It took me a while to fully appreciate the issue that Twitter user Yoshimasa Niwa was describing. At first glance I and many others assumed that setting Japanese over English would solve his app library sorting issue.
Then I realized that wasn’t his point at all. The software app in the screenshot is the Yahoo Japan ‘Norikae Annai’ transit app, one of the most popular free stand alone transit apps in Japan. I use it all the time. It’s a Japanese app with a Japanese name but the basic iOS English sorting algorithm ignores this and assumes all Chinese characters used everywhere must follow modern mainland China’s Simplified Chinese rules for reading and sorting.
This is ridiculous as assuming that all Roman based character sets everywhere must follow modern Italian reading and sorting rules. I always find that westerners assume the Kanji culture flow was always one way from China which it is not, with different and unique readings, usages, and Japanese Kanji like shitsuke 躾 traveling the other way over the centuries. The same is true for other cultures that adapted the Chinese writing system for their languages.
It amounts to cultural destruction by neglect and ignorance by large western based technology companies who think things are ‘good enough’. Or are just bugs to fix in a later software update that usually never appears. Modern computer software has pretty much destroyed traditional kanji culture publishing this way, with many countries abandoning mainstream traditional vertical text layout for western style layout because ‘it’s easier’, i.e. western tech companies couldn’t be bothered getting Asian language typography right. All these years later web browsers still can’t do vertical text worth a damn.
A veteran Japanese font engineer whose entire career was devoted to preserving high end Japanese typography in the digital age recently told me, “I don’t think anybody cares anymore.” In the end it all too often comes down to this: I don’t care cultural death by I don’t care companies who have the money and power to care.
That’s bitter irony in our age that purports to champion cultural diversity.
2020 is the coming out party for Apple designed OpenType variable fonts, both the SF Pro and SF Compact system fonts and the all-new New York font shipping in iOS 14, watchOS 7 and macOS 11. The Apple created variable font technology is not new of course. It has been around since the QuickDraw GX days along with the TrueType GX enhanced Skia font. It was due to be standard in MacOS Copland system fonts including a Japanese variable font created by FontWorks. Then Steve Jobs returned to Apple and everything changed.
Yes, it has taken 25 years for an Apple created technology to make it into the basic system. It proves my long stated belief that font technology doesn’t matter unless it is built into every nook and cranny of the OS foundation. The TrueType GX Skia variable font has been with us all this time, but only matters now because the SF Pro system font has gone variable.
Why is It Taking So Long? iOS 14 and macOS 11 variable font basics are covered in an excellent WWDC20 video, ‘The Details of UI Typography’. It’s important to remember that while OpenType variable font technology is ‘world ready’, at this stage they only apply to Roman based font sets. It’s going to be a long time before we see a Japanese language system font in variable format.
There are many reasons. In the WWDC20 video Loïc Sander of the Apple design team drops a big hint when he explains that while digital technology (PostScript fonts) “gave us a lot more flexibility in handling text,” it also “made typography a bit more crude than it used to be.” The statement shows how clueless designers and engineers outside of Japan can be about Japanese fonts and typography.
While a ‘bit more crude’ might be true for Roman based fonts and text layout, PostScript fonts completely broke traditional Japanese font design and composition models. Everything was thrown out because Adobe made no accommodation outside of western typography needs when creating the PostScript font DTP foundation.
Another big problem was that Adobe relations with Japanese PostScript licensees in the 1990’s was not healthy. Adobe stuck with closed print device font licensing for far too long and discouraged independent font production wherever they could. Because of this situation, digital font progress in Japan was slow and very expensive.
Here are some challenges facing Japanese variable fonts.
Once Upon a Time One basic flaw of OpenType font outline technology is that it’s extremely inefficient for kanji glyph production and storage. Every glyph has to be created and stored separately and doesn’t scale well. This is why OpenType CJK fonts on tiny devices like Apple Watch are a match made in hell. One solution to this problem is stroke fonts. Stroke fonts use a library of basic glyph parts to efficiently create complex glyphs.
Stroke fonts are a perfect fit for kanji font production and for small constrained devices like Apple Watch because reusable parts don’t take up precious resources. On the desktop, stroke fonts can do weight variations over the full range from Light through Ultra Bold without losing typographic details, all in a single 4 MB font while an equivalent OpenType variable font can weigh in around 18 MB.
The technology has been around for a long time and was supported up until macOS 9 but lost out when Apple quietly dropped the QuickDraw GX derived Open Font Scaler architecture in the migration from classic to macOS X.
While stroke fonts are not supported in the current Apple OS lineup, on the font tool side stroke font technology has appeared in software such as the classic MacOS Gaiji Master from FontWorks. The lead engineer of that effort is currently working independently on a similar gaiji glyph tool for Windows based on stroke font technology that is much more advanced than the old and long unavailable FontWorks software. I plan to cover developments in a future post.
The Japanese Font Production Challenge The Hiragino iOS/macOS Japanese system font was not created by Apple, it was licensed from Screen Holdings (SH), originally created by independent font design studio Jiyukobo in the early 1990’s. There is much more work involved creating a Japanese font compared to Roman based languages. Hand drawn glyphs are created, scanned and cleaned up for digital production.
The Adobe Japan 1-7 glyph collection requires 23,060 glyphs for a single weight, multiply this work by the different weights for one family and you get an idea how massive the undertaking is. From Osamu Torinoumi, one of the key designers of the Apple licensed Hiragino font on its creation:
On average, one person would (hand) draw 12 or 13 glyphs a day, which is not much change of pace from the days of creating block type…the whole process, from start to finish, took three years.
One might think that a single CJK (Chinese-Japanese-Korean) font sharing a common design can streamline the process but this is a huge misconception. Each culture has centuries worth of different design aesthetics that good design must incorporate: what looks good to a Chinese designer and works well in a Chinese text design, looks terrible in Japanese context. I have yet to see a decent digital ‘kana’ design from a Chinese font designer. Osamu Torinoumi on the differences in creating the Simplified Chinese Hiragino Sans GB:
“We worked with the Adobe GB 1-4 character set (29,064 glyphs) at 2 weights. Basically we had to finish one weight in 6 months. One year for the entire project. At first we only thought we would be there as backup, but Screen kept passing us all the questions from Beijing. It turned out to be a lot more work than we anticipated.”
Jiyukobo sent all the original Hiragino design data to Hanyi Keyin through Screen and they adapted the designs for China. Torinoumi said that one of the major differences is that Chinese design demands that Gothic (sans serif) characters mimic handwritten style. This means the character should be slightly off center within the virtual body. “Even after the project was over I still didn’t understand the difference between Japan and Chinese “Kokoro” glyph which the Chinese designers insisted were different.”
The Variable font UI Challenge Finally we get to a problem on the Apple OS platform side that has been around since the GX days: how to present advanced typography features in a useful and easy to understand system UI that works everywhere. What works on macOS obviously won’t work on iOS, but iPad OS will need some degree of advanced typography feature access. Sliders have their place but I agree with Adobe Type Senior Manager Dan Rhatigan who made a very good point in his TYPO Talk 2016 presentation: there has to be a better UI control concept out there.
Japanese typography is unique in that it has preserved its own print ‘moji bunka’ cultural history and vision that China and Korea have largely abandoned in the face of western centric computer culture that all too often pretends to care about such things, which it does not. If it did we’d have vertical text in web browsers by now that actually works. I hope a rich text culture can be preserved and conveyed to future generations even in such small details as a well designed and executed Japanese variable font for computers and smart-devices.
The once mighty Shaken KK, who’s founder co-created the modern Japanese typesetter with Morisawa in 1924 and a huge highly profitable font library that once dominated the entire Japanese print industry, is not quite dead but the last vestiges are quickly disappearing. Japanese blogger tkri notes that the Shaken Saitama production facility is being demolished, the same fate of the Kawagoe factory in 2017. The very last Shaken ‘digital’ typesetter machines serving the last 100 customers are due to go out of service by 2024.
Shaken had everything but never made it into the desktop publishing era by their own choice:
Hiragino started just about the time that Japanese PostScript arrived in the late 1980’s. At that time, the king of the typesetting market was Shaken KK. Anyone who knows Japanese typography knows Shaken. It had the most sought-after type library, the best designers, the biggest market share. Shaken made so much money that it became arrogant and absolutely refused any offers from Adobe and Apple to jump on the PostScript bandwagon.
But there was trouble in paradise. A few designers felt Shaken was becoming stagnant and left the company. Each sought his own vision, yet they came together to create the fonts that would eventually end up in MacOS X.
Shaken kept its proprietary typesetters and formats, losing 40 percent of its market share by 1999 along with lots of prestige and talent in the bargain. Ex-Shaken people filled the Japanese print industry. The low point came that year when government tax raiders found several safes in the company basement filled with 10,000-yen bills; president Hiroko Ishii, daughter of the founder, had stashed away some 15 million USD in the early ’80s and reported it later as profit. The directors of the company tried to oust her but failed. She died in the presidents chair at age 92 in 2018 leaving a ghost company as her legacy.
This sorry state of affairs had two huge repercussions. It denied a vital part of the Japanese font design legacy to designers working with DTP. Imagine a world with younger generations of designers denied the use of Helvetica and Times and you get the idea. It also left the only other competitor, Morisawa, which did license its library to Adobe, with a practical monopoly in the new DTP era.
Tomihisa Uchida who retired from Iwata Corporation a few years back knows more about Japanese font programming and typography than anybody else in this world. He was involved with Japanese digital font production from the start, working at its very heart: Shaken KK. I interviewed him in 2003.
Uchida: I was with Shaken for 23 years. I entered right out of college, where I was a chemistry major. My first job there was working with analog plates and mechanical processes. That involved high-resolution plates, similar to what is used for IC chip manufacture, to produce high-quality typography. I did that for 10 years; then digital fonts came along in the early ’70s. Shaken was the first Japanese vendor to have computerized layout. It also did the Japanese version of Ikarus. There wasn’t any real competition and it had the market to itself.
Question: So Shaken made the transfer to digitalized fonts and computer-based layout successfully?
Uchida: Yes, it purchased the Japanese rights to Autologic technology to produce a hybrid product where the software was a customized Autologic engine running on Shaken hardware. That didn’t last too long, as Shaken had been developing in-house technology and soon released its own original product.
We simply implemented Japanese typography and composition rules on the computer with outline fonts that output on an imagesetter using proprietary technology. At that time, Shaken systems were extensively used in newspaper production. However, Shaken lost that market because it didn’t have strong network capability, which newspaper production demands.
One of the ways Shaken was able to build up a strong type library in a fairly short time was by sponsoring a typeface competition. It would pay the winner and purchase his typeface. It raised lots of young designers that way (like Suzuki-san, who would later create the Hiragino font used in Apple’s MacOS X) and really expanded the market with new typefaces for comic books and such.
Later on, my job was creating different weights. The designer would create the basic design, then we’d use the Ikarus system to make the weights. That was the late ’70s. The systems we designed ran on hardware from the likes of DEC and Wang. I had a group of people who were basically a font production line. When Japanese PostScript first arrived, it wasn’t immediately apparent that things would change as they did, it took forever to print. Shaken systems always had excellent performance.
Shaken sent Uchida san to programming school then put him in charge of their digital font engineering group. He created the Shaken proprietary font format for their digital typesetter machines. He joked that if Shaken ever wanted to convert their font library to OpenType format, they would have to hire him back.
The very last time that Shaken made any kind of product announcement these past 20 years was at the July 2011 International eBook Expo Tokyo. At the time I blogged:
The once mighty Shaken finally waved the white flag and demonstrated OpenType versions of their fonts running in InDesign and on an iPad today at International eBook Expo Tokyo. Shaken’s Toshiro Ito said they don’t have a set release date, but hope to make an announcement soon; Japanese designers, who have been waiting for Shaken packaged software fonts since 1989, can finally look forward to using Shaken fonts on personal computers and mobile devices.
Blogger Danbo was there too and took some video of the Shaken demonstration. I remember most of the audience being near retirement age. That was the last word from Shaken. The OpenType product never appeared. Uchida san told me that Shaken probably didn’t have any designers and engineers left to build out the smaller legacy proprietary digital font collections into Adobe Japan 1-4 character sets for modern designers and devices.
Any time I talk with ex-Shaken people like Torinoumi san of Jiyukobo, or Toyoizumi san of Screen, or Uchida san of Iwata, there’s a wistful quality to their shared memories, even though they all went on to create new and better things. It’s bittersweet legacy only they can know: building something great then watching it rot away, destroyed by greedy deluded people in charge of protecting it. And now that it is almost gone, with most designers of that time in retirement, it is largely forgotten.
Some people will want to write the Shaken story off as Japanese culture unable to change but that’s just snobbery. What happened to Shaken can happen to any company with a monopoly. Steve Jobs definitively explained how monopolistic market power rots a company in his lost interview:
Sales and marketing people end up running the companies and the product people get driven out of decision making forums. And the companies forget what it means to make great products. The product sensibility and product genius that brought them to that monopolistic position gets rotted out by people running these companies who have no conception of a good product vs. a bad product. They have no conception of the craftsmanship that’s required to take a good idea and turn it into a good product. And they really have no feeling in their hearts usually about wanting to really help the customers.
When the last person turns out the lights at Shaken KK, I hope they open the vaults and set the Shaken font library free. Only by taking flight and having a life of its own can it ever hope to live on in the hearts and imaginations of future Japanese designers. Only then can the Shaken tragedy be reborn into something new.
Postscript A reader forwarded a Japanese public record document from 2008 regarding a Shaken company labor union dispute with management over mandatory retirement policy. It’s a pretty damning document. Company sales data from page 11 on clearly shows in ugly detail how management ruined the company business: focusing exclusively on milking customers with outrageously expensive font library rental royalties on proprietary hardware at the expense of everything else.
Shaken had the talent, technical expertise and opportunity to lead Japan into the DTP era with non-proprietary digital fonts and imagesetters. Instead of innovating, management clung to a do-nothing easy money strategy that destroyed the company. Meanwhile, the Japanese print industry moved on to better modern production infrastructure; a perfect example of Steve Jobs’s explanation.
One last bit: the Shaken KK company registration was updated in August 2019 and lists digital font production and sales as a main company objective even though real estate holdings will probably fund most company operations.
This article was published in The Seybold Report Volume 30 Number 6, November 2000. It was a strange time when QuickDraw GX 3rd party developments like Japanese stroke fonts and plug-in font scalers were up in the air without a clear migration path to the newly announced MacOS X ATSUI Carbon framework, OpenType Japanese fonts and InDesign J had been announced but were months away from shipping, and Japanese print customers were angry with Adobe and Morisawa for forcing font upgrades on everybody. It’s a snapshot of Japanese publishing in transition as Steve Jobs was saving Apple and tossing the Japanese print market a few morsels like extended Japanese glyph sets, Adobe was caught flat footed and was furious with Apple.
Japanese DTP arrived with Apple’s NTX-J PostScript printer and Linotype’s first Japanese PostScript imagesetter in 1989. They came at the right time: The early- ’90s economy was bubbling, companies had money to burn and Japanese DTP took off. It was a young, booming market and it forgave many mistakes that would haunt the industry later.
By 1996, the go-go days were gone and they would not come back. By this time, DTP tools (Quark Xpress, Illustrator, Photoshop and, to a smaller extent, PageMaker) had captured nearly 40 percent of the production process. For a conservative industry like Japanese publishing, this was phenomenal―until compared to the West. There, in the same amount of time, practically the entire industry converted to DTP production. Japan is still about 40 percent and holding.
What happened? Three things: the economy, the failure of Western technology to address Japanese issues, and the same old cultural differences Westerners have been running into since Commodore Perry knocked down the doors in 1853. They just do things differently here, get used to it.
The economic crunch hit the publishing market hard, and it hasn’t really recovered. Consider the book and magazine market: On average, this year’s revenues are down 3.4 percent over last year.
The advertising industry has been down, but has recently shown signs of recovering: On average, revenues are up 10.4 percent so far this year.
Basically, the publishing industry overextended itself in the bubble years of the early ’90s and is still adjusting.
Some of the difficulty is the publication distribution structure. Anybody in the business will tell you there just isn’t enough shelf space and return rates are climbing. Internet publishing is not much of a factor yet. Japanese book retail laws, which prohibit stores from selling below suggested retail prices, will be relaxed soon and will put tremendous pressure on the current price structure, which supports lots of marginally profitable items
The Japanese market is also a highly segmented one. There are the major newspapers, which have very specialized workflow systems and in-house fonts. There is the book industry, which runs the gamut from standard tanko bon hardbacks to the ingenious, very popular pocket-size bunko hon paperbacks (which really have no equal in the West). There are lots of printers, using both DTP and traditional systems, that cover everything in-between. In a class of their own are manga: the hefty weekly comic books you can find on subway luggage racks, left there by salarymen too embarrassed to take them home.
On the whole, the industry is still in a slump and has been treading water for the last few years, with relatively little investment in new technologies such as computer-to-plate. At the same time, competition and consolidation in the printing sector have forced prices down-more than 30 percent in the past year alone-so there is a great need to reduce production costs and increase efficiency. This is the great challenge of the Japanese publishing industry. Meeting it will require new investment in hardware and software. The challenge for vendors is to provide the right products. Those who watch the market and listen to customers carefully will find opportunity.
The font technology failure A designer once said, “Fonts are like air.” They are the environment in which DTP lives and thrives. Compared to the West, the Japanese DTP air is pretty thin.
One of the great tragedies was the refusal of Shaken KK, the largest, most popular, and perhaps most beautiful Japanese font library, to license its faces to Adobe or have anything to do with PostScript. It kept its proprietary machines and formats, losing over 40 percent of its market share (and lots of prestige and talent) in the bargain. The industry is filled with ex-Shaken people. A low point came a year and a half ago when tax raiders found several safes in the company basement filled with 10,000-yen bills; the president of Shaken, Yuko Ishii had stashed away the money in the early ’80s and was now reporting it as profit.
This sorry state of affairs had two huge repercussions. It denied a large part of the Japanese font legacy to designers working with DTP. (Imagine a world without Helvetica and Times and you get the idea.) It also left the only other competitor, Morisawa, which did license its library to Adobe, with a practical monopoly. But these were not the only problems.
Initial problems The first implementation of Japanese PostScript Type 1 fonts had serious shortcomings. First of all, they could not be downloaded on a per-job basis, but had to reside permanently on the printer. This turned the production process upside down; service bureaus and printers were suddenly dictating to designers which fonts could and could not be used. For every PostScript device, users had to invest in font licenses― and Japanese fonts are very expensive. In a market where making a single face can easily require ¥30,000,000 (about $280,000) and consume two years, font vendors made sure they got maximum return on their investments.
To this end, Morisawa and Adobe came up with the idea of marketing two flavors of PostScript printer fonts: low-resolution (up to 600 dpi) and unlimited. A single unlimited-resolution Japanese font costs ¥218,000 (about $2,000).
The situation became more complex when ATM J arrived on the scene in late 1993. Morisawa’s ATM fonts used hardware binding to lock the font to the hard disk. This resulted in a high maintenance operation, because hardware binding required installer floppies for each font and for every CPU. (This remains true even today.) If the user got a new hard disk, he had to send the old floppies back to the vendor and exchange them, for a fee, for new ones. Large operations had the extra burden of keeping track of the floppy that had been installed on each CPU. On top of all this, each installation took time; putting a family of 15 Kanji fonts on two or three Macs could easily eat up a day.
In addition, compared to traditional systems, these fonts had primitive typographic features: lack of pair kerning, lack of proper metrics and poor Gaiji support. (Gaiji are Kanji characters outside of the current JIS [Japanese Industrial Standard] and Unicode encoding sets and are not included in a standard font. They comprise many “unofficial” Kanji characters, mistakes and misinterpretations, and seldom used Kanji passed down for generations, long before printing presses and governments created standards.
These Gaiji characters are widely used in people and place names. To this day, they are a reason for publishers to hang on to their proprietary systems. Last but not least, Japanese PostScript fonts were not cross platform.
The first fix Adobe has been trying to fix the font problem. Its first attempt was CID (character-key ID). CID divorced two-byte PostScript fonts from dependence on any encoding scheme and allowed character subsets to be downloaded to the printer on a job basis, laying the foundation for PDF font embedding in Japanese Acrobat 4.0 as well.
Unfortunately the CID font upgrade was a marketing washout. Upgrades were not cheap, and there were few benefits the end user could really see. Morisawa also decided to tweak some glyph designs and the font metrics. Designers would open old files with the new fonts and find they had to redo everything. They stayed away in droves. As a result, the installed base is still overwhelmingly OCF (original composite font), which has slowed acceptance of PDF as a major production format.
Dueling fixes Adobe’s final solution for the Japanese font problem is OpenType. OpenType will be a very big deal and, in the long term, fundamentally change the market. OpenType finally brings true font parity between Windows and Mac. Morisawa and Fontworks are busy preparing OpenType upgrades. It marks the first time their font libraries will be available cross-platform. Both companies see Windows as an opportunity for revenue growth―something that has been missing on the Mac side for years. Users will finally be able to choose the platform that best fits their needs and budgets without worrying about font problems.
This will be especially interesting because Mac OS X and OpenType are both arriving in 2001. According to a Japanese Publishing Consortium survey in 1999, Apple has nearly 80 percent of the DTP installed base. However, a large share of the machines in the printing and publishing areas (but not in design) are pre-G3 PowerPC systems that will have to be upgraded to run Mac OS X. Apple Japan’s “We’ll let the market decide” marketing statements may become a true nightmare if the market chooses Windows instead of Mac OS X
Adobe Japan 1-4 The most important feature of Japanese OpenType will be the Adobe Japan 1-4 character collection. This is the big Gaiji fix everybody has been waiting for. AJ 1-4 addresses the Gaiji encoding problem and provides a consistent and cross-platform way for font developers to add Gaiji to OpenType fonts. Adobe says the collection is “to provide professional publishers with a glyph set that will suit their needs.” AJ 1-4 will add 6,000 glyphs. They will include additional Latin characters: macroned vowels, italic forms, fractions, third and quarter-width numerals and punctuation and Latin ligatures.
They will also offer a rich set of Kanji options:
Annotated forms: These are popular ‘shorthand’ glyphs for such Kanji as “corporation limited” as well as specially designed numbers:
Hiragana, katakana and Kanji ligatures
Alternate kana for horizontal and vertical writing
Ruby glyphs: These are special kana characters that are placed next to difficult Kanji to show readers how to pronounce them. Many software DTP packages offer this feature. However, traditional ruby glyphs are a different design so they are easy to read at the small point sizes in which ruby glyphs are always used. With OpenType, these special designs can finally be included in a Japanese font.
Kanji and Kanji variants Anyone familiar with Japanese knows the language has changed more in 200 years than English has. Particularly after World War II, many Kanji were simplified and, even though the older, more complex Kanji disappeared from newspapers, magazines and school textbooks, they still live on in books, maps and proper names. There are also many different variations of certain Kanji, such as the ‘nabe’ character for the name ‘Watanabe’ (see illustration).
AJ 1-4 was created by Adobe in cooperation with Morisawa, Fontworks and other industry experts. It’s not going to satisfy everybody, but it’s an excellent effort that should take good care of the majority. The only drawback is timing, coming as it does on the heels of the CID upgrade catastrophe. Adobe and Morisawa are planning to market two packages: Standard, which will have the same features as current CID fonts, and Pro, which will have AJ 1- 4 Gaiji and advanced layout.
Character layout Advanced typography is another major OpenType feature and one that, until now, has been an elusive goal. After all, Apple has been pushing advanced typography for years, first with QuickDraw GX and now with ATSUI (Apple Text Services for Unicode Imaging) in Mac OS X. Apple is still pushing its solution; the Hiragino fonts for Mac OS X will include both OpenType layout and Apple Advanced Layout (AAT) tables.
Advanced layout is not a trivial feature to add, particularly for a complex written language such as Japanese. As noted in the Gaiji section above, there are all kinds of ligatures, vertical variants and arcane Kanji. But a well thought-out and executed set of features can vastly simplify the user experience and address many shortcomings of Japanese DTP.
One example of how this can be used is Morisawa’s new Mincho Kyoiku font. This font family has special glyphs required in Japanese school texts that help children learn how to write.
Because it is based on the standard Mincho design, Morisawa digital type manager Nobuo Tomita said, it would make sense to implement the different glyphs as variants in the standard Mincho font package instead of selling them separately as they do now.
GX legacies At the World Wide Developer Conference and at an Apple font developers’ conference in Tokyo shortly afterward, Apple stated that Apple Advanced Typography tables were the “recommended” way to add advanced layout features. One Apple engineer described the differences between the two approaches this way. “AAT is pretty much a superset of OpenType [layout], with much better performance.
There’s no real pressure for us to do this [support OpenType layout directly in ATSUI].” The Hiragino OpenType fonts for Mac OS X have both AAT and OpenType layout tables.
ATSUI is a system-level API, while OpenType layout is an application-level API. One very nice thing about ATSUI is that application developers don’t have to know all the details of font table formats and how to process them. ATSUI just takes care of it.
OpenType layout requires the application to do all the processing, so developers have to know a lot about a language if they want the features to work right. On the Windows side, there is the OpenType Layout Services Library, which helps somewhat; but whether on Windows or Mac, application developers will not find it easy to implement OpenType layout.
Adobe, of course, has the advantage over the competition here because its own type layout library, Cooltype, is built into every Adobe application. Adobe will pull out the stops and bundle the KozukaMincho Pro OpenType Kanji font, which of course does not have AAT tables, with InDesign J, due to be shipped in the first quarter of 2001.
Unhappy developers Apple’s AAT solution might be better technology, but some font developers are not amused. Morisawa’s Nobuo Tomita said, “Apple has not been very forward with their plans…. AAT has lots [of features] that don’t have much to do with Japanese. We plan to go with Adobe’s cross-platform solution.” If Morisawa doesn’t support AAT, then the case is pretty much closed.
Adobe, predictably, isn’t very enthusiastic about AAT either. “I went over to Apple and told them to just kill it…. We went through this whole thing with GX already,” said Adobe’s Asian Font product manager Julie Ma. Given Apple’s lack of marketing enthusiasm for ATSUI―it is ignoring some fantastic third-party technology that makes use of it, such as Fontworks International’s stroke font technology (announced at Tokyo Seybold two years ago)―it is understandable that developers are hesitant. Quark, for one, is hedging its bets; when asked which technology it plans to go with, Masato Nishimura of Quark Japan said, “We’ll wait to hear from our customers before we make any decisions.”
Another Apple move not sitting well with font developers is the additional Gaiji characters that are part of the 17,000-character set in the Mac OS X Hiragino OpenType Pro fonts. These go well beyond the Adobe Japan 1-4 collection.
Not much is clear at this point, but the extra Gaiji appear to be another ATSUI-only feature guaranteed not to work with Adobe software (nor with any other application that does not support the Apple API). The feature by itself is not bad, but Apple has not communicated with Japanese font developers well, and that is bad. The developers need detailed information to define their own OpenType features and to make sure it all works seamlessly with Hiragino and Mac OS X, and that information has not been forthcoming. As one font developer moaned, “Just when things are looking better, this happens and everybody loses.” If Apple really wants Mac OS X to be the best publishing solution, it will have to get its Developer Relations act in high gear, and fast.
In the end, Japanese OpenType’s greatest achievement will be that it delivers most of the features the high-end customers have been asking for: Gaiji, robust cross-platform support (allowing developers to put both OpenType layout and Apple’s AAT layout in the same package), and embedding and dynamic downloading. It will finally bring the Japanese printing environment to the same level the West has had for years. As Adobe Japan’s Seiichiro Miyajima, group manager for product marketing said, “To put it simply, it will be the same as using Roman [one-byte].” The timing for Japanese OpenType is difficult, but the benefits are very real. The industry will get there―eventually.
Japanese layout and the promise of InDesign J In order to really understand the differences between Japanese and Western layout, you have to go back to elementary school. Think back to second grade when you were just learning cursive; the teacher gave you a piece of paper with lines. This was the bottom line, that was the top line. She taught you to write along the bottom line.
In Japan, the students get a piece of paper with rows of little boxes. The teacher tells the kids to write Kanji characters in the centers of the boxes. Japanese typographic layout is exactly the same; it is all based on little boxes, known as virtual bodies. It is also called the grid system, because the middle of each box is one center point on a grid. Everything is calculated from the center; there is no baseline. The whole system makes sense when you realize it all comes from the days of block type; the virtual bodies are blocks of metal type set end to end. The vocabulary, the measurements and the aesthetics all evolved from that.
Virtual bodies Unlike DTP layout, which is graphics-driven, Japanese text composition, called kumihan, is driven by how much text will fit in a given space. Designers know how many characters (virtual bodies) are supposed to be on a line and on a page before they start composition, and this is how they discuss layout with writers and editors. Western composition is calculated from margins, a wholly different concept.
Adobe Japan InDesign product manager Hiroshi Miyamoto, who has typesetting experience, explained the difference. “It’s very important for [Japanese] operators to know the number of characters on each page, and it’s difficult to work that way with PageMaker, Quark Xpress and Illustrator. PostScript Japanese fonts (OCF, CID, OpenType) have no virtual-body information whatsoever, and historically there is no baseline in Japanese kumihan. But since PostScript fonts have only baseline information, that’s how all DTP software developed. Japanese fonts have different baseline positioning, so that when you change the font, the line breaks change.
Traditional systems used the grid system, so as long as you kept the same size, any font line breaking stayed uniform, and even if you did change the size, the grid made it very easy to make adjustments. If you are trying to calculate grids in Xpress, the point system doesn’t match well, so you end up with too many, or too few characters on the page, and the designer compensates by incrementing in .01-point values, which is overkill…. Also it’s very difficult to handle pages that have both English and Japanese, a real problem for magazines.”
It’s not impossible to do quality kumihan with DTP. It just takes more effort than traditional systems, and everything has to be done manually, which is inefficient. For text-heavy and specialty market segments still using traditional or proprietary systems (newspapers, books, manga), the cost benefits of DTP are just not there.
Japanese engine InDesign J, announced this past February at MacWorld Tokyo and due to be shipped in the first quarter of 2001, is the first major piece of DTP software that follows Japanese typesetting conventions. To this end, InDesign J has a special Japanese typesetting engine and Japanese layout grid. Because there is only baseline information in PostScript fonts, InDesign J calculates and creates a full-width zenkaku virtual body, and this allows the designer to calculate the number of characters in a line, on a page or in an InDesign text frame, based on point size and line spacing.
Another basic Japanese kumihan function that professionals expect to have is known as Ji-Dori-Gyo-Dori. In traditional systems the priority is: (1) Calculate additions to the previous line, if the first character of the next line is a punctuation mark, (2) Calculate the shift kana so the first character of the next line is not a punctuation mark, (3) Calculate the average spacing of each kana.
This sounds easy to do, but it’s not, for two reasons. First, as noted above, everything is calculated from the virtual body’s central grid point. Second, there are special sets of and Kanji known as “Classes.” Proprietary systems have these highly specialized line-breaking rules and Class settings built in, so they can automatically calculate and adjust the spacing very quickly.
They are also very good with mixed Japanese and English, which is half-width: two Roman characters equal the width of one Kanji. InDesign J has all these features. InDesign also has support for the kyu measurement standard used in traditional systems, where one kyu equals 0.25 millimeters. Quark Xpress supports kyu too, but Adobe promised that InDesign will have a more accurate method to calculate Japanese kyu and will eliminate errors that occur in the current versions of Xpress and PageMaker.
Quark’s reaction Quark is not standing on the sidelines. “We foresee strong growth in the market as Japanese font technologies become richer and easier to use. More and more designers will feel comfortable using DTP and especially Quark Xpress instead of traditional typesetting methods,” Desktop Systems Product Manager Masato Nishimura explained. He could not discuss specific features of Quark Xpress 5.0J, other than saying it will not be carbon-compliant. “That version will be a release subsequent to 5.0.” But he did mention some features requested by Japanese Xpress users that will very likely appear: placing each character in a desirable position (center) even if a line contains characters of different font sizes; scaling characters by center line grid in line-width increments in vertical text; enabling users to easily keep a full-width zenkaku type setting mode for all Japanese characters and punctuation.
However, Quark will not pursue InDesign’s approach of matching all the features of traditional kumihan. Adobe has followed the Japanese Industrial Standard (JIS) X4051 typesetting and composition specification (written by an ex-Shaken employee, no less), which is the kumihan “bible.” (Follow it and you’ll never commit compositional sin.) Quark has not. “We want to give the most important features to the majority of users,” Nishimura explained.
“Many designers feel they have enough features already,” a sentiment I have heard from others in the industry. From a layout perspective, the current tools are good enough. But they are not good enough for the unconverted 60 percent who are dealing with composition rather than layout. So the big question is, will this new environment, the second wave of Japanese DTP arriving in 2001―OpenType, advanced layout, Gaiji extended character sets, cross platform Kanji, InDesign J, Quark Xpress 5.0J―finally convince the unconverted to make the change to the open standards-based digital workflow that DTP has to offer? The answer: definitely maybe.
Cultural differences “At this point I don’t think the issue has to do with technical reasons,” Toppan Printing KK Publishing Information Center’s Shinichi Konno, told me. “It has to do with middle managers and their way of thinking. Many companies know they have to move to a digital workflow at some point; some are even feeling panicky, but those kind of people don’t want to take the responsibility for changing over.” Having worked in the Japanese publishing industry for more than ten years, I can attest to the very conservative mentality of middle managers, and I can even agree―up to a point. I have been in many situations where a company desperately wants to change but doesn’t know how. If you listen carefully and talk with the managers on a level they understand, amazing transformations can take place. But it takes time and patient effort.
There is another factor at work, too: shokunin mentality. “Shokunin” is a tricky Japanese word. The dictionary translation lists the word as artisan or craftsman, but the nuance implies a long apprenticeship, an obsession with quality and detail and pride― traits the Japanese are famous for. I call them specialists, and Japanese culture does cherish specialists. The publishing industry is full of them, and if there is one thing you don’t do, it is tell a specialist his business. But specialists tend not to see the big picture. They focus on their own little area of expertise and have trouble understanding company-wide benefits of digital workflow.
This is where Adobe is playing a smart, subtle hand with InDesign J. It is using Japanese standards such as JIS X4051 and is also using all the specialty words. The marketing material is carefully scrubbed clean of foreign words such as layout (reeauhto), and text (tekisuto), instead using native specialist words like kumihan. Even the official product name foregoes the lazy ‘J’ for Japanese that most product names go for, using the formal “Nihongo ban” instead.
Quark, too, has a solid, if slightly different, vision. “The nucleus of our strategy is media-independent publishing, the next generation publishing model that encompasses print, Web, E-books …and beyond,” Nishimura explained. He said Quark hopes to free publishers to create and manage content so they can deliver it anywhere at low cost, with a layout appropriate to each media format.
It sounds good, but here again―and this is the whole point of this article―developers simply cannot do a localization job and sling their products out in the market. It takes time, careful study of market needs, and thorough knowledge of the segments and specialties therein. For example, media asset management has not done very well as a category in Japan. But it can, as it has for CRS Systems, the Japanese localizer and distributor of Canto’s Cumulus. CRS has seen Cumulus gradually grow from a software package into a intranet integration business, as Japanese managers slowly understand the cost savings from integrated workflow. However, like the font and layout situation, it has to happen in a Japanese context to which shokunin, the specialists, and middle managers can relate.
Conclusion Current Japanese DTP workflow is far from perfect. It is expensive and unreliable; it has primitive typography and layout and a limited variety of Kanji fonts that lack necessary Gaiji characters. Perfect Japanese DTP workflow should have a rich set of fonts, rich typography, layout and composition, Gaiji character sets, and theability to pass files between computers seamlessly and print without any problems.
The second wave of Japanese DTP will not magically convert the unconverted. It will be a gradual change, as companies see and truly understand digital workflow as cost reduction and potential revenue growth. The highly segregated nature of the Japanese market also means there are plenty of specialty niches to fill with all manner of InDesign plug-ins, Quark Xtensions and flexible media management systems, giving the 60 percent all the more reason to convert to a standards-based digital workflow.
A friend once told me that Japanese society is like an egg. In the West, the shell is brittle. A small amount of force from the inside can break through and bring change. But the Japanese shell is tough and rubbery; a small force can only bend the walls. It takes much more force to break the Japanese egg. But once it breaks, change can come very quickly. The publishing market will take time, but it will change. And change is opportunity.
Stroke Fonts: The Future of Japanese Type?
OpenType, the Adobe Japan 1-4 Character Collection and Unicode do a good job of breaking the current limitations of Japanese fonts. Unfortunately, encoding is not the only problem. You also have to create all those extra Kanji. The proposed Adobe Japan 1- 4 character set alone is 6,000 characters. One of the glaring deficiencies of current outline technology is that every single Kanji must be traced and tweaked extensively. A regular Japanese font has nearly 7,000 characters; add the Adobe Japan 1-4 character set for OpenType and you have a whopping 13,000 glyphs.
Take that total and multiply by each weight (light, demi-bold, bold) that has to be designed, and you have an idea of how much work goes into font-making and why Japanese fonts are so expensive. The fonts are large—anywhere from 3 to 8 MB each—and they are not very efficient, which is one of the reasons why Japanese Multiple Master fonts will never happen.
An interesting feature of Mac OS (both 9 and X) is the Open Font Architecture (OFA) that evolved from QuickDraw GX. As the name implies, OFA is an open plug-in architecture. It works with any font technology, be it PostScript, TrueType, or a completely new technology, such as stroke fonts created specifically for Japanese, Chinese and Korean writing systems.
Recombinant parts Anybody who has studied Chinese or Japanese knows that although each Kanji is unique, certain parts occur again and again, recombining to create new characters. You can get a good feel for this by looking at Chinese or Japanese calligraphy. The brush is the most natural way to write Kanji, and with a little study, you quickly comprehend the strict order of each stroke.
In a similar way, Fontworks International broke down its Kanji fonts into parts that loosely correspond to brush stokes. These stroke parts are kept in a library that the stroke-font scaler uses to draw the character, resulting in a much smaller and more efficient font. Torsten Buck, Fontworks technical director, told me that stroke technology “allows us to do weight variations over the full range from Light through Ultra Bold without losing typographic details,” all in a 4 MB font. An equivalent PostScript Kanji font family can weigh in around 18 MB.
Where stroke technology really shines is character creation. Once a base library of parts has been created, a designer can create high quality Kanji quickly and easily. A key feature of stroke technology is that it preserves the stroke width as the part is scaled, which is impossible to do with PostScript outlines.
To create a new Kanji, you simply use a similar character, swap out the parts that need to be changed and, perhaps, fine-tune them. Stroke fonts are a great technology to break the Japanese font logjam of high cost and limited design. It could do for Japanese fonts what Fontographer did for Western typography: put real power in the hands of designers and open up the floodgates.
Apple fumbles Unfortunately, Apple doesn’t see this as an opportunity. It creates great things like ATSUI and OFA and then forgets about them, leaving third-party developers holding the baby. When Fontworks approached Adobe’s InDesign team to discuss stroke font technology, Adobe didn’t even know what ATSUI was. Clearly somebody in Apple Developer Relations wasn’t doing the job.
Fontworks’s original plan was to have full-blown Japanese stroke fonts with advanced typography and multiple weights. But in the face of Apple’s indifference, it postponed the plan and instead re- leased Gaiji Master last February. This is a slimmed down, Gaiji only version of 2×2, Fontworks’s heavy-duty in-house stroke font development environment. It is a fascinating piece of software, and it reminds you of the early Macintosh days when the graphics software was thrilling and cutting-edge.
E-book opportunity One reason why Fontworks went ahead with this is that it believes E-books and E-publishing could be a huge market in Japan. And stroke fonts could be the key. As Fontworks President Ross Evans put it, “For that to happen, fonts have to be as easy to read on a screen as they are in a book, otherwise people won’t adapt to it. Fonts have to be designed for the technology.”
Japanese and Chinese Kanji are complex and don’t hold up well at small point sizes on computer screens. They break down completely with bold and italic faces. All of the currently popular Japanese font faces were designed for print and are not screen-opti- mized. LCD screens that display up to 210 dpi will be coming soon from Sharp. But as Evans explained, “Even that is far less than a laser printer at 600 dpi. It has taken Shaken 18 years to build up their library of 150 fonts. How are they going to optimize them for the screen? With stroke technology we can do that far, far faster than anyone can.”
He showed me demonstrations of what the technology can do. The stroke-font scaler rasterizes everything in real time, without any heavy multimedia code. The quality appears very attractive for Web publishing. Evans continued, “This is the future of type. How can you have this stuff happening in Kanji unless you have stroke fonts? Apple can do this and nobody else can … if they wanted to support the technology.”
With Unicode adding more and more useless emoji, and seemly doing little else, it’s time to ask an important question: what the fuck is the Unicode Consortium supposed to be doing anyway?
It’s time to dust off Howard Oakley’s excellent blog post Why we can’t keep stringing along with Unicode, and think about the Normalization problem for file names and the Glyph Variation problem of CJK font sets. These problems fit together surprisingly well. My take is the problems must be tackled together as one thing to find a solution. Let’s take a look at the essential points that Oakley makes:
Unicode is one of the foundations of digital culture. Without it, the loss of world languages would have accelerated greatly, and humankind would have become the poorer. But if the effect of Unicode is to turn a tower of Babel into a confusion of encodings, it has surely failed to provide a sound encoding system for language.
Neither is normalisation an answer. To perform normalisation sufficient to ensure that users are extremely unlikely to confuse any characters with different codes, a great many string operations would need to go through an even more laborious normalisation process than is performed patchily at present.
Pretending that the problem isn’t significant, or will just quietly go away, is also not an answer, unless you work in a purely English linguistic environment. With increasing use of Unicode around the world, and increasing global use of electronic devices like computers, these problems can only grow in scale…
Having grown the Unicode standard from just over seven thousand characters in twenty-four scripts, in Unicode 1.0.0 of 1991, to more than an eighth of a million characters in 135 scripts now (Unicode 9.0), it is time for the Unicode Consortium to map indistiguishable characters to the same encodings, so that each visually distinguishable character is represented by one, and only one, encoding.
The Normalization Problem and the Gylph Variation Problem As Oakley explains earlier in the post: the problem for file system naming boils down to the fact that Unicode represents many visually-identical characters using different encodings. Older file systems like HFS+ used Normalization to resolve the problem, but it is incomplete and inefficient. Modern file systems like APFS avoid Normalization to improve performance.
Glyph variations are the other side of the coin. Instead of identical looking characters using different encodings, we have different looking characters that are variations of the same ‘glyph’. They have the same encoding but they have to be distinguished as variation 1, 2, 3, etc. of the parent glyph. Because this is CJK problem, western software developers traditionally see it as a separate problem for the OpenType partners to solve and not worth considering.
Put another way there needs to be an unambiguous 1-to-1 mapping and an unambiguous 1-1/1-2/1-3-to-1 mapping. I say the problems are two sides of the same coin and must be solved together. Unicode has done a good job of mapping things but it is way past time for Unicode to evolve beyond that and tackle bigger things: lose the western centric problem solving worldview (i.e. let’s fix western encoding issues first and deal with CJK issues later), and start solving problems from a truly globally viewpoint.