Stroke Fonts Forever

The start point is the end point. The first time I saw Japanese stroke fonts in action, I had the same revelation Steve Jobs did when he visited Xerox PARC and was blinded seeing the first Graphical User Interface: this is the way it’s supposed to work. Stroke fonts are the way digital Kanji based CJK fonts are supposed to work. But they don’t work that way.

When one set of cultural priorities drive a technology standard, everybody else outside that culture is forced to adapt. This is the duality of technology and context at play. In one context a technology can be constructive and foster innovation, in another context the same technology can be stifling or even destructive. When DTP and PostScript fonts (the baseline font metrics layout model) were brought to Japan (the virtual body layout model) in the late 1980’s, it was both. It brought some innovation, but didn’t address important market needs because the technology was western centric and deemed ‘good enough’ for everywhere else.

Limitations of current outline font technology and the Japanese glyph set problem
When writing Hiragino Shock, it struck me how little things have changed, that 20 years later the most basic problems of digital Kanji fonts are still with us: a poor imitation of traditional virtual body layout using inadequate baseline font metrics, large files for each different weight of the same font family, confusing collections of different glyph sets. Regarding the last one I wrote:

For many developers (Adobe included), creating larger and larger fonts was not the best solution to handle the ever-evolving character standards. Adobe did go on to create more Japanese glyph collections but their ability to rally the industry around them diminished over time. Back in 2002 I thought that most Japanese fonts would probably stop at AJ 1-4, leaving Apple in the enviable position of giving users a industry standard super-font with every copy of Mac OS X…not a bad place to be. It’s pretty much how things panned out.

Former Apple MacOS text and layout architecture engineer Yasuo Kida echoed the same opinion and nailed it in the Hiragino Shock interview:

Another regret was that we should have created a solid subset (or subsets) of Apple Publishing Glyph Set / Adobe Japan 1-5. Applications that really need the whole set of APGS are books. Display typefaces obviously do not need the whole set, nor do magazines and so on. We were concerned that font developers might think it necessary, or be pressured from customers to develop whole AJ 1-5 or whatever whole set for every single font that they have when it is not really necessary.

To demonstrate this was not the case, we intentionally left some of our bundle fonts with a smaller subset. But it was not enough, it did not establish a solid character set category that everyone can follow. I should have worked with Adobe to develop a good standard subset based on X 0213 and give it a name. AJ 1-4 is not a good subset as it contains itaiji that many of applications do not need, and it does not contain important characters from JIS X 0213. AJ 1-4 is effectively used as a fallback subset right now.

The necessary glyph set depends on the job. Japanese book publishers need the Adobe Japan 1-5 glyph set, Japanese newspapers publishers need Adobe Japan 1-6 or 1-7, most magazines can make do with Adobe Japan 1-4 while most digital device display for apps only needs Adobe Japan 1-3. The situation is similar for all Kanji based CJK font collections in their respective markets and countries.

Font creation and end-user dilemmas
So what are customers supposed to do? Buy the most expensive font license subscriptions with the largest variety of glyph collections for each and every computer? Mix and match? And how much time and effort goes into managing all that when the production line or designer is juggling many different jobs that require many different fonts?

Going back to the original list of problems, let’s look at large glyph collections and large font file issues as one problem, and examine that problem on the font creation side and the font use side.

First of all you have to create all those Kanji glyphs. One of the glaring deficiencies of current outline technology is that every single Kanji must be traced and tweaked extensively. A ‘Standard’ Japanese font has 9,354 glyphs, the Adobe Japan 1-4 character set has 15,444 glyphs, Adobe Japan 1-5 has 20,317 glyphs.

Take that total and multiply by each weight (light, demi-bold, bold, etc.) that has to be designed and tweaked. It can and does take years to create a high quality Japanese font family, and so it goes. You have an idea of how much work goes into font-making and why Japanese fonts are so expensive. Last but not least the font files themselves are large—anywhere from 3 to 8 MB each—because the basic outline model is not efficient for complex shapes like Kanji.

Font file size is one of the problems that TrueType GX née OpenType Font Variation format (OFVF) is supposed to fix, but none of the major Japanese font vendors has released anything in OFVF. It’s a lot more work to do OFVF with Japanese fonts and the payback on all that extra work just isn’t there.

Put another way, if your business choice is using limited resources, do you use those limited resources: (1) to create new font designs that delight customers using the same production methods, or (2) to update and re-release old fonts in a slightly more convenient web font format? It’s a no-brainer that font development resources don’t go into OpenType Variable fonts.

The stroke font solution: taking outline fonts to the next level
Any solution has to be two fold: better tools to create large collections of Kanji based CJK fonts more efficiently and economically, and better ways to use them on devices that capture that same efficiency and economy. Stroke fonts address both of these problems.

Stroke fonts have been around for a long time in various forms. The beauty and eternal appeal of the technology is that it is very efficient at creating a large number of glyphs from a small reusable library of parts. This makes them perfect for small, power and memory contained devices, like Apple Watch. In 1995 Fontworks International was busy developing stroke based Japanese fonts of their library that used their built customer font scaler and OFA. What’s that? Let me back up and explain.

Anybody who has studied Chinese or Japanese knows that although each Kanji is unique, certain parts, i.e. strokes, occur again and again, recombining to create new characters. You can get a good feel for this by looking at Chinese or Japanese calligraphy. The brush is the most natural way to write Kanji, and with a little study, you quickly comprehend the order of strokes.

A group of researchers at Stanford in the late 70s~early 80s created prototype ‘Chinese vector fonts‘ based on the Metafont79 system that attempted to use Kanji stroke parts for computer display complex fonts in the extremely restricted memory and storage environments of early computer systems. Don Hosek started a project to refine and enhance those concepts based on Metafont84 before abandoning it for lack of financial support.

One of the real groundbreaking but unnoticed features of QuickDraw GX was Open Font Architecture, OFA for short. It was a simple but powerful concept: plug-in digital font scaler architecture that let font developers create new font technologies that ‘just worked’ by adding plug-in scalers: a plug-in scaler for PostScript, a plug-in scaler for TrueType, a plug-in scaler for stroke fonts and so on. Asian font developers such as Fontworks and DynaLab used OFA to create GX stroke based fonts.

In a similar way to Chinese and Japanese calligraphy and vector font concepts, Fontworks International and DynaLab broke down their Kanji outline fonts into parts that loosely correspond to brush stokes. The crucial difference of these GX stroke fonts was that instead of earlier primitive ‘vector fonts’, they took outline font technology to the next level: smart outline stroke font technology for Kanji based fonts.

These smart outline stroke parts were kept in a library that the stroke-font scaler used to draw the character, resulting in a much smaller and more efficient font. At the time the Fontworks technical director, said that stroke technology “allows us to do weight variations over the full range from Light through Ultra Bold without losing typographic details all in a 4 MB font.” An equivalent OpenType Kanji font family can weigh in around 18 MB although a single OFVF file would likely reduce the size somewhat.

Stroke technology shines is character creation. Once a base library of stroke parts has been created, a designer can create high quality Kanji quickly and easily make adjustment and modifications. Fontworks then applied what they called ‘recipes’ to create different weights from the basic stroke part library. A key feature of the stroke font approach is that it preserves the stroke width as the part is scaled, which is impossible to do with regular outlines. This means efficient high quality blending is possible over the entire rage of font weights, difficult to with OpenType Kanji variable fonts.

Smarter stroke fonts take outline fonts to the next level

Post GX stroke font development
Tomihisa Uchida was the lead font engineer for the Fontworks stroke font project that had two goals: QuickDraw GX stroke font versions of the Fontworks library to be bundled by Apple in Copland OS, a font productions tool suite called ‘2X2’ that used stroke technology for glyph creation and editing, but could export in multiple formats (Illustrator, etc.). After Apple hit the kill switch on Copland, Fontworks scrapped their GX stroke font project. The 2X2 production tool morphed into Gaiji Editor that shipped in 2000. Unfortunately Gaiji Master was killed in 2001 when Uchida san left Fontworks and joined Iwata KK to lead their font engineering team.

Since retiring from Iwata, Uchida san is working again on a stroke font based production tool. He showed it to me in 2019. The video I took shows some the features: the ability to switch between ‘full outline’ and ‘stroke outline’, intelligent point handles for efficient part editing and much more. Watching it in action is like seeing his entire Japanese font engineering career knowledge compressed into an application.

Demo of Tomihisa Uchida’s stroke outline font editing tool. The ability to edit and switch back and forth from full outline to stroke parts might look mundane to the untrained eye but can greatly accelerate Kanji glyph creation

Stroke fonts saw quite a bit of action in Japan in the pre-iPhone handset era. The explosion of 3G Internet capable Symbian OS Japanese handsets with Docomo iMode and compatible services, leap frogging display sizes and specs demanded high quality scalable Japanese fonts that fit tiny storage and memory requirements. With their tiny overhead using a library of font parts to create a large variety of fonts, stroke fonts were the perfect solution. There were many handset stroke fonts: Morisawa had KeiType, Ascender Corp (later bought by Monotype) had Compact Asian font technology and Taiwan font developer DynaComware had DigiType.

Stroke fonts: a perfect match for smart devices
As far as I know, none of these are still used in the smartphone era. However the advantages of stroke font technology grow exponentially as device sizes shrink. Apple Watch, health trackers, AR glasses. Tiny compact high quality Kanji fonts with a wide variety of weights are essential. There are other non-Roman writing systems that could benefit as well. In 2016 industry sources said Apple was actively searching for “the best stroke font technology.” Maybe Apple plans to do something with it, maybe not.

The problem with OpenType fonts is not the technology, it’s simply that the current OpenType standard is a desktop era solutioin that has not evolved: it has not evolved to address the western cultural priorities that inform the standard, it has not evolved for the smart device era with storage and memory constraints. Let’s assume Apple is doing something with stroke fonts. They can fix a few problems:

  • Stroke font scaler and format: a stroke based system font doesn’t have to fit within the restrictive OpenType format, but it can be developed in mind to be upwardly compatible, if and when OpenType evolves.
  • Solving CJK glyph set confusion: because stroke fonts reuse the same basic library parts, supporting the largest CJK glyph sets is not a problem. It also makes CJK all-in-one fonts practical and also solves the problem with current CJK fonts: one design doesn’t fit every culture sensibility, what looks good to Chinese users does not to Japanese users and so on. Glyph variation ‘recipies’ for different cultural regions correctly display the versions that look best.

In addition to this there is one last problem not immediate to stroke fonts but related to vertical layout. As Adobe’s Nat McCully pointed out, ‘real’ vertical layout is impossible to do across apps and on the web with the current OpenType baseline model:

  • No font metrics for virtual body/em-box glyph space placement: everything has to be accomplished with baseline metrics
  • No reliable space control
  • No reliable line breaks
kanji box 3
A virtual body Kanji with approximate baseline overlay red line.
character spacing@2x
Character spacing and tracking are different concepts in Japanese typography as well.
Kumihan character spacing is the amount of space measured between outer virtual box boundaries, the 10% red area
On the other hand tracking, ji-tsume in Japanese, is the amount of space measured between type face boundaries and virtual body boundaries, the blue area

Right now InDesign J is the only application that does real vertical layout because Adobe created proprietary Japanese font table metrics for virtual body layout. There needs to be open standard virtual body metrics included in font tables for robust real vertical layout that works across applications and on web pages because using CSS will never cut it. Along with the stroke fonts Apple could deploy new AAT tables incorporating virtual body metrics, again in mind to be upwardly compatible at a later time, just like TrueType GX variation font AAT was for OpenType variable fonts.

Not that any of this will happen, but I wanted to write about it one last time in the hope that by laying out the issues, the solutions can somehow live on. The end point is the start point.

Many thanks to Tomihisa Uchida and many other great folks from Fontworks, Iwata, Morisawa, Adobe and Apple who shared their time, thoughts and opinions over the years. It was a blast.


TextKit 2 and Apple text layout architecture evolution

No matter what kind of fancy fonts you have, they look bad with poor typography.

Tomihisa Uchida, former lead font engineer of Sha-Ken, FontWorks, Iwata Font

As I wrap my typography related writing, it’s fitting to post about Apple’s text layout architecture evolution one last time. There have been a few changes over the decades. Actually it has been nothing but changes, going forward with a bold vision, pulling back, going forward again in bits and pieces instead of one united comprehensive vision.

Part of my current job includes dealing with lots of vertical text Japanese documents with lots of traditional Kanji characters that juggle different Japanese encoding standards depending on when the documents were created. For that reason I’m keen on robust vertical text layout and easy to access, easy to use high end Japanese font typography features. InDesign is there of course, but its high end page layout features are overkill and time consuming when a good word processor will do. And the those high end Japanese typography features, some of them proprietary, are only available in the InDesign J app, not anywhere else in the Creative Suit apps, certainly not available outside of the Adobe orbit.

Unfortunately there are very few choices outside of Creative Suite InDesign J (and only the J version). Japanese high end features in Word are clunky and confusingly scattered around the UI, Pages vertical text and advanced font feature UI are a joke. The best Japanese word processor on macOS is egword Universal 2, egword has a long history and has used every single macOS text engine at some point. It’s survival is a miracle. Why it is so hard to get international savvy, insanely great typography and layout features that should be standard, universal and easy to use?

QuickDraw GX is the start point for built-in advanced typography on personal computers. Thought it was short-lived, GX parts and concepts live on in Apple OS platforms to this day such as the SF variable system fonts. It’s hard to explain how revolutionary GX was in Asian markets and how much exciting development it fostered, utterly impossible to comprehend from the western biased wikipedia entry. Suffice to say GX was the first multilingual internationally savvy text layout vision: an architecture where all writing systems, languages, various font technologies, scripts and layout models were equally and very well supported, right to left, vertical, contextual and so on.

After Copland OS was cancelled in August 1996, GX text technology morphed into Apple Type Services for Unicode Imaging aka ATSUI. ATSUI along with Apple Advanced Typography (AAT) font tables were some of the technology that made the Hiragino Apple Publishing Glyph Set feature, aka Hiragino Shock, possible in OS X 10.1. The better performing Cocoa based 64-bit Core Text replaced 32-bit Carbon ATSUI in OS X Leopard. macOS Lion AppKit finally gained some vertical text support but it wasn’t very robust as was demonstrated later when TextKit functions migrated to UIKit in iOS 7. Tomihisa Uchida, lead font engineer of Iwata Corporation had this to say:

TextKit (and UIKit) doesn’t support real vertical text layout, the Japanese punctuation and glyph spacing are all wrong. The easiest thing for an app developer to do is bundle a display only Japanese vertical font just for displaying vertical text in the app. Go ask the programmers at Monokakido (egword Universal 2), that’s certainly what they are doing for their iOS Japanese dictionary apps.

‘Real’ vertical text layout remains a low level Core Text coding exercise which Apple really doesn’t encourage unless, “you must do text layout and font handling at a low level, such as developers of layout engines. You should develop your app using a higher-level framework if possible.” In other words, use TextKit.

I suspect one reason for Apple’s recommendation to use TextKit instead of Core Text whenever possible, is the parade of Core Text rendering bugs and security leaks that started cropping up in 2013 and continue to trickle. Low level Core Text grunt work exposes rendering and security bugs so high level TextKit in iOS UIKit and macOS AppKit is the recommended text layout method. Higher level APIs abstract away the complications, which is where Apple wants developers to go. With abstraction in place, Apple can make low level changes with much less developer pain.

TextKit is older than QuickDraw GX?!
There’s a trade-off however: TextKit is 30 years old, far older than Core Text, older than QuickDraw GX even, and shows its age. It’s the last NextStep holdout that never got an upgrade, its text layout features and performance were good enough back in the NextStep days, but not very good today. And it doesn’t abstract well. Apple recognizes this and is finally doing something about it: TextKit 2, Apple’s next-generation text engine.

TextKit 2: abstraction, abstraction abstraction
The best place to start for all things TextKit 2 is the WWDC21 video: Meet TextKit 2. It’s all about abstraction, getting developers away from all the complex, and risky, glyph layout grunt work. The big new takeaway points are:

Non-linear viewport-based layout: TextKit 2 only lays out the appropriate text area that needs to be displayed • edited unlike linear layout which does the whole text document sequentially. You are probably familiar when scrolling long text documents and it takes forever for the text to render down to your current location, like standing on the beach waiting for the last tip of a wave to creep up and reach your feet. That wait is gone in TextKit 2.

TextKit 2 does this using viewport-based layout and rendering. When I saw the WWDC video my first reaction was ‘this reminds me of that shitty new WordPress block editor.’ Other developers had the same reaction. The thing is, once you get used to the shitty WordPress block editor, you don’t want to go back. It’s way more convenient even if the performance isn’t great.

The new TextKit 2 block editor approach promises to deliver convenience and high performance along with, “a robust set of customization points, making it simple to extend the layout system and add your own behaviors” that “also lends itself well to mixing non-text elements into your text layout.” Great, just what I wanted, more emoji + text + whatever-you-want-to-insert-here mishmash.

Glyph Abstraction: aka correct rendering for complex scripts because Apple does the grunt work (in Core Text) so you don’t have to. It removes the drudge, a lot of drudge. On a practical level text selection will work correctly no matter the script or language. With UIKit•TextKit 1, text selection is never reliable even for Japanese.

Safety aka more abstraction: there’s a huge middle section that discusses the bulk of the new TextKit 2 calls and how they work, the goal being using layout ‘elements’ instead of glyphs, strings, paragraphs, etc. Like glyph abstraction, it’s another kind of abstraction and illustrates the TextKit 2 mantra of using higher level objects to control layout. Eliminating the many TextKit 1 layout details developers have to juggle is a good thing.

Inclusive layout yes, but is it high quality layout?
TextKit 2 is the start of a long term migration. When it is done we will only have TextKit 2 that is simply called TextKit. For now, most of TextKit 2 is ‘opt in’ for compatibility. macOS 12 AppKit has all TextKit 2 functions. On the iOS side, UITextField (single-line editable text) has been updated to use TextKit 2 but UITextView (multiline text) has not.

The migration is going to take a while. We won’t get the full TextKit 2 story until WWDC22. Once the transition is complete I suspect Core Text 2 is next. We shall see. Japanese developer reactions have been muted along the lines of, “Apple wants to be inclusive, which is nice, but it doesn’t look like a high end solution.” After all, Japanese developers have been down this road many times before.

And so we circle back to my original question, does TextKit 2 finally deliver international savvy, insanely great typography and layout as standard universal features all developers can implement easily in apps? Time will tell as TextKit 2 has to mature but there is also the basic shortcoming of the baseline font model.

Western created DTP layout is graphics-driven and calculated by margins and font baselines. The western baseline typography model and font metrics is how PostScript and OpenType fonts, and all layout engines evolved. As Adobe is well acquainted with the shortcomings of their own font technology, InDesign J gets around the problems by adding proprietary Kanji virtual body font metrics and Japanese line break algorithms.

So there will always be limits to what TextKit 2 can do, but even if it does eventually delivers insanely great typography for the rest of us, it will only be for apps, never for WebKit. That’s a whole other story.

WWDC22 Update
Sure enough, TextKit 2 has gone default in iOS 16 and macOS Ventura, with UITextView/NSTextView (multiline text) now automatic opt in. All text controls use TextKit 2. The WWDC22 what’s new in TextKit session’s main focus is migration strategies with a few new features. Non-simple text containers have also been added for wrap-around text and text list support has also been added to iOS 16 so all TextKit 2 functionality is the same on both platforms.

On the Safari side WebKit has added CSS ic unit which supposedly delivers “real” vertical text layout on web pages… to which I say, I’ll believe it when I really, truly see beautiful looking vertical text layout web pages everywhere.

Another new WWDC…another new text layout api chart
If I had a dollar for every new CSS feature promise that delivers ‘real’ vertical text layout, I’d be very rich by now.

Japanese Typography and Font Posts

This is a collection of long form Japanese typography posts. They were written as stand alone pieces, so there is some background explanation overlap, always a weak point of the blog format.

Sha-Ken finally launches a website

The Sha-Ken web site uses Noto Sans JP…

Sha-Ken finally got a website, listing themselves as Sha-Ken Co., Ltd. Big deal so what, except that it’s not 1991 or even 2001. A font company doing business without a website until 2021 is tantamount to not doing business at all. That it has taken them some 30 years to acknowledge digital fonts on the web says all that you need to know about the tragic Sha-Ken saga. The site has samples of famous classic Sha-Ken fonts that are certainly the OpenType launch candidates due in 2024, a full archive is due to go online in May. There is also a dreamy catchphrase: ‘building the next future of fonts and type culture,’ but the site design uses Google Noto Sans JP fonts…not a Sha-Ken web font to be found. Actually Sha-Ken is building supermarkets while leaving digital font development of their venerable library to Morisawa. And as FeliCa Dude gleefully notes, the company hagiography does not mention the 1999 infamous tax scandal.

There is another 1999 scandal that few people outside of Apple and Sha-Ken of that time know about: Apple almost bought Sha-Ken, or at least the library. In 1998 and 1999 I was close to Ross Evans, the founder of Fontworks. I wrote about Fontworks’ QuickDraw GX based Japanese stroke font technology and font designs that were due to be bundled as the default Japanese system font in the ill-fated MacOS Copeland. That did not pan out obviously but Ross had many close contacts within the Apple typography engineering and publishing market groups and kindly plugged me into that world including Jeff Martin who was the VP overseeing DTP marketing and developer relation efforts.

One day in early February 1999 I woke up to find a email from Jeff’s secretary asking me to contact Jeff right away. I did and his email immediately bounced back from the Apple corporate email server ‘user does not exist’, so did the secretary email address. In the space of 5 hours Jeff Martin’s position in Apple, along with his email address had vanished into thin air.

I later pieced together the story from Ross and others in Apple, many of whom soon followed Jeff out the door: Jeff had arranged a deal between Apple and Sha-Ken but the deal along with Jeff’s career at Apple were ‘Steved’ at the last second, right about the time that Japanese tax officers found millions of Japanese yen hidden away in basement safes at Sha-Ken corporate headquarters (worth more than 200 million worth in USD at the time). I never found out what he wanted to ask me.

Apple of course signed a licensing deal with Screen for the Hiragino fonts that became the MacOS X Japanese font, and iOS Japanese font later on. Perhaps the Screen deal was another reason for the collapse of the Sha-Ken one. We’ll never know. Still it’s fun to think about what would have happened had the Apple Sha-Ken deal gone through as planned.

The Second Wave of Japanese Desktop Publishing

This article was published in The Seybold Report Volume 30 Number 6, November 2000. It was a strange time when QuickDraw GX 3rd party developments like Japanese stroke fonts and plug-in font scalers were up in the air without a clear migration path to the newly announced MacOS X ATSUI Carbon framework, OpenType Japanese fonts and InDesign J had been announced but were months away from shipping, and Japanese print customers were angry with Adobe and Morisawa for forcing font upgrades on everybody. It’s a snapshot of Japanese publishing in transition as Steve Jobs was saving Apple and tossing the Japanese print market a few goodies like extended Japanese glyph sets, Adobe was caught flat footed and was furious with Apple.

Japanese DTP arrived with Apple’s NTX-J PostScript printer and Linotype’s first Japanese PostScript imagesetter in 1989. They came at the right time: The early- ’90s economy was bubbling, companies had money to burn and Japanese DTP took off. It was a young, booming market and it forgave many mistakes that would haunt the industry later.

By 1996, the go-go days were gone and they would not come back. By this time, DTP tools (Quark Xpress, Illustrator, Photoshop and, to a smaller extent, PageMaker) had captured nearly 40 percent of the production process. For a conservative industry like Japanese publishing, this was phenomenal―until compared to the West. There, in the same amount of time, practically the entire industry converted to DTP production. Japan is still about 40 percent and holding.

What happened? Three things: the economy, the failure of Western technology to address Japanese issues, and the same old cultural differences Westerners have been running into since Commodore Perry knocked down the doors in 1853. They just do things differently here, get used to it.

The economic crunch hit the publishing market hard, and it hasn’t really recovered. Consider the book and magazine market: On average, this year’s revenues are down 3.4 percent over last year.

The advertising industry has been down, but has recently shown signs of recovering: On average, revenues are up 10.4 percent so far this year.

Basically, the publishing industry overextended itself in the bubble years of the early ’90s and is still adjusting.

Some of the difficulty is the publication distribution structure. Anybody in the business will tell you there just isn’t enough shelf space and return rates are climbing. Internet publishing is not much of a factor yet. Japanese book retail laws, which prohibit stores from selling below suggested retail prices, will be relaxed soon and will put tremendous pressure on the current price structure, which supports lots of marginally profitable items

The Japanese market is also a highly segmented one. There are the major newspapers, which have very specialized workflow systems and in-house fonts. There is the book industry, which runs the gamut from standard tanko bon hardbacks to the ingenious, very popular pocket-size bunko hon paperbacks (which really have no equal in the West). There are lots of printers, using both DTP and traditional systems, that cover everything in-between. In a class of their own are manga: the hefty weekly comic books you can find on subway luggage racks, left there by salarymen too embarrassed to take them home.

Hiragino: In Japan, Mac OS X will include a family of fonts, Hiragino, with an impressive 17,000 characters.

On the whole, the industry is still in a slump and has been treading water for the last few years, with relatively little investment in new technologies such as computer-to-plate. At the same time, competition and consolidation in the printing sector have forced prices down-more than 30 percent in the past year alone-so there is a great need to reduce production costs and increase efficiency. This is the great challenge of the Japanese publishing industry. Meeting it will require new investment in hardware and software. The challenge for vendors is to provide the right products. Those who watch the market and listen to customers carefully will find opportunity.

The font technology failure
A designer once said, “Fonts are like air.” They are the environment in which DTP lives and thrives. Compared to the West, the Japanese DTP air is pretty thin.

One of the great tragedies was the refusal of Shaken KK, the largest, most popular, and perhaps most beautiful Japanese font library, to license its faces to Adobe or have anything to do with PostScript. It kept its proprietary machines and formats, losing over 40 percent of its market share (and lots of prestige and talent) in the bargain. The industry is filled with ex-Shaken people. A low point came a year and a half ago when tax raiders found several safes in the company basement filled with 10,000-yen bills; the president of Shaken, Yuko Ishii had stashed away the equivalent of 150 million USD in the early ’80s and was now reporting it as profit. Shaken was not persecuted as the statue of limitations for tax evasion had elapsed, but the company reputation was destroyed.

This sorry state of affairs had two huge repercussions. It denied a large part of the Japanese font legacy to designers working with DTP. (Imagine a world without Helvetica and Times and you get the idea.) It also left the only other competitor, Morisawa, which did license its library to Adobe, with a practical monopoly. But these were not the only problems.

Initial problems
The first implementation of Japanese PostScript Type 1 fonts had serious shortcomings. First of all, they could not be downloaded on a per-job basis, but had to reside permanently on the printer. This turned the production process upside down; service bureaus and printers were suddenly dictating to designers which fonts could and could not be used. For every PostScript device, users had to invest in font licenses― and Japanese fonts are very expensive. In a market where making a single face can easily require ¥30,000,000 (about $280,000) and consume two years, font vendors made sure they got maximum return on their investments. 

To this end, Morisawa and Adobe came up with the idea of marketing two flavors of PostScript printer fonts: low-resolution (up to 600 dpi) and unlimited. A single unlimited-resolution Japanese font costs ¥218,000 (about $2,000).

The situation became more complex when ATM J arrived on the scene in late 1993. Morisawa’s ATM fonts used hardware binding to lock the font to the hard disk. This resulted in a high maintenance operation, because hardware binding required installer floppies for each font and for every CPU. (This remains true even today.) If the user got a new hard disk, he had to send the old floppies back to the vendor and exchange them, for a fee, for new ones. Large operations had the extra burden of keeping track of the floppy that had been installed on each CPU. On top of all this, each installation took time; putting a family of 15 Kanji fonts on two or three Macs could easily eat up a day.

In addition, compared to traditional systems, these fonts had primitive typographic features: lack of pair kerning, lack of proper metrics and poor Gaiji support. (Gaiji are Kanji characters outside of the current JIS [Japanese Industrial Standard] and Unicode encoding sets and are not included in a standard font. They comprise many “unofficial” Kanji characters, mistakes and misinterpretations, and seldom used Kanji passed down for generations, long before printing presses and governments created standards.

These Gaiji characters are widely used in people and place names. To this day, they are a reason for publishers to hang on to their proprietary systems. Last but not least, Japanese PostScript fonts were not cross platform.

The first fix
Adobe has been trying to fix the font problem. Its first attempt was CID (character-key ID). CID divorced two-byte PostScript fonts from dependence on any encoding scheme and allowed character subsets to be downloaded to the printer on a job basis, laying the foundation for PDF font embedding in Japanese Acrobat 4.0 as well.

Unfortunately the CID font upgrade was a marketing washout. Upgrades were not cheap, and there were few benefits the end user could really see. Morisawa also decided to tweak some glyph designs and the font metrics. Designers would open old files with the new fonts and find they had to redo everything. They stayed away in droves. As a result, the installed base is still overwhelmingly OCF (original composite font), which has slowed acceptance of PDF as a major production format.

Dueling fixes
Adobe’s final solution for the Japanese font problem is OpenType. OpenType will be a very big deal and, in the long term, fundamentally change the market. OpenType finally brings true font parity between Windows and Mac. Morisawa and Fontworks are busy preparing OpenType upgrades. It marks the first time their font libraries will be available cross-platform. Both companies see Windows as an opportunity for revenue growth―something that has been missing on the Mac side for years. Users will finally be able to choose the platform that best fits their needs and budgets without worrying about font problems.

This will be especially interesting because Mac OS X and OpenType are both arriving in 2001. According to a Japanese Publishing Consortium survey in 1999, Apple has nearly 80 percent of the DTP installed base. However, a large share of the machines in the printing and publishing areas (but not in design) are pre-G3 PowerPC systems that will have to be upgraded to run Mac OS X. Apple Japan’s “We’ll let the market decide” marketing statements may become a true nightmare if the market chooses Windows instead of Mac OS X

Adobe Japan 1-4
The most important feature of Japanese OpenType will be the Adobe Japan 1-4 character collection. This is the big Gaiji fix everybody has been waiting for. AJ 1-4 addresses the Gaiji encoding problem and provides a consistent and cross-platform way for font developers to add Gaiji to OpenType fonts. Adobe says the collection is “to provide professional publishers with a glyph set that will suit their needs.” AJ 1-4 will add 6,000 glyphs. They will include additional Latin characters: macroned vowels, italic forms, fractions, third and quarter-width numerals and punctuation and Latin ligatures.

They will also offer a rich set of Kanji options:

  • Annotated forms: These are popular ‘shorthand’ glyphs for such Kanji as “corporation limited” as well as specially designed numbers:
  • Hiragana, katakana and Kanji ligatures
  • Alternate kana for horizontal and vertical writing
  • Ruby glyphs: These are special kana characters that are placed next to difficult Kanji to show readers how to pronounce them. Many software DTP packages offer this feature. However, traditional ruby glyphs are a different design so they are easy to read at the small point sizes in which ruby glyphs are always used. With OpenType, these special designs can finally be included in a Japanese font.
  • Kanji and Kanji variants
    Anyone familiar with Japanese knows the language has changed more in 200 years than English has. Particularly after World War II, many Kanji were simplified and, even though the older, more complex Kanji disappeared from newspapers, magazines and school textbooks, they still live on in books, maps and proper names. There are also many different variations of certain Kanji, such as the ‘nabe’ character for the name ‘Watanabe’ (see illustration).
Kanji glyph variations

AJ 1-4 was created by Adobe in cooperation with Morisawa, Fontworks and other industry experts. It’s not going to satisfy everybody, but it’s an excellent effort that should take good care of the majority. The only drawback is timing, coming as it does on the heels of the CID upgrade catastrophe. Adobe and Morisawa are planning to market two packages: Standard, which will have the same features as current CID fonts, and Pro, which will have AJ 1- 4 Gaiji and advanced layout.

Character layout
Advanced typography is another major OpenType feature and one that, until now, has been an elusive goal. After all, Apple has been pushing advanced typography for years, first with QuickDraw GX and now with ATSUI (Apple Text Services for Unicode Imaging) in Mac OS X. Apple is still pushing its solution; the Hiragino fonts for Mac OS X will include both OpenType layout and Apple Advanced Layout (AAT) tables.

Advanced layout is not a trivial feature to add, particularly for a complex written language such as Japanese. As noted in the Gaiji section above, there are all kinds of ligatures, vertical variants and arcane Kanji. But a well thought-out and executed set of features can vastly simplify the user experience and address many shortcomings of Japanese DTP.

One example of how this can be used is Morisawa’s new Mincho Kyoiku font. This font family has special glyphs required in Japanese school texts that help children learn how to write.

Because it is based on the standard Mincho design, Morisawa digital type manager Nobuo Tomita said, it would make sense to implement the different glyphs as variants in the standard Mincho font package instead of selling them separately as they do now.

GX legacies
At the World Wide Developer Conference and at an Apple font developers’ conference in Tokyo shortly afterward, Apple stated that Apple Advanced Typography tables were the “recommended” way to add advanced layout features. One Apple engineer described the differences between the two approaches this way. “AAT is pretty much a superset of OpenType [layout], with much better performance.

There’s no real pressure for us to do this [support OpenType layout directly in ATSUI].” The Hiragino OpenType fonts for Mac OS X have both AAT and OpenType layout tables.

ATSUI is a system-level API, while OpenType layout is an application-level API. One very nice thing about ATSUI is that application developers don’t have to know all the details of font table formats and how to process them. ATSUI just takes care of it.

OpenType layout requires the application to do all the processing, so developers have to know a lot about a language if they want the features to work right. On the Windows side, there is the OpenType Layout Services Library, which helps somewhat; but whether on Windows or Mac, application developers will not find it easy to implement OpenType layout.

Adobe, of course, has the advantage over the competition here because its own type layout library, Cooltype, is built into every Adobe application. Adobe will pull out the stops and bundle the KozukaMincho Pro OpenType Kanji font, which of course does not have AAT tables, with InDesign J, due to be shipped in the first quarter of 2001.

Unhappy developers
Apple’s AAT solution might be better technology, but some font developers are not amused. Morisawa’s Nobuo Tomita said, “Apple has not been very forward with their plans…. AAT has lots [of features] that don’t have much to do with Japanese. We plan to go with Adobe’s cross-platform solution.” If Morisawa doesn’t support AAT, then the case is pretty much closed. 

Adobe, predictably, isn’t very enthusiastic about AAT either. “I went over to Apple and told them to just kill it…. We went through this whole thing with GX already,” said Adobe’s Asian Font product manager Julie Ma. Given Apple’s lack of marketing enthusiasm for ATSUI―it is ignoring some fantastic third-party technology that makes use of it, such as Fontworks International’s stroke font technology (announced at Tokyo Seybold two years ago)―it is understandable that developers are hesitant. Quark, for one, is hedging its bets; when asked which technology it plans to go with, Masato Nishimura of Quark Japan said, “We’ll wait to hear from our customers before we make any decisions.”

Another Apple move not sitting well with font developers is the additional Gaiji characters that are part of the 17,000-character set in the Mac OS X Hiragino OpenType Pro fonts. These go well beyond the Adobe Japan 1-4 collection.

Not much is clear at this point, but the extra Gaiji appear to be another ATSUI-only feature guaranteed not to work with Adobe software (nor with any other application that does not support the Apple API). The feature by itself is not bad, but Apple has not communicated with Japanese font developers well, and that is bad. The developers need detailed information to define their own OpenType features and to make sure it all works seamlessly with Hiragino and Mac OS X, and that information has not been forthcoming. As one font developer moaned, “Just when things are looking better, this happens and everybody loses.” If Apple really wants Mac OS X to be the best publishing solution, it will have to get its Developer Relations act in high gear, and fast.

In the end, Japanese OpenType’s greatest achievement will be that it delivers most of the features the high-end customers have been asking for: Gaiji, robust cross-platform support (allowing developers to put both OpenType layout and Apple’s AAT layout in the same package), and embedding and dynamic downloading. It will finally bring the Japanese printing environment to the same level the West has had for years. As Adobe Japan’s Seiichiro Miyajima, group manager for product marketing said, “To put it simply, it will be the same as using Roman [one-byte].” The timing for Japanese OpenType is difficult, but the benefits are very real. The industry will get there―eventually.

Japanese layout and the promise of InDesign J
In order to really understand the differences between Japanese and Western layout, you have to go back to elementary school. Think back to second grade when you were just learning cursive; the teacher gave you a piece of paper with lines. This was the bottom line, that was the top line. She taught you to write along the bottom line.

In Japan, the students get a piece of paper with rows of little boxes. The teacher tells the kids to write Kanji characters in the centers of the boxes. Japanese typographic layout is exactly the same; it is all based on little boxes, known as virtual bodies. It is also called the grid system, because the middle of each box is one center point on a grid. Everything is calculated from the center; there is no baseline. The whole system makes sense when you realize it all comes from the days of block type; the virtual bodies are blocks of metal type set end to end. The vocabulary, the measurements and the aesthetics all evolved from that.

Virtual bodies
Unlike DTP layout, which is graphics-driven, Japanese text composition, called kumihan, is driven by how much text will fit in a given space. Designers know how many characters (virtual bodies) are supposed to be on a line and on a page before they start composition, and this is how they discuss layout with writers and editors. Western composition is calculated from margins, a wholly different concept.

Adobe Japan InDesign product manager Hiroshi Miyamoto, who has typesetting experience, explained the difference. “It’s very important for [Japanese] operators to know the number of characters on each page, and it’s difficult to work that way with PageMaker, Quark Xpress and Illustrator. PostScript Japanese fonts (OCF, CID, OpenType) have no virtual-body information whatsoever, and historically there is no baseline in Japanese kumihan. But since PostScript fonts have only baseline information, that’s how all DTP software developed. Japanese fonts have different baseline positioning, so that when you change the font, the line breaks change.

Traditional systems used the grid system, so as long as you kept the same size, any font line breaking stayed uniform, and even if you did change the size, the grid made it very easy to make adjustments. If you are trying to calculate grids in Xpress, the point system doesn’t match well, so you end up with too many, or too few characters on the page, and the designer compensates by incrementing in .01-point values, which is overkill…. Also it’s very difficult to handle pages that have both English and Japanese, a real problem for magazines.”

It’s not impossible to do quality kumihan with DTP. It just takes more effort than traditional systems, and everything has to be done manually, which is inefficient. For text-heavy and specialty market segments still using traditional or proprietary systems (newspapers, books, manga), the cost benefits of DTP are just not there.

Japanese engine
InDesign J, announced this past February at MacWorld Tokyo and due to be shipped in the first quarter of 2001, is the first major piece of DTP software that follows Japanese typesetting conventions. To this end, InDesign J has a special Japanese typesetting engine and Japanese layout grid. Because there is only baseline information in PostScript fonts, InDesign J calculates and creates a full-width zenkaku virtual body, and this allows the designer to calculate the number of characters in a line, on a page or in an InDesign text frame, based on point size and line spacing.

Another basic Japanese kumihan function that professionals expect to have is known as Ji-Dori-Gyo-Dori. In traditional systems the priority is: (1) Calculate additions to the previous line, if the first character of the next line is a punctuation mark, (2) Calculate the shift kana so the first character of the next line is not a punctuation mark, (3) Calculate the average spacing of each kana.

This sounds easy to do, but it’s not, for two reasons. First, as noted above, everything is calculated from the virtual body’s central grid point. Second, there are special sets of and Kanji known as “Classes.” Proprietary systems have these highly specialized line-breaking rules and Class settings built in, so they can automatically calculate and adjust the spacing very quickly.

They are also very good with mixed Japanese and English, which is half-width: two Roman characters equal the width of one Kanji. InDesign J has all these features. InDesign also has support for the kyu measurement standard used in traditional systems, where one kyu equals 0.25 millimeters. Quark Xpress supports kyu too, but Adobe promised that InDesign will have a more accurate method to calculate Japanese kyu and will eliminate errors that occur in the current versions of Xpress and PageMaker.

Quark’s reaction
Quark is not standing on the sidelines. “We foresee strong growth in the market as Japanese font technologies become richer and easier to use. More and more designers will feel comfortable using DTP and especially Quark Xpress instead of traditional typesetting methods,” Desktop Systems Product Manager Masato Nishimura explained. He could not discuss specific features of Quark Xpress 5.0J, other than saying it will not be carbon-compliant. “That version will be a release subsequent to 5.0.” But he did mention some features requested by Japanese Xpress users that will very likely appear: placing each character in a desirable position (center) even if a line contains characters of different font sizes; scaling characters by center line grid in line-width increments in vertical text; enabling users to easily keep a full-width zenkaku type setting mode for all Japanese characters and punctuation.

However, Quark will not pursue InDesign’s approach of matching all the features of traditional kumihan. Adobe has followed the Japanese Industrial Standard (JIS) X4051 typesetting and composition specification (written by an ex-Shaken employee, no less), which is the kumihan “bible.” (Follow it and you’ll never commit compositional sin.) Quark has not. “We want to give the most important features to the majority of users,” Nishimura explained.

“Many designers feel they have enough features already,” a sentiment I have heard from others in the industry. From a layout perspective, the current tools are good enough. But they are not good enough for the unconverted 60 percent who are dealing with composition rather than layout. So the big question is, will this new environment, the second wave of Japanese DTP arriving in 2001―OpenType, advanced layout, Gaiji extended character sets, cross platform Kanji, InDesign J, Quark Xpress 5.0J―finally convince the unconverted to make the change to the open standards-based digital workflow that DTP has to offer? The answer: definitely maybe.

Cultural differences
“At this point I don’t think the issue has to do with technical reasons,” Toppan Printing KK Publishing Information Center’s Shinichi Konno, told me. “It has to do with middle managers and their way of thinking. Many companies know they have to move to a digital workflow at some point; some are even feeling panicky, but those kind of people don’t want to take the responsibility for changing over.” Having worked in the Japanese publishing industry for more than ten years, I can attest to the very conservative mentality of middle managers, and I can even agree―up to a point. I have been in many situations where a company desperately wants to change but doesn’t know how. If you listen carefully and talk with the managers on a level they understand, amazing transformations can take place. But it takes time and patient effort.

There is another factor at work, too: shokunin mentality. “Shokunin” is a tricky Japanese word. The dictionary translation lists the word as artisan or craftsman, but the nuance implies a long apprenticeship, an obsession with quality and detail and pride― traits the Japanese are famous for. I call them specialists, and Japanese culture does cherish specialists. The publishing industry is full of them, and if there is one thing you don’t do, it is tell a specialist his business. But specialists tend not to see the big picture. They focus on their own little area of expertise and have trouble understanding company-wide benefits of digital workflow.

This is where Adobe is playing a smart, subtle hand with InDesign J. It is using Japanese standards such as JIS X4051 and is also using all the specialty words. The marketing material is carefully scrubbed clean of foreign words such as layout (reeauhto), and text (tekisuto), instead using native specialist words like kumihan. Even the official product name foregoes the lazy ‘J’ for Japanese that most product names go for, using the formal “Nihongo ban” instead.

Quark, too, has a solid, if slightly different, vision. “The nucleus of our strategy is media-independent publishing, the next generation publishing model that encompasses print, Web, E-books …and beyond,” Nishimura explained. He said Quark hopes to free publishers to create and manage content so they can deliver it anywhere at low cost, with a layout appropriate to each media format.

It sounds good, but here again―and this is the whole point of this article―developers simply cannot do a localization job and sling their products out in the market. It takes time, careful study of market needs, and thorough knowledge of the segments and specialties therein. For example, media asset management has not done very well as a category in Japan. But it can, as it has for CRS Systems, the Japanese localizer and distributor of Canto’s Cumulus. CRS has seen Cumulus gradually grow from a software package into a intranet integration business, as Japanese managers slowly understand the cost savings from integrated workflow. However, like the font and layout situation, it has to happen in a Japanese context to which shokunin, the specialists, and middle managers can relate. 

Current Japanese DTP workflow is far from perfect. It is expensive and unreliable; it has primitive typography and layout and a limited variety of Kanji fonts that lack necessary Gaiji characters. Perfect Japanese DTP workflow should have a rich set of fonts, rich typography, layout and composition, Gaiji character sets, and theability to pass files between computers seamlessly and print without any problems.

The second wave of Japanese DTP will not magically convert the unconverted. It will be a gradual change, as companies see and truly understand digital workflow as cost reduction and potential revenue growth. The highly segregated nature of the Japanese market also means there are plenty of specialty niches to fill with all manner of InDesign plug-ins, Quark Xtensions and flexible media management systems, giving the 60 percent all the more reason to convert to a standards-based digital workflow.

A friend once told me that Japanese society is like an egg. In the West, the shell is brittle. A small amount of force from the inside can break through and bring change. But the Japanese shell is tough and rubbery; a small force can only bend the walls. It takes much more force to break the Japanese egg. But once it breaks, change can come very quickly. The publishing market will take time, but it will change. And change is opportunity.

Stroke Fonts: The Future of Japanese Type?

OpenType, the Adobe Japan 1-4 Character Collection and Unicode do a good job of breaking the current limitations of Japanese fonts. Unfortunately, encoding is not the only problem. You also have to create all those extra Kanji. The proposed Adobe Japan 1- 4 character set alone is 6,000 characters. One of the glaring deficiencies of current outline technology is that every single Kanji must be traced and tweaked extensively. A regular Japanese font has nearly 7,000 characters; add the Adobe Japan 1-4 character set for OpenType and you have a whopping 13,000 glyphs.

Take that total and multiply by each weight (light, demi-bold, bold) that has to be designed, and you have an idea of how much work goes into font-making and why Japanese fonts are so expensive. The fonts are large—anywhere from 3 to 8 MB each—and they are not very efficient, which is one of the reasons why Japanese Multiple Master fonts will never happen.

An interesting feature of Mac OS (both 9 and X) is the Open Font Architecture (OFA) that evolved from QuickDraw GX. As the name implies, OFA is an open plug-in architecture. It works with any font technology, be it PostScript, TrueType, or a completely new technology, such as stroke fonts created specifically for Japanese, Chinese and Korean writing systems.

Recombinant parts
Anybody who has studied Chinese or Japanese knows that although each Kanji is unique, certain parts occur again and again, recombining to create new characters. You can get a good feel for this by looking at Chinese or Japanese calligraphy. The brush is the most natural way to write Kanji, and with a little study, you quickly comprehend the strict order of each stroke.

In a similar way, Fontworks International broke down its Kanji fonts into parts that loosely correspond to brush stokes. These stroke parts are kept in a library that the stroke-font scaler uses to draw the character, resulting in a much smaller and more efficient font. Torsten Buck, Fontworks technical director, told me that stroke technology “allows us to do weight variations over the full range from Light through Ultra Bold without losing typographic details,” all in a 4 MB font. An equivalent PostScript Kanji font family can weigh in around 18 MB.

Where stroke technology really shines is character creation. Once a base library of parts has been created, a designer can create high quality Kanji quickly and easily. A key feature of stroke technology is that it preserves the stroke width as the part is scaled, which is impossible to do with PostScript outlines.

To create a new Kanji, you simply use a similar character, swap out the parts that need to be changed and, perhaps, fine-tune them. Stroke fonts are a great technology to break the Japanese font logjam of high cost and limited design. It could do for Japanese fonts what Fontographer did for Western typography: put real power in the hands of designers and open up the floodgates.

Apple fumbles
Unfortunately, Apple doesn’t see this as an opportunity. It creates great things like ATSUI and OFA and then forgets about them, leaving third-party developers holding the baby. When Fontworks approached Adobe’s InDesign team to discuss stroke font technology, Adobe didn’t even know what ATSUI was. Clearly somebody in Apple Developer Relations wasn’t doing the job.

Fontworks’s original plan was to have full-blown Japanese stroke fonts with advanced typography and multiple weights. But in the face of Apple’s indifference, it postponed the plan and instead re- leased Gaiji Master last February. This is a slimmed down, Gaiji only version of 2×2, Fontworks’s heavy-duty in-house stroke font development environment. It is a fascinating piece of software, and it reminds you of the early Macintosh days when the graphics software was thrilling and cutting-edge.

E-book opportunity
One reason why Fontworks went ahead with this is that it believes E-books and E-publishing could be a huge market in Japan. And stroke fonts could be the key. As Fontworks President Ross Evans put it, “For that to happen, fonts have to be as easy to read on a screen as they are in a book, otherwise people won’t adapt to it. Fonts have to be designed for the technology.”

Japanese and Chinese Kanji are complex and don’t hold up well at small point sizes on computer screens. They break down completely with bold and italic faces. All of the currently popular Japanese font faces were designed for print and are not screen-opti- mized. LCD screens that display up to 210 dpi will be coming soon from Sharp. But as Evans explained, “Even that is far less than a laser printer at 600 dpi. It has taken Shaken 18 years to build up their library of 150 fonts. How are they going to optimize them for the screen? With stroke technology we can do that far, far faster than anyone can.”

He showed me demonstrations of what the technology can do. The stroke-font scaler rasterizes everything in real time, without any heavy multimedia code. The quality appears very attractive for Web publishing. Evans continued, “This is the future of type. How can you have this stuff happening in Kanji unless you have stroke fonts? Apple can do this and nobody else can … if they wanted to support the technology.”

Japanese Typography and Font Posts

This is a collection of long form Japanese typography posts. They were written as stand alone pieces, so there is some background explanation overlap, always a weak point of the blog format.