I have not used Adobe Illustrator much the past few years and certainly don’t use it enough to justify buying a Creative Suite subscription that only lasts 12 months. Recently a localization project came in where I needed to edit the original Illustrator file data text. The printer sent me their Illustrator print files and I blithely opened the file with a name that ended with ‘OL’.
As soon as I clicked the body text I realized what OL meant: outline. All the text in a 2 page document with lots of text had been converted to outlines via the Illustrator convert text to outline feature. I couldn’t edit anything. I contacted the printer and received a backup file with the text intact that had not been converted to outlines.
I reflected on this basic Japanese designer practice of converting all Illustrator file text to outlines before sending work files to the printer. It took me back to my days setting up some of the first Japanese PostScript DTP production lines for print companies in Shizuoka. Any printer or high end print service like Lithmatic (a great service company by the way) always requests designers to submit Illustrator work files with all the text data pre-converted to outlines. I hate doing this because it strips away all the font hinting. Font hinting is now only thought of as a screen display thing, but printer font hinting was necessary back in the days of 300~600 DPI PostScript laser printers.
Maybe printer font hinting is no longer necessary in this era of high resolution CTP (computer to plate) on-demand small print runs. Even so, to my eye, stripping out the font hinting reduces the Japanese typographic quality of smaller printed kanji text with their complex glyph strokes. Why is it necessary in this age of PDF workflows to even bother converting text to outline anymore?
It all goes back to the many original sins of the first Adobe Japanese PostScript fonts, the biggest sin being they could not be downloaded to the printer on a job basis…they had to reside on the printer. And they were not cheap: ¥300,000 a pop (back in 1990 when that kind of price was a lot heavier on the wallet) for a single unlimited resolution Japanese PostScript printer font. Not only that, early Japanese PostScript print drivers sucked. They were slow and print jobs would often trip up the RIP job with a memory error or something arcane. Like it or not, print job managers learned to read voodoo tea leave PostScrip error codes to decipher problems, fix the Illustrator file and run the job again. Late work nights were common for production staff.
Usually it was just easier to convert text to outlines which was the godsend feature that arrived with Illustrator v5 along with Japanese Adobe ATM. Instead of buying expensive printer fonts and dealing with incomprehensible PostScript output errors, it was easier (and cheaper) for print service bureaus to require all Illustrator file text data be converted to outlines. This was a time when Illustrator was the workhorse choice for DTP designers in Japan.
All of the PostScript problems were eventually fixed with OpenType fonts and PDF workflows, PostScript fonts themselves will officially die on January 2023. But the PostScript font damage done in Japan will never be fixed. There’s just too much legacy data out there, both in data files, and printer fonts still installed on high end output devices. And Morisawa will always provide legacy OCF fonts for their Passport customers that need them, no matter what Adobe says.
PostScript fonts may be going away, but the ghosts of PostScript fonts, the fine art of outlining Illustrator text data, will be haunting Japan for a very long time.
The start point is the end point. The first time I saw Japanese stroke fonts in action, I had the same revelation Steve Jobs did when he visited Xerox PARC and was blinded seeing the first Graphical User Interface: this is the way it’s supposed to work. Stroke fonts are the way digital Kanji based CJK fonts are supposed to work. But they don’t work that way.
When one set of cultural priorities drive a technology standard, everybody else outside that culture is forced to adapt. This is the duality of technology and context at play. In one context a technology can be constructive and foster innovation, in another context the same technology can be stifling or even destructive. When DTP and PostScript fonts (the baseline font metrics layout model) were brought to Japan (the virtual body layout model) in the late 1980’s, it was both. It brought some innovation, but didn’t address important market needs because the technology was western centric and deemed ‘good enough’ for everywhere else.
Limitations of current outline font technology and the Japanese glyph set problem When writing Hiragino Shock, it struck me how little things have changed, that 20 years later the most basic problems of digital Kanji fonts are still with us: a poor imitation of traditional virtual body layout using inadequate baseline font metrics, large files for each different weight of the same font family, confusing collections of different glyph sets. Regarding the last one I wrote:
For many developers (Adobe included), creating larger and larger fonts was not the best solution to handle the ever-evolving character standards. Adobe did go on to create more Japanese glyph collections but their ability to rally the industry around them diminished over time. Back in 2002 I thought that most Japanese fonts would probably stop at AJ 1-4, leaving Apple in the enviable position of giving users a industry standard super-font with every copy of Mac OS X…not a bad place to be. It’s pretty much how things panned out.
Former Apple MacOS text and layout architecture engineer Yasuo Kida echoed the same opinion and nailed it in the Hiragino Shock interview:
Another regret was that we should have created a solid subset (or subsets) of Apple Publishing Glyph Set / Adobe Japan 1-5. Applications that really need the whole set of APGS are books. Display typefaces obviously do not need the whole set, nor do magazines and so on. We were concerned that font developers might think it necessary, or be pressured from customers to develop whole AJ 1-5 or whatever whole set for every single font that they have when it is not really necessary.
To demonstrate this was not the case, we intentionally left some of our bundle fonts with a smaller subset. But it was not enough, it did not establish a solid character set category that everyone can follow. I should have worked with Adobe to develop a good standard subset based on X 0213 and give it a name. AJ 1-4 is not a good subset as it contains itaiji that many of applications do not need, and it does not contain important characters from JIS X 0213. AJ 1-4 is effectively used as a fallback subset right now.
The necessary glyph set depends on the job. Japanese book publishers need the Adobe Japan 1-5 glyph set, Japanese newspapers publishers need Adobe Japan 1-6 or 1-7, most magazines can make do with Adobe Japan 1-4 while most digital device display for apps only needs Adobe Japan 1-3. The situation is similar for all Kanji based CJK font collections in their respective markets and countries.
Font creation and end-user dilemmas So what are customers supposed to do? Buy the most expensive font license subscriptions with the largest variety of glyph collections for each and every computer? Mix and match? And how much time and effort goes into managing all that when the production line or designer is juggling many different jobs that require many different fonts?
Going back to the original list of problems, let’s look at large glyph collections and large font file issues as one problem, and examine that problem on the font creation side and the font use side.
First of all you have to create all those Kanji glyphs. One of the glaring deficiencies of current outline technology is that every single Kanji must be traced and tweaked extensively. A ‘Standard’ Japanese font has 9,354 glyphs, the Adobe Japan 1-4 character set has 15,444 glyphs, Adobe Japan 1-5 has 20,317 glyphs.
Take that total and multiply by each weight (light, demi-bold, bold, etc.) that has to be designed and tweaked. It can and does take years to create a high quality Japanese font family, and so it goes. You have an idea of how much work goes into font-making and why Japanese fonts are so expensive. Last but not least the font files themselves are large—anywhere from 3 to 8 MB each—because the basic outline model is not efficient for complex shapes like Kanji.
Font file size is one of the problems that TrueType GX née OpenType Font Variation format (OFVF) is supposed to fix, but none of the major Japanese font vendors has released anything in OFVF. It’s a lot more work to do OFVF with Japanese fonts and the payback on all that extra work just isn’t there.
Put another way, if your business choice is using limited resources, do you use those limited resources: (1) to create new font designs that delight customers using the same production methods, or (2) to update and re-release old fonts in a slightly more convenient web font format? It’s a no-brainer that font development resources don’t go into OpenType Variable fonts.
The stroke font solution: taking outline fonts to the next level Any solution has to be two fold: better tools to create large collections of Kanji based CJK fonts more efficiently and economically, and better ways to use them on devices that capture that same efficiency and economy. Stroke fonts address both of these problems.
Stroke fonts have been around for a long time in various forms. The beauty and eternal appeal of the technology is that it is very efficient at creating a large number of glyphs from a small reusable library of parts. This makes them perfect for small, power and memory contained devices, like Apple Watch. In 1995 Fontworks International was busy developing stroke based Japanese fonts of their library that used their built customer font scaler and OFA. What’s that? Let me back up and explain.
Anybody who has studied Chinese or Japanese knows that although each Kanji is unique, certain parts, i.e. strokes, occur again and again, recombining to create new characters. You can get a good feel for this by looking at Chinese or Japanese calligraphy. The brush is the most natural way to write Kanji, and with a little study, you quickly comprehend the order of strokes.
A group of researchers at Stanford in the late 70s~early 80s created prototype ‘Chinese vector fonts‘ based on the Metafont79 system that attempted to use Kanji stroke parts for computer display complex fonts in the extremely restricted memory and storage environments of early computer systems. Don Hosek started a project to refine and enhance those concepts based on Metafont84 before abandoning it for lack of financial support.
One of the real groundbreaking but unnoticed features of QuickDraw GX was Open Font Architecture, OFA for short. It was a simple but powerful concept: plug-in digital font scaler architecture that let font developers create new font technologies that ‘just worked’ by adding plug-in scalers: a plug-in scaler for PostScript, a plug-in scaler for TrueType, a plug-in scaler for stroke fonts and so on. Asian font developers such as Fontworks and DynaLab used OFA to create GX stroke based fonts.
In a similar way to Chinese and Japanese calligraphy and vector font concepts, Fontworks International and DynaLab broke down their Kanji outline fonts into parts that loosely correspond to brush stokes. The crucial difference of these GX stroke fonts was that instead of earlier primitive ‘vector fonts’, they took outline font technology to the next level: smart outline stroke font technology for Kanji based fonts.
These smart outline stroke parts were kept in a library that the stroke-font scaler used to draw the character, resulting in a much smaller and more efficient font. At the time the Fontworks technical director, said that stroke technology “allows us to do weight variations over the full range from Light through Ultra Bold without losing typographic details all in a 4 MB font.” An equivalent OpenType Kanji font family can weigh in around 18 MB although a single OFVF file would likely reduce the size somewhat.
Stroke technology shines is character creation. Once a base library of stroke parts has been created, a designer can create high quality Kanji quickly and easily make adjustment and modifications. Fontworks then applied what they called ‘recipes’ to create different weights from the basic stroke part library. A key feature of the stroke font approach is that it preserves the stroke width as the part is scaled, which is impossible to do with regular outlines. This means efficient high quality blending is possible over the entire rage of font weights, difficult to with OpenType Kanji variable fonts.
Post GX stroke font development Tomihisa Uchida was the lead font engineer for the Fontworks stroke font project that had two goals: QuickDraw GX stroke font versions of the Fontworks library to be bundled by Apple in Copland OS, a font productions tool suite called ‘2X2’ that used stroke technology for glyph creation and editing, but could export in multiple formats (Illustrator, etc.). After Apple hit the kill switch on Copland, Fontworks scrapped their GX stroke font project. The 2X2 production tool morphed into Gaiji Editor that shipped in 2000. Unfortunately Gaiji Master was killed in 2001 when Uchida san left Fontworks and joined Iwata KK to lead their font engineering team.
Since retiring from Iwata, Uchida san is working again on a stroke font based production tool. He showed it to me in 2019. The video I took shows some the features: the ability to switch between ‘full outline’ and ‘stroke outline’, intelligent point handles for efficient part editing and much more. Watching it in action is like seeing his entire Japanese font engineering career knowledge compressed into an application.
Stroke fonts saw quite a bit of action in Japan in the pre-iPhone handset era. The explosion of 3G Internet capable Symbian OS Japanese handsets with Docomo iMode and compatible services, leap frogging display sizes and specs demanded high quality scalable Japanese fonts that fit tiny storage and memory requirements. With their tiny overhead using a library of font parts to create a large variety of fonts, stroke fonts were the perfect solution. There were many handset stroke fonts: Morisawa had KeiType, Ascender Corp (later bought by Monotype) had Compact Asian font technology and Taiwan font developer DynaComware had DigiType.
Stroke fonts: a perfect match for smart devices As far as I know, none of these are still used in the smartphone era. However the advantages of stroke font technology grow exponentially as device sizes shrink. Apple Watch, health trackers, AR glasses. Tiny compact high quality Kanji fonts with a wide variety of weights are essential. There are other non-Roman writing systems that could benefit as well. In 2016 industry sources said Apple was actively searching for “the best stroke font technology.” Maybe Apple plans to do something with it, maybe not.
The problem with OpenType fonts is not the technology, it’s simply that the current OpenType standard is a desktop era solutioin that has not evolved: it has not evolved to address the western cultural priorities that inform the standard, it has not evolved for the smart device era with storage and memory constraints. Let’s assume Apple is doing something with stroke fonts. They can fix a few problems:
Stroke font scaler and format: a stroke based system font doesn’t have to fit within the restrictive OpenType format, but it can be developed in mind to be upwardly compatible, if and when OpenType evolves.
Solving CJK glyph set confusion: because stroke fonts reuse the same basic library parts, supporting the largest CJK glyph sets is not a problem. It also makes CJK all-in-one fonts practical and also solves the problem with current CJK fonts: one design doesn’t fit every culture sensibility, what looks good to Chinese users does not to Japanese users and so on. Glyph variation ‘recipies’ for different cultural regions correctly display the versions that look best.
In addition to this there is one last problem not immediate to stroke fonts but related to vertical layout. As Adobe’s Nat McCully pointed out, ‘real’ vertical layout is impossible to do across apps and on the web with the current OpenType baseline model:
No font metrics for virtual body/em-box glyph space placement: everything has to be accomplished with baseline metrics
No reliable space control
No reliable line breaks
Right now InDesign J is the only application that does real vertical layout because Adobe created proprietary Japanese font table metrics for virtual body layout. There needs to be open standard virtual body metrics included in font tables for robust real vertical layout that works across applications and on web pages because using CSS will never cut it. Along with the stroke fonts Apple could deploy new AAT tables incorporating virtual body metrics, again in mind to be upwardly compatible at a later time, just like TrueType GX variation font AAT was for OpenType variable fonts.
Not that any of this will happen, but I wanted to write about it one last time in the hope that by laying out the issues, the solutions can somehow live on. The end point is the start point.
Many thanks to Tomihisa Uchida and many other great folks from Fontworks, Iwata, Morisawa, Adobe and Apple who shared their time, thoughts and opinions over the years. It was a blast.
As I wrap my typography related writing, it’s fitting to post about Apple’s text layout architecture evolution one last time. There have been a few changes over the decades. Actually it has been nothing but changes, going forward with a bold vision, pulling back, going forward again in bits and pieces instead of one united comprehensive vision.
Part of my current job includes dealing with lots of vertical text Japanese documents with lots of traditional Kanji characters that juggle different Japanese encoding standards depending on when the documents were created. For that reason I’m keen on robust vertical text layout and easy to access, easy to use high end Japanese font typography features. InDesign is there of course, but its high end page layout features are overkill and time consuming when a good word processor will do. And the those high end Japanese typography features, some of them proprietary, are only available in the InDesign J app, not anywhere else in the Creative Suit apps, certainly not available outside of the Adobe orbit.
Unfortunately there are very few choices outside of the Creative Suite world. Japanese high end features in Word are clunky and confusingly scattered around the UI, Pages vertical text and advanced font feature UI are a joke. The best Japanese word processor on macOS is egword Universal 2, egword has a long history and has used every single macOS text engine at some point. It’s survival is a miracle. Why it is so hard to get international savvy, insanely great typography and layout features that should be standard, universal and easy to use?
QuickDraw GX is the start point for built-in advanced typography on personal computers. Thought it was short-lived, GX parts and concepts live on in Apple OS platforms to this day such as the SF variable system fonts. It’s hard to explain how revolutionary GX was in Asian markets and how much exciting development it fostered, utterly impossible to comprehend from the western biased wikipedia entry. Suffice to say GX was the first multilingual internationally savvy text layout architecture where all writing systems, languages, various font technologies, scripts and layout models were equally and very well supported, right to left, vertical, contextual and so on.
After Copland OS was cancelled in August 1996, GX text technology morphed into Apple Type Services for Unicode Imaging aka ATSUI. ATSUI along with Apple Advanced Typography (AAT) font tables were some of the technology that made the Hiragino Apple Publishing Glyph Set feature, aka Hiragino Shock, possible in OS X 10.1. The better performing Cocoa based 64-bit Core Text replaced 32-bit Carbon ATSUI in OS X Leopard. macOS Lion AppKit finally gained some vertical text support but it wasn’t very robust as was demonstrated later when TextKit functions migrated to UIKit in iOS 7. Tomihisa Uchida, lead font engineer of Iwata Corporation had this to say:
TextKit (and UIKit) doesn’t support real vertical text layout, the Japanese punctuation and glyph spacing are all wrong. The easiest thing for an app developer to do is bundle a display only Japanese vertical font just for displaying vertical text in the app. Go ask the programmers at Monokakido (egword Universal 2), that’s certainly what they are doing for their iOS Japanese dictionary apps.
‘Real’ vertical text layout remains a low level Core Text coding exercise which Apple really doesn’t encourage unless, “you must do text layout and font handling at a low level, such as developers of layout engines. You should develop your app using a higher-level framework if possible.” In other words, use TextKit.
I suspect one reason for Apple’s recommendation to use TextKit instead of Core Text whenever possible, is the parade of Core Text rendering bugs and security leaks that started cropping up in 2013 and continue to trickle. Low level Core Text grunt work exposes rendering and security bugs so high level TextKit in iOS UIKit and macOS AppKit is the recommended text layout method. Higher level APIs abstract away the complications, which is where Apple wants developers to go. With abstraction in place, Apple can make low level changes with much less developer pain.
TextKit is older than QuickDraw GX?! There’s a trade-off however: TextKit is 30 years old, far older than Core Text, older than QuickDraw GX even, and shows its age. It’s the last NextStep holdout that never got an upgrade, its text layout features and performance were good enough back in the NextStep days, but not very good today. And it doesn’t abstract well. Apple recognizes this and is finally doing something about it: TextKit 2, Apple’s next-generation text engine.
TextKit 2: abstraction, abstraction abstraction The best place to start for all things TextKit 2 is the WWDC21 video: Meet TextKit 2. It’s all about abstraction, getting developers away from all the complex, and risky, glyph layout grunt work. The big new takeaway points are:
Non-linear viewport-based layout: TextKit 2 only lays out the appropriate text area that needs to be displayed • edited unlike linear layout which does the whole text document sequentially. You are probably familiar when scrolling long text documents and it takes forever for the text to render down to your current location, like standing on the beach waiting for the last tip of a wave to creep up and reach your feet. That wait is gone in TextKit 2.
TextKit 2 does this using viewport-based layout and rendering. When I saw the WWDC video my first reaction was ‘this reminds me of that shitty new WordPress block editor.’ Other developers had the same reaction. The thing is, once you get used to the shitty WordPress block editor, you don’t want to go back. It’s way more convenient even if the performance isn’t great.
The new TextKit 2 block editor approach promises to deliver convenience and high performance along with, “a robust set of customization points, making it simple to extend the layout system and add your own behaviors” that “also lends itself well to mixing non-text elements into your text layout.” Great, just what I wanted, more emoji + text + whatever-you-want-to-insert-here mishmash.
Glyph Abstraction: aka correct rendering for complex scripts because Apple does the grunt work (in Core Text) so you don’t have to. It removes the drudge, a lot of drudge. On a practical level text selection will work correctly no matter the script or language. With UIKit•TextKit 1, text selection is never reliable even for Japanese.
Safetyaka more abstraction: there’s a huge middle section that discusses the bulk of the new TextKit 2 calls and how they work, the goal being using layout ‘elements’ instead of glyphs, strings, paragraphs, etc. Like glyph abstraction, it’s another kind of abstraction and illustrates the TextKit 2 mantra of using higher level objects to control layout. Eliminating the many TextKit 1 layout details developers have to juggle is a good thing.
Inclusive layout yes, but is it high quality layout? TextKit 2 is the start of a long term migration. When it is done we will only have TextKit 2 that is simply called TextKit. For now, most of TextKit 2 is ‘opt in’ for compatibility. macOS 12 AppKit has all TextKit 2 functions. On the iOS side, UITextField (single-line editable text) has been updated to use TextKit 2 but UITextView (multiline text) has not.
The migration is going to take a while. We won’t get the full TextKit 2 story until WWDC22. Once the transition is complete I suspect Core Text 2 is next. We shall see. Japanese developer reactions have been muted along the lines of, “Apple wants to be inclusive, which is nice, but it doesn’t look like a high end solution.” After all, Japanese developers have been down this road many times before.
And so we circle back to my original question, does TextKit 2 finally deliver international savvy, insanely great typography and layout as standard universal features all developers can implement easily in apps? Time will tell as TextKit 2 has to mature but there is also the basic shortcoming of the baseline font model.
Western created DTP layout is graphics-driven and calculated by margins and font baselines. The western baseline typography model and font metrics is how PostScript and OpenType fonts, and all layout engines evolved. As Adobe is well acquainted with the shortcomings of their own font technology, InDesign J gets around the problems by adding proprietary Kanji virtual body font metrics and Japanese line break algorithms.
So there will always be limits to what TextKit 2 can do, but even if it does eventually delivers insanely great typography for the rest of us, it will only be for apps, never for WebKit. That’s a whole other story.
WWDC22 Update Sure enough, TextKit 2 has gone default in iOS 16 and macOS Ventura, with UITextView/NSTextView (multiline text) now automatic opt in. All text controls use TextKit 2. The WWDC22 what’s new in TextKit session’s main focus is migration strategies with a few new features. Non-simple text containers have also been added for wrap-around text and text list support has also been added to iOS 16 so all TextKit 2 functionality is the same on both platforms.
On the Safari side WebKit has added CSS ic unit which supposedly delivers “real” vertical text layout on web pages… to which I say, I’ll believe it when I really, truly see beautiful looking vertical text layout web pages everywhere.
…could not be downloaded on a per-job basis, had to reside permanently on the printer and turned the production process upside down: service bureaus and printers were suddenly dictating to designers which fonts could and could not be used.
For every PostScript device, users had to invest in font licenses― and Japanese fonts were very expensive…Morisawa and Adobe came up with the idea of marketing two flavors of PostScript printer fonts: low-resolution (up to 600 dpi) and unlimited. A single unlimited-resolution RIP Japanese font cost ¥218,000 (about $2,000).
Adobe fixed some of the problems with CID Japanese PostScript fonts but the expensive and forced upgrade in 1995 did not go well with Japanese customers. Actually it was more of a revolt. Morisawa was forced to backdown and support the older format. I remember spending endless hours upgrading output devices, feeding piles of unique key Morisawa font floppy disks into a Mac that downloaded the CID update to the output device, and often broke in the middle of an install.
Adobe didn’t really fix most Japanese font problems until the OpenType Japanese format arrived a few years later…with yet another expensive upgrade. Adobe and Morisawa wanted to move everybody to OpenType as quickly as possible but the CID upgrade disaster killed that possibility and customers stuck with OCF Type 1 Japanese fonts on output devices as long as they could.
Japanese designers and printers who don’t want to deal with OpenType upgrades and options for extended character sets, IVS enabled fonts, etc., have annual licensing programs like Morisawa Passport to download what they need, but at ¥49,800 per year per CPU it does not come cheap. A close reading of the Morisawa font catalog lists everything still available in OCF format and it’s not exactly clear if OCF is disappearing January 2023 with regular Type 1 (the Japanese notice does not specifically list OCF) but it really needs to go. At this point I imagine it’s mostly there for compatibility with very old RIP systems and files. CID font support ended in 2020. I hope Morisawa uses the opportunity to streamline their catalog option clutter and start delivering Japanese OpenType variable fonts though I think it’s a long shot.
It’s hard for people not of that era to understand how important the Seybold publication was, the vanguard of computers and printing, and the genius of Jonathan Seybold creating connections between the two with his Seybold Seminars. It’s not an exaggeration to say that desktop publishing (DTP) and web browsers would not have happened the way they did without him and his father John Seybold. That’s why I was thrilled to find the Oral History of Jonathan Seybold courtesy of the Computer History Museum. There are lots of mentions of Seybold out there but very little history. It’s great that his contribution and history is on record now, required viewing if you have any interesting in computers and print and how they came together. A revolution that changed print forever.