TextKit 2 and Apple text layout architecture evolution

No matter what kind of fancy fonts you have, they look bad with poor typography.

Tomihisa Uchida, former lead font engineer of Sha-Ken, FontWorks, Iwata Font

As I wrap my typography related writing, it’s fitting to post about Apple’s text layout architecture evolution one last time. There have been a few changes over the decades. Actually it has been nothing but changes, going forward with a bold vision, pulling back, going forward again in bits and pieces instead of one united comprehensive vision.

Part of my current job includes dealing with lots of vertical text Japanese documents with lots of traditional Kanji characters that juggle different Japanese encoding standards depending on when the documents were created. For that reason I’m keen on robust vertical text layout and easy to access, easy to use high end Japanese font typography features. InDesign is there of course, but its high end page layout features are overkill and time consuming when a good word processor will do. And the those high end Japanese typography features, some of them proprietary, are only available in the InDesign J app, not anywhere else in the Creative Suit apps, certainly not available outside of the Adobe orbit.

Unfortunately there are very few choices outside of Creative Suite InDesign J (and only the J version). Japanese high end features in Word are clunky and confusingly scattered around the UI, Pages vertical text and advanced font feature UI are a joke. The best Japanese word processor on macOS is egword Universal 2, egword has a long history and has used every single macOS text engine at some point. It’s survival is a miracle. Why it is so hard to get international savvy, insanely great typography and layout features that should be standard, universal and easy to use?

QuickDraw GX is the start point for built-in advanced typography on personal computers. Thought it was short-lived, GX parts and concepts live on in Apple OS platforms to this day such as the SF variable system fonts. It’s hard to explain how revolutionary GX was in Asian markets and how much exciting development it fostered, utterly impossible to comprehend from the western biased wikipedia entry. Suffice to say GX was the first multilingual internationally savvy text layout vision: an architecture where all writing systems, languages, various font technologies, scripts and layout models were equally and very well supported, right to left, vertical, contextual and so on.

After Copland OS was cancelled in August 1996, GX text technology morphed into Apple Type Services for Unicode Imaging aka ATSUI. ATSUI along with Apple Advanced Typography (AAT) font tables were some of the technology that made the Hiragino Apple Publishing Glyph Set feature, aka Hiragino Shock, possible in OS X 10.1. The better performing Cocoa based 64-bit Core Text replaced 32-bit Carbon ATSUI in OS X Leopard. macOS Lion AppKit finally gained some vertical text support but it wasn’t very robust as was demonstrated later when TextKit functions migrated to UIKit in iOS 7. Tomihisa Uchida, lead font engineer of Iwata Corporation had this to say:

TextKit (and UIKit) doesn’t support real vertical text layout, the Japanese punctuation and glyph spacing are all wrong. The easiest thing for an app developer to do is bundle a display only Japanese vertical font just for displaying vertical text in the app. Go ask the programmers at Monokakido (egword Universal 2), that’s certainly what they are doing for their iOS Japanese dictionary apps.

‘Real’ vertical text layout remains a low level Core Text coding exercise which Apple really doesn’t encourage unless, “you must do text layout and font handling at a low level, such as developers of layout engines. You should develop your app using a higher-level framework if possible.” In other words, use TextKit.

I suspect one reason for Apple’s recommendation to use TextKit instead of Core Text whenever possible, is the parade of Core Text rendering bugs and security leaks that started cropping up in 2013 and continue to trickle. Low level Core Text grunt work exposes rendering and security bugs so high level TextKit in iOS UIKit and macOS AppKit is the recommended text layout method. Higher level APIs abstract away the complications, which is where Apple wants developers to go. With abstraction in place, Apple can make low level changes with much less developer pain.

TextKit is older than QuickDraw GX?!
There’s a trade-off however: TextKit is 30 years old, far older than Core Text, older than QuickDraw GX even, and shows its age. It’s the last NextStep holdout that never got an upgrade, its text layout features and performance were good enough back in the NextStep days, but not very good today. And it doesn’t abstract well. Apple recognizes this and is finally doing something about it: TextKit 2, Apple’s next-generation text engine.

TextKit 2: abstraction, abstraction abstraction
The best place to start for all things TextKit 2 is the WWDC21 video: Meet TextKit 2. It’s all about abstraction, getting developers away from all the complex, and risky, glyph layout grunt work. The big new takeaway points are:

Non-linear viewport-based layout: TextKit 2 only lays out the appropriate text area that needs to be displayed • edited unlike linear layout which does the whole text document sequentially. You are probably familiar when scrolling long text documents and it takes forever for the text to render down to your current location, like standing on the beach waiting for the last tip of a wave to creep up and reach your feet. That wait is gone in TextKit 2.

TextKit 2 does this using viewport-based layout and rendering. When I saw the WWDC video my first reaction was ‘this reminds me of that shitty new WordPress block editor.’ Other developers had the same reaction. The thing is, once you get used to the shitty WordPress block editor, you don’t want to go back. It’s way more convenient even if the performance isn’t great.

The new TextKit 2 block editor approach promises to deliver convenience and high performance along with, “a robust set of customization points, making it simple to extend the layout system and add your own behaviors” that “also lends itself well to mixing non-text elements into your text layout.” Great, just what I wanted, more emoji + text + whatever-you-want-to-insert-here mishmash.

Glyph Abstraction: aka correct rendering for complex scripts because Apple does the grunt work (in Core Text) so you don’t have to. It removes the drudge, a lot of drudge. On a practical level text selection will work correctly no matter the script or language. With UIKit•TextKit 1, text selection is never reliable even for Japanese.

Safety aka more abstraction: there’s a huge middle section that discusses the bulk of the new TextKit 2 calls and how they work, the goal being using layout ‘elements’ instead of glyphs, strings, paragraphs, etc. Like glyph abstraction, it’s another kind of abstraction and illustrates the TextKit 2 mantra of using higher level objects to control layout. Eliminating the many TextKit 1 layout details developers have to juggle is a good thing.

Inclusive layout yes, but is it high quality layout?
TextKit 2 is the start of a long term migration. When it is done we will only have TextKit 2 that is simply called TextKit. For now, most of TextKit 2 is ‘opt in’ for compatibility. macOS 12 AppKit has all TextKit 2 functions. On the iOS side, UITextField (single-line editable text) has been updated to use TextKit 2 but UITextView (multiline text) has not.

The migration is going to take a while. We won’t get the full TextKit 2 story until WWDC22. Once the transition is complete I suspect Core Text 2 is next. We shall see. Japanese developer reactions have been muted along the lines of, “Apple wants to be inclusive, which is nice, but it doesn’t look like a high end solution.” After all, Japanese developers have been down this road many times before.

And so we circle back to my original question, does TextKit 2 finally deliver international savvy, insanely great typography and layout as standard universal features all developers can implement easily in apps? Time will tell as TextKit 2 has to mature but there is also the basic shortcoming of the baseline font model.

Western created DTP layout is graphics-driven and calculated by margins and font baselines. The western baseline typography model and font metrics is how PostScript and OpenType fonts, and all layout engines evolved. As Adobe is well acquainted with the shortcomings of their own font technology, InDesign J gets around the problems by adding proprietary Kanji virtual body font metrics and Japanese line break algorithms.

So there will always be limits to what TextKit 2 can do, but even if it does eventually delivers insanely great typography for the rest of us, it will only be for apps, never for WebKit. That’s a whole other story.

WWDC22 Update
Sure enough, TextKit 2 has gone default in iOS 16 and macOS Ventura, with UITextView/NSTextView (multiline text) now automatic opt in. All text controls use TextKit 2. The WWDC22 what’s new in TextKit session’s main focus is migration strategies with a few new features. Non-simple text containers have also been added for wrap-around text and text list support has also been added to iOS 16 so all TextKit 2 functionality is the same on both platforms.

On the Safari side WebKit has added CSS ic unit which supposedly delivers “real” vertical text layout on web pages… to which I say, I’ll believe it when I really, truly see beautiful looking vertical text layout web pages everywhere.

---

WWDC23 Update
TextKit 2 continues to mature and gradually work through the iOS and macOS framework ecosystem. Enhancements and tweaks in AppKit are covered in What’s new with text and text interactions, of particular interest is the Internationalization section.

iOS 17 TextKit 2 surprises
There are a few pleasant text surprises for Japanese users in iOS 17. The biggest overall feature is vertical text functionality. We have ‘Vertical text recognition‘ for Live Text (finally, although it’s not as robust as Google’s version found in Google iOS apps), ‘Vertical text support‘ (Apple: “Chinese and Japanese text can be displayed vertically in supported apps and components like Contact Posters, Photos Memories, and Calendar widgets.”), ‘Punctuation white space‘ (“Reduced punctuation white space in Chinese and Japanese scripts improves overall readability and design.”). While not CJK specific Apple also lists ‘Text clipping and collision‘ (Character spacing dynamically adjusts to better accommodate non-Latin scripts) and ‘Improved line breaks‘ (Enhanced line breaking logic results in better readability of rendered text results.) It all reads exactly like TextKit 2 and UIKit dynamic text enhancements covered in the WWDC23 What’s new with text and text interactions presentation.

---