Tag: TextKit 2

WWDC22 Wish List

It is hard to be enthusiastic about this year’s WWDC when Apple’s entire integrated software/hardware business model is coming under attack. With so much distraction these days there’s not much of a wish list, just a few observations for Apple Pay, Apple Maps and Text Layout.

Apple Pay
First up of course, is Apple Pay. After Jennifer Bailey’s WWDC21 appearance where she announced home keys, hotel keys, office keys and ID for iOS 15 Wallet, and the separate Tap to Pay on iPhone PR announcement release in January, I don’t think Jennifer will be in the WWDC22 keynote. She’s not going to appear just to explain that Apple Pay is not a monopoly, that’s Tim’s job with CEO level pay grade, it’s unlikely she’s doing to appear to just recap details of what’s already been announced.

Bailey’s job is to announce new features, and I don’t think that after the big iOS 15 rollout of new Wallet features and Tap to Pay on iPhone there’s nothing really new. And it’s not her job to announce new frameworks, that’s what the sessions are for. Things that I have been wishing for these past few years such include easier, more open NFC Pass certification process and/or new frameworks for developers to access the secure element for payments or use Tap to Pay on iPhone. There needs to a clearer path for developers who want to use the secure element for payments (Wallet) or iPhone as payment terminal (Tap to Pay on iPhone).

Apple needs to open up the NFC/Secure Element Pass certification process and clarify the process

The only possible ‘new’ Apple Pay Wallet feature I can think of is the long in the works Code Payments. It has been lurking in the iOS shadows since iOS 13, so long that Apple legal inserted official mention in a recent Apple Pay & Privacy web page update: “When you make a payment using a QR code pass in Wallet, your device will present a unique code and share that code with the pass provider to prevent fraud.” If Apple Pay delivers native device generated QR code payments without a network connection, just like all Apple Pay cards to date, it would be quite a coup but by itself, but probably not worth a Jennifer Bailey appearance. Other future goodies like passport in Wallet or ID in Wallet for other countires are too far out to mention, at least in the iOS 16 time frame.

Apple Maps
The only new Apple Maps feature that suggests itself is AR enhanced ‘Look Around’ indoor maps for stations. That’s the conclusion after examining the current (February ~ May 2022) backpack image collection in Tokyo, Osaka, Kyoto and Nagoya. It is highly focused on stations, and stations such as Shinjuku, Tokyo, Shibuya, Ikebukuro, etc., are mostly underground, surrounded with densely packed extensive maze like malls.

This means Apple image collection in Japan is going indoors for the first time, likely at pre-arranged times when people are scarce. This is hard to do at a place like Shinjuku station as multiple companies collectively manage the entire site (JR East, Odakyu, Keio, Seibu, Tokyo Metropolitan Bureau of Transportation, Tokyo Metro, just to name a few).

Apple needs something new with indoor maps as the current incarnation is inadequate for stations. As Google Maps Live has shown in Tokyo station, AR walking guidance is a good fit for indoor maps that navigate users through intricate, information dense underground station mazes, though Google’s version has its problems. New and improved, AR enhanced “Look Around” style indoor station maps with walking directions that seamlessly guide users from transit gate to final destination would be far more useful than they are now.

Recent image collection suggests Indoor Station Maps might be coming in iOS 16
Apple backpack Image Collection was mostly focused on station areas in 2021

Overall, I am not optimistic that Apple Maps in Japan can become a top tier digital map service. The local 3rd party map and transit data suppliers that Apple depends on to make up the bulk of the Japanese service are decidedly not top tier. Old problems remain unfixed. In the case of the main Japanese map data supplier things have deteriorated.

Increment P (IPC) was 100% owned by Pioneer but was sold to Polaris Capital Group in June 2021 with a new CEO (ex Oracle Japan) who quickly changed the name to GeoTechnologies Inc. Under hedge fund Polaris Capital Group led management the company has been busy inflating the number of cushy company director positions, never a good sign, and pushing out shitty ad-ware apps like Torima. The focus is leveraging assets not building them.

Apple’s Japanese map problem can only be fixed by dumping low quality GeoTechnologies for a top quality digital map supplier like Zenrin (the amateurish UK backed Open Street Map effort in Japan is not worth serious consideration) or Apple aggressively mapping Japan themselves. Apple has not pursued either option: the image collection effort in Japan is leisurely and limited, its use remains restricted to Look Around. Until this changes, expect more of the same old fundamental Japanese map problems in iOS 16 and beyond. Apple Maps is a collection of many different service parts. Some evolve and improve, some do not. Let’s hope for a good outcome with the data Apple is collecting for indoor station maps.

Apple Typography TextKit 2 migration
WWDC21 saw the unveiling of TextKit 2, the next generation replacement for the 30 year old TextKit, older than QuickDraw GX even, but much less capable. TextKit 2 marked the start of a long term migration with most of TextKit 2 initially ‘opt in’ for compatibility. We’ll find out how much of TextKit 2 will evolve to default on with an ‘opt out’. There are holes to fill too: the iOS side didn’t get all the TextKit 2 features of macOS such as UITextView (multiline text), some of the planned features like NSTextContainer apparently didn’t make the final cut either. We should get a much more complete package at WWDC22. Once the TextKit 2 transition is complete, I wonder if a Core Text reboot is next.

watchOS 9 Express Cards with Power Reserve?
Mark Gurman reported that watchOS 9 will have “a new low-power mode that is designed to let its smartwatch run some apps and features without using as much battery life.” While this sounds like Express Cards with Power Reserve (transit cards, student ID, hotel-home-car-office keys) and it might even mimic the iPhone feature to some degree, it will not be the real thing. Power Reserve on iPhone is a special mode where iOS powers down itself down but leaves the lights on for direct secure element NFC transactions. iOS isn’t involved at all.

Real Power Reserve requires an Apple silicon design that supports the hardware feature on Apple Watch, it cannot be added with a simple software upgrade. Until that happens, a new watchOS 9 low-power mode means that watchOS still babysits Express Cards, but anything that gives us better battery life than what we have now is a good thing. We’ll find out later this year if Apple Watch series 8 is the real Power Reserve deal.

Enjoy the keynote and have a good WWDC.

TextKit 2 and Apple text layout architecture evolution

No matter what kind of fancy fonts you have, they look bad with poor typography.

Tomihisa Uchida, former lead font engineer of Sha-Ken, FontWorks, Iwata Font

As I wrap my typography related writing, it’s fitting to post about Apple’s text layout architecture evolution one last time. There have been a few changes over the decades. Actually it has been nothing but changes, going forward with a bold vision, pulling back, going forward again in bits and pieces instead of one united comprehensive vision.

Part of my current job includes dealing with lots of vertical text Japanese documents with lots of traditional Kanji characters that juggle different Japanese encoding standards depending on when the documents were created. For that reason I’m keen on robust vertical text layout and easy to access, easy to use high end Japanese font typography features. InDesign is there of course, but its high end page layout features are overkill and time consuming when a good word processor will do. And the those high end Japanese typography features, some of them proprietary, are only available in the InDesign J app, not anywhere else in the Creative Suit apps, certainly not available outside of the Adobe orbit.

Unfortunately there are very few choices outside of Creative Suite InDesign J (and only the J version). Japanese high end features in Word are clunky and confusingly scattered around the UI, Pages vertical text and advanced font feature UI are a joke. The best Japanese word processor on macOS is egword Universal 2, egword has a long history and has used every single macOS text engine at some point. It’s survival is a miracle. Why it is so hard to get international savvy, insanely great typography and layout features that should be standard, universal and easy to use?

QuickDraw GX is the start point for built-in advanced typography on personal computers. Thought it was short-lived, GX parts and concepts live on in Apple OS platforms to this day such as the SF variable system fonts. It’s hard to explain how revolutionary GX was in Asian markets and how much exciting development it fostered, utterly impossible to comprehend from the western biased wikipedia entry. Suffice to say GX was the first multilingual internationally savvy text layout vision: an architecture where all writing systems, languages, various font technologies, scripts and layout models were equally and very well supported, right to left, vertical, contextual and so on.

After Copland OS was cancelled in August 1996, GX text technology morphed into Apple Type Services for Unicode Imaging aka ATSUI. ATSUI along with Apple Advanced Typography (AAT) font tables were some of the technology that made the Hiragino Apple Publishing Glyph Set feature, aka Hiragino Shock, possible in OS X 10.1. The better performing Cocoa based 64-bit Core Text replaced 32-bit Carbon ATSUI in OS X Leopard. macOS Lion AppKit finally gained some vertical text support but it wasn’t very robust as was demonstrated later when TextKit functions migrated to UIKit in iOS 7. Tomihisa Uchida, lead font engineer of Iwata Corporation had this to say:

TextKit (and UIKit) doesn’t support real vertical text layout, the Japanese punctuation and glyph spacing are all wrong. The easiest thing for an app developer to do is bundle a display only Japanese vertical font just for displaying vertical text in the app. Go ask the programmers at Monokakido (egword Universal 2), that’s certainly what they are doing for their iOS Japanese dictionary apps.

‘Real’ vertical text layout remains a low level Core Text coding exercise which Apple really doesn’t encourage unless, “you must do text layout and font handling at a low level, such as developers of layout engines. You should develop your app using a higher-level framework if possible.” In other words, use TextKit.

Core Text has been in the news sometimes, not in a good way

I suspect one reason for Apple’s recommendation to use TextKit instead of Core Text whenever possible, is the parade of Core Text rendering bugs and security leaks that started cropping up in 2013 and continue to trickle. Low level Core Text grunt work exposes rendering and security bugs so high level TextKit in iOS UIKit and macOS AppKit is the recommended text layout method. Higher level APIs abstract away the complications, which is where Apple wants developers to go. With abstraction in place, Apple can make low level changes with much less developer pain.

TextKit is older than QuickDraw GX?!
There’s a trade-off however: TextKit is 30 years old, far older than Core Text, older than QuickDraw GX even, and shows its age. It’s the last NextStep holdout that never got an upgrade, its text layout features and performance were good enough back in the NextStep days, but not very good today. And it doesn’t abstract well. Apple recognizes this and is finally doing something about it: TextKit 2, Apple’s next-generation text engine.

TextKit 2: abstraction, abstraction abstraction
The best place to start for all things TextKit 2 is the WWDC21 video: Meet TextKit 2. It’s all about abstraction, getting developers away from all the complex, and risky, glyph layout grunt work. The big new takeaway points are:

Non-linear viewport-based layout: TextKit 2 only lays out the appropriate text area that needs to be displayed • edited unlike linear layout which does the whole text document sequentially. You are probably familiar when scrolling long text documents and it takes forever for the text to render down to your current location, like standing on the beach waiting for the last tip of a wave to creep up and reach your feet. That wait is gone in TextKit 2.

TextKit 2 does this using viewport-based layout and rendering. When I saw the WWDC video my first reaction was ‘this reminds me of that shitty new WordPress block editor.’ Other developers had the same reaction. The thing is, once you get used to the shitty WordPress block editor, you don’t want to go back. It’s way more convenient even if the performance isn’t great.

The new TextKit 2 block editor approach promises to deliver convenience and high performance along with, “a robust set of customization points, making it simple to extend the layout system and add your own behaviors” that “also lends itself well to mixing non-text elements into your text layout.” Great, just what I wanted, more emoji + text + whatever-you-want-to-insert-here mishmash.

Glyph Abstraction: aka correct rendering for complex scripts because Apple does the grunt work (in Core Text) so you don’t have to. It removes the drudge, a lot of drudge. On a practical level text selection will work correctly no matter the script or language. With UIKit•TextKit 1, text selection is never reliable even for Japanese.

Safety aka more abstraction: there’s a huge middle section that discusses the bulk of the new TextKit 2 calls and how they work, the goal being using layout ‘elements’ instead of glyphs, strings, paragraphs, etc. Like glyph abstraction, it’s another kind of abstraction and illustrates the TextKit 2 mantra of using higher level objects to control layout. Eliminating the many TextKit 1 layout details developers have to juggle is a good thing.

Text engine evolution map

Inclusive layout yes, but is it high quality layout?
TextKit 2 is the start of a long term migration. When it is done we will only have TextKit 2 that is simply called TextKit. For now, most of TextKit 2 is ‘opt in’ for compatibility. macOS 12 AppKit has all TextKit 2 functions. On the iOS side, UITextField (single-line editable text) has been updated to use TextKit 2 but UITextView (multiline text) has not.

The migration is going to take a while. We won’t get the full TextKit 2 story until WWDC22. Once the transition is complete I suspect Core Text 2 is next. We shall see. Japanese developer reactions have been muted along the lines of, “Apple wants to be inclusive, which is nice, but it doesn’t look like a high end solution.” After all, Japanese developers have been down this road many times before.

And so we circle back to my original question, does TextKit 2 finally deliver international savvy, insanely great typography and layout as standard universal features all developers can implement easily in apps? Time will tell as TextKit 2 has to mature but there is also the basic shortcoming of the baseline font model.

Western created DTP layout is graphics-driven and calculated by margins and font baselines. The western baseline typography model and font metrics is how PostScript and OpenType fonts, and all layout engines evolved. As Adobe is well acquainted with the shortcomings of their own font technology, InDesign J gets around the problems by adding proprietary Kanji virtual body font metrics and Japanese line break algorithms.

So there will always be limits to what TextKit 2 can do, but even if it does eventually delivers insanely great typography for the rest of us, it will only be for apps, never for WebKit. That’s a whole other story.

Japanese developer reactions have been muted too…

WWDC22 Update
Sure enough, TextKit 2 has gone default in iOS 16 and macOS Ventura, with UITextView/NSTextView (multiline text) now automatic opt in. All text controls use TextKit 2. The WWDC22 what’s new in TextKit session’s main focus is migration strategies with a few new features. Non-simple text containers have also been added for wrap-around text and text list support has also been added to iOS 16 so all TextKit 2 functionality is the same on both platforms.

On the Safari side WebKit has added CSS ic unit which supposedly delivers “real” vertical text layout on web pages… to which I say, I’ll believe it when I really, truly see beautiful looking vertical text layout web pages everywhere.

Another new WWDC…another new text layout api chart
If I had a dollar for every new CSS feature promise that delivers ‘real’ vertical text layout, I’d be very rich by now.

Japanese Typography and Font Posts

This is a collection of long form Japanese typography posts. They were written as stand alone pieces, so there is some background explanation overlap, always a weak point of the blog format.