Rethinking Face ID in the Face Mask Touchless Era (Updated)

When iPhone X came out in November 2017, IT journalist Tsutsumu Ishikawa named Suica the Apple Pay winner. What he really meant to say was that Suica Express Transit was the only easy way to use Face ID Apple Pay. It took me a long time to get used to Face ID Apple Pay but now with the COVDID-19 crisis and regulation face masks, the choices are back at square one: (1) yank down the face mask to Face ID anything, (2) use a passcode instead, (3) use Apple Pay Suica set with Express Transit. Yeah, the last one. More people have Express Transit now in China, TfL-land and little bits of the MTA OMNY system but nobody has it for purchases. Except Apple Pay Suica, still the only Express Transit card for contactless payments at stores.

In the sudden era of face masks and plastic curtained checkout areas, dealing with Face ID as little as possible, and using Apple Pay Suica as much as possible, makes life easier and safer: experts in Japan instruct people not to touch face mask surfaces and you don’t want to be yanking down a face mask to use Face ID Apple Pay at close proximity checkout. The interim solution is Apple Pay on Apple Watch which does not use Face/Touch ID at all. But there is that social distance problem: your arm has to reach the reader. That’s the thing about NFC, it’s close proximity technology. So are QR Codes.

The Touchless Distance
When I first saw the NTT Docomo Ultra Wideband Touchless Mobile FeliCa demo I though why would anybody want to pay a few feet away from the reader? Outside of paying while sitting in the drive thru I could not think of a reason. After living with Face ID, face masks and COVID-19 social distancing, I see the reason now at every checkout at every store. I want it. You will too (the 1:20 mark):

And for cars too, CarKey will work like this at some point (0:13 mark):

Touchless Transit Gate vs Facial Recognition
The COVID-19 crisis upends another Face ID related technology fantasy: facial recognition transit gates. NEC is working on face recognition that works with face masks. If anybody can deliver viable face recognition with face masks NEC will certainly be one of the first, but there are cost, performance and privacy issues to consider for transit gates: how fast is the transaction speed, how well does it scale for commuter rush, how do you register faces? Who controls all that transit gate face data and is it stored domestically or data farmed out internationally?

Mobile FeliCa and MIFARE Touchless is the same device level security model we have now with Apple Pay Suica and Student ID, and what we will have with CarKey and shared ‘keys’. UWB is a new hardware layer on top of what already exists, it bridges the NFC infrastructure and contactless payment methods we have now and extends it to the future instead of junking it.

Osaka Metro plans to have face recognition transit gates deployed in time for Osaka Expo 2025. It’s a risky transition plan. Touchless transit gates are the safer bet. Sony, Docomo, NXP, JR East, JREM are doing the necessary hardware and software development with the same embedded secure element security and local processing architecture we have now. Osaka Metro can buy the finished goods from them instead of reinventing the wheel.

Fixing Face ID Shortcomings
On the smartphone side Apple already has the Ultra Wideband U1 chip in iPhone 11. The next step is Apple Pay support as outlined in the iOS 14 Apple Pay post. I hope Apple uses the opportunity of adding UWB Touchless Apple Pay to enhance Face ID with improved technology and controls. Express Card/Express Transit is the Apple Pay method to bypass Face/Touch ID for transit, purchases (Suica) and ID door access (Student ID and CarKey). Extending the Express Card/Express Transit model as much as possible, while keeping the high level of security, is one practical way Apple Pay can address some of the Face ID in face mask era pain points.

Longer term, Face ID has to evolve to securely read faces with face masks reliably. If Face ID cannot be secure, intuitive and face mask user friendly, I don’t see a future for it, or being the iPhone model that customers want to buy. This is why iPhone SE is looking like Apple’s most important product launch of 2020.

Last but not least I don’t see Open Loop transit ever working with Touchless technology. Open Loop will likely remain a NFC only service because EMVCo partners are invested in lower common hardware standards like ISO14443 and plastic cards and probably loath to update them. Certainly they don’t want to lose the plastic card issue business because it’s more profitable than issuing digital ones. EMVCo certainly didn’t see the current situation coming, nor did Apple of course. But then again who did?


UPDATE

iOS 13.5 beta 3 has a Face ID tweak: when it detects a face mask it no longer delays the swipe up Passcode pop up with a 2nd read attempt, it goes straight to Passcode pop up. This small tweak remove a tiny bit of Face ID with face mask stress, but tiny things add up when unlocking iPhone many times a day. But for me Passcode pop up was only one stumbling block, a second bigger stumbling block is Passcode entry via the numeric keyboard.

There is a curious lag between what your fingers are tapping, the feedback click sound and what tap the iPhone actually registers. If you closely inspect the visual tap feedback, it flashes white then fades slowly, while the click just clicks.Taken all together, my brain wants to type fast and tells me the my 2 thumb input is going fast, but the iPhone Passcode numeric keyboard wants me to type slow with 1 thumb. Perhaps it’s just me but I only get correct passcode entry 50% of the time unless I slow way down and type with 1 thumb.

Overall the Face ID with face mask tweak seems more for iPhone unlock, it’s much less useful for Apple Pay. I hope Apple continues to tweak Face ID before iOS 13.5 ships but the reality is Apple can’t do very much in a short time.

John Gruber had an interesting observation regarding another iOS 13.5 beta 3 tweak, this one for Group FaceTime:

methinks a lot of folks at Apple (executives included) are using group FaceTime chats more than ever before lately, and have realized that in practice, especially in larger groups, it’s not a good experience.

Daring Fireball: ANOTHER IOS 13.5 BETA TWEAK: AN OPTION TO DISABLE ‘AUTOMATIC PROMINENCE’ FOR THE CURRENT SPEAKER’S TILE IN GROUP FACETIME

Unfortunately it’s the same for Face ID: Apple is only addressing it because Apple execs are wearing face masks. It’s very frustrating that Apple is only dealing with the Face ID with face mask issue now that it’s on their face. Customers in Asia have been wrestling with it since iPhone X day one November 2017. At any rate I hope Apple puts the experience to good use for a better future version of Face ID.

5G Contactless Payments Part 1: Fast QR vs Ultra Wide Band enhanced FeliCa and MIFARE

Payment empire players envision a brave new world of 5G enhanced contactless payment solutions, seen in recent moves by JR East and other major Japanese transit companies to replace expensive legacy mag strip ticketing with lower cost QR Code ticketing. 5G flavored QR Code and ‘Touchless’ Ultra Wide Band (UWB) Mobile FeliCa solutions were also on display at last months Docomo Open House 2020. How can it be that Docomo is developing Ultra Wide Band Mobile FeliCa and QR Code solutions?

The endless push pull of ‘this contactless payment works great for me’ that drives somebody else crazy is endless fascinating. We have more choices than ever: digital wallets, plastic cards, face recognition, NFC, QR Codes, etc. 5G and UWB promise to mix things up even more.

Ultra Wide Band enhanced FeliCa and MIFARE Apple CarKey?
The evolution of EMV, FeliCa, MIFARE and other similar protocols as they transition from plastic smartcards to digital wallets devices opens up opportunities to include other radio technologies like Ultra Wide Band and Bluetooth in addition to NFC. Ultra Wide Band Touchless FeliCa on display at the Docomo Open House was all about cars, not Touchless walkthrough transit gates that will appear in a few years.

Touchless FeliCa makes great sense as a ‘NFC car key’ that utilizes UWB for operation at greater distance and better accuracy when needed. Touchless makes even more sense as a ‘keep phone in pocket’ touchless payment method for drive thru purchases. The addition of UWB into the mix makes smartcard protocols much more useful than just NFC. I would certainly welcome a smartphone UWB powered Touchless FeliCa replacement that ditches the need for automobile ETC cards and readers on Japanese expressways.

How UWB enhanced FeliCa would fit with Apple’s new CarKey feature said to be coming with iOS 13.4 is unknown but iPhone already supports FeliCa. UWB touchless support for iPhone 11 and later models is a logical evolution. Sony and Docomo are developing the technology with NXP which certainly means that MIFARE will also support UWB enhancements. The long history of FeliCa and MIFARE as keycard solution providers is a natural fit with Apple CarKey. NFC is the only protocol that has been discovered in iOS 13.4 beta CarKey framework so far but I would not be surprised if UWB code references turn up at some point.

5G Cloud vs Local Processing
The Docomo Open House also showcased a QR Code transit gate with 200 millisecond (ms) transaction processing but the real star was the speed of 5G. 5G powered cloud processing promises to upend the current advantage of locally processed prepaid stored value cards…cards like Suica.

The basic promise of 5G is that IT system designers finally achieve a nirvana of everywhere, always available, big pipe central processing without wires, the big cloud. The original Suica card design effort back in the 1980’s had to leverage local processing because central processing wasn’t up to the task of handling massive transaction volumes of a Tokyo-Shinjuku-Ikebukuro station at peak rush hour. This is why Suica cards are stored value by design, the FeliCa technology behind the card design delivers 200 ms and faster transaction times for local processing at the transit gate. What happens when 5G promises, in theory, to deliver 200 ms central processing?

Kill mag strip paper tickets first then Suica?
As Junya Suzuki points out in his article ‘Is QR the future of Suica?‘, transit QR Codes on the complex Japanese transit network only need be a unique local passkey with everything else, verification, transaction, etc., done in the 5G cloud. The same concept applies for facial recognition systems where the registered face is the unique local passkey. With the power and speed of 5G, Suzuki san argues that the need for Suica-like local processing falls away. In his scenario all Suica needs to be is a unique passkey that can lose stored value functions.

I understand his point, Suzuki san comes from an IT system background, as a journalist he has covered JP transit payment system developments for a long time. For low traffic stations a Suica-lite 5G cloud based network makes sense and does away with the expensive hard wired transit gates. Just one year ago JR East said they are building a cloud networked Suica to cover all non-Suica areas.

However the old Tokyo-Shinjuku-Ikebukuro station peak rush hour central processing crunch problem remains. I’m not convinced super fast 5G enabled cloud processing is going to solve that problem any better or cheaper than Suica does now, and reliability is a complete unknown. We also have the next generation ‘Super Suica’ format and FeliCa OS coming in the next 12 months, the design goals here include a flexible, modular cloud friendly architecture and lower costs. Next generation Suica coupled with a flexible local processing~cloud processing backend may be a compelling solution that finally delivers a practical inexpensive Suica infrastructure to the little end of the line station which only gets a few trains or buses a day.

New JR East Suica / QR Code transit gate for Takanawa Gateway station

JR East, Hanshin and Osaka Metro are testing QR Codes and facial recognition ID ticketing to replace mag strip paper. As Junya Suzuki points out, mechanical paper ticket transit gates are more expensive to install and maintain than IC transit card gates but the real expense is mag strip paper recycling costs. Mundane but not surprising. The more important long term question is this: do transit companies keep the current more expensive cash base paper ticket fare vs less expensive IC card fare structures in place, or do away with it when QR Codes replace mag strip tickets? I don’t think we’ll see an answer to that question for a few years.

There is no doubt that 5G will enable new payment possibilities, and a lot of debate. But I don’t see 5G cloud completely upending and replacing the need for local processing and stored value cards. Both are evolving, both have their place. It doesn’t have to be, and should not be a one size fits all solution. Each approach has strengths that can be complementary and build a better stronger system.

For me it comes down to one simple thing. My Apple Watch can be buried under multiple sleeve layers but Apple Pay Suica works great going through rush hour transit gates every time. It’s the best argument for UWB enhanced FeliCa and MIFARE touchless transit gates and stored value local processing I can think of. QR can never match that, nor can face recognition…think face masks during an epidemic or pollen season.

In the next installment I hope to explore 5G and the evolution of digital wallets.

The Super Suica Touchless Connection

The recent flurry of press releases and news reports for touchless walkthrough transit gates and handsfree touchless store payments sheds considerably more light on the next generation Suica architecture and FeliCa OS. The new Suica card due in spring 2021 does not have an official name. I call it Super Suica. Here’s what has been announced so far.

Next Generation Suica “2 cards in 1” architecture, new FeliCa OS, new IC card format announced by Sony, JR East, JR East Mechatronics (JREM) in September 2018.

Handsfree touchless Mobile FeliCa payments technology based on UWB+Bluetooth on Mobile FeliCa announced by Docomo, Sony, NXP Semiconductors in December 2019. A new JR East touchless transit gate was also reported by Kyodo News around the same time and was confirmed by JR East. The new touchless payments technology uses FeliCa for transactions but uses a UWB+Bluetooth front-end instead of NFC.

No delivery date for touchless gates or touchless payments has been announced but as Junya Suzuki pointed out in his recent article, Japanese transit infrastructure investment runs in 7~8 year cycles. The Takanawa Gateway station opening and the Tokyo Olympics in 2020 are the kickoff for the next transit infrastructure cycle. I see 3 basic transitions for JR East and the other major transit companies.

  • Suica transition from legacy architecture to next generation ‘2 cards in 1’ Super Suica staring in spring 2021.
  • FeliCa transition from NFC only front-end to incorporate UWB+Bluetooth radio technologies for handsfree touchless payments. News reports suggest deployment of JR East touchless walkthrough gates starting in 2023.
  • QR Code transition from legacy magnetic strip and other paper ticketing. Testing and evaluation is due to start at Takanawa Gateway station in 2020 with new Suica+QR Code dual reader transit gates.

Next generation Suica and Touchless Mobile FeliCa represent an interesting twist in that both require a new version of FeliCa. My take is that the new versions of FeliCa OS are one and the same, and that both Super Suica and Touchless incorporate UWB and Bluetooth protocols for transactions in addition to NFC-F.

Zero-sum Game Reset?
People are already complaining ‘oh no, not more JR East/FeliCa proprietary BS,’ but that snap judgement is way too early. Outside of the basic technologies we don’t know what standards are involved for handsfree touchless payments, but we do know that NXP is partnering with Docomo and Sony on the effort. That means MIFARE is already working on it too. JR East announced at the 2016 Tokyo NFC Forum conference that they are dedicated to working for open compatible transit payments (i.e. open ticketing between transit operators, not EMV).

Let’s take JR East at their word and assume that there is just one flavor of UWB+Bluetooth touchless, that it is fast, that it is open. In this scenario the same UWB+Bluetooth touchless front-end could be used by anybody from the large established proprietary players like EMV, FeliCa and MIFARE to open transit payment associations like Calypso. I hope this is the scenario that plays out. We don’t need a repeat of the ‘let’s make NFC A-B (Philips and Motorola) an open standard and shut NFC-F (Sony) out of the game’ nonsense that didn’t help anybody except QR Code players.

The Apple angle is interesting. Global NFC support put Apple Pay ahead of the curve. Apple putting UWB into iPhone 11 this year could be another ‘get ahead of the curve’ move so that everything is ready to roll with Super Suica on iOS 15/watchOS 8 in late 2021. I doubt anybody will see it this way, but I think touchless Mobile FeliCa and JR East plans for it are one factor in Apple’s decision.

Handsfree Touchless Smartcards?
One very important question: does this stuff work on smartcards? So far only smartphones have been mentioned in the press releases. Indications are that Super Suica is launching with new IC smartcard issue, by necessity it will have be backwards compatible with current transit card IC infrastructure.

If JR East plans to deploy touchless gates systemwide starting in 2023, Super Suica plastic transit cards must work seamlessly with the new gates. It doesn’t make any sense to issue yet another card, Super Duper Suica, to work with handsfree touchless. It also doesn’t make sense if touchless is only for smartphones. If it’s going to work in the minds of transit users and be used at all, all of it has to work perfectly, out of the gate.

Touchless FeliCa with UWB

The technology behind the recent JR East Touchless transit gate got a little clearer today. Sony and Docomo announced a new ‘touchless’ Osaifu Keitai settlement process that combines Ultra Wide Band (UWB), Bluetooth and FeliCa technologies from Sony, NXP Semiconductors and Docomo and does away with NFC. The press release does not mention the new version of FeliCa that Sony is developing for Super Suica, but I expect the next generation FeliCa OS is what powers touchless transactions.

In addition to touchless ‘keep smartphone in bag or pocket’ transactions (what about authentication?), the press release also outlines digital car key and digital signage uses for the technology. It all sounds very similar to the earlier JR East Suica touchless walkthrough gate reported by Kyodo News. Does this mean that touchless gates require iPhone 11 and later?

Technology demonstrations including hands on will be held from December 10 to the end of January 2020 though the press release does not specify exactly where. A last minute venue announcement to keep competitors guessing perhaps? A public showing with hands on will also take place at the “Docomo Open House 2020” at Tokyo Big Site on January 23~24.