I really like YouTuber kenzy201’s latest post regarding the mysterious disappearance of the ¥170 paper ticket button at JR Kyushu Kokura station kiosks. Kenzy’s simple no frills talk style might look boring on the surface but his analysis is keen and enlightening, the kind of analysis we used to get with good journalists who don’t have the time for it anymore. I highly recommend listening to what he has to say here.
When JR Kyushu revealed the reasons behind their elimination of the ¥170 paper ticket option from JR Kokura station kiosks, it shined an uncomfortable light on their operations. Because of poor management decisions over the past decade that continued JR Kyushu’s reliance on paper tickets at the expense of leveraging SUGOCA, the result was 90% of ¥170 tickets sold at Kokura station were being used for fare evasion. Cheap passage to any paper ticket unmanned station on their system. This is because JR Kyushu has a continued reliance on legacy paper tickets instead of increasing SUGOCA transit card coverage and promoting its use, and because of cost cutting there are an increasing number of staff-less stations that are an invitation to paper ticket fare evasion.
As Kenzy points out, JR East Suica and Tokyu led PASMO revolutionized transit in the Tokyo region to the point where legacy paper tickets are hardly used, over 90% of fares are Suica • PASMO, with the old style transit operator interchange paper tickets already eliminated. There are station entrance/exit areas that only accept transit IC. JR Kyushu on the other hand, and despite having their own card and system, have not done much with SUGOCA. It’s a me too transit card without innovation or marketing muscle. The usage area is limited but more than that, JR Kyushu offers almost zero incentive to use it over paper tickets. No JRE POINT or WESTER-like reward point retail/service platform effort to encourage SUGOCA use and sell services, no mobile roadmap.
It’s a huge wasted opportunity that, unfortunately, has resulted in an embarrassing fare evasion problem that won’t be solved by eliminating a few paper ticket fare options. JR Kyushu management, and certainly SMBC group stera transit, may think that open loop will fix the problems, but it will not fix them. A bolt-on thin client like stera transit needs an existing hardware base to work. JR Kyushu has to fix and enhance the SUGOCA infrastructure they already have first.
The first indication that something was up the FeliCa chip supply chain came on May 31 as an small announcement from Iwatekenkotsu Co., Ltd that the scheduled last leg of their Suica 2 in 1 Iwate Green Pass bus support rollout would be delayed due to new IC reader device procurement delays.
This was quickly followed by a large joint press release from JR East and PASMO on June 2 announcing that unregistered Suica and PASMO cards, the plastic ones that people can buy in Tokyo area station kiosks, would not be available starting June 8, ‘until further notice’.
The official reason for the Suica plastic card sales suspension is ‘the global chip shortage,’ but that doesn’t sound right when there are gluts out there. These things can be complex so I asked an old colleague who specializes in chip production analysis about the situation. He had this to say:
Domestic manufacturers of non-volitive memory of the type used in FeliCa chips, reduced manufacturing capacity (the NAND market segment is currently in a recession due to overproduction and excess inventory). This reduction came when there was an increase of inbound visitors to Japan buying Suica and PASMO cards. There was also increased demand for new Suica purchases due to the expansion of the Suica area to the Tohoku region, because the population is not as large as Kanto and Kansai, demand is expected to settle down soon.
However as production capacity will not return to previous levels, there is a high possibility that supply and demand will continue to be tight to some extent.
Other Japanese sources say another factor is that all transit IC manufacturing has been sub-contracted out to Taiwan. In short, buckle up folks, it’s going to be a long bumpy ride as in addition to manufacturers cutting production due to the non-volitive memory glut, Japanese IC card production (FeliCa chip, antenna, card, etc.) has been farmed out to Taiwan. As the saying goes, they’re always a great risk putting all ones eggs in one basket, especially with chip production. But no matter how many times companies learn this hard lesson, companies soon forget.
The May 27 Tohoku Suica launch and Suica 2 in 1 Region Affiliate card launches are certainly a short term factor in the FeliCa chip shortage as JR East is still selling both registered and unregistered Suica cards in the Tohoku region and Suica 2 in 1 Region Affiliate cards. Outside of the Tohoku area JR East and PASMO will continue to sell plastic commuter passes and inbound speciality cards like Welcome Suica and PASMO PASSPORT, however inventory is tight and JR East reduced the number of Welcome Suica sales outlets on August 2.
People assume that Transit IC cards from other regions like ICOCA or SUGOCA are readily available but this is not the case as operators are quietly limiting sales outlets. SUGOCA for example is not available at station kiosks but kept ‘under the counter’: buyers have to go the nearest JR Train Reservation office and ask for it. Expect this to start happening for all regular non-commuter pass Transit IC cards as well, partly to control inventory, partly to keep scalpers from cleaning out transit cards from kiosk station machines for resale.
Why not use Mobile Suica and Mobile PASMO then? Unfortunately the mobile situation for inbound visitors isn’t great: visitors with only VISA cards or Android are basically out of luck.
So unless VISA lifts their foreign card Mobile Suica blockade, and it has been in place for a year now, even Apple Pay Suica • PASMO • ICOCA users are limited to using Mastercard and Amex cards…using cash recharge. More on that situation in later post.
The easiest solution for JR East and PASMO is to encourage domestic Suica and PASMO users to go with Mobile Suica and Mobile PASMO instead of plastic cards. They are already doing that but expect more Mobile Suica and Mobile PASMO promotion campaigns and reward point enticements. If anything, the plastic Suica • PASMO card shortage is the best Mobile Suica • PASMO promotion ever.
Mobile Suica has had a rough 2 weeks. On June 24 a construction error during server center power supply expansion work left JR East Mobile Suica and Eki-Net online reservation services offline for 12 hours (0:00~12:00). It was an embarrassing mishap but the actual damage was small, limited to refunding Eki-Net ticket holders who couldn’t change ticket reservations. Mobile Suica was offline so no refunding was necessary because nobody could use the Mobile Suica credit card recharge service. No need to refund what people can’t buy.
A shorter but much more problematic outage happened on June 27. Media mistakenly reported that Mobile Suica was down but this was not the case as Mobile Suica on Android was working just fine. It was an Apple Pay problem: Apple Pay servers went down from heavy demand on Apple Pay ICOCA launch day, taking down not only Apple Pay Suica recharge but also PASMO, ICOCA, nanaco, WAON, Octopus, China T-Union, adding credit cards and other Wallet services worldwide. As the outage took place during the Japanese business day, JR East had to refund iOS Suica App users who attempted to buy or use Suica Green Car tickets during the Apple Pay outage.
Just as things were settling down, another even shorter 40 minute period of trouble occurred on July 8 at 12:00~12:40 JST. Again the media reported that Mobile Suica was down, again they were mistaken, and again it wasn’t an JR East or Mobile Suica problem, it was a much wider, and unreported, EMV credit card payment network outage. EMV transactions on readers everywhere were not responding, and they were not working for Apple Pay or Google Pay. However FeliCa payment network cards were working.
And finally there was, yet another, Apple Pay and Wallet outage on July 14 from 17:45 to 18:30 JST, with another round of Japanese media bashing poor old Mobile Suica without checking for the wider Apple Pay outage.
Mobile Suica caught the media blame because they were the only company duly reporting the problems on Mobile Suica support SNS services. JR East never lays an outage blame on Apple Pay, or any other service partner because they know Mobile Suica users don’t care, they only want to know when things are not working and when they will be fixed. This is the way it should be done because they are giving their users fast, accurate, service information…even if that means they have to take the media and SNS blame that comes with it.
But despite all the Mobile Suica outages including the EMV payment network one, the Suica card itself always remained working, both digital or plastic versions. As long as there is money on the card it works for transit and payments, and cash recharge is available 24/7. This is an under appreciated but very important aspect of the Transit IC system: there is always a non-network fail safe cash backup. Japanese never put all their household finances in one basket, cash is always the one thing that works after an earthquake, typhoon, natural or manmade infrastructure damaging disaster strikes.
In the EMV credit card payment network outage there was, without doubt, unreported trouble with open loop system test deployments on Nankai, Fukuoka Metro and other QUADRAC • stera transit operated systems, which all open loop systems in Japan use: it’s the only open loop player in town.
Unlike Mobile Suica however, when the credit card payment processing network goes down, open loop doesn’t have a fail safe cash backup. And while that’s not a problem now with small installation test sites and a tiny user base, it will be when open loop goes big time. The transit companies deploying open loop have an obligation to take care of their customers, but will they take JR East-like responsibility when QUADRAC goes down, or stera goes down, or NTT Data CAFIS, payment processing centers, or mobile carrier networks? Because believe me they will. All highly connected, interdependent networks do. That’s why we always need alternative methods and networks. Too bad that VISA is working to remove the non-EMV transit gate competition in Japan.
The Suica cross region problem, no thru transit going from the Suica area to the TOICA area for example, is a well known and criticized shortcoming of the Transit IC system. There has been some recent progress with cross region thru transit commuter passes but barriers remain for regular Suica use, a headache for both local residents and longer distance travelers. Despite all the fancy technology, the cheapest cross region thru transit fare choice is paper tickets.
A lesser known Suica barrier remains on the JR East network: Suica service region gaps. Currently there are 3 Suica regions: Tokyo, Sendai and Niigata. There are also some curious gaps between them illustrated below:
Fortunately this is all about to change for the better.
Filling the Suica gaps In 2019 JR East CEO Yuji Fukusawa said the company planned to have 100% Suica deployment by March 2022 but that didn’t happen. Why? Transit use killing COVID, the resulting red ink and redeployed resources are a big reason of course, but system development snags certainly contributed to the missed deadline. There was also a shift from a narrow focus of a lower cost Suica system to a wider focus of Suica 2 in 1, Cloud Suica, a lot of new service parts built around a cloud based centralized fare processing system. JR East’s Suica vision is evolving to a wider, transit service platform encompassing a range of technologies, with FeliCa as one component of a larger whole and flexible new system.
In October 2022 JR Central announced that TOICA is expanding to all JR Central lines and stations. The pressure is now on JR East to complete their delayed Suica rollout to all stations first. But there is something else: it’s an open secret that JR East hosts the TOICA system. JR Central would not make such a big TOICA commitment publicly unless JR East had a new system in place to facilitate the expansion. This new system, which I call Suica 2.0, started operation on May 27 in the Tohoku region.
The launch brings Suica to 45 stations in the Akita, Aomori and Morioka regions but only 9 of these are fully automatic transit gate installations similar to what you find in Tokyo area stations (the same new QR equipped gates shown in the press announcement are installed in Yoyogi station). The rest, 36 in all, are Suica 2.0 validators. Performance is an obvious concern. Suica users are accustomed to the fastest transit gate fare processing speeds on the planet. Will Suica 2.0 performance satisfy the Suica 1.0 experienced customer base who expectation Suica to ‘just work’ like it always has? To understand how Suica 1.0 fare gates achieve speedy performance apart from FeliCa technology, we need to examine why Suica regions exist and how they relate to transit gate performance.
Suica regions and gate processing speed Suica stands for “Super Urban Intelligent CArd” (but there is also ‘IC’ in the name for integrated chip) and was designed for heavily used urban transit as a smart card recreation of visually inspected paper commuter passes. JNR (pre JR East) researchers wanted to eliminate the time it took urban commuters to pull their magnetic commuter pass out of a wallet or case and feed it into the ticket gate slot. This clogged up major station gates at rush hour. The researchers also wanted a centrally processed card system but the networks and processing power of that time could not deal with the rush hour traffic volume. So the Suica architecture was built around locally transit gate processed stored fare (SF) balance from the card. Instead of centrally processed payments, fares are processed at the station level and synced with the central server, said to be about 6 times a day.
Transit gates have very little memory, most of it dedicated to their main task of local processing Suica fare at the exit point. Low overhead is a necessity. They can’t hold massive fare tables, hot card lists, dead card lists and so on. Only the bare minimum information required to do the local processing job is periodically synced with the central server. Limiting fare processing to specific heavy use regions is a necessary strategy in keeping the local fare processing overhead low and speedy. This is why a Tokyo Suica/PASMO region transit exit gate only processes the fare from a Suica or PASMO (or any Transit IC card) that started the journey in the same region. It’s also the reason why Transit IC cards are generally limited to 200 km point to point trips in their respective local regions, though there are some interesting loopholes.
It’s the same situation writ large with when traveling across transit IC card regions. Border stations like Atami (Suica and TOICA) have 2 sets of exit gates: one for travelers from the Suica region, one for travelers from the TOICA region. Continuous cross region transit across Suica and TOICA regions is limited to special cross region commuter passes and special cross region stations, again to keep the local processing overhead low.
It’s important to note however that IC coverage extensions to border stations with 2 sets of different gates and cross region commuter passes, are very recent 2021 developments. This is the JR Group companies laying the foundation to remove IC transit barriers in the near future. Because Suica 2.0 can process any and all Transit IC fare configurations, transit gate memory limits for local processing are no longer a concern. The barriers will come down when gate hardware • firmware is updated and Suica 2.0 cloud servers are in place.
Conceptually, Suica 2.0 is simply going back to what the creators of Suica originally envisioned: centralized fare processing. Specifically the Suica fare processing hockey puck is moving from the station level to centralized cloud servers. The Suica card itself is exactly the same as it is now, transit gates still handle all mutual authentication read~write functions.
The original aims of Cloud Suica with lower costs and flexibility are still there, the JR East Suica 2.0 press release builds on those with emphasis on a distributed server processing system for both Suica service expansion (more stations and no region barriers) and service functions (all kinds of cloud linked services). Let’s examine the new kinds of services JR East is promising to deliver with Suica 2.0.
① Barrier Free Suica transit with no more region gaps. A main goal of Suica 2.0 and bigger than it might seem. Eki-Net Shinkansen eTickets are already ‘barrier free’ with Suica, through clever use of Shinkansen transit gates, but Ticketless Limited Express trains are stuck with Suica barriers such as the Tokyo to Sendai Hitachi and Tokiwa Limited Express trains. Suica users have long complained that service gaps forces them to travel with paper tickets, or they are forced to pay in cash at the exit gate because they tapped in with Suica in Tokyo and forgot the Suica barriers. This problem, and many more barrier Suica gap issues will be eliminated.
② Automated Fare Discounts Part 1: Commute Plan Lite. This is similar to the recently launched Off-Peak Commuter Passes, think of it as short term ‘commute plan lite’ with tons of options. You buy a discounted fare option for certain routes, use times or frequency and it’s automatically linked with your Suica. And unlike the current Suica App method, the items are added in the cloud, not written and stored on the card itself.
③ Automated Fare Discounts Part 2: Fare Discount Gift Coupons. In a similar vein, fare discount reward coupons for store purchases with Suica can be automatically gifted with a tap at the payment terminal. Kinda like the old free parking ticket with store purchase gimmick only far more useful.
④ Linked MaaS services. JR East has been experimenting with MaaS programs like RingoPass but linking MaaS services with Suica 1.0 is a real pain. Suica 2.0 should make bundling much easier, it’s also an opportunity to clean up the current mess of apps.
Reality check and missing pieces Glossy JR East press releases are one thing but reading between the lines of the Suica Service Roadmap there are hints of missing pieces. Suica 2.0 is all about eliminating physical transit barriers but in the mobile app era there are lots of software barriers that need to be addressed too. Right now JR East online services are hosted in a bunch of apps that don’t fit together very well. It’s a maze of walled gardens: lots of service apps each with different accounts and login, making them work together is a real pain. The real problem is there is no one app to see and manage all the services and tickets attached or linked to your Suica.
A few things need to happen to make Suica 2.0 truly useful.
The current version of Suica App lives a double life: one half pulls things off the Mobile Suica cloud, one half does local housekeeping attaching Commute Plans, Green Car Seat Tickets and recharges to Suica card. Meanwhile Shinkansen eTickets, MaaS and other online services live in different apps with different accounts IDs. Wouldn’t it be nice to have all these services living in one cloud savvy Suica App that shows and manages everything attached to your Suica? Absolutely yes please.
Local Processing Fail-Safe?
We all know that cloud and mobile services fail. Stuff happens. Safe railroad operation requires fail-safe design. Japanese IT journalists like to pooh-pooh FeliCa and Suica reliability, heaping praise on how ‘fail-safe’ the Transit for London open loop Oyster system is. But London transit doesn’t have to deal with major earthquakes, tsunami, typhoons, torrential rain and flooding, train communication cable arsonists, communication cable damaging trackside fire disasters, not to mention sarin gas and cable cutting terrorists. Japanese tend to take safety and security for granted but these infrastructure risks are very real. They have all happened. Suica 2.0 will be a highly centralized system, the higher the centralization, the higher the associated risks when it fails.
Does Suica 2.0 have a fail-safe backup? Here’s a possible, and from emerging details, likely scenario. We all know programmers don’t like using a new API for mission-critical programs unless they have to. They like to stick with what they already have for compatibility with a smooth gradual transition strategy to the new API. Same for Suica 2.0. Automatic Suica transit gates could be upgraded with both the Suica 1.0 ‘Suica Region local processing API’ and the new Suica 2.0 ‘Region-Free central processing API’. If something goes wrong with the Suica 2.0 central servers, the exit gates switch to reliable local processing Suica 1.0 API mode to keep passengers moving with station level fare processing or perhaps regional level fare processing depending on the JR East distributed server setup. Long story short, If this backup is not in place we can expect this to happen.
Suica 2.0 rollout and the QR Eki-Net Connection By all accounts the Suica 2.0 May 27 launch seems to be a happy marriage of ‘truth in the card’ Suica Stored Fare balance + central fare processing without any loss of fast transit gate speed. It works just like the Suica we’ve always had. Region barriers remain however, and they have increased: the 3 new Tohoku additions (Aomori•Morioka•Akita) each represent a new Suica region. Transit IC card barriers will eventually go away when Suica 2.0 is deployed across the entire JR East system. People can travel anywhere on the transit IC network not having to think about barrier nonsense, just like paper tickets. Sounds great but when does it happen? Let’s take a look at the JR East rollout schedule.
As of May 27, 2023 we have six Suica regions, however Suica extensions announced for 2024 and 2025 give us a clue how the barriers will drop. The 2024 Yamagata Suica extension will be added to the Sendai region, the 2025 Nagano Suica extension will be added to the Tokyo region. Sequentially we can expect Sendai to be folded into the Tokyo region first, then Niigata and finally Tohoku as missing station gaps between regions are filled.
JR East says the Suica region barriers will drop by 2026, at the latest, when Suica 2.0 is rolled out across the entire rail network.
An interesting point here is that QR Eki-Net service starts in the very same Suica 2.0 Tohoku launch region which means that QR Eki-Net uses the same Suica 2.0 fare validation system. Suica 2.0 does QR too. When Suica 2.0 goes wide, so does QR. It’s one package with 2 parts as shown in the Suica Service Roadmap: the Suica 2.0 Platform and a ‘new’ (and unnamed) Ticketing system, which might be the venerable (and earthquake hardened) JR Group MARS system updated for the mobile transit era.
And when does seamless cross region IC transit for Suica, TOICA, et al. happen? Hopefully the JR Group is coordinating so that the Suica 2.0 rollout is mirrored by the other JR Group companies. The JR Central TOICA announcement certainly suggests so. Slight differences are already apparent: JR East prefers cloud connected Suica 2.0 validators at unmanned stations. JR Central and JR West prefer the bus style approach of having on board enter and exit validators for rural lines with unmanned stations. Either way is fine, just get it done as quickly as possible. Let the Transit IC barriers drop away into the past where they belong. Because with Suica 2.0 in place and barriers gone, the way is also cleared for fare capping, automated discounts, specialty ticketing and lots of new cloud based transit services.
This post was originally published 2023-02-27 and was reposted with the latest information from JR East on 2023-06-21.
Android 10 devices: the Osaifu Keitai supported devices list is limited to modern Android 10 devices that support multiple transit cards (Suica + PASMO + ICOCA) with only one card set as the default ‘Express Transit’ card.
WESTERID required: this new point system + single ID for multiple JR West online services was put in place just for Mobile ICOCA and a registered account is required for Mobile ICOCA. Similar to JRE POINT, users earn points from transit use, recharge and purchases at participating stores. WESTER POINT to Mobile ICOCA recharge is built in and self contained, more streamlined than JRE POINT app to Suica app recharge. J-WEST credit cards earn the most WESTER POINT for recharge and commute plan purchase.
Users can register 3-D Secure 2.0 credit cards for recharge in the Mobile ICOCA app but ‘some foreign issue cards’ are not supported (looking at you foreign issue VISA)
No wearables (Garmin, Fitbit, etc.) yet, Apple Watch will be first hopefully followed by WearOS devices later
No auto-charge support…a real pain point, stick with plastic if you want this feature.
Mobile Commute Plans: most regular and university student commute plans are supported including JR West + Osaka Metro/Hankyu/Keihan/Nankai/Hanshin routes, but other commute plan types/routes are not. These include, FREX Shinkansen plans, HS/JHS student plans (launching on Mobile Suica and PASMO March 18), disability discount commute plans, bus and trolley routes, and routes that include the JR West Tsuruga~Ishikawa Hokuriku line that will transfer from JR West to third sector ownership in 2024. Stand alone ICOCA commuter passes for Hankyu, Keihan, Osaka Metro routes that do not include JR West lines are not supported.
Suica and PASMO together account for roughly 80% of all Japanese transit card issue, ICOCA added in makes that 90%. In other words, 90% of the issued transit card market has mobile service. It will be very interesting to see how the Mobile ICOCA migration works out. I’ll update this post with details as they become available.