NTT Flet’s fails in the Covid traffic crunch

NTT FLET’S internet service has been around forever in many configurations, the latest being Flet’s Hikari ‘optical fiber’. I call it flexible fiber because NTT uses the term Hikari when they should not. My Hikari only comes into the apartment building junction box then branches into each apartment with good old cooper wire phone lines and a VDSL modem. NTT calls that Hikari, I don’t.

PPPoE/IPv4 traffic has been tapped out in Tokyo since at least 2017. When I first upgraded from PPPoE/IPv4 to IPoE/IPv6, I saw a pleasant bump in speed with none of the night time internet traffic meltdowns when using PPPoE.

I thought my problems were solved but over time IPoE/IPv6 download speed has slowed down while iPhone NTT Docomo 4G LTE speed has skyrocketed past NTT Flet’s:

A year ago Twitter user shao, who posts wonderful network and payment tech tweets with the deep tech background to back them up, noted that the Japanese Internet Provider Association was in a collective hissy fit with NTT. IPoE/IPv6 junction points to NTT main lines where tapping out and providers needed more junction points, they also wanted IPoE access pricing brought in line with PPPoE and better traffic control. NTT gave internet providers the cold shoulder with ‘we’ll consider it if you do the work.’ The result of that is NTT East/West Flet’s service is seriously slowing down in face of stay home telework, bored kids streaming content and too much online shopping.

As shao notes 4G and KDDI au Hikari nuro service are, so far, unaffected. The strange thing here is that KDDI is simply renting NTT dark fiber for nuro. So yes, NTT has the capacity, but doesn’t seem inclined to put in the effort to share it unless providers do the work, and also pay up. To be fair I think one of the problems is hinted at in a recent annual NTT financial report: a shortage of field engineers and technicians. Somehow it seems fitting that the human problem of Covid is also the human problem of slow internet speeds.

Japanese Text Layout for the Future* (hint: there isn’t one)

I finally had time to catch Adobe Nat McCully’s ATypl Tokyo 2019 presentation. He covers the topic that I have covered in depth many times before: the (sad) state of CJK typography. As Nat points out most software developers and system engineers talk about CJK support as typography without any idea of what it means. Throwing CJK glyphs on a screen is not typography, they are not the same thing at all.

The defining feature of CJK typography and layout in general and Japanese typography in particular is that space is an essential composition element equal with text and graphics, with fine space element control way beyond a baseline. Instead of thinking about how much space should be between text, flip it around and think about how much text should be between the space. Baseline font metrics will never deliver great CJK typography because there are too many limitations. So everybody implements the missing stuff on the fly and everybody does it different. Unfortunately the irony of it all is that Adobe played a huge role in how these limitations played out in the evolution of digital fonts, desktop publishing (DTP) and the situation we have today.

QuickDraw GX was probably the only time in computer history that fonts, layout engine and the basic OS came together to solve these limitations for all language systems, all language typography as equal from the bottom up. Parts of that effort survived, such as Apple’s San Francisco variable system font based on the TrueType GX model, and the inclusion of the TrueType GX model as the base technology for OpenType Variable fonts. Nice as this is, it’s only a tiny sliver of the GX vision pie that survived, all the other baseline font metric and CJK typography limitations still exist. Outside of a handful of people like Nat at Adobe, and the Adobe CJK typography ghetto approach of keeping all the good stuff corralled in InDesign J, very little is being done to address them.

Call me a pessimist but after 20 years of watching things slide sideways, I don’t see much hope for the future evolution of great CJK typography on digital devices. Most western software development people think that having CKJ glyphs on a screen is ‘good enough’ CJK typography, end of story.

Already I see the OpenType Variable Font effort devolving into a bauble for web developer geeks, always stuck in demo-hell, never going mainstream. It is the same story for quality CJK typography on digital devices. When the current Adobe CJK leaders like McCully and Ken Lunde reach retirement age, whom have devoted their careers to fixing these problems, I think it will be the end of an era. In many ways we are already there.

Apple prides itself on having good typography but cannot be bothered with such Japanese typography basics as not mixing Gothic and Ryumin Japanese font styles seen here in the Photos app

UPDATE
Ken Lunde posted a wonderful overview of his Adobe career to date, also his ATypl Tokyo 2019 presentation.

NTT Docomo rolls out 4G LTE Gigabit service

In case you missed it, try this if you are a Docomo iPhone customer: open the Docomo Speed Test app and tap the Area Map button. The previous red area has been replaced by yellow. The app needs to be updated but the red now indicates the areas with 1288Mbps~988Mbps Gigabit-class ‘Premium 4G‘ service, just in time for the iPhone 11 and iPhone 11 Pro release.

I am fortunate to live in a red 4G Gigabit speed area and my iPhone XS 4G speed is faster than my NTT East FLET’S HIKARI ‘mansion type’ VDSL service. It’s an older apartment building where telephone lines and VDSL are the only way to connect to the internet. That’s depressing to think about, but it will have to do until I can move to a place with direct fiber connection service. At least my iPhone XS 4G LTE is fast and will get faster if I upgrade to iPhone 11 Pro.

KDDI au is offering similar 4G LTE Gigabit-class carrier aggregation service for iPhone 11 customers. Be sure to check details and coverage with your carrier.

Bug Bounties, Public Betas and Risk Management

I love Paul Jorgensen’s blog and his unique take on cyber security issues. It is his chosen profession and he was one of the very few to notice and take interest in the August 2017 Google BGP leak that brought down Apple Pay Suica services and major parts of the Japanese internet. He was also one of the few to blog about China Telecom spoofing the BGP protocol to poison internet routes to suck up massive amounts of American and Canadian internet traffic for intelligence analysis.

In his post today Paul quotes Katie Moussouris on bug bounties and risk management. Specifically, relying on public bug bounty programs that just create the “appearance of diligence”:

“This is not appropriate risk management. This is not getting better when it comes to security vulnerability management..

A lot of the patterns [have] not actually shifted that much from where we were when I started out professionally 20 years ago as a penetration tester…

We’ve created a $170 billion industry, which, we’re really good at a few things, security not exactly being one of them. Marketing, definitely.”

As Paul points out, “bug bounties are a tool, but only one tool. And it’s a game, so people will look to take advantage.”

To draw a close analogy I would also say that the public beta approach that Apple now uses for iOS and macOS development is similar in that it just conjures the appearance of diligence, not diligence itself. It creates an atmosphere of reduced expectations, both on the engineering side and the user side: “it’s just a beta, we can still work out the bugs.” I wonder if we would be better off without a public beta, a better developer beta program with robust bug reporting tools might set a higher bar.

As others such as John Gruber have noted, iOS 13 has been one of the buggiest beta development cycles in recent memory. Perhaps I am being nostalgic, but I think when Steve Jobs still walked the halls in Cupertino, his drive to deliver an excellent shipping product, and fear of his wrath when things didn’t measure up, was due diligence that instilled the Apple development culture of that time.

People perceive quality even if they cannot put it into words, the old look and feel thing. As Moussouris points out, marketing is a poor substitute for diligence and quality. The risk of the current environment is that Apple ships software products that have lower expectations which no amount of marketing can make up for.

Choosing the right WiFi router for Japan

UPDATE: Japanese internet service providers are under a lot of speed stress from many people working at home during the Covid crisis. Make sure you are using IPoE IPv6 service outlined below. A good stress free service is KDDI au Hikari nuro which uses excess bandwidth that KDDI rents directly from NTT dark fiber for nuro.


My father had WiFi problems in his apartment, too many dead spots for a decent FaceTime conversation unless he stayed tethered near his Comcast Xfinity WiFi box. Like most people my father likes to walk around and talk resulting in broken connections and conversations.

I picked up a Linksys Velop mesh WiFi router set for him while in the USA, set bridge mode on his Xfinity box and plugged in the Velop router. It could not have worked out better. All the WiFi dead spots were gone, my father can FaceTime wherever he wanders. Velop truly ‘just works’ out of the box.

Linksys has been absent from Japan for some time but seems to be using Velop to dip a toe back into the Japanese market. Velop is a good product but I do not recommend it for WiFi use in Japan: it’s a poor match for the IPv6 protocols used by Japanese internet providers and the NTT backbone.

Goodbye PPPoE (IPv4) Hello IPoE (IPv6)
The problem with Velop is the same one with the Apple AirPort Extreme (part 1, part 2) and Google Nest WiFi: no support for DS-Lite and Map-E IPv6 protocols. Both DS-Lite and Map-E use IPv6 IPoE (IP over Ethernet) that replaces the older IPv4 only PPPoE connection protocol. IPoE is all called IPv4 over IPv6. This means IPv4 packets are encapsulated inside IPv6. All internet connections in Japan actually use IPv6. PPPoE/IPv4 is like an old studio backlot, a false front with nothing but IPoE/IPv6 behind it.

Any router that does not support IPoE/IPv6 on the internet in Japan does not get priority routing at crucial exchange points between local area lines, the internet provider, and the NTT backbone. PPPoE is worthless because PPPoE/IPv4 in Japan is ‘tapped out’ and sits in a traffic jam on the local internet highway ramp while IPoE/IPv6 whizzes by on the IPv6 super highway.

Japan Internet Setup
Do yourself a favor and do not waste time and money with any WiFi router that does not support DS-Lite/Map-E protocols and IPoE/IPv6 service. Completely eliminate PPPoE on your home network if you want fast internet service in Japan.

All of the Japanese internet providers offer free ‘v6 Plus’ or ‘IPoE’ service options for connecting your home internet directly with IPv6. I highly recommend adding a free IPv6 option and either renting the WiFi router from the internet provider, or purchasing one. Make sure your WiFi router supports the IPv6 DS-Lite and Map-E protocols. The major Japanese WiFi home router manufacturers all support those protocols and maintain IPv6/IPoE lists of internet providers and services qualified with their WiFi equipment:

Always make sure your WiFi router is updated with the latest firmware.

If you are not a DIY networking guru, you can save time by renting a pre-configured WiFi router from your Japanese internet service provider. Rental prices vary, So-Net for example charges ¥400 a month. If you are in Japan for the long-term and futzing with internet configurations is not a problem, a good WiFi router investment from the list above can save you money.