The promise of pen computing

Something exciting happened in the personal computing domain of the early 90's: the GO corporation launched their PenPoint OS. The computer industry had just adopted the graphical user interface - inspired by Xerox, brought to market by Apple and copied by Microsoft - this was the way how we would use computers from now on. Fueled by this magic, visionaries (among them Robert Carr) already saw the next logical evolution in the user input interfaces: the pen!

The promise of using a pen to interact with the computer was tempting to the point where Microsoft saw its nascent monopoly challenged and decided to copy once more an idea, so it went after the PenPoint OS with a special version of Windows 3.1 for pens. Similarly, Apple - not wanting to miss the boat - under the leadership of ex-Pepsi Cola CEO John Sculley, saw it's pen future in a device called the Newton. The pen computing idea was simple: why not use the handwriting that we have all known since we've entered primary school and use it as a way to interface with the computer. The metaphor is pen & paper - but with a smart computer screen.

Implementing this vision proved to be much more of a challenge than anyone had anticipated. There were limitations on processing performance and handwriting recognition dependent very much on how precise the handwriting was. But also the general notion of how to combine the now ubiquitous mouse input with pen was not really addressed. In short, none of these commercial attempts succeeded - the only successful device in that space was the Palm PDA which didn't even use handwriting recognition. And thus, every pen device disappeared from the market (Newton was killed by Steve Jobs himself - he hated Pens). However Microsoft kept at it - a bit in secret and a bit hidden (Bill Gates is a big fan), and while Apple killed the Newton, the handwriting recognition technology made it in fact into MacOs and (later) iOS.

Enter artificial intelligence and the 2010s and all the usual suspects are back at the table with their pen computing devices. Microsoft built it into their Windows 10 operating system and Apple uses it in its line of iPad. So how does it perform today?

Sadly, not great. The promise of pen computing from the early 90’s to the late 2010’s (a time span of 25 years), has not in fact changed the game in any meaningful way and AI has not been the white-knight of saving the “genre”. We could say that pen input has enjoyed some modest success among the creative type - designers, artists etc. but has not had any meaningful impact for the rest of us. I know, because I’ve been trying for 25 years to make it work. Today’s latest Microsoft Surface with pen, feels not that different from my Newton that I had 20 years ago. The challenge is still the same: my handwriting is not properly recognized. Just like autonomous driving - this is a zero sum game. Autonomous driving has to work under any circumstance because if it doesn’t people die. It’s not as dramatic with handwriting recognition but it's the same principle: if it doesn’t work 100% of the time, it is useless because the time it takes to fix your errors takes away from that promise of increased productivity and in the end you’re still better off with a keyboard (quiz question: how long would it have taken me to write these short paragraphs with a pen-input device? I suspect around 4 times longer). But the far bigger crime is the fact that nobody has ever thought beyond the graphical user interface that has relied on mouse- and keyboard input for the last 30 years because it doesn’t really work for pen input devices. My perfect example is Excel: try to enter a formula or a number in a cell with a pen. It’s impossible because the applications have not been conceptualized for pen input. Microsoft only has a single application that tries to show the promise of pen computing and that is OneNote - except that it doesn’t help. OneNote is such a monstrosity of application where you can enter text and graphics in a multitude of ways - inconsistent as hell.

At the end of this decade, I have to sadly admit that the promise of pen computing is still very much elusive. Don’t get me wrong, it works to capture notes and to draw or design something on screen - in short for niche applications. There is no keyboard replacement on the horizon for sure and it may in fact never arrive unless we completely rethink how we would interact with a computer if we didn’t have a mouse or a keyboard. But given all the investments into existing apps and operating systems, this may never be viable. I fear we will leapfrog straight into controlling computers with our minds. That’s a pity because the pen would be such a phenomenal tool.