Out of the way

the problem, idea one, idea two

The Problem

Once upon a time, Web applications were pretty simple. You had forms, and they sent information. Or you had javascript, and it didn't do anything very important.

Then came interaction events, where you could make things happen when the mouse hovered something, or a user clicked on it. And the DOM, which made it much easier to do interesting things when the user did their mouse magic. And later still came SVG, and XMLHttpRequest, and muli-touch screens and various other cooler and cooler technologies for making the Web into a really interesting platform.

But way back then, when we had a handful of mouse events, people realised there could be a problem. We got a few keyboard events like onkeypress and onkeydown. And this would solve the problems...

Of course, it was a bit trickier. Because while people knew something about common mouse interactions (there were differences - web patterns typically used hover and click, where others used click or double-click) keyboard interactions were pretty much a mess. There have been well-established but different ways of moving around since before there were graphic interfaces of any quality. But they relied on wildly different keyboard mappings. (One reason for this is that mice were pretty much all the same, but keyboards come in many many varieties).

So keyboard interaction was (and today still is) based on waiting for a particular key to be pressed. And hoping that the user has that key. And trying to work out whether it is the key that has a particular label on it, or a particular key in a (guessed-at) layout of the keyboard. And assuming that doing this won't interfere with the user interface, so that trying to open a file or quit an application does something unexpected instead.

In short, it's a mess. It doesn't work well, it didn't work well, and as devices get more and more diverse (what are the keys on a touch-screen, or a Wii remote control, or a 20-key qwerty keypad?), it seems likely to work less and less well. And yet people continue to provide the advice that making things keyboard accessible is a matter of writing increasingly complex javascript and pretending things work reasonably well.

So what's the alternative?

And there is the rub. I don't know. I don't think anyone else does, either. Here are a two ideas that might work, although they need a lot more fleshing out. But having thought that for a few years, I am hoping others might be able to help with the fleshing process.

1: Intent-based events

A decade ago I was involved in a long series of discussions which tried to replace the action-specific event triggers (hovering, clicking, pressing a key) with a model based on abstract interactions such as selecting, moving, activating, and so on. The idea was simple enough - if you activate something, the author doesn't care if you did it with a mouse or by a spoken command, they just know it was activated and react. Similarly you could select something ready to activate, whether by hovering over it with a mouse cursor, or moving to it with a 4-way joystick (then rare, now very very common on phones in particular). Some of these things got taken up - but as separate events, that were not implemented in browsers and therefore not useful in practice.

A development of this idea was "namespaced events" - essentially the ability to define new events. The trick here was to work out what would trigger them (and the simplest thinking was to fall straight back into trapping low-level actions as if the author could somehow predict what set of actions were available to the user). Opera even implemented this proposal - and it wasn't all that hard, at least to do what we did.

At the same time, it became clear that the click event (the most common event used) had to respond to other activation. It's still not clear exactly how this works (and what is the interaction of a complex set of things when you use a key to trigger the click), but at least it is possible to do simple things and no longer require a physical mouse (or something horrible like mousekeys) to make them work. So we have, today, something like an activation event (called click, and fairly messy around the edges). But we still have things like onmousemove and the CSS :hover property that are pretty-much mouse-only, and no apparent push to fix them.

The question now is can they be fixed? It would take implementation in probably half a dozen browsers to push any change, and then it requires teaching authors to change - and there are zillions of those. With agreement, goodwill, and an understanding that we are currently dividing applications into different platforms (except the enormous ones that try to cover everything), I think it is possible. I believe we could change the Web in under a decade. But it will take a lot of work to get there.

"accesskey" mode?

(I use that name because I don't have a good one. It relies on reinterpreting the accesskey attribute, more like HTML5 now does it than like HTML4 did it, because although the markup required by HTML4 was OK, the specification was stupid - and yet people followed it and made stupid implementations, which gave the whole thing including the useful bits a bad reputation it hasn't yet recovered).

The idea starts by fixing accesskey itself. Instead of being primarily a request to listen for a particular key (in this respect SVG's version is even worse than HTML 4's), it is primarily a statement that some control is important enough that you should be able to directly activate it (rather than moving through the page to it). There are lots of ways to do this - you can use the letter to associate a key with it, as iCab used to do. You can use the letter, or a word, to associate a voice command. You can associate it to a mouse gesture, or any other input mechanism available to the user. This should be determined by the user agent, although information supplied by the author is helpful (memorable letters/words, a role or rel attribute, etc).

For simple page interaction, or common applications, this is pretty much enough. And we implemented something that handles it (more or less) in Opera, a few years ago. Entering accesskey mode pops up a menu of the accesskeys defined - and you can either select the relevant letter, or navigate the menu with the mouse.

(You should be able to navigate the menu with the keyboard, it should handle the case where the attribute is more than a single letter, it should do a better job of explaining what happens, etc, but those things are in development still. It also makes little sense that shift-esc is the way to make this happen - I reassign that to the "." key, but there are other alternatives. Finally, we need to make this work on Opera Mobile and Opera Mini too. Which is all a matter of getting those bits of work done).

But this still leaves out one important use case. A pop-up menu is fine when you're interacting with a complex spreadsheet application, or an internal business system. But it really isn't the way to go for games, air traffic control systems, and other cases where you want continuous interaction. In that case, something that tells the system to "keep the accesskey mode on", that is to listen for the accelerated controls until told not to is probably required. The WAI-ARIA role application is sort-of used to do that (I think the current explanation of it is too simplistic, but it points to something like what I describe), and it doesn't seem impossible (or even especially difficult).

The "until told not to" is important. The problem with plugins that provide their own user interface is their inability to return the user to what they were doing before they dealt with the plugin. But again, this is a problem that can be (and occasionally has been) solved.

So where now? I don't know. Thoughts welcome.