So yesterday's Apple event was quite interesting. I read about it afterward and although the Air is really cool, most of my attention went to OSX 10.7 or Lion. Here is my take on the new features.

Basically all the new features are tied together in one theme: how do we make a computer better suited to use with touch? I really, really like the fact that they only want to do horizontal touch, which means they are focusing on every-day use and not some gimmicky interactions. They really want useful stuff you will use everyday because it is just so natural and easy to use.

Apple begins to transition away from applications into tasks with fullscreen applications. They have a ton of experience now from iOS devices and how people use them, so why not apply that. In the pictures it is unclear however how that works with notification and other system stuff. The current time and battery status are still useful, even in fullscreen mode. And does the dock automatically go into hide or does it just disappear?

But fullscreen lets you really focus on your task and talking about tasks... Since we are still talking about a single display with a keyboard, task switching is based around indirect manipulation and I think Apple has some work to do here. Sure, multitouch made exposé so much more useful (2-finger swipes!), but the interface still is very dynamic and requires me to reorientate myself every time I open it. I am still waiting for the killer switcher. I guess the touch area needs to further expand for that, we'll see.

I definitely am waiting to see how they combine dashboard and spaces together in Mission Control because spaces+exposé = real tasks. If I can group my windows into tasks and switch between them easily, they will make a lot of friends, including myself. I noticed that during our task bar presentation at UIST this year; people really want to organise their windows into the tasks they are doing. That is an operation across applications, while Apple until now grouped windows per application. I need a browser window in my thesis-work and in my coding, for me those two windows are completely different from each other. It would also be awesome if I can close that entire group and restore it again. With the expertise with fast-switching apps on iOS, that should be right up Apple's sleeve.

It is definitely the way to go in my opinion. At first, the task bar was just a window bar; then it became an application bar with the dock of OSX or a hybrid with the window-grouping of XP (which turned into the application bar of W7). The next step is going to be a real task bar, where the user can group their windows as they see fit and switch between them.

We are quite addicted to keyboards these days. They are a really good mechanical way to input text into a system. Same goes for the mouse; they give feedback, they almost never fail you.

There is a catch which the world seems to forget: mice and keyboards still have quite noticeable learning curves. A curve almost everybody has already taken. Seniors are the only ones who consciously have to choose to learn to use a computer and we are closing in on the mark the first computer user generation is becoming senior. It seems like a ridiculous idea to throw the collectively learned interaction with our computers away.

I think I agree with that one, but I want to make a slightly stylised comparison to alpine sports at this moment. Until late in the 20th century, skiing was the way to go if you wanted to go down hill. It was fast, fun and once you got used to it, you did not want to do it any different. When snowboarding came along, some people frowned, others tried it. It took a different approach to operate it, but a lot of them had more fun. Some went back to skis and complained about the lack of control and other stuff. Hills do not really discriminate and both skis and boards are seen today.

In this case we have one way to change the way we operate our computers, but to make it work, we have to use the materials we give ourselves; it is not a good idea to treat it like it should be like a keyboard and mouse. So what does material excel in and what can we use it for?

Keyboards work in a very simple way; you press a button and that button or key resembles a character or modifier. In the traditional sense keys have edges and we use our motor memory to locate a key, while using the physical feedback from the key to determine where we pressed it and get a better mental model of how the keyboard is oriented. This way, we make better judgements about where keys are located; a nice feedback loop.

When we talk about a multitouch panel, we lose quite a bit of feedback. We do not have keys, we do not press down anything. We are left with a slate and the pressure feedback from our fingers. These surfaces are however able to draw an interface anywhere on top of it, so let us find a way to use that and I believe we are already seeing the first steps into really using multitouch surfaces for text input.

Swype actually uses the fact that you do not have physical keys. The basic idea is that you swipe across a virtual keyboard and it uses the changes in directionality to type and uses a word list to make corrections or suggestions. It is however based on direct touch (for corrections) and still uses a keyboard. Because Swype is used in direct touch, the keyboard is present where you look, so you can actually see which "keys" you are hitting. So the swiping is a nice idea, but to make this indirect, we need more.

So... motor memory, pressure sensing, low feedback, flexible interface surface... We need something that does not require positioning because that is the bane of screen surfaces. You cannot position your fingers anywhere with precision, so we have to take the text input to the fingers.

Microsoft actually patented an idea of mine to use wrist detection. They relate it to positioning a virtual keyboard, which is a good idea: rely on that motor memory! If the keys are in the same exact spot every time, that might work. It has some minor problems with occlusion and not being able to lift your hand to check what is under it (the system would lose your wrist to position it).

Another idea is to come back to the swiping, but inverse the relation between key and finger; if we can track the hand when it is slightly above the surface, we can use real simple gestures by every finger to input text. Just on the top of my head are a lot of practical issues with comfort and high error rate, but one hand could probably power enough to avoid the sandwich problem*. It just screams to be tested.

By the way, I am not saying we should ditch the keyboard, but I like snowboarding, maybe I like this as well ;-)

Next time: "Surface real estate"

*The sandwich problem is the problem of people preferring to eat a sandwich with one hand and operate their computer with the other. I really think designers forget this way too often...

I am slowly getting a better picture of what I want to do with my master thesis or, actually, what direction I want to take my expertise. When talking with people at the excellent UIST conference last week I really got the feeling that every-day multitouch is not really in focus of researchers. A lot of very cool "session work" on tabletops and wall screens, but not much work on how our personal computer can benefit from it. As it goes with researching somewhere some researcher wants to shoot me right now.

But I got a very interesting question at UIST: where can I find your blog, it sounds really interesting. Oops, forgot that. So from now on, for sake of fun and prior-work claims, more about what I am working on.

The idea is to incorporate the benefits and overcome the hurdles of using multitouch. For the laymen: multitouch means that you can use more than one cursor as input on your computer. In practice this means using your fingers. But be aware of marketing; a lot of "multitouch" all-in-one's you can buy nowadays are actually dualtouch, they can only detect 2 contact points at the same time.

Multitouch has a lot of hurdles to overcome, mainly because I am looking to replace the omnipresent default keyboard and mouse with it. Things like feedback while typing, occlusion from your hands and the fact your fingers are really fat come up while first thinking about it. The benefits are mainly in the flexible nature of multitouch solutions and the higher input shared between cursor and text input.

Most work done is done on a 'darling' of this generation: direct touch. Our current setup is indirect, we type and move our mouse somewhere in the periphery of where we are looking; we manipulate external objects which though cable or wireless transmission lets the computer update the state of your screen. Suddenly words appear where we are looking or windows move. Direct touch is the opposite where we directly manipulate what we want to interact with. You see this on our current smart-phones, tablets and tabletop/wall-screen setups. Direct touch solves a lot of confusing elements of finding your cursor, actually: you do not have to find out the state of your computer before using it.

But, and this is a big but, indirect manipulation has one golden advantage: it follows the natural resting positions of our body. That is why keyboards and mice are still alive; try to use your tablet all day, try to tap on your screen all day. It gets annoying quite fast. That is why I call the fancy new stuff "session work". You have a meeting, arrange stuff on the fancy Perceptive Pixel wall screen, sort your pictures on the ubiquitous multitouch tabletop project, walk through election results, etc. When the session is over, you continue with your "normal" work.

So... indirect multitouch. Most people know 10/Gui, which is already a very cool starting point. It recognises that mouse input could really benefit by upgrading your single cursor to 10 controllable ones. But 10/Gui acknowledges the superiority of the keyboard. Not a bad stance, but where is the fun in that? What can we do to make a big slate of multitouch tabletop screen real useful in daily use cases?

First up next: "I am never giving up my keyboard."

So this one is a straight-up steal from a coffee joint I encountered in NYC. They serve very good coffee (hard to find in the States) and also very decent sandwiches. This one is for sale there.

You need:

• A baguette-style piece of bread
• A green apple (Granny Smith or Golden Delicious will do just fine)
• Some brie
• Some honey
• Some balsamico di modena

Depending on the type and state of the bread: put it in the oven or just cut it straight away. Slice the apple in thin slices and divide the brie in decent parts. Place the apple slices on the bread in a roof-tile pattern and stack the brie on top of it. Drizzle some honey and balsamico on top of it and there you go!

Warning: it's really good.