
10 finger multi-touch UI designed to replace the mouse - zain
http://10gui.com/
======
TrevorJ
Having a non-focused, non-active window arbitrarily take up 1/3 of my viewable
monitor at all times would drive me nuts.

It also makes no sense to me to go to the trouble of allowing a user to use
all ten fingers, and then restricting application interaction to only two or
three fingers. I work in rich, deep applications with tons of controls and
buttons, if I'm going to go to the trouble of learning to use ten fingers to
navigate a computer, you better allow me to make use of all of them withing
the individual programs that I'm using.

~~~
stcredzero
I think you're onto something. OS interaction/application switching should be
simple, and the individual app interaction should be more complex. I think 4
app selection modes are almost too much. 3 is perfect.

Prediction: I suspect that Apple is taking Exposé in exactly this direction.

~~~
TrevorJ
The real logical gap in the thinking here is that they are focusing on
replacing the _mouse_ but in reality, they are in fact replacing functions
that can be executed simply and quickly with a _keyboard_. Application
switching etc can be done efficiently with a keyboard. At the very least I
question the wisdom of moving away from the modifier key paradigm because it
is an elegant and easy method of interaction.

I also believe that as we engage more and more of the body in a control
interface, for it to be natural it is going to need to mimic real-world
interaction so that we can rely on our lifetime of experience interacting with
physical objects.

The better solution in my opinion would contain:

-2 mirrored screens so that the one under your hands can be glanced at for reference as well.

-A UI approach that allows you to interact with objects and windows in the same ways you would with real objects. (Grabbing the edges moves the window, pushing down on it 'sticks' it in place, etc.)

------
Semiapies
Talk about multi-touch reminds me of a friend's perennial complaint, "The
future of interfaces seems designed to screw me over."

He has a congenital deformity of the arms, but he can type (and is one of
those clacky-IBM keyboard fetishists). With his feet, he can use mice and
controllers. But multi-touch systems are beyond his ability - while he can
pivot and shift to type remarkably quickly, he can't get more than one hand on
something at the same time. Worse, each hand is effectively only one digit for
him.

And then, there are the folks who don't even have his degree of dexterity.

I can't help thinking that it might be a good thing for general interface
designers to consider the implications of disabilities for users of these
designs, instead of shrugging that off onto providers of interfaces
specifically for the disabled.

~~~
gecko
Without disagreeing, I also don't think it makes sense to discount potentially
massive improvements in user interfaces merely because a portion of the
population cannot use them.

~~~
joe_the_user
No, but his problem actually points to this being a problem for anyone.

It becomes a problem when the machine requires all of my dexterity just to
accomplish a given task. I dislike all Apple products specifically because
they _require_ mouse operations to accomplish things where Linux and MS
interfaces tend to have keyboard shortcuts.

~~~
andreyf
In my experience, OSX has _much_ better thought-out keyboard shortcuts than
Windows does. A simple example: you'd like to force-kill the focused
application. In OSX, you type command-option-esc (one hand), return, return
(other hand). I don't know what it is in Vita, but in XP, it would require
quite a bit of mousework.

~~~
deyan
Did you (all of you not just andreyf) seriously manage to degrade the
discussion on the GUI into a OSX vs. Windows debate? Or perhaps my
expectations are just way too high..

~~~
andrewtj
Straw, camel back... sorry

Keep expectations high and the community will self-correct.

------
jpwagner
Phenomenal video.

The ideas are definitely interesting, but I am amazed at the clarity of the
presentation.

~~~
deyan
Thanks for posting, it looks quite interesting. The interface is something
that I can imagine myself getting adjusted to (for example, I am a big Opera
user and use gestures all the time - once you get used to them they are
phenomenal).

I had a similar idea a while ago and might try to implement it this summer as
a side project using Johnny Wii's tricks perhaps + Windows 7 (not sure how
much of the interface I can pull off). I would be curious to see in particular
how tired I am (i.e. my hands) after using such an interface for a while.

------
snprbob86
What I find most compelling about this design is that it would support
existing applications. This is a proposal for a new Window Manager, not a
ground-up rewrite. Looking at the local context commands, it seems like
traditional top-level menus could be automatically converted to something
left-hand friendly.

------
extension
It seems pretty obvious to me that the most efficient and intuitive way to
manage windows is just a boring list at the bottom/side of the screen, but I
do hold out hope, like everyone else, that it turns out to be some flashy
zooming 3D multitouch thing.

It is refreshing though, to see research that dispenses with the legacy of GUI
convention we've built up over the years, instead of just piling on new
features.

------
tc
While we're rethinking the human interface, could we please stop staggering
typing keys and put frequently used keys like carriage return and backspace
under strong fingers? [1]

I was disappointed to see at the end of the 10gui video a concept keyboard
with staggered keys and with backspace way over in the corner. If we can't
move keyboards past the constraints imposed by mechanical typewriters, what
hope do we have at changing the mouse paradigm?

[1] Typematrix is the only keyboard maker I know of that currently does this
while still building a relatively standard keyboard (and something that could
be integrated into a laptop).

~~~
RevRal
Colemak layout turns your caps lock into a backspace.

This works surprisingly well, but I really wish the spacebar was split in
half. And I've always felt shift+spacebar should = backspace.

~~~
mjgoins
I use a keyboard with Control where Caps Lock would be on most keyboards (a
sun unix keyboard). That combined with the fact that Control-H is backspace in
a standard unix environment means I don't have to reach for the backspace key.

------
jrockway
Emacs is a 10 finger multi-touch UI. And it works very well.

~~~
stcredzero
Multi-touch is right. A roommate once told me about a guy he knew who rigged
up foot-pedal modifier keys to increase his Emacs productivity. And this was
in the PS2 heyday! (No flames. I was an avid Emacs user in College & grad
school.)

~~~
jrockway
Foot pedals are indeed an excellent idea. There don't seem to be any available
on the market that are programmable under non-Windows-XP, however, and I have
not had the chance to build my own yet.

~~~
dkersten
I believe the kinesis keyboards have an (optional) foot pedal.

~~~
jrockway
I am not too excited by their keyboards, unfortunately. They do sell separate
USB foot pedals, but they act as the mouse buttons until you reprogram them on
a Windows computer.

I don't know anyone that uses Windows.

~~~
stcredzero
Get hold of a Win2K disk image and put it in VirtualBox. (No details given,
but you should be able to get one, plus install key for, _ahem_ , cheap.) You
can connect the virtual image to the USB device. You might need one of the
service packs to get USB to work, however.

~~~
zandorg
I don't know what that is, but you could use a MIDI pedal and hook it up with
some controller software!

~~~
dkersten
Actually, thats not a bad idea.

I once wrote a python program to trigger callback functions when specific keys
on my MIDI keyboard are pressed. The MIDI protocol was surprisingly simple to
figure out (for simple things like "button-x was pressed at 25% pressure"
anyway). It had a learning mode where you assign a key by pressing it and an
action mode where pressing the assigned keys would trigger functions to be
called.

I had intended to use it as a shortcut panel to run a shell scripts, but never
actually finished it.. might do so sometime soon :-)

------
CWuestefeld
That's cool as far as it goes. But what about the keyboard?

The mouse is just one aspect of the computer interaction. I can't remove my
keyboard from my desk just to make room for this mouse replacement. The
keyboard seems a big barrier (literally!) for getting this device on the
desktop.

~~~
joe_the_user
Indeed, double Indeed, what about the keyboard?

Funny how much of the video seems act as if the mouse was the _ONLY_ way that
users actually input information into computers in the last twenty years. But
even the newest user is going to be using the keyboard and the more
experienced a user gets, the more they use the keyboard. And even more, as
computers become _a normal part of human society_ , interfaces _pandering to
novices_ will become less important, not more important. The question will
change, is changing now, from "how do I use this thing?" to "How can I be more
productive with this thing"?

Conside that MOST of the data that users input is going to be text, which a
form of _discreet_ digital data. A mouse or manipulative multitouch interface,
no matter how well designed, is useless for this because it inherently
produces analog input which is then translated to a digit action - dragging a
file to the trash versus just pressing the delete key.

Photo editing or moving windows are exceptional cases but UI designers are
constantly developing clever ways to do them specifically because they allow
all sorts of kinesthetic analogs. Yet none of this solves the problem that
this isn't the interface a user wants to do most their tasks.

~~~
stcredzero
I would like a keyboard like the small Apple aluminum one, but with a multi-
touch pad on the side. Also make the pad a small display like the iPhone. Have
it switch modes depending on the Application. It would still be great for
application switching and contextual menu selection as in the 10Gui video. One
could also have a 10-key mode. Widgets could also appear there.

------
TimothyFitz
This basically already exist(ed): <http://www.fingerworks.com/ST_product.html>

Apple bought them up and stopped making their products (but we got multitouch
on the iPhone, an interesting trade-off).

~~~
lief79
I had interned with Fingerworks prior to the formal release of the project.
The main problem I heard about was the demand cost/issues.

Outside of the proposed UI changes, Fingerworks product did all of this. I
still wish I had bought one when I had the chance.

Anyway, if they are serious about getting these made, I believe University of
Delaware owns a fair amount of Wayne Westerman's patents, as the basic idea
was the premise for his doctorate. He was a professor starting in '99, so that
should make them easier to locate.

------
zain
There's a similar device I read about with a hardware prototype that actually
works. It doesn't have the software support of 10gui but it's basically a
resistive touchscreen mapped to keypress macros (copy/paste, app switching,
etc).

Here's some pictures:

[http://www.flickr.com/photos/pauric/sets/72157616093748066/w...](http://www.flickr.com/photos/pauric/sets/72157616093748066/with/3488471659/)

~~~
NathanKP
Is that the same device? It doesn't look anything like the prototype design in
the video.

I also don't see any correlation between Pauric on Flickr and C. Miller of
10/GUI, at least not by exploring Pauric's Flickr profile or Twitter account.

Perhaps you can explain why you think that those images are of 10/GUI's
device? How did you find them?

~~~
zain
You're right. It isn't the exact same device. I ran across it while doing more
research on 10gui and mistakenly thought it was by the same person. I updated
my post.

Sorry about that.

~~~
NathanKP
Okay no problem. I thought that something was funny as I looked through those
Flickr photos. ;)

------
agazso
I'm looking forward to use a multitouch touchpad with my desktop keyboard just
as I currently do with my laptop. However I will definitely want to keep a
mouse also when in need for more precision for tasks like photo editing etc.

10gui seems to be a very cool prototype and I guess that Apple and Microsoft
are already working on things like this. I also spotted a weakness of this
demo: 10gui uses the little finger to invoke contextual menus. Most of the
people who cannot touch type with all ten fingers (and my guess this is about
the 99% of all computer users :) will find this uncomfortable.

~~~
NathanKP
I agree. A ten finger system is going to appeal greatly to those of us who
already touch type and are used to using multiple fingers. But I suspect that
many people aren't going to be too happy with the coordination required to use
such a system.

But I may be wrong. Perhaps when it comes down to it the interface will feel
so intuitive that there will be no problems at all. We'll have to wait and
see.

~~~
timgasper
That was one of the biggest things I worry about - the learning curve of all
the gestures. A mouse and keyboard are quite straightforward in terms of how
the operate. A 10GUI interface would require a significant amount of training
and practice. At which point maybe yes - it will become completely natural.

------
zacechola
I'd like to see it with the pad acting as the keyboard, too. though, that's
more software implementation, while the video is talking hardware interface.

Although, the problem with that kind of implementation is that it doesn't have
the tactile feel of a keyboard. however, iPhone users seem to have acclimated
after first bitching about it.

~~~
NathanKP
I personally think that a real physical keyboard will always be preferred.
10Gui is doing the right thing by putting the keyboard above the multi-touch
pad.

~~~
lief79
<http://www.fingerworks.com/faq.html>

Apparently people could get up to 70 wpm with what could be considered the
prior version of this.

For what it is worth, at least when I was involved with it, most people found
a keyboard a lot easier to use for rapid typing.

------
bprater
They've certainly put lots of thought into the design of this device/software.
But the whole time I kept thinking: "what about the keyboard?" (Eventually, at
the end, they showed the device in a traditional trackpad position below a
keyboard.)

I wish they had considered a way to make the keyboard a part of the device.
Perhaps an on-screen keyboard, some haptics? The challenge with that part of
the input device is that to touch type, you need that spacial reference a
keyboard gives you.

------
alexandros
I wonder if they could combine the keyboard with the touch surface. Not by
having the surface be the keyboard, but by overlaying a traditional keyboard
with touch sensitive key-sized tiles that together make a touch-sensitive
area. Keys should still be pressable, but when you treat it like a touch
surface it should behave so. If the gaps between keys are snall enough it
should work for fingers. (not styluses though)

------
bryansum
The video mentions the problem of always having one's hands in the way with a
direct interface. This could also be alleviated by moving the hands underneath
the display ala the capacitive mouse from Microsoft Research:
[http://arstechnica.com/hardware/news/2009/10/video-
microsoft...](http://arstechnica.com/hardware/news/2009/10/video-microsoft-
research-demoes-five-multitouch-mice.ars)

------
RevRal
As for a keyboard, why not have the virtual buttons relative to where the the
fingers are placed?

As in, place all eight fingers onto the pad and this automatically becomes
home row. Simply strike above and below home row. Depending on the finger you
moved, it'll know the key you're striking. I rarely have more than one finger
off home row, per hand, when typing.

You heard it here first.

~~~
izcs
was thinking the same thing. i think i've something similar before, and
something tells me it may not be practical.

------
gnubardt
Interesting, would be cool to see worked up with one of thse
<http://www.technologyreview.com/computing/22358/>

(I saw Ken Perlin of the nyu media lab give a presentation on that earlier in
the year and it was incredible, high resolution and cheap to make)

------
mitko
Scrolling alone already had decreased a lot of tension in my arms. 10/GUI
would probably have great effect on RSI reduction.

It is amazing how intuitive touch can be. I just love the multi-touch on the
touchpad that Ubuntu for example supports. I never knew my old Thinkpad T43
would be able to do it but it does.

------
surfmike
Like this example of bold but concrete thinking on a future computing
metaphor.

Two of the concepts seem very heavily Palm Pre-inspired: The window management
scheme (reminiscent of 'cards') and the gesture areas on each side, one of
which is used to launch apps just like the Pre

------
username
I'd rather have more modifier keys and mouse buttons or modifier mouse
buttons. Instead of pinching with fingers, you could move the mouse while
holding another mouse button. For three or more finger gestures, you could
hold down another mouse button.

------
zokier
I'm not quite convinced that multi-touch panel that large would be practical,
but smaller, one hand version could be nice. Imho it would also be better if
the panel was on the side of a keyboard(ie in the place of numpad and arrow
keys), not under.

------
chrismear
"Who would build something like this?

Just as in the early days of the mouse and windowed GUI, a system based upon
10/GUI's principles would likely be best suited to a systems builder capable
of full vertical integration."

I wonder who they're thinking of.

------
saurabh
It certainly looks cool. I am just not sure about the ergonomics. What would
really be awesome is not having to lift my hand at all by switching between
touch mode and keyboard mode at a flick of a finger.

------
da5e
I wonder if they could with a tap bring up an virtual keyboard on the touchpad
were the fingers already are and do away with a keyboard altogether. This sort
of harkens back to the chord keyset.

------
thomasfl
A ten finger multitouch pad for desktops makes perfectly sense. The multitouch
pad on mac book pro is way more pleasant to use than a mouse, but can't be
used with an external monitor.

~~~
jodrellblank
It isn't more pleasant to use than a mouse.

If I have to scrape two surfaces together constantly to use a computer, I'd
rather one of them wasn't the most sensitive part of my finger.

------
th0ma5
Multitouch innovators should take heed. A lot of the NUIGroup community is
centered around hardware innovations, which is cool and all, but we need more
and more software!

------
mcav
Anyone happen to know what software likely produced that video?

------
tomstuart
Not sure they should've let Patrick Bateman narrate the video.

"This is Sussudio, a great, great song, a personal favorite."

------
DannoHung
Seems interesting, but why not use a fully dynamic window manager instead of
the side to side thing?

------
aik
Pretty cool. I could see my productivity increase from something like that.

------
mhb
tl;dw

30 seconds in and they haven't shown it yet?

~~~
gjm11
It's worse than that. The first time they say anything about their interface
is at about 2:00. That's extremely vague (it's a multitouch sensor! where your
mouse would be!). No more specifics until about 4:00 (windows are arranged in
only one dimension). They don't actually show the damn thing until 4:45.

Oh, and all accompanied by the Most Annoying Music In The World.

~~~
NathanKP
Patience. Any good idea requires a full explanation to build it up. I feel
that they did the right thing by contrasting the mouse with their new system
because it helped show the benefits of a ten figure multi-touch pad compared
with a single point mouse.

~~~
DanielStraight
I'm not entirely convinced they showed such a benefit. I can already use
multiple fingers to move windows by clicking anywhere on them... two of those
fingers are just on buttons. Not to mention, I still only have two eyes. I
can't be looking at 4 things on the screen at the same time anyway.

------
bjclark
"A single finger manipulates items inside of an application. <insert finger
painting>"

Ding. Ding. Ding. We have #fail.

It's pretty cool and all, but this isn't going to change anyone's life. The
best idea here is just putting a big touch pad on the desktop in front of your
monitor.

