Hacker News new | past | comments | ask | show | jobs | submit login
Looking Glass, a revolutionary window manager revealed in 2006
174 points by pierlooqup 6 months ago | hide | past | web | favorite | 124 comments
I recently redescovered the keynote video in which Looking Glass was announced and realized it’s now 12 years old. I remember watching this video in awe and still today I findit quite remarkable although a bit gimmicky.


I wonder what was the motivation behind this project and why it never really took off. Also what are the people behind it up to nowadays - does anyone here know the backstory to this?

When 3D acceleration became mainstream and people started developing 3D-accelerated desktop it was very trendy to have a bunch of pointless 3D stuff going on. I think it was around that time that compiz and its 3D burning cubes and other mostly (but not entirely) useless eye candy were released: https://www.youtube.com/watch?v=4QokOwvPxrE

I think there were two reasons for this:

- It looked cool, even if sometimes very impractical. Now it's a bit mundane but at the time it was like breaking the 3rd wall, we were so used to the flatness of the desktop that it was really weird to view it as a 3D object.

- Apple and Microsoft were in no rush to bring these features to their commercial OSs (and proably for good reasons, MS tried to have the 3D carousel in Vista but even that wasn't very useful). That meant that people in the Linux world in particular could show off "hey, can your Windows XP do that?"

But of course eventually we realized that 90% of these features were counter-productive so we only kept the bits that made sense (ability to scale down windows in real time easily to make thumbnails or previews, faster rendering, a bit of transparency etc...).

Compiz (and the wobbly windows in particular) were what got me to try Linux in the first place. Previously I'd always thought of it as a server/workstation OS that was very stuffy and minimal. A friend showed me the desktop effects, which completely changed my view; I'd never before realized that linux could look cool.

This reminds me how old I am now and how young I was back then. At the time, stuff like this filled me with excitement. Now I look at it and think it's obvious that the annoyance factor and lost performance will vastly outweigh the legitimately cool applications.

It also reminds me that I used to tweak a bunch of settings in OS X to speed up and disable animations, but I gave up years ago because Apple kept breaking them. Now I just use OS X with its default settings and stay away from things like Spaces that insist on cramming animations into my attention. That makes me sad, so I am going to go back to not thinking about it.

Hi I'm a recent convert to OSX and found the animations of 3-finger-swiping between desktops annoying.

Recently found the "reduce motion" checkbox in System Preferences > Accessibility and instead of sliding it just does fade-in-place which is way less annoying and I think feels faster because you start to see the other window right away, even if there's a window fading away on top of it. The slide from desktop to desktop feels like I need a split second to orient myself after being spun in a chair.

The only good implementation of a 3D desktop effect (outside of straight-up window rendering) is Expose, which Apple demoed in 2002/2003 for 10.3.

I remember, circa 2005, there being compiz extensions that enabled something similar.

It could expose all the windows in the current monitor, on all monitors of the current workspace, or all windows period.

Virtual desktops which I believe linux supported long before spaces acquired a nice visual metaphor with compiz which allowed you to zoom out to see them all and drag windows between one and the other.

Windows could be set to have a small amount of resistance when passing other windows to make it easy to stick them side be size.

The effects for windows creation and destruction were quite cool and super configurable.

Translucent windows when moving and resizing is really nice (KDE).

Having had to move to Windows for work, I actually miss wobbly windows. All the fancy over the top effects and 3D stuff? No. But for some reason for me, the fluid windows felt so much better than the perfect stiff boxes I have to use now.

I have the exact opposite response. The wobbly windows look wacky and distracting. To my eye, windows are simply not meant to do that. I suppose it's a matter of what you're used to (as is often the case when it comes to UI opinions).

Yeah, I most likely got seduced by all the fancy stuff in the early Compiz days and then used the wobbly windows long enough that they are what feel normal.

Rotating cubes and wobbly windows were in macOS.

I don't think they were...

The cube animation was introduced in 2002-2003 if I remember correctly, and it is still present in macOS when changing users using fast-user-switching.

I remember testing it on my Ubuntu partition soon after I learned about the project. I recorded a little video showing some of its capabilities. It became my most viewed video ever on YouTube with over 500K


Some good quality images of it running with my blog in the old days:





Hey, I know you from the OSX articles in magazines, you sure would like that ala Exposé ;)

Windows, and the walls they rest in, are basically 2D at their principle point of consumption (indoors, looking out).

Desks provide much more creative value across the first two dimensions than the 3rd: you want the stuff you're working on within sight, touch and arm's reach. "Piling up" is a "storage" or "attention reducing" strategy because you can't work with the piled items or the content with your hands. You largely don't care what they even are unless you're working on them... so pushing virtual representations of work surfaces deeper down an imaginary z-axis makes little sense.

The demos in the video look fun and exciting, and probably justified the applause at the time. I think the desktop metaphor could be worked over. Tablets and phones dropped it from the get-go. But I'm yet to be convinced that visually arranging data across a projected third dimension except when you want to draw the user through (like in a game) adds much to the experience.

The flipping and "piling" you get in Windows and macOS seem like decent uses of 3D. Those effects basically just add+hide 2D surface. It's the vanishing-point stuff that seems gimmicky to me.

>I'm yet to be convinced that visually arranging data across a projected third dimension except when you want to draw the user through (like in a game) adds much to the experience.

While not entirely convinced myself, I was persuaded to think that there might actually be some value in this after watching the various videos where people have built very large functional constructions in Minecraft. Watching the creators explain what they've done, and seeing how easy it is for them to navigate even tremendously complex and interlinked structures through reference to the relative positions of different pieces certainly makes it seem like there's something to it to me.

Note: most games are also 2D with a bit of 3D visuals thrown on top. True 3D controls are super tough for new players.

Not to mention disorienting. I loved the Descent [1] series but damn was it ever difficult to wrap your head around, at least at the beginning.

[1] https://en.wikipedia.org/wiki/Descent_(1995_video_game)

Hardly revolutionary.

I was following this back then -- Apple had just come out with 3D manipulations in the desktop (e.g. turning a widget window around in dashboard around to view their configuration on the "other side").

Some engineer at Sun, in their spare time, put together a quick demo with 3D windows (using Java/OpenGL bindings IIRC). Basically a quick play on the idea of windows being objects rendered in a 3D space by a compositor (which OS X had introduced to the mainstream a while before).

OS X still rendered them as 2D projection, except for the occasional effect like the flip-around in 3D space, while this demo had them floating by default (but still only shown the same effects, flip around to write notes etc).

The Looking Glass demo was very basic, it got some talk in forums at the time, and it was even presented a few months later in a SUN "keynote". For a while they pretended like they had something there, instead of just a half-arsed demo.

There was never any big project around it, nor much thought. The highlight was ...turning a window around to write notes, configure, etc (e.g. a copy of what Apple had done commercially and already shipped with minor changes).

Even the very concept of Windows in 3D space is not that novel, Microsoft had done something similar before Apple, as well as others.

It didn't went anywhere, because (at least as implemented this far) it doesn't solve any problem better than the regular desktop.

In short: Looking Glass never went anywhere because it was a proof of concept by 1-2 Sun employees, when window compositing became possible in XWindows/Java.

This was just part of Sun's JDS (Java Desktop System). The motivation behind it I can only speculate: to get attention for Sun's JDS, demo Java + 3D while at the same time differentiate from default Linux desktop, and to demo the successor of CDE; basically it all boils down to stock price of SUNW/JAVA.

Remember, Sun didn't do well after the dotcom boom and Sun wasn't cheap but Sun did invest a lot in desktop UNIX (Gnome HIG, documentation, internationalisation, and a major contributor with LOC). Sun GPLed this and many other things (among which Java itself and Solaris) right before they were sold out (to Oracle). We can only thank Jonathan Schwartz & Co for that because Oracle would've kept it proprietary (speculation though).

It isn't very innovative. It was one of the many window managers which did this. 3dwm, 3d desktop [1] [2] (from around 2002 IIRC), Beryl, Compiz, and Enlightenment (E17 & onward) are some other examples though not all of these were 3D accelerated.

As you said, the problem with this is its gimmicky; not productive. In gaming nice 3D effects can add something to the experience (immersion) but on a desktop it shouldn't be very noisy or abundant. So after the initial wow-effect was over these effects didn't get a long lasting stay in products.

[1] http://linuxreviews.org/features/3ddesktop/

[2] http://desk3d.sourceforge.net/

Thanks for the info and I agree - it was a pretty doomed UX to begin with and definitely not the first time 3d made an appearance on the desktop. The keynote definitely had a wow effect to it though, and it came from an unusual “competitor”.

> As you said, the problem with this is its gimmicky; not productive.

Things like that being gimmicky is not the major problem. The major problem is they rely on 3D hardware which (or its drivers probably) is always more or less quirky. I have not seen a single PC on which Compiz would perform reliably without occasional crashes and/or artifacts and I can see no practical reason for a window manager to use hardware 3D as window decorations are not that hard to render, alt+tab task-switching 3D cube is hardly useful and wobbly windows effect is not just useless but also annoying. I believe there should be a category of "wow! cool!" window managers with all kinds of Hollywood effects for fancy demos, cool kids, drivers debugging and concept experimentation but no serous distro should rely on them the way Canonical did cementing a Compiz into Unity.

At the same time I would love to try a WM like the one by the link running in a VR environment and controlled with old-school cyberpunk VR gloves or something.

Just a small nitpick: Solaris wasn't GPLed, but remains CDDL

The source was reclosed back in 2010 when Oracle bought Sun.

This is only partially accurate. Parts of Solaris continued to be published under the CDDL and other licenses after that. But yes, the kernel and parts of the system libraries were no longer published. Remember that unlike Linux, Solaris is more than just a kernel.

Looking at their source code dump for 11.4, nearly everything that isn't GPL is missing. It's less open than macos.

Then you're probably looking in the wrong place for the non-GPL items. Those are typically on github, for example:


That's the only Solaris item on Oracle's github, and it's just the package manager. Where's the rest of the OS?

I wonder if after that, the copies that were distributed under GPL could still be distributed as such. Are the GPL kernel and system library sources still available to download from somewhere (like archive.org or maybe a Solaris fan's website)?

Yes, they could, but no significant part of OpenSolaris was GPL -- it was all, as much as possible, CDDL. When Oracle killed OpenSolaris that did not cause the previously CDDL'ed code to be no-longer open source, and indeed, there was a fork: Illumos.

Oh wow! Any insights for the curious? I’d love to know more than what’s already on wikipedia from a first hand source! Cheers

Someone put Synapse on Wikipedia? Do you have a link?

My friend Nick McKinney and I borrowed $25k from a chiropractor and spent a year trying to get Linux users to replace X with Synapse (which was a paid offering that wasn't compatible with anything.) We had no idea what we were doing, no idea how to get customers, etc. So we gave up and released it as open source.

It was a lot of fun! I have tons of stories, but I gotta go because I am trying to prepare for a talk (currently posted on http://js.la/)

Good luck then!

I have no idea, but I guess that it was built because it was technically feasible.

Xgl [1] was released in the same year and had quite similar capabilities [2]. Xgl, in turn, lead to Compiz/Beryl which is the technical ancestor of the most popular window managers we run on Linux desktops nowadays (e.g., Kwin in KDE).

So to some extent, we still use the tech which was developed back then, the effects are just a lot more subtle.

[1]: https://de.wikipedia.org/wiki/Xgl

[2]: https://www.youtube.com/watch?v=-CgqWlX_GsI

Likewise re: subtlety, the Windows Vista/7 Win+Tab animation used a 3D card stack, the Windows 10 ones uses a simple zoom-out. Both are 3D, but people seem to prefer the ones which don't scream "3D!" as much.

There's acryllic everywhere too, which is blurry transparency, but you might not notice it either.

That said: I miss Compiz wobbly windows. They were cool.

Wobbly windows and cube animation for extra oomph! :-)

> I miss Compiz wobbly windows. They were cool.

Last I checked, Kwin still had that functionnality.

wobbly windows is so 2006. Enjoy clothy windows. https://videopress.com/v/zmiBKUyQ

For bonus points, Arcan & Durden are a lot more hackable than Xorg & Beryl :-)

at the very least, they'll have to work -hard- to get it as easy to hack as https://github.com/letoram/durden/blob/master/durden/tools/f... :-)

Is this using LuaJIT or just plain Lua?

Because I get the impression this (not-C, not-Rust, not-stereotypically-fast) code is what's running at 60(?)fps, doing the actual deformation animation.


Both are supported, but it's not particularly heavy - it's only running at a monotonic clock of 25Hz (decoupled from display updates and deliberately chosen to not be evenly divisible with 60Hz to pinpoint animation issues).

Alas, the improvement that should really be made is to encode both n and n+1 in the same update and interpolate in the vertex shader, not hard here but I was lazy..

where is that from?

part of the 'flair' tool from: durden.arcan-fe.com

Ah cool. I miss them because I jumped off Linux and switched to OSX/macOS for a few years and then moved to Windows.

That said, I just found https://www.stardock.com/products/windowfx/

Update: Compiz style wobbliness on Windows:


Wobbly windows are definitely available with Kwin 5.13.5 (September 2018). I love that effect too, so it is always enabled on my PC :-)

I think most compositing window managers do, having adopted a lot of plugins from Compiz and Beryl.

Xgl was attempt to do the same thing by SuSE, which led to lot of opposition from other Linux distributors. This spawned competing AIGLX project (mostly associated with RedHat in that times), which solved the same "problem" from the other end and enabled such graphical effects mostly as an side effect and in contrast to Xgl without degrading performance of OpenGL applications (Xgl did not allow it's clients to use hardware acceleration).

The difference is that both Looking Glass and Xgl works as an proxy X server. While AIGLX was an effort to make client-side libgl implementations usable from the X server itself, which is good for Acceleration of GLX in Indirect mode (hence AIGLX) but also as a general HW acceleration backend for things like XRender and XVideo (for example intel driver emulates XVideo in terms of OpenGL) and with addition of one simple OpenGL extension (ability to create GL texture from contents of X11 window/pixmap) allows all the 3D window manager tricks.

I remember this coming out my first year of college and taking the boot disk for a spin.

At the time it was pretty cutting edge and the demos made it seem very polished and useful. In actuality the live version was rough round the edges, fonts and rendering looked soft. It was definitely a proof of concept and didn't really have much breadth beyond the few use cases presented in the video above.

That said yes, it was very cool but even back in 2006 I and my classmates were questioning quite what you'd do with such a product. It seemed like a solution in search of a problem.

Also OS X 10.4 Tiger was when Apple's Mac platform became very solid indeed and started making inroads in academia and enterprise. Most of our professors for example switched from Linux to Mac around then.

I think Looking Glass failed then not just due to legal pressure from Apple but competitive pressure too. I also doubt Sun - which if memory serves had financial issues and was suffering diminishing sales of their SPARC platform - had the clout or will to invest in this particular project.

Anyone remember Bumptop

description from wikipedia

BumpTop was a skeuomorphic desktop environment app that simulates the normal behavior and physical properties of a real-world desk and enhances it with automatic tools to organize its contents. It is aimed at stylus interaction, making it more suitable for tablet computers and handheld PCs. It was created at the University of Toronto as Anand Agarawala's master's thesis. Anand Agarawala also gave a presentation at the TED conference about his idea. The 1.0 version was released on April 8, 2009, along with a fully featured pro version as a paid upgrade.[1] On April 30, 2010 the author announced that BumpTop was being discontinued and that they were taking the software "in an exciting new direction."[2] Two days later, it was announced that the company had been acquired by Google. On January 5, 2011, Google released a sneak preview video of Android 3.0 Honeycomb[3] showing a 3D desktop with features purportedly taken from BumpTop.[4][5]


Yeah, awesome stuff - I used some of Anand's ideas in my own research at the time.

I remember being wowed by this at the time, but, looking back on it now, I think I was maybe just captivated by the fact that they were throwing a bunch of technology at it, not what it was accomplishing.

I see two ideas for how it might be useful that are practical: Putting windows "out of the way" when they're not being used, and attaching notes to files.

For the first, I'm not sure the 3D works well with how I like to work, because I like to use up all my screen real estate, and a bunch of floating tilted windows are still in the way (because they take up a bunch of space - maybe 15-20% of the real estate of the full window, in the examples from the demo) using this approach. On OS X, by contrast, I have several options, all of which really do get the window completely out of the way - I can hide it, I can minimize it, or I can stick it on its own desktop. Throw a window tiling tool like Spectacle in there, and I get a pretty large amount of power, while still keeping things simple.

Adding notes to files by writing on the back of the window feels odd to me, UX-wise. To view the notes, do I have to open the file and then flip the window around? How do I know that a file has a note, without doing that? I think I'd rather just have a "notes" bit of metadata that is viewable in the file manager.

In general, I think that 3D is maybe just awkward for organization. Evidence: Out in the real world, we have invented all sorts of objects for 2d-izing - or even linearizing - things for organizational purposes. Shelves and file cabinets, for example.

Yes, hiding notes on the back of a window doesn't look a good idea. Furthermore it hides the object of the note from the user when the user is writing the note. One really needs good memory.

I looked through the video and I didn't see anything I want to use. Some concepts made their way into some current desktops (the dock, floating windows on alt tab) or maybe were copied from some previous one. I don't use docks, I take my time to remove them from the desktops I use. If I can't, I use a different OS.

3D might have been a solution to limited pixel space. Now, pixels are cheap and abundant, so you can lay everything out and switch back and forth with a flick of your eyes.

I guess. . . but even around that time, when I was doing a lot of work on a pretty pixel-limited laptop screen, I typically made real estate by just having several different virtual terminal sessions (plus one for X). A flick of an F-key to switch VT's wasn't as quick as a flick of the eyes, but it was close enough.

And way better than anything that involves reaching for the mouse, which would seem to be an essential part of doing it in a 3D DE.

Nothing of value was lost. None of what this WM offers seems to be really useful.

I agree. This was a nice technical exercise/demo, but did not offer any notable usability improvements over other window managers.

I agree. In terms of fancy visual effects and CPU usage, I prefer my window manager at the opposite end of the spectrum, e.g. i3wm.

I'm still waiting for VR pixel density to get high enough to have an actually 3D window manager.

No more buying monitors, just buy one headset and I'm done.

Is there any modern prototypes/demos/products doing a code explorer/editor in a VR environment? The vision of 360 navigation and lots of screen space, with added Z depth, is very enticing, but have yet to found anything in this direction.


Doesn't really compete with a regular keyboard+monitor in terms of ergonomics/eye health/fatigue/input methods right now. Maybe in 5 years

I recall some tentative stabs at it, but I think the hardware's unreadiness really puts a damper on development efforts. There isn't much point jumping out years in advance, with the way display managers have gone over the past decade it's pretty much a trivial problem now. (Or at least, trivial as such problems go. Still a lot of coding and I'm sure a lot of fresh new corner cases, but nothing fundamentally unknown or impossible.) We know that if the hardware was there, we could create such a desktop environment now. Anything you did now would run the risk of being completely obsolete before the hardware got there.

> the hardware's unreadiness really puts a damper on development efforts

And regrettably it means that now, when COTS hardware components are arguably sufficient for HMDs to start competing with screens, there's no code, no widespread perception of need, and thus no clear market to target. And thus no hardware for sale. Bootstrap deadlock.

Given the effect Minority Report's UI footage seemed to have on popular imagination, perhaps it might be helpful to have some nice demos? So say software devs can have a "I want that!" moment. And industry can have a "oh, well that, that we can build for you now" head-out-of- err, heads-up moment.

Sent from my Lenovo Explorer WMR HMD running on an old laptop's integrated graphics. Simple custom stack. (But only briefly, so I could say that.)

"Given the effect Minority Report's UI footage seemed to have on popular imagination, perhaps it might be helpful to have some nice demos?"

Well, I think that might be a more minor contributor to the problem, but definitely a contributor. The Minority Report interface is utter garbage and I seriously doubt anything we build is going to look like that in the end. Vague gestures that even strong AI is unlikely to be able to correctly determine the target for reliably, arms held in positions that are completely impossible to be held in for any period of time, the entire "interface" is a usability nightmare. Screens far more separated than they should be... just because I can spread my workspace out over 210 degrees doesn't mean that's actually going to be a good idea.

But it's not as cool if you have anything conventional in it.

I'm playing the recent Switch The World Ends With You re-release. It has a mode where you nominally point the controller at the screen to control a cursor. But there's no optical component to it; it just centers when you hit a button and uses the accelerometer. So of course I don't sit there like a dope trying to point at the screen. The controller just sits in my hand where ever it is comfortable, and I twitch in the appropriate manner. It may be "pointed" 60 degrees off the actual screen, but it works fine. I have some similar stories for the Wii, where I still may have had to point the remote at the screen, but I certainly didn't play the games like the commercials. I played them in a much less awesome manner... but a much more comfortable one.

The first Wii Golf was the funniest... eventually I settled on holding the remote pointing straight up, and wiggling it back and then forward again for my stroke. It was not exactly a realistic interpretation of how one swings a golf club. Exquisite control compared to trying to do it the "right" way, though. The second Wii Golf got smart enough that didn't work anymore.

What's a real pity is that there probably is a really cool demo waiting for this. It just involves thinking a bit more creatively.

I'm sorry, I was unclear.

I agree the Minority Report UI was poor. But seeing it, seemed to get people talking. People who normally wouldn't think about possibilities for future UIs. It seemed to fire imagination, to create anticipation. It had random TV news people talking about UI design. My impression is an "Iron Man" UI was similar. So UI demos, at least in blockbuster movies, can create popular interest.

There currently seems an absence of anticipation for VR/AR "office work" screen-replacement UIs. And there is currently a failure to market VR hardware with tradeoffs made with that focus, rather than for gaming. I was speculating that we might get such hardware sooner, by creating a perception of a market worth attending to, by increasing popular anticipation, by making demos.

Because waiting to get it as a side-effect of gaming tech has already cost a year or few, and seems likely to cost more than a couple more. I'd rather not lose a decade unnecessarily waiting. I can't spare one, and it's not clear society can either.

Regards controllers, oh yes. Remember skeuomorphic design for user interfaces? Things should look like their physical counterparts? A calendar app should have a worn leather border? To ease on-boarding of all the new users not yet trained on phones? I suggest VR is in its skeuomorphic design phase. In contrast, my own interest is in expert UIs for all-day every-day use. So as you say, they are minimum-viable-twitch ergonomics. And any resemblance to the physical world, with its many and unmotivated constraints, is a design smell. But people are focused on gaming, training, CAD, and tours, and Reality is even part of the name. While aphysical UIs seem clearly the way to go, for much non-novice use.

> thinking a bit more creatively.

Do you think there would be any interest in a demo gif of a laptop with "mouse ear" mirrors, providing parallax to a laptop webcam with a clip-on fisheye, for high-resolution hand and chopstick tracking?

On a similar note, I'm looking forward to innovations relating to keyboard/mouse input. Without the restriction of a monitor, various advancements could be made to improve how information is entered.

Motorcar is a cool demo of this: https://www.youtube.com/watch?v=Dgtba_GpG-U

I think we may actually be there with the Vive Pro, but that's around $1000 for the HMD + tracking gear alone, and I can imagine that using a keyboard isn't going to be easy, even for touch typists.

>> No more buying monitors

> I think we may actually be there with the Vive Pro

Perhaps if ones work environment is green terminals. I once heard rumor of an ops person being happy. 1600px sounds like a lot. But Vive's PenTile subpixel layout means only green is full resolution. And at least with Vive and WMR lenses, there's only a small region a few hundred pixels across where pixels aren't blurred together. Non-PenTile (non-Odyssey) WMR gets you color and subpixel rendering. But most devs will likely be unhappy peering through a VGA portal, even if you minimize head sloshing by remapping it. So basically no, current HMDs are far from competitive with screens.

With better lenses and a larger panel, maybe. But the market seems mistakenly blocked on a misconception of "need immersion -> need high fps -> need next-gen GPU and game/cad support". Not "you can run existing HMDs on old laptop integrated graphics" (I do), and at least some people will buy an HMD with 4K panels, and even use them with laptop low-end integrated graphics, as soon as someone gets around to selling them. Perhaps even with the same blurry lenses. Desktop-ish screen replacement is very different than gaming. Gaming has lots of severe constraints. Which was nice for pushing some tech progress, but now it's causing a stall.

This is cool for animation demo. From UX stand point is almost a joke. Apple intentionally is using animations only when user attention is low. My problem with window managers are not animations. My problem is that this is old paradigm and desktop needs to be reinvented by extending and changing user experience. 3d interaction with information objects in virtual space is future. But the big companies don't care anymore. They are focused in cashing the cow and capitalizing on old ideas of Apple and S.Jobs. There is a big room for innovation but centralized investment in R&D and testing are needed. I have tinkered with this and generated some ideas, but after estimation the result is that this is too big for small dev teams. So we will wait GUI to become cool again and wait for someone with imagination and skill to penetrate the walls of reason and get the blessing from excel gods.:)

> From UX stand point is almost a joke [...] ideas

Any favorite ideas or literature pointers?

I've an interest in coding in VR/AR. But even pushing current HMDs with custom stacks, it's not, for me, yet competitive with screens. Since the hardware market is being so dysfunctionally slow to do the easy, I'm stuck to doing hybrids. Blending of screen 2D, anaglyph, and HMD. Laptop with kludged optical hand and chopstick tracking. Annoying eyewear swapping.

Point is, I've been surprised by how unhelpfully limited the 3D UX literature I've seen has been. It's almost like no one has pursued it with serious intent. That "almost a joke" resonated. Perhaps by the time someone struggles with interface devices, and a bit of code, well, you have your paper or thesis, and there's no big market, so you're done.

But with improving web dev and python ecosystems, it's now more plausible than ever to do full-stack reinvention, devices to environment to apps. No market, but something for itch scratching and joy. Still daunting, but seemingly small-team/silly-person accessible. But I'm missing that warm fuzzy feeling of knowing just how I'm positioned wrt an organized professional research effort. And the collaboration.

> 3d interaction with information objects in virtual space is future

Perhaps it is because we're interacting with 3D objects in real space all the time. Unfortunately we're still lacking a convenient and widely available device to interact with virtual objects. VR/AR helmets are not convenient. <sarcasm>Imagine the success of 2D desktops if we had to wear sensor gloves instead of using mice and touchpads</sarcasm>. Even them were not optimal. Touching a screen with a finger was so better (for low precision pointing) that for some people it's the only interaction device with a computing device.

>3d interaction with information objects in virtual space is future

[citation needed]


To be fair this doesn't necessarily mean AR/VR is the future. It could mean a physical designed for computing like Dynamicland or a single device that has intelligent real-time understanding of the physical world that isn't and hmd or a phone.

Bumptop was even worse from a UX standpoint.

Ah, Sun and their prototype user interfaces.

How about Starfire (1993)? This was Bruce Tognazzini's big project. See how many modern UI elements you can pick out of this concept video:


It's easy to forget context. So here's the 1993 ground-breaking PowerBook[1] (640×400; not yet with optional color). And the Thinkpad 700C[2], with internal 3.5in floppy drive (not on a cable)! 2G and GSM are ariving, with flip phones.[3] And available next year, the first PDA-phone combo.[4] Tech moves very slowly, but if you live long enough, there is progress.

[1] https://en.wikipedia.org/wiki/PowerBook_100_series [2] http://oldcomputers.net/ibm-thinkpad.html [3] https://www.mobilephonehistory.co.uk/motorola/motorola_flip_... [4] https://en.wikipedia.org/wiki/IBM_Simon

I remember that and feel it still shows several elements we're missing today:

1. Very large form factor concave displays. (We're getting closer, but the HD era really slowed display innovation.)

2. Good gesture input.

3. Seamless integration of drafting/drawing surface and display.

4. Personal application omnipresence across all input devices (personally, I think this is the most significant element often seen in science fiction that is missing from modern computing.)

> 3. Seamless integration of drafting/drawing surface and display.

A current analog is pairing an art display tablet with normal screens. But people arrange them variously, and not-infrequently on separate and movable arms. So seamless physical integration might not be the right thing. Another analog is the new Lenovo Yoga Book C930 (2-in-1 with keyboard replaced by an e-ink touch/pen screen).

According to https://en.wikipedia.org/wiki/Project_Looking_Glass, it was killed by Apple (thanks software patents). It is open source though.

That page lists some descendants.

"Although we ended up abandoning Looking Glass, Steve’s threat didn’t figure into our decision (the last thing enterprises wanted was a new desktop – in hindsight, exactly the wrong audience to poll (we should’ve been asking developers, not CIO’s)" -- linked from the wikipedia article https://jonathanischwartz.wordpress.com/2010/03/09/good-arti...

I'm guessing it didn't help though.

Oh thanks. I should’ve googled cause this part of the Wikipedia article has all the answers:

After unveiling the prototype, Steve Jobs called Schwartz's office and told him that Apple would sue Sun if they moved forward to commercialize it. Jobs claimed that the project is infringing Apple's IP.[3] Regardless of the threat, Sun determined that the project was not a priority and decided not to put more resource to develop it further into product quality.

Still I wonder what Sun thought they were going to do with this... I find it fascinating they’d pursue a heavyweight UX project such as a window manager with no clear stake in the game... a last chance at pushing for Solaris adoption?!

>I find it fascinating they’d pursue a heavyweight UX project such as a window manager with no clear stake in the game... a last chance at pushing for Solaris adoption?!

Not sure. But around that time Sun had invested heavily in the Gnome project and made a lot of commits that enabled lots of more "boring" things to be added. eg. i18n, accessibility etc. This was because with Linux based Java Desktop (i.e. their polished gnome implementation) they had won various corporate & government contracts in different industries which required these changes, benefiting the gnome project immensely.

Whether the very popular compositing WMs (e.g. Metacity/Compiz) indicated a necessary investment in that area (hacker prelude to new industry), I don't know.

I don't think they had ever really intended it for widespread use. Sun had already jumped on the GNOME bandwagon back in around 2000, given that CDE was extremely dated looking by then.

In 1997 my Comp Sci undergrad dissertation was on the '3D desktop'. The C++ code I wrote for my demo did software 2D on svgalib on the Linux console (there was some way to get a high res mode, I don't remember what exactly).

I didn't have video or semi-transparency, but I'd gone for a similar idea of clearing windows out of the way - sitting at angles on the sides of the screen.

I realised too late that what I really wanted wasn't 3D, but documents as first class citizens, with modular tools working on documents rather than monolithic applications.

If you clicked on a file icon it would graphically 'open' the file and bring it to the foreground for viewing, then you'd bring tools to it - for example, you'd have application-agnostic text tools for text editing.

I started adding things like 'filters' - where you'd drop a file on a filter icon and it would be animated going through the filter and dropping onto the 'desktop' underneath - e.g. you would be able to apply troff+pic and get a diagram.

It was all naive and probably based on ideas that others were having and I'd seen around (e.g. articles about OS/2), but it was enjoyable discovering more and more ideas as I dug further. My real inspiration was, I believe, TkDesk, which wowed me when I saw I could edit the functionality of my desktop /on the fly/.

What we have in modern desktop environments is great, so I'm a bit sad that the idea of a powerful desktop environment seems to never have taken off as far as some of us hoped was possible. Maybe it just doesn't work!

Anyway, back to the command line...

so basically OLE in Windows or OpenDoc in Mac. Both platforms for some reasons abandon that concept.

Makes me wonder about what happened to compiz [1]

[1] http://www.compiz.org/

Ubuntu shipped with Compiz by default up until last year. https://www.omgubuntu.co.uk/2018/09/wobbly-windows-new-liban... I think the control panel to enable the weird effects was not installed by default though?

The original author and main developer, Dave Reveman, started working on other projects for Novell (Silverlight, etc.), then joined Google, where he worked on ChromeOS compositing (obviously) and, more recently, on Crostini. He's a really crazy dude, even outside of computing.

With Novell out of the picture, IIRC the project was forked, rewritten and merged a couple of times. Canonical ended up driving most of the development since. Last time I tried it, a year or two ago, it still had those cracktastic special effects, e.g. the flames circling the pointer when you activate the "Show mouse" shortcut.

It's still around, still seems to be getting maintained, people still use it. KDE has had most of the 'cool' compiz effects baked into KWin for ages now, just not turned on by default.

somehow the jelly windows never got old for me :)

I think here is the newest version existent: https://github.com/compiz-reloaded

14 y/olds grew up.

Where "growing up" of course means becoming cynical and bitter, and pretending that hating fun is "mature"? And before you tell me that it was impractical... sure, a bit, but I really doubt the wm is a significant limiting factor in anyone's workflow, and it's still every bit as useful as whatever Windows uses these days.

Fun doesn't belong in a window manager for daily use, I'm sorry. Cute UI animations have a very strong "Pepsi taste test" effect: when you are just trying them out they look interesting and cool, but over the long term, over daily use, they are distracting and present other disadvantages: all compositing WMs, including those on macOS and Windows, introduce latency that straightforward tiling or stacking WMs lack.

I put goofy things on my terminal backgrounds and desktop. For me they don't seem to get old. I like charm in my life and my UI. As long as it doesn't compromise the utility.

So far I haven't.

I remember this. It has cool visuals, but I remember thinking something along the lines of this is the window manager that would be developed if you went and got some devs from the demo scene and said, build a window manager. Looks great, I really would not like to use it in real life.

Now I really want to see what the output of a demo competition focused on window managers would look like.

Lots and lots of transition animations, and potentially very poor UI/UX model.

Demos are not (largely) interactive, so the demoscene is not known for collective skill designing GUIs.

The most similar/related type of things I can think of that are interactive are crack/keygen programs that include chiptunes and maybe a cute graphic effect somewhere.

For some time I've actually wanted to go track down such programs to analyze/compare/study the progression of design. Kind of difficult to do so though, for obvious reasons (unfortunately).

But my impression from the one or two programs I do very vaguely remember using, half for entertainment purposes was that the UI design was sort of at the "48%-50%" mark - good enough, but flitting just below the halfway mark instead of just above it. Not remarkable; maybe a few rough edges.

Thanks for your comment. It prompted some mental gears to spin and for me to realize the above.


I do think I understand the spirit of what you're getting at, though, and there are a few things that my brain has decided is relevant, although I can't describe the working-out process that led me to list these. ¯\_(ツ)_/¯

- SymbOS (https://en.wikipedia.org/wiki/SymbOS): preemptive multitasking, windowed GUI, storage on devices up to 128GB, runs on home computers using the Z80 (starting at 4MHz) with up to (_up to_) 1MB of RAM. (I found this rambling but very interesting demo of what it can do with one of the more capable machines it supports: https://www.youtube.com/watch?v=2-oBNh0UkQc)

- kOS (https://www.youtube.com/watch?v=kTrOg19gzP4): limited-access OS and brutalist GUI using the K functional programming language (hmm, I should watch this video again)

- KolibriOS/MenuetOS: you're likely already familiar with this; it's a pure-assembly-language GUI that runs on 32/64-bit x86 PCs

- AtheOS: Amiga/C64/etc inspired kernel + GUI + application suite, written (for x86) by one person over circa 10 years before it became boring; a small team picked it up a few years later and branded their continuation of the OS as Syllable, although the Syllable website seems to have died at some point

- Contiki: runs a GUI on a Commodore 64 within 30KB of RAM

- There is of course TempleOS, FWIW

> why it never really took off.

It seems since 2006 the world has gone exactly in the opposite direction: simpler UIs based on windows that fill the screen, one at a time or, in some cases, docked. The idea of having multiple windows open on the same screen space was fancy but often confusing for a lot of people. I have relatives who have had their first approach with computers at the age of 80 and can use quite proficiently an Android phone; others that are still struggling, after twenty years, with the concept of "windows (partially) hidden behind other windows" :).

Relevant quote (in the first 20 seconds): https://www.youtube.com/watch?v=Ep_kaY4K5a8

I'm a bit older and remember this and much more getting demo'd at Xerox Parc in the mid 80ies, when powerful graphic cards were bring developed (for VR) being used for the 3D window managers being developed at Parc (WIMP GUI) in the 70ies. They were definitely not gimmicks but functional for certain use cases, eg. keyboard less interaction, information overload, voice control and such. I remember 6 different window manager styles there, where only 2 made it into production 30 years later, compiz and looking glass.

tldr: The revolution happened at Parc in the 70ies.

there is a great article by Jonathan Schwartz - ex CEO of sun about Looking Glas, that Steve Jobs threatened to sue him over this.


Steve didn't, but Jonathan killed the project non the less, because he listend to consultants instead of listenting to devs.

Zoowm did some more recent push on this space. Personally felt that the idea of inifinite desktop really made sense. Similarly as in physical world, when you need more space on the table, you spread your stuff on a broader area.

Would have liked to see this thing fly.

[1] https://www.youtube.com/watch?v=Gcrp54A-GX8

tl;dr: An integrated workspace and window manager, emphasizing zooming and panning.

Might be helpful in current, resolution-limited, VR.

It is so satisfying to look at. They got animations exactly right. I'm not sure I want full 3D applications (or notes on the back of windows), but the hint of 3D space for window management seems just right.

This is not difficult to do with today's technology. Maybe a similar WM will come along again.

For anyone interested, here's a Live CD: https://sourceforge.net/projects/lg3d-livecd/files/

Whoa this is cool, I wonder if this is the inspiration for Looking Glass technology featured in Prey.


I loved the demo, but I think that 3D interfaces very rarely bring any value to your brain. After all your visual system is built around the idea of turning n-dimensional noise into a non 3d abstraction.

I agree that the looking glass style 3D is fun and gimicky and hard to see added benefit.

At the same time, speaking from personal experience - 3D in VR can give a much stronger sense of overview of many datasources etc. while at the same time can reduce semantic noise.

I get quite a few productive hours out of a successor to this: https://www.youtube.com/watch?time_continue=1&v=Js7Y1H5D8cY

LOL Remember the 3d UI from jurassic park? "Oh its a linux system!" And then she proceeds to use a 3d RTS-style interface to manage the park :)

I think i disliked it then because it was written in Java, and if i remember it performed poorly on most machines compared to Compiz Fusion.

>I wonder why it never really took off

Because it's useless garbage that nobody needs and that doesn't fix any problems.

I sent away for the demo cd. it's probably still in a stack someplace. very sad this didn't take off more.

Seems more gimmicky than useful. I don't need to play Morrowind just to interact with my filesystem.

Thank you for this. I don't have anything insightful to add, but it sure brought back memories.

I wish something would have came from Sun's Project Wonderland.

So that's where OS X got its 3D app dock from...

The app dock was included with the first releases of OS X; the 3D effect isn't really a 3D effect (they just replaced the backdrop image in horizontal modes).

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact