| This article serves two purposes. It is a discussion | | | | capitalize on what 3D has to offer. The precise |
| about the future of computer interfaces; and it is a | | | | nature of how and where 3D can best be |
| means by which I can purge myself of thoughts that | | | | incorporated is an open question, and a framework |
| have been accumulating on this topic for quite a few | | | | to evaluate these questions seems appropriate. As a |
| years. Even if it fails as intelligent discourse on the | | | | rough starting point it seems reasonable to divide the |
| first, it will have succeeded in the second. Previously | | | | attempts into two broad categories: those that are |
| the title was Where Are Computer Interfaces Going? | | | | trying to simulate the physical world and those that |
| but after writing it I noticed a significant number of | | | | prefer more abstract representations. If you'll indulge |
| predictive passages and decided to be bold and | | | | me, I'd like to call these two approaches, |
| move the "are". Of course now I feel obliged to add | | | | respectively, the "Physical Simulation Approach" (PSA) |
| a disclaimer. I admit right here, or at least in the next | | | | and the "Abstract Representation Approach" |
| sentence, that I don't know where computer | | | | (ARA).Developers in the PSA camp are taking |
| interfaces are going. I don't know.With that out of | | | | physical simulations and hanging applications, web |
| the way, I'd like to start, as many interfaces do, with | | | | sites, movies, and pictures on simulated walls. |
| the metaphor. In the 80s and 90s successful | | | | Simulated desks have functional simulated calculators |
| interface design and an appropriate metaphor were | | | | on them. And, perhaps, there is a simulated sun |
| taken to be nearly synonymous. Although a good | | | | outside. It's all very familiar and comes with a nice |
| metaphor is important, it imposes unnecessary and | | | | minimal learning curve.The ARA camp are working on |
| artificial restrictions. So why is it so important? The | | | | strange visualization techniques to view complexity |
| best, perhaps only, reason is familiarity. Unfortunately, | | | | and patterns in large amounts of data. They have |
| familiarity comes at a cost: the shorter learning curve | | | | general graphs floating around in space with links |
| can require speed and ability to be sacrificed.Consider | | | | joining concepts and words together in arbitrary |
| the ubiquitous desktop metaphor. What is more | | | | ways. They have nifty algorithms that filter the |
| powerful, the abstract construct of a tree, or a single | | | | salient characteristics of large data sets so you don't |
| flat surface to place your papers on? Well, a tree is. | | | | get overwhelmed. Their attempts are, by far, much |
| In fact it is so much more powerful that it is the | | | | harder to describe with these mere words.In practice |
| cornerstone of all modern file systems. Trees are | | | | many attempts will combine aspects of both |
| great, they impose an organizational order that is | | | | philosophies. I suspect that successful attempts at a |
| common in natural systems. General graphs are, | | | | 3D interface will have to balance these two |
| perhaps, too general. DAGs (Directed Acyclic Graphs) | | | | extremes in appropriate ways. Objects in a functional |
| are a good contender; largely because of their | | | | 3D interface should probably be represented with |
| acyclicness, but also because they extend trees in a | | | | models that are familiar, just like the icons on your |
| well defined way. I suspect that trees are so useful | | | | desktop are often imitations of familiar real-world |
| because we can't move backwards in time. Species | | | | objects. This is a PSA property. On the other hand, |
| speciate, languages extend, and software bloats. To | | | | tree-based organizational systems would be well |
| fight these is to fight the increasing entropy of the | | | | advised. Very much an ARA concept.Text should |
| universe.Would it be a good idea not to allow folders | | | | always be view-plane aligned, as should images. This |
| within folders within folders just because it would be | | | | is one of those 2D features mentioned earlier. |
| physically cumbersome, and at some point impossible? | | | | Images and text may be scaled, but they should not |
| Probably not. Do icons have a real-world counterpart? | | | | present themselves at an angle. Vertical and |
| Not really. Metaphors should be, and have been, | | | | horizontal edges need to remain vertical and |
| taken only so far.So what does the future hold? Will | | | | horizontal. Of course, these features are trivially |
| interfaces be 3D? Will we be stuck with rectangles | | | | present with your desktop interface as well.And |
| forever? I think it's reasonable to say both have their | | | | there's an important lesson: build on the backs of |
| place. People on the 3D side think that we humans | | | | giants. The desktop UI is successful for a reason, not |
| see, work, live, and play in 3D. We don't. They say | | | | simply because it has a familiar analogue in the |
| they can't wait until there are fully 3D monitors that | | | | physical world, but rather because it behaves in that |
| you can walk around. Why? Our retinas, as well as | | | | same useful way that real desks behave. It takes |
| birds whose eyes are plastered on the side of their | | | | advantage of a well-established ability; spatial |
| heads, are two-dimensional surfaces. Birds have | | | | memory. You put something down and it stays |
| flatter vision than we do, if not as Euclidean, because | | | | there.Useable interfaces need a certain amount of |
| they don't have the benefit of the tiny bit of 3D | | | | persistence in their structure. Having objects stay |
| depth perception a predator gets by overlapping | | | | where you leave them is one good way to achieve |
| images. I've heard graphics programmers explain that | | | | persistence. Placing objects manually, whether on |
| their 3D scene was being projected onto a flat 2D | | | | your desktop or in a 3D environment, takes |
| screen and so it was no longer really 3D. But consider | | | | advantage of spatial memory. We can remember, in |
| this: everything you see in this world is like that. It all | | | | context, where we've left hundreds of objects |
| gets projected onto our flat retinas. We just have | | | | (notwithstanding car keys; they get moved around |
| really big brains. A 3D scene is constructed in our | | | | too much). You probably know where your camera is |
| mind regardless of whether what we're viewing is on | | | | and where the light switches in your home are. By |
| a flat computer monitor or in that nether-world | | | | positioning objects manually you can give them some |
| known as real life. In fact, most brains do a decent | | | | context; perhaps by placing pictures of your family to |
| job of scene construction even with one eye closed. | | | | the left, and panoramic vistas to the right. Contextual |
| From 2D to 3D. Impressive!People on the 2D side | | | | clues help you remember.I've heard the assertion |
| think that we humans see, work, live, and play in 2D. | | | | that adding a single extra dimension doesn't buy you |
| We do, after all, have flat retinas, like playing tennis | | | | much organizational power and that the added |
| on flat tennis courts, and eat dinners from flat plates | | | | navigational complexity isn't worth it. Others think |
| on flat tables. But we don't live in 2D. Our brains are | | | | that we need an n-dimensional space to do a good |
| really big. 1.3 litres big. More than enough dendrites, | | | | job. Aside from the obvious observation that we |
| axons and other brain-things to contain a nice 3D | | | | seem to exist in a macroscopically three-dimensional |
| representation of the world we live in. Clues to build | | | | world (macroscopically was added just to keep any |
| the scene abound: motion, foreshortening, and the | | | | physicists-who-may-know-better reading) and are |
| aforementioned depth perception.The truth is some | | | | therefore good at 3D manipulations, there is evidence |
| things are better in 2D and some 3D. Writing a letter? | | | | that the jump from 2 to 3 dimensions is of a more |
| Use a desk. Put a flat piece of paper on it. Want to | | | | fundamental significance. If you draw a bunch of dots |
| file that letter away? Wouldn't it be cool if you could | | | | on a piece of paper you will not be able to draw lines |
| just let it hover in some large 3D organizational | | | | joining the dots in all possible configurations unless the |
| space? Here's what I think. | | | | lines cross (given some sufficiently large number of |
| Text: 2D | | | | dots. I think 5 might do it). However, once you hit |
| Reading and Writing: 2D | | | | three dimensions, all configurations are possible |
| Organizing and Grouping: 3D | | | | without crossings. Adding a fourth or fifth doesn't |
| Visualization of Data: Depends | | | | have any further beneficial effect. Admittedly there is |
| It has occurred to me that 2D representations | | | | some hand-waving going on here; but the result has |
| should be considered a feature of an interface. It's | | | | implications for some possible interface designs; and it |
| beneficial that text documents are lined up nicely for | | | | points to using three dimensions.So why haven't |
| you in a window. If head-or-eye-tracking hardware | | | | interfaces changed much in the last 20 years? One |
| were more widespread, we'd have software that | | | | possibility is that the desktop is in some way an |
| could compensate for (single) users who are not | | | | optimal representation. More likely, however, is that it |
| directly in front of their screens. Imagine looking at | | | | is simply a functional representation; no need to |
| your monitor from an angle but still having the text | | | | change when change takes effort, right? We expect |
| of this article appear flat. That would be a pretty | | | | to be able to sit down in front of a new interface |
| neat feature (on the other hand, it might just look | | | | and immediately be as productive as we were |
| strange and make you sick; hard to tell without trying | | | | before. We have all learned to use the desktop and |
| it).Because the input is essentially 2D, I predict pure | | | | menu-driven interfaces because we haven't had a |
| 3D imaging devices will prove to be a novelty even if | | | | choice. It has taken time; just as learning to read and |
| the enormous bandwidth problems can be solved. A | | | | write took years when we were younger. Even the |
| graphics card that draws a 480x480 pixel scene at | | | | keyboard and the mouse, although perhaps easier |
| 60fps would take 8 seconds to update a | | | | than writing, have taken time and effort to master. |
| 480x480x480 cube. Yes, I understand this is a vast | | | | New interfaces will face the same hurdles. Their |
| simplification. Somehow restricting rendering to the | | | | designs will need tweaking to reduce the learning |
| surfaces of an object might help, but it sounds tricky. | | | | curve as much as possible. The users of these new |
| Regardless, the same or better effect will be | | | | interfaces will need the patience to develop efficient |
| achievable by feeding a couple of 2D images to each | | | | usage patterns; and the interfaces themselves will |
| eye. Technology that takes this approach will be | | | | need to be entertaining enough to mitigate the |
| more successful. Devices that project images directly | | | | patience required. All these efforts will yield interfaces |
| onto the retina seem like a reasonable approach; | | | | that are not only more enjoyable, but faster and |
| along with any tracking technology that may go with | | | | more useful.Tristan Grimmer is technical director at |
| them.The next 10 years will be a transitional phase | | | | Upper Bounds Interactive Inc. Previously a |
| for interface design. 3D rendering technologies | | | | video-game programmer, and much earlier than that |
| already have a stable home in the entertainment, | | | | a young boy trying to get his Vic-20 to accurately |
| video game, simulation, and design sectors. Although | | | | compute Pi, Tristan now spends his time working on |
| 2D interfaces have dominated everything else, I | | | | Tactile 3D in an attempt to rid the world of |
| expect we will start seeing more 3D incursions. | | | | rectangles. |
| Operating systems and applications are beginning to | | | | |