- You can use augmented reality or VR with something like Oculus to display things in 3D space.
- You can use Motion Capture technologies, plus maybe some convenient remote-control/keyboard-like devices to put a human in that virtual space and manipulate things.
Only 2 technologies(Oculus + Mocap) would be enough to accomplish something like this:
Give a bunch of people oculus rifts and put them in the same virtual reality - and we have the dynamic stage he is talking about.
They don't even have to be in the same room of course, we have 3D realtime MMORPGs and second-life that already solved that problem!! You could attend a virtual talk by sitting in your apartment with oculus rift on your head.
And of course "downloading a room" instead of a book is already very doable. We already have computer games and engines. That room is like a video game level. Downloading it is not a problem, to explore it all you need is oculus and a simple game controller or a keyboard/mouse.
And in terms of creating such content, the closest thing I can think of is 3D editors, like Maya or Houdini, especially houdini(http://www.sidefx.com), which is the brilliant combination of computer graphics and programming, that allows you to model and program dynamic things visually.
That screens are problematic asserts a moral imperative for programmers to avoid writing applications that are designed to be experiential cul-de-sacs. This, in turn, goes very much against the grain of most technologists and the people who fund them, who are all looking to make the most potent screens they can to pull people out of real life as much as possible. This makes great economic sense, but it is unethical.
Most of us who are so fundamentally tied to a screen for so many years at a time would do well to systematically disengage and interact with the real-world as much as possible. What I wouldn't give at this point in my life to put on a hard-hat and help build a bridge!
Great talk overall, a lot of interesting insights!
Changing the analogy to a fish in a tank suggests an assertion that these other intellectual modes are not important for human experience and wouldn't make communication more effective and efficient.
To tweak the analogy further: Otters are very good at swimming, but their habitats should include land.
Personally I don't think his usage of the term "[in]humane" to describe human media is appropriate. Who is responsible for this inhumanity? Our entire race is collectively treating itself inhumanely by not inventing entirely new forms of media and expression? That seems like a stretch. Are we inhumane to dogs because we haven't yet helped them discover their fullest and richest capabilities as a species, whatever those may be? I love Bret and his vision and talks, but I feel like this particular usage is heavy-handed shame rhetoric (I get that he is trying to light a fire under people, but it still seems forced).
some critical and important experiences and behaviors are excluded.
What defines experiences as critical and important, such that to exclude them would be inhumane? Is it inhumane that I haven't traveled on a spaceship to Mars? That seems like a pretty important experience. But of course that's ridiculous, because you can't include the infinite landscape of experiences that might be possible. Personally I don't think either analogy (dog or fish) is particularly apt, but I think the fish one is better because a) our generally known and accepted capabilities are not being forcibly restricted by another agent, and b) the stuff Bret is talking about is entirely new and not possible within our natural environment (i.e. there are many natural and accessible ways to write symbols on a surface, but nothing like that for arbitrary-dynamic-physical-hologram-things).
A) Our capabilities are being restricted by the circumstances we find ourselves in. A dog happens to be in a cage built hundreds of years ago. There must be some words to describe this situation, I think inhumane applies.
B) Now we can start opening the cage.
Edit: our natural environment is inhumane and awful. Your commentary seems to heavily favor the status quo in a way I can't relate to.
The additional inhumanity of media is from 2 things: the fact that information processing has been mostly limited, and that information processing has become really important lately, so a lot of people are spending a lot of their time in that limited environment.
I'm actually quite interested in this spatial learning thing. I had an aboriginal acquaintance who explained to me how his culture would attach meaning to places and even physical orientation (North South East West etc). However, it has been so far very very difficult for me to wrap my head around that. Visual notation imo, has been the most efficient way of communicating between humans. It also helps a lot in understanding.
A lot of what he says harks back to a 1980s/1990s/early 2000s style of teaching with an increasing use of "aids". Using props to teach math is a bit silly (well, asides from geometry, which is inherently spatial). I really cannot see how we can represent the learning of linear algebra in a dynamic spatial manner.
How for example, would one represent higher dimensional spaces spatially, when you really are limited to only 3 dimensions in real life? I am aware that Iron Man style projection tech may actually help in visualizing it, but again, that's a visual aspect of learning, not spatial. The most efficient way of doing so unfortunately is still in algebraic notation.
Perhaps I'm doubtful because the use of props, spaces, songs, moving your body etc has never worked for me in school. Ever. Can anyone share any studies done on the use of these cheesy 1980s-1990s educational aids (the closest analogue to all the things Brett mentioned) in aiding learning?
EDIT: If you're interested, there's been a good book that I read, in attempt to wrap my head around the concept (however to no avail) - Wisdom Sits in Places.
I had a Spanish teacher and later a Latin teacher (teaching adults) who favored TPR and would tell people to jump a specified number of times, or to touch their heads, and so on. I don't have a clear opinion about the effectiveness of TPR but I thought I'd mention the name so you could use it to search for research if you like.
There is also a century-old approach to using colors to visualize 4D geometry (I think the color represents the offset in the 4th spatial dimension), proposed by Charles H. Hinton; they're called "Hinton cubes". Martin Gardner describes them and reports that some people have found them too effective, I guess becoming mildly obsessed with them or having the learned associations interfere somehow with day-to-day visual perception.
There are some pretty big limitations on how far in algebra you can go with that. I do think it created a solid foundation for my mental model as I transitioned to only using symbolic equations.
look to the best line, and i'd argue one of the most important ideas to ingestively portray to a mainstream population, in 2008 cinema, also an argument could be made to increase that reach.. perhaps in the last decade, century, however hyperbolically you care to extend the reach.. from the film burn after reading(i):
CIA Superior: What did we learn, Palmer?
CIA Officer: I don't know, sir.
CIA Superior: I don't fuckin' know either. I guess we learned not to do it again.
CIA Officer: Yes, sir.
CIA Superior: I'm fucked if I know what we did.
when quantum mechanics was young there was a sudden rush of interest in quantum understanding and one of the modes of expressing improvements in that understanding was to create models(ii) sometimes completely replacing old models, if i remember victor even says bohrs name in relation to this concept
this i think is the sorts of things victor is alluding to, stead seeing it as learning think of it as new perspectives to improve understanding.. interestingly though one would have to learn the concepts, both current and avant, in order to know when a new expressive medium improves on the old
so knowledge accumulation is a part of this sort of environment, but arguable only subordinate to the true goals
the shot of the stick figure pushing on a bond between elements in a molecular model i feel is the best representation of this idea
i'd love to have a room like that:
a central tool with supporting data surrounding it
the central tool is imbued with evidencial data w
when i manipulate the tool new data is expressed in different ways around me
i apply my changes physically
the the data maps all changing in sync
in the picture the stick figure is applying force on that bond, perhaps the idea the figure is testing is what if i bend this bond or strain it, what then? or if i move this bond over here does this better represent our understanding of molecules?
my issue with the talk was two fold, i thought victor tended to talk down efforts others have already made or are making to do something with similar goals but in a wholly different means, and victor also failed to mention the number of people already doing the very things he is suggesting,
it came off as unnecessarily dramatic and i found those were the times when he really lost me to wandering thoughts
and also i found it interesting that victor seemed to suggest this new dynamic reality would manifest without the use of current forms of creation
things like, ~programming of the future will be different than programming of today~ which i both agree and disagree with, i think both the current and future ways to program will be in tandem and it will be more of a matter of who touches what aspects of it,
i think like hardware design and manufacture programming will become a specialised field
look at compilers, that is people really interested in low level bit moving granting people with high level interests.. in the programming sense stead some cogitive nobile sense.. access to their expertise from within the comforts of the higher level
the same i believe will be true but that gap will likely grow as it has with hardware manufacture
there will always be a need and opportunity for tinkers to challenge the manufacture to keep it fresh, like the hardware scene(iii), but realistically this shift in software will most likely have the same dramatic effect on quality of life as the hardware model has
1. Walking around a gallery and waving your arms is not enough exercise
2. Incremental progress is giving us VR through Tactical Haptics, Oculus Rift, etc. He says his thing is not VR but R or dynamic R. The distinction is unclear, sounds like marketing.
I also had the idea of talking about systems, particularly political systems through models and simulations. Having spent too much time talking about politics, economics, and computer holy wars in text, it feels like a few interactive models could have saved all that time for millions of people.
That said, simulations are widely used to communicate about financial topics, mainly spreadsheets. And the effects of using spreadsheets are pretty interesting https://medium.com/backchannel/a-spreadsheet-way-of-knowledg...
People start thinking in spreadsheets. A lot of the weird merger acquisitions since the 1980's were basically caused by them. Economists already use fancier models. And as with spreadsheets, garbage in, garbage out. Still I feel they'd be useful in casual conversation.
2. He's talking beyond VR. He never says VR isn't a stepping stone. But he does make a good point about it being dehumanising.
One person who has completely gone overboard with that is Jacob Lurie, he has practically single handedly written down a foundational work of higher algebra and algebraic topology, which essentially precisely corresponds to what you are asking. How does mathematics look like if we weren't constrained to 2d paper to write down equations. Of course to actually write down those things you have to somehow project the equations to several essentially 1d sections or invoke very abstract methods to reason about them. Of course algebraic topology already has tools for that.
The early impetus came from Grothendieck I believe, who began to create a theory of "higher groupoids" in several thousand pages, ordinary groupoids being categories in which every morphism is an isomorphism.
On TV in the Netherlands we have a program that aims to educate the public (state sponsored tv), every couple of months there is a new episode, they are listed here (but they are in Dutch): http://dewerelddraaitdoor.vara.nl/DWDD-University.2772.0.htm...
So far they were about the big bang, the quantum world, Einstein (All from Robbert Dijkgraaf, now director of the Institute for Advanced Study in Princeton), The cow and silicon valley. They were always set in a large theater (with live public) with a stage full of props (Inflating balloons to represent the inflating universe, buckets of sand to represent the amount of stars in a galaxy, cameras on rails to represent relativity, living cows as well as all their nice parts of meat on display.) These "presentations" are very nice to watch and my wife who is not a scientist always enjoys them as well.
There are also museums like Nemo in Amsterdam (https://www.e-nemo.nl/nl/ontdek/brein/) and Naturalis in Leiden (http://www.naturalis.nl/en/), these places are filled with rooms with stuff to touch and experiments to perform. I'm sure the US is laced with those as well so it is not that ground-breaking of an idea.
A holo-deck would be the killer feature for this way of communicating.
Programmers can't easily streamline like this. Maybe there are multiple forces locking us in: the keyboard, text editor, and programming language. Sure, maybe the optimal way to write code is with a string of ascii characters arranged left to right top to bottom but I doubt it. So even in the visual-symbolic space there is unexplored potential.
I'm glad someone is out there tackling these assumptions and much more, I should keep an eye on his work.
On an unrelated note, his website is entertaining: http://worrydream.com/
I don't think Bret Victor's work is really about design (most of the UX designers I know have never heard of him), it is about invention and is a bit more on the programmer side. Putting him in the designer bucket is too inaccurate (but buckets are always inaccurate).
I saw this talk in person at SPLASH; the puppy in cage for life really pulls at some heart strings. But it didn't really do it for me like the previous talks did (though he didn't have any prototypes to show off as with the previous talks).
I discovered his portfolio a couple years ago through:
and since then it's come up in conversation with UX designers as a cool portfolio 3 or 4 times. But the points you make are valid.
Are you being serious?
Edit: he later dismisses this, but I don't think he's right to do so.
(edit: relevant link, Victor's rant on hands in interaction design. http://worrydream.com/ABriefRantOnTheFutureOfInteractionDesi...)