7 ms·

Holy crap, great idea! Imagine if adding a "monitor" (virtually speaking, of course) to your setup was no more difficult than splitting a Vim window... that wou

Holy crap, great idea! Imagine if adding a "monitor" (virtually speaking, of course) to your setup was no more difficult than splitting a Vim window... that would be pretty rad. An Oculus Rift is much less expensive than your typical multi-monitor setup too, FWIW.

zachrose 12y ago
Finally, a virtual reality so immersive it can simulate a room full of teletypes.
- JoshTriplett 12y ago
  "A mouse is a device used to point at the xterm you want to type in." A VR headset is a device used to look at the xterm you want to type in.
- e12e 12y ago
  Hehe. Well, text will be text... on a side note, those interested in this, might also like: https://github.com/brianpeiris/RiftSketch https://github.com/brianpeiris/RiftSketch
moron4hire 12y ago
Actually, I'm hoping to be able to experiment with code layout ideas that do not strictly adhere to a window. 2D elements tend to not map well to 3D spaces. One of the problems with the text-in-a-texture method that I've done here is that there is no relief of depth to the text, and relief is actually an important part of how we make sense out of objects in 3D space. It's fine on a 2D screen because the context is so restricted that it all stays consistent. But in 3D, it's kind of disconcerting to see completely flat textures. Like, why do we still use plain text in flat files for source code (for the most part, there are a few exceptions-that-prove-the-rule)? Why are there so many examples of graphical programming systems that have completely failed? Is it because there is something inherent to text that makes it superior for representing algorithms? Or is it because 1-dimensional text maps to our IO systems the best? I suspect there is a healthy dose of the latter, but we haven't ever had a significant "other" IO system to ever test that theory. In a lot of ways, Primrose isn't even my goal, more like a benchmark from which to start developing other ideas in text representation in VR UIs. I don't know what they will be, but at least I now have a starting point.
- bct 12y ago
  > Like, why do we still use plain text in flat files for source code Information density. Images and graphs are exceptions to the general rule that text can contain a lot more information on one page than graphics can.
  - e12e 12y ago
    Well... if you look at things like (colour)forth (and highlighting in IDEs/editors) -- most people don't actually code in "plain" text (there's colour/typographic hints that indicates meaning). Other than that -- we do tend to still use text to represent mathematics, so I don't think it's that strange that we usually represent code as text. For another exception to the rule: see graphical editors for UI layout, as seen in many IDEs (ie: using graphical programming for representing graphics) -- or 3d modellers like Blender (when you add bones etc, you could argue that you're "programming"). Then there's of course spread sheets -- but they tend to be pretty text-focused -- even if they use layout to great effect.
- vidarh 12y ago
  > Like, why do we still use plain text in flat files for source code Text transports with full fidelity between print and screen, and between pretty much any system. Anything beyond text becomes a) harder to communicate (e.g. consider how many people would struggle with even reading out loud maths formulas with symbols they are unfamiliar with; now extend that to trying to talk about representations that have extra spatial information), b) requires extra tooling for far more varied environments than you ever thought people actually code in (e.g. I have an idea; I want to write down code in my phones note-taking app) Serializing it for IO is trivial. Representing it in a way that allows us to handle the two problems above in a way that makes it seem superior to text in enough cases is a very, very hard problem. It's definitively intersting, though.
- orbifold 12y ago
  I believe the reason is that graphical (2d) notations are harder to communicate and therefore not standardized. If they are standardized, they are usually very application specific and not really general enough. For example lambda calculus has a nice graphical representation as string diagrams, since it corresponds to calculations in a cartesian closed category. Typed lambda calculus has a similar representation, but with multicolored strings. Pretty much any problem easily modeled as data flow has a nice a nice representation in an appropriate (traced, braided) monoidal category, examples would be electrical circuits, chemical reaction networks (simple atmosphere models, so called "Box models" are an example), hardware control software (LabVIEW), general ODE/Statemachine simulation (SIMULINK), "business processes" (forgot the name of the vendor), not to mention the various video/audio/shader programming environments. Probably the intersection between the people capable of building a "generic" programming language for graphical programming and people who know enough category theory to inform the design is zero, even though software that solves subproblems of this kind is very successful. Ultimately you really want at least a 3d graphical language, because you want to express relations between 2d processes (for example the equation a+b = b+a, should be really understood as such a 3d process). Coincidentally relations between functions is something even programming languages like Haskell fail to address, it has type classes, but no way of specifying the equations that are required to hold. The only language I'm aware of that does that is Magma, which unsurprisingly is based on Operads (essentially multicolored monoidal category with a fixed set of monoidal functors and natural transformations) and was designed by mathematicians and to a lesser extend Mathematica and Prolog.

zachrose 12y ago

Finally, a virtual reality so immersive it can simulate a room full of teletypes.

JoshTriplett 12y ago
"A mouse is a device used to point at the xterm you want to type in." A VR headset is a device used to look at the xterm you want to type in.
e12e 12y ago
Hehe. Well, text will be text... on a side note, those interested in this, might also like: https://github.com/brianpeiris/RiftSketch https://github.com/brianpeiris/RiftSketch

moron4hire 12y ago

Actually, I'm hoping to be able to experiment with code layout ideas that do not strictly adhere to a window. 2D elements tend to not map well to 3D spaces. One of the problems with the text-in-a-texture method that I've done here is that there is no relief of depth to the text, and relief is actually an important part of how we make sense out of objects in 3D space. It's fine on a 2D screen because the context is so restricted that it all stays consistent. But in 3D, it's kind of disconcerting to see completely flat textures. Like, why do we still use plain text in flat files for source code (for the most part, there are a few exceptions-that-prove-the-rule)? Why are there so many examples of graphical programming systems that have completely failed? Is it because there is something inherent to text that makes it superior for representing algorithms? Or is it because 1-dimensional text maps to our IO systems the best? I suspect there is a healthy dose of the latter, but we haven't ever had a significant "other" IO system to ever test that theory. In a lot of ways, Primrose isn't even my goal, more like a benchmark from which to start developing other ideas in text representation in VR UIs. I don't know what they will be, but at least I now have a starting point.

bct 12y ago
> Like, why do we still use plain text in flat files for source code Information density. Images and graphs are exceptions to the general rule that text can contain a lot more information on one page than graphics can.
- e12e 12y ago
  Well... if you look at things like (colour)forth (and highlighting in IDEs/editors) -- most people don't actually code in "plain" text (there's colour/typographic hints that indicates meaning). Other than that -- we do tend to still use text to represent mathematics, so I don't think it's that strange that we usually represent code as text. For another exception to the rule: see graphical editors for UI layout, as seen in many IDEs (ie: using graphical programming for representing graphics) -- or 3d modellers like Blender (when you add bones etc, you could argue that you're "programming"). Then there's of course spread sheets -- but they tend to be pretty text-focused -- even if they use layout to great effect.
vidarh 12y ago
> Like, why do we still use plain text in flat files for source code Text transports with full fidelity between print and screen, and between pretty much any system. Anything beyond text becomes a) harder to communicate (e.g. consider how many people would struggle with even reading out loud maths formulas with symbols they are unfamiliar with; now extend that to trying to talk about representations that have extra spatial information), b) requires extra tooling for far more varied environments than you ever thought people actually code in (e.g. I have an idea; I want to write down code in my phones note-taking app) Serializing it for IO is trivial. Representing it in a way that allows us to handle the two problems above in a way that makes it seem superior to text in enough cases is a very, very hard problem. It's definitively intersting, though.
orbifold 12y ago
I believe the reason is that graphical (2d) notations are harder to communicate and therefore not standardized. If they are standardized, they are usually very application specific and not really general enough. For example lambda calculus has a nice graphical representation as string diagrams, since it corresponds to calculations in a cartesian closed category. Typed lambda calculus has a similar representation, but with multicolored strings. Pretty much any problem easily modeled as data flow has a nice a nice representation in an appropriate (traced, braided) monoidal category, examples would be electrical circuits, chemical reaction networks (simple atmosphere models, so called "Box models" are an example), hardware control software (LabVIEW), general ODE/Statemachine simulation (SIMULINK), "business processes" (forgot the name of the vendor), not to mention the various video/audio/shader programming environments. Probably the intersection between the people capable of building a "generic" programming language for graphical programming and people who know enough category theory to inform the design is zero, even though software that solves subproblems of this kind is very successful. Ultimately you really want at least a 3d graphical language, because you want to express relations between 2d processes (for example the equation a+b = b+a, should be really understood as such a 3d process). Coincidentally relations between functions is something even programming languages like Haskell fail to address, it has type classes, but no way of specifying the equations that are required to hold. The only language I'm aware of that does that is Magma, which unsurprisingly is based on Operads (essentially multicolored monoidal category with a fixed set of monoidal functors and natural transformations) and was designed by mathematicians and to a lesser extend Mathematica and Prolog.