5 ms·
Ask HN: What would an IDE built for the Apple Vision Pro look like?
The OS for Apple Vision Pro brings a new design language focusing on depth and utilizing space beyond a square screen.
If IDEs weren't confined to a 2D window, what would it look like? Are there any features you can think of that would make AR coding more productive than simply on a monitor?
- vdievsowgdj 3y agojust look at what IDEs that runs exclusively on ios are like: crap. nobody invest into a closed platform. i expect jetbrains to came up with something marvelous for ar/vr, but it will run on the upcoming version of Microsoft or HP glasses. you know, the only ones today that works just like an external monitor without a locked in ecosystem like apple or facebook. the silly apps and games and such will net millions tho.
- omeze 3y agoThis is flame-baity but honestly what Id also expect. No one is going to invest in developing IDEs for this thing to rival VS Code or whatever if Apple wants a stranglehold on its ecosystem. Maybe there will be extensions to existing editors since people can ise it as a display extension for their macs. We didn’t see this happen on windows with game dev and the quest though.
- tikimcfee 3y agoI mean.. I’ve been doing exactly this for a while, and I fully plan on releasing something when I can actually find the time to superhero it to completion or find help. https://github.com/tikimcfee/LookAtThat https://github.com/tikimcfee/LookAtThat
- omeze 3y agoThis is very cool! I was thinking of a VS code extension that lets me put files or directories “physically” on my desk. Itd be emulating the way that I read text books. Instead of keeping notes and bookmarks physically in the book I could drop sticky notes that represent files directly on my desk and show a preview by by just shifting my eye focus to the AR/VR note.
- tikimcfee 3y agoWow this is an awesome use case I hadn’t thought of and it makes so much darn sense! In fact, the APIs include automatic plane detection, which means being able to look at your table, directly in front of you and “popping a note down“ shouldn’t be too terribly impossible! In fact I can see you doing things like duplicating snippets of code, and putting the scratchpad next to your monitor, your keyboard, or anywhere else you would stick a stickie. What a great idea, I hope you don’t mind if I ruminate on it some more!
- simse 3y agoTotally. Any iOS, or in this case realityOS, IDE would necessarily have to be remote only to actually be useful.
- simse 3y agoedit: visionOS not realityOS
- veidr 3y agoI just want VS Code floating in my 100-room magical memory palace mansion. With tools like Remote Development extensions, VS Code already works superbly from a desktop or laptop, editing locally but using a remote machine (your own or some cloud thing) for the filesystem, git, compilation, dependency management, development server if its a web/network app, etc. It also works pretty great on the iPad, other than the tiny screen (which yeah, is a dealbreaker IMHO, but wouldn't be on a many-screens device like Vision Pro). It also works pretty fabulously in a web browser with that same setup (e.g. Github Codespaces, but you can roll your own, too). I doubt we will see a great IDE that does everything locally on the device. But a great IDE client is eminently doable; it's basically already done. The IDE is one application type where the tooling around offloading all the compute, and just keeping the UI local, has flourished. VS Code was the first to really do it so it was just as good or better as developing 100% locally. JetBrains is right behind them. Others are doing it too. This was the only way iPad OS became usable as development machines. I don't expect there to be any good native IDEs for Vision OS, either. But if you can deal with requiring a network connection, and using VS Code, you will be able to use Vision Pro to develop on a remote Linux machine (or Mac or Windows, if that is your thing). Whether JetBrains will make an IDE client for it, enable third parties to make that, or not support it at all remains to be seen, I guess.
- thesuperbigfrog 3y agoMost programming work is still text editing with various amounts of debugging, refactoring, and autocompletion. It is not obvious to me how an AR interface would make a difference other than more virtual screen real estate. You would still need a way to enter text, build code, run tests, etc. This means a keyboard and pointing device (mouse, trackpad, etc.) are still needed unless something else can do it better. Granted, for the same cost as the Vision Pro you could get several large, high resolution monitors and have lots of screen to work on.
- kevinmershon 3y agoAgreed. So, in a word, Emacs. It's just Emacs buffers everywhere
- kabirgoel 3y ago> Most programming work is still text editing [...] Text no longer needs to be the primary way of conveying programs. There are practical reasons text works best on screens, but if your coding environment is boundless then there’s no reason to believe you can’t do fancier things like direct manipulation of ASTs pretty easily. Imagine "grabbing" an AST node and attaching it to a different parent, all in space. Beyond simple AST manipulation, the Vision Pro will probably enable Dynamicland-esque "programming spaces" where you manipulate objects in your virtual environment to construct programs.
- stoniejohnson 3y agoThis seems like a very literal interpretation of "spatial computing"; I don't think anyone will be physically manipulating ASTs with any regularity.
- klyrs 3y agoWe already have plenty of these, and they suck at doing anything serious.
- jazzyjackson 3y ago
- solomatov 3y ago> would make AR coding more productive than simply on a monitor One obvious thing is much more space. I.e. unlimited number of extra monitors, or entities which work as such.
- jungturk 3y agoThinking about depth: Using it to present stacks of information (version history, undo/redo chain) Using it to render background information that doesn't need to be swapped into the foreground to be useful - the architecture/module that the code you're working in serves, the remote services that fulfill certain commands, the test coverage available to you in this module.
- satvikpendem 3y agoIt would look like a 3D world, like this: https://www.youtube.com/watch?v=z4FGzE4endQ https://www.youtube.com/watch?v=z4FGzE4endQ
- colinplamondon 3y agoI would expect extracting out more of the meta-logic via GPT, and showing relevant panes dynamically. If you get an error, automatically search for the answer and propose the change. If you add a new flow uncovered by tests, propose the test. Generally, have panes that are dynamic to what you are doing, and tightly couple them. I could imagine looking at different zoom levels of a code file, folder, or architecture, and working primarily on abstractions, approving / rejecting the resulting proposed edits. Strategic coding more akin to a game like Supreme Commander or Planetary Annihilation.
- icapybara 3y agoThis sounds kind of frustrating because it would be hard to predict or trigger. “Last time I ran into this error a super helpful window popped up, this time I’m getting nothing?”
- esafak 3y ago"Design for spatial input": https://developer.apple.com/videos/play/wwdc2023/10073/ https://developer.apple.com/videos/play/wwdc2023/10073/
- lurker137 3y agoI think it might make large scale code visualization in a similar way to how SourceTrail does it more feasible: https://github.com/CoatiSoftware/Sourcetrail https://github.com/CoatiSoftware/Sourcetrail
- mindcrash 3y agoI think it will be a wholesome natural interface, so I imagine some form of a LLM will be included which can be used to actually make the interface elements work and connect them to the outside world, while having the possibility to design and modify the interface you are building with your hands/fingers (think Interface Builder, but spatial). If they manage to pull something like this off and be first to market I guess Magic Leap, HoloLens, and whatever Meta is cooking up (if they still are doing something in that space anyway) will very likely be pretty much dead, by the way.
- armchairhacker 3y agoMore open windows. But in all seriousness, you can have a tree of small code windows connected by various dependencies. So it will be much easier to see a piece of code and it’s uses / definitions, and thus understand a codebase’s architecture
- 1propionyl 3y agoGet in the car, we're reinventing HyperCard/Smalltalk!
- womitt 3y agoMake the window active you look at - tried to make it with oculus developer kit didn't work that time, maybe with this hw
- sturza 3y agoI imagine something like https://pharo.org/ https://pharo.org/
- ehutch79 3y agoNeovim, but curved slightly
- dzhiurgis 3y agoGo to definition/usage opens new window but lets you easily toggle between previous windows easily (using eyeballs)
- giancarlostoro 3y agoSounds to me like you want to use the old Delphi IDE where all the editor windows are separate.
- zingerlio 3y agoI think a robust eye-tracking functionality for code editors has a ton of potential for the Vision Pro. A lot of the barriers and benefits of vim/emacs style navigation can be replaced or augmented by smart, interpretive eye tracking. It’s like digging into the future where machine starts to read our minds, and an intimate integration of where you look is a big step.
- tikimcfee 3y agoCan I throw my hat in the ring? I’ve been working on this for.. a long time. AR VR iOS and macOS app for arbitrary code rendering in 3D space. Terminal like rendered glyph by glyph means perfect control over ever mesh and texture. The iOS demo is fun. You can walk around your code like an art museum, draw lines of executed traces, and perform visual hierarchical search. https://github.com/tikimcfee/LookAtThat https://github.com/tikimcfee/LookAtThat
- simse 3y agoLooks very interesting, thanks for sharing this.
- tikimcfee 3y agoYou’re quite welcome, I’m all too glad to share my obsessions, haha. Let me know if you’re interested in a little personal demo of the latest stuff- the Metal based implementation I’ve been cooking up has pretty awesome performance, on the order of millions of glyphs at 60+ FPS. Thanks for asking around and letting me share
- juliusgeo 3y agoWow, this looks very impressive. Going to check it out. Is there any plan to support languages other than Swift? I’m assuming that has quite a few technical hurdles associated.
- tikimcfee 3y agoThank you very much, and absolutely! My hope is to convert to a treesitter implementation instead of the SwiftSyntax direct parser. There’s a file filter in there that can be turned off to load any particular file up for viewing, but the analysis stuff won’t work of course. The tracing is more complicated. SwiftTrace is a bit tangly, and I’m still trying to think of a way for folks to “drop in” something to their code base to produce a structured set of traces. There’s a file-backed array that is is used to read compacted run logs, so as long as I can can get the format right, it should work for most cases. I even experimented with a syntax rewriter to insert logs! What languages do you use? What would you like to see supported?
- jones1618 3y agoI'd want a 3D node editor interface much like Blender's texture or geometry creation interface. The nodes would be 3D objects (cubes, cylinders, etc) and nested groups of objects w/ physical ports that could be wired together. Nodes would include class/object/method nodes w/ code blocks. So, an important AR/VR UX feature would be the ability to collapse/dive-into nodes & groups of nodes much like code-outliners in 2D IDEs. Another awesome feature would be the ability to affix dials/gauges and other displays to the outside of nodes & node groups that would provide indicators of the unit's state: How "full" is a collection node, how often/frequently was this node invoked, the health of the node (errors, exceptions, slow-execution times), etc.
- aaron695 3y ago[dead]
- tetek 3y agoVim but you can quit with hand gesture
- Austin_Conlon 3y agoXcode's view debugger in 3D would be great.
- giantg2 3y agoI've heard about people using Unity for other AR/VR stuff. Not sure if they might expand to include it. I've played around with Unity a little bit. I guess we have to see if Apple is using it's own proprietary stuff (as usual), or if they will be closer to the industry standards, or at least providing support for including their language in existing IDEs.