4 ms·

Wow. Thanks for all of this info and for your very interesting perspective. I have a passing familiarity from HN and elsewhere with some of these new shells, bu

Wow. Thanks for all of this info and for your very interesting perspective. I have a passing familiarity from HN and elsewhere with some of these new shells, but am not well informed enough to really comment on their usefulness/etc. With that said, you've given an excellent primer in your comment -- thanks very much! A few things you said stuck out to me: > For me, the great thing about a shell language is that it's a programming language I get to _live_ in. > The idea with these new shells is to try to find a way to enhance the programming capabilities of shells without making them any less convenient for navigating the filesystem and performing simple tasks with external programs. Generally, I agree, and I actually use `eshell` in Emacs for this kind of thing a lot. Being in a pseudo-shell LISP layer allows me to get arbitrarily manipulate text I get back from the command line with elisp and allows for some really flexible workflows. With that said, eshell does of course have many trade offs and limitations, which I won't get into here. I think where these things always fall apart for me though is in the Elvish shell example you provided. To me, that is not functionally better than using Bash and jq, it's just different. Maybe it's better, I'm not sure, but my first impression is that it's neither more concise or more readable, it's just different syntax. I think there is a credible argument to be made for "batteries included" type shells, with something like native json parsing, but at what point does it then tip into being like Powershell or Python, where once again you've gotten away from the native/accessible experience because you need to support `n` kinds of structured data inputs/outputs on `n` platforms? As I was reading your comment, the thought that occurred to me was: Why not just make a python library that can be loaded into the REPL that abstracts over some of the more cumbersome parts of interacting with the OS/filesystem in a shell like way? That seems to be the best of both worlds, and from what I gather that seems like what Rush and Xonsh are doing? I need to look into them further. > Developers and sysadmins want to be able to pull in structured data from whatever source and query it and manipulate it right there inside their shells. But each format gets its own utilities for that kind of querying and manipulation, and they don't necessarily know how to talk to each other. I agree re: structured data manipulation, but I think the solution that has naturally emerged exists for a reason. I can use pipes, and programs like jq to push and pull data into and out of whatever format I need, and each layer of that ecosystem can be maintained in parallel, adapting to changes in the overall landscape. In a sentence, it's the core of the Unix philosophy. One thing and one thing well and all that. I dunno. I've thought a lot about it while typing this comment out. I think it sounds like I'm disagreeing with you, but I'm not. I don't even really think we're debating. I think these new ideas for shells are good things, and I will definitely investigate them more and see if they make my life easier. I just can't seem to shake this feeling that we can't have our cake and eat it too. Maybe I'm looking at it the wrong way though. Maybe in 20 years time shells like Oil and Elvish will be the norm, and we'll be complaining about how they don't handle quantum data structures well without lots of pipes and fd's :D Either way, this has been a very interesting digression. Thanks again for your thoughtful comment and insight.

pxc 5y ago
Glad you found my long comment useful! lol > Generally, I agree, and I actually use `eshell` in Emacs for this kind of thing a lot. Being in a pseudo-shell LISP layer allows me to get arbitrarily manipulate text I get back from the command line with elisp and allows for some really flexible workflows. FWIW, Xiaq at least (initial author and lead developer of Elvish) envisions the terminal of the future as somewhat Emacs-like (this I recall from conversation— maybe you'd enjoy dropping by one of Elvish's online chat thingies and asking what folks think of eshell and Emacs as a model for rich, programmable interactivity with text), and looks to functional programming languages (especially Lisps and Schemes) for elements of Elvish's language design. (For example: its approach to numerical types is based on Scheme's; its arithmetic operators use prefix notation and parentheses just like in any Lisp; and lists and maps are immutable, like in Clojure or most functional programming languages of the ML heritage.) > To me, [using a shell to ingest and manipulate structured data] is not functionally better than using Bash and jq, it's just different. Maybe it's better, I'm not sure, but my first impression is that it's neither more concise or more readable, it's just different syntax. One way that it's better is that since your shell is handling the data, it can offer you tab completion, syntax highlighting, and previews of your manipulation and querying of the data, whereas with jq and bash, your jq query is going to be enclosed in quotation marks (probably single quotes). It's going to contain pipes that don't mean the same thing as your shell's pipes. It's also kind of a pain to pull things out of the ‘middle’ of a jq pipeline, since you have to either write a block of code or split your pipeline into multiple jq invocations and do weird things with tee and maybe open extra fds, idk. That said, jq is still a great tool, and stuff like jiq can be helpful for letting you compose jq pipelines with an interactive experience. > [A]t what point does it then tip into being like Powershell or Python, where once again you've gotten away from the native/accessible experience because you need to support `n` kinds of structured data inputs/outputs on `n` platforms? For my part, I think the way PowerShell handles this is actually pretty nice. But basically what you want is a large library ecosystem that includes wrappers for the external tools you might want to use. In easy cases, this will mostly be adding like a `--json` flag or something. This might sound like a lot, but I think it's basically comparable to the existing way we distribute completions for popular command-line programs. But yes, this does push you to prefer a ‘pure’ approach to some extent, or you end up munging text outputs into structured ones yourself, although for serious scripting cases you still end up doing that kind of output processing even in Bash— you just don't have very nice types to put your data into when you're done. > As I was reading your comment, the thought that occurred to me was: Why not just make a python library that can be loaded into the REPL that abstracts over some of the more cumbersome parts of interacting with the OS/filesystem in a shell like way? That seems to be the best of both worlds, and from what I gather that seems like what Rush and Xonsh are doing? I need to look into them further. My impression is that Xonsh is the most successful of that type of new shell, and that its regular users really love it, so I'd definitely check that one out first, if you like Python at all. Oil also seems aimed at Python fans, being written in Python and having some syntax inspired by it. Maybe those two would be good ones for you to try and compare to get a sense of the two approaches to offering shells with better programmability!

pxc 5y ago

Glad you found my long comment useful! lol > Generally, I agree, and I actually use `eshell` in Emacs for this kind of thing a lot. Being in a pseudo-shell LISP layer allows me to get arbitrarily manipulate text I get back from the command line with elisp and allows for some really flexible workflows. FWIW, Xiaq at least (initial author and lead developer of Elvish) envisions the terminal of the future as somewhat Emacs-like (this I recall from conversation— maybe you'd enjoy dropping by one of Elvish's online chat thingies and asking what folks think of eshell and Emacs as a model for rich, programmable interactivity with text), and looks to functional programming languages (especially Lisps and Schemes) for elements of Elvish's language design. (For example: its approach to numerical types is based on Scheme's; its arithmetic operators use prefix notation and parentheses just like in any Lisp; and lists and maps are immutable, like in Clojure or most functional programming languages of the ML heritage.) > To me, [using a shell to ingest and manipulate structured data] is not functionally better than using Bash and jq, it's just different. Maybe it's better, I'm not sure, but my first impression is that it's neither more concise or more readable, it's just different syntax. One way that it's better is that since your shell is handling the data, it can offer you tab completion, syntax highlighting, and previews of your manipulation and querying of the data, whereas with jq and bash, your jq query is going to be enclosed in quotation marks (probably single quotes). It's going to contain pipes that don't mean the same thing as your shell's pipes. It's also kind of a pain to pull things out of the ‘middle’ of a jq pipeline, since you have to either write a block of code or split your pipeline into multiple jq invocations and do weird things with tee and maybe open extra fds, idk. That said, jq is still a great tool, and stuff like jiq can be helpful for letting you compose jq pipelines with an interactive experience. > [A]t what point does it then tip into being like Powershell or Python, where once again you've gotten away from the native/accessible experience because you need to support `n` kinds of structured data inputs/outputs on `n` platforms? For my part, I think the way PowerShell handles this is actually pretty nice. But basically what you want is a large library ecosystem that includes wrappers for the external tools you might want to use. In easy cases, this will mostly be adding like a `--json` flag or something. This might sound like a lot, but I think it's basically comparable to the existing way we distribute completions for popular command-line programs. But yes, this does push you to prefer a ‘pure’ approach to some extent, or you end up munging text outputs into structured ones yourself, although for serious scripting cases you still end up doing that kind of output processing even in Bash— you just don't have very nice types to put your data into when you're done. > As I was reading your comment, the thought that occurred to me was: Why not just make a python library that can be loaded into the REPL that abstracts over some of the more cumbersome parts of interacting with the OS/filesystem in a shell like way? That seems to be the best of both worlds, and from what I gather that seems like what Rush and Xonsh are doing? I need to look into them further. My impression is that Xonsh is the most successful of that type of new shell, and that its regular users really love it, so I'd definitely check that one out first, if you like Python at all. Oil also seems aimed at Python fans, being written in Python and having some syntax inspired by it. Maybe those two would be good ones for you to try and compare to get a sense of the two approaches to offering shells with better programmability!