21 ms·

Cicada – Unix shell written in Rust

244 points by mitnk 9y ago

cies 9y ago
There's also this project: https://github.com/redox-os/ion https://github.com/redox-os/ion It's intially build with RedoxOS (in Rust as well) underneath, but now also runs on Linux. Seems to be a bit more mature. I wonder what sets these two projects apart, and/or if the devs of Casada know of Ion's existence.
- floatboth 9y ago
  Cicada seems to be more POSIX-y, while Ion "it is not, nor will it ever be, compliant with POSIX". What they have in common though… is that job control (Ctrl+Z) is not implemented yet :(
  - eridius 9y ago
    I've been giving thought lately to writing my own shell, because there's stuff I want to fix that no current shell does right, but job control is one of the big reasons why I haven't actually started writing code yet (namely, I haven't yet figured out how to reconcile the stuff I want to do with the need to let the user suspend or background tasks including shell scripts).
rocky1138 9y ago
Looks fully featured! Cool stuff. I think people are going to ask "Why?" and "What's the difference between this and bash?" Might want to cover that in your README.
- withjive 9y ago
  The fact that there is a "Won't do list" stating that functions won't be supported... I don't think this is anything more than a toy to play with rust.
  - Brian_K_White 9y ago
    This.
  - valarauca1 9y ago
    Generally speaking anytime you start to breakout bash/zsh/sh/csh functions you've already entered a zone where just writing a python/perl script would be the easier and a more maintainable solution in the long term.
    - gkya 9y ago
      I have a dozen of functions in my bashrc that are basically a bit more sophisticated aliases. Why would I want to load a whole interpreter to run them while bash can easily do that?
      - frou_dh 9y ago
        > I have a dozen of functions in my bashrc that are basically a bit more sophisticated aliases. Me too. Looking at them, they all start with something like test $# -eq 2 || return ...since in sh/bash you cannot even name your function's arguments or indicate how many you expect to receive. Speaking of primitiveness, in strict POSIX sh, there's no such thing as local variables in functions!!
    - falcolas 9y ago
      Not if I want to actually affect the current shell environment. For example: export MARKPATH=$HOME/.marks function jump { cd -P $MARKPATH/$1 2> /dev/null || (echo "No such mark: $1" && marks) } function mark { mkdir -p $MARKPATH; ln -s $(pwd) $MARKPATH/$1 } function unmark { rm -i $MARKPATH/$1 } function marks { ls -l $MARKPATH | sed 's/ / /g' | cut -d' ' -f9- && echo } _jump() { local cur=${COMP_WORDS[COMP_CWORD]} COMPREPLY=( $(compgen -W "$( ls $MARKPATH )" -- $cur) ) } complete -F _jump jump This would be much harder to do in a script.
    - eikenberry 9y ago
      Shell functions are useful for much more than scripting. They are like aliases++, letting you add small commands to your shell where aliases don't quite cut it. They let you handle arguments, export environment variables, use conditionals, etc. I have 100 or so that I've added over the years. An interactive shell without functions would not cut it.
brandur 9y ago
Amazing work! Between this, Alacritty [1], and coreutils [2], we're getting pretty close to a plausible all-Rust CLI stack. While Cicada is pretty clearly modeled on the "old" generation of shells (sh, Bash, Zsh, etc.), one has to wonder what a more modern shell might look to address some of the problems of its predecessors like pipes that are essentially text-only, poor composability (see `cut` or `xargs`), and terribly obscure syntax. PowerShell seems to have been a flop outside of the Windows ecosystem, and between overly elaborate syntax and COM baggage, was probably rightfully passed over, but it probably had the right idea. Imagine a future where we're piping structured objects between programs, scripting in a modern language, and have IntelliSense quality auto-completion for every command. --- [1] https://github.com/jwilm/alacritty https://github.com/jwilm/alacritty [2] https://github.com/uutils/coreutils https://github.com/uutils/coreutils
- zdkl 9y ago
  My structured objects aren't your structured objects. In that use case why not use a dedicated program (like say ae python/js/clojure interpreter) to handle more complex pipes and keep the shell level primitves... Well, primitive. Text is compatible with all systems, past and present, and can be used to model more complex objects. Let's not add features to the core on a "why not" please.
  - vog 9y ago
    The so-called "plain text formats" are also just representations of complex objects. Often these formats are ad-hoc and hard to parse correctly. Multiple text files in a file system hierarchy (e.g. /etc) are also just nested structures. So in principle, treating those as complex object structures is the right way to go. Also, I believe this is the idea behind D-Bus and similar modern Unix developments. However, what's hard is to provide good tooling with a simple syntax that is simple to understand: * Windows Registry and Power Shell show how not to do it. * The XML toolchain demonstrate that any unnecessary complexity in the meta-structure will haunt you through every bit of the toolchain (querying, validation/schema, etc.). * "jq" goes somewhat into the right direction for JSON files, but still hasn't found wide adoption. This appears to be a really hard design issue. Also, while deep hierarchies are easier to process by scripts, a human overview is mostly achieved by advances searching and tags rather than hierarchies. A shell needs to accomodate for both, but maybe we really just need better command line tools for simple hierarchial processing of arbitrary text file formats (ad-hoc configs, CSV, INI, JSON, YAML, XML, etc.).
Animats 9y ago
Note: Rust environment is needed for installation. Why?
- foodmart 9y ago
  A rust installation is needed due to you having to compile it from source manually. If binaries were to be distributed, a rust installation would not be needed.
  - mitnk 9y ago
    Yes, I haven't provided the pre-built binaries yet. Maybe I should..
    - IceyEC 9y ago
      Check out snapcraft, it can build rust binaries!
oconnor0 9y ago
Oh, bummer, I was really hoping it would support Windows.
- mitnk 9y ago
  You need xonsh - http://xon.sh/ http://xon.sh/ :)
city41 9y ago
Being able to do arithmetic right in the shell is handy.
- icebraining 9y ago
  $ echo $((1 + 2 * 3 - 4)) 3
  - vbernat 9y ago
    With zsh, you can even save a few keystrokes. function c() { echo $(($@)) } alias c='noglob c'
    - jplatte 9y ago
      zsh can handle floating point arithmetic too, while bash can't.
- tyingq 9y ago
  They way they chose to do it though seems to create some ambiguity. Like, what does this do? I can think of 3 distinctly different things it could do. $ echo test 1 + 2>err Or what if there are files with these names? $ cp /bin/grep 42 $ 42 + 1
Klasiaster 9y ago
Escaping and passing strings as arguments (without losing e.g. " characters) is often a problem. It would be so nice if this would be solved in a shell the way it works for function calls in propper programming languages like Python or Rust (i.e. various ways to specify strings with special characters and a good format method for string composition) with Haskell-like function call syntax.
- IshKebab 9y ago
  And lists, and maths, and error handling, and ... To fix all those things you wouldn't end up with a traditional Unix shell, which I think would be a great project, but there are clearly two mutually exclusive projects to do: 1. A Unix-like shell (this) 2. A robust shell suitable for writing scripts
peteretep 9y ago
Apologies for the off-topic nature of this, but: Something that's been capturing my imagination recently is sshing into a machine, pushing an appropriate shell binary to /tmp, and then executing it. If you have an exotic preference for shell (or vim config, or whatever), we've presumably generally got sufficient bandwidth these days to do a virtually instant user-space install and execution, no?
- falcolas 9y ago
  With a judicious copying of certain dotfiles and directories, this should be quite possible. Just put your shell in ${HOME}/bin, and set your ${HOME}/.ssh/authorized_keys to execute that shell on login, and you should be in good shape. Just expect a few questions from the admins when you do this.
- Skunkleton 9y ago
  I wouldn't copy binaries between arbitrary systems (specific systems is fine), but copying configs is pretty easy with SSH as-is: #!/bin/bash -x HOST=$1 scp -o ControlMaster=auto -o ControlPersist=30 ~/.vimrc $HOST:~/.vimrc ssh -o ControlMaster=auto -o ControlPersist=30 $HOST With the above, you will only create one SSH connection, and you will only authenticate once.
- tomjakubowski 9y ago
  The team I work on had similar needs†, so we built a tool, atop the Python library fabric, to sync a project to (multiple) remote machines, build it, and execute it with configurable command line arguments. http://github.com/Oblong/obi http://github.com/Oblong/obi †: the specific use-case was for development of multi-machine GUI applications that run in configurations like "display walls" or "immersion rooms". for every developer to have their own "display wall" or "immersion room" would cost lots of $$$, so many of the design decisions were based around the needs of a team of N developers that "shares" these systems.
- solidsnack9000 9y ago
  Arx is intended to allow exactly this instant user-space install and execution: https://github.com/solidsnack/arx https://github.com/solidsnack/arx
arca_vorago 9y ago
I just have to say, while I respect the work and have been looking forward to nix tools being ported to rust, the first thing I do is see if it's GPL or not and if it's not it immediately loses points (not all of them), because I have been working hard to free up my stacks. I wish people would consider making core system tools like this GPL instead of MIT/BSD styles. Tivoization is a real threat to the freedom of users and devs. (which is what mit license enables, despite it being listed as "gpl compat") For example, I forced myself to learn screen really well because even though I like tmux and it has some features lacking in screen, I would rather support a tool I know is more aligned with freedom for the user. Same with i3 vs awesome, etc. An interesting side benefit of this is I have noticed the complexity of my stack has been reduced because of this selectiveness.
- Sir_Cmpwn 9y ago
  In fact, the complexity of your system has probably increased. I find the quality of software generally available under the GPL is inferior to that available under more permissive licenses, and in most cases that's due to huge amounts of unnecessary complexity. This may just be bias introduced by GNU, though. In some cases the GPL gets in the way of progress, such as the decable over GCC being able to export its AST for other tools to interact with - a feature RMS rejected on the grounds that it would make it possible to make nonfree tools that integrate with GCC.
  - mhh__ 9y ago
    The GCC AST thing was due to politics inside GNU, not explicitly related to the GPL.
- notheguyouthink 9y ago
  I'm not too familiar with licensing, can someone ELI5? Eg, normally I just choose MIT because licensing is not a concern of mine. I'd like credit, but beyond that, do whatever the hell you want with my work. Now, I'm not making core nix tools, but should I be choosing a different license? Man do I hate licensing.
tomByrer 9y ago
" Won't do list * functions * Windows support " /cry

244 points by mitnk 9y ago

cies 9y ago

There's also this project: https://github.com/redox-os/ion https://github.com/redox-os/ion It's intially build with RedoxOS (in Rust as well) underneath, but now also runs on Linux. Seems to be a bit more mature. I wonder what sets these two projects apart, and/or if the devs of Casada know of Ion's existence.

floatboth 9y ago
Cicada seems to be more POSIX-y, while Ion "it is not, nor will it ever be, compliant with POSIX". What they have in common though… is that job control (Ctrl+Z) is not implemented yet :(
- eridius 9y ago
  I've been giving thought lately to writing my own shell, because there's stuff I want to fix that no current shell does right, but job control is one of the big reasons why I haven't actually started writing code yet (namely, I haven't yet figured out how to reconcile the stuff I want to do with the need to let the user suspend or background tasks including shell scripts).

rocky1138 9y ago

Looks fully featured! Cool stuff. I think people are going to ask "Why?" and "What's the difference between this and bash?" Might want to cover that in your README.

withjive 9y ago
The fact that there is a "Won't do list" stating that functions won't be supported... I don't think this is anything more than a toy to play with rust.
- Brian_K_White 9y ago
  This.
- valarauca1 9y ago
  Generally speaking anytime you start to breakout bash/zsh/sh/csh functions you've already entered a zone where just writing a python/perl script would be the easier and a more maintainable solution in the long term.
  - gkya 9y ago
    I have a dozen of functions in my bashrc that are basically a bit more sophisticated aliases. Why would I want to load a whole interpreter to run them while bash can easily do that?
    - frou_dh 9y ago
      > I have a dozen of functions in my bashrc that are basically a bit more sophisticated aliases. Me too. Looking at them, they all start with something like test $# -eq 2 || return ...since in sh/bash you cannot even name your function's arguments or indicate how many you expect to receive. Speaking of primitiveness, in strict POSIX sh, there's no such thing as local variables in functions!!
  - falcolas 9y ago
    Not if I want to actually affect the current shell environment. For example: export MARKPATH=$HOME/.marks function jump { cd -P $MARKPATH/$1 2> /dev/null || (echo "No such mark: $1" && marks) } function mark { mkdir -p $MARKPATH; ln -s $(pwd) $MARKPATH/$1 } function unmark { rm -i $MARKPATH/$1 } function marks { ls -l $MARKPATH | sed 's/ / /g' | cut -d' ' -f9- && echo } _jump() { local cur=${COMP_WORDS[COMP_CWORD]} COMPREPLY=( $(compgen -W "$( ls $MARKPATH )" -- $cur) ) } complete -F _jump jump This would be much harder to do in a script.
  - eikenberry 9y ago
    Shell functions are useful for much more than scripting. They are like aliases++, letting you add small commands to your shell where aliases don't quite cut it. They let you handle arguments, export environment variables, use conditionals, etc. I have 100 or so that I've added over the years. An interactive shell without functions would not cut it.

brandur 9y ago

Amazing work! Between this, Alacritty [1], and coreutils [2], we're getting pretty close to a plausible all-Rust CLI stack. While Cicada is pretty clearly modeled on the "old" generation of shells (sh, Bash, Zsh, etc.), one has to wonder what a more modern shell might look to address some of the problems of its predecessors like pipes that are essentially text-only, poor composability (see `cut` or `xargs`), and terribly obscure syntax. PowerShell seems to have been a flop outside of the Windows ecosystem, and between overly elaborate syntax and COM baggage, was probably rightfully passed over, but it probably had the right idea. Imagine a future where we're piping structured objects between programs, scripting in a modern language, and have IntelliSense quality auto-completion for every command. --- [1] https://github.com/jwilm/alacritty https://github.com/jwilm/alacritty [2] https://github.com/uutils/coreutils https://github.com/uutils/coreutils

zdkl 9y ago
My structured objects aren't your structured objects. In that use case why not use a dedicated program (like say ae python/js/clojure interpreter) to handle more complex pipes and keep the shell level primitves... Well, primitive. Text is compatible with all systems, past and present, and can be used to model more complex objects. Let's not add features to the core on a "why not" please.
- vog 9y ago
  The so-called "plain text formats" are also just representations of complex objects. Often these formats are ad-hoc and hard to parse correctly. Multiple text files in a file system hierarchy (e.g. /etc) are also just nested structures. So in principle, treating those as complex object structures is the right way to go. Also, I believe this is the idea behind D-Bus and similar modern Unix developments. However, what's hard is to provide good tooling with a simple syntax that is simple to understand: * Windows Registry and Power Shell show how not to do it. * The XML toolchain demonstrate that any unnecessary complexity in the meta-structure will haunt you through every bit of the toolchain (querying, validation/schema, etc.). * "jq" goes somewhat into the right direction for JSON files, but still hasn't found wide adoption. This appears to be a really hard design issue. Also, while deep hierarchies are easier to process by scripts, a human overview is mostly achieved by advances searching and tags rather than hierarchies. A shell needs to accomodate for both, but maybe we really just need better command line tools for simple hierarchial processing of arbitrary text file formats (ad-hoc configs, CSV, INI, JSON, YAML, XML, etc.).

Animats 9y ago

Note: Rust environment is needed for installation. Why?

foodmart 9y ago
A rust installation is needed due to you having to compile it from source manually. If binaries were to be distributed, a rust installation would not be needed.
- mitnk 9y ago
  Yes, I haven't provided the pre-built binaries yet. Maybe I should..
  - IceyEC 9y ago
    Check out snapcraft, it can build rust binaries!

oconnor0 9y ago

Oh, bummer, I was really hoping it would support Windows.

mitnk 9y ago
You need xonsh - http://xon.sh/ http://xon.sh/ :)

city41 9y ago

Being able to do arithmetic right in the shell is handy.

icebraining 9y ago
$ echo $((1 + 2 * 3 - 4)) 3
- vbernat 9y ago
  With zsh, you can even save a few keystrokes. function c() { echo $(($@)) } alias c='noglob c'
  - jplatte 9y ago
    zsh can handle floating point arithmetic too, while bash can't.
tyingq 9y ago
They way they chose to do it though seems to create some ambiguity. Like, what does this do? I can think of 3 distinctly different things it could do. $ echo test 1 + 2>err Or what if there are files with these names? $ cp /bin/grep 42 $ 42 + 1

Klasiaster 9y ago

Escaping and passing strings as arguments (without losing e.g. " characters) is often a problem. It would be so nice if this would be solved in a shell the way it works for function calls in propper programming languages like Python or Rust (i.e. various ways to specify strings with special characters and a good format method for string composition) with Haskell-like function call syntax.

IshKebab 9y ago
And lists, and maths, and error handling, and ... To fix all those things you wouldn't end up with a traditional Unix shell, which I think would be a great project, but there are clearly two mutually exclusive projects to do: 1. A Unix-like shell (this) 2. A robust shell suitable for writing scripts

peteretep 9y ago

Apologies for the off-topic nature of this, but: Something that's been capturing my imagination recently is sshing into a machine, pushing an appropriate shell binary to /tmp, and then executing it. If you have an exotic preference for shell (or vim config, or whatever), we've presumably generally got sufficient bandwidth these days to do a virtually instant user-space install and execution, no?

falcolas 9y ago
With a judicious copying of certain dotfiles and directories, this should be quite possible. Just put your shell in ${HOME}/bin, and set your ${HOME}/.ssh/authorized_keys to execute that shell on login, and you should be in good shape. Just expect a few questions from the admins when you do this.
Skunkleton 9y ago
I wouldn't copy binaries between arbitrary systems (specific systems is fine), but copying configs is pretty easy with SSH as-is: #!/bin/bash -x HOST=$1 scp -o ControlMaster=auto -o ControlPersist=30 ~/.vimrc $HOST:~/.vimrc ssh -o ControlMaster=auto -o ControlPersist=30 $HOST With the above, you will only create one SSH connection, and you will only authenticate once.
tomjakubowski 9y ago
The team I work on had similar needs†, so we built a tool, atop the Python library fabric, to sync a project to (multiple) remote machines, build it, and execute it with configurable command line arguments. http://github.com/Oblong/obi http://github.com/Oblong/obi †: the specific use-case was for development of multi-machine GUI applications that run in configurations like "display walls" or "immersion rooms". for every developer to have their own "display wall" or "immersion room" would cost lots of $$$, so many of the design decisions were based around the needs of a team of N developers that "shares" these systems.
solidsnack9000 9y ago
Arx is intended to allow exactly this instant user-space install and execution: https://github.com/solidsnack/arx https://github.com/solidsnack/arx

arca_vorago 9y ago

I just have to say, while I respect the work and have been looking forward to nix tools being ported to rust, the first thing I do is see if it's GPL or not and if it's not it immediately loses points (not all of them), because I have been working hard to free up my stacks. I wish people would consider making core system tools like this GPL instead of MIT/BSD styles. Tivoization is a real threat to the freedom of users and devs. (which is what mit license enables, despite it being listed as "gpl compat") For example, I forced myself to learn screen really well because even though I like tmux and it has some features lacking in screen, I would rather support a tool I know is more aligned with freedom for the user. Same with i3 vs awesome, etc. An interesting side benefit of this is I have noticed the complexity of my stack has been reduced because of this selectiveness.

Sir_Cmpwn 9y ago
In fact, the complexity of your system has probably increased. I find the quality of software generally available under the GPL is inferior to that available under more permissive licenses, and in most cases that's due to huge amounts of unnecessary complexity. This may just be bias introduced by GNU, though. In some cases the GPL gets in the way of progress, such as the decable over GCC being able to export its AST for other tools to interact with - a feature RMS rejected on the grounds that it would make it possible to make nonfree tools that integrate with GCC.
- mhh__ 9y ago
  The GCC AST thing was due to politics inside GNU, not explicitly related to the GPL.
notheguyouthink 9y ago
I'm not too familiar with licensing, can someone ELI5? Eg, normally I just choose MIT because licensing is not a concern of mine. I'd like credit, but beyond that, do whatever the hell you want with my work. Now, I'm not making core nix tools, but should I be choosing a different license? Man do I hate licensing.

tomByrer 9y ago

" Won't do list * functions * Windows support " /cry