7 ms·

Show HN: Using Rust to write shell-script like tasks

Fiahil 6y ago
This is great! What about "set -euxo pipefail"? I see you were using eprintln before the commands, could we have a "set" macro that would do that for us?
- rustshellscript 6y ago
  You can consider it was enabled by default, and any failed command would return error unless you mask it with “xx || true”.
jitl 6y ago
I love this. I tried to do something similar in Go, because it was in use on my team at Airbnb, and we were looking to port a 2000 line make-and-bash tool to... something not make-and-bash. But as you know, Go doesn’t have macros - so I spent all this effort trying to build a @decorator comment macro system in my personal time (abandoned). Rust seems like a perfect fit for this! We did have a teammate pitching rust, but no one wanted to learn it. Anyways, congrats on the release. It looks fabulous. Safely splicing shell command snippets together is surprisingly annoying, so it’s really cool to see a hygienic yet user-friendly approach.
yisonPylkita 6y ago
My god, this super useful when you have a mix o shell commands and processing text output from them. Bash isn’t particularly easy to work with parsing non-trivial strings in a readable way (I’m looking at you awk)
- scns 6y ago
  Where are nontrivial strings easy to parse?
  - enhray 6y ago
    Ruby? Of all of the scripting languages out there, I find most suitable to write small scripts to transform complex text to something more suitable.
- ilovetux 6y ago
  I apologize, but I must disagree. Awk is literally amazing once you get used to writing actual scripts instead of trying for the ever-elusive and often-untenable one-liners. Noone is using `python -c` syntax for constructing one-liners and i think thats helping adoption of python. I have no idea why one-liners are seen as desirable when they're often hard to read and debug.
  - qchris 6y ago
    When I first was getting into coding while doing test engineering, this was one of my greatest complaints. The software engineers would hand me bash scripts filled with very clever, but unexplained and esoteric one-liners. Whenever something didn't work (and it didn't, because perfectly interfacing with embedded hardware is tough), I had two options: spend literal hours on Google and Stack Overflow, or go stand outside their cubicle and hope they had time for me. I'll take a verbose function with clearly-followable logic over an amazing one-liner with a maze of options and hacks any day.
- thesuperbigfrog 6y ago
  This is why Perl and regular expressions have been so popular on Unix and Linux systems for the past 30 years or so. Python and Ruby are also very handy for these kinds of tasks. People joke about regular expressions being another problem to solve, but they really are an elegant solution to handle a LARGE portion of text processing needs.
laumars 6y ago
Please people, don’t do stuff like this for anything other than personal projects. You might think it’s safer than writing Bash but it isn’t. It results in unsafe Rust code since you’re now forking external code that might be missed by people who are strictly vetting for code inside “unsafe” blocks. Ironically anyone who writes she’ll scripts will know that there are problems with shell scripting but thankfully dot-sh files stand out and bring attention to themselves as files that need to be audited. This wouldn’t. If you need to embed other languages or even just the approximate concept of then, then please at least keep those language files separate rather than inlining them. Then you have an issue that people who are already aware of the pitfalls of shell scripts would know to read through any such scripts but this introduces a newer and unfamiliar scripting language to audit (eg how do we knew that what’s been declared is run but free?). At least Bash et al has had many years of eyeballs on it.
- kazinator 6y ago
  > unsafe Rust ... since you’re now forking external code Are you saying that Rust becomes unsafe because it used a C program as a subroutine? E.g. "tar xvf -" or whatever? What is the fix: rewrite tar, awk, scp and whatever else as Rust functions? That's a lot of work. I'm surprised that you're simultaneously overlooking what ought to be a more gaping problem: that every system call made by a Rust program is a trip through a kernel written in C.
  - laumars 6y ago
    There’s a few problems with forking out: 1. Do those programs exist and what happened if they don’t? That behaviour is already understood in Bash, less so in random 3rd party Rust libraries. 2. Is ‘tar’ calling ./tar, /bin/tar or some other instance of tar? And how do you find out? (eg easy to check $PATH in Bash but does this library honour that? Easy to ‘which tar’ but is that going to be the same tar that this library forks?) 3. Are the people using this software even aware what external programs are being executed? How do they validate this? A .sh file clearly signals that there are external dependencies that need to be audited. A .rs file does not. This problem becomes magnified if you then start shipping compiled binaries rather than source. I get people who like Rust are unlikely to be people who like writing shell scripts but the better way to think of this is like an MVC-like design where you have separate concerns that should be clearly separated in source.
rhn_mk1 6y ago
I am not a huge fan of copying the shell language wholesale and wrapping inside a macro. Since macros can execute arbitrary code, this makes me feel uneasy that the strings are just executed within a shell context, with all the appropriate, bug-prone, expansion done by the shell. Seeing "ls /nofile || true;" makes me worry that "||" is actually passed to the shell wholesale. There's also no transparency about how the binary names are resolved. I much prefer an approach more integrated with the language, like Plumbum: https://plumbum.readthedocs.io/en/latest/local_commands.html#guide-local-commands https://plumbum.readthedocs.io/en/latest/local_commands.html... This no longer looks like the POSIX shell, but instead clearly integrates the good parts directly into the language, even if some complexity bubbles through. I don't have to worry that "grep["world"] < sys.stdin" is piped into an actual shell, because it gets converted into an AST on the way to execution.
- IshKebab 6y ago
  I agree, this is a misguided idea. The whole point of not using Bash is that you don't have to use it's terrible design and syntax surely?
  - foota 6y ago
    This only seems to use the good parts of the shell, easy piping and redirection, while dropping the language for logic.
    - IshKebab 6y ago
      It also appears to use some bad parts, e.g. command line switches and unquoted arguments.
      - rustshellscript 6y ago
        Unquoted argument is not an issue here, see some examples here: https://github.com/rust-shell-script/rust_cmd_lib/issues/10 https://github.com/rust-shell-script/rust_cmd_lib/issues/10
- 6y ago
geowwy 6y ago
> A lot developers just choose shell(sh, bash, ...) scripts for such tasks, > by using < to redirect input, > to redirect output and '|' to pipe outputs. > In my experience, this is the only good parts of shell script. If you try to use shell as a general purpose programming language, of course it sucks. If you treat shell as a DSL for files and streams, nothing can beat it. Shell is amazing. I'm sceptical a bunch of Rust macros can beat shell. I think you'd better off writing a few smaller programs that use STDIO and stringing them together with shell.
- masklinn 6y ago
  > If you try to use shell as a general purpose programming language, of course it sucks. > If you treat shell as a DSL for files and streams, nothing can beat it. Shell is amazing. The problem is that any non-trivial shell script is a mix of the two, so you find yourself torn apart by the inconvenience of "file and streams" in most languages (though really it's mostly subprocesses), and the inconvenience of literally everything else in shells.
  - dijit 6y ago
    I often find myself composing smaller programs which I then call/pipe/chain with bash, this looks pretty messy because I’m a sysadmin, not a programmer. But I do think it works better than trying to do everything in one place.
  - amalcon 6y ago
    This is the one space where I actually like Perl: for shell scripts that grew up a bit, but still amount to mostly manipulating text files and streams.
- jabirali 6y ago
  > If you treat shell as a DSL for files and streams, nothing can beat it. Shell is amazing. On the other hand, wouldn’t you just define anything that beats it also a shell? In my opinion, Fish beats Bash and Zsh in this area, and I would definitely call it a shell even though it’s not a POSIX-compatible shell. A more extreme example would be PowerShell (I’m not a fan but some people love it). Where would you draw the line between a “shell” and an “interpreted scripting language that beats POSIX shells on dealing with files and streams”?
deleted 6y ago
[deleted]
darthrupert 6y ago
If this is a fun proof of concept, it's nice. If somebody uses this in an actual system, it's terrifying. edit oh, Rust is now a thing where even the bad ideas need to be praised without caveats. Gotcha
- laumars 6y ago
  It’s pretty crazy isn’t it. Things like this are fun as pet projects but the stuff of nightmares in real codebase that need supporting for years and by a department that will have the usual churn of staff. I’ve managed enough teams and enough code bases in my time to know that sometimes the smartest code is the least clever. If someone if finding the need to write a shell script in Rust then I’d suggest they need to re-evaluate the problem they’re trying to solve.
- MetaDark 6y ago
  I don't think you deserve to be down voted for this. You bring up a valid concern. Although, I don't agree that this project should be outright dismissed either. So many times, I've run into the issue where I've wanted to chain a set of commands with a concise syntax (specifically in Python) without having to shell out to bash. What I really like about this library is that it gives you the concise composability of bash, without having to deal with its pitfalls (eg. variable escaping, lack of Windows support, clunky interface for anything that's not a command invocation...). Using a DSL will always come with certain tradeoffs, and it won't be the best solution for every use case, but I think this library fills a certain need very well.
- Ar-Curunir 6y ago
  I mean, no one is suggesting that it be used in a critical system yet, so your suggestion is kinda unnecessary
LockAndLol 6y ago
This reminds me of the python version of this called xonsh https://xon.sh/ https://xon.sh/ I really like the idea, but it was missing some simple features that bash had. I can't recall them right now but after an hour of trying to convert a simple bash script, I gave up. That was a year ago. Maybe things changed. I'll give this and xonsh a go again because I just really dislike bash. Thanks for the project!
- rustshellscript 6y ago
  Thanks, yes since I just finished the core functionality of this project and I am not surprised if it is still missing some critical features when converting bash script. Pls file bugs if you find anything :)
- oldsj 6y ago
  I’ve been playing with xonsh lately and really liking it so far! From what I can tell it’s pretty close to feature parity with fish shell which has a lot of nice things like command auto completion but you don’t have to learn yet another shell syntax it’s just python. Wrote up a quick trip report at https://blog.jamesolds.me/post/xonsh-aws-example/ https://blog.jamesolds.me/post/xonsh-aws-example/
- Siira 6y ago
  You might like my alternative design choice: https://github.com/NightMachinary/brish https://github.com/NightMachinary/brish Xonsh is a superset of Python, which introduces a lot of complexity for little gain. Brish chooses to use Python metaprogramming abilities to solve the problem within the language itself, and so is a much simpler solution.
barrenko 6y ago
I really don't see the point of these. Probably just need to explore available tools a bit more.
rossmohax 6y ago
Nothing can fix the fact, that pipes carry dumb byte streams. Powershell addressed this, but sadly remains unpopular in Unix crowd
- hnlmorg 6y ago
  Actually there are several shells out there that fix that problem and still support existing UNIX tools too (which Powershell doesn't play nice with). My own shell, https://github.com/lmorg/murex https://github.com/lmorg/murex does this by passing type information along with the byte stream. So _murex_ aware tools can have structured data passed and POSIX tools can fall back to byte streams. Best of both worlds. The problem, however, is that as long as Bourne Shell and Bash are installed everywhere, people will write scripts for it. This is less about the popularity of UNIX tools and more about the ubiquity of them (though the two points aren't mutually exclusive).
  - carlmr 6y ago
    >The problem, however, is that as long as Bourne Shell and Bash are installed everywhere, people will write scripts for it. This is also an issue of interpreted languages. Often I write bash and very constricted python2/3 compatible code, because I can be fairly sure the target audience has both of these. You need to have everyone install (and maybe even use) your shell/language for them to be able to use it. Or have them recreate your environment (docker or cxfreeze). With Rust it's easy to distribute a small self contained binary.
- eska 6y ago
  You might be interested in nushell.
oconnor663 6y ago
I'm going to shamelessly plug my own library here: https://github.com/oconnor663/duct.rs https://github.com/oconnor663/duct.rs I wanted to solve the same problem, originally in Python (https://github.com/oconnor663/duct.py https://github.com/oconnor663/duct.py). It's surprisingly annoying to do pipelines and redirections, compared to how easy they are to do in the shell. Lots of libraries try to address this, but most of them seem do it by emulating shell syntax within the host language, using operator overloading or other magic like that. I think that's a limiting choice. (For example, can you use `cd` to change the working dir for the left half of a pipeline but not the right half? In Bash you would use a "subshell" for this.) Instead, I think it's sufficient to build an API out of regular objects with regular methods. The result doesn't look like shell code, but it's easier to reason about, and more consistent across different languages.
- rustshellscript 6y ago
  It can be supported with internal APIs, even without macros: Cmds::from_cmd(Cmd(...).current_dir(..)) .pipe(Cmd(...).current_dir(...)) .run_cmd(...) As you can see, it is very verbose and that's why I choose to hide the lower APIs at this moment.
  - joobus 6y ago
    I like your approach more than duct.rs :)
- laumars 6y ago
  ‘cd’ is a shell builtin so you couldn’t use ‘cd’ in any of these solutions unless they then spawn a shell instance...and that worries me if you are because then you really might as well just have a separate .sh file and launch that instead (at least that is more auditable with tools like Shellcheck than any inlined code would be).
  - dllthomas 6y ago
    As I read the parent comment, the broad context is turning "shell-like behavior" into rust code, and the comment is choosing to talk about that projection by focusing on elements in the source and assuming that it's understood that they're really talking about the resulting image. You can't use the shell's cd, but you can call chdir and set the working directory - and hopefully you can do that for only part of your pipeline. If they were in fact describing implementation, then I mostly agree - it's likely better to write shell directly than generate it, at least short of treating it seriously as a compilation target.
sneak 6y ago
Does anyone know of something similar for Go?
staktrace 6y ago
I wrote a tool to do the opposite thing: allow writing "shell scripts" using Rust. Still early days but https://github.com/staktrace/khaki https://github.com/staktrace/khaki is where it lives.
ausjke 6y ago
what's the difference between this approach and shell scripts? Thanks.
lugoues 6y ago
There is also https://github.com/igor-petruk/scriptisto https://github.com/igor-petruk/scriptisto which would allow you to wrap any compiled language.
gitgud 6y ago
Interesting, it seems that it allows the exact syntax of shell commands, without using strings. // valid rust code and shell code, no strings run_cmd!(du -ah . | sort -hr | head -n 10)?; How does rust parse the statement within run_cmd()? Can rust parse other languages like this? run_html!(<div>COOL</div>)
- steveklabnik 6y ago
  https://crates.io/crates/typed-html-macros https://crates.io/crates/typed-html-macros
- qwertycrackers 6y ago
  Yeah, there's a system for macro definition where you define the language the macro accepts. It's very powerful and can probably do most of what you're imagining in this comment.
- gabssnake 6y ago
  Yes! If you are interested in doing front-end in Rust using Wasm, checkout Yew : https://github.com/yewstack/yew https://github.com/yewstack/yew The data flow is inspired by React, you’ll feel right at home.
implfuture 6y ago
This is so awesome! Any tips on parallelizing/is it async compatible? I personally found granular error handling combined with parallelization to be impossible to get just right in pure bash.

This is great! What about "set -euxo pipefail"? I see you were using eprintln before the commands, could we have a "set" macro that would do that for us?

rustshellscript 6y ago
You can consider it was enabled by default, and any failed command would return error unless you mask it with “xx || true”.

jitl 6y ago

I love this. I tried to do something similar in Go, because it was in use on my team at Airbnb, and we were looking to port a 2000 line make-and-bash tool to... something not make-and-bash. But as you know, Go doesn’t have macros - so I spent all this effort trying to build a @decorator comment macro system in my personal time (abandoned). Rust seems like a perfect fit for this! We did have a teammate pitching rust, but no one wanted to learn it. Anyways, congrats on the release. It looks fabulous. Safely splicing shell command snippets together is surprisingly annoying, so it’s really cool to see a hygienic yet user-friendly approach.

yisonPylkita 6y ago

My god, this super useful when you have a mix o shell commands and processing text output from them. Bash isn’t particularly easy to work with parsing non-trivial strings in a readable way (I’m looking at you awk)

scns 6y ago
Where are nontrivial strings easy to parse?
- enhray 6y ago
  Ruby? Of all of the scripting languages out there, I find most suitable to write small scripts to transform complex text to something more suitable.
ilovetux 6y ago
I apologize, but I must disagree. Awk is literally amazing once you get used to writing actual scripts instead of trying for the ever-elusive and often-untenable one-liners. Noone is using `python -c` syntax for constructing one-liners and i think thats helping adoption of python. I have no idea why one-liners are seen as desirable when they're often hard to read and debug.
- qchris 6y ago
  When I first was getting into coding while doing test engineering, this was one of my greatest complaints. The software engineers would hand me bash scripts filled with very clever, but unexplained and esoteric one-liners. Whenever something didn't work (and it didn't, because perfectly interfacing with embedded hardware is tough), I had two options: spend literal hours on Google and Stack Overflow, or go stand outside their cubicle and hope they had time for me. I'll take a verbose function with clearly-followable logic over an amazing one-liner with a maze of options and hacks any day.
thesuperbigfrog 6y ago
This is why Perl and regular expressions have been so popular on Unix and Linux systems for the past 30 years or so. Python and Ruby are also very handy for these kinds of tasks. People joke about regular expressions being another problem to solve, but they really are an elegant solution to handle a LARGE portion of text processing needs.

laumars 6y ago

Please people, don’t do stuff like this for anything other than personal projects. You might think it’s safer than writing Bash but it isn’t. It results in unsafe Rust code since you’re now forking external code that might be missed by people who are strictly vetting for code inside “unsafe” blocks. Ironically anyone who writes she’ll scripts will know that there are problems with shell scripting but thankfully dot-sh files stand out and bring attention to themselves as files that need to be audited. This wouldn’t. If you need to embed other languages or even just the approximate concept of then, then please at least keep those language files separate rather than inlining them. Then you have an issue that people who are already aware of the pitfalls of shell scripts would know to read through any such scripts but this introduces a newer and unfamiliar scripting language to audit (eg how do we knew that what’s been declared is run but free?). At least Bash et al has had many years of eyeballs on it.

kazinator 6y ago
> unsafe Rust ... since you’re now forking external code Are you saying that Rust becomes unsafe because it used a C program as a subroutine? E.g. "tar xvf -" or whatever? What is the fix: rewrite tar, awk, scp and whatever else as Rust functions? That's a lot of work. I'm surprised that you're simultaneously overlooking what ought to be a more gaping problem: that every system call made by a Rust program is a trip through a kernel written in C.
- laumars 6y ago
  There’s a few problems with forking out: 1. Do those programs exist and what happened if they don’t? That behaviour is already understood in Bash, less so in random 3rd party Rust libraries. 2. Is ‘tar’ calling ./tar, /bin/tar or some other instance of tar? And how do you find out? (eg easy to check $PATH in Bash but does this library honour that? Easy to ‘which tar’ but is that going to be the same tar that this library forks?) 3. Are the people using this software even aware what external programs are being executed? How do they validate this? A .sh file clearly signals that there are external dependencies that need to be audited. A .rs file does not. This problem becomes magnified if you then start shipping compiled binaries rather than source. I get people who like Rust are unlikely to be people who like writing shell scripts but the better way to think of this is like an MVC-like design where you have separate concerns that should be clearly separated in source.

rhn_mk1 6y ago

I am not a huge fan of copying the shell language wholesale and wrapping inside a macro. Since macros can execute arbitrary code, this makes me feel uneasy that the strings are just executed within a shell context, with all the appropriate, bug-prone, expansion done by the shell. Seeing "ls /nofile || true;" makes me worry that "||" is actually passed to the shell wholesale. There's also no transparency about how the binary names are resolved. I much prefer an approach more integrated with the language, like Plumbum: https://plumbum.readthedocs.io/en/latest/local_commands.html#guide-local-commands https://plumbum.readthedocs.io/en/latest/local_commands.html... This no longer looks like the POSIX shell, but instead clearly integrates the good parts directly into the language, even if some complexity bubbles through. I don't have to worry that "grep["world"] < sys.stdin" is piped into an actual shell, because it gets converted into an AST on the way to execution.

IshKebab 6y ago
I agree, this is a misguided idea. The whole point of not using Bash is that you don't have to use it's terrible design and syntax surely?
- foota 6y ago
  This only seems to use the good parts of the shell, easy piping and redirection, while dropping the language for logic.
  - IshKebab 6y ago
    It also appears to use some bad parts, e.g. command line switches and unquoted arguments.
    - rustshellscript 6y ago
      Unquoted argument is not an issue here, see some examples here: https://github.com/rust-shell-script/rust_cmd_lib/issues/10 https://github.com/rust-shell-script/rust_cmd_lib/issues/10
6y ago

geowwy 6y ago

> A lot developers just choose shell(sh, bash, ...) scripts for such tasks, > by using < to redirect input, > to redirect output and '|' to pipe outputs. > In my experience, this is the only good parts of shell script. If you try to use shell as a general purpose programming language, of course it sucks. If you treat shell as a DSL for files and streams, nothing can beat it. Shell is amazing. I'm sceptical a bunch of Rust macros can beat shell. I think you'd better off writing a few smaller programs that use STDIO and stringing them together with shell.

masklinn 6y ago
> If you try to use shell as a general purpose programming language, of course it sucks. > If you treat shell as a DSL for files and streams, nothing can beat it. Shell is amazing. The problem is that any non-trivial shell script is a mix of the two, so you find yourself torn apart by the inconvenience of "file and streams" in most languages (though really it's mostly subprocesses), and the inconvenience of literally everything else in shells.
- dijit 6y ago
  I often find myself composing smaller programs which I then call/pipe/chain with bash, this looks pretty messy because I’m a sysadmin, not a programmer. But I do think it works better than trying to do everything in one place.
- amalcon 6y ago
  This is the one space where I actually like Perl: for shell scripts that grew up a bit, but still amount to mostly manipulating text files and streams.
jabirali 6y ago
> If you treat shell as a DSL for files and streams, nothing can beat it. Shell is amazing. On the other hand, wouldn’t you just define anything that beats it also a shell? In my opinion, Fish beats Bash and Zsh in this area, and I would definitely call it a shell even though it’s not a POSIX-compatible shell. A more extreme example would be PowerShell (I’m not a fan but some people love it). Where would you draw the line between a “shell” and an “interpreted scripting language that beats POSIX shells on dealing with files and streams”?

deleted 6y ago

[deleted]

darthrupert 6y ago

If this is a fun proof of concept, it's nice. If somebody uses this in an actual system, it's terrifying. edit oh, Rust is now a thing where even the bad ideas need to be praised without caveats. Gotcha

laumars 6y ago
It’s pretty crazy isn’t it. Things like this are fun as pet projects but the stuff of nightmares in real codebase that need supporting for years and by a department that will have the usual churn of staff. I’ve managed enough teams and enough code bases in my time to know that sometimes the smartest code is the least clever. If someone if finding the need to write a shell script in Rust then I’d suggest they need to re-evaluate the problem they’re trying to solve.
MetaDark 6y ago
I don't think you deserve to be down voted for this. You bring up a valid concern. Although, I don't agree that this project should be outright dismissed either. So many times, I've run into the issue where I've wanted to chain a set of commands with a concise syntax (specifically in Python) without having to shell out to bash. What I really like about this library is that it gives you the concise composability of bash, without having to deal with its pitfalls (eg. variable escaping, lack of Windows support, clunky interface for anything that's not a command invocation...). Using a DSL will always come with certain tradeoffs, and it won't be the best solution for every use case, but I think this library fills a certain need very well.
Ar-Curunir 6y ago
I mean, no one is suggesting that it be used in a critical system yet, so your suggestion is kinda unnecessary

LockAndLol 6y ago

This reminds me of the python version of this called xonsh https://xon.sh/ https://xon.sh/ I really like the idea, but it was missing some simple features that bash had. I can't recall them right now but after an hour of trying to convert a simple bash script, I gave up. That was a year ago. Maybe things changed. I'll give this and xonsh a go again because I just really dislike bash. Thanks for the project!

rustshellscript 6y ago
Thanks, yes since I just finished the core functionality of this project and I am not surprised if it is still missing some critical features when converting bash script. Pls file bugs if you find anything :)
oldsj 6y ago
I’ve been playing with xonsh lately and really liking it so far! From what I can tell it’s pretty close to feature parity with fish shell which has a lot of nice things like command auto completion but you don’t have to learn yet another shell syntax it’s just python. Wrote up a quick trip report at https://blog.jamesolds.me/post/xonsh-aws-example/ https://blog.jamesolds.me/post/xonsh-aws-example/
Siira 6y ago
You might like my alternative design choice: https://github.com/NightMachinary/brish https://github.com/NightMachinary/brish Xonsh is a superset of Python, which introduces a lot of complexity for little gain. Brish chooses to use Python metaprogramming abilities to solve the problem within the language itself, and so is a much simpler solution.

barrenko 6y ago

I really don't see the point of these. Probably just need to explore available tools a bit more.

rossmohax 6y ago

Nothing can fix the fact, that pipes carry dumb byte streams. Powershell addressed this, but sadly remains unpopular in Unix crowd

hnlmorg 6y ago
Actually there are several shells out there that fix that problem and still support existing UNIX tools too (which Powershell doesn't play nice with). My own shell, https://github.com/lmorg/murex https://github.com/lmorg/murex does this by passing type information along with the byte stream. So _murex_ aware tools can have structured data passed and POSIX tools can fall back to byte streams. Best of both worlds. The problem, however, is that as long as Bourne Shell and Bash are installed everywhere, people will write scripts for it. This is less about the popularity of UNIX tools and more about the ubiquity of them (though the two points aren't mutually exclusive).
- carlmr 6y ago
  >The problem, however, is that as long as Bourne Shell and Bash are installed everywhere, people will write scripts for it. This is also an issue of interpreted languages. Often I write bash and very constricted python2/3 compatible code, because I can be fairly sure the target audience has both of these. You need to have everyone install (and maybe even use) your shell/language for them to be able to use it. Or have them recreate your environment (docker or cxfreeze). With Rust it's easy to distribute a small self contained binary.
eska 6y ago
You might be interested in nushell.

oconnor663 6y ago

I'm going to shamelessly plug my own library here: https://github.com/oconnor663/duct.rs https://github.com/oconnor663/duct.rs I wanted to solve the same problem, originally in Python (https://github.com/oconnor663/duct.py https://github.com/oconnor663/duct.py). It's surprisingly annoying to do pipelines and redirections, compared to how easy they are to do in the shell. Lots of libraries try to address this, but most of them seem do it by emulating shell syntax within the host language, using operator overloading or other magic like that. I think that's a limiting choice. (For example, can you use `cd` to change the working dir for the left half of a pipeline but not the right half? In Bash you would use a "subshell" for this.) Instead, I think it's sufficient to build an API out of regular objects with regular methods. The result doesn't look like shell code, but it's easier to reason about, and more consistent across different languages.

rustshellscript 6y ago
It can be supported with internal APIs, even without macros: Cmds::from_cmd(Cmd(...).current_dir(..)) .pipe(Cmd(...).current_dir(...)) .run_cmd(...) As you can see, it is very verbose and that's why I choose to hide the lower APIs at this moment.
- joobus 6y ago
  I like your approach more than duct.rs :)
laumars 6y ago
‘cd’ is a shell builtin so you couldn’t use ‘cd’ in any of these solutions unless they then spawn a shell instance...and that worries me if you are because then you really might as well just have a separate .sh file and launch that instead (at least that is more auditable with tools like Shellcheck than any inlined code would be).
- dllthomas 6y ago
  As I read the parent comment, the broad context is turning "shell-like behavior" into rust code, and the comment is choosing to talk about that projection by focusing on elements in the source and assuming that it's understood that they're really talking about the resulting image. You can't use the shell's cd, but you can call chdir and set the working directory - and hopefully you can do that for only part of your pipeline. If they were in fact describing implementation, then I mostly agree - it's likely better to write shell directly than generate it, at least short of treating it seriously as a compilation target.

sneak 6y ago

Does anyone know of something similar for Go?

staktrace 6y ago

I wrote a tool to do the opposite thing: allow writing "shell scripts" using Rust. Still early days but https://github.com/staktrace/khaki https://github.com/staktrace/khaki is where it lives.

ausjke 6y ago

what's the difference between this approach and shell scripts? Thanks.

lugoues 6y ago

There is also https://github.com/igor-petruk/scriptisto https://github.com/igor-petruk/scriptisto which would allow you to wrap any compiled language.

gitgud 6y ago

Interesting, it seems that it allows the exact syntax of shell commands, without using strings. // valid rust code and shell code, no strings run_cmd!(du -ah . | sort -hr | head -n 10)?; How does rust parse the statement within run_cmd()? Can rust parse other languages like this? run_html!(<div>COOL</div>)

steveklabnik 6y ago
https://crates.io/crates/typed-html-macros https://crates.io/crates/typed-html-macros
qwertycrackers 6y ago
Yeah, there's a system for macro definition where you define the language the macro accepts. It's very powerful and can probably do most of what you're imagining in this comment.
gabssnake 6y ago
Yes! If you are interested in doing front-end in Rust using Wasm, checkout Yew : https://github.com/yewstack/yew https://github.com/yewstack/yew The data flow is inspired by React, you’ll feel right at home.

implfuture 6y ago

This is so awesome! Any tips on parallelizing/is it async compatible? I personally found granular error handling combined with parallelization to be impossible to get just right in pure bash.