9 ms·

Zig's new CLI progress bar explained

122 points by fmystic 2y ago

lionkor 2y ago
I really enjoy reading Zig, I always struggle writing it because its so subtly different from the other languages I use, like C, Rust and Go, that I get confused often. I really hope Zig becomes stable soon so I can use it on non-throwaway projects
- voidhorse 2y ago
  My experience is exactly the same. It deviates just enough in really minor ways that it becomes difficult to write. There's a lot of upfront effort to unlearn all the syntactic patterns that have already been established, the fact that the differences are really small makes it harder, not easier. I think the language design includes a lot of good justification for these choices, but I worry they might have underestimated the sheer power of habit and historical precedence.
  - wredue 2y ago
    Personally, I do not find that zig differs from other languages all that greatly. I am actually a little curious about the minor and subtle differences, cause I personally did not find that this was the case coming from C, Java, C#, a little rust, COBOL and NATURAL. Edit: I do admit that you need to understand what said other languages are doing in order to accomplish things. No hidden flow definitely increases a bit of the surface area of the code, but if you understood how other languages accomplish the stuff they hide form you, then zig does not feel like it has minor differences.
  - anacrolix 2y ago
    This is my impression too. Rust is drastically different from mainstream languages, so it's worth the jump. Zig is quite close, and I'm not sure it provides enough. I would just use C.
- mr90210 2y ago
  > I really hope Zig becomes stable soon so I can use it on non-throwaway projects This may or may not give you some comfort, but I recall hearing one of the core maintainers of Zig talking about its use at Amazon to leverage its cross platform capabilities.
- wredue 2y ago
  0.12 is meant to be the version that you can probably stop tracking master. It isn’t stable, but it is.
- hansvm 2y ago
  Here's my opinion from the Zig evangelism strike force: Aside from the removal of async (which was very disruptive for a few of my projects), the rest of the changes over the last few years have been minor. It takes a few hours once a year to update a few tens of thousands of lines of code to the new syntax, build system, and stdlib. The current 0.12.0 is supposed to be a semi-stable release for people to be happy with pre-1.0, reducing that effort further, and language stability is an express design goal, so once 1.0 hits (ETA 3-5yrs?) it'll almost certainly be good enough for you. I do think the initial learning curve was higher than I would have expected for such a simple language. The docs are much better now, and there are more examples and learning resources (off-topic, definitely use the zig-help channel in their discord server). I just finished making a compelling enough prototype that $WORK wants to use Zig for some performance-critical software. I suspect it'll be easier for my team to learn than it was for me. We'll see (whether it succeeds or fails, that sounds like a fun first blog post). Despite that higher-than-expected learning curve (which IMO was mostly because nuances like the in-memory representation of a 5-bit integer weren't documented, so you had to experiment to find out the exact behavior if you were doing anything "interesting"), once you've picked it up it really is a simple language. IMO it's worth pushing through. I've done hardly anything with Go yet, but Zig fits happily in the mind of this particular C/Rust/Forth/Scala/Python/... programmer. Pushing through the first ~100 Ziglings exercises might be a pretty fast way to work through most of the syntactic differences if you're already comfortable with C and Rust. Or maybe your prior is that new languages need to prove themselves and stabilize a little longer before you sink time into them. That's fine too. I'm just commenting since you seem to be a bit on the fence and because I think it's better than your current impression of it is.
  - binary132 2y ago
    I stan Zig but the OP literally starts with “there was a subtle bug in my fork() wrapper for years”.
thefaux 2y ago
Unsolicited writing advice: drop the self-aggrandizing grandiosity. It is distracting and undermines the work itself. Zig is a language for writing perfect programs? The fact that a progress bar is needed at all is a symptom of the lack of perfection of Zig. A perfect compiler would compile instantly (or at least faster than human perception is capable of registering) with no need for a progress bar.
- YuukiRey 2y ago
  I highly doubt that was serious. I think you just missed the joke. Or did I miss something? I really hope I didn’t :(
  - AndyKelley 2y ago
    It's amusing to me that multiple people on HN think I'm joking. Have you ever tried to write thread-safe, lock-free, infallible, non-heap-allocating code before? I'm curious what kind of API the people who think this problem is easy would have come up with, and what its performance characteristics would be compared to mine. I had a blast working on this problem!
    - YuukiRey 2y ago
      I think the part that might raise eyebrows is the “programming language designed for making perfect software”. Perfect is, after all, highly subjective, yet at the same time a maybe unnecessarily hyperbolic term. I don’t mind the language though. I have no doubt that you’re at the top of your game and the problems you’ve outlined in the post do indeed sound challenging.
      - AndyKelley 2y ago
        https://youtu.be/Z4oYSByyRak?t=161 https://youtu.be/Z4oYSByyRak?t=161 Software Should Be Perfect - A journey of questioning assumptions and building a new programming language This has been my goal since the beginning. I wasn't kidding
    - 2y ago
ximeng 2y ago
The visualisation is really neat. Would be interesting to have a mechanism to track runtime of each step and compare across runs (maybe with a unique generated key?), so that you could benchmark speed up and slow down. This would enable per time rather than per step progress bars.
ai_ 2y ago
I'm not sure if it's actually any easier to read. When lots of things are changing it honestly becomes harder to read and figure out what's important and what's superfluous. With the old progress system, everything was on one line. This honestly isn't horrible to me since I can easily glance text from left to right to figure out what the gist of the text is. When it's changing between the same two steps it still isn't too much of an issue since the information is all still in the same place and it's not actually changing too much between each stage. I can identify each step, figure out what's changing between them, and look for that information specifically. The new progress system dumps a lot more information at the user, most of it detailing what file is being analyzed and compiled, each one taking maybe 2-4 frames of screen time, on an excessive number of lines, just a complete barrage of pointless information. None of this is really important to me since the only time it would be important is if a file or step took more than 3 seconds to be processed. With items constantly appearing and disappearing, the things that are taking time on the more macro scale like build-lib and build-exe steps that are more important to me will constantly move around the terminal. It's much much harder to read something if it's jumping up and down randomly every 2 frames vs if it's being swapped to share a single line. If the line literally leaves my field of view, it becomes frustrating to follow. I much prefer the Bazel approach to this problem. When running a series of actions concurrently, the 6 actions taking the most amount of time will be visible in the action list showing how long they're taking, but all other actions will be minimized to a "and X other actions..." line. This looks cool on the surface, but in practice is not that good at giving you progress information. Which is what progress indicators should do! At best this is a better indicator of how much work is being done, not how much progress is being made. Like a bunch of bleeps, bloops, and random LEDs toggling when a computer does work in a sci-fi TV show. I think this would be better if individual files being processed got removed unless they start taking too long. Keep the build-exe and build-lib steps around but make them a little sticky. When they complete, have it say "Done" and then remove things in groups or on a set interval. Don't change the number of lines too often and don't reorganize lines either. Generally, it should be easy to parse what's going on and frequent changes to the number of lines and how much information is on each makes that hard.
- epage 2y ago
  This is really good feedback! I'm looking at redesigning Cargo's output (after I wrap up other projects).
Groxx 2y ago
I wish all multi-task systems had a UI like this. It makes it SO much easier to spot where your biggest latencies are coming from, which is an excellent passive motivator to improve them. Silent / opaque progress just teaches people that they can't do anything about it except wait, so all it does is get worse and worse.
- deleted 2y ago
  [deleted]
epage 2y ago
I've seen guides for detecting terminal support for colors, hyperlinks, emoji, etc but not "fancy" terminal control like this. Anyone know of any? In one CLI I work on, some others are concerned about the long tail of users and wanting to make sure no one has a bad experience by default.
- rockorager 2y ago
  This isn't using any "fancy" terminal features, aside from the synchronized update sequences (which terminals that don't support typically will ignore, though windows has a special case where it must be ignored). That said, you can query the terminal for support for this sequence if you wanted to. Other than that, it's using standard ANSI sequences: `\r` to return the cursor to the beginning of the line, `\x1bM` (Reverse Index) to move the cursor up a single line (repeated n times), and then `\x1b[J` to clear the screen below the cursor position. All of these are sequences defined at least since the VT220, probably the VT100.
  - epage 2y ago
    > `\x1bM` (Reverse Index) to move the cursor up a single line (repeated n times), and then `\x1b[J` to clear the screen below the cursor position. All of these are sequences defined at least since the VT220, probably the VT100. For whatever reason, the people I'm collaborating with have the impression these aren't universal enough. And of course termcaps is out of vogue.
    - saurik 2y ago
      Simultaneously caring about terminals that don't support basic movement but also refusing to use termcap seems like an unfortunate combination :(.
Filligree 2y ago
> The key insight I had here is that, since the end result must be displayed on a terminal screen, there is a reasonably small upper bound on how much memory is required, beyond which point the extra memory couldn't be utilized because it wouldn't fit on the terminal screen anyway. How large are your terminals? I'm not sure if this represents lines, or perhaps something else. But on occasion I need to copy things out of my terminal, scrolling while copying doesn't work well, and so I shrink the text size to, let's say, one or two pixels per letter. I suspect Zig won't survive this.
- vanderZwan 2y ago
  One pixel per letter on a 4k screen would be 3840 × 2160 chars at most, so around 8 megabytes if we assume one byte per "character tile" on the screen. I highly doubt Zig would die from this.
  - Filligree 2y ago
    The article suggests it’s using a fixed-size buffer of 200 somethings. Seems worth checking, though I have no goose in this race.
    - vanderZwan 2y ago
      If I understand correctly then that's for the number of threads, although I also think that the sudden switch from terminal size back to the topic of threads is confusing so I agree that it's not entirely clear what the connection between the two is - or if there is none, that the structure of the article makes that confusing
Animats 2y ago
Not sure if this is a joke.
one-punch 2y ago
Reminds me of nix-output-monitor [1], for example see [2]. It makes it easy to understand how individual steps are progressing, and how individual steps relate to the overall plan. It enables me to locate expensive build steps, and possibly to avoid them if steps are failing. [1]: https://github.com/maralorn/nix-output-monitor https://github.com/maralorn/nix-output-monitor [2]: https://asciinema.org/a/7hJXH2iFLEkKxG1lL25lspqNn https://asciinema.org/a/7hJXH2iFLEkKxG1lL25lspqNn
deleted 2y ago
[deleted]
binary132 2y ago
One thing that the latest generation of languages has taught me (or that I have learned in the process?) is that “languages” aren’t “really a real thing” in and of themselves as much as they are merely composable APIs over compilers / interpreters (which are basically just dynamic compilers, or compilers are interpreters?), and compilers are an insane, fragmented dumpster fire with no or barely usable APIs. Only LLVM, GCC, TinyCC, and Terra admit this reality, neither LLVM nor GCC’s APIs are really user-oriented, and Terra isn’t really a realistic option for most people. I also personally don’t really feel that LLVM has accomplished (or can accomplish) its potential for a variety of reasons — most LLVM projects are forks or patches, although Zig has done well at this — and GCC has limitations as well although I’m very pleased with gccjit in Emacs. I’m very curious to see what the future holds especially if we can get an optionally-verifiable yet usable compiler API.

122 points by fmystic 2y ago

I really enjoy reading Zig, I always struggle writing it because its so subtly different from the other languages I use, like C, Rust and Go, that I get confused often. I really hope Zig becomes stable soon so I can use it on non-throwaway projects

voidhorse 2y ago
My experience is exactly the same. It deviates just enough in really minor ways that it becomes difficult to write. There's a lot of upfront effort to unlearn all the syntactic patterns that have already been established, the fact that the differences are really small makes it harder, not easier. I think the language design includes a lot of good justification for these choices, but I worry they might have underestimated the sheer power of habit and historical precedence.
- wredue 2y ago
  Personally, I do not find that zig differs from other languages all that greatly. I am actually a little curious about the minor and subtle differences, cause I personally did not find that this was the case coming from C, Java, C#, a little rust, COBOL and NATURAL. Edit: I do admit that you need to understand what said other languages are doing in order to accomplish things. No hidden flow definitely increases a bit of the surface area of the code, but if you understood how other languages accomplish the stuff they hide form you, then zig does not feel like it has minor differences.
- anacrolix 2y ago
  This is my impression too. Rust is drastically different from mainstream languages, so it's worth the jump. Zig is quite close, and I'm not sure it provides enough. I would just use C.
mr90210 2y ago
> I really hope Zig becomes stable soon so I can use it on non-throwaway projects This may or may not give you some comfort, but I recall hearing one of the core maintainers of Zig talking about its use at Amazon to leverage its cross platform capabilities.
wredue 2y ago
0.12 is meant to be the version that you can probably stop tracking master. It isn’t stable, but it is.
hansvm 2y ago
Here's my opinion from the Zig evangelism strike force: Aside from the removal of async (which was very disruptive for a few of my projects), the rest of the changes over the last few years have been minor. It takes a few hours once a year to update a few tens of thousands of lines of code to the new syntax, build system, and stdlib. The current 0.12.0 is supposed to be a semi-stable release for people to be happy with pre-1.0, reducing that effort further, and language stability is an express design goal, so once 1.0 hits (ETA 3-5yrs?) it'll almost certainly be good enough for you. I do think the initial learning curve was higher than I would have expected for such a simple language. The docs are much better now, and there are more examples and learning resources (off-topic, definitely use the zig-help channel in their discord server). I just finished making a compelling enough prototype that $WORK wants to use Zig for some performance-critical software. I suspect it'll be easier for my team to learn than it was for me. We'll see (whether it succeeds or fails, that sounds like a fun first blog post). Despite that higher-than-expected learning curve (which IMO was mostly because nuances like the in-memory representation of a 5-bit integer weren't documented, so you had to experiment to find out the exact behavior if you were doing anything "interesting"), once you've picked it up it really is a simple language. IMO it's worth pushing through. I've done hardly anything with Go yet, but Zig fits happily in the mind of this particular C/Rust/Forth/Scala/Python/... programmer. Pushing through the first ~100 Ziglings exercises might be a pretty fast way to work through most of the syntactic differences if you're already comfortable with C and Rust. Or maybe your prior is that new languages need to prove themselves and stabilize a little longer before you sink time into them. That's fine too. I'm just commenting since you seem to be a bit on the fence and because I think it's better than your current impression of it is.
- binary132 2y ago
  I stan Zig but the OP literally starts with “there was a subtle bug in my fork() wrapper for years”.

thefaux 2y ago

Unsolicited writing advice: drop the self-aggrandizing grandiosity. It is distracting and undermines the work itself. Zig is a language for writing perfect programs? The fact that a progress bar is needed at all is a symptom of the lack of perfection of Zig. A perfect compiler would compile instantly (or at least faster than human perception is capable of registering) with no need for a progress bar.

YuukiRey 2y ago
I highly doubt that was serious. I think you just missed the joke. Or did I miss something? I really hope I didn’t :(
- AndyKelley 2y ago
  It's amusing to me that multiple people on HN think I'm joking. Have you ever tried to write thread-safe, lock-free, infallible, non-heap-allocating code before? I'm curious what kind of API the people who think this problem is easy would have come up with, and what its performance characteristics would be compared to mine. I had a blast working on this problem!
  - YuukiRey 2y ago
    I think the part that might raise eyebrows is the “programming language designed for making perfect software”. Perfect is, after all, highly subjective, yet at the same time a maybe unnecessarily hyperbolic term. I don’t mind the language though. I have no doubt that you’re at the top of your game and the problems you’ve outlined in the post do indeed sound challenging.
    - AndyKelley 2y ago
      https://youtu.be/Z4oYSByyRak?t=161 https://youtu.be/Z4oYSByyRak?t=161 Software Should Be Perfect - A journey of questioning assumptions and building a new programming language This has been my goal since the beginning. I wasn't kidding
  - 2y ago

ximeng 2y ago

The visualisation is really neat. Would be interesting to have a mechanism to track runtime of each step and compare across runs (maybe with a unique generated key?), so that you could benchmark speed up and slow down. This would enable per time rather than per step progress bars.

ai_ 2y ago

I'm not sure if it's actually any easier to read. When lots of things are changing it honestly becomes harder to read and figure out what's important and what's superfluous. With the old progress system, everything was on one line. This honestly isn't horrible to me since I can easily glance text from left to right to figure out what the gist of the text is. When it's changing between the same two steps it still isn't too much of an issue since the information is all still in the same place and it's not actually changing too much between each stage. I can identify each step, figure out what's changing between them, and look for that information specifically. The new progress system dumps a lot more information at the user, most of it detailing what file is being analyzed and compiled, each one taking maybe 2-4 frames of screen time, on an excessive number of lines, just a complete barrage of pointless information. None of this is really important to me since the only time it would be important is if a file or step took more than 3 seconds to be processed. With items constantly appearing and disappearing, the things that are taking time on the more macro scale like build-lib and build-exe steps that are more important to me will constantly move around the terminal. It's much much harder to read something if it's jumping up and down randomly every 2 frames vs if it's being swapped to share a single line. If the line literally leaves my field of view, it becomes frustrating to follow. I much prefer the Bazel approach to this problem. When running a series of actions concurrently, the 6 actions taking the most amount of time will be visible in the action list showing how long they're taking, but all other actions will be minimized to a "and X other actions..." line. This looks cool on the surface, but in practice is not that good at giving you progress information. Which is what progress indicators should do! At best this is a better indicator of how much work is being done, not how much progress is being made. Like a bunch of bleeps, bloops, and random LEDs toggling when a computer does work in a sci-fi TV show. I think this would be better if individual files being processed got removed unless they start taking too long. Keep the build-exe and build-lib steps around but make them a little sticky. When they complete, have it say "Done" and then remove things in groups or on a set interval. Don't change the number of lines too often and don't reorganize lines either. Generally, it should be easy to parse what's going on and frequent changes to the number of lines and how much information is on each makes that hard.

epage 2y ago
This is really good feedback! I'm looking at redesigning Cargo's output (after I wrap up other projects).

Groxx 2y ago

I wish all multi-task systems had a UI like this. It makes it SO much easier to spot where your biggest latencies are coming from, which is an excellent passive motivator to improve them. Silent / opaque progress just teaches people that they can't do anything about it except wait, so all it does is get worse and worse.

deleted 2y ago
[deleted]

epage 2y ago

I've seen guides for detecting terminal support for colors, hyperlinks, emoji, etc but not "fancy" terminal control like this. Anyone know of any? In one CLI I work on, some others are concerned about the long tail of users and wanting to make sure no one has a bad experience by default.

rockorager 2y ago
This isn't using any "fancy" terminal features, aside from the synchronized update sequences (which terminals that don't support typically will ignore, though windows has a special case where it must be ignored). That said, you can query the terminal for support for this sequence if you wanted to. Other than that, it's using standard ANSI sequences: `\r` to return the cursor to the beginning of the line, `\x1bM` (Reverse Index) to move the cursor up a single line (repeated n times), and then `\x1b[J` to clear the screen below the cursor position. All of these are sequences defined at least since the VT220, probably the VT100.
- epage 2y ago
  > `\x1bM` (Reverse Index) to move the cursor up a single line (repeated n times), and then `\x1b[J` to clear the screen below the cursor position. All of these are sequences defined at least since the VT220, probably the VT100. For whatever reason, the people I'm collaborating with have the impression these aren't universal enough. And of course termcaps is out of vogue.
  - saurik 2y ago
    Simultaneously caring about terminals that don't support basic movement but also refusing to use termcap seems like an unfortunate combination :(.

Filligree 2y ago

> The key insight I had here is that, since the end result must be displayed on a terminal screen, there is a reasonably small upper bound on how much memory is required, beyond which point the extra memory couldn't be utilized because it wouldn't fit on the terminal screen anyway. How large are your terminals? I'm not sure if this represents lines, or perhaps something else. But on occasion I need to copy things out of my terminal, scrolling while copying doesn't work well, and so I shrink the text size to, let's say, one or two pixels per letter. I suspect Zig won't survive this.

vanderZwan 2y ago
One pixel per letter on a 4k screen would be 3840 × 2160 chars at most, so around 8 megabytes if we assume one byte per "character tile" on the screen. I highly doubt Zig would die from this.
- Filligree 2y ago
  The article suggests it’s using a fixed-size buffer of 200 somethings. Seems worth checking, though I have no goose in this race.
  - vanderZwan 2y ago
    If I understand correctly then that's for the number of threads, although I also think that the sudden switch from terminal size back to the topic of threads is confusing so I agree that it's not entirely clear what the connection between the two is - or if there is none, that the structure of the article makes that confusing

Animats 2y ago

Not sure if this is a joke.

one-punch 2y ago

Reminds me of nix-output-monitor [1], for example see [2]. It makes it easy to understand how individual steps are progressing, and how individual steps relate to the overall plan. It enables me to locate expensive build steps, and possibly to avoid them if steps are failing. [1]: https://github.com/maralorn/nix-output-monitor https://github.com/maralorn/nix-output-monitor [2]: https://asciinema.org/a/7hJXH2iFLEkKxG1lL25lspqNn https://asciinema.org/a/7hJXH2iFLEkKxG1lL25lspqNn

deleted 2y ago

[deleted]

binary132 2y ago

One thing that the latest generation of languages has taught me (or that I have learned in the process?) is that “languages” aren’t “really a real thing” in and of themselves as much as they are merely composable APIs over compilers / interpreters (which are basically just dynamic compilers, or compilers are interpreters?), and compilers are an insane, fragmented dumpster fire with no or barely usable APIs. Only LLVM, GCC, TinyCC, and Terra admit this reality, neither LLVM nor GCC’s APIs are really user-oriented, and Terra isn’t really a realistic option for most people. I also personally don’t really feel that LLVM has accomplished (or can accomplish) its potential for a variety of reasons — most LLVM projects are forks or patches, although Zig has done well at this — and GCC has limitations as well although I’m very pleased with gccjit in Emacs. I’m very curious to see what the future holds especially if we can get an optionally-verifiable yet usable compiler API.