6 ms·

Show HN: A Swiss army knife for testing HTTP from the terminal

110 points by hassy 4y ago

Hiya HN! Just released Artillery Probe - a Swiss army knife for testing HTTP from the CLI. Think mini-curl with better UX for common use-cases, plus a couple of extra features. Would love for you to try it and give some feedback! https://www.artillery.io/blog/swiss-army-knife-for-http-testing and: npm install artillery@latest What does it do? - First of all, it's a HTTP client! It does all the usual stuff you'd expect from a HTTP client... HTTP methods, request bodies, custom headers, forms, Basic Auth etc. - Got some JSON or XML back? It'll pretty-print it, and syntax highlight it for you. - It'll show you request waterfalls like this one: (inspired by httpstat) DNS Lookup | TCP Connection | SSL Handshake | Time to First Byte | Content Transfer 56ms | 14ms | 19ms | 181ms | 88ms | | | | | | 56ms | | | | 70ms | | | 89ms | | 270ms | total:358ms - JSON responses can be queried and sliced and diced with JMESPath (same syntax as AWS CLI) - no need to reach for jq. XML & HTML may be queried with a jQuery-like syntax too. - You can set expectations on the response, e.g. have the CLI check that the response is a 200, or that a certain header is set, and exit with non-zero code if not. Super handy for quick acceptance testing. We've got lots of ideas for improvements, but would love to hear what you think!

ushakov 4y ago
does this require Node.JS? edit: it looks like you need to install 1276 npm dependencies to use this tool https://github.com/artilleryio/artillery/network/dependencies https://github.com/artilleryio/artillery/network/dependencie... i'll pass, the risk is unacceptable for value it offers, especially in corporate environments
- hassy 4y ago
  sure, that’s a valid concern in some environments. fwiw we use Snyk.io for dependency scanning
  - ushakov 4y ago
    no, this is a valid concern in all environments, but in some environments this is more of a concern > Snyk.io for dependency scanning so what? does the CLI update itself automatically?
    - hassy 4y ago
      Yes, those dependency trees can be large. Yes, supply chain attacks are a real threat. But Node isn’t that different than Python or Ruby in that regard. How far down the stack do you personally choose to go? I trust you’re familiar with that famous paper published by a certain mr Thompson in the mid-80s? The world is a big place. There’s a lot of software out there written in Node.js, used happily and productively by millions of developers, many of them in corporate environments. Given the opinions you expressed elsewhere in the thread here I think it’s clear that this tool is not for you. I hope no one is forcing you to use it.
- jsd1982 4y ago
  This is essentially what I was referring to with my comment below. If you can improve the installation experience such that I don't have to care that you developed with node.js, so much the better. If I were a node.js developer I surely wouldn't want my global package name space polluted like that. npm is not for end users.
  - ushakov 4y ago
    the solution is not write CLIs in JavaScript, unless you have a JS-library that you also want to make available as a CLI packaging Node in a binary won't solve the problem of JavaScript's plagued ecosystem and in fact i'd rather prefer to be able to mitigate the issues/update packages myself rather than waiting for a new binary to release
    - jsd1982 4y ago
      I agree with your conclusion to not write cli tools in Javascript. However, packaging node in a binary is pretty much what electron does today, albeit with a GUI. Regardless, it's not a problem I wish to spend any effort to solve since it's not my problem and I've (we've) already dismissed the premise (writing CLI tools in js) lol.
      - spoiler 4y ago
        Another option is using Deno!
- ushakov 4y ago
  honest question to the authors: why cannot this tool not be written in JavaScript? what technically prevents you from designing the tool independent of the Node.JS and JavaScript ecosystem?
  - throwawayboise 4y ago
    Maybe it's what they know. Common advice for startups is to not get mired in language/framework debates: use what you already know.
- deleted 4y ago
  [deleted]
- heywoodlh 4y ago
  > Think mini-curl with better UX for common use-cases, plus a couple of extra features. > it looks like you need to install 1276 npm dependencies to use this tool Sincere question (probably highlights my lack of understanding Node package management): how can this be a "mini" curl with that many dependencies?
  - hassy 4y ago
    “mini” in the sense that it doesn’t do everything that curl does. curl does a whole lot. this tool focuses on more common use cases and makes them friendlier. plus with request waterfalls and assertions it does things curl can’t do. the number of packages is a bit of a misnomer anyway. Artillery Probe is part of Artillery which does load testing, with multiple protocols, support for multi-step scenarios, publishing to a variety of monitoring systems (Datadog, Prometheus etc) and more. That’s what most of those packages enable.
    - heywoodlh 4y ago
      I see. Thanks for the info and thanks for creating/sharing a great tool!
- Uptrenda 4y ago
  1276 deps... how is that even possible for a simple HTTP fetcher. Does node have no standard library? actually amazing.
hassy 4y ago
clickable url: https://www.artillery.io/blog/swiss-army-knife-for-http-testing https://www.artillery.io/blog/swiss-army-knife-for-http-test...
arinlen 4y ago
Other than pretty printing the output, what does this offer that curl doesn't already do?
- hassy 4y ago
  A few things :) Syntax highlighting Request performance waterfalls Built-in JMESPath queries for JSON (same syntax as in AWS CLI). XML and HTML can be queried too. Ability to set checks on the response. This one is super handy for quick acceptance tests.
jsd1982 4y ago
> npm install Sorry, lost me at this part. Have a better installation story like a single executable. Beyond that, what does this do that curl, wget, et al do not? How is your interface better than those? Is it robust in the face of errors? What http protocol versions does it support?
- Eduard 4y ago
  What makes this a worse "installation story" than e.g. apt install, pip install, git clone && ./configure && make && make install?
- ushakov 4y ago
  > what does this do that curl, wget, et al do not they don’t get VC-funded by YCombinator, do they? /s
- hassy 4y ago
  if you’re on a Mac, Artillery has a Homebrew formula too. We have an official Docker image too. Other ways of installing are on the roadmap too (self contained tarballs and binaries). As to what it does better than curl/wget. Probe is geared towards interactive use with HTTP APIs. So you get syntax highlighting and pretty printing for JSON, built-in querying, request waterfall visualizations, and ability to set expectations on every response. HTTP/2 is supported.
  - jsd1982 4y ago
    https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... Does artillery properly escape the environment variables into json strings? I don't see how it could if you're using bash expansion in double quotes. This feature would only work with string values without spaces or special characters. Don't really see any improvement here. https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... So by embedding a json query evaluator in your http client, this means that every time I iterate on my JMESPath query I'm making a new request? Is there an offline mode? What if my Json file is 200MiB? Or if the request was not idempotent? Just saying there's good reasons why json query features are not typically built into one-shot http request tools. However, if your tool was not one-shotand you had an interactive session and cached/stored the entire result and then allow the user to interactively query it, then I could see some better value here.
dj_mc_merlin 4y ago
Nice project. How are the request waterfalls done?
- hassy 4y ago
  thank you! we just grab the timings info for the request from the underlying HTTP library, and sprinkle some ASCII art on top. That part was inspired by httpstat [1] We want to extend those with support for Server-Timing next, and also Core Web Vitals [3] (via Playwright) for web pages. 1. https://github.com/reorx/httpstat https://github.com/reorx/httpstat 2. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se... 3. https://web.dev/vitals/ https://web.dev/vitals/
ovebepari 4y ago
It would take me something extraordinary to move away from curl
- rgoulter 4y ago
  In Emacs, I found the restclient plugin quite useful. https://github.com/pashky/restclient.el https://github.com/pashky/restclient.el I imagine other editors have similar plugins.
  - throwawayboise 4y ago
    Awesome! This comment made reading this thread worthwhile.
- blacksmith_tb 4y ago
  Sure, if you've got muscle memory for all the flags (plus it lets you reason about all the other programs that pipe stuff to curl). I personally find myself using httpie [1] at least as much. 1: https://httpie.io/ https://httpie.io/
  - deleted 4y ago
    [deleted]
  - ratorx 4y ago
    I think curlie is the middle ground and the best (IMO) in the space. A nicer CLI for interactive use and can use and output curl flags for scripting.
    - jicea 4y ago
      As a maintainer, I feel obligated to mention Hurl [1], a tool for testing HTTP with plain text and curl. It’s a wrapper around libcurl, in a single binary, with syntactic sugar for asserts. You can use it also like curl to give you an HTTP output. [1] https://hurl.dev https://hurl.dev
numlock86 4y ago
Nice idea. NodeJS and thousands of NPM dependencies are a big no-go with a red flag on top, though. One of the things something like Go probably would have been a better choice for. See "bombardier" for example.
- hassy 4y ago
  Bombardier is cool, but serves a very different use case. I mentioned elsewhere in the comments that we have a Docker image, and are working on other methods of installing the CLI to alleviate some of these dependency-related concerns. Getting side tracked here, but there seems to be a common sentiment when it comes to Node.js that it's uniquely insecure. Node.js has indeed had some unfortunate press when it comes to supply-chain security, but every other runtime is susceptible to those attacks (PiPy, Gems, Maven, Rust Crates). Ultimately of course, if you choose to avoid using any software built on top of those stacks, that's your choice. Artillery specifically is no different to any other Node.js-based project in how large the dependency tree is. VSCode for instance is used by millions of developers has 1.6k dependencies [1]. 1. https://github.com/microsoft/vscode/network/dependencies https://github.com/microsoft/vscode/network/dependencies
  - numlock86 4y ago
    > Bombardier is cool, but serves a very different use case. Yes, of course. I was just taking an arbitrary example from my common HTTP toolbox. > VSCode for instance is used by millions of developers has 1.6k dependencies [1]. VSCode is maintained by Microsoft plus a huge community and they are transparent about their process in regards of auditing/freezing dependencies.

110 points by hassy 4y ago

ushakov 4y ago

does this require Node.JS? edit: it looks like you need to install 1276 npm dependencies to use this tool https://github.com/artilleryio/artillery/network/dependencies https://github.com/artilleryio/artillery/network/dependencie... i'll pass, the risk is unacceptable for value it offers, especially in corporate environments

hassy 4y ago
sure, that’s a valid concern in some environments. fwiw we use Snyk.io for dependency scanning
- ushakov 4y ago
  no, this is a valid concern in all environments, but in some environments this is more of a concern > Snyk.io for dependency scanning so what? does the CLI update itself automatically?
  - hassy 4y ago
    Yes, those dependency trees can be large. Yes, supply chain attacks are a real threat. But Node isn’t that different than Python or Ruby in that regard. How far down the stack do you personally choose to go? I trust you’re familiar with that famous paper published by a certain mr Thompson in the mid-80s? The world is a big place. There’s a lot of software out there written in Node.js, used happily and productively by millions of developers, many of them in corporate environments. Given the opinions you expressed elsewhere in the thread here I think it’s clear that this tool is not for you. I hope no one is forcing you to use it.
jsd1982 4y ago
This is essentially what I was referring to with my comment below. If you can improve the installation experience such that I don't have to care that you developed with node.js, so much the better. If I were a node.js developer I surely wouldn't want my global package name space polluted like that. npm is not for end users.
- ushakov 4y ago
  the solution is not write CLIs in JavaScript, unless you have a JS-library that you also want to make available as a CLI packaging Node in a binary won't solve the problem of JavaScript's plagued ecosystem and in fact i'd rather prefer to be able to mitigate the issues/update packages myself rather than waiting for a new binary to release
  - jsd1982 4y ago
    I agree with your conclusion to not write cli tools in Javascript. However, packaging node in a binary is pretty much what electron does today, albeit with a GUI. Regardless, it's not a problem I wish to spend any effort to solve since it's not my problem and I've (we've) already dismissed the premise (writing CLI tools in js) lol.
    - spoiler 4y ago
      Another option is using Deno!
ushakov 4y ago
honest question to the authors: why cannot this tool not be written in JavaScript? what technically prevents you from designing the tool independent of the Node.JS and JavaScript ecosystem?
- throwawayboise 4y ago
  Maybe it's what they know. Common advice for startups is to not get mired in language/framework debates: use what you already know.
deleted 4y ago
[deleted]
heywoodlh 4y ago
> Think mini-curl with better UX for common use-cases, plus a couple of extra features. > it looks like you need to install 1276 npm dependencies to use this tool Sincere question (probably highlights my lack of understanding Node package management): how can this be a "mini" curl with that many dependencies?
- hassy 4y ago
  “mini” in the sense that it doesn’t do everything that curl does. curl does a whole lot. this tool focuses on more common use cases and makes them friendlier. plus with request waterfalls and assertions it does things curl can’t do. the number of packages is a bit of a misnomer anyway. Artillery Probe is part of Artillery which does load testing, with multiple protocols, support for multi-step scenarios, publishing to a variety of monitoring systems (Datadog, Prometheus etc) and more. That’s what most of those packages enable.
  - heywoodlh 4y ago
    I see. Thanks for the info and thanks for creating/sharing a great tool!
Uptrenda 4y ago
1276 deps... how is that even possible for a simple HTTP fetcher. Does node have no standard library? actually amazing.

hassy 4y ago

clickable url: https://www.artillery.io/blog/swiss-army-knife-for-http-testing https://www.artillery.io/blog/swiss-army-knife-for-http-test...

arinlen 4y ago

Other than pretty printing the output, what does this offer that curl doesn't already do?

hassy 4y ago
A few things :) Syntax highlighting Request performance waterfalls Built-in JMESPath queries for JSON (same syntax as in AWS CLI). XML and HTML can be queried too. Ability to set checks on the response. This one is super handy for quick acceptance tests.

jsd1982 4y ago

> npm install Sorry, lost me at this part. Have a better installation story like a single executable. Beyond that, what does this do that curl, wget, et al do not? How is your interface better than those? Is it robust in the face of errors? What http protocol versions does it support?

Eduard 4y ago
What makes this a worse "installation story" than e.g. apt install, pip install, git clone && ./configure && make && make install?
ushakov 4y ago
> what does this do that curl, wget, et al do not they don’t get VC-funded by YCombinator, do they? /s
hassy 4y ago
if you’re on a Mac, Artillery has a Homebrew formula too. We have an official Docker image too. Other ways of installing are on the roadmap too (self contained tarballs and binaries). As to what it does better than curl/wget. Probe is geared towards interactive use with HTTP APIs. So you get syntax highlighting and pretty printing for JSON, built-in querying, request waterfall visualizations, and ability to set expectations on every response. HTTP/2 is supported.
- jsd1982 4y ago
  https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... Does artillery properly escape the environment variables into json strings? I don't see how it could if you're using bash expansion in double quotes. This feature would only work with string values without spaces or special characters. Don't really see any improvement here. https://www.artillery.io/blog/swiss-army-knife-for-http-testing#post-some-json https://www.artillery.io/blog/swiss-army-knife-for-http-test... So by embedding a json query evaluator in your http client, this means that every time I iterate on my JMESPath query I'm making a new request? Is there an offline mode? What if my Json file is 200MiB? Or if the request was not idempotent? Just saying there's good reasons why json query features are not typically built into one-shot http request tools. However, if your tool was not one-shotand you had an interactive session and cached/stored the entire result and then allow the user to interactively query it, then I could see some better value here.

dj_mc_merlin 4y ago

Nice project. How are the request waterfalls done?

hassy 4y ago
thank you! we just grab the timings info for the request from the underlying HTTP library, and sprinkle some ASCII art on top. That part was inspired by httpstat [1] We want to extend those with support for Server-Timing next, and also Core Web Vitals [3] (via Playwright) for web pages. 1. https://github.com/reorx/httpstat https://github.com/reorx/httpstat 2. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se... 3. https://web.dev/vitals/ https://web.dev/vitals/

ovebepari 4y ago

It would take me something extraordinary to move away from curl

rgoulter 4y ago
In Emacs, I found the restclient plugin quite useful. https://github.com/pashky/restclient.el https://github.com/pashky/restclient.el I imagine other editors have similar plugins.
- throwawayboise 4y ago
  Awesome! This comment made reading this thread worthwhile.
blacksmith_tb 4y ago
Sure, if you've got muscle memory for all the flags (plus it lets you reason about all the other programs that pipe stuff to curl). I personally find myself using httpie [1] at least as much. 1: https://httpie.io/ https://httpie.io/
- deleted 4y ago
  [deleted]
- ratorx 4y ago
  I think curlie is the middle ground and the best (IMO) in the space. A nicer CLI for interactive use and can use and output curl flags for scripting.
  - jicea 4y ago
    As a maintainer, I feel obligated to mention Hurl [1], a tool for testing HTTP with plain text and curl. It’s a wrapper around libcurl, in a single binary, with syntactic sugar for asserts. You can use it also like curl to give you an HTTP output. [1] https://hurl.dev https://hurl.dev

numlock86 4y ago

Nice idea. NodeJS and thousands of NPM dependencies are a big no-go with a red flag on top, though. One of the things something like Go probably would have been a better choice for. See "bombardier" for example.

hassy 4y ago
Bombardier is cool, but serves a very different use case. I mentioned elsewhere in the comments that we have a Docker image, and are working on other methods of installing the CLI to alleviate some of these dependency-related concerns. Getting side tracked here, but there seems to be a common sentiment when it comes to Node.js that it's uniquely insecure. Node.js has indeed had some unfortunate press when it comes to supply-chain security, but every other runtime is susceptible to those attacks (PiPy, Gems, Maven, Rust Crates). Ultimately of course, if you choose to avoid using any software built on top of those stacks, that's your choice. Artillery specifically is no different to any other Node.js-based project in how large the dependency tree is. VSCode for instance is used by millions of developers has 1.6k dependencies [1]. 1. https://github.com/microsoft/vscode/network/dependencies https://github.com/microsoft/vscode/network/dependencies
- numlock86 4y ago
  > Bombardier is cool, but serves a very different use case. Yes, of course. I was just taking an arbitrary example from my common HTTP toolbox. > VSCode for instance is used by millions of developers has 1.6k dependencies [1]. VSCode is maintained by Microsoft plus a huge community and they are transparent about their process in regards of auditing/freezing dependencies.