5 ms·

I wonder if it would work do design something that was able to say test 1: do step 1, assert step 1 test 2: requires: test 1 do

I wonder if it would work do design something that was able to say test 1: do step 1, assert step 1 test 2: requires: test 1 do step 2, assert step 2 test 3: requires: test 2 do step 3, assert step 3 test 4: requires: test 2 do step 3, assert step 3 If the assertion of each test doesn't change any state, that might make things easier to read. Though, given that I haven't spent much time pondering it, I expect it could have it's own problems. But it could also do things like skip test 3 if tests 2 or 1 failed - because it knows about the relationship.

troupo 1mo ago
Yup. I'd love to have dependency declarations for tests like this.
- jaggederest 1mo ago
  That's... A thing in many frameworks. If it's not in yours, you could add it. Forgive an old man some ruby: it 'relies on mobile setup defined elsewhere', :mobile => true do # test that relies on mobile setup here end
brabel 1mo ago
Yes that is the way to do it, in Spock that’s what @Stepwise does.
mamcx 1mo ago
This make too much sense! In special when testing against a DB.
latencyharbor 1mo ago
A dependency graph can help with reporting, but I’d avoid making “test 1 ran successfully” part of test 2’s fixture. Those are two separate ideas: 1. Execution dependency: if a cheap contract test fails, skip expensive downstream tests because their results won’t be informative. 2. State dependency: test 2 consumes state left behind by test 1. The first can be useful. The second tends to create order dependence, awkward retries, and failures that are hard to reproduce when a CI runner shards or parallelizes the suite. A safer model is a DAG of independently reproducible tests: each node declares prerequisites for scheduling/reporting, but creates or restores its own input state. Then a downstream test can be marked “blocked by X” rather than failed, while still being runnable by itself when debugging. That distinction also keeps the dependency metadata from becoming a hidden setup mechanism.
drdexebtjl 1mo ago
The hard parts to design about this, imo, are: - How to reconcile this with tests that execute many times with varying input data. You’d need some way to express requirements with specific inputs or shared inputs. - Passing state between test dependencies. - When, if ever, it’s fine to share step results between tests. If tests B and C require A, can you run A just once? Not always, but you should be able to when it’s safe. I don’t think I’ve ever used a test framework that gets these things right.
- RHSeeger 1mo ago
  > Passing state between test dependencies Actually, I wasn't even thinking about passing state. I was thinking about shared setup steps. I'm perfectly happy for the same steps to run for each test - as long as each test doesn't need to list the steps (when they're setup steps not directly related to the thing being tested)
  - drdexebtjl 1mo ago
    For example, suppose you want to write a test for the shipOrder(orderId) function, and you want it to depend on the test for the placeOrder(shoppingCart) -> OrderId function. Even if you are fine calling placeOrder twice, once for the placeOrder test and once for the placeOrderAndShipOrder test, you still need the placeOrder test to provide an order ID to the second test, and not just a confirmation that it completed successfully.
avensec 1mo ago
The pattern can work, but the domain matters, the test type matters (unit, integration, ui) and the trade-off associated matter. e.g. If I am running a long-running UI-test scenario, I absolutely don't want test-5 to walk through 80% of the UI that was already exercised in tests 1-4. I am creating test coupling, but I'm saving cost/time by doing so. But, you'll also hear why not to do this, because it creates test coupling / breaks atomic tests, which is generally seen as bad. If that is a local integration test and those early steps run is millis? Then maybe we keep things uncoupled to allow the system to exercise the pathways without explicit expectations.
- RHSeeger 1mo ago
  The question was less about speed and more about not having the same code duplicated over and over across tests. Which is how I read the article talking about it. Allowing one test to "depend" on another makes it clear they use the same setup (presumably with the second test going "a bit further", but not necessarily). I wouldn't have a problem with something like test-1: setup: do-the-thing verification assert-the-thing-happened test-2 setup: depends-on: test-1 // tells it to run test-1's setup do-the-next-thing verification assert-the-next-thing-happened The format is awful, but the idea is that most tests are of the form GIVEN Some initial setup WHEN I run command THEN The result of that command is what is expected And, in that context, the GIVEN frequently contains noise not directly related to understanding what is being tested. I actually use the GIVEN/WHEN/THEN keywords in my tests, to make them easier to read

troupo 1mo ago

Yup. I'd love to have dependency declarations for tests like this.

jaggederest 1mo ago
That's... A thing in many frameworks. If it's not in yours, you could add it. Forgive an old man some ruby: it 'relies on mobile setup defined elsewhere', :mobile => true do # test that relies on mobile setup here end

brabel 1mo ago

Yes that is the way to do it, in Spock that’s what @Stepwise does.

mamcx 1mo ago

This make too much sense! In special when testing against a DB.

latencyharbor 1mo ago

A dependency graph can help with reporting, but I’d avoid making “test 1 ran successfully” part of test 2’s fixture. Those are two separate ideas: 1. Execution dependency: if a cheap contract test fails, skip expensive downstream tests because their results won’t be informative. 2. State dependency: test 2 consumes state left behind by test 1. The first can be useful. The second tends to create order dependence, awkward retries, and failures that are hard to reproduce when a CI runner shards or parallelizes the suite. A safer model is a DAG of independently reproducible tests: each node declares prerequisites for scheduling/reporting, but creates or restores its own input state. Then a downstream test can be marked “blocked by X” rather than failed, while still being runnable by itself when debugging. That distinction also keeps the dependency metadata from becoming a hidden setup mechanism.

drdexebtjl 1mo ago

The hard parts to design about this, imo, are: - How to reconcile this with tests that execute many times with varying input data. You’d need some way to express requirements with specific inputs or shared inputs. - Passing state between test dependencies. - When, if ever, it’s fine to share step results between tests. If tests B and C require A, can you run A just once? Not always, but you should be able to when it’s safe. I don’t think I’ve ever used a test framework that gets these things right.

RHSeeger 1mo ago
> Passing state between test dependencies Actually, I wasn't even thinking about passing state. I was thinking about shared setup steps. I'm perfectly happy for the same steps to run for each test - as long as each test doesn't need to list the steps (when they're setup steps not directly related to the thing being tested)
- drdexebtjl 1mo ago
  For example, suppose you want to write a test for the shipOrder(orderId) function, and you want it to depend on the test for the placeOrder(shoppingCart) -> OrderId function. Even if you are fine calling placeOrder twice, once for the placeOrder test and once for the placeOrderAndShipOrder test, you still need the placeOrder test to provide an order ID to the second test, and not just a confirmation that it completed successfully.

avensec 1mo ago

The pattern can work, but the domain matters, the test type matters (unit, integration, ui) and the trade-off associated matter. e.g. If I am running a long-running UI-test scenario, I absolutely don't want test-5 to walk through 80% of the UI that was already exercised in tests 1-4. I am creating test coupling, but I'm saving cost/time by doing so. But, you'll also hear why not to do this, because it creates test coupling / breaks atomic tests, which is generally seen as bad. If that is a local integration test and those early steps run is millis? Then maybe we keep things uncoupled to allow the system to exercise the pathways without explicit expectations.

RHSeeger 1mo ago
The question was less about speed and more about not having the same code duplicated over and over across tests. Which is how I read the article talking about it. Allowing one test to "depend" on another makes it clear they use the same setup (presumably with the second test going "a bit further", but not necessarily). I wouldn't have a problem with something like test-1: setup: do-the-thing verification assert-the-thing-happened test-2 setup: depends-on: test-1 // tells it to run test-1's setup do-the-next-thing verification assert-the-next-thing-happened The format is awful, but the idea is that most tests are of the form GIVEN Some initial setup WHEN I run command THEN The result of that command is what is expected And, in that context, the GIVEN frequently contains noise not directly related to understanding what is being tested. I actually use the GIVEN/WHEN/THEN keywords in my tests, to make them easier to read