points by manmal 4 days ago | 0 comments
Why not use gherkin given/when/then stories, and build E2E runners and maybe also unit tests for them? I don’t understand why a new spec format is necessary, which forever binds you to using agents for evaluation. Deterministic tests also have the advantage that they are harder to cheat on. If they are green, they give me a strong signal I can trust.
[–] nickabe 3 days ago
[–] manmal 3 days ago
But good gherkin tests encode pretty well what I’m trying to achieve, because they can be super high level - any flight level I want them to be. The only precondition is that the test env can deterministically measure the conditions.
Given the user is logged in
And the XYZ endpoint is offline
And the dashboard is visible
When the user clicks on the endpoint status
Then an overview of XYZ endpoint errors is shown
Ultimately, you spec out these tests and create deterministic code around them - but that doesn't mean that the tests didn't drift from the intended spec.
I think I agree with the author in that what you were TRYING to achieve is much more important than what you actually achieved (in code or tests) and actually using the product is your best defense to see if code / spec has drifted.