inaseer

7 ms·

1.
by inaseer 3mo ago
Have you looked at model-based testing? One way to think of it as property-based testing for stateful system, though that's underselling it a little. It's surprisingly easy to come up models/specs for most stateful systems, i
2.
by inaseer 3mo ago
There are really two separate concerns here. The first is that some effect happens asynchronously, potentially interleaved with other operations. Whether a client observes completion by polling or by receiving an event from a message broker
3.
by inaseer 3mo ago
Property-based testing and model-based testing are closely related. Both ask the developer to state the expected behavior of a system (whether you call it a property, invariant, model, specification, or contract) and then validate that beha
4.
by inaseer 3mo ago
I've been working on a framework for writing executable specs in .NET called Accordant, developed at Microsoft and open sourced recently. Github: https://github.com/microsoft/accordant Docs: https://mic
5.
by inaseer 6mo ago
Soon!
6.
by inaseer 6mo ago
Yes, I understand why you made these design decisions. And I also agree that sticking to JS/TS keeps things simple (for humans, and LLMs). I generally default to the s and s' way of specifying things (in a C# property-based testin
7.
by inaseer 6mo ago
Bombadil takes a fresh approach to UI testing I haven't seen before: online monitoring through LTL formulas. Unlike model-checking (say by TLC), LTL formulas over here unfold in lock-step with the UI and allow users to express interest
8.
by inaseer 1y ago
+1. We have used Coyote/P# not just for model checking an abstract design (which no doubt is very useful) but testing real implementations of production services at Microsoft.
9.
by inaseer 2y ago
Microsoft Azure Storage | Seattle, WA | Onsite or Remote (US only) Our team in Azure Storage is working on applying automated reasoning and validation techniques to scalably generate hundreds of thousands of tests to check data integrity an
10.
So Farewell, TypeScript(twitter.com)
5 points by inaseer 3y ago|0 comments
11.
by inaseer 3y ago
My team used Coyote to test their distributed service against network race conditions. It requires a little bit of setup to ensure all components that typically run on separate machines can run in a single process, and inter-process communi
12.
by inaseer 4y ago
TLA+ and Coyote pair quite well actually. Verifying the high level design in TLA+ while checking subtle bugs as you translate that design to an implemention.
13.
by inaseer 4y ago
Check out Coyote (which is an evolution of P) and allows you to "model check" .NET code to find hard to find race condition-y bugs in your implementation. We have used it very successfully on a number of projects at Microsoft. I s
14.
by inaseer 5y ago
Coyote (concurrency exploration tool for .NET programs) can be used to do something "similar". My team often writes tests which set up focused concurrency between different APIs, the tests use Coyote to explore different ways that
15.
Finding interesting concurrency bugs through relatively simple mocks(twitter.com)
2 points by inaseer 5y ago|0 comments
16.
by inaseer 5y ago
Visualizations do help a lot when model checkers and concurrency schedule exploration tools like Coyote find bugs. Coyote include the ability to visualize the traces if you express your concurrency using actors (see https://micro
17.
by inaseer 5y ago
Loom's style of exploration (and that of aws shuttle mentioned below) can be quite effective. Coyote is the equivalent library in the .NET world (formerly known as P#) and comes with a bunch of exploration "schedulers" (from
18.
by inaseer 5y ago
There has been a bunch of content on Coyote shared before as well but the quality of animations and how they explain how it works under the hood in this introduction was excellent.
19.
by inaseer 5y ago
Hi HN, The tweet links to a couple of tutorials showing how to test an extremely simple CRUD service using Coyote to find concurrency bugs. Developers write simple unit tests whose concurrency is explored by Coyote to find bugs. You might b
20.
Finding concurrency bugs in .NET services using Coyote(twitter.com)
1 points by inaseer 5y ago|1 comments
21.
by inaseer 6y ago
Shameless plug: http://imnaseer.net/paxos-from-the-ground-up.html I worked on an explanation of Paxos where we start with a simple but incorrect implementation of the protocol. The bug is then fixed and the protocol refined
22.
by inaseer 6y ago
Concurrency is hard - this was a great investigation into the bug using TLA+ by the author which suggested simplifications to the code in addition to the bug fix. This reminded me of another reader/writer concurrency bug posted by Tom
23.
by inaseer 6y ago
This is a fair call-out. I'll like to share a counter-example where we 'model check' our real-world service written in C# with pretty decent effectiveness. As you rightly call out, the number of execution points around which
24.
by inaseer 6y ago
This is a good callout. P language evolved into the P# framework (as opposed to a language) which then evolved into Coyote ( https://microsoft.github.io/coyote/ ) Coyote allows developers to systematically test concurren
25.
by inaseer 6y ago
Neat to learn about thread sanitizer. It sounds similar to another tool from Microsoft Research called Torch ( https://www.microsoft.com/en-us/research/project/torch/ ) which automatically instruments bina
26.
by inaseer 6y ago
That's a great question. Stress testing, which is what you are suggesting helps, but is not super effective and often misses bugs. You need tools which can precisely control the task/go-routine scheduling during testing and system
27.
by inaseer 6y ago
That's a fair point. Having said that, it's also interesting to realize that most distributed systems (which are concurrent by their very nature) don't have that luxury. Our micro-services interact with databases, event queue
28.
by inaseer 6y ago
Concurrency is hard and we have very poor support for testing correctness of concurrent and distributed systems. Language abstractions help but they aren't nearly enough (as evidenced by this post). My team at Microsoft leverages Coyot
29.
by inaseer 6y ago
Looms looks quite interesting and is similar to Coyote in a lot of ways. The exploration engine used by Coyote is based on years of investigation by Microsoft Research so it scales nicely on real world programs. I believe the Coyote team is
30.
by inaseer 6y ago
We've used Coyote ( https://microsoft.github.io/coyote/ ) which offers an actor-based programming model, as well as as async/await programming model on top of .NET's Tasks to express the concurrency in our

More ›