rrnewton

6 ms·

1.
by rrnewton 4y ago
I thought it was very cool how gVisor is multi-backend (their “sentry” implemented vie either ptrace or kvm), which is pretty unusual with instrumentation tools. We could maybe have shared this logic to intercept syscalls and redirect them
2.
by rrnewton 4y ago
The distinction that we often have trouble getting across is between eliminating/controlling nondeterminism, vs recording what happens to occur. Undo, like rr, and Microsoft TTD, records basically all syscalls, but doesn’t determinize
3.
by rrnewton 4y ago
There’s a bit of shared lineage there. Joseph Devietti and I started working in this area together around 2015. And Joe had worked on the deterministic dOS at UW during his PhD (which led to Jinx). Another related effort is antithesis.com
4.
by rrnewton 4y ago
Yeah, it's interesting to think about persisting the state we would need to make the file system more sympatico with Hermit. If we were willing to have a daemon.... Meta develops this "watchman" tool that our build infrastruc
5.
by rrnewton 4y ago
Well, we don't prove the absence of concurrency bugs -- that would be more a job for formal verification, type systems, at the source level. But we can tell when our `--chaos` stress tests cease to produce crashes in reasonable numbers
6.
by rrnewton 4y ago
Yep, good point -- should have mentioned it. That's still roughly accurate because Hermit is, today, still ptrace-powered. I'll quote my reply from elsewhere about the WIP high-perf backend: > The `experimental/reverie-sa
7.
by rrnewton 4y ago
Thanks for all the questions! Whew, here goes. > How are you sanitizing the result of stat(), for example? Ah, that part's the same as it was described in the ASPLOS'20 paper ( https://dl.acm.org/doi/10.114
8.
by rrnewton 4y ago
Well, the starting datetime at the beginning of execution in the container is whatever you set it to: $ hermit run --epoch=2022-01-01T00:00:00Z /bin/date Fri Dec 31 16:00:00 PST 2021 We, somewhat eccentrically, put it in last mill
9.
by rrnewton 4y ago
Ah, at least you were in good hands with Michael Vollmer. Btw, he's now a prof at University of Kent in the UK ( http://recurial.com/ ).
10.
by rrnewton 4y ago
Well, probably not on device ;-). The underlying Reverie instrumentation layer works on ARM, but Hermit isn't ported yet, and we haven't touched RISC-V yet at all. (Contributions welcome!) One thing we haven't tried yet is
11.
by rrnewton 4y ago
Well, gettimeofday is a syscall, and we do intercept it (along with clock_gettime, clock_gettres, time, nanosleep, and the rdtsc instruction, even though that last one is not a syscall). When we intercept it, we report virtual time back to
12.
by rrnewton 4y ago
Yes, I'm very interested in that as well. I've been involved with the ACM Artifact Evaluation process which has been going on in several conferences for a while. https://www.acm.org/publications/policies/
13.
by rrnewton 4y ago
Alas the performance overhead in realtime is not great yet. It still uses ptrace currently, which often results in a multiple-X slowdown (but at least it doesn't "subscribe" to every syscall like strace does, because some
14.
by rrnewton 4y ago
Sure! So it really breaks down into two cases: internal and external networking relative to the container Hermit creates. (1) internal networking If you run a test like `rust/network_hello_world.rs` under Hermit, then the communication
15.
by rrnewton 4y ago
Sure! I think underpinning your question is a really subtle point there. And I think the answer is in the different purposes of regression testing and bug finding. In regression testing (CI), you're testing if the code introduced new
16.
by rrnewton 4y ago
Reverted a badly-timed breaking change that came through the sync system. Will fix it properly shortly (and add a Dockerfile and release tag). But for now you may have better luck on the main branch after that reversion, which yielded 6cb5
17.
by rrnewton 4y ago
(1) rr [formerly Mozilla rr] We're big fans of rr! Hermit is different in that creating a deterministic OS semantics is different than recording whatever nondeterministic behavior occurs under normal Linux. BUT, there's a lot of
18.
by rrnewton 4y ago
It's a good question. We would like to make it usable as a platform for dynamic analysis. The idea being that you can control all these external factors (like thread scheduling), find a crashing run, and then ask introspective questio
19.
by rrnewton 4y ago
Heh, I implore you to consider the engineers-eye view. We're tech geeks inside or outside of FAANG, with all the usual incentives. I've been the tech lead on this project for 5 years, through 2 companies, and of course I hope so
20.
by rrnewton 4y ago
Yes indeed. That concurrency testing capability is a pretty well-studied area and we implement a couple existing algorithms. The first is our adaptation of the PCT algorithm (ASPLOS'10 https://www.microsoft.com/en-us&#x
21.
by rrnewton 4y ago
Note that this is a follow-on project from the earlier Dettrace system, which was applied mainly to reproducible builds (as in the academic paper, https://dl.acm.org/doi/10.1145/3373376.3378519 , and presented to t
22.
Deterministic Linux for controlled testing and software bug-finding(developers.facebook.com)
167 points by rrnewton 4y ago|69 comments
23.
by rrnewton 4y ago
This has been the culmination of several years of work intercepting and sanitizing the Linux system call API. It's now open source.
24.
by rrnewton 7y ago
Hmm, why is this a good thing compared to just using Nix / nix-pkgs to manage JS packages?
25.
by rrnewton 7y ago
That really stupid naming was a function of fitting that table into one column ;-). Page limits.
26.
by rrnewton 7y ago
@firethief -- glad to hear you're interested in this idea too. Do you have an implementation for Re that we could learn more about? Maybe we could swap some ideas.
27.
by rrnewton 7y ago
Hi Kenton, TL;DR: we'd love to get some pointers on our Cap'n Proto usage -- let's set up a call between all us authors and you. > difficult to take this approach in an imperative model. Indeed, we are eventually ad
28.
by rrnewton 7y ago
One thing that might not be clear from the paper: while the Gibbon compiler currently handles a small subset of Haskell, we plan to deploy it by integrating with GHC. That is, we want to make it possible to compile a subset of your Haskell
29.
by rrnewton 7y ago
Our main focus of the Gibbon project is to make tree traversals very fast, and only secondarily to work with serialized data. CapnProto is a wonderful library interface for dealing with serialized data, but I think the usual library-vs-comp
30.
by rrnewton 7y ago
Well, if you have a big document collection or something, the faster you can process each individual document the more you can scale, right? I guess I'm not sure what scalability would mean here other than speeding up that individual,

More ›