huntaub

5 ms·

1.
by huntaub 3mo ago
This is the problem that we're building for at Archil [1]. Would be super curious to hear more about your use case (and specifically why you're not able to use the hyperscaler managed offerings). Feel free to email me at hleath [a
2.
Bash is the SQL for file systems(archil.com)
5 points by huntaub 5mo ago|0 comments
3.
S3 Files: the right product on the wrong foundation(archil.com)
4 points by huntaub 5mo ago|0 comments
4.
by huntaub 5mo ago
I don't think there's much for Amazon to gain from publishing these sorts of internal details. Amazon's services are used by developers who are looking to tightly optimize their usage. If Amazon were to publish detailed inter
5.
by huntaub 5mo ago
I imagine (hope) that they are doing some kind of intelligent read-ahead in the frontend servers to optimize for sequential reads that would avoid this looking terrible for applications.
6.
by huntaub 5mo ago
What does DuckDB need that NFS/SMB do not provide?
7.
by huntaub 5mo ago
Of course, you don't need to, this is just a way to opt-in to getting file semantics on top of S3. The purpose of S3 isn't to be cheap, it's to be simple.
8.
by huntaub 5mo ago
Notably, this is going to manage your data in it's native format (i.e. you can actually read-write the files out of the S3 bucket as if they were actual objects, mapping 1:1 to each file). The ZFS backend is (almost certainly) a block-
9.
by huntaub 5mo ago
This is pretty different than s3fs. s3fs is a FUSE file system that is backed by S3. This means that all of the non-atomic operations that you might want to do on S3 (including edits to the middle of files, renames, etc) are run on the mach
10.
by huntaub 6mo ago
howdy! two things on the archil front: 1. we're not NFS, we wrote our own protocol to get much better performance 2. we're planning on coming out with native branching this month, which should make these kinds of workloads much ea
11.
by huntaub 6mo ago
Well, I think this is what our company, Archil, is working on. We basically built an SSD clustering layer that proxies/caches/and assembles requests into object storage so that you can run a POSIX file system directly on top. Th
12.
by huntaub 7mo ago
We just released a driver that allows users of just-bash to attach a full Archil file system, synced to S3. This would let you run just-bash in an enrivonment where you don't have a full VM and get high-performance access to data that&
13.
by huntaub 7mo ago
It's 100% because the number of operations happening on Github has likely 100x'd since the introduction of coding agents. They built Github for one kind of scale, and the problem is that they've all of a sudden found themselv
14.
by huntaub 7mo ago
This turns out to be a more and more important primitive for companies who are building their own models [1]. [1] https://si.inc/posts/the-heap/
15.
Connect just-bash directly to data in S3 with Archil(archil.com)
1 points by huntaub 8mo ago|0 comments
16.
Why file systems are here to stay for agents(archil.com)
1 points by huntaub 8mo ago|0 comments
17.
by huntaub 8mo ago
It does! We see a lot of users choose Cloudflare's R2 because it has no egress fees and then pair it with Archil so they get fast, local caching of their content from wherever their compute lives.
18.
Archil: An elastic, scale-out file system that syncs to S3(archil.com)
4 points by huntaub 8mo ago|3 comments
19.
by huntaub 8mo ago
Our product is Archil [1], and we are building our service on top of a durable, distributed SSD storage layer. As a result, we have the ability to: (a) store and use data in S3 in its native format [not a block based format like the other s
20.
by huntaub 8mo ago
tl;dr it doesn't. I'm not sure what they're planning in this capacity (I haven't checked out sprites myself), but I would guess that it's going to be a function of "snapshots" as a mechanism to give multip
21.
by huntaub 8mo ago
These are basic POSIX features, but I think the high-level point that Kurt is trying to make is that building a FUSE file system signs you up for a nearly unlimited amount of compatibility work (if you want to support most applications) whe
22.
by huntaub 8mo ago
It depends on what level of FUSE you're working with. If you're running a FUSE adapter provided by a third party (Mountpoint, GCS FUSE), odds are that you aren't going to get great performance because it's going to have
23.
by huntaub 8mo ago
> FUSE is full of gotchas I want to go ahead and nominate this for the understatement of the year. I expect that 2026 is going to be filled with people finding this out the hard way as they pivot towards FUSE for agents.
24.
by huntaub 8mo ago
Yes, I'm working in the space too. I think it's fine to do benchmarks, I don't think it's necessary to immediately post them any time a competitor comes up on HN. I don't want to see the cloud storage sector turn as
25.
by huntaub 8mo ago
> * The bloody .nfs000000 files. I think this might be fixed by NFSv4 but it seems like nobody actually uses that. (Not helped by the fact that CentOS 7 is considered "modern" to EDA people.) Unfortunately, NFSv4 also has the s
26.
by huntaub 8mo ago
Respect to your work on ZeroFS, but I find it kind of off-putting for you to come in and immediately put down JuiceFS, especially with benchmark results that don't make a ton of sense, and are likely making apples-to-oranges comparison
27.
by huntaub 8mo ago
I think that number 4 is the big one. AWS only has so much capacity to work on new hardware types, and the number of companies who want to work with on-device NVMe are WAY smaller than then number of companies who just want to slap Kubernet
28.
by huntaub 8mo ago
Yeah, I would keep in mind that not everyone is optimmizing for $/perf, some use cases (where data is colder) are optimizing for $/GiB stored.
29.
by huntaub 9mo ago
> At scale, we still think scripts will be better for their cost, performance, and debuggability aspects This actually makes a ton of sense to me in lots of the LLM contexts (e.g. seeing how we are starting to prefer having LLMs write on
30.
by huntaub 9mo ago
This is a super interesting product, guys. I get that agents aren't great for everything right now, but I'd expect that they'll continue to improve over time (like everything in the LLM space). How do you see the product evol

More ›