benesch

7 ms·

1.
by benesch 29d ago
Yeah, we've had a PlanetScale cluster since the early days that powers the dashboard experience (turbopuffer.com/dashboard). That's a fairly standard webapp (customers, orgs, users, API keys, namespace stats) that benefits fr
2.
by benesch 4mo ago
Yes, that’s right.
3.
by benesch 8mo ago
I agree our sample may not be representative but we try to stay focused on the current and next crop of tpuf customers rather than the software industry as a whole. So far "CI prohibits network access during tests" just hasn'
4.
by benesch 8mo ago
Thanks, appreciate this! Jotted down some notes on our roadmap.
5.
by benesch 8mo ago
Yep, we're well aware of the selection bias effects in product feedback. As we grow we're thinking about how to make our product more accessible to small orgs / hobby projects. Introducing a local dev environment may be part
6.
by benesch 8mo ago
The exact CPU depends on the region/cloud provider, but this Granite Rapids CPU is representative: https://www.intel.com/content/www/us/en/products/sku/240777/...
7.
by benesch 8mo ago
So I can note this down on our roadmap, what's the root of your requirement here? Supporting local dev without internet (airplanes, coffee shops, etc.)? Unit test speed? Something else?
8.
by benesch 8mo ago
My point is it's enough of a hassle to set up that I've yet to see that level of restriction in practice (across hundreds of CI systems).
9.
by benesch 8mo ago
> in many CI environments unit tests don't have network access, it's not purely a price consideration. I've never seen a hard block on network access (how do you install packages/pull images?) but I am sympathetic to
10.
by benesch 8mo ago
Often not a dealbreaker, actually! We can spin up new tpuf regions and procure dedicated interconnects to minimize latency to the on-prem network on request (and we have done this). When you're operating at the 100B scale, you're
11.
by benesch 8mo ago
For local dev + testing, we recommend just hitting the production turbopuffer service directly, but with a separate test org/API key: https://turbopuffer.com/docs/testing Works well for the vast majority of our cu
12.
by benesch 1y ago
It’s hard to overstate the amount of service Ian provided to the Go community, and the programming community at large. In addition to gccgo, Ian wrote the gold linker, has blogged prolifically about compiler toolchains, and maintains huge s
13.
by benesch 2y ago
Yes, the three major open table formats are all quite similar. When AWS launched S3 Tables last month I wrote a blog post with my first impressions: https://meltware.com/2024/12/04/s3-tables There may be more
14.
by benesch 2y ago
> I'm sure they'd quickly argue it's wire compatibility, but even then it's a slippery slope and wire compatible is left open to however the person wants to interpret it. I actually think that they'd argue they i
15.
by benesch 2y ago
> I liked the author's write-up, but as an old programmer take umbrage at the idea that changing your parser in the middle of a program is "crazy", we used to do this... well maybe not all the time... but with a greater fr
16.
by benesch 2y ago
Yes! I’m actively working on it, in fact. We’re waiting on the next release of the Rust `object_store` crate, which will bring support for S3’s native conditional puts. If you want to follow along: https://github.com/slatedb
17.
by benesch 2y ago
> Anecdotally I have had to do this in js a few times. I have never had to do this in Rust. Probably because Rust projects are likely to ship with fewer bugs. Still anecdotal, but I have worked on a large Rust codebase (Materialize) for
18.
by benesch 2y ago
Ah, I misunderstood! Yes, we may have invented that. I whipped up the cron job a few years back in response to concerns from our legal team. I’m not aware of any prior art for automatically advancing the change date for the BSL.
19.
by benesch 2y ago
We haven't benchmarked TimescaleDB, so I can't say. Results tend to vary heavily by workload, too. What I can say is that the research at the heart of Materialize ( https://dl.acm.org/doi/10.1145/2517349.2
20.
by benesch 2y ago
We did not originate the Business Source License (BSL/BUSL). It was originally developed by the folks behind MariaDB. Wikipedia has a good article that covers the history: https://en.wikipedia.org/wiki/Business_Sou
21.
by benesch 2y ago
Not to my knowledge. I believe TimescaleDB has their own incremental view maintenance engine.
22.
by benesch 2y ago
Those updates are not retroactive. They apply on a go forward basis. Each day's changes become Apache 2.0 licensed on that day four years in the future. For example, v0.28 was released on October 18, 2022, and becomes Apache 2.0 licens
23.
by benesch 2y ago
> It would be so much better if this were a Postgres extension instead. I've thought about this counterfactual a lot. (I'm a big part of the reason that Materialize was not built as a PostgreSQL extension.) There are two majo
24.
by benesch 2y ago
(Materialize CTO here.) > It's becoming more mainstream with Materialize, which is technically open-source, but they are quite aggressive with pushing their expensive cloud and offuscating on-prem usage. Quick but important clarific
25.
by benesch 3y ago
(Materialize CTO here.) Partial materialization is indeed Noria's major contribution to dataflow technology, and it's impressive stuff. But I want to call out that there are a number of techniques that folks use with Materialize t
26.
by benesch 3y ago
> Materialize no longer provide the latest code as an open-source software that you can download and try. It turned from a single binary design to cloud-only micro-service Materialize CTO here. Just wanted to clarify that Materialize has
27.
by benesch 3y ago
You can still do this. In fact, it’s even easier since the construction of the new mezzanine beneath Moynihan. There are glass windows on the mezzanine looking out onto the tracks beneath. If you’re good at differentiating train set for yo
28.
by benesch 4y ago
As mentioned in the blog post, clusters allow horizontal scalability and daisy chaining, so you can allocate more memory for your views even if you run up against the limits of how much memory you can fit on a single machine. We've got
29.
by benesch 4y ago
The biggest problem we've encountered with existing tools in the Kafka ecosystem (and the homegrown solutions that we've seen) is that nearly all of them sacrifice consistency. Debezium and most other Kafka Connect plugins will pr
30.
by benesch 4y ago
> well, minimal SQL support - there's no RETURNING, ON CONFLICT or json We support both `INSERT ... RETURNING` [0] and the `jsonb` data type [1]. The only feature in your list that we're actually missing is UPSERT (i.e., `INSER

More ›