mfreed

6 ms·

1.
by mfreed 6mo ago
github.com/timescale/tigerfs
2.
by mfreed 6mo ago
Hey, author here. Just noticed this. Yes, TigerFS works well with local PG - in fact, all of my testing framework uses this setup. Big upside of this is latency; main downside is that such a deployment is primarily "single user mode.&
3.
by mfreed 11mo ago
None of this. It's in the blog post in a lot of detail =) The 5ms write latency is because the backend distributed block storage layer is doing synchronous replication to multiple servers for high availability and durability before ac
4.
by mfreed 11mo ago
Currently support multi-AZ clusters and multi-region disaster recovery (continuous PITR between regions). We're continuing to evaluate demand for multi-region clusters, love to hear from you.
5.
by mfreed 11mo ago
I'm not aware of any published source for this time limit, nor ways to reduce it. The docs do say, however, "If the volume has been impaired for more than 20 minutes, you can contact the AWS Support Center." [0] which suggest
6.
by mfreed 11mo ago
Tiger Cloud certainly continues to run on AWS. We have built it to rely on fairly low-level AWS primitives like EC2, EBS, and S3 (as opposed to some of the higher-level service offerings). Our existing Postgres fleet, which uses EBS for sto
7.
by mfreed 11mo ago
A few datapoints that might help frame this: - EBS typically operates in the millisecond range. AWS' own documentation suggests "several milliseconds"; our own experience with EBS is 1-2 ms. Reads/writes to local disk al
8.
Replacing EBS and Rethinking Postgres Storage from First Principles(tigerdata.com)
105 points by mfreed 11mo ago|59 comments
9.
by mfreed 2y ago
TimescaleDB primarily serves operational use cases: Developers building products on top of live data, where you are regularly streaming in fresh data, and you often know what many queries look like a priori, because those are powering your
10.
by mfreed 2y ago
Our experience is that Clickhouse and Timescale are designed for different workloads, and that Timescale is optimized for many of the time-series workloads people use in production: - https://www.timescale.com/blog/what
11.
by mfreed 3y ago
All depends on the benchmark :) https://www.timescale.com/blog/what-is-clickhouse-how-does-i...
12.
by mfreed 3y ago
That PoC benchmark didn't turn on Timescale's columnar compression, which every real deployment uses. So misleading at best. (Timescaler)
13.
by mfreed 3y ago
Check out how TimescaleDB adds columnar compression to PostgreSQL, typically saving 95% of storage overhead: https://www.timescale.com/blog/building-columnar-compression...
14.
by mfreed 3y ago
https://www.timescale.com/blog/how-we-scaled-postgresql-to-3... Staring at a >trillion rows in a TimescaleDB hypertable on PostgreSQL.
15.
by mfreed 3y ago
Also, Timescale similarly introduced S3 for bottomless data tiering: https://www.timescale.com/blog/expanding-the-boundaries-of-p...
16.
by mfreed 3y ago
Can you say more about "dynamic labels"? Do you just mean that as you evolve, you want to add a new type of "key-value" pair? The most common approach here is just to store the step of "dynamic" labels in JSON
17.
by mfreed 3y ago
If you are open, would love to hear more about some of the challenges you had with Timescale, esp. with your workload. mike (at) timescale or DM on twitter?
18.
by mfreed 4y ago
Do you use AWS Athena as the "database/serving layer" for customer-facing applications where performance matters? Our experience is that isn't a common use case for Athena, while it is the primary use case for TimescaleD
19.
by mfreed 4y ago
Very much agree. Even without this new capability of S3-based storage, Timescale's native columnar compression often gets like 95% storage reduction, even while staying fully in the PostgreSQL ecosystem. (We often see our users amazed
20.
by mfreed 4y ago
This is a great observation. As you point out, this was designed for the workload patterns we typically see with time-series, events, and analytical data, where the query (& insert) patterns differ across time. So I agree that it's
21.
by mfreed 4y ago
"Can I use Timescale to provide a SaaS service that collects application traces, and I provide a DSL to query the database that is not exposing the DB directly?" Yes you can. (Timescale co-founder here) The vast majority of peopl
22.
by mfreed 4y ago
TimescaleDB supports compression for all data types, it just uses type-aware compression algorithms that it applies automatically/transparently based on typing. - Gorilla compression for floats - Delta-of-delta + Simple-8b with run-len
23.
by mfreed 4y ago
We see those types of queries commonly in TimescaleDB. And, for example, both compression and "horizontal" scale out has ways where you can optimize your code for these types of analytical queries. More concrete, we see a lot of
24.
by mfreed 4y ago
I'm not sure what "recent license changes" you are referring to? The Timescale License was introduced in late 2018, although we never _relicensed_ any of our Apache 2 code, we only created a space for _new_ capabilities to be
25.
by mfreed 4y ago
We actually do this within Timescale Cloud, and it's amazing. It allows us to cohort performance data across data stored in others microservice databases (e.g., by account types, projects, billing data, etc.). JOINs across foreign data
26.
by mfreed 4y ago
Actually, try https://slack.timescale.com/ instead if you are signing up for the first time. The link above is the right slack group, but you need to go in through the URL I provided to get the right invite tokens. Sorry, S
27.
by mfreed 5y ago
Yes, we support the rich set of PostgreSQL's JOIN operations, including against hypertables. It's generally smart enough to only apply these JOINs against the right subset of time-series data if you also have any time predicates
28.
by mfreed 5y ago
To replicate, please see the Time Series Benchmark Suite, which is open-source and has many vendor-contributed configurations: - https://github.com/timescale/tsbs - https://github.com/timescale/tsb
29.
by mfreed 5y ago
Hi! So the team is over 100 at this point, but engineering effort is spread across multiple products at this point. The core timescaledb repo [0] currently has 10-15 primary engineers, with a few others working on DB hyperfunctions and our
30.
by mfreed 5y ago
Yep! And in fact, that's exactly what TimescaleDB supports - things like hyperloglog to support approximate count distinct, including as part of continuous aggregates. [0] This blog post - "How PostgreSQL aggregation works and how

More ›