philbe77

8 ms·

1.
by philbe77 4mo ago
TypeScript: https://www.npmjs.com/package/@quack-protocol/sdk
2.
by philbe77 4mo ago
Two new Quack client drivers: ADBC: https://github.com/gizmodata/adbc-driver-quack JDBC: https://github.com/gizmodata/quack-jdbc
3.
SHOW HN: DuckDB / DuckLake Server (With Arrow Flight SQL) for iOS
2 points by philbe77 5mo ago|0 comments
4.
by philbe77 7mo ago
Now it is easy to use Power BI with a DuckDB server you can self-host: GizmoSQL. Powered by Arrow Flight SQL - you can run terabyte-scale analytics for dirt-cheap...
5.
by philbe77 10mo ago
Oh, and GizmoData Cloud (SaaS option) is coming soon - to make it easier than ever to provision GizmoSQL instances...
6.
by philbe77 10mo ago
GizmoSQL is definitely a good option. I work at GizmoData and maintain GizmoSQL. It is an Arrow Flight SQL server with DuckDB as a back-end SQL execution engine. It can support independent thread-safe concurrent sessions, has robust secu
7.
by philbe77 11mo ago
good point :) - we can re-aggregate HyperLogLog (HLL) sketches to get a pretty accurate NDV (Count Distinct) - see Query.farm's DataSketches DuckDB extension here: https://github.com/Query-farm/datasketches We als
8.
by philbe77 11mo ago
hi sdairs, we did store the data on the worker nodes for the challenge, but not in memory. We wrote the data to the local NVMe SSD storage on the node. Linux may cache the filesystem data, but we didn't load the data directly into me
9.
by philbe77 11mo ago
This is something we are trying to take a novel approach to as well. We have a video demonstrating some TPC-H SF10TB queries which perform inner joins, etc. - with GizmoEdge as well: https://www.youtube.com/watch?v=hlSx0E2j
10.
by philbe77 11mo ago
I've since learned (from a DuckDB blog) - that DuckDB seems to do better when the XFS filesytem. I used ext4 for this, so I may be able to get another 10 to 15% (maybe!). DuckDB blog: https://duckdb.org/2025/10&#x
11.
by philbe77 11mo ago
Hi mosselman, GizmoEdge is not open-source. DeepSeek has "smallpond" however, which is open-source: https://github.com/deepseek-ai/smallpond I plan on getting GizmoEdge to production-grade quality eventually
12.
by philbe77 11mo ago
hi djhworld. The 5s does not include the download/materialization step. That parts takes the worker about 1 to 2 minutes for this data set. I didn't know that this was going on HackerNews or would be this popular - I will try t
13.
by philbe77 11mo ago
If I used "spot" instances - it would have been 63 x $0.732/hr for a total of: $45.99 / hr.
14.
by philbe77 11mo ago
I'm not exactly sure yet. My goal was to not have the shards be too large so as to be un-manageable. In theory - I could just have had 63 (or 64) huge shards - and 1 worker per K8s node, but I haven't tried it. There are so many
15.
by philbe77 11mo ago
:D that is scary!
16.
by philbe77 11mo ago
You should do it then, and post it here. I did do it with one machine as well: https://gizmodata.com/blog/gizmosql-one-trillion-row-challen...
17.
by philbe77 11mo ago
Challenge accepted - I'll try it on a 4XL Snowflake to get actual perf/cost
18.
by philbe77 11mo ago
Cost-wise, 64 4xl Snowflake clusters would cost: 64 x $384/hr - for a total of: $24,576/hr (I believe)
19.
by philbe77 11mo ago
Good point - I'll update it...
20.
by philbe77 11mo ago
Hi nodesocket - I tried to do 4 CPUs per node, but Kubernetes takes a small (about 200m) CPU request amount for daemon processes - so if you try to request 4 (4000m) CPUs x 16 - you'll spill one pod over - fitting only 15 per node. I w
21.
by philbe77 11mo ago
At 5 seconds - the query technically cost: $0.3276
22.
by philbe77 11mo ago
Yes - I also had GizmoSQL (a single-node DuckDB database engine) take the challenge - with very good performance (2 minutes for $0.10 in cloud compute cost): https://gizmodata.com/blog/gizmosql-one-trillion-row-challen.
23.
by philbe77 11mo ago
A Standard E64pds v6 costs: $3.744 / hr on demand. At 63 nodes - the cost is: $235.872 / hr - still cheaper than a Snowflake 4XL cluster - costing: 128 credits / hr at $3/credit = $384 / hr.
24.
by philbe77 11mo ago
Hi MobiusHorizons, I happened to use websockets b/c it was the technology I was familiar with. I will try to learn more about normal sockets to see if I could perhaps make them work with the app. Thanks for the suggestion...
25.
by philbe77 11mo ago
Hi shinypenguin - the dataset and challenge are detailed here: https://github.com/coiled/1trc The data is in a publicly accessible bucket, but the requester is responsible for any egress fees...
26.
GizmoSQL completed the 1T row challenge
1 points by philbe77 1y ago|0 comments
27.
GizmoEdge runs 10TB TPC-H queries in under 5 seconds [video](youtube.com)
2 points by philbe77 1y ago|0 comments
28.
by philbe77 1y ago
Here is a video demo I uploaded: https://youtube.com/watch?v=bYmFd8KBuE4&si=YbcH3ILJ7OS8Ns47
29.
Show HN: GizmoEdge – Distributed IoT SQL Engine
4 points by philbe77 1y ago|1 comments
30.
Show HN: GizmoSQL – now easy to try online for FREE(app.gizmodata.com)
2 points by philbe77 1y ago|1 comments

More ›