6 ms·

I just discovered Hacker News offers a JSON API, ducking awesome, if only there was a nice tool to crunch data as JSON files... I think I have just found out y

I just discovered Hacker News offers a JSON API, ducking awesome, if only there was a nice tool to crunch data as JSON files... I think I have just found out yet another great DuckDB-weekend project!

tylerhannan 3y ago
It's definitely a fun dataset to explore. Dale from ClickHouse wrote a pretty extensive blog series on the Hacker News dataset, ingest approach, some queries of interest... Could be a good bit of reading alongside the project? https://clickhouse.com/blog/getting-data-into-clickhouse-part-1 https://clickhouse.com/blog/getting-data-into-clickhouse-par...
- carlopi 3y ago
  Thanks, cool read, I took a couple of ideas, will see where we land.
carlopi 3y ago
Btw, it's already happening: Go to https://shell.duckdb.org https://shell.duckdb.org, and type FROM 'https://hacker-news.firebaseio.com/v0/item/37663308.json https://hacker-news.firebaseio.com/v0/item/37663308.json'; Querying hacker news, from a browser tab (passing through a bunch of database and Web technology that make it possible for DuckDB to be executed within a browser tab)
- ddanieltan 3y ago
  woah! Didn't know this existed https://shell.duckdb.org https://shell.duckdb.org, it's pretty cool!