platypii

8 ms·

1.
by platypii 7mo ago
Sylvain Lesage’s cool interactive explainer on visualizing extreme row counts—think billions—inside the browser. His technical deep dive explains how the open-source library HighTable works around scrollbar limits by: - Lazy loading - Virtu
2.
A visual explainer of how to scroll through billions of rows in the browser(rednegra.net)
3 points by platypii 7mo ago|1 comments
3.
by platypii 8mo ago
We're willing to spend money, but I've had the "datadog billing problem" before where it starts reasonable and then grows to a non-trivial percent of saas budget, and then theres a scramble to refactor. Trying to get ahe
4.
Ask HN: Where are you keeping your LLM logs?
1 points by platypii 8mo ago|4 comments
5.
Show HN: Squirreling: a browser-native SQL engine(blog.hyperparam.app)
2 points by platypii 9mo ago|0 comments
6.
Best way to annotate large parquet LLM logs without full rewrites?
2 points by platypii 9mo ago|2 comments
7.
Ask HN: Local tools for working with LLM datasets?
2 points by platypii 9mo ago|0 comments
8.
by platypii 9mo ago
Makes sense. I'm not currently in snowflake because I'm mostly working with local parquet files. Would prefer not to have to pay for snowflake just to explore my data. I'm interested in better data UIs though so I might need
9.
What UI do you use on top of data engineering tools to look at data?
1 points by platypii 9mo ago|3 comments
10.
by platypii 10mo ago
I started Hyperparam one year ago because I knew that the world of data was changing, and existing tools like Python and Jupyter Notebooks were not built for the scale of LLM data. The weights of LLMs may be tensors, but the input and outpu
11.
Show HN: We built an AI tool for working with massive LLM chat log datasets(hyperparam.app)
16 points by platypii 10mo ago|1 comments
12.
Lessons from Hyperparam's year of open source data transformation(blog.hyperparam.app)
1 points by platypii 10mo ago|1 comments
13.
by platypii 10mo ago
This is a Q&A I did on what I learned from a year of open source data transformation. Most of all, it reinforced my belief that browser-native tools aren’t “toys” that don’t work for real systems. When Hugging Face integrated my librari
14.
by platypii 11mo ago
As with anything, there are engineering tradeoffs. What I've found is that moving data processing toward the browser has been for one, a refreshing developer experience because I don't need to build a pair of backend+frontend. Fro
15.
Ask HN: How far can we push the browser for large-scale data parsing?
1 points by platypii 11mo ago|2 comments
16.
by platypii 1y ago
Why not? We are trying to evaluate AI's capabilities. It's OBVIOUS that we should compare it to our only prior example of intelligence -- humans. Saying we shouldn't compare or anthropomorphize machine is a ridiculous hill to
17.
by platypii 1y ago
This is the story of how I spent a year making the world's fastest Parquet loader in JavaScript. The goal: - Make a faster, more interactive viewer for AI datasets (which are mostly parquet format) - Simplify the stack by doing everyth
18.
The Quest for Instant Data(blog.hyperparam.app)
16 points by platypii 1y ago|1 comments
19.
by platypii 1y ago
I don’t have benchmarks specifically against duckdb. I’m sure native C++ will run faster than JavaScript. But whats important is that with Hyperparam you can do it in the browser, where the bottleneck will always be network-bound not cpu-bo
20.
by platypii 1y ago
Funny you say that, because I built these tools because I wanted to build something very much like what you're describing! I was trying to look at, filter, and transform large AI datasets, and I was frustrated with how bad the existing
21.
by platypii 1y ago
Yea except with parquet you don't need to load the entire file, the parquet metadata let's you do http range requests for just the data you need. For example this parquet is the entire english wikipedia (400mb) but loads less than
22.
by platypii 1y ago
That's fair criticism... to be honest when I started the project it was more focused on hyperparameters, and it evolved into this javascript-for-ai mission. But now I just kind of liked the name.
23.
by platypii 1y ago
It does support using S3 presigned requests, but it's admittedly a little awkward to ask a server for a presigned request before every fetch. But does still have the benefit that you can have a small and light server just handing out s
24.
by platypii 1y ago
Duckdb and datafusion are super cool! But they are VERY large wasm blobs (30-40mb each). This is often larger than the data you’re trying to load. And they add complexity with serving and deploying wasm files. Hyparquet is 10kb of pure js,
25.
by platypii 1y ago
Zero telemetry, fully local. It spawns `http-server` on port 2048 and opens your browser at `localhost`. Similar pattern as Jupyter Notebooks. Feel free to audit the code... the server is <200 LOC.
26.
Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser(hyperparam.app)
77 points by platypii 1y ago|21 comments
27.
by platypii 2y ago
FSD makes Tesla superior to any car out there. No other car comes even close. Although I heard that FSD was already crippled in eu so maybe they aren't missing out as much.
28.
by platypii 2y ago
AI is even better at writing tests than writing code. So have it write the tests first and then write the code.
29.
Hyparquet.js: World's Smallest and Most Conformant Parquet File Parser(github.com)
3 points by platypii 2y ago|1 comments
30.
by platypii 2y ago
My goal is to build tools which enable working with large-scale ML datasets in the browser. The browser is critical for building compelling UIs, but previous parquet js libraries had gone abandoned. Apache Parquet is a very complicated form

More ›