dmpetrov

6 ms·

1.
Anthropic's in-house data analytics with Claude(claude.com)
4 points by dmpetrov 4mo ago|1 comments
2.
Code as Agent Harness(code-as-harness.github.io)
3 points by dmpetrov 4mo ago|0 comments
3.
Issue tracking for AI-assisted software work(github.com)
3 points by dmpetrov 4mo ago|0 comments
4.
How to Build a Data Agent in 2026(twitter.com)
2 points by dmpetrov 6mo ago|0 comments
5.
Entire - hooks into your Git workflow to capture AI agent sessions(github.com)
2 points by dmpetrov 7mo ago|0 comments
6.
by dmpetrov 7mo ago
I like the idea a lot but it's still unclear from the docs what the hard security boundary is once you start calling LLMs - can it avoid "breaking out" into the host env in practice?
7.
Monty: A minimal, secure Python interpreter written in Rust for use by AI(github.com)
323 points by dmpetrov 7mo ago|164 comments
8.
by dmpetrov 2y ago
Good point - it does sound a bit like marketing bs. We’ll rephrase it.
9.
Overcoming data inconsistency with a universal semantic layer(infoworld.com)
1 points by dmpetrov 2y ago|0 comments
10.
by dmpetrov 2y ago
Please share your feedback!
11.
by dmpetrov 2y ago
Good question! I’m not so familiar with it. It looks like Daft is closer to Lance with it’s own data format and engine. But I’d appreciate more insights from users or the creators.
12.
by dmpetrov 2y ago
Lance is just a data format. Lance DB might be more comparable to DataChain. DataChain focuses on data transformation and versioning, whereas LanceDB appears to be more about retrieving and serving data. Both designed for multimodal use cas
13.
by dmpetrov 2y ago
It's not a format :) It's simpliy about linking metadata from a json to a corresponding image or video file, like pairing data003.png & data003.json to a single, virtual record. Some format use this approach: open-image or lai
14.
by dmpetrov 2y ago
Absolutely, that's a common scenario! Just connect from your Python code (like the lambda in the example) to DB and extract the necessary data.
15.
by dmpetrov 2y ago
I guess, it involves splitting a file into smaller document snippets, getting page numbers and such, and calculating embeddings for each snippet—that’s the usual approach. Specific signals vary by use case. Hopefully, @jerednel can add more
16.
by dmpetrov 2y ago
Exactly! DataChain does lazy compute. It will read metadata/json while applying filtering and only download a sample of data files (jpg) based on the filter. This way, you might end up downloading just 1% of your data, as defined by th
17.
by dmpetrov 2y ago
DataChain has no assumptions about metadata format. However, some formats are supported out of the box: WebDataset, json-pair, openimage, etc. Extract metadata as usual, then return the result as JSON or a Pydantic object. DataChian will au
18.
by dmpetrov 2y ago
Yes, it's not meant to replace data engineering tools like Prefect or Temporal. Instead, it serves as a transformation engine and ad-hoc analytics for images/video/text data. It's pretty much DBT use case for text and im
19.
by dmpetrov 2y ago
Yay! Excited to see DataChain on the front page :) Maintainer and author here. Happy to answer any questions. We built DataChain because our DVC couldn't fully handle data transformations and versioning directly in S3/GCS/Azu
20.
by dmpetrov 2y ago
Can this work statistically? For a giving number of attempts, you can ger a required number of successes to make sure it's a statistically meaningful result. In theory, this approach could help address the non-determinism of LLMs.
21.
by dmpetrov 2y ago
Thank you! Just shoot an email to support and mention HN. I’ll read and reply.
22.
by dmpetrov 2y ago
Right, DVC caches data for consistency and reproducibility. If caching is not needed and streaming required, we've created a sister tool DataChain. It's even supports WebDataset and can stream from tar archives and filter images b
23.
by dmpetrov 2y ago
Yes. And if you track transformations of the binaries or ml training
24.
by dmpetrov 2y ago
In this cases, you need DVC if: 1. File are too large for Git and Git LFS. 2. You prefer using S3/GCS/Azure as a storage. 3. You need to track transformations/piplines on the file - clean up text file, train mode, etc. Otherw
25.
by dmpetrov 2y ago
hi there! Maintainer and author here. Excited to see DVC on the front page! Happy to answer any questions about DVC and our sister project DataChain https://github.com/iterative/datachain that does data versioning with
26.
by dmpetrov 2y ago
Hey! I'm one of the creators of DataChain. DataChain works on your local machine and manages files in storage (like images and PDFs in S3 or GCP). Users can slice and dice their files using metadata. Example: - Download only files labe
27.
by dmpetrov 2y ago
It uses SQLite in open-source. In SaaS - proprietary data warehouses where your engine can be integrated.
28.
by dmpetrov 2y ago
Bridging the gap between AI and data warehouses is crucial, but I’m not sure SQL is the best fit for AI engineers who mainly work with Python and AI APIs. At DataChain, we are solving this by creating a Python API that translates to SQL und
29.
by dmpetrov 4y ago
Well... I need to rent a property with +1 bedroom to WFH. It is a substantial family spending.
30.
by dmpetrov 4y ago
> I do not treat Kubernetes as Cloud-Native... for end-users to achieve cloud native goals. Do you mean that Cloud-Native is about applications that are built on top of K8S, not K8S itself? What are the cloud native goals? Abstracting ou

More ›