geoHeil

6 ms·

1.
by geoHeil 2mo ago
I maintain a similar library more focused on data engineering needs: https://docs.metaxy.io/ maybe it is useful for more people.
2.
by geoHeil 1y ago
Benchmarks are like single match scorelines: interesting, but rarely comparable. We built a public ELO score leaderboard for analytics engines that treats every public benchmark (TPC-DS, TPC-H, SSB, vendor blogs, community posts - even the
3.
Show HN: ELO-Ranking Analytics Engines(data-inconsistencies.datajourney.expert)
6 points by geoHeil 1y ago|1 comments
4.
by geoHeil 1y ago
We are currently preparing a submission for JOSE and are looking for feedback but also more people who are interested in joining in and refining the template
5.
LLM-in-a-Box: A Templated, Self-Hostable Framework for Generative AI(github.com)
3 points by geoHeil 1y ago|3 comments
6.
by geoHeil 1y ago
Together with www.linkedin.com/in/aaron-cul... I have created a template for making LLMs (from different vendors and even self hosted ones) easily accessible to researchers - including advanced document RAG with docling.
7.
by geoHeil 1y ago
Nice. What about tracking the learning process and comparing against the peer group/ class?
8.
by geoHeil 1y ago
As some others already posted: There is more to copying data than just moving it. It is about observability. A lot of companies have created their own frameworks. One example I find useful for a lot of usecases is dagster. You can define re
9.
Getting Better at Data Engineering(georgheiler.com)
3 points by geoHeil 1y ago|0 comments
10.
Mix data SaaS workloads (Databricks vs. EMR) to save money by using dagster(georgheiler.com)
7 points by geoHeil 2y ago|2 comments
11.
by geoHeil 2y ago
Did you ever want to find an alternative to cloud data PaaS/SaaS vendor lock-in? In my blog post, I show how a 0-cost switch between local, Databricks and AWS EMR environments can be achieved to save cost by using dagster. Additionally
12.
by geoHeil 3y ago
Interesting why? What were the pain points?
13.
by geoHeil 3y ago
If you use bigquery in big lake mode - i.e. with parquet or iceberg or delta files on GCS (or other object store) you can easily pull in the data into duckdb as well
14.
by geoHeil 3y ago
Hi, Georg - one of the authors is here. What we argue is that: - for a great software/data engineering/creation experience we recommend such a stack that is only on when needed (when transformations occur) - for a great data consu
15.
by geoHeil 3y ago
Well local can mean your laptop. But could mean your local server. However, it can also mean a VM on your cloud provider of choice colocated to the object store. In such a case the network transfer (for many cases) can be almost irrelevant
16.
by geoHeil 3y ago
Not to forget https://www.starrocks.io/ or https://risingwave.com/ - see also a recent blog post of a friend: https://juhache.substack.com/p/moving-from-bi-to-data-apps-p...
17.
by geoHeil 3y ago
Hi, Georg - one of the authors is here. Indeed, we used LLMs/GPT4 for proof reading and enhancing the English language (we are not native speakers). We were thinking about breaking up the content - but decided that one long post is a b
18.
by geoHeil 3y ago
Hi, Georg - one of the authors is here. In fact, this is exactly what we argue: Keep data the consumer experience high a Paas platform (Fabric, BigLake, Databricks, SF, ...) can make a lot of sense, whereas for the best data development&#x2
19.
The new local data stack: Integrating Dagster, dbt and DuckDB(georgheiler.com)
144 points by geoHeil 3y ago|50 comments
20.
by geoHeil 3y ago
We (Aleksandar and Georg) want to share our new blog post on: "Dagster, dbt, DuckDB as new local MDS" here. It re-introduces the local environment to enhance the developer productivity for data pipelines by bringing back software
21.
Ask HN: Electronic data interchange for structured financial products
2 points by geoHeil 3y ago|0 comments
22.
Show HN: Harmonized Data Platform(databarnum.com)
2 points by geoHeil 3y ago|0 comments