6 ms·
when you got out of Spark, what did you go to?
by ajma 1y ago
when you got out of Spark, what did you go to?
- forgetfulness 1y agoBigQuery ELT, the org I went to was rather immature in their data practice, and I sold them on getting some proper orchestration (Dataform, their preference over DBT, and Airflow), and keeping the architecture coherent. I'd have rather stuck with Spark just because I prefer Scala or Python to SQL (and that comes with e.g. being far easier to unit test), but life happened and that ecosystem was getting disrupted anyway.