Alanhlwang

6 ms·

1.
by Alanhlwang 3y ago
Definitely, I can jump into what we meant by brittle—we mainly meant that SQL scripts are hard to debug/undescriptive, you can't parametrize and customize error messages that you receive from transforms, and you can only execute o
2.
by Alanhlwang 3y ago
This is a great point—in terms of dynamic tables/materialized views, the software engineering best practices ie modularization, testing, version control are not as intuitive/straight-forward to apply in comparison to dbt and Serra
3.
by Alanhlwang 3y ago
We see this working more alongside Airflow—we see Airflow mainly centered as an orchestrator/scheduler to chain together your ETL steps after you've written your data transformations and connections. With Serra, we're a flexi
4.
by Alanhlwang 3y ago
Thanks for these links! We consider ourselves an ELT and ETL tool—if you run a Serra job in your own warehouse (ie Databricks), you can easily specify extracting from AWS, loading the parquets into your warehouse, then transforming them wit
5.
by Alanhlwang 3y ago
These are great links, we'll take a look
6.
by Alanhlwang 3y ago
This is super insightful thanks a ton for this gold mine. On the python vs yaml part—definitely could've made that way more clear in the demo. Right now are framework lets you call these python objects in your yaml file, but we are wor
7.
by Alanhlwang 3y ago
Thank you we appreciate the support!
8.
by Alanhlwang 3y ago
Yep we only showed our configuration file which instantiates the said Python objects. Definitely would've made the demo clearer, and if you want to look at the actual objects that our config is referencing, definitely check out our rep
9.
by Alanhlwang 3y ago
These are all great points, and no, Disney didn't use dbt. We wanted the demo to show how you can modularize SQL into reusable objects that you can fully customize error logs for, while also adding the value of handling all steps of ET
10.
by Alanhlwang 3y ago
Definitely a great point, adding one today
11.
Launch HN: Serra (YC S23) – Open-core, Python-based dbt alternative
139 points by Alanhlwang 3y ago|84 comments
12.
by Alanhlwang 3y ago
This is awesome, the ability to assess image composition according to photography 'rules' especially is super intriguing. Could you provide some examples of the specific rules your algorithm considers and how they contribute to th