9 ms·

Networking is worth an honorable mention I agree. > If you don't learn about grammars, lexers, parsers, ASTs,... you're at a huge disadvantage in many situatio

Networking is worth an honorable mention I agree. > If you don't learn about grammars, lexers, parsers, ASTs,... you're at a huge disadvantage in many situations. Can you get an example? I don't recall a time where I used this knowledge in the last 5 years as a systems generalist in big tech.

sn9 4y ago
https://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html https://steve-yegge.blogspot.com/2007/06/rich-programmer-foo...
cue_the_strings 4y ago
Your (non-technical) clients want to query the company's data, which is stored in who-knows-what way (Redis + Mongo + Postgres + Kafka + hdf5 + network filesystems with Apache Parquet files, whatever wakes you up at night). They just want to write filters in a trivial `some_field > 3 and otherField * 5 < thirdField` way. You implement this, but they keep asking for more functionality and syntax to this little DSL. The obvious answer is writing a parser, what the parser generates is dependent on the backend. Maybe you generate a lambda filter for the collection, maybe you generate an intermediary AST which you then traverse to generate a query for some DB or multiple of them. I've had similar requests in both medical and finance fields. Your predecessors had zero regard for C programming best practices, leaving you with a pile of supremely brittle code that just barely works, but you have to tiptoe around it in silk gloves whenever you want to modify it in any way, because of unreasonable standards on ownership, calling conventions, concurrency, whatever. They might have had a reason in the beginning, but now it took over creating a monster that no regular programmer can work on. They didn't think that the code will ever need to be multithreaded. This is no environment for a sane person. A young, inexperienced fool would suggest to rewrite it from scratch. You know better. The whole business logic of a working, profitable business is imprisoned in this pile of shit, and yet it's too far gone to continue forward with it. Nobody understands all the silly rules of engagement with the codebase. You decide to: 1) break everything into chunks, 2) write extensive unit tests, 3) use automated tools to fix the whole codebase yourself to some sane standard. Some of the automation can be just grep + xargs + sed, some is much more complex. Regex replacements are unreasonably efficient, but not omnipotent. The only sane way to fix 40k LOC at a time is to write tools that operate on the AST, find toxic patterns and smartly replace them. This takes dozens of iterations, but you can always roll back with git. You're not transforming the code incrementally, you're writing a transformer. Eventually, your fixers are good enough that they fix 95% of the issues, you fix the rest manually. Your company can now hire any old mediocre C dev off the street, and this dev can productively contribute to the rejuvinated codebase. The company no longer needs psychics, regular C devs will do. If they stray from the path, the newly set up CI with dozens of custom clang-tidy plugins swiftly puts them back in their place. Your company is in a highly regulated space that has insane coding standards. There are (shitty) compliance tools that cost >$100k a year for your team. You're able to replicate 80% of the functionality of the tools, fix some issues and generate compliance reports from the code on this in ~8 months. Slap on some CI, and the bean counters don't even have to do anything but download the latest PDF report.

sn9 4y ago

https://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html https://steve-yegge.blogspot.com/2007/06/rich-programmer-foo...

cue_the_strings 4y ago

Your (non-technical) clients want to query the company's data, which is stored in who-knows-what way (Redis + Mongo + Postgres + Kafka + hdf5 + network filesystems with Apache Parquet files, whatever wakes you up at night). They just want to write filters in a trivial `some_field > 3 and otherField * 5 < thirdField` way. You implement this, but they keep asking for more functionality and syntax to this little DSL. The obvious answer is writing a parser, what the parser generates is dependent on the backend. Maybe you generate a lambda filter for the collection, maybe you generate an intermediary AST which you then traverse to generate a query for some DB or multiple of them. I've had similar requests in both medical and finance fields. Your predecessors had zero regard for C programming best practices, leaving you with a pile of supremely brittle code that just barely works, but you have to tiptoe around it in silk gloves whenever you want to modify it in any way, because of unreasonable standards on ownership, calling conventions, concurrency, whatever. They might have had a reason in the beginning, but now it took over creating a monster that no regular programmer can work on. They didn't think that the code will ever need to be multithreaded. This is no environment for a sane person. A young, inexperienced fool would suggest to rewrite it from scratch. You know better. The whole business logic of a working, profitable business is imprisoned in this pile of shit, and yet it's too far gone to continue forward with it. Nobody understands all the silly rules of engagement with the codebase. You decide to: 1) break everything into chunks, 2) write extensive unit tests, 3) use automated tools to fix the whole codebase yourself to some sane standard. Some of the automation can be just grep + xargs + sed, some is much more complex. Regex replacements are unreasonably efficient, but not omnipotent. The only sane way to fix 40k LOC at a time is to write tools that operate on the AST, find toxic patterns and smartly replace them. This takes dozens of iterations, but you can always roll back with git. You're not transforming the code incrementally, you're writing a transformer. Eventually, your fixers are good enough that they fix 95% of the issues, you fix the rest manually. Your company can now hire any old mediocre C dev off the street, and this dev can productively contribute to the rejuvinated codebase. The company no longer needs psychics, regular C devs will do. If they stray from the path, the newly set up CI with dozens of custom clang-tidy plugins swiftly puts them back in their place. Your company is in a highly regulated space that has insane coding standards. There are (shitty) compliance tools that cost >$100k a year for your team. You're able to replicate 80% of the functionality of the tools, fix some issues and generate compliance reports from the code on this in ~8 months. Slap on some CI, and the bean counters don't even have to do anything but download the latest PDF report.