9 ms·

Rawdogging SQL when you're not a seasoned DB administrator basically makes an arcane art look occult. Most people reach out towards an ORM or query building en

by ljm 24d ago

Rawdogging SQL when you're not a seasoned DB administrator basically makes an arcane art look occult. Most people reach out towards an ORM or query building engine and otherwise don't really go far beyond the basic CRUD, joins, and some simple aggregations with groups. Since they try to be DB agnostic you'll rarely get an adaptor over CTEs or window functions or partitioning. An LLM is great at exposing what a database is capable of doing with SQL and might even manage to navigate the most poorly designed of schemas. And it might even manage to design one to an acceptable standard if it has enough domain knowledge in its context.

vjvjvjvjghv 24d ago
The problem in my view is that there aren't good tools to debug advanced SQL stuff within the context of the whole system which is usually written in a higher level language. I just spent a few weeks modifying some code where the original dev put a lot of logic into stored procedures. That's in principle fine but it's really hard to figure the actual business logic when it's spread out over C# and then also SQL. It doesn't help that the SQL code looks like FORTRAN code from 1985. Personally I think we need ORMs that allow expressing advanced SQL stuff with other high level languages. Or even better: The ORM detects where advanced SQL makes sense and uses it.
- ljm 24d ago
  If I had to pick, I'd try to make the ORM redundant by making 'lower level' SQL easier to deploy rather than depending on sending strings of SQL queries and mutations over the wire. I haven't worked in a single setup where raw SQL has been encouraged, because it always requires DB migrations and not all of them are safe. Nobody dares touch the DB server's resources by setting up stored procedures, materialised views, etc. etc. and instead people are blowing money on Redis instances and caching and shit. I don't have an answer to this but I've hit a lot of issues in my career where I think, "this could have been solved months ago by pivoting a couple of tables or creating a new function." You have been able to 'script' the DB for decades but you lose a lot of what you gain from the traditional SDLC at the app layer.
  - grebc 24d ago
    Dapper in .net is fantastic to deal with raw sql, to the point I think I’m delusional because it’s so damn simple to send outrageous queries to the database and have those multiple mixed results turned into objects very simply. I’ve never had an issue of raw SQL requiring migrations? Unless you’re talking of changing database engine? In which case I think it’s a bit of folly to imagine changing the database engine will not mean changes to your stack higher up the chain.
  - bruce511 24d ago
    For me, one of the primary benefits of ORMs is that they can parameterize requests which then prevents SQL injection attacks. Passing raw SQL to the database needs very careful attention to the dynamic parts, and it's too easy for user-generated data to be included. Yes, it's possible to pass user generated text through a sanitizer but now you just have an arms race between the sanitizer and "clever" users.
    - jedwards1211 23d ago
      It’s not that hard to pass values as query parameters of a manually written query. With inferior databases that don’t support array parameters it’s a bit more work to construct the correct number of $ parameters in the query, but still not that hard
      - bruce511 23d ago
        "Can it be done?" is one question. "Is it done dilligently by all the programmers on the team?" is quite another.
sgarland 24d ago
> Rawdogging SQL when you're not a seasoned DB administrator basically makes an arcane art look occult. Isn't that true of most languages? SQL has pretty simple syntax; I think the only reason it's sometimes seen as arcane is that fewer and fewer people bother to learn it.
catlifeonmars 24d ago
> Rawdogging SQL when you're not a seasoned DB administrator basically makes an arcane art look occult. This is kind of a hot take. Most devs I know know PostGreSQL well. They know how to write complex queries with CTAS, joins, etc, know how to create indexes, views, and add user defined functions.
alliao 24d ago
dba here and I really don't get why SQL is so feared... I get that it requires very different way to think about data but it is quite simple in terms of you tell it what to do, and if it does it badly you probably told it wrong so just try something different...
- mike_hearn 23d ago
  Because: 1. SQL isn't composable (you can't assign fragments to variables except for CTEs) so you can't easily test out subparts and build them up incrementally without just copy/pasting stuff around. 2. Joins are an unnatural way to dereference pointers. 3. SQL is more than SELECT. Once you get into updates you encounter lots of scary edge cases and traps. How many engineers really understand isolation levels? Why doesn't skipping the column list in an INSERT substitute nulls for the nullable columns that aren't provided? What changes can you make to a schema that are 'safe' for your environment (won't take table locks)? What locks are being taken by the RDBMS behind your back - sometimes it matters! 4. Site outages caused by optimizer plan shifts are scary because people don't feel in control. Good databases have features to ameliorate these issues, but most people's experience is of databases that are merely OK and not good.
  - euroderf 23d ago
    > 2. Joins are an unnatural way to dereference pointers. There's gotta be a simple & clear alternative to this obstruction. Maybe it just hasn't been invented yet.
    - mike_hearn 23d ago
      Most query languages do fix that. GraphQL is one example.
      - jedwards1211 23d ago
        GraphQL joins are entirely up to the underlying resolvers, it has no syntax for expressing different kinds of joins or filter conditions on values from multiple joined relations

by ljm 24d ago

vjvjvjvjghv 24d ago

The problem in my view is that there aren't good tools to debug advanced SQL stuff within the context of the whole system which is usually written in a higher level language. I just spent a few weeks modifying some code where the original dev put a lot of logic into stored procedures. That's in principle fine but it's really hard to figure the actual business logic when it's spread out over C# and then also SQL. It doesn't help that the SQL code looks like FORTRAN code from 1985. Personally I think we need ORMs that allow expressing advanced SQL stuff with other high level languages. Or even better: The ORM detects where advanced SQL makes sense and uses it.

ljm 24d ago
If I had to pick, I'd try to make the ORM redundant by making 'lower level' SQL easier to deploy rather than depending on sending strings of SQL queries and mutations over the wire. I haven't worked in a single setup where raw SQL has been encouraged, because it always requires DB migrations and not all of them are safe. Nobody dares touch the DB server's resources by setting up stored procedures, materialised views, etc. etc. and instead people are blowing money on Redis instances and caching and shit. I don't have an answer to this but I've hit a lot of issues in my career where I think, "this could have been solved months ago by pivoting a couple of tables or creating a new function." You have been able to 'script' the DB for decades but you lose a lot of what you gain from the traditional SDLC at the app layer.
- grebc 24d ago
  Dapper in .net is fantastic to deal with raw sql, to the point I think I’m delusional because it’s so damn simple to send outrageous queries to the database and have those multiple mixed results turned into objects very simply. I’ve never had an issue of raw SQL requiring migrations? Unless you’re talking of changing database engine? In which case I think it’s a bit of folly to imagine changing the database engine will not mean changes to your stack higher up the chain.
- bruce511 24d ago
  For me, one of the primary benefits of ORMs is that they can parameterize requests which then prevents SQL injection attacks. Passing raw SQL to the database needs very careful attention to the dynamic parts, and it's too easy for user-generated data to be included. Yes, it's possible to pass user generated text through a sanitizer but now you just have an arms race between the sanitizer and "clever" users.
  - jedwards1211 23d ago
    It’s not that hard to pass values as query parameters of a manually written query. With inferior databases that don’t support array parameters it’s a bit more work to construct the correct number of $ parameters in the query, but still not that hard
    - bruce511 23d ago
      "Can it be done?" is one question. "Is it done dilligently by all the programmers on the team?" is quite another.

sgarland 24d ago

> Rawdogging SQL when you're not a seasoned DB administrator basically makes an arcane art look occult. Isn't that true of most languages? SQL has pretty simple syntax; I think the only reason it's sometimes seen as arcane is that fewer and fewer people bother to learn it.

catlifeonmars 24d ago

> Rawdogging SQL when you're not a seasoned DB administrator basically makes an arcane art look occult. This is kind of a hot take. Most devs I know know PostGreSQL well. They know how to write complex queries with CTAS, joins, etc, know how to create indexes, views, and add user defined functions.

alliao 24d ago

dba here and I really don't get why SQL is so feared... I get that it requires very different way to think about data but it is quite simple in terms of you tell it what to do, and if it does it badly you probably told it wrong so just try something different...

mike_hearn 23d ago
Because: 1. SQL isn't composable (you can't assign fragments to variables except for CTEs) so you can't easily test out subparts and build them up incrementally without just copy/pasting stuff around. 2. Joins are an unnatural way to dereference pointers. 3. SQL is more than SELECT. Once you get into updates you encounter lots of scary edge cases and traps. How many engineers really understand isolation levels? Why doesn't skipping the column list in an INSERT substitute nulls for the nullable columns that aren't provided? What changes can you make to a schema that are 'safe' for your environment (won't take table locks)? What locks are being taken by the RDBMS behind your back - sometimes it matters! 4. Site outages caused by optimizer plan shifts are scary because people don't feel in control. Good databases have features to ameliorate these issues, but most people's experience is of databases that are merely OK and not good.
- euroderf 23d ago
  > 2. Joins are an unnatural way to dereference pointers. There's gotta be a simple & clear alternative to this obstruction. Maybe it just hasn't been invented yet.
  - mike_hearn 23d ago
    Most query languages do fix that. GraphQL is one example.
    - jedwards1211 23d ago
      GraphQL joins are entirely up to the underlying resolvers, it has no syntax for expressing different kinds of joins or filter conditions on values from multiple joined relations