7 ms·

A Short Story About SQL’s Biggest Rival

256 points by huy 6y ago

Ericson2314 6y ago
Good history. But, thanks to https://github.com/haskell-beam/beam https://github.com/haskell-beam/beam I no longer need to worry about the COBOL-ness of SQL without giving up the semantics. Finally!
kmeisthax 6y ago
>If the world worked differently, we wouldn’t still be writing on QWERTY keyboards, or speaking English; technically superior alternatives like Dvorak and Esperanto would have taken over. Bad metaphor: There is no evidence for Dvorak's technical superiority to QWERTY, neither is there for Esperanto over other languages.
- nine_k 6y ago
  Esperanto is similar to a bunch of Roman languages widespread in Europe. One can see it as an attempt to collect them back into a common "Latin 2.0". Outside Western Europe, Esperanto makes rather little sense. It's highly regular, which is nice — but, say, Japanese is also highly regular.
  - ithkuil 6y ago
    Yes it's based off european languages; it was designed in a time when removing barriers from neighboring countries in a culturally very fragmented continent was perceived of higher practical value than creating a pan-human language; intercontinental travel wasn't yet as commonplace as now, yet many european countries had different cultures living within the boundaries of the same states and usually members of the majority culture were privileged as a result; there was hope (espero in esperanto) that people could forget about differences among them and see what they have in common. Today we can be tempted to frame it as european chauvinism, but that's just because of our expanded horizons as a global society. People who believes in esperanto in the early days would likely share the same feeling now
- cbsmith 6y ago
  I think the "technically superior" bit wasn't really the right choice of words. Esperanto isn't intended to be superior. It's value is on it being equally foreign yet approachable for all the salient parties and therefore a conceivable acceptable neutral turf for everyone to share. Ironically, the case for Dvorak keyboards is kind of the opposite: QWERTY was intentionally designed to avoid jams, which if anything biased towards making it more difficult. It's not so much technical superiority as having a design objective that is more appropriate for the problem space. One can similarly argue about whether "QUEL" is really technically superior to "SQL", but the design objective is (at least as perceived by the author) better aligned with the solution space.
  - thaumasiotes 6y ago
    > Esperanto isn't intended to be superior. It's value is on it being equally foreign yet approachable for all the salient parties and therefore a conceivable acceptable neutral turf for everyone to share. This was not even attempted; Esperanto is a Romance language. Unless you think the only salient parties are Spain, France, Portugal, Italy, and Latin America, this "value" does not exist and was not a goal.
    - mijamo 6y ago
      Esperanto is not a romance language. It incorporates elements from most european languages families (balto-slavic, germanic, romance), and the grammar is most particularly inspired by slavic languages. And it was pretty much a goal of its creator to be both familiar and foreign for different european language families, as he had experienced division in Poland between speakers of those different families and wanted to have something more universal to gather them all together.
      - eska 6y ago
        We're still ignoring the entire Asian and African continents then, though.
        6y ago
- tbenst 6y ago
  Dvorak is 25% more efficient than Qwerty using some very reasonable calculations: http://mkweb.bcgsc.ca/carpalx/?dvorak http://mkweb.bcgsc.ca/carpalx/?dvorak
  - bsder 6y ago
    And by some other measures (hand alternation of fingers) Dvorak is less efficient. QWERTY isn't as bad as people make it out to be, and Dvorak isn't as good as people make it out to be. There are keyboard configurations that are better than both, but nobody uses them because you'll never be able to use anybody else's keyboard.
    - edjroot 6y ago
      > nobody uses them because you'll never be able to use anybody else's keyboard I doubt that's the main reason, and it may be unwarranted as I say below. Personally, I thought it would take too long for me to be able to type as fast as in QWERTY. Fortunately, I was wrong about that too. I learned the QGMLW optimized layout from Carpalx (linked above) about a year ago. I trained on https://keybr.com https://keybr.com and then on https://typeracer.com https://typeracer.com for a few days and in less than a week I reached 70wpm. I stopped training shortly after and now I can usually reach 90-100wpm. Not particularly fast, but I'm not any faster in QWERTY, and I believe I could get faster if I trained more. Even though I almost never type in QWERTY anymore, when I do I'm still just as fast as I used to be. I never learned to type in QWERTY "the right way", though, and I suppose this is actually what keeps me from confusing the two "modes". Whenever I try to keep my fingers on their "appropriate" keys (like a properly trained typist would), it's like my brain switches to "Carpalx mode" (since I did make an effort to use the right finger positioning to learn it). It's kinda like switching between thinking in my native language and English - I can think in both, but it doesn't "feel" the same, and I'm more likely to confuse the two where they overlap more. Pretty interesting, really.
      - greenshackle2 6y ago
        Yeah I did the same thing but I learned colemak. I never learned to touch-type qwerty, when I learned touch-typing I switched to colemak. I feel like it made things easier as I didn't have bad habits to fall back on. I can still type qwerty but I'm not very fast at it (I never was).
- deleted 6y ago
  [deleted]
exabrial 6y ago
Really the the only issue I have with SQL is NULL != NULL. This creates an impedance mismatch with most languages... MySQL sort of solves this problem with a <=> operator, which I wish was the default for ORMs to use. There are a lot of other minor nitpicks but a lot of criticisms come down to the actual RDMS not SQL itself.
- nendroid 6y ago
  What about the main point of the article that SQL is not composable.
  - jmalicki 6y ago
    CTEs (common table expressions) and views definitely do help with this, though they are new-ish where they exist and often have optimization issues. But being able to use them extensively in a newer database where they work well helps this quite a bit.
    - nendroid 6y ago
      Tables are composable but the expression itself cannot be decomposed. I cannot reuse a where clause somewhere else; that is the fundamental problem the article addresses.
- sumtechguy 6y ago
  I just wish they had flipped from and select around.
  - NickSharp 6y ago
    Or "set" and "where"
  - nendroid 6y ago
    It's not about flipping. The two concepts are actually commutative. You could in theory create syntax that looks like this: FROMCLAUSE * SELECTCLAUSE * WHERECLAUSE = SQLEXPRESSION SELECTCLAUSE * WHERECLAUSE * FROMCLAUSE = SQLEXPRESSION ... The issue is that not only does SQL syntax force an artificial order on these clauses, but that these clauses Cannot be decomposed to be used elsewhere. I cannot reuse a WHERECLAUSE or a SELECTCLAUSE in another expression.
jrochkind1 6y ago
can anyone find examples of what QUEL looked like? If we had a more composable query language being used instead of SQL, I wonder if that would have effected the course of ORMs, which arguably end up being as much about composable models of queries as they do about actual object mapping.
- knodi123 6y ago
  sure. https://en.wikipedia.org/wiki/QUEL_query_languages#Usage https://en.wikipedia.org/wiki/QUEL_query_languages#Usage
- GordonS 6y ago
  I kept searching the article, thinking I must have somehow missed the examples - nope! How very odd to wrote an article about how much better QUEL was than SQL, and not have a single example of either!
eecc 6y ago
are there any usable implementations of QUEL in the wild?
alquemist 6y ago
> … The language (SQL) is not very composable. This is a fact that most SQL users are not aware of. The relational algebra that SQL is based on is absolutely composable but SQL is not due to the inherent limitation of the language (as it was designed to be natural language-like). When you write "select x from a where z", you are actually building something along the lines of "from a" => "where z" => "select x" in the algebra and you can actually compose each portion separately. If you are familiar with dplyr, Spark or pandas you would get this instantly. Hmmm. Not a big lover of SQL, but this is a bit imprecise. While the unit of composability is slightly smaller for from=>where=>select (expression) vs select+from+where (subquery), in practice they both encode the same fundamental compositional principles, based on relational algebra. Any from=>where=>select query can be translated into a select+where+from query almost 1:1, if only via: from t ::= select * from t q => where p ::= select * from q where p q => select xs ::= select xs from q Sometimes the select+where+from ends up more verbose, sometimes there is more brain twisting to grok a given select+where+from query, but that's not a composition limiting factor. Granted, the readability of SQL is sometimes lacking, but it is fully capable to compose recursive relational algebra queries.
- contravariant 6y ago
  You're arguing about capability, whereas the post is arguing about the grammar itself. What you're describing is how to build a separate domain specific language that compiles to SQL (which there are quite a few of; e.g. C#'s Linq). Quite a few query languages are equally capable (including, surprisingly, quite a few so-called graph languages, provided the SQL dialect provides a transitive closure), but SQL as a language has some undesirable properties (most of which are trade-offs for the fact that basic SQL is very easy to parse).
- asah 6y ago
  Subqueries, named VIEWs, CTEs, etc all make SQL compostable ?
  - radiowave 6y ago
    Yes, but it's extremely clunky. The number of times I've had to write out a whole chain of CTEs just because I wanted to apply one window function to the output of another. In which context, "compostable" is a fantastic Freudian typo.
mamcx 6y ago
I'm working in a relational language that if it work as I wish, could eventually be put on top of a RDBM like sqlite or layer for other DBs: https://github.com/Tablam/TablaM https://github.com/Tablam/TablaM It have ideas similar to QUEL...
- prostodata 6y ago
  Just for comparison, a functional approach is a major alternative to relational and set-oriented models and query languages. The difference is that functions and operations with functions are first class elements of the model. One version of it is implemented in this project: https://github.com/prostodata/prosto https://github.com/prostodata/prosto - Functions matter! No join-groupby, No map-reduce
nendroid 6y ago
The article talks about how SQL lacks composability. I would like to know everyones thoughts about this. This is a huge issue with programming in general not exclusive to SQL. Everyone would like to build programs that are modular and reusable but programming paradigms have been traveling in directions that prevent this from happening. Many people turn to design patterns or microservices to try to deal with this organizational issue but they fail to see that the lower level programming paradigm itself is the precursor to the problem. In SQL the problem occurs in the statement itself. The WHERE clause or the SELECT clause cannot be reused anywhere else. I can't modularize a where clause and put it in another SQL statement. I have to rewrite the entire clause to reuse it. In OOP the same issue occurs. In OOP your class tends to contain methods that are not combinators, or in other words methods that modify a free variable. Due to this like the SQL expression, Objects cannot be decomposed either. I cannot reuse a setter in another class or anywhere else outside of the context of the free variable it modifies. In both cases there comes a time in the future of an application where programmers realize that similar logic could be reused but structural problems are preventing the reuse from happening so they have to implement a hack to get around it. The issue is that everyone is unaware of this trend at a low level. They are unaware that SQL lacks composability just like how they are unaware that OOP lacks composability as well. But they are aware of this issue at a higher level and they tend to call it "technical debt" or some high level design problem. Case in point: https://news.ycombinator.com/item?id=24732789 https://news.ycombinator.com/item?id=24732789 Most commenters above talk about minor syntactical issues and fail to address what is not only IMO the main issue, but the main issue that the article itself is addressing. Likely because they're all unaware of the true nature of the composability issue and just didn't completely understand what the article was saying. Also note that when I talk about composition in OOP I am not talking about "object composition." These are completely different usages of the word.
- cafard 6y ago
  Is this really a SQL feature or a relational feature? The essence of the relational model is that names are known up front: an attribute of one relation is not an attribute of another.
  - 6y ago
cafard 6y ago
"Mike Stonebraker of Ingres didn’t even bother to show up at the committee meeting to make the (quite strong) case for adopting QUEL because he was ideologically opposed to setting technology standards. It was the behavior of an intellectually arrogant academic rather than a prudent businessman protecting the interests of his company." Some might call the behavior principled, rather than arrogant.
- lisper 6y ago
  Those are not mutually exclusive. (One could add "foolish" and "short-sighted" to the list of potentially compatible adjectives.)
  - cheschire 6y ago
    And RMS being a good example of a principled and arrogant man that was neither foolish nor short sighted.
    - znpy 6y ago
      The more we go forward with technology and everything, the more I understand: Stallman was 110% damn right the whole time, and was lightyears ahead in seeing what other people couldn't see. Proprietary software is a cancer.
- titanomachy 6y ago
  In retrospect, standards turned out to be rather important.
jgalt212 6y ago
> technically superior alternatives like Dvorak and Esperanto would have taken over. The best thing about English is the lack of accent marks. Makes each glyph unique (other than casing). Sorts are faster and not ambiguous.
- samatman 6y ago
  It's also untrue. It's good, for some purposes, that English can be written conventionally by ignoring the accent in words such as résumé. But there are plenty of contexts, and I would say most of them, where this is going to bite you.
  - jgalt212 6y ago
    please provide a few real world examples.
jhallenworld 6y ago
I don't like either syntax. Wikipedia has this example: QUEL: range of E is EMPLOYEE retrieve into W (COMP = E.Salary / (E.Age - 18)) where E.Name = "Jones" SQL: select (e.salary / (e.age - 18)) as comp from employee as e where e.name = "Jones" I would prefer an operator syntax that directly mimics relational algebra. Something like: w = employee(name == "Jones")[comp = salary / (age - 18)] So () is "where", [] is "project" (choose or create columns) and you can use * for join and + for union. The result is a table with a column named comp.
- brian_herman 6y ago
  That sounds like a cool language.
- boramalper 6y ago
  I would reorder round and square brackets, since I may want to filter on computed/created columns, and the ordering makes it clearer.
  - jhallenworld 6y ago
    It's up to you (and the query optimizer can reorder), but you have to make sure the column is available when you use it. Broken up: I had: a = employee(name == "Jones") a has name, salary and age w = a[comp = salary / (age - 18)] w has comp You want: b = employee[name, comp = salary / (age - 18)] b has name and comp w = b(name == "Jones" && comp > 500) w has name and comp In one line: w = employee[name, comp = salary / (age - 18)](name == "Jones" && comp > 500)
  - a1369209993 6y ago
    I'm not sure if you misunderstood them or I misunderstood you, but there's no legitimate reason you can't filter on computed columns as is: w = employee(name == "Jones")[comp = salary / (age - 18)](comp > 2000)
- roenxi 6y ago
  They should implement something using straight functions, extremely spartan with no special syntax at all, and let everyone build their own favoured DSL over the top. One of my major complaints about SQL is the syntax is so finicky that it is really hard to replace it with a [something -> sql] layer, because the something layer can't generate all the silly syntactic forms that SQL uses. Eg, personal favourite, it is easy to have a dsl that translates select(y = fn(x)) -> select fn(x) as y that then breaks down because it can't construct ??? -> select extract(month from x) as y and that is the only syntax the SQL database decided to understand. There are too many cases like that that need special handling, especially once SQL dialect-specific stuff comes into play.
hackerfromthefu 6y ago
Would anyone familiar with both LINQ and QUEL be able to comment on similarities and differences?
brian_herman 6y ago
I want him to continue telling the Postgres Story.
samatman 6y ago
> In a real coup we hired a superb team from Xerox PARC. Once again, California's absence of noncompetes plays a critical role in the success of a business.
tannhaeuser 6y ago
Had expected to read sth about Datalog, rivaling SQL at least in academic DB literature.
- dreamcompiler 6y ago
  Same here. Datalog is superior to both, but alas.
  - scott_meyer 6y ago
    Don't despair. Datalog is alive and well. Yes, it does compose beautifully. However, with composition solved, you'll discover that a naive implementation of relational algebra will suffer from spurious cross products. https://engineering.linkedin.com/blog/2020/liquid-the-soul-of-a-new-graph-database-part-1 https://engineering.linkedin.com/blog/2020/liquid-the-soul-o... https://engineering.linkedin.com/blog/2020/liquid--the-soul-of-a-new-graph-database--part-2 https://engineering.linkedin.com/blog/2020/liquid--the-soul-...
    - refset 6y ago
      This is pretty interesting stuff, thanks for sharing! Does temporal information get any special treatment in the economic graph at the moment? Are temporal queries of any interest? I ask because I work on Crux [0] which at its core is a point-in-time bitemporal Datalog engine, and I see a lot of similarities (schemaless core, Worst-Case Optimal Join etc.) [0] https://opencrux.com https://opencrux.com
      - scott_meyer 6y ago
        Aside from what comes for free with "log-structured" there is no special treatment for temporal data.
- eternalban 6y ago
  Apparently it is not easy to create general purpose datalog engines that scale. "In this paper, we started with the observation that Datalog engines do not translate across domains. We experimentally evaluated the advantages and disadvantages of existing techniques, and compared them with our own baseline, a general-purpose, parallel, in-memory Datalog solver (RecStep) built upon a rdbms. "We presented the necessary optimizations and guidelines to achieve efficiency, and demonstrated that RecStep is scalable, applicable to a range of application domains, and is competitive with highly op- timized and specialized Datalog solvers." vldb.org/pvldb/vol12/p695-fan.pdf (2019)
timpark 6y ago
Long ago, I worked for a company that had to convert its entire accounting system from QUEL to SQL. Fortunately, I was able to write a parser to find the queries and rewrite them, at least for whatever subset of the language they used. It's been a while, but I think there was an issue with multi-query transactions, so the program warned that you'd have to convert and/or verify some parts yourself, but fortunately there weren't too many of those.
somurzakov 6y ago
SQL is perfect. if you want composability of SQL, just do SQL code-generation with your bare hands and compose whatever and however you want
jp0d 6y ago
I agree with some of the comments about composabilty of SQL. I've been doing SQL since more than 14 years. Most of it was spent working on ETL projects for Finance and Utilities industries. Even today 80% of the code I write in pySpark is just plain SQL. It's been my bread and butter. However, I spend a lot of time trying to think about a solution in SQL. It's not an easy language to use when it comes to implementing complex transformations. I could write the same logic in Python in a lot less time. I use SQL mostly because it's easily portable across systems and most analysts and to some extent tech managers understand it. I work primarily on proof of concept data products and it does the job for that. Then a real developer takes over and implements it in .Net.
hn_throwaway_99 6y ago
Minor point, and others have brought up more details around composability, but I think it's an absolute mistake to reference properties on an object before that object is declared. I.e. if SQL had the FROM clause before the SELECT clause autocomplete support could be much more intelligent about helping with SELECT columns. Same problem with declaring imports in javascript. Python got it correct, where I write e.g. 'from somemodule import ...', so by the time I get to the import the parser has enough info to help with autocomplete. Javascript's 'import { Something } from "foo"' means the parser can't help me autocomplete Something.
- earthboundkid 6y ago
  I find imports in Python completely backwards conceptually compared to JS. I hadn’t thought about the autocomplete issue though. In practice, I know what I want to import, I guess.
  - ric2b 6y ago
    I think it makes more sense the way Python does it. You get all the dependencies more or less in a column on the left, instead of jumbled at different widths on the right side.
me2i81 6y ago
In my first job I used Ingres/Quel. Probably because of that, I still find SQL hideous. Quel was a lot more orthogonal and clean, but by now SQL has so many more features that they're not really comparable. The first version of Ingres ran on a PDP-11/70, and the different components (parser, query optimizer, query executor, etc.) ran in separate processes connected via pipes (this was pre-socket Berkeley Unix) because each process could only be 64K 16-bit words. It was hideously slow--ran a lot faster once it was ported to the VAX and everything could run in one process. INGRES originally stood for something like Interactive Graphics Retrieval System, IIRC because the original funding agency wanted a graphics database. Stonebraker wanted to build a relational DBMS so he just went ahead and did it, but gave it a grant-compliant name and wrote some bullshit about graphics to make the funding agency happy.
nl 6y ago
Composition might be desirable but in the 1980s and early 90s it didn't really matter because the databases of the day weren't powerful enough to do the multiple table joins where composition is useful with enough speed to be usable. DBAs of the day would spend a long time optimising physical storage of tables and building aggregation tables to make up for this performance deficit.
ilaksh 6y ago
This is a fundamental difference that I have with almost all of humanity. People do not know the difference between popularity and merit. They don't realize that the reason we do things is because that is how we do things. And they put a lot of effort into rationalizing the way we do things, without realizing the subconscious psychological (rather than rational) basis for that. Most people fundamentally are generally unable to question assumptions about technology or how the world works. This is one reason why, even though I am rooting for the human species, I am doubtful we will be able to stay relevant for long as autonomous general machine intelligence is built and deployed. Even with extensive augmentation, it's obvious that there are just severe limitations to the human mind. The nice thing is that we have the opportunity to design successors that will not be limited in so many ways.
mitchtbaum 6y ago
The path forward https://github.com/speakeasy-engine/speakeasy/issues/26 https://github.com/speakeasy-engine/speakeasy/issues/26
unnouinceput 6y ago
Quote: "Of course, there was a silver lining to the whole saga. Stonebraker had forked the Ingres codebase in 1982 to create his company. Defeated by the bruising database wars of the 80s, he returned to Berkeley in 1985, and started a post-Ingres database project. Naturally, he named that database post-gres — as in, after Ingres. And thus PostgreSQL was born." And 35 years later PostgreSQL is kicking Oracle's butt at every corner.
geophile 6y ago
Dupe: https://news.ycombinator.com/item?id=24709031 https://news.ycombinator.com/item?id=24709031 As I said the last time this came around: The end of this isn't quite right. The Postgres project started in 1986. I don't recall what language it used, QUEL perhaps, but it wasn't SQL. SQL support was added between 1994 and 1996, and that's when PostgreSQL was born.
- wenc 6y ago
  I don't see the contradiction you mention. Stonebraker returned to Berkeley in 1985. The article doesn't say the Postgres project started in 1985. It says Stonebraker started a post-Ingres project then. The query language at that time might have been either QUEL or POSTQUEL [1] But the ending as it is written seems correct to me. [1] https://dsf.berkeley.edu/papers/ERL-M85-95.pdf https://dsf.berkeley.edu/papers/ERL-M85-95.pdf
  - geophile 6y ago
    I didn't say that there was a contradiction. The paper is about QUEL vs. SQL. Given that Postgres/PostgreSQL is introduced, I would think that the initial use of QUEL (PostQUEL), and how and when and why it transitioned to SQL would be highly relevant. But the end of the paper is needlessly fuzzy on this topic.
    - wenc 6y ago
      This is the ending. Quote: "The world has since standardised on SQL, and the dreams of an alternate history exists only in the heads of those who had a hand in the early database wars. It was simply a quirk of history that System R was built within IBM, the single most powerful company in the computer industry at the time; it was a quirk that the engineers who built System R came up with a fiddly language interface as an afterthought, and it was a quirk that IBM then took that language and pushed it to become a standard … one that has lasted till today. Of course, there was a silver lining to the whole saga. Stonebraker had forked the Ingres codebase in 1982 to create his company. Defeated by the bruising database wars of the 80s, he returned to Berkeley in 1985, and started a post-Ingres database project. Naturally, he named that database post-gres — as in, after Ingres. And thus PostgreSQL was born." I was trying to figure what you meant. I think I have an inkling now -- let me know if this is correct. Your quibble is with the fact the implication here is that since SQL won, Stonebraker jumped on the SQL bandwagon and created a SQL database, when in fact, he didn't -- he merely created Postgres, which ran on QUEL and didn't have SQL until much later. I think the author made a stylistic choice to omit that detail to drive home a point, but even so nothing was said that was non-factual.
ngcc_hk 6y ago
Learn cj date and codd those days before even oracle is a thing. It is such a hard sell we have to go through IMS and network db and the only rdb is dbase II. At least we get db2 before we do foxbase.
- deleted 6y ago
  [deleted]
TehShrike 6y ago
I'd been aware of Codd's relational theory, but never heard of QUEL – now I want to find a database engine that implements QUEL!
- runroader 6y ago
  As far as I'm aware only Ingres supports QUEL Edit: In case you actually do want to play with it, Ingres is now Actian IngresX. Though I'd recommend thinking about what could have been instead of actually spending any time fighting with and configuring Ingres.
  - rrgmitchell 6y ago
    The latest iteration is actually called 'ActianX', and it's a continuation of Ingres plus support for column-based tables from the 'Vector' product acquired by Actian. QUEL is still there, but its functionality has been frozen for years and, in terms of bells and whistles, is way behind modern SQL. If you do want to try it, I had no trouble installing it on my home Ubuntu system a couple of years ago. Ingres is pretty easy to use in some ways. If you want to make a database, you just type "createdb mydatabase" at the command line.
  - znpy 6y ago
    Uh, it's nice to see the name Actian come out for once. My previous employer used Versant by Actian (now called Actian NoSQL) heavily. It's much more of a NoSQL database: it's a real object oriented database. You don't store tuples, you store objects. You don't make queries with selection and projection, you make a cut of a graph of objects. It's insanely fast, multithreaded, has very good tooling, scales vertically very well, can do online schema evolution (class definition evolution, really). Sadly it's almost impossible to scale horizontally (I'd be glad to be proven wrong). It's basically what the industry needs to avoid the object-relational mismatch: an object oriented database. But everybody only learns SQL...
- cameldrv 6y ago
  Postgres originally implemented QUEL, until Postgres95 which implemented SQL (later renamed to Postgresql)

256 points by huy 6y ago

Good history. But, thanks to https://github.com/haskell-beam/beam https://github.com/haskell-beam/beam I no longer need to worry about the COBOL-ness of SQL without giving up the semantics. Finally!

kmeisthax 6y ago

>If the world worked differently, we wouldn’t still be writing on QWERTY keyboards, or speaking English; technically superior alternatives like Dvorak and Esperanto would have taken over. Bad metaphor: There is no evidence for Dvorak's technical superiority to QWERTY, neither is there for Esperanto over other languages.

nine_k 6y ago
Esperanto is similar to a bunch of Roman languages widespread in Europe. One can see it as an attempt to collect them back into a common "Latin 2.0". Outside Western Europe, Esperanto makes rather little sense. It's highly regular, which is nice — but, say, Japanese is also highly regular.
- ithkuil 6y ago
  Yes it's based off european languages; it was designed in a time when removing barriers from neighboring countries in a culturally very fragmented continent was perceived of higher practical value than creating a pan-human language; intercontinental travel wasn't yet as commonplace as now, yet many european countries had different cultures living within the boundaries of the same states and usually members of the majority culture were privileged as a result; there was hope (espero in esperanto) that people could forget about differences among them and see what they have in common. Today we can be tempted to frame it as european chauvinism, but that's just because of our expanded horizons as a global society. People who believes in esperanto in the early days would likely share the same feeling now
cbsmith 6y ago
I think the "technically superior" bit wasn't really the right choice of words. Esperanto isn't intended to be superior. It's value is on it being equally foreign yet approachable for all the salient parties and therefore a conceivable acceptable neutral turf for everyone to share. Ironically, the case for Dvorak keyboards is kind of the opposite: QWERTY was intentionally designed to avoid jams, which if anything biased towards making it more difficult. It's not so much technical superiority as having a design objective that is more appropriate for the problem space. One can similarly argue about whether "QUEL" is really technically superior to "SQL", but the design objective is (at least as perceived by the author) better aligned with the solution space.
- thaumasiotes 6y ago
  > Esperanto isn't intended to be superior. It's value is on it being equally foreign yet approachable for all the salient parties and therefore a conceivable acceptable neutral turf for everyone to share. This was not even attempted; Esperanto is a Romance language. Unless you think the only salient parties are Spain, France, Portugal, Italy, and Latin America, this "value" does not exist and was not a goal.
  - mijamo 6y ago
    Esperanto is not a romance language. It incorporates elements from most european languages families (balto-slavic, germanic, romance), and the grammar is most particularly inspired by slavic languages. And it was pretty much a goal of its creator to be both familiar and foreign for different european language families, as he had experienced division in Poland between speakers of those different families and wanted to have something more universal to gather them all together.
    - eska 6y ago
      We're still ignoring the entire Asian and African continents then, though.
      - 6y ago
tbenst 6y ago
Dvorak is 25% more efficient than Qwerty using some very reasonable calculations: http://mkweb.bcgsc.ca/carpalx/?dvorak http://mkweb.bcgsc.ca/carpalx/?dvorak
- bsder 6y ago
  And by some other measures (hand alternation of fingers) Dvorak is less efficient. QWERTY isn't as bad as people make it out to be, and Dvorak isn't as good as people make it out to be. There are keyboard configurations that are better than both, but nobody uses them because you'll never be able to use anybody else's keyboard.
  - edjroot 6y ago
    > nobody uses them because you'll never be able to use anybody else's keyboard I doubt that's the main reason, and it may be unwarranted as I say below. Personally, I thought it would take too long for me to be able to type as fast as in QWERTY. Fortunately, I was wrong about that too. I learned the QGMLW optimized layout from Carpalx (linked above) about a year ago. I trained on https://keybr.com https://keybr.com and then on https://typeracer.com https://typeracer.com for a few days and in less than a week I reached 70wpm. I stopped training shortly after and now I can usually reach 90-100wpm. Not particularly fast, but I'm not any faster in QWERTY, and I believe I could get faster if I trained more. Even though I almost never type in QWERTY anymore, when I do I'm still just as fast as I used to be. I never learned to type in QWERTY "the right way", though, and I suppose this is actually what keeps me from confusing the two "modes". Whenever I try to keep my fingers on their "appropriate" keys (like a properly trained typist would), it's like my brain switches to "Carpalx mode" (since I did make an effort to use the right finger positioning to learn it). It's kinda like switching between thinking in my native language and English - I can think in both, but it doesn't "feel" the same, and I'm more likely to confuse the two where they overlap more. Pretty interesting, really.
    - greenshackle2 6y ago
      Yeah I did the same thing but I learned colemak. I never learned to touch-type qwerty, when I learned touch-typing I switched to colemak. I feel like it made things easier as I didn't have bad habits to fall back on. I can still type qwerty but I'm not very fast at it (I never was).
deleted 6y ago
[deleted]

exabrial 6y ago

Really the the only issue I have with SQL is NULL != NULL. This creates an impedance mismatch with most languages... MySQL sort of solves this problem with a <=> operator, which I wish was the default for ORMs to use. There are a lot of other minor nitpicks but a lot of criticisms come down to the actual RDMS not SQL itself.

nendroid 6y ago
What about the main point of the article that SQL is not composable.
- jmalicki 6y ago
  CTEs (common table expressions) and views definitely do help with this, though they are new-ish where they exist and often have optimization issues. But being able to use them extensively in a newer database where they work well helps this quite a bit.
  - nendroid 6y ago
    Tables are composable but the expression itself cannot be decomposed. I cannot reuse a where clause somewhere else; that is the fundamental problem the article addresses.
sumtechguy 6y ago
I just wish they had flipped from and select around.
- NickSharp 6y ago
  Or "set" and "where"
- nendroid 6y ago
  It's not about flipping. The two concepts are actually commutative. You could in theory create syntax that looks like this: FROMCLAUSE * SELECTCLAUSE * WHERECLAUSE = SQLEXPRESSION SELECTCLAUSE * WHERECLAUSE * FROMCLAUSE = SQLEXPRESSION ... The issue is that not only does SQL syntax force an artificial order on these clauses, but that these clauses Cannot be decomposed to be used elsewhere. I cannot reuse a WHERECLAUSE or a SELECTCLAUSE in another expression.

jrochkind1 6y ago

can anyone find examples of what QUEL looked like? If we had a more composable query language being used instead of SQL, I wonder if that would have effected the course of ORMs, which arguably end up being as much about composable models of queries as they do about actual object mapping.

knodi123 6y ago
sure. https://en.wikipedia.org/wiki/QUEL_query_languages#Usage https://en.wikipedia.org/wiki/QUEL_query_languages#Usage
GordonS 6y ago
I kept searching the article, thinking I must have somehow missed the examples - nope! How very odd to wrote an article about how much better QUEL was than SQL, and not have a single example of either!

eecc 6y ago

are there any usable implementations of QUEL in the wild?

alquemist 6y ago

> … The language (SQL) is not very composable. This is a fact that most SQL users are not aware of. The relational algebra that SQL is based on is absolutely composable but SQL is not due to the inherent limitation of the language (as it was designed to be natural language-like). When you write "select x from a where z", you are actually building something along the lines of "from a" => "where z" => "select x" in the algebra and you can actually compose each portion separately. If you are familiar with dplyr, Spark or pandas you would get this instantly. Hmmm. Not a big lover of SQL, but this is a bit imprecise. While the unit of composability is slightly smaller for from=>where=>select (expression) vs select+from+where (subquery), in practice they both encode the same fundamental compositional principles, based on relational algebra. Any from=>where=>select query can be translated into a select+where+from query almost 1:1, if only via: from t ::= select * from t q => where p ::= select * from q where p q => select xs ::= select xs from q Sometimes the select+where+from ends up more verbose, sometimes there is more brain twisting to grok a given select+where+from query, but that's not a composition limiting factor. Granted, the readability of SQL is sometimes lacking, but it is fully capable to compose recursive relational algebra queries.

contravariant 6y ago
You're arguing about capability, whereas the post is arguing about the grammar itself. What you're describing is how to build a separate domain specific language that compiles to SQL (which there are quite a few of; e.g. C#'s Linq). Quite a few query languages are equally capable (including, surprisingly, quite a few so-called graph languages, provided the SQL dialect provides a transitive closure), but SQL as a language has some undesirable properties (most of which are trade-offs for the fact that basic SQL is very easy to parse).
asah 6y ago
Subqueries, named VIEWs, CTEs, etc all make SQL compostable ?
- radiowave 6y ago
  Yes, but it's extremely clunky. The number of times I've had to write out a whole chain of CTEs just because I wanted to apply one window function to the output of another. In which context, "compostable" is a fantastic Freudian typo.

mamcx 6y ago

I'm working in a relational language that if it work as I wish, could eventually be put on top of a RDBM like sqlite or layer for other DBs: https://github.com/Tablam/TablaM https://github.com/Tablam/TablaM It have ideas similar to QUEL...

prostodata 6y ago
Just for comparison, a functional approach is a major alternative to relational and set-oriented models and query languages. The difference is that functions and operations with functions are first class elements of the model. One version of it is implemented in this project: https://github.com/prostodata/prosto https://github.com/prostodata/prosto - Functions matter! No join-groupby, No map-reduce

nendroid 6y ago

The article talks about how SQL lacks composability. I would like to know everyones thoughts about this. This is a huge issue with programming in general not exclusive to SQL. Everyone would like to build programs that are modular and reusable but programming paradigms have been traveling in directions that prevent this from happening. Many people turn to design patterns or microservices to try to deal with this organizational issue but they fail to see that the lower level programming paradigm itself is the precursor to the problem. In SQL the problem occurs in the statement itself. The WHERE clause or the SELECT clause cannot be reused anywhere else. I can't modularize a where clause and put it in another SQL statement. I have to rewrite the entire clause to reuse it. In OOP the same issue occurs. In OOP your class tends to contain methods that are not combinators, or in other words methods that modify a free variable. Due to this like the SQL expression, Objects cannot be decomposed either. I cannot reuse a setter in another class or anywhere else outside of the context of the free variable it modifies. In both cases there comes a time in the future of an application where programmers realize that similar logic could be reused but structural problems are preventing the reuse from happening so they have to implement a hack to get around it. The issue is that everyone is unaware of this trend at a low level. They are unaware that SQL lacks composability just like how they are unaware that OOP lacks composability as well. But they are aware of this issue at a higher level and they tend to call it "technical debt" or some high level design problem. Case in point: https://news.ycombinator.com/item?id=24732789 https://news.ycombinator.com/item?id=24732789 Most commenters above talk about minor syntactical issues and fail to address what is not only IMO the main issue, but the main issue that the article itself is addressing. Likely because they're all unaware of the true nature of the composability issue and just didn't completely understand what the article was saying. Also note that when I talk about composition in OOP I am not talking about "object composition." These are completely different usages of the word.

cafard 6y ago
Is this really a SQL feature or a relational feature? The essence of the relational model is that names are known up front: an attribute of one relation is not an attribute of another.
- 6y ago

cafard 6y ago

"Mike Stonebraker of Ingres didn’t even bother to show up at the committee meeting to make the (quite strong) case for adopting QUEL because he was ideologically opposed to setting technology standards. It was the behavior of an intellectually arrogant academic rather than a prudent businessman protecting the interests of his company." Some might call the behavior principled, rather than arrogant.

lisper 6y ago
Those are not mutually exclusive. (One could add "foolish" and "short-sighted" to the list of potentially compatible adjectives.)
- cheschire 6y ago
  And RMS being a good example of a principled and arrogant man that was neither foolish nor short sighted.
  - znpy 6y ago
    The more we go forward with technology and everything, the more I understand: Stallman was 110% damn right the whole time, and was lightyears ahead in seeing what other people couldn't see. Proprietary software is a cancer.
titanomachy 6y ago
In retrospect, standards turned out to be rather important.

jgalt212 6y ago

> technically superior alternatives like Dvorak and Esperanto would have taken over. The best thing about English is the lack of accent marks. Makes each glyph unique (other than casing). Sorts are faster and not ambiguous.

samatman 6y ago
It's also untrue. It's good, for some purposes, that English can be written conventionally by ignoring the accent in words such as résumé. But there are plenty of contexts, and I would say most of them, where this is going to bite you.
- jgalt212 6y ago
  please provide a few real world examples.

jhallenworld 6y ago

I don't like either syntax. Wikipedia has this example: QUEL: range of E is EMPLOYEE retrieve into W (COMP = E.Salary / (E.Age - 18)) where E.Name = "Jones" SQL: select (e.salary / (e.age - 18)) as comp from employee as e where e.name = "Jones" I would prefer an operator syntax that directly mimics relational algebra. Something like: w = employee(name == "Jones")[comp = salary / (age - 18)] So () is "where", [] is "project" (choose or create columns) and you can use * for join and + for union. The result is a table with a column named comp.

brian_herman 6y ago
That sounds like a cool language.
boramalper 6y ago
I would reorder round and square brackets, since I may want to filter on computed/created columns, and the ordering makes it clearer.
- jhallenworld 6y ago
  It's up to you (and the query optimizer can reorder), but you have to make sure the column is available when you use it. Broken up: I had: a = employee(name == "Jones") a has name, salary and age w = a[comp = salary / (age - 18)] w has comp You want: b = employee[name, comp = salary / (age - 18)] b has name and comp w = b(name == "Jones" && comp > 500) w has name and comp In one line: w = employee[name, comp = salary / (age - 18)](name == "Jones" && comp > 500)
- a1369209993 6y ago
  I'm not sure if you misunderstood them or I misunderstood you, but there's no legitimate reason you can't filter on computed columns as is: w = employee(name == "Jones")[comp = salary / (age - 18)](comp > 2000)
roenxi 6y ago
They should implement something using straight functions, extremely spartan with no special syntax at all, and let everyone build their own favoured DSL over the top. One of my major complaints about SQL is the syntax is so finicky that it is really hard to replace it with a [something -> sql] layer, because the something layer can't generate all the silly syntactic forms that SQL uses. Eg, personal favourite, it is easy to have a dsl that translates select(y = fn(x)) -> select fn(x) as y that then breaks down because it can't construct ??? -> select extract(month from x) as y and that is the only syntax the SQL database decided to understand. There are too many cases like that that need special handling, especially once SQL dialect-specific stuff comes into play.

hackerfromthefu 6y ago

Would anyone familiar with both LINQ and QUEL be able to comment on similarities and differences?

brian_herman 6y ago

I want him to continue telling the Postgres Story.

samatman 6y ago

> In a real coup we hired a superb team from Xerox PARC. Once again, California's absence of noncompetes plays a critical role in the success of a business.

tannhaeuser 6y ago

Had expected to read sth about Datalog, rivaling SQL at least in academic DB literature.

dreamcompiler 6y ago
Same here. Datalog is superior to both, but alas.
- scott_meyer 6y ago
  Don't despair. Datalog is alive and well. Yes, it does compose beautifully. However, with composition solved, you'll discover that a naive implementation of relational algebra will suffer from spurious cross products. https://engineering.linkedin.com/blog/2020/liquid-the-soul-of-a-new-graph-database-part-1 https://engineering.linkedin.com/blog/2020/liquid-the-soul-o... https://engineering.linkedin.com/blog/2020/liquid--the-soul-of-a-new-graph-database--part-2 https://engineering.linkedin.com/blog/2020/liquid--the-soul-...
  - refset 6y ago
    This is pretty interesting stuff, thanks for sharing! Does temporal information get any special treatment in the economic graph at the moment? Are temporal queries of any interest? I ask because I work on Crux [0] which at its core is a point-in-time bitemporal Datalog engine, and I see a lot of similarities (schemaless core, Worst-Case Optimal Join etc.) [0] https://opencrux.com https://opencrux.com
    - scott_meyer 6y ago
      Aside from what comes for free with "log-structured" there is no special treatment for temporal data.
eternalban 6y ago
Apparently it is not easy to create general purpose datalog engines that scale. "In this paper, we started with the observation that Datalog engines do not translate across domains. We experimentally evaluated the advantages and disadvantages of existing techniques, and compared them with our own baseline, a general-purpose, parallel, in-memory Datalog solver (RecStep) built upon a rdbms. "We presented the necessary optimizations and guidelines to achieve efficiency, and demonstrated that RecStep is scalable, applicable to a range of application domains, and is competitive with highly op- timized and specialized Datalog solvers." vldb.org/pvldb/vol12/p695-fan.pdf (2019)

timpark 6y ago

Long ago, I worked for a company that had to convert its entire accounting system from QUEL to SQL. Fortunately, I was able to write a parser to find the queries and rewrite them, at least for whatever subset of the language they used. It's been a while, but I think there was an issue with multi-query transactions, so the program warned that you'd have to convert and/or verify some parts yourself, but fortunately there weren't too many of those.

somurzakov 6y ago

SQL is perfect. if you want composability of SQL, just do SQL code-generation with your bare hands and compose whatever and however you want

jp0d 6y ago

I agree with some of the comments about composabilty of SQL. I've been doing SQL since more than 14 years. Most of it was spent working on ETL projects for Finance and Utilities industries. Even today 80% of the code I write in pySpark is just plain SQL. It's been my bread and butter. However, I spend a lot of time trying to think about a solution in SQL. It's not an easy language to use when it comes to implementing complex transformations. I could write the same logic in Python in a lot less time. I use SQL mostly because it's easily portable across systems and most analysts and to some extent tech managers understand it. I work primarily on proof of concept data products and it does the job for that. Then a real developer takes over and implements it in .Net.

hn_throwaway_99 6y ago

Minor point, and others have brought up more details around composability, but I think it's an absolute mistake to reference properties on an object before that object is declared. I.e. if SQL had the FROM clause before the SELECT clause autocomplete support could be much more intelligent about helping with SELECT columns. Same problem with declaring imports in javascript. Python got it correct, where I write e.g. 'from somemodule import ...', so by the time I get to the import the parser has enough info to help with autocomplete. Javascript's 'import { Something } from "foo"' means the parser can't help me autocomplete Something.

earthboundkid 6y ago
I find imports in Python completely backwards conceptually compared to JS. I hadn’t thought about the autocomplete issue though. In practice, I know what I want to import, I guess.
- ric2b 6y ago
  I think it makes more sense the way Python does it. You get all the dependencies more or less in a column on the left, instead of jumbled at different widths on the right side.

me2i81 6y ago

In my first job I used Ingres/Quel. Probably because of that, I still find SQL hideous. Quel was a lot more orthogonal and clean, but by now SQL has so many more features that they're not really comparable. The first version of Ingres ran on a PDP-11/70, and the different components (parser, query optimizer, query executor, etc.) ran in separate processes connected via pipes (this was pre-socket Berkeley Unix) because each process could only be 64K 16-bit words. It was hideously slow--ran a lot faster once it was ported to the VAX and everything could run in one process. INGRES originally stood for something like Interactive Graphics Retrieval System, IIRC because the original funding agency wanted a graphics database. Stonebraker wanted to build a relational DBMS so he just went ahead and did it, but gave it a grant-compliant name and wrote some bullshit about graphics to make the funding agency happy.

nl 6y ago

Composition might be desirable but in the 1980s and early 90s it didn't really matter because the databases of the day weren't powerful enough to do the multiple table joins where composition is useful with enough speed to be usable. DBAs of the day would spend a long time optimising physical storage of tables and building aggregation tables to make up for this performance deficit.

ilaksh 6y ago

This is a fundamental difference that I have with almost all of humanity. People do not know the difference between popularity and merit. They don't realize that the reason we do things is because that is how we do things. And they put a lot of effort into rationalizing the way we do things, without realizing the subconscious psychological (rather than rational) basis for that. Most people fundamentally are generally unable to question assumptions about technology or how the world works. This is one reason why, even though I am rooting for the human species, I am doubtful we will be able to stay relevant for long as autonomous general machine intelligence is built and deployed. Even with extensive augmentation, it's obvious that there are just severe limitations to the human mind. The nice thing is that we have the opportunity to design successors that will not be limited in so many ways.

mitchtbaum 6y ago

The path forward https://github.com/speakeasy-engine/speakeasy/issues/26 https://github.com/speakeasy-engine/speakeasy/issues/26

unnouinceput 6y ago

Quote: "Of course, there was a silver lining to the whole saga. Stonebraker had forked the Ingres codebase in 1982 to create his company. Defeated by the bruising database wars of the 80s, he returned to Berkeley in 1985, and started a post-Ingres database project. Naturally, he named that database post-gres — as in, after Ingres. And thus PostgreSQL was born." And 35 years later PostgreSQL is kicking Oracle's butt at every corner.

geophile 6y ago

Dupe: https://news.ycombinator.com/item?id=24709031 https://news.ycombinator.com/item?id=24709031 As I said the last time this came around: The end of this isn't quite right. The Postgres project started in 1986. I don't recall what language it used, QUEL perhaps, but it wasn't SQL. SQL support was added between 1994 and 1996, and that's when PostgreSQL was born.

wenc 6y ago
I don't see the contradiction you mention. Stonebraker returned to Berkeley in 1985. The article doesn't say the Postgres project started in 1985. It says Stonebraker started a post-Ingres project then. The query language at that time might have been either QUEL or POSTQUEL [1] But the ending as it is written seems correct to me. [1] https://dsf.berkeley.edu/papers/ERL-M85-95.pdf https://dsf.berkeley.edu/papers/ERL-M85-95.pdf
- geophile 6y ago
  I didn't say that there was a contradiction. The paper is about QUEL vs. SQL. Given that Postgres/PostgreSQL is introduced, I would think that the initial use of QUEL (PostQUEL), and how and when and why it transitioned to SQL would be highly relevant. But the end of the paper is needlessly fuzzy on this topic.
  - wenc 6y ago
    This is the ending. Quote: "The world has since standardised on SQL, and the dreams of an alternate history exists only in the heads of those who had a hand in the early database wars. It was simply a quirk of history that System R was built within IBM, the single most powerful company in the computer industry at the time; it was a quirk that the engineers who built System R came up with a fiddly language interface as an afterthought, and it was a quirk that IBM then took that language and pushed it to become a standard … one that has lasted till today. Of course, there was a silver lining to the whole saga. Stonebraker had forked the Ingres codebase in 1982 to create his company. Defeated by the bruising database wars of the 80s, he returned to Berkeley in 1985, and started a post-Ingres database project. Naturally, he named that database post-gres — as in, after Ingres. And thus PostgreSQL was born." I was trying to figure what you meant. I think I have an inkling now -- let me know if this is correct. Your quibble is with the fact the implication here is that since SQL won, Stonebraker jumped on the SQL bandwagon and created a SQL database, when in fact, he didn't -- he merely created Postgres, which ran on QUEL and didn't have SQL until much later. I think the author made a stylistic choice to omit that detail to drive home a point, but even so nothing was said that was non-factual.

ngcc_hk 6y ago

Learn cj date and codd those days before even oracle is a thing. It is such a hard sell we have to go through IMS and network db and the only rdb is dbase II. At least we get db2 before we do foxbase.

deleted 6y ago
[deleted]

TehShrike 6y ago

I'd been aware of Codd's relational theory, but never heard of QUEL – now I want to find a database engine that implements QUEL!

runroader 6y ago
As far as I'm aware only Ingres supports QUEL Edit: In case you actually do want to play with it, Ingres is now Actian IngresX. Though I'd recommend thinking about what could have been instead of actually spending any time fighting with and configuring Ingres.
- rrgmitchell 6y ago
  The latest iteration is actually called 'ActianX', and it's a continuation of Ingres plus support for column-based tables from the 'Vector' product acquired by Actian. QUEL is still there, but its functionality has been frozen for years and, in terms of bells and whistles, is way behind modern SQL. If you do want to try it, I had no trouble installing it on my home Ubuntu system a couple of years ago. Ingres is pretty easy to use in some ways. If you want to make a database, you just type "createdb mydatabase" at the command line.
- znpy 6y ago
  Uh, it's nice to see the name Actian come out for once. My previous employer used Versant by Actian (now called Actian NoSQL) heavily. It's much more of a NoSQL database: it's a real object oriented database. You don't store tuples, you store objects. You don't make queries with selection and projection, you make a cut of a graph of objects. It's insanely fast, multithreaded, has very good tooling, scales vertically very well, can do online schema evolution (class definition evolution, really). Sadly it's almost impossible to scale horizontally (I'd be glad to be proven wrong). It's basically what the industry needs to avoid the object-relational mismatch: an object oriented database. But everybody only learns SQL...
cameldrv 6y ago
Postgres originally implemented QUEL, until Postgres95 which implemented SQL (later renamed to Postgresql)