9 ms·

Little languages are the future of programming

179 points by JohnyTex 4y ago

deleted 4y ago
[deleted]
eterps 4y ago
I think the explanation in https://www.oilshell.org/blog/2022/02/diagrams.html https://www.oilshell.org/blog/2022/02/diagrams.html is also related to this.
nottorp 4y ago
Weren't they called DSLs a while ago? The OP even uses "Domain Specific Language" in the article.
SleepyMyroslav 4y ago
Author and comments are sticking to text based languages for some reason. I think visual ones have proven to be more accessible to non-programmers. There are downsides ofc ... for programmers =) As a programmer myself I do not think it will be great to push visual languages too far but business is business.
vfinn 4y ago
AI will kill the need for all these abstractions. There will be as many or as little abstractions you want. It doesn't matter. New languages of all shapes and forms will be generated eventually on the fly. You will just have to know what you want.
sargstuff 4y ago
Lisp macros much more unified approach to DSL/little language creation/usage.
martin1975 4y ago
a.k.a. Domain Specific Languages Hello ANTLR.. https://martinfowler.com/books/dsl.html https://martinfowler.com/books/dsl.html
jph 4y ago
The Rust `nom` parser crate works very well IMHO for creating little languages, DSLs, custom markup, and your own kinds of annotations, all with the Rust compiler guarantees and good speed.
- tov_objorkin 4y ago
  Oh, please, making reasonable good DSL might take a month or even years. Also wrestling with parser and borrow checker not an easy task for average user.
  - intelVISA 4y ago
    I'm no Rust zealot but the borrow checker is arguably one of the core benefits for the average user. Wrestling with it means they are not yet ready for systems programming and need to understand move semantics more deeply.
    - tov_objorkin 4y ago
      Borrow checker is fine. But from the library writer perspective its pain and take enormous amount of time to make it sound. One does not simple checkout "nom" and test thing in few minutes.
smitty1e 4y ago
My first goal with a challenge at the office is "make the problem smaller". Now I have a folder of little one-offs and REPL scraps. A triumph of tactics over strategy that defies passing on to anyone who didn't author it. Looking at the JVM and JSON, I wonder to what degree that languages contribute some piece of goodness or idea toward some final Tool To Rule Them All...
neilv 4y ago
In addition to the Racket article that the article links, Racket's language support has become more powerful and smoother since then. A more recent treatment is Matthew Butterick's book: https://beautifulracket.com/ https://beautifulracket.com/ It doesn't have to be a big standalone DSL with a separate compiler or preprocessor. It can also be an embedded little language, like when you sprinkle HTML templates throughout your normal general-purpose language, and as only a syntax extension: https://docs.racket-lang.org/html-template/ https://docs.racket-lang.org/html-template/ (Aside: I'm seeing tasteful Racket and Scheme influences in Rust, even though they're very-very different languages. I'm hoping to contribute a little more influences.)
lmm 4y ago
This article claims the problems it's trying to solve are: Hard to onboard new hires, code breaks because of lack of understanding of dependencies, and code changes become harder to manage. In my experience SQL, regexes, unix shell, and listening to Alan Kay, far from solving those problems, are the very things that most exacerbate them. General-purpose languages that are expressive enough to let one write business logic in the language of the domain, but without breaking the rules of the language or requiring new tooling - "internal" rather than "external" DSLs - are a far better way forward.
- sargstuff 4y ago
  > " General-purpose languages that are expressive enough to let one write business logic in the language of the domain, but without breaking the rules of the language or requiring new tooling .." gawk compiled to webassembly would seem to fit the bill -- just shifts gawk from "external" dsl to "internal" dsl. orthogonality allows for usage of all modern interface trappings without any retooling and/or breaking rules of gawk language. Makes gawk a module in group of customized modules to form a general-purpose program out of dsl/little languages.
  - lmm 4y ago
    Transpiling to your main language doesn't make it an internal DSL. Reimplementing what gawk does as a library you can use within your language (and not just by passing opaque strings to it, but by actually expressing awk-style commands in a proper datastructure) is the kind of thing I'm advocating for.
    - sargstuff 4y ago
      Transpiling the language changes the operational domain. Reimplimenting/supporting the library in language means, have to do that for each OS operational domain that the gawk interpreter runs in (aka each relevant OS & OS version for mac, unix, windows, non-pc OS's, plus mobile versions). Modifying language also means that anything that relies on lanuage will have to be re-evaluated to assess potential language change issues/implications. Transpiling to change the operational domain means can just focus on task at hand, as long as not tied to specifics outside the operational domain of webassembly. aka file paths Avoids the need to assess potential language change issues/implications. Changing the operation domain in this case means can leverage the operational environments orthononality to 'express awk-style commands in a propper data structures' that awk doesn't provide without modifiying the language. (or writing the feature in awk) Changing the operation environment also means no required new retooling. Changing the language may force/require retooling. Case study context example: Long term 10-40 year studies where same code written at start of study needs to be runnable / usable / understandable at the end of the study without being rewritten. Note: Hardware and/or OS used at start may be entirely different than that used at end of study. Fairly simple to get 10,000+ awk scripts running under IBM/DEC/Mac/Windows/Unix/BeOS to run under any of the other OS domains using webassembly by transpiling awk. Fairly trivial to leverage browser to permit a user customizable GUI usable under any of the aformentioned OS domains without having to modify/recertify the awk scripts. Finding / recompiling / supporting awk for each of those OS domains (OS & verions variants, GUI libs) bit more time consuming (short term & long term).
laurencerowe 4y ago
I think it's more likely the future will be ever more 'languages' which conform to a schema but are simply expressed as JSON/YAML. Being able to trivially deserialize with a simple JSON.parse or equivalent provides a huge head start. DSLs defined in a type/schema system atop JSON/YAML end up being far easier to write tools around than DSLs which require a custom parser (e.g. Dockerfile.) That said, there are definitely a subset of languages like JSONPath that would not work written out as an AST.
zetsurin 4y ago
Having dabbled in this a bit myself (using small dsl's to solve specific problems), some problems that immediately occur to me: - more difficulty on boarding - more difficulty adding new features - more challenges with best practices/linting/code reviews - different runtime behaviour between different languages This all adds up to more complexity. New languages are often really neat, and enjoyable for their own sake, but I'm not all that interested in maintaining a large swath of different languages for different tools.
chewxy 4y ago
I don't think it's so much "little" languages (commonly DSL) that matter. It's more the jumps in expressivity. You don't use a full on Turing-complete language when you need to match strings written in a regular language. Instead, we write the language we want as a regexp, and then use a regexp engine to match it. I agree with much of the problems listed in the article. The author even manages to stumble onto some of the solutions (e.g. Dhall being a total language). "Expressiveness is co-decidability" is the main theme of these things. The crux of the issue is in our everyday programming tasks, we have many levels of decidability, ranging from RE all the way to things that require full Turing completeness. The majority of work however, lies in the middle. There are so many things that can be done with pushdown automatons, or with deterministic automatons. Most codebases don't actually use those though. An issue is that there is a dearth of "mini" languages that support these things. Another issue is that somehow we are enamoured with the idea that our languages must be able to express everything under the sun (up to TC/Recursively Enumerable). This seems to be more of an industry attitude than anything - there is this chase for the most powerful language (a lisp, clearly... everything else is a blub). I've recently experimented with embedding an APL into my usual programming language, and it was a very interesting experience. It feels like having the power to do regular expression stuff, but with arrays. I want to do the same for the other levels of expressiveness.
- tempodox 4y ago
  What's a “total language”?
  - Tainnor 4y ago
    To slightly oversimplify: A language in which every function provably terminates.
  - TuringTest 4y ago
    It's a programming language where every written program terminates. As a corollary, such languages can't be Turing complete, since universal Turing machines can represent programs which don't terminate. [1] https://en.wikipedia.org/wiki/Total_functional_programming https://en.wikipedia.org/wiki/Total_functional_programming
AlotOfReading 4y ago
People have been making this argument since the 80s and possibly even earlier. My experience is often the opposite. Little languages are usually far, far harder than (mis-)using "big" languages for small tasks. The problem is that your DSL has to be understood by other people, including future you. Programming tasks are vast, combinatorially explosive state spaces full of weird potential interactions between features. Once you get above the complexity and universal familiarity of say, arithmetic, it's difficult for others to understand what's going on just by looking at 1-2 live examples. You have to heavily invest in proper docs and tooling (if your language doesn't provide it for free). By the time you've completed that your "little language" usually isn't such a little effort anymore. If you don't, you've just made the next CMake. Congrats you monster.
- CipherThrowaway 4y ago
  I think EDSLs are the happy medium here.
- lovecg 4y ago
  And even if you do invest in proper documentation and support, you still have to overcome the hurdle that people just _don’t want to spend time learning your one-off language_ - there’s nontrivial opportunity cost in learning something that won’t be useful anywhere else. So people will just do the bare minimum which will lead to misunderstanding and bugs.
  - nottorp 4y ago
    > people just _don’t want to spend time learning your one-off language_ Which people though? If you make a DSL that non programmers in your organization use, I'm sure they will appreciate not having to learn the intricacies of Rust or whatever's in fashion this week.
    - Yizahi 4y ago
      We don't use DSL per se, but a custom tool for writing QA tests, which looks like a kinda Visio block diagram software, only each block is a function or other logical entity. Anyway, after a few years struggling with it, for many different reasons, we are slowly and painfully migrating to writing tests in Python, and every single QA supports it. Custom languages, with limited support, limited community, limited extendability etc. are just like that - limited. And as soon as you hit a wall with them, transition will cost more (in both time and money) than saved in the first place by using "easier" tooling for non-programmers.
bloppe 4y ago
The size of the language is a red herring. You really just want programs that are well structured, which can be greatly helped by choosing a "perfect language" for each task, but often helped just as well by choosing (or creating) a great library to express the business logic.
- martindbp 4y ago
  I think "little" refers to the domain rather than other metrics of language size
lolinder 4y ago
A "little language" is just an abstraction over a small part of the domain. Equivalent abstractions can be and are written as libraries. In a Turing-complete host language, the primary difference is that these libraries don't get the privilege of inventing new syntax, but that's almost always a good thing. We already have major headaches switching between JS, SQL, and {insert backend language here}. Introducing tens of little languages into a codebase may marginally increase readability of each chunk of code in isolation, but the amount of context-switching and required background knowledge it introduces would more than make up the difference. In a abstraction strategy that's based around libraries, every library agrees on the same basic syntax and semantics. You interact with an embedded DSL via a well-defined interface that all libraries respect. The syntax may not be perfectly ideal to any one part of the project, but it's consistent throughout every part of the project. That has real value. I think it's also a red herring to argue about lines of code: stringing together a bunch of little languages is not likely to lead to fewer lines of code than pulling in an equivalent number of third-party libraries, and it will almost certainly increase the total amount of code in your distribution, because each little language must not only implement the functionality desired, it also needs its own parser and interpreter. Take McIlroy's shell script: if you add up the C code to implement each of those little languages, you're at about 10k lines of code to make that bit of code golf possible. I'm a huge fan of DSLs, and I like the analogy of modern programming as pyramid building. I just don't think independent, chained-together DSLs are the answer. I'd rather have a language like Kotlin that is designed around embedded DSLs that all respect the same rules and use the same runtime.
- Kamq 4y ago
  > Take McIlroy's shell script: if you add up the C code to implement each of those little languages, you're at about 10k lines of code to make that bit of code golf possible. This is a fair point, but part of this has to be how battle tested the language in question is, right? Bringing in a single language that's been run through its paces (bourne shell in this case) for text processing, seems like a much lower risk than bringing in a dozen different languages and hoping the places that they interface doesn't blow up (hope someone tried that particular combination before).
- yellowapple 4y ago
molteanu 4y ago
Lisp
- nonrandomstring 4y ago
  I was a little surprised he took until the final paragraph to mention Racket. If he had played around with it then I think the article might have read differently. " Racket is a Lisp dialect that’s specifically designed for creating new languages (a technique sometimes referred to as language-oriented programming). I haven’t had time to play around much with Racket myself, but it looks like a very suitable tool for creating 'little languages'."
ssivark 4y ago
Here's something I don't understand: How are "little languages" different from a bunch of functionality wrapped into a library/module? Is it just that (with some convenient syntax sprinkled on top), or is there more to it? I would imagine that most of the value comes from being able to "refactor" thought patterns to match the best way to cleave the domain into composable concepts -- and it seems like we do this all the time (and in all programming languages?).
- honungsburk 4y ago
  I agree, they are the same! regex is an example of an eDSL that most general-purpose languages support. The king of eDSL:s is of course Haskell. I recommend looking up parser combinators for anyone who has ever struggled with an understanding way to complex regex expressions.
- bazoom42 4y ago
  An “internal DSL” is a library with a a design which makes it “feel” like a language. JQuery is the classic example. An “external DSL” has its own syntax, like regexes.
- francasso 4y ago
  With a DSL you need to learn syntax, semantics and either the API for interacting with your GP language or the tooling for external DSLs. With a library you just need to learn the new semantics. I much prefer this to a DSL. Plus a library has all the debug tools the language has. DSLs usually don't have any debug tools except for printing when you are lucky
- hackandthink 4y ago
  "How are "little languages" different from a bunch of functionality wrapped into a library/module?" This is called a shallow embedding in the Haskell world. Deep Embedding is more like writing a full blown interpreter. Then there is tagless final (Oleg Kiselyov) - it feels shallow but is more flexible as simple library functions and it is optimisable like deep embedded DSLs. http://www.cse.chalmers.se/~josefs/publications/TFP12.pdf http://www.cse.chalmers.se/~josefs/publications/TFP12.pdf https://wiki.haskell.org/Embedded_domain_specific_language https://wiki.haskell.org/Embedded_domain_specific_language
auxfil 4y ago
This is a “deepity”. We already do this. We constantly do this in programming: “The idea is that as you start to find patterns in your application, you can encode them in a little language—this language would then allow you to express these patterns in a more compact manner than would be possible by other means of abstraction. Not only could this buck the trend of ever-growing applications, it would actually allow the code base to shrink during the course of development!” Functions, frameworks, little languages. It’s all abstractions on top of abstractions. You are shifting the knowledge of the abstraction for the more fundamental knowledge underneath that does the actual work. You end up just sweeping the codebase growth under some other layer’s rug and blissfully forget about the woes of future maintainers. The code is still there, abstracted and exposed by the “little language”. Hiding this behind a cute moniker doesn’t seduce. This isn’t the future of programming. This is already programming.
- Spark_Ed 4y ago
  Maybe the author is trying to predict that there will be boom in DSLs like there was for JS frameworks? Funnily enough, I'm just wrapping up a DSL for our in-house web component engine that creates an abstract data layer all components share. The pattern was easy enough that I'll probably build more DSLs like it when an API isn't flexible enough
  - valenterry 4y ago
    The better the programming language, the less need for a custom language. You can then create the DSL inside of the host language. This requires flexibile syntax to some degree and a fairly advanced typesystem if it should be statically typed - hence not too many languages are a good choice here.
    - giraffe_lady 4y ago
      I don't think there's anything in that comment to indicate that they didn't do it that way. There's a robust discussion about this elsewhere, but people making embedded DSLs is basically routine. They just don't always call them that or have awareness that what they're doing fits that description. But almost all of those will be written in the "host language" of just whatever the containing system happens to be written in. Some languages are particularly suited for this but other than racket and ruby it seems mostly accidental/incidental to their design.
deafpolygon 4y ago
I think it's just trendy again. We've had this on-off on-again love affair with tiny languages and they simply don't stand the test of time.
btschaegg 4y ago
On that note: Am I the only one that's constantly surprised by the absence of proper sandboxing solutions when so many programming languages now provide (otherwise pretty useful) means of running code dynamically in a script-like fashion? In C#, I can pull in Roslyn, and compile a string on the fly as a C# script; but the way the .NET standard library is structured makes it pretty much unfeasible to prohibit outside interactions I don't want to allow (in my case, e.g: `DateTime.Now`, while allowing the Handling of `DateTime` values). It's possbile to embed the Typescript compiler into a website, but running code on the fly and some simple sandboxing is not feasible without a serious pile of hacks. I've recently read a forum thread about a library for compiling/running Elixir code as a script, but guess what: The runtime (apparently) makes sandboxing really hard. And so on and so on. I just wished that the LUA approach of "if I don't give you a hook, you cannot do that" were just the default. I've seen so many overcomplicated enterprise-y solutions that are basically just a plea for a well-designed, local and small scripting API…
- orangea 4y ago
  Ruby used to have the $SAFE feature for sandboxing, but it was removed because it was buggy, added a lot of complexity, and wasn't actually that useful. Linux has all the various isolation features that make Docker work, but people still recommend not running untrusted code in Docker containers because of the potential for oversights/"bugs" in Linux's API. I suspect that programming languages / VMs don't include these features because they are very hard to get right and add a disproportionate amount of complexity for their utility.
- kaba0 4y ago
  As far as I know most in-process sandboxing has been deprecated because it is in contrast to maintainability. E.g. Java decided against its Security Manager, because it is way too easy to leave the proper checks out of a new feature, leaving the whole thing vulnerable with a false sense of safety. Instead, process-level isolation is recommended.
- rng_civ 4y ago
  I think WASM is filling in that gap to some extent. From the spec: > Any interaction with the environment, such as I/O, access to resources, or operating system calls, can only be performed by invoking functions provided by the embedder and imported into a WebAssembly module And IIRC, the core instruction set is reasonably compact.
ArcMex 4y ago
My concern was how will putting these little languages together work and it turns out that the author also had these questions. Where do we draw the line on "enough DSLs" for example? And what happens to the gains from using several DSLs in tandem as opposed to a high-level language with libraries that accomplish the same thing?
helsinkiandrew 4y ago
> I’ve become convinced that “little languages”—small languages designed to solve very specific problems—are the future of programming, Yes, but over time very specific problems become bigger/different problems which the little language isn't ideal for, the original developers move on leaving someone new to figure out the problem and language which is probably poorly documented and very brittle. Application developers probably aren't suited to writing and maintaining language code. The only caveat is an external system provided with its own language - like RDB/SQL - which is proven and well maintained - but its hard to call SQL a little language.
bayesian_horse 4y ago
I can hardly get my coworkers to understand one language well.
diffxx 4y ago
DSLs are not a replacement for but a complement to any existing language, general purpose or specialized. I have come to think of DSLs as programs for writing programs (similar to but not identical to macros). With a DSL, you can specify the grammar of a specific problem/program. Once that has been done, it is often quite straightforward to implement the grammar in any number of target languages. As an application developer, this may not be a huge advantage (though DSLs can also shine in any client/server interactions), but if you are a library author this can be very compelling because your library may be easily portable to most commonly used language runtimes in a generally rote kind of way. The port might not be optimal, but it should be correct, provided the high level logic of the DSL is. Performance optimizations can be done where needed. What is great about this approach as an individual is that it requires you to tighten your ideas. When you have to implement all of the functionality in a DSL, you really start thinking about what you truly need. A big language nudges you towards using all of its features while a small language challenges you to consider what is truly essential. Of course DSLs always run the risk of being write only and/or only comprehensible by the original author. Like any powerful tool, DSLs should be used judiciously and responsibly. Often that isn't the case, in part because I don't think the tooling for writing DSLs is generally very good. But I am betting that new tools that make DSL writing easy will have a profound effect on software development.
samsquire 4y ago
Join any company and organisation and look at their build and deployment tooling. Unless they are using Kubernetes and even then, you shall find a very complicated bunch of languages: - shell scripts - Dockerfiles - Kubernetes YAML - Makefiles - Bazel - Ansible - python scripts - Jenkins XML - Groovy scripts - Ruby scripts - CloudFormation - Terraform - Fabric or other deployment deployment script It's very hard to fit together and understand from a high level. The last thing they were working on at my previous company was a YAML format to define a server, to go through the organisational structure of the company to manage computer systems. Some people mentioned LISP in this comment thread. For me LISP is an intermediate language, I would never want to build a large system in LISP. It's not how I think about computation.
- throwaway17_17 4y ago
  Can you expand on you mean when you say that Lisp is not ‘how [you] think about computation’. I have never seen that phrase used regarding Lisp, usually it’s mentioned in regards to logic or other much less mainstream paradigm than the imperative/semi-functional paradigm in Lisp.
  - samsquire 4y ago
    I see LISP as being useful for codegen and intermediate representation and AST representation but I wouldn't want to program with it directly without a tool to create a structure that is understandable to me. All the parenthesis! I wouldn't want to maintain or work on a large Clojure codebase written by other people. I've done that three times. For reference, I think Python is easy to write and read and understand. I wrote a simple toy multithreaded interpreter and I've written part of a compiler that does codegen to target the imaginary interpreter. It's basic but my AST is a tree that could be represented in LISP. I use a handwritten recursive descent pratt parser. The language looks similar to Javascript. I know it can fixate your thinking if you think of it too much in this manner, but I think of modern computers as turing machines. They loop or iterate through over memory addresses which are data or instructions and execute them. That said, my perspective is not traditional. I design and try implementing programming languages. I am interested in the structure of problems and code, asynchrony, coroutines, parallelism and multithreading more than anything else. Even more than type systems. I think the expression problem is a huge problem that doesn't have good solutions for managing complexity. I find other people's LISP code to be difficult to read whereas I can understand a Python, Java algorithm. What am I trying to say? The structure of the program in the developer and compiler's head is different from the instructions actually executed by the computer. LISP is nearer to the instructions executed by the computer than what exists in my mind. In my mind exists relationships, ideas more complicated and not structured in post order traversal. A post order traversal of LISP is the codegen.
galaxyLogic 4y ago
The problem with multiple small languages is that it is not trivial to make the call each other
adamddev1 4y ago
Is this just like the push for microservices, but for languages?
hkt 4y ago
Empirically speaking, it seems more like rich languages are the past, present and future of programming. Rich as in, has a strong standard library. Python, go, etc. The only "little" languages I can think of that I'd reasonably ask people to use at work are lua, make(~), awk, and (ba)sh.
Mikhail_Edoshin 4y ago
Unlikely. On one hand notations should be a commodity and they indeed should be unique to the task. There is no point and no way to try making a unified notation for music and chess. On the other hand there is no point to make multiple notations for the same thing. And programming is indeed the same thing from lowest to highest levels, composable like a Russian doll, or we won't be able to build large systems. So the future of programming is a single notation that actually reflects what programming is. We do not have it yet, this is why we have so many "programming languages".
brundolf 4y ago
This is what I always heard Lisp was best at. Instead of making totally new languages (with parsers, tooling, etc) you'd create little DSLs within your own code in the form of macros: come up with a "little language" for describing one part of your app, write a macro for it, and then it integrates smoothly with everything around it Whether or not you agree this philosophy is a good one, and whether or not you like Lisp specifically, I think we can all agree that macros (in whichever language) are a much better way to do it than creating a bunch of tiny languages from scratch. I was surprised not to see the word "macro" appear in the article at all
- brabel 4y ago
  Macros are very hard to write tooling for, and hence difficult to author and use. I think that, perhaps something like Polyglot GraalVM[1], which was designed to host many languages and let them all seamlessly talk with each other, while automatically making existing tooling "just work" with any new languages created with the framework [2] would be a better way. [1] https://www.graalvm.org/22.0/reference-manual/polyglot-programming/ https://www.graalvm.org/22.0/reference-manual/polyglot-progr... [2] https://www.graalvm.org/22.3/graalvm-as-a-platform/implement-language/ https://www.graalvm.org/22.3/graalvm-as-a-platform/implement...
- dmux 4y ago
  Lisp macros are nice, but as Gumby said above [0] a common approach is to use Lisp's quoting abilities to construct a data structure that represents the problem at hand (in its own terms) and then create functions to parse / manipulate that data structure. A classic example comes from Peter Norvig's "Principles of Artificial Intelligence" wherein he defines a subset of English grammar as a data structure [1]: '((sentence -> (noun-phrase verb-phrase)) (noun-phrase -> (Article Noun)) (verb-phrase -> (Verb noun-phrase)) (Article -> the a) (Noun -> man ball woman table) (Verb -> hit took saw liked))) He then goes on to define a function "generate" that uses the above to create simplistic English sentences. Additional rules can be added by a non-programmar so long as they understand how their domain logic has been mapped to Lisp. [0] https://news.ycombinator.com/item?id=33705558 https://news.ycombinator.com/item?id=33705558 [1] https://github.com/norvig/paip-lisp/blob/main/docs/chapter2.md#21-a-grammar-for-a-subset-of-english https://github.com/norvig/paip-lisp/blob/main/docs/chapter2....
015a 4y ago
Is the engineering footprint of an organization really better if everything is implemented in twenty different languages, versus just three or four? Everything else aside, quality of the language, scope, etc; just the number. You have to expect everyone to know each language; know the ins, outs, idioms, gotchas, etc. You have to be able to hire for the languages. You need the language runtimes in your environment, everywhere, docker, local dev machines. You have to keep up to date in X times more changelogs, version upgrades, CVEs. The article pulls Shell as an early example. Shell did not become the powerhouse it is because its "great" (though some would argue it is, I'm not here to debate that); or because its small; or because its general purpose; or because its single-purpose. It became a powerhouse because its Old and Omnipresent. See, the problem with inventing New Things is that they are, by definition, not Old, nor Omnipresent. New Things have to start somewhere, but you're starting in last place. > Regular expressions and SQL won’t let you express anything but text search and database operations, respectively. Oh mylanta. Did you know that after the addition of back-expressions, Regular Expressions became turing complete? They are, functionally, a real programming language, just like C; well, except, far more annoying to write. And naturally, SQL "won't let you express anything but database operations", which is to say nothing about "SELECT 1+1"... let alone the little corner of the language called "Stored Procedures".
- gpderetta 4y ago
  That's the inner platform syndrome, and yes, it is bad.
- 0x445442 4y ago
  There’s a Groovy DSL that will allow you to perform operations on collections using SQL syntax. I’d argue it’s an improvement over the traditional procedural or functional approach’s. https://groovy-lang.org/using-ginq.html https://groovy-lang.org/using-ginq.html
  - 015a 4y ago
    Well, maybe this will start a flame war: I will absolutely die on the hill that SQL is shit. But, like shell, is old (48 years old!) and omnipresent; so it sticks around. Like, the thing I find comical about the original article is: they reach for shell, regex, and SQL as Prime Example of "mini languages done right". Be more domain specific, look at this its already happening. All of those examples are kinda shit languages! They're popular because of their omnipresence. Maybe regex is "fine", though the line "try to solve a problem with a regex and now you have two problems" is well known for a reason. But my broader opinion is: if you're building an application, service, tool, whatever in LANGUAGE_X; having to "dip out" of that language into an entirely different language should be viewed as, fundamentally, a Negative Thing. There may be reasons why you should; the good may outweigh the bad; but there is Bad there. There will always be an interpretation layer; extra tooling; that's More Things that can go wrong, have to be configured, statically analyzed, tested, its a failure point. SQL injection is a thing. Why? Because, for a time, people had the thought "hey, its a string, lets just template the string"; but that's no good, so now we have A Layer between the Java and the SQL to keep us safe. We can only support So Many Layers; we need to make things simpler, not more complex, there needs to be Fewer Things. Its time is gone, but I'll always view Heroku Buildpacks as a paragon of system design. Consider: You could write a nodejs application in javascript, write a package.json in json (also javascript), and end-to-end get that thing on a URL on the internet, all in one language. Fantastic! Today, a typical app will have the app (say, Golang), a go.mod (different syntax than go itself), dockerfiles (language 3), kubernetes yamls (language 4), maybe helm or cdk8s (language 5), shell scripts, makefiles, maybe you're also writing sql... this isn't better and it doesn't have to be like this. But right now, it is, and to some degree I'm happy for it because its 80% of the reason why I get paid six figures.
chantepierre 4y ago
I am currently building a tiny language designed only for data manipulation in a long-term personal project : data made of number/strings/booleans in maps or lists goes in, can be re-shaped, and goes out. It's written in Typescript as part of a web app. It just has map/list access & creation, string manipulation & concatenation, basic arithmetic/comparison, lambdas as first-class values, function calls. No way to do I/O, just data in-data out. I need some non-programmers friends to collaborate on the tiny data transforms and it is quite easy to write a 1:1 text <> visual editor for this language thanks to its limited features, allowing to go from visual to text or text to visual. The exercise in itself is interesting and worth it. I think I wouldn't do that in a commercial context though.
macrolime 4y ago
If a lot of little languages /DSLs were going to be practical as another layer of abstraction up from APIs, I think it would have to be implemented in something like Unison. Unison is made so that that you can have different syntaxes for the language or parts or the language as the language is not stored as text,but more or less the abstract syntax tree. Then again so is Lisp more or less. I suspect that instead what will happen as the next big thing in terms of new abstractions in programming is that AI code generators will keep getting better and get better tooling and that's going to be the tool of choice for high level needs. While these AI tools may not be that accurate today there is vast potential in improving the models and tooling. In the IDE it could be more like you describe what you want, it generates some code and tests and then run the tests and maybe a visual time traveling debugger so you can visualize what it's doing to see if it's doing the right thing.
jstx1 4y ago
If the point is that DSLs will be around, sure. They already are. If the point is that we'll replace complex full-featured languages with a stack of DSLs - no way.
yellowapple 4y ago
I think this works best if the little languages all share as much syntax and semantics as possible. A good example of this is OpenBSD's assortment of configuration file syntaxen for OpenSMTPd, pf, httpd/relayd, etc.; each of those "little languages" differ considerably in their problem domains, but they all seem to share a vaguely-Tcl-ish syntax and have largely converged in semantics and typical structure. Another important consideration is that these languages are typically best when declarative as possible; if you can avoid Turing-completeness and stick entirely to something representing static data, then that's the ideal.
- frou_dh 4y ago
  GNU Guile was supposed to be such an ubiquitous language but AFAIK it didn't really take off.
caporaltito 4y ago
Remember, when a guy writes a post with an absolute title, the answer is always "No".
- aargh_aargh 4y ago
  Including yours.
kaba0 4y ago
While I also believe several commenter’s opinion that libraries, frameworks can easily become DSLs themselves (e.g. isn’t Java streams basically a DSL for stream processes inside another language?), one really can’t talk about polyglottism without mentioning the ingenuity of GraalVM. You basically write a dumb, easy parser and AST interpreter for your language, and it will magically turn into a JIT compiled dynamic language with state-of-the-art GCs and better performance than what you could likely come up with. And the best thing is that it really unifies the computing model where you can pass a python object to some js lib, effectively giving you every library ever written for any (implemented) language, which is the real productivity booster.
- adql 4y ago
  Yes but library that turns a part of syntax of language into DSL to solve certain problem have advantage of developers already knowing that language, developers needing to learn only one language, and ability to use with that DSL anything else language libraries offer. The problem is really "does the language app uses works for target audience that will be doing the DSL". If it does (Ruby makes pretty decent one), job done, if it doesn't, we end up in the mess of making toy languages (usually because developers want to do something fun and writing new small language can be pretty fun) or plugging something like Lua into it.
iovrthoughtthis 4y ago
whats old is new dsl's are now little languages
lysecret 4y ago
Interesting article, I immediately had to think about terraform. The problem I'm having with specialized languages (even with SQL) is that it always creates additional interfaces and almost always creates ugly string formatting if you want to integrate (SQL) or ugly duplication (Terraform) if you keep it separate. I like the idea of abstraction but in my mind it is very easy to have the power of a "little language" inside an all purpose language by using a package. E.g. SQLAlchemy or Pulumi as the alternative to the little languages of SQL and TF.
LordHeini 4y ago
I disagree and would say DSLs should go away if possible. DSLs like SQL are the norm and you can see the problem of them in basically every project. You either use ORMs or you end up hand rolling SQL rows into Structs or Classes. The whole mapping usually looks like crap and contains a bunch of implicit corner cases, which eventually end up being a footgun for someone. Usually the SQL sever runs somewhere else, the ports are wrong, the language version is wrong, or a migration failed and a function is missing yada yada.... The same is usually true for Regexp. There are a billion dialects and every single one of them is basically unreadable, incomplete or just weird. The same is true for microservices with tons of config files for dev, staging, testing and production... Everything has its own version, can be down or mutate some random state somewhere while depending one other servcies. It always breaks at the seams. Increasing the amount of DSLs increases the amount of seams and thus makes software worse.
- zelphirkalt 4y ago
  But your ORM example could be considered a DSL in itself, so it doesn't really work that well. ORMs can be implemented in various ways too, as a syntactically looking different DSL or as something like objects and method calls or whatever the language already provides. Regexes can be abstracted from exactly by making a more readable DSL. Config files and the stuff accepted therein are small languages as well. What are the alternatives to making little languages?
- Octoth0rpe 4y ago
  > DSLs like SQL are the norm and you can see the problem of them in basically every project. I think your points re: sql are true of straightforward crud apps, but not true at all of an analytics app. In those cases, the sql is often _very_ complex, and while the results of a query may be eventually mapped into a struct or something, the query generation is rarely a simple mapping of properties in an object to select columns. > There are a billion dialects and every single one of them is basically unreadable, incomplete or just weird. Sure, but in the vast majority of cases, one only has to deal with at most 3 dialects, and there's a good chance you won't be hitting the corner cases that make each dialect significantly different.
cies 4y ago
Some more little language not listed by the article: Elm (browser apps w/o runtime errors). The creator (BDFL?) of Elm, Evan, specifically avoids making it general purpose.
hnbad 4y ago
Calling SQL a "little language" must surely be a joke. Even if you ignore the differences between its many dialects, SQL is only "little" in that it is domain-specific rather than general purpose. In fact the entire article seems to boil down to "DSLs are the future", which I'm sure I've seen articles about back when Ruby on Rails was dominating web technologies, Cucumber (and its various ports) created "BDD" testing fad and DevOps started gaining traction on top of various "Ruby DSLs" used as configuration formats. I don't think DSLs are going to go away any time soon. But there is a trade-off between domain-specific "little languages" and general purpose programming languages (or "DSLs" that are actually subsets of the latter). It can be fun to have to work with a little language, it's not so fun to work with dozens of them, each with different rules you have to memorize, instead of just being able to use the same language for all (and in truth, this was the source of the Ruby DSL craze because developers were already using Ruby on Rails).
BiteCode_dev 4y ago
> SQL is a little language for describing database operations Yeah, SQL is not a little language anymore. It started as one, but because a lot has been added to it and SQL flavors for Oracle or Postgres are anything but tiny. Windowing, nesting, json handling... I think the author is kinda proving with this that a successful little language does not stay little, and hence little languages are not the future. And don't get me started on DSL in general. Just lookup my username and "DSL" on hackernews for endless rambling.
- raydiatian 4y ago
  DSLs are great for job security in small organizations. They might as well be regarded as criminal.
- williamcotton 4y ago
  The reason why the author is using the term little language is because the term DSL doesn’t mean custom language any longer. Yes, SQL is bigger than it used to be but it is still a domain specific language, which is what the article is actually about.
- redanddead 4y ago
  Maybe the term modular would be more appropriate than little? Every solution that has gained widespread dominance, take the shipping container, has been thanks to its flexibility, and modularity.
ergonaught 4y ago
I feel like lisp refutes this article, in that most of the “proposed solution” already exists and already failed in actual practice.
hoseja 4y ago
I don't think programming has yet stagnated enough for this to hold. For domain specific languages you need specified domains.
mlatu 4y ago
hm, im a little unhappy about the author comparing Knuth's solution to a handfull of shell utilities. for one, the author says knuths program written in WEB was 10 pages long, discounting the fact that these 10 pages are HEAVILY annotated. my other point is: tr has 1917 LOC, sort has 4856, uniq has 663 and sed is in its own package at around 10 MB all including comments and docs for sed it's fine and good that you can use composition with shell utilities, but come on, write that example program in C99 and you'll be a not very happy coder at all. in general i find the comparison rather rude. Knuth was supposed to show ?his? programming language WEB and as a "critique" McIlroy farts out a shellscript like "lmao first". indeed, you do not often need to count word frequencies. but what was this article supposed to be really about? software engineering 101 aka dont-reinvent-the-wheel/DRY? or perhaps literate programming?
- cb321 4y ago
  Related: https://buttondown.email/hillelwayne/archive/donald-knuth-was-framed/ https://buttondown.email/hillelwayne/archive/donald-knuth-wa... As you say, Knuth was asked to demonstrate his literate programming... In some ways this is a direct request for the non-pithy, articulated, first principles answer. I would more say Knuth was set up than that he was framed, but tomato-tomato. :)
nigamanth 4y ago
Wouldn't that require programmers to use 50 languages in a project though? Assuming that this is the future of programming, the number of languages that a programmer would have to learn to create a simple project would be ridiculous. Why use many languages when you can do the same process in one? Sure, even if the "big" language isn't specifically made for a certain job, doing that job in a little language requires time for the programmer to learn the little language. The syntax may be different, so to convert from the big languages to little languages it would take time. This is why this hasn't happened yet; people are lazy to learn new languages so they just learn the "big" languages that can complete all the tasks that they require.
bvrmn 4y ago
It seems DSL proponents grow from environment without rich standard library. And classic Knuth argument, of course. It's so hard to count words. General purpose language: >>> from collections import Counter >>> import re >>> Counter(re.findall('\w+', 'boo foo boo +dfd zii')).most_common(1) [('boo', 2)]
beanjuiceII 4y ago
just what i want, a zillion little languages to remember their syntax and semantic quirks.. its bad enough the way it is right now where everyone tells me "the right tool for the right job" then i get there and realize they are all the same tool
gumby 4y ago
Little languages are the past and yes, the future. We just don't recognise them. It was common in the 60s and 70s to have the hardware manufacturer ship all the OS and languages with their hardware. The languages were often designed for specific problem domains. The idea of general purpose languages (FORTRAN, PL/1, etc) was uncommon. You can see this in K&R (the original edition anyway) where they justify the idea of a general purpose language, even though C itself derived from prior general languages (B & BCPL) and they had gotten the idea from their experience on Multics (written in PL/1, a radical idea at the time). So a 20 year old idea was still barely diffused into the computing Zeitgeist. Most Lisp development (since the early 70s at least) is writing a domain-specific representation (data structures and functions) and then writing your actual problem in it. I used both Lisp and Smalltalk this way at PARC in the early 80s. More rigid languages (the more modern algolish languages like python, c++, rust, C, js etc -- almost every one of them) doesn't have these kind of affordances but instead do the same via APIs. Every API is itself a "little language" What are called little languages in the Bently sense is simply a direct interface for domain experts. And after all what was a language like, say, Macsyma but a (large) "little language"?
- Verdex 4y ago
  > Every API is itself a "little language" I came to this conclusion early in my career. It went something like this: A - "To do this, just create this object, fill in these properties, and call these methods." B - "Okay, I did that, but it crashed." A - "Yeah, it's because you set the properties in the wrong order. This property relies on this other property under the hood. Set them in this order." B - "Still crashes." A - "Yeah, you called the methods in the wrong order. This method relies on that method. Call them in this order and it works." My conclusion was that the lisp philosophy of building a lot of little sub language was equivalent to what people were doing with OO in C#/Java. Either way you have to learn the "right" way to put things together which is dictated by unseen forces behind the scene. Of course, I also concluded that most people work differently than I do. For most people, if the code "looks right" (ie recognizable syntax) then they're able to tell themselves a story that it's familiar and their intuition is able to pick up the slack for finding the right enough way to use most arbitrary APIs (just as long as they don't exceed some level of incomprehensibility). On the other hand, I have to understand the underlying logic or I use the API the wrong way pretty much every time. So for most people lots of APIs is actually a much better cognitive way for them to work whereas for me API soup and lisp macros are the same conundrum.
raydiatian 4y ago
> For example, SQL is a little language for describing database operations. Regular expressions is a little language for text matching. Dhall is a little language for configuration management, and so on. > There are a few other names for these languages: Domain-specific languages (DSL:s), problem-oriented languages, etc. However, I like the term “little languages”, partially because the term “DSL” has become overloaded to mean anything from a library with a fluent interface to a full-blown query language like SQL, but also because “little languages” emphasizes their diminutive nature. Ahem, so is SQL diminuitive or not? Because, SQL is NOT diminuitive. SQL is Turing complete.
jkingsbery 4y ago
Among the problems this article ignores is the relatively large fixed cost needed to go from a framework/internal-DSL to a separate "language." Unless you are in a situation where lots of people have that same problem, then it actually creates more complexity than it solves to have an separate language to solve the sub problem.
CyberDildonics 4y ago
A huge problem with this approach is that very few general purpose languages reach the point where they have big standard libraries, great external libraries and tools on top of them. Every time someone invents a new language it wipes away the decades of progress of the general purpose language that could be used instead, so it needs to have a massive advantage or be completely necessary.
kwhitefoot 4y ago
I've always regarded the principal task when implementing a program to be creating the necessary data structures and a set of functions that represent a vocabulary for manipulating them. So every program I write contains a small domain specific language. Isn't this what everyone does? 'Official' small languages only make sense when the user base is large, this is why SQL exists but a mainstream language for manipulating Maxwell's equations does not.
oscarb92 4y ago
I am very familiar with DSLs but had not heard them called "little languages". I cannot say I find it fitting as it makes me think more of languages with reduced syntax or semantics.
- giraffe_lady 4y ago
  I haven't done any research on this but I suspect that usage predates "DSL," or at least was popular before the term DSL was widely known and used for it. I mostly see "little language" used in older sources, mid-90s or earlier. It seems to be making a comeback though! I prefer it too. It's not really more descriptive than DSL but it's not much less, and is less jargony and just cuter.
  - radiator 4y ago
    This seems corect. The AWK Programming Language, 1988, Chapter 6: Little Languages https://archive.org/details/pdfy-MgN0H1joIoDVoIC7/page/n11/mode/2up https://archive.org/details/pdfy-MgN0H1joIoDVoIC7/page/n11/m...
sunarch 4y ago
I've seen other replies highlighting Lisp in general and Racket in particular, but when it comes to a "little language" I think it's valuable to have a link to Paul Graham's article on The Roots of Lisp: http://www.paulgraham.com/rootsoflisp.html http://www.paulgraham.com/rootsoflisp.html It underlines all the points already made here about Lisp, and also includes a CL version of the original code. 67 lines including comments and empty lines. Endless possibilities.
garethrowlands 4y ago
Rule of least power also relevant here, https://en.m.wikipedia.org/wiki/Rule_of_least_power https://en.m.wikipedia.org/wiki/Rule_of_least_power The web is designed using a variety of little languages. It could just be JavaScript with an API but it's not. And it's probably best as it is.
beefok 4y ago
I've always enjoyed reading this book that espouses this same idea: Constructing Language Processors for Little Languages 1st Edition by Randy M. Kaplan [1]. It's no longer being produced, but used it's $6 or less. It's dated, from 1994, but it is a fun enjoyable discussion on the benefit of tiny specific languages. It also has a nice tutorial on the use of lex and yacc. [1] https://www.amazon.com/gp/product/0471597546 https://www.amazon.com/gp/product/0471597546
OhNoNotAgain_99 4y ago
anikom15 4y ago
DSLs and general purpose languages are not comparable. DSLs are generally designed to handle data, while general purpose languages can handle everything. Where DSLs fall apart is when you either need to (a) work with I/O in a different format than the DSL handles data in (including working with hardware) or (b) the scope of data changes into something that now incorporates information outside the DSL’s domain. For (a), a simple example is an application that reads information from a DAQ, stores it in a database in a compressed format, and later sends the data to a printer. We have a DSL that can easily implement the database, but we need to translate the data from the DAQ into a compatible input, so that requires a different language. Furthermore, the printer requires postscript, so we need some other language or tool for that. Then we need to figure out how to glue it all together. In some cases, it becomes tempting to try to ‘hack’ the DSL into trying to do things it was never designed for. Or we could just use a general purpose language and libraries.

179 points by JohnyTex 4y ago

deleted 4y ago

[deleted]

I think the explanation in https://www.oilshell.org/blog/2022/02/diagrams.html https://www.oilshell.org/blog/2022/02/diagrams.html is also related to this.

nottorp 4y ago

Weren't they called DSLs a while ago? The OP even uses "Domain Specific Language" in the article.

SleepyMyroslav 4y ago

Author and comments are sticking to text based languages for some reason. I think visual ones have proven to be more accessible to non-programmers. There are downsides ofc ... for programmers =) As a programmer myself I do not think it will be great to push visual languages too far but business is business.

vfinn 4y ago

AI will kill the need for all these abstractions. There will be as many or as little abstractions you want. It doesn't matter. New languages of all shapes and forms will be generated eventually on the fly. You will just have to know what you want.

sargstuff 4y ago

Lisp macros much more unified approach to DSL/little language creation/usage.

martin1975 4y ago

a.k.a. Domain Specific Languages Hello ANTLR.. https://martinfowler.com/books/dsl.html https://martinfowler.com/books/dsl.html

jph 4y ago

The Rust `nom` parser crate works very well IMHO for creating little languages, DSLs, custom markup, and your own kinds of annotations, all with the Rust compiler guarantees and good speed.

tov_objorkin 4y ago
Oh, please, making reasonable good DSL might take a month or even years. Also wrestling with parser and borrow checker not an easy task for average user.
- intelVISA 4y ago
  I'm no Rust zealot but the borrow checker is arguably one of the core benefits for the average user. Wrestling with it means they are not yet ready for systems programming and need to understand move semantics more deeply.
  - tov_objorkin 4y ago
    Borrow checker is fine. But from the library writer perspective its pain and take enormous amount of time to make it sound. One does not simple checkout "nom" and test thing in few minutes.

smitty1e 4y ago

My first goal with a challenge at the office is "make the problem smaller". Now I have a folder of little one-offs and REPL scraps. A triumph of tactics over strategy that defies passing on to anyone who didn't author it. Looking at the JVM and JSON, I wonder to what degree that languages contribute some piece of goodness or idea toward some final Tool To Rule Them All...

neilv 4y ago

In addition to the Racket article that the article links, Racket's language support has become more powerful and smoother since then. A more recent treatment is Matthew Butterick's book: https://beautifulracket.com/ https://beautifulracket.com/ It doesn't have to be a big standalone DSL with a separate compiler or preprocessor. It can also be an embedded little language, like when you sprinkle HTML templates throughout your normal general-purpose language, and as only a syntax extension: https://docs.racket-lang.org/html-template/ https://docs.racket-lang.org/html-template/ (Aside: I'm seeing tasteful Racket and Scheme influences in Rust, even though they're very-very different languages. I'm hoping to contribute a little more influences.)

lmm 4y ago

This article claims the problems it's trying to solve are: Hard to onboard new hires, code breaks because of lack of understanding of dependencies, and code changes become harder to manage. In my experience SQL, regexes, unix shell, and listening to Alan Kay, far from solving those problems, are the very things that most exacerbate them. General-purpose languages that are expressive enough to let one write business logic in the language of the domain, but without breaking the rules of the language or requiring new tooling - "internal" rather than "external" DSLs - are a far better way forward.

sargstuff 4y ago
> " General-purpose languages that are expressive enough to let one write business logic in the language of the domain, but without breaking the rules of the language or requiring new tooling .." gawk compiled to webassembly would seem to fit the bill -- just shifts gawk from "external" dsl to "internal" dsl. orthogonality allows for usage of all modern interface trappings without any retooling and/or breaking rules of gawk language. Makes gawk a module in group of customized modules to form a general-purpose program out of dsl/little languages.
- lmm 4y ago
  Transpiling to your main language doesn't make it an internal DSL. Reimplementing what gawk does as a library you can use within your language (and not just by passing opaque strings to it, but by actually expressing awk-style commands in a proper datastructure) is the kind of thing I'm advocating for.
  - sargstuff 4y ago
    Transpiling the language changes the operational domain. Reimplimenting/supporting the library in language means, have to do that for each OS operational domain that the gawk interpreter runs in (aka each relevant OS & OS version for mac, unix, windows, non-pc OS's, plus mobile versions). Modifying language also means that anything that relies on lanuage will have to be re-evaluated to assess potential language change issues/implications. Transpiling to change the operational domain means can just focus on task at hand, as long as not tied to specifics outside the operational domain of webassembly. aka file paths Avoids the need to assess potential language change issues/implications. Changing the operation domain in this case means can leverage the operational environments orthononality to 'express awk-style commands in a propper data structures' that awk doesn't provide without modifiying the language. (or writing the feature in awk) Changing the operation environment also means no required new retooling. Changing the language may force/require retooling. Case study context example: Long term 10-40 year studies where same code written at start of study needs to be runnable / usable / understandable at the end of the study without being rewritten. Note: Hardware and/or OS used at start may be entirely different than that used at end of study. Fairly simple to get 10,000+ awk scripts running under IBM/DEC/Mac/Windows/Unix/BeOS to run under any of the other OS domains using webassembly by transpiling awk. Fairly trivial to leverage browser to permit a user customizable GUI usable under any of the aformentioned OS domains without having to modify/recertify the awk scripts. Finding / recompiling / supporting awk for each of those OS domains (OS & verions variants, GUI libs) bit more time consuming (short term & long term).

laurencerowe 4y ago

I think it's more likely the future will be ever more 'languages' which conform to a schema but are simply expressed as JSON/YAML. Being able to trivially deserialize with a simple JSON.parse or equivalent provides a huge head start. DSLs defined in a type/schema system atop JSON/YAML end up being far easier to write tools around than DSLs which require a custom parser (e.g. Dockerfile.) That said, there are definitely a subset of languages like JSONPath that would not work written out as an AST.

zetsurin 4y ago

Having dabbled in this a bit myself (using small dsl's to solve specific problems), some problems that immediately occur to me: - more difficulty on boarding - more difficulty adding new features - more challenges with best practices/linting/code reviews - different runtime behaviour between different languages This all adds up to more complexity. New languages are often really neat, and enjoyable for their own sake, but I'm not all that interested in maintaining a large swath of different languages for different tools.

chewxy 4y ago

I don't think it's so much "little" languages (commonly DSL) that matter. It's more the jumps in expressivity. You don't use a full on Turing-complete language when you need to match strings written in a regular language. Instead, we write the language we want as a regexp, and then use a regexp engine to match it. I agree with much of the problems listed in the article. The author even manages to stumble onto some of the solutions (e.g. Dhall being a total language). "Expressiveness is co-decidability" is the main theme of these things. The crux of the issue is in our everyday programming tasks, we have many levels of decidability, ranging from RE all the way to things that require full Turing completeness. The majority of work however, lies in the middle. There are so many things that can be done with pushdown automatons, or with deterministic automatons. Most codebases don't actually use those though. An issue is that there is a dearth of "mini" languages that support these things. Another issue is that somehow we are enamoured with the idea that our languages must be able to express everything under the sun (up to TC/Recursively Enumerable). This seems to be more of an industry attitude than anything - there is this chase for the most powerful language (a lisp, clearly... everything else is a blub). I've recently experimented with embedding an APL into my usual programming language, and it was a very interesting experience. It feels like having the power to do regular expression stuff, but with arrays. I want to do the same for the other levels of expressiveness.

tempodox 4y ago
What's a “total language”?
- Tainnor 4y ago
  To slightly oversimplify: A language in which every function provably terminates.
- TuringTest 4y ago
  It's a programming language where every written program terminates. As a corollary, such languages can't be Turing complete, since universal Turing machines can represent programs which don't terminate. [1] https://en.wikipedia.org/wiki/Total_functional_programming https://en.wikipedia.org/wiki/Total_functional_programming

AlotOfReading 4y ago

People have been making this argument since the 80s and possibly even earlier. My experience is often the opposite. Little languages are usually far, far harder than (mis-)using "big" languages for small tasks. The problem is that your DSL has to be understood by other people, including future you. Programming tasks are vast, combinatorially explosive state spaces full of weird potential interactions between features. Once you get above the complexity and universal familiarity of say, arithmetic, it's difficult for others to understand what's going on just by looking at 1-2 live examples. You have to heavily invest in proper docs and tooling (if your language doesn't provide it for free). By the time you've completed that your "little language" usually isn't such a little effort anymore. If you don't, you've just made the next CMake. Congrats you monster.

CipherThrowaway 4y ago
I think EDSLs are the happy medium here.
lovecg 4y ago
And even if you do invest in proper documentation and support, you still have to overcome the hurdle that people just _don’t want to spend time learning your one-off language_ - there’s nontrivial opportunity cost in learning something that won’t be useful anywhere else. So people will just do the bare minimum which will lead to misunderstanding and bugs.
- nottorp 4y ago
  > people just _don’t want to spend time learning your one-off language_ Which people though? If you make a DSL that non programmers in your organization use, I'm sure they will appreciate not having to learn the intricacies of Rust or whatever's in fashion this week.
  - Yizahi 4y ago
    We don't use DSL per se, but a custom tool for writing QA tests, which looks like a kinda Visio block diagram software, only each block is a function or other logical entity. Anyway, after a few years struggling with it, for many different reasons, we are slowly and painfully migrating to writing tests in Python, and every single QA supports it. Custom languages, with limited support, limited community, limited extendability etc. are just like that - limited. And as soon as you hit a wall with them, transition will cost more (in both time and money) than saved in the first place by using "easier" tooling for non-programmers.

bloppe 4y ago

The size of the language is a red herring. You really just want programs that are well structured, which can be greatly helped by choosing a "perfect language" for each task, but often helped just as well by choosing (or creating) a great library to express the business logic.

martindbp 4y ago
I think "little" refers to the domain rather than other metrics of language size

lolinder 4y ago

A "little language" is just an abstraction over a small part of the domain. Equivalent abstractions can be and are written as libraries. In a Turing-complete host language, the primary difference is that these libraries don't get the privilege of inventing new syntax, but that's almost always a good thing. We already have major headaches switching between JS, SQL, and {insert backend language here}. Introducing tens of little languages into a codebase may marginally increase readability of each chunk of code in isolation, but the amount of context-switching and required background knowledge it introduces would more than make up the difference. In a abstraction strategy that's based around libraries, every library agrees on the same basic syntax and semantics. You interact with an embedded DSL via a well-defined interface that all libraries respect. The syntax may not be perfectly ideal to any one part of the project, but it's consistent throughout every part of the project. That has real value. I think it's also a red herring to argue about lines of code: stringing together a bunch of little languages is not likely to lead to fewer lines of code than pulling in an equivalent number of third-party libraries, and it will almost certainly increase the total amount of code in your distribution, because each little language must not only implement the functionality desired, it also needs its own parser and interpreter. Take McIlroy's shell script: if you add up the C code to implement each of those little languages, you're at about 10k lines of code to make that bit of code golf possible. I'm a huge fan of DSLs, and I like the analogy of modern programming as pyramid building. I just don't think independent, chained-together DSLs are the answer. I'd rather have a language like Kotlin that is designed around embedded DSLs that all respect the same rules and use the same runtime.

Kamq 4y ago
> Take McIlroy's shell script: if you add up the C code to implement each of those little languages, you're at about 10k lines of code to make that bit of code golf possible. This is a fair point, but part of this has to be how battle tested the language in question is, right? Bringing in a single language that's been run through its paces (bourne shell in this case) for text processing, seems like a much lower risk than bringing in a dozen different languages and hoping the places that they interface doesn't blow up (hope someone tried that particular combination before).
yellowapple 4y ago

molteanu 4y ago

Lisp

nonrandomstring 4y ago
I was a little surprised he took until the final paragraph to mention Racket. If he had played around with it then I think the article might have read differently. " Racket is a Lisp dialect that’s specifically designed for creating new languages (a technique sometimes referred to as language-oriented programming). I haven’t had time to play around much with Racket myself, but it looks like a very suitable tool for creating 'little languages'."

ssivark 4y ago

Here's something I don't understand: How are "little languages" different from a bunch of functionality wrapped into a library/module? Is it just that (with some convenient syntax sprinkled on top), or is there more to it? I would imagine that most of the value comes from being able to "refactor" thought patterns to match the best way to cleave the domain into composable concepts -- and it seems like we do this all the time (and in all programming languages?).

honungsburk 4y ago
I agree, they are the same! regex is an example of an eDSL that most general-purpose languages support. The king of eDSL:s is of course Haskell. I recommend looking up parser combinators for anyone who has ever struggled with an understanding way to complex regex expressions.
bazoom42 4y ago
An “internal DSL” is a library with a a design which makes it “feel” like a language. JQuery is the classic example. An “external DSL” has its own syntax, like regexes.
francasso 4y ago
With a DSL you need to learn syntax, semantics and either the API for interacting with your GP language or the tooling for external DSLs. With a library you just need to learn the new semantics. I much prefer this to a DSL. Plus a library has all the debug tools the language has. DSLs usually don't have any debug tools except for printing when you are lucky
hackandthink 4y ago
"How are "little languages" different from a bunch of functionality wrapped into a library/module?" This is called a shallow embedding in the Haskell world. Deep Embedding is more like writing a full blown interpreter. Then there is tagless final (Oleg Kiselyov) - it feels shallow but is more flexible as simple library functions and it is optimisable like deep embedded DSLs. http://www.cse.chalmers.se/~josefs/publications/TFP12.pdf http://www.cse.chalmers.se/~josefs/publications/TFP12.pdf https://wiki.haskell.org/Embedded_domain_specific_language https://wiki.haskell.org/Embedded_domain_specific_language

auxfil 4y ago

This is a “deepity”. We already do this. We constantly do this in programming: “The idea is that as you start to find patterns in your application, you can encode them in a little language—this language would then allow you to express these patterns in a more compact manner than would be possible by other means of abstraction. Not only could this buck the trend of ever-growing applications, it would actually allow the code base to shrink during the course of development!” Functions, frameworks, little languages. It’s all abstractions on top of abstractions. You are shifting the knowledge of the abstraction for the more fundamental knowledge underneath that does the actual work. You end up just sweeping the codebase growth under some other layer’s rug and blissfully forget about the woes of future maintainers. The code is still there, abstracted and exposed by the “little language”. Hiding this behind a cute moniker doesn’t seduce. This isn’t the future of programming. This is already programming.

Spark_Ed 4y ago
Maybe the author is trying to predict that there will be boom in DSLs like there was for JS frameworks? Funnily enough, I'm just wrapping up a DSL for our in-house web component engine that creates an abstract data layer all components share. The pattern was easy enough that I'll probably build more DSLs like it when an API isn't flexible enough
- valenterry 4y ago
  The better the programming language, the less need for a custom language. You can then create the DSL inside of the host language. This requires flexibile syntax to some degree and a fairly advanced typesystem if it should be statically typed - hence not too many languages are a good choice here.
  - giraffe_lady 4y ago
    I don't think there's anything in that comment to indicate that they didn't do it that way. There's a robust discussion about this elsewhere, but people making embedded DSLs is basically routine. They just don't always call them that or have awareness that what they're doing fits that description. But almost all of those will be written in the "host language" of just whatever the containing system happens to be written in. Some languages are particularly suited for this but other than racket and ruby it seems mostly accidental/incidental to their design.

deafpolygon 4y ago

I think it's just trendy again. We've had this on-off on-again love affair with tiny languages and they simply don't stand the test of time.

btschaegg 4y ago

On that note: Am I the only one that's constantly surprised by the absence of proper sandboxing solutions when so many programming languages now provide (otherwise pretty useful) means of running code dynamically in a script-like fashion? In C#, I can pull in Roslyn, and compile a string on the fly as a C# script; but the way the .NET standard library is structured makes it pretty much unfeasible to prohibit outside interactions I don't want to allow (in my case, e.g: `DateTime.Now`, while allowing the Handling of `DateTime` values). It's possbile to embed the Typescript compiler into a website, but running code on the fly and some simple sandboxing is not feasible without a serious pile of hacks. I've recently read a forum thread about a library for compiling/running Elixir code as a script, but guess what: The runtime (apparently) makes sandboxing really hard. And so on and so on. I just wished that the LUA approach of "if I don't give you a hook, you cannot do that" were just the default. I've seen so many overcomplicated enterprise-y solutions that are basically just a plea for a well-designed, local and small scripting API…

orangea 4y ago
Ruby used to have the $SAFE feature for sandboxing, but it was removed because it was buggy, added a lot of complexity, and wasn't actually that useful. Linux has all the various isolation features that make Docker work, but people still recommend not running untrusted code in Docker containers because of the potential for oversights/"bugs" in Linux's API. I suspect that programming languages / VMs don't include these features because they are very hard to get right and add a disproportionate amount of complexity for their utility.
kaba0 4y ago
As far as I know most in-process sandboxing has been deprecated because it is in contrast to maintainability. E.g. Java decided against its Security Manager, because it is way too easy to leave the proper checks out of a new feature, leaving the whole thing vulnerable with a false sense of safety. Instead, process-level isolation is recommended.
rng_civ 4y ago
I think WASM is filling in that gap to some extent. From the spec: > Any interaction with the environment, such as I/O, access to resources, or operating system calls, can only be performed by invoking functions provided by the embedder and imported into a WebAssembly module And IIRC, the core instruction set is reasonably compact.

ArcMex 4y ago

My concern was how will putting these little languages together work and it turns out that the author also had these questions. Where do we draw the line on "enough DSLs" for example? And what happens to the gains from using several DSLs in tandem as opposed to a high-level language with libraries that accomplish the same thing?

helsinkiandrew 4y ago

> I’ve become convinced that “little languages”—small languages designed to solve very specific problems—are the future of programming, Yes, but over time very specific problems become bigger/different problems which the little language isn't ideal for, the original developers move on leaving someone new to figure out the problem and language which is probably poorly documented and very brittle. Application developers probably aren't suited to writing and maintaining language code. The only caveat is an external system provided with its own language - like RDB/SQL - which is proven and well maintained - but its hard to call SQL a little language.

bayesian_horse 4y ago

I can hardly get my coworkers to understand one language well.

diffxx 4y ago

DSLs are not a replacement for but a complement to any existing language, general purpose or specialized. I have come to think of DSLs as programs for writing programs (similar to but not identical to macros). With a DSL, you can specify the grammar of a specific problem/program. Once that has been done, it is often quite straightforward to implement the grammar in any number of target languages. As an application developer, this may not be a huge advantage (though DSLs can also shine in any client/server interactions), but if you are a library author this can be very compelling because your library may be easily portable to most commonly used language runtimes in a generally rote kind of way. The port might not be optimal, but it should be correct, provided the high level logic of the DSL is. Performance optimizations can be done where needed. What is great about this approach as an individual is that it requires you to tighten your ideas. When you have to implement all of the functionality in a DSL, you really start thinking about what you truly need. A big language nudges you towards using all of its features while a small language challenges you to consider what is truly essential. Of course DSLs always run the risk of being write only and/or only comprehensible by the original author. Like any powerful tool, DSLs should be used judiciously and responsibly. Often that isn't the case, in part because I don't think the tooling for writing DSLs is generally very good. But I am betting that new tools that make DSL writing easy will have a profound effect on software development.

samsquire 4y ago

Join any company and organisation and look at their build and deployment tooling. Unless they are using Kubernetes and even then, you shall find a very complicated bunch of languages: - shell scripts - Dockerfiles - Kubernetes YAML - Makefiles - Bazel - Ansible - python scripts - Jenkins XML - Groovy scripts - Ruby scripts - CloudFormation - Terraform - Fabric or other deployment deployment script It's very hard to fit together and understand from a high level. The last thing they were working on at my previous company was a YAML format to define a server, to go through the organisational structure of the company to manage computer systems. Some people mentioned LISP in this comment thread. For me LISP is an intermediate language, I would never want to build a large system in LISP. It's not how I think about computation.

throwaway17_17 4y ago
Can you expand on you mean when you say that Lisp is not ‘how [you] think about computation’. I have never seen that phrase used regarding Lisp, usually it’s mentioned in regards to logic or other much less mainstream paradigm than the imperative/semi-functional paradigm in Lisp.
- samsquire 4y ago
  I see LISP as being useful for codegen and intermediate representation and AST representation but I wouldn't want to program with it directly without a tool to create a structure that is understandable to me. All the parenthesis! I wouldn't want to maintain or work on a large Clojure codebase written by other people. I've done that three times. For reference, I think Python is easy to write and read and understand. I wrote a simple toy multithreaded interpreter and I've written part of a compiler that does codegen to target the imaginary interpreter. It's basic but my AST is a tree that could be represented in LISP. I use a handwritten recursive descent pratt parser. The language looks similar to Javascript. I know it can fixate your thinking if you think of it too much in this manner, but I think of modern computers as turing machines. They loop or iterate through over memory addresses which are data or instructions and execute them. That said, my perspective is not traditional. I design and try implementing programming languages. I am interested in the structure of problems and code, asynchrony, coroutines, parallelism and multithreading more than anything else. Even more than type systems. I think the expression problem is a huge problem that doesn't have good solutions for managing complexity. I find other people's LISP code to be difficult to read whereas I can understand a Python, Java algorithm. What am I trying to say? The structure of the program in the developer and compiler's head is different from the instructions actually executed by the computer. LISP is nearer to the instructions executed by the computer than what exists in my mind. In my mind exists relationships, ideas more complicated and not structured in post order traversal. A post order traversal of LISP is the codegen.

galaxyLogic 4y ago

The problem with multiple small languages is that it is not trivial to make the call each other

adamddev1 4y ago

Is this just like the push for microservices, but for languages?

hkt 4y ago

Empirically speaking, it seems more like rich languages are the past, present and future of programming. Rich as in, has a strong standard library. Python, go, etc. The only "little" languages I can think of that I'd reasonably ask people to use at work are lua, make(~), awk, and (ba)sh.

Mikhail_Edoshin 4y ago

Unlikely. On one hand notations should be a commodity and they indeed should be unique to the task. There is no point and no way to try making a unified notation for music and chess. On the other hand there is no point to make multiple notations for the same thing. And programming is indeed the same thing from lowest to highest levels, composable like a Russian doll, or we won't be able to build large systems. So the future of programming is a single notation that actually reflects what programming is. We do not have it yet, this is why we have so many "programming languages".

brundolf 4y ago

This is what I always heard Lisp was best at. Instead of making totally new languages (with parsers, tooling, etc) you'd create little DSLs within your own code in the form of macros: come up with a "little language" for describing one part of your app, write a macro for it, and then it integrates smoothly with everything around it Whether or not you agree this philosophy is a good one, and whether or not you like Lisp specifically, I think we can all agree that macros (in whichever language) are a much better way to do it than creating a bunch of tiny languages from scratch. I was surprised not to see the word "macro" appear in the article at all

brabel 4y ago
Macros are very hard to write tooling for, and hence difficult to author and use. I think that, perhaps something like Polyglot GraalVM[1], which was designed to host many languages and let them all seamlessly talk with each other, while automatically making existing tooling "just work" with any new languages created with the framework [2] would be a better way. [1] https://www.graalvm.org/22.0/reference-manual/polyglot-programming/ https://www.graalvm.org/22.0/reference-manual/polyglot-progr... [2] https://www.graalvm.org/22.3/graalvm-as-a-platform/implement-language/ https://www.graalvm.org/22.3/graalvm-as-a-platform/implement...
dmux 4y ago
Lisp macros are nice, but as Gumby said above [0] a common approach is to use Lisp's quoting abilities to construct a data structure that represents the problem at hand (in its own terms) and then create functions to parse / manipulate that data structure. A classic example comes from Peter Norvig's "Principles of Artificial Intelligence" wherein he defines a subset of English grammar as a data structure [1]: '((sentence -> (noun-phrase verb-phrase)) (noun-phrase -> (Article Noun)) (verb-phrase -> (Verb noun-phrase)) (Article -> the a) (Noun -> man ball woman table) (Verb -> hit took saw liked))) He then goes on to define a function "generate" that uses the above to create simplistic English sentences. Additional rules can be added by a non-programmar so long as they understand how their domain logic has been mapped to Lisp. [0] https://news.ycombinator.com/item?id=33705558 https://news.ycombinator.com/item?id=33705558 [1] https://github.com/norvig/paip-lisp/blob/main/docs/chapter2.md#21-a-grammar-for-a-subset-of-english https://github.com/norvig/paip-lisp/blob/main/docs/chapter2....

015a 4y ago

Is the engineering footprint of an organization really better if everything is implemented in twenty different languages, versus just three or four? Everything else aside, quality of the language, scope, etc; just the number. You have to expect everyone to know each language; know the ins, outs, idioms, gotchas, etc. You have to be able to hire for the languages. You need the language runtimes in your environment, everywhere, docker, local dev machines. You have to keep up to date in X times more changelogs, version upgrades, CVEs. The article pulls Shell as an early example. Shell did not become the powerhouse it is because its "great" (though some would argue it is, I'm not here to debate that); or because its small; or because its general purpose; or because its single-purpose. It became a powerhouse because its Old and Omnipresent. See, the problem with inventing New Things is that they are, by definition, not Old, nor Omnipresent. New Things have to start somewhere, but you're starting in last place. > Regular expressions and SQL won’t let you express anything but text search and database operations, respectively. Oh mylanta. Did you know that after the addition of back-expressions, Regular Expressions became turing complete? They are, functionally, a real programming language, just like C; well, except, far more annoying to write. And naturally, SQL "won't let you express anything but database operations", which is to say nothing about "SELECT 1+1"... let alone the little corner of the language called "Stored Procedures".

gpderetta 4y ago
That's the inner platform syndrome, and yes, it is bad.
0x445442 4y ago
There’s a Groovy DSL that will allow you to perform operations on collections using SQL syntax. I’d argue it’s an improvement over the traditional procedural or functional approach’s. https://groovy-lang.org/using-ginq.html https://groovy-lang.org/using-ginq.html
- 015a 4y ago
  Well, maybe this will start a flame war: I will absolutely die on the hill that SQL is shit. But, like shell, is old (48 years old!) and omnipresent; so it sticks around. Like, the thing I find comical about the original article is: they reach for shell, regex, and SQL as Prime Example of "mini languages done right". Be more domain specific, look at this its already happening. All of those examples are kinda shit languages! They're popular because of their omnipresence. Maybe regex is "fine", though the line "try to solve a problem with a regex and now you have two problems" is well known for a reason. But my broader opinion is: if you're building an application, service, tool, whatever in LANGUAGE_X; having to "dip out" of that language into an entirely different language should be viewed as, fundamentally, a Negative Thing. There may be reasons why you should; the good may outweigh the bad; but there is Bad there. There will always be an interpretation layer; extra tooling; that's More Things that can go wrong, have to be configured, statically analyzed, tested, its a failure point. SQL injection is a thing. Why? Because, for a time, people had the thought "hey, its a string, lets just template the string"; but that's no good, so now we have A Layer between the Java and the SQL to keep us safe. We can only support So Many Layers; we need to make things simpler, not more complex, there needs to be Fewer Things. Its time is gone, but I'll always view Heroku Buildpacks as a paragon of system design. Consider: You could write a nodejs application in javascript, write a package.json in json (also javascript), and end-to-end get that thing on a URL on the internet, all in one language. Fantastic! Today, a typical app will have the app (say, Golang), a go.mod (different syntax than go itself), dockerfiles (language 3), kubernetes yamls (language 4), maybe helm or cdk8s (language 5), shell scripts, makefiles, maybe you're also writing sql... this isn't better and it doesn't have to be like this. But right now, it is, and to some degree I'm happy for it because its 80% of the reason why I get paid six figures.

chantepierre 4y ago

I am currently building a tiny language designed only for data manipulation in a long-term personal project : data made of number/strings/booleans in maps or lists goes in, can be re-shaped, and goes out. It's written in Typescript as part of a web app. It just has map/list access & creation, string manipulation & concatenation, basic arithmetic/comparison, lambdas as first-class values, function calls. No way to do I/O, just data in-data out. I need some non-programmers friends to collaborate on the tiny data transforms and it is quite easy to write a 1:1 text <> visual editor for this language thanks to its limited features, allowing to go from visual to text or text to visual. The exercise in itself is interesting and worth it. I think I wouldn't do that in a commercial context though.

macrolime 4y ago

If a lot of little languages /DSLs were going to be practical as another layer of abstraction up from APIs, I think it would have to be implemented in something like Unison. Unison is made so that that you can have different syntaxes for the language or parts or the language as the language is not stored as text,but more or less the abstract syntax tree. Then again so is Lisp more or less. I suspect that instead what will happen as the next big thing in terms of new abstractions in programming is that AI code generators will keep getting better and get better tooling and that's going to be the tool of choice for high level needs. While these AI tools may not be that accurate today there is vast potential in improving the models and tooling. In the IDE it could be more like you describe what you want, it generates some code and tests and then run the tests and maybe a visual time traveling debugger so you can visualize what it's doing to see if it's doing the right thing.

jstx1 4y ago

If the point is that DSLs will be around, sure. They already are. If the point is that we'll replace complex full-featured languages with a stack of DSLs - no way.

yellowapple 4y ago

I think this works best if the little languages all share as much syntax and semantics as possible. A good example of this is OpenBSD's assortment of configuration file syntaxen for OpenSMTPd, pf, httpd/relayd, etc.; each of those "little languages" differ considerably in their problem domains, but they all seem to share a vaguely-Tcl-ish syntax and have largely converged in semantics and typical structure. Another important consideration is that these languages are typically best when declarative as possible; if you can avoid Turing-completeness and stick entirely to something representing static data, then that's the ideal.

frou_dh 4y ago
GNU Guile was supposed to be such an ubiquitous language but AFAIK it didn't really take off.

caporaltito 4y ago

Remember, when a guy writes a post with an absolute title, the answer is always "No".

aargh_aargh 4y ago
Including yours.

kaba0 4y ago

While I also believe several commenter’s opinion that libraries, frameworks can easily become DSLs themselves (e.g. isn’t Java streams basically a DSL for stream processes inside another language?), one really can’t talk about polyglottism without mentioning the ingenuity of GraalVM. You basically write a dumb, easy parser and AST interpreter for your language, and it will magically turn into a JIT compiled dynamic language with state-of-the-art GCs and better performance than what you could likely come up with. And the best thing is that it really unifies the computing model where you can pass a python object to some js lib, effectively giving you every library ever written for any (implemented) language, which is the real productivity booster.

adql 4y ago
Yes but library that turns a part of syntax of language into DSL to solve certain problem have advantage of developers already knowing that language, developers needing to learn only one language, and ability to use with that DSL anything else language libraries offer. The problem is really "does the language app uses works for target audience that will be doing the DSL". If it does (Ruby makes pretty decent one), job done, if it doesn't, we end up in the mess of making toy languages (usually because developers want to do something fun and writing new small language can be pretty fun) or plugging something like Lua into it.

iovrthoughtthis 4y ago

whats old is new dsl's are now little languages

lysecret 4y ago

Interesting article, I immediately had to think about terraform. The problem I'm having with specialized languages (even with SQL) is that it always creates additional interfaces and almost always creates ugly string formatting if you want to integrate (SQL) or ugly duplication (Terraform) if you keep it separate. I like the idea of abstraction but in my mind it is very easy to have the power of a "little language" inside an all purpose language by using a package. E.g. SQLAlchemy or Pulumi as the alternative to the little languages of SQL and TF.

LordHeini 4y ago

I disagree and would say DSLs should go away if possible. DSLs like SQL are the norm and you can see the problem of them in basically every project. You either use ORMs or you end up hand rolling SQL rows into Structs or Classes. The whole mapping usually looks like crap and contains a bunch of implicit corner cases, which eventually end up being a footgun for someone. Usually the SQL sever runs somewhere else, the ports are wrong, the language version is wrong, or a migration failed and a function is missing yada yada.... The same is usually true for Regexp. There are a billion dialects and every single one of them is basically unreadable, incomplete or just weird. The same is true for microservices with tons of config files for dev, staging, testing and production... Everything has its own version, can be down or mutate some random state somewhere while depending one other servcies. It always breaks at the seams. Increasing the amount of DSLs increases the amount of seams and thus makes software worse.

zelphirkalt 4y ago
But your ORM example could be considered a DSL in itself, so it doesn't really work that well. ORMs can be implemented in various ways too, as a syntactically looking different DSL or as something like objects and method calls or whatever the language already provides. Regexes can be abstracted from exactly by making a more readable DSL. Config files and the stuff accepted therein are small languages as well. What are the alternatives to making little languages?
Octoth0rpe 4y ago
> DSLs like SQL are the norm and you can see the problem of them in basically every project. I think your points re: sql are true of straightforward crud apps, but not true at all of an analytics app. In those cases, the sql is often _very_ complex, and while the results of a query may be eventually mapped into a struct or something, the query generation is rarely a simple mapping of properties in an object to select columns. > There are a billion dialects and every single one of them is basically unreadable, incomplete or just weird. Sure, but in the vast majority of cases, one only has to deal with at most 3 dialects, and there's a good chance you won't be hitting the corner cases that make each dialect significantly different.

cies 4y ago

Some more little language not listed by the article: Elm (browser apps w/o runtime errors). The creator (BDFL?) of Elm, Evan, specifically avoids making it general purpose.

hnbad 4y ago

Calling SQL a "little language" must surely be a joke. Even if you ignore the differences between its many dialects, SQL is only "little" in that it is domain-specific rather than general purpose. In fact the entire article seems to boil down to "DSLs are the future", which I'm sure I've seen articles about back when Ruby on Rails was dominating web technologies, Cucumber (and its various ports) created "BDD" testing fad and DevOps started gaining traction on top of various "Ruby DSLs" used as configuration formats. I don't think DSLs are going to go away any time soon. But there is a trade-off between domain-specific "little languages" and general purpose programming languages (or "DSLs" that are actually subsets of the latter). It can be fun to have to work with a little language, it's not so fun to work with dozens of them, each with different rules you have to memorize, instead of just being able to use the same language for all (and in truth, this was the source of the Ruby DSL craze because developers were already using Ruby on Rails).

BiteCode_dev 4y ago

> SQL is a little language for describing database operations Yeah, SQL is not a little language anymore. It started as one, but because a lot has been added to it and SQL flavors for Oracle or Postgres are anything but tiny. Windowing, nesting, json handling... I think the author is kinda proving with this that a successful little language does not stay little, and hence little languages are not the future. And don't get me started on DSL in general. Just lookup my username and "DSL" on hackernews for endless rambling.

raydiatian 4y ago
DSLs are great for job security in small organizations. They might as well be regarded as criminal.
williamcotton 4y ago
The reason why the author is using the term little language is because the term DSL doesn’t mean custom language any longer. Yes, SQL is bigger than it used to be but it is still a domain specific language, which is what the article is actually about.
redanddead 4y ago
Maybe the term modular would be more appropriate than little? Every solution that has gained widespread dominance, take the shipping container, has been thanks to its flexibility, and modularity.

ergonaught 4y ago

I feel like lisp refutes this article, in that most of the “proposed solution” already exists and already failed in actual practice.

hoseja 4y ago

I don't think programming has yet stagnated enough for this to hold. For domain specific languages you need specified domains.

mlatu 4y ago

hm, im a little unhappy about the author comparing Knuth's solution to a handfull of shell utilities. for one, the author says knuths program written in WEB was 10 pages long, discounting the fact that these 10 pages are HEAVILY annotated. my other point is: tr has 1917 LOC, sort has 4856, uniq has 663 and sed is in its own package at around 10 MB all including comments and docs for sed it's fine and good that you can use composition with shell utilities, but come on, write that example program in C99 and you'll be a not very happy coder at all. in general i find the comparison rather rude. Knuth was supposed to show ?his? programming language WEB and as a "critique" McIlroy farts out a shellscript like "lmao first". indeed, you do not often need to count word frequencies. but what was this article supposed to be really about? software engineering 101 aka dont-reinvent-the-wheel/DRY? or perhaps literate programming?

cb321 4y ago
Related: https://buttondown.email/hillelwayne/archive/donald-knuth-was-framed/ https://buttondown.email/hillelwayne/archive/donald-knuth-wa... As you say, Knuth was asked to demonstrate his literate programming... In some ways this is a direct request for the non-pithy, articulated, first principles answer. I would more say Knuth was set up than that he was framed, but tomato-tomato. :)

nigamanth 4y ago

Wouldn't that require programmers to use 50 languages in a project though? Assuming that this is the future of programming, the number of languages that a programmer would have to learn to create a simple project would be ridiculous. Why use many languages when you can do the same process in one? Sure, even if the "big" language isn't specifically made for a certain job, doing that job in a little language requires time for the programmer to learn the little language. The syntax may be different, so to convert from the big languages to little languages it would take time. This is why this hasn't happened yet; people are lazy to learn new languages so they just learn the "big" languages that can complete all the tasks that they require.

bvrmn 4y ago

It seems DSL proponents grow from environment without rich standard library. And classic Knuth argument, of course. It's so hard to count words. General purpose language: >>> from collections import Counter >>> import re >>> Counter(re.findall('\w+', 'boo foo boo +dfd zii')).most_common(1) [('boo', 2)]

beanjuiceII 4y ago

just what i want, a zillion little languages to remember their syntax and semantic quirks.. its bad enough the way it is right now where everyone tells me "the right tool for the right job" then i get there and realize they are all the same tool

gumby 4y ago

Little languages are the past and yes, the future. We just don't recognise them. It was common in the 60s and 70s to have the hardware manufacturer ship all the OS and languages with their hardware. The languages were often designed for specific problem domains. The idea of general purpose languages (FORTRAN, PL/1, etc) was uncommon. You can see this in K&R (the original edition anyway) where they justify the idea of a general purpose language, even though C itself derived from prior general languages (B & BCPL) and they had gotten the idea from their experience on Multics (written in PL/1, a radical idea at the time). So a 20 year old idea was still barely diffused into the computing Zeitgeist. Most Lisp development (since the early 70s at least) is writing a domain-specific representation (data structures and functions) and then writing your actual problem in it. I used both Lisp and Smalltalk this way at PARC in the early 80s. More rigid languages (the more modern algolish languages like python, c++, rust, C, js etc -- almost every one of them) doesn't have these kind of affordances but instead do the same via APIs. Every API is itself a "little language" What are called little languages in the Bently sense is simply a direct interface for domain experts. And after all what was a language like, say, Macsyma but a (large) "little language"?

Verdex 4y ago
> Every API is itself a "little language" I came to this conclusion early in my career. It went something like this: A - "To do this, just create this object, fill in these properties, and call these methods." B - "Okay, I did that, but it crashed." A - "Yeah, it's because you set the properties in the wrong order. This property relies on this other property under the hood. Set them in this order." B - "Still crashes." A - "Yeah, you called the methods in the wrong order. This method relies on that method. Call them in this order and it works." My conclusion was that the lisp philosophy of building a lot of little sub language was equivalent to what people were doing with OO in C#/Java. Either way you have to learn the "right" way to put things together which is dictated by unseen forces behind the scene. Of course, I also concluded that most people work differently than I do. For most people, if the code "looks right" (ie recognizable syntax) then they're able to tell themselves a story that it's familiar and their intuition is able to pick up the slack for finding the right enough way to use most arbitrary APIs (just as long as they don't exceed some level of incomprehensibility). On the other hand, I have to understand the underlying logic or I use the API the wrong way pretty much every time. So for most people lots of APIs is actually a much better cognitive way for them to work whereas for me API soup and lisp macros are the same conundrum.

raydiatian 4y ago

> For example, SQL is a little language for describing database operations. Regular expressions is a little language for text matching. Dhall is a little language for configuration management, and so on. > There are a few other names for these languages: Domain-specific languages (DSL:s), problem-oriented languages, etc. However, I like the term “little languages”, partially because the term “DSL” has become overloaded to mean anything from a library with a fluent interface to a full-blown query language like SQL, but also because “little languages” emphasizes their diminutive nature. Ahem, so is SQL diminuitive or not? Because, SQL is NOT diminuitive. SQL is Turing complete.

jkingsbery 4y ago

Among the problems this article ignores is the relatively large fixed cost needed to go from a framework/internal-DSL to a separate "language." Unless you are in a situation where lots of people have that same problem, then it actually creates more complexity than it solves to have an separate language to solve the sub problem.

CyberDildonics 4y ago

A huge problem with this approach is that very few general purpose languages reach the point where they have big standard libraries, great external libraries and tools on top of them. Every time someone invents a new language it wipes away the decades of progress of the general purpose language that could be used instead, so it needs to have a massive advantage or be completely necessary.

kwhitefoot 4y ago

I've always regarded the principal task when implementing a program to be creating the necessary data structures and a set of functions that represent a vocabulary for manipulating them. So every program I write contains a small domain specific language. Isn't this what everyone does? 'Official' small languages only make sense when the user base is large, this is why SQL exists but a mainstream language for manipulating Maxwell's equations does not.

oscarb92 4y ago

I am very familiar with DSLs but had not heard them called "little languages". I cannot say I find it fitting as it makes me think more of languages with reduced syntax or semantics.

giraffe_lady 4y ago
I haven't done any research on this but I suspect that usage predates "DSL," or at least was popular before the term DSL was widely known and used for it. I mostly see "little language" used in older sources, mid-90s or earlier. It seems to be making a comeback though! I prefer it too. It's not really more descriptive than DSL but it's not much less, and is less jargony and just cuter.
- radiator 4y ago
  This seems corect. The AWK Programming Language, 1988, Chapter 6: Little Languages https://archive.org/details/pdfy-MgN0H1joIoDVoIC7/page/n11/mode/2up https://archive.org/details/pdfy-MgN0H1joIoDVoIC7/page/n11/m...

sunarch 4y ago

I've seen other replies highlighting Lisp in general and Racket in particular, but when it comes to a "little language" I think it's valuable to have a link to Paul Graham's article on The Roots of Lisp: http://www.paulgraham.com/rootsoflisp.html http://www.paulgraham.com/rootsoflisp.html It underlines all the points already made here about Lisp, and also includes a CL version of the original code. 67 lines including comments and empty lines. Endless possibilities.

garethrowlands 4y ago

Rule of least power also relevant here, https://en.m.wikipedia.org/wiki/Rule_of_least_power https://en.m.wikipedia.org/wiki/Rule_of_least_power The web is designed using a variety of little languages. It could just be JavaScript with an API but it's not. And it's probably best as it is.

beefok 4y ago

I've always enjoyed reading this book that espouses this same idea: Constructing Language Processors for Little Languages 1st Edition by Randy M. Kaplan [1]. It's no longer being produced, but used it's $6 or less. It's dated, from 1994, but it is a fun enjoyable discussion on the benefit of tiny specific languages. It also has a nice tutorial on the use of lex and yacc. [1] https://www.amazon.com/gp/product/0471597546 https://www.amazon.com/gp/product/0471597546

OhNoNotAgain_99 4y ago

anikom15 4y ago

DSLs and general purpose languages are not comparable. DSLs are generally designed to handle data, while general purpose languages can handle everything. Where DSLs fall apart is when you either need to (a) work with I/O in a different format than the DSL handles data in (including working with hardware) or (b) the scope of data changes into something that now incorporates information outside the DSL’s domain. For (a), a simple example is an application that reads information from a DAQ, stores it in a database in a compressed format, and later sends the data to a printer. We have a DSL that can easily implement the database, but we need to translate the data from the DAQ into a compatible input, so that requires a different language. Furthermore, the printer requires postscript, so we need some other language or tool for that. Then we need to figure out how to glue it all together. In some cases, it becomes tempting to try to ‘hack’ the DSL into trying to do things it was never designed for. Or we could just use a general purpose language and libraries.