5 ms·

Lisp in Web-Based Applications (2001)

124 points by bschne 4mo ago

gwern 4mo ago
Anyone know how redundant this is with the pg essays on his website? Not sure I've seen much about 'Rtml' or other technical details of Viaweb before.
- rmunn 4mo ago
  This is one of the essays linked on his site: https://paulgraham.com/lwba.html https://paulgraham.com/lwba.html Which just links to the same ASCII-text link at https://sep.turbifycdn.com/ty/cdn/paulgraham/bbnexcerpts.txt https://sep.turbifycdn.com/ty/cdn/paulgraham/bbnexcerpts.txt, so it's exactly the same link as the one posted on HN here.
vindarel 4mo ago
I like History but I'll comment about Lisp in web apps today, if I may. We have a choice of web servers and web libraries (https://github.com/CodyReichert/awesome-cl/#web-development https://github.com/CodyReichert/awesome-cl/#web-development), and we live in happy times where HTMX or Datastar are great fit for Lisp -as with any stack. See these Datastar examples built in CL: https://github.com/fsmunoz/datastar-cl https://github.com/fsmunoz/datastar-cl I use and like the Mito ORM too, which comes with automatic migrations and, since last year, a composable query engine (SxQL). Here you will find some screenshots of some of today's web applications built in CL: http://lisp-screenshots.org/ http://lisp-screenshots.org/ and here an opinionated tutorial: https://web-apps-in-lisp.github.io/ https://web-apps-in-lisp.github.io/ One example: ScreenShotBot https://screenshotbot.io/ https://screenshotbot.io/ a successful open-source product and company. It now replaced Facebook's automatic screenshot testing tool (source: their blog). So, web apps in Lisp are possible -with a right amount of learning time and elbow grease. The incremental development and interactive top-level are still precious and unmatched.
- ares623 4mo ago
  The productivity high folks get from using tools like Claude Code are the same as what one gets from learning a Lisp and using something like Emacs as the editor. There is obviously a difference in capability, but do the cost differences scale linearly? Maybe when the party is over, the lost will yearn for the parentheses to get just a shadow of that high back.
  - cess11 4mo ago
    It's not "productivity" that sucks them in, it's the feeling of being seen by an apparently nice and subservient person.
    - satvikpendem 4mo ago
      Lol, come on now, you think it's not productivity and only some sort of feeling of being acquiesced to?
      - cess11 4mo ago
        Yes, absolutely.
        satvikpendem 4mo ago
        Tell that to all the AI users then because that's a ridiculous assertion that even some notable fraction use it for the purpose you're talking about, much less all. Most people simply use it for information, very few treat it like a person.
        cess11 4mo ago
        Why? The creator of ELIZA tried that, it didn't work then, why would it now? We've had code generation tools for decades, abstracting over code generation has been a solved problem for a long time, and as it happens, this is not in itself a very rewarding practice except for a small segment of professional developers that get a kick out of the elegance of some technique or other. The new thing about these chat bots is that they provide a feeling of being seen and appreciated, and unlike a coworker they seemingly carry very little risk of embarrassment, pain and so on while they also require very little effort compared to human relations.
neilv 4mo ago
I contributed to a contemporaneous large and complex system in Lisp (Scheme) that used most of those. It was one of those things where 1-3 programmers, working with the right tools and thinking, could out-perform all competitors, no matter how much resources competitors threw at the problems. (I can't take credit for the initial architecture and implementation: that was all someone else.) The system actually outlived Viaweb by decades, and I suspect still uses Scheme for a lot, especially for the necessarily complex backend. (Half the problem for Scheme uptake was that most students only saw it school, as presented by a CS professor in an intro class, and then they did annoying homework, so they hoped never to use it again. They should've seen what we did with it, making it fly in ways that a CS professor wouldn't in first-year classes, compared to how it would've been done in other languages at the time.) > One way we used macros was to generate Html. There is a very natural fit between macros and Html, because Html is a prefix notation like Lisp, and Html is recursive like Lisp. So we had macro calls within macro calls, generating the most complicated Html, and it was all still very manageable. We just used lists and functions for HTML. And often with quasiquote, when we wanted to splice "dynamic" bits into a substantial chunk of "static". I didn't try to use macros for HTML until a decade after this talk. I liked the simple DSL, and leaning on buffered I/O ports and string ports, but nobody else seemed to like it as much as I did (maybe because they didn't have a prolific colleague doing 10x code-writing in a rather more delicate way of generating HTML): https://www.neilvandyke.org/racket/html-template/ https://www.neilvandyke.org/racket/html-template/ (Don't look at the code, though. I wrote it before Racket got submodules, which are great if you have a strict, non-CL-ish module system and syntax extension mechanism that otherwise make some things much harder than in CL. I have a TODO note in there to refactor it with submodules, but I had to redirect my time towards other languages, for reasons having nothing to do with language merit.)
- bschne 4mo ago
  I wonder what your take is on the current landscape --- do you see this approach still having an edge? If you wanted to outcompete for a current-day SaaS app, do you see an undervalued approach that would still confer an advantage?
  - 4mo ago
anti-zionist 4mo ago
[dead]
tosh 4mo ago
love this on keyword parameters: > Rtml even depended heavily on keyword parameters, which up to that time I had always considered one of the more dubious features of Common Lisp. Because of the way Web-based software gets released, you have to design the software so that it's easy to change. And Rtml itself had to be easy to change, just like any other part of the software. Most of the operators in Rtml were designed to take keyword parameters, and what a help that turned out to be. If I wanted to add another dimension to the behavior of one of the operators, I could just add a new keyword parameter, and everyone's existing templates would continue to work. A few of the Rtml operators didn't take keyword parameters, because I didn't think I'd ever need to change them, and almost every one I ended up kicking myself about later. If I could go back and start over from scratch, one of the things I'd change would be that I'd make every Rtml operator take keyword parameters.
david04 4mo ago
"Closures Simulate Subroutines" - I really love this way of coding for the web, it's what I did when I created FastScala: https://www.fastscala.com/ https://www.fastscala.com/ (same logic)
alexpotato 4mo ago
I found out recently that CloudFlare uses a dialect of Lisp for managing there DNS records: https://blog.cloudflare.com/topaz-policy-engine-design/ https://blog.cloudflare.com/topaz-policy-engine-design/
t0mpr1c3 4mo ago
Paul Graham glosses over the details: writing LISP in continuation passing style is not for the faint hearted. For a modern implementation that hides the gnarly stuff, see https://docs.racket-lang.org/web-server/stateless.html https://docs.racket-lang.org/web-server/stateless.html The disadvantage of this kind of stateless server is that you need to keep around a hash full of continuations, basically one for every page visit. Each of these continuations contains an entire environment and stack, and you have to manage all that memory allocation somehow. You need to serialize the continuations in order to cache them. Then you need to figure out which ones to keep around. Authentication is potentially also an issue because absent some other security mechanism, anyone who knows the hash of the continuation can visit the page with the same permissions as the original user. https://docs.racket-lang.org/web-server/faq.html https://docs.racket-lang.org/web-server/faq.html (10.7) Apparently this website runs on LISP so I would be interested to know how it works under the hood. For me, anything more than a SPA seemed like too much pain.
- graemep 4mo ago
  A BEAM process per user would have similar advantages, is pretty memory efficient, and does not need a lot of extra work.
- bjoli 4mo ago
  Using delimited continuations in a web framework is magic. In remember the first time I saw things like pharo and wodered how it was even possible. Doing that stuff in Python is still hard. Doing it in racket is easy peasy.

124 points by bschne 4mo ago

gwern 4mo ago

Anyone know how redundant this is with the pg essays on his website? Not sure I've seen much about 'Rtml' or other technical details of Viaweb before.

rmunn 4mo ago
This is one of the essays linked on his site: https://paulgraham.com/lwba.html https://paulgraham.com/lwba.html Which just links to the same ASCII-text link at https://sep.turbifycdn.com/ty/cdn/paulgraham/bbnexcerpts.txt https://sep.turbifycdn.com/ty/cdn/paulgraham/bbnexcerpts.txt, so it's exactly the same link as the one posted on HN here.

vindarel 4mo ago

I like History but I'll comment about Lisp in web apps today, if I may. We have a choice of web servers and web libraries (https://github.com/CodyReichert/awesome-cl/#web-development https://github.com/CodyReichert/awesome-cl/#web-development), and we live in happy times where HTMX or Datastar are great fit for Lisp -as with any stack. See these Datastar examples built in CL: https://github.com/fsmunoz/datastar-cl https://github.com/fsmunoz/datastar-cl I use and like the Mito ORM too, which comes with automatic migrations and, since last year, a composable query engine (SxQL). Here you will find some screenshots of some of today's web applications built in CL: http://lisp-screenshots.org/ http://lisp-screenshots.org/ and here an opinionated tutorial: https://web-apps-in-lisp.github.io/ https://web-apps-in-lisp.github.io/ One example: ScreenShotBot https://screenshotbot.io/ https://screenshotbot.io/ a successful open-source product and company. It now replaced Facebook's automatic screenshot testing tool (source: their blog). So, web apps in Lisp are possible -with a right amount of learning time and elbow grease. The incremental development and interactive top-level are still precious and unmatched.

ares623 4mo ago
The productivity high folks get from using tools like Claude Code are the same as what one gets from learning a Lisp and using something like Emacs as the editor. There is obviously a difference in capability, but do the cost differences scale linearly? Maybe when the party is over, the lost will yearn for the parentheses to get just a shadow of that high back.
- cess11 4mo ago
  It's not "productivity" that sucks them in, it's the feeling of being seen by an apparently nice and subservient person.
  - satvikpendem 4mo ago
    Lol, come on now, you think it's not productivity and only some sort of feeling of being acquiesced to?
    - cess11 4mo ago
      Yes, absolutely.
      - satvikpendem 4mo ago
        Tell that to all the AI users then because that's a ridiculous assertion that even some notable fraction use it for the purpose you're talking about, much less all. Most people simply use it for information, very few treat it like a person.
        cess11 4mo ago
        Why? The creator of ELIZA tried that, it didn't work then, why would it now? We've had code generation tools for decades, abstracting over code generation has been a solved problem for a long time, and as it happens, this is not in itself a very rewarding practice except for a small segment of professional developers that get a kick out of the elegance of some technique or other. The new thing about these chat bots is that they provide a feeling of being seen and appreciated, and unlike a coworker they seemingly carry very little risk of embarrassment, pain and so on while they also require very little effort compared to human relations.

neilv 4mo ago

I contributed to a contemporaneous large and complex system in Lisp (Scheme) that used most of those. It was one of those things where 1-3 programmers, working with the right tools and thinking, could out-perform all competitors, no matter how much resources competitors threw at the problems. (I can't take credit for the initial architecture and implementation: that was all someone else.) The system actually outlived Viaweb by decades, and I suspect still uses Scheme for a lot, especially for the necessarily complex backend. (Half the problem for Scheme uptake was that most students only saw it school, as presented by a CS professor in an intro class, and then they did annoying homework, so they hoped never to use it again. They should've seen what we did with it, making it fly in ways that a CS professor wouldn't in first-year classes, compared to how it would've been done in other languages at the time.) > One way we used macros was to generate Html. There is a very natural fit between macros and Html, because Html is a prefix notation like Lisp, and Html is recursive like Lisp. So we had macro calls within macro calls, generating the most complicated Html, and it was all still very manageable. We just used lists and functions for HTML. And often with quasiquote, when we wanted to splice "dynamic" bits into a substantial chunk of "static". I didn't try to use macros for HTML until a decade after this talk. I liked the simple DSL, and leaning on buffered I/O ports and string ports, but nobody else seemed to like it as much as I did (maybe because they didn't have a prolific colleague doing 10x code-writing in a rather more delicate way of generating HTML): https://www.neilvandyke.org/racket/html-template/ https://www.neilvandyke.org/racket/html-template/ (Don't look at the code, though. I wrote it before Racket got submodules, which are great if you have a strict, non-CL-ish module system and syntax extension mechanism that otherwise make some things much harder than in CL. I have a TODO note in there to refactor it with submodules, but I had to redirect my time towards other languages, for reasons having nothing to do with language merit.)

bschne 4mo ago
I wonder what your take is on the current landscape --- do you see this approach still having an edge? If you wanted to outcompete for a current-day SaaS app, do you see an undervalued approach that would still confer an advantage?
- 4mo ago

anti-zionist 4mo ago

[dead]

tosh 4mo ago

love this on keyword parameters: > Rtml even depended heavily on keyword parameters, which up to that time I had always considered one of the more dubious features of Common Lisp. Because of the way Web-based software gets released, you have to design the software so that it's easy to change. And Rtml itself had to be easy to change, just like any other part of the software. Most of the operators in Rtml were designed to take keyword parameters, and what a help that turned out to be. If I wanted to add another dimension to the behavior of one of the operators, I could just add a new keyword parameter, and everyone's existing templates would continue to work. A few of the Rtml operators didn't take keyword parameters, because I didn't think I'd ever need to change them, and almost every one I ended up kicking myself about later. If I could go back and start over from scratch, one of the things I'd change would be that I'd make every Rtml operator take keyword parameters.

david04 4mo ago

"Closures Simulate Subroutines" - I really love this way of coding for the web, it's what I did when I created FastScala: https://www.fastscala.com/ https://www.fastscala.com/ (same logic)

alexpotato 4mo ago

I found out recently that CloudFlare uses a dialect of Lisp for managing there DNS records: https://blog.cloudflare.com/topaz-policy-engine-design/ https://blog.cloudflare.com/topaz-policy-engine-design/

t0mpr1c3 4mo ago

Paul Graham glosses over the details: writing LISP in continuation passing style is not for the faint hearted. For a modern implementation that hides the gnarly stuff, see https://docs.racket-lang.org/web-server/stateless.html https://docs.racket-lang.org/web-server/stateless.html The disadvantage of this kind of stateless server is that you need to keep around a hash full of continuations, basically one for every page visit. Each of these continuations contains an entire environment and stack, and you have to manage all that memory allocation somehow. You need to serialize the continuations in order to cache them. Then you need to figure out which ones to keep around. Authentication is potentially also an issue because absent some other security mechanism, anyone who knows the hash of the continuation can visit the page with the same permissions as the original user. https://docs.racket-lang.org/web-server/faq.html https://docs.racket-lang.org/web-server/faq.html (10.7) Apparently this website runs on LISP so I would be interested to know how it works under the hood. For me, anything more than a SPA seemed like too much pain.

graemep 4mo ago
A BEAM process per user would have similar advantages, is pretty memory efficient, and does not need a lot of extra work.
bjoli 4mo ago
Using delimited continuations in a web framework is magic. In remember the first time I saw things like pharo and wodered how it was even possible. Doing that stuff in Python is still hard. Doing it in racket is easy peasy.