5 ms·

Introducing lisp.js: a Lisp for node.js

64 points by devijvers 16y ago

mark_l_watson 16y ago
I like this! I just put a reminder on my calendar for 3 months from now to more-fully check out the project.
maxtilford 16y ago
Awesome. I've got it up and running. I've been looking for a solution for the client/server distinction for a while. How much have you thought about the implementation of that?
- devijvers 16y ago
  Good to hear! What do have in mind when you say "client/server distinction"?
  - maxtilford 16y ago
    Mainly, I'm tired of using $.getJson, and I liked your on-server macro. Also, how is the server going to watch references on the server and vice-versa?
    - devijvers 16y ago
      Basically, the server "knows" (as in can know) the references that exist on the client and vice versa. When a watched reference changes value on the client an HTTP request/response is sent over the wire. The request carries the old and new value, the response is basically redundant but since it's HTTP it needs to be there. Hence, the response might carry new forms to be executed on the client. The client watching references on the server is obviously more tricky. node.js already supports keep-alive by default. There probably needs to be some kind of chuncked reply mechanism that allows the server to send changes without the client having to send polling requests. Obviously, if the reference is not going to change frequently it might be cheaper to just poll every once in a while.
    - Qerub 16y ago
      on-server reminds me of Hop. Take a look at http://hop.inria.fr/ http://hop.inria.fr/, maybe http://hop.inria.fr/usr/local/lib/hop/2.1.1/weblets/home/articles/hop-lang/article.html#HOP-services http://hop.inria.fr/usr/local/lib/hop/2.1.1/weblets/home/art... in particular.
gruseom 16y ago
While it's always great to see experimentation in the Lisp realm, I'm unclear as to what "a Lisp for node.js" will do that the mature Lisp-on-top-of-JS (Parenscript) doesn't already do.
- devijvers 16y ago
  Thanks for the reference to Parenscript which I'm unfamiliar with. The main point of lisp.js is to benefit from node.js' event-loop/non-blocking-i/o environment. node.js will be able to handle far greater loads than web servers that use blocking I/O. I'm not sure whether or not the Common Lisp web server exclusively uses non-blocking I/O on the account that I'm not familiar with that web server. I'm erring on the side of blocking I/O in which case node.js/lisp.js would run circles around that web server.
  - gruseom 16y ago
    Using Parenscript with Node.js would presumably mean throwing out any Common Lisp web server and relying on Node to do the I/O. Otherwise, as you point out, you don't get much benefit from Node. In such a setup, Node (or Node behind a reverse proxy) would be the web server, Parenscript code compiled to JS would do whatever was needed in Node, and if you wanted application logic in a Lisp at runtime, you could call it however Node does IPC. The value that Parenscript adds here is the same as for programs that run in a web browser: you write your JS in a Lispy way at a higher level of abstraction, mostly because you get the full power of Lisp macros. There are other wins too, but that's the big one. The runtime environment, though, is entirely JS, because PS is a compiler that targets JS. So I still don't see a difference in intent here between what PS is good for and what you're doing. I don't want to discourage you at all. Just curious about the differentiator.
    - devijvers 16y ago
      I've scanned the Parenscript docs and it doesn't seem immediately obvious how to combine node.js and Parenscript without having to include Common Lisp. Also, there's a number of things on my wish list that compelled me to write my own lisp implementation (there are other JavaScript lisp implementations): * only requirement is node.js and the lisp source distribution, no other dependencies. * ability to debug lisp code from the browser. * immutable data structures. I feel I can implement a lisp that can run the examples in <i>Let Over Lambda</i> in a couple of weeks. The lisp won't be as stable as Parenscript but that's ok. Only at that point can the real work start of building the web framework that is lisp.js. I think the most important bit for me is to have a base that I can modify as the needs arise. I'm already trying to include as many lisp concepts in the test source as I can find to make sure lisp.js is a general purpose lisp. In that respect I'm setting several goals: get lisp-in-lisp working, get Let Over Lambda examples working, ... any other useful lisp constructs that I come across. That's a journey I can only make if I do my own lisp implementation.
tlrobinson 16y ago
So you mean "Lisp for JavaScript"? Why limit it to node.js?
- devijvers 16y ago
  It might one day become a general lisp for javascript but for now the concurrency-free server-side environment of node.js is assumed.
DTrejo 16y ago
lisp.js will be a lisp-2 (like Common Lisp and unlike Clojure and Scheme) Can you talk more about this decision?
- devijvers 16y ago
  Macros are important, that's why I seeking guidance in Let Over Lambda. The most suitable lisps for writing macros are lisp-2, hence the preference for a lisp-2 dual namespace.
  - pwpwp 16y ago
    Lisp-2's have less potential for trouble with unhygienic macros, but they still have it. Hygienic macro systems work with Lisp-1 as well as with Lisp-2. Apropos, I've written a Lisp->JS compiler a while ago; you might find some of the code useful: http://github.com/manuel/cyberlisp/ http://github.com/manuel/cyberlisp/
    - DTrejo 16y ago
      You might also check out the Racket->JS compiler: http://planet.plt-scheme.org/display.ss?package=moby.plt&owner=dyoo http://planet.plt-scheme.org/display.ss?package=moby.plt&#38... I used it just last year — the developer, dyoo, is super responsive. I'm sure he'd be happy to talk shop with you: http://github.com/dyoo/moby-scheme http://github.com/dyoo/moby-scheme
    - sedachv 16y ago
      In regards to hygienic macros and Hoyte's Let Over Lambda, the type of variable capture hygiene is designed to prevent is actually used as a programming technique in the book; Hoyte calls it free variable injection. Lisp-1 vs Lisp-2 actually has very little to do with macros. Most of the inconvenience comes when writing functions - things like having to name list parameters "lst." The way Common Lisp gets around the variable capture problem is to forbid the rebinding of special forms, functions, and macros defined in the standard.* This is an ugly hack that forces early binding, and obviously does nothing to prevent capture of user-defined functions. The Scheme wiki has a good article on hygiene: http://community.schemewiki.org/?hygiene-versus-gensym http://community.schemewiki.org/?hygiene-versus-gensym * - This wasn't done for the sake of macros. The rationale behind the decision as I heard it is to prevent the hypothetical scenario of some standard function like cdr that's called inside the garbage collector or allocator from being rebound to something that tries to allocate memory and crashes the gc or sends the allocator into an infinite loop. I don't really buy it; it's easy enough to prevent these types of scenarios.

64 points by devijvers 16y ago

I like this! I just put a reminder on my calendar for 3 months from now to more-fully check out the project.

Awesome. I've got it up and running. I've been looking for a solution for the client/server distinction for a while. How much have you thought about the implementation of that?

devijvers 16y ago
Good to hear! What do have in mind when you say "client/server distinction"?
- maxtilford 16y ago
  Mainly, I'm tired of using $.getJson, and I liked your on-server macro. Also, how is the server going to watch references on the server and vice-versa?
  - devijvers 16y ago
    Basically, the server "knows" (as in can know) the references that exist on the client and vice versa. When a watched reference changes value on the client an HTTP request/response is sent over the wire. The request carries the old and new value, the response is basically redundant but since it's HTTP it needs to be there. Hence, the response might carry new forms to be executed on the client. The client watching references on the server is obviously more tricky. node.js already supports keep-alive by default. There probably needs to be some kind of chuncked reply mechanism that allows the server to send changes without the client having to send polling requests. Obviously, if the reference is not going to change frequently it might be cheaper to just poll every once in a while.
  - Qerub 16y ago
    on-server reminds me of Hop. Take a look at http://hop.inria.fr/ http://hop.inria.fr/, maybe http://hop.inria.fr/usr/local/lib/hop/2.1.1/weblets/home/articles/hop-lang/article.html#HOP-services http://hop.inria.fr/usr/local/lib/hop/2.1.1/weblets/home/art... in particular.

gruseom 16y ago

While it's always great to see experimentation in the Lisp realm, I'm unclear as to what "a Lisp for node.js" will do that the mature Lisp-on-top-of-JS (Parenscript) doesn't already do.

devijvers 16y ago
Thanks for the reference to Parenscript which I'm unfamiliar with. The main point of lisp.js is to benefit from node.js' event-loop/non-blocking-i/o environment. node.js will be able to handle far greater loads than web servers that use blocking I/O. I'm not sure whether or not the Common Lisp web server exclusively uses non-blocking I/O on the account that I'm not familiar with that web server. I'm erring on the side of blocking I/O in which case node.js/lisp.js would run circles around that web server.
- gruseom 16y ago
  Using Parenscript with Node.js would presumably mean throwing out any Common Lisp web server and relying on Node to do the I/O. Otherwise, as you point out, you don't get much benefit from Node. In such a setup, Node (or Node behind a reverse proxy) would be the web server, Parenscript code compiled to JS would do whatever was needed in Node, and if you wanted application logic in a Lisp at runtime, you could call it however Node does IPC. The value that Parenscript adds here is the same as for programs that run in a web browser: you write your JS in a Lispy way at a higher level of abstraction, mostly because you get the full power of Lisp macros. There are other wins too, but that's the big one. The runtime environment, though, is entirely JS, because PS is a compiler that targets JS. So I still don't see a difference in intent here between what PS is good for and what you're doing. I don't want to discourage you at all. Just curious about the differentiator.
  - devijvers 16y ago
    I've scanned the Parenscript docs and it doesn't seem immediately obvious how to combine node.js and Parenscript without having to include Common Lisp. Also, there's a number of things on my wish list that compelled me to write my own lisp implementation (there are other JavaScript lisp implementations): * only requirement is node.js and the lisp source distribution, no other dependencies. * ability to debug lisp code from the browser. * immutable data structures. I feel I can implement a lisp that can run the examples in <i>Let Over Lambda</i> in a couple of weeks. The lisp won't be as stable as Parenscript but that's ok. Only at that point can the real work start of building the web framework that is lisp.js. I think the most important bit for me is to have a base that I can modify as the needs arise. I'm already trying to include as many lisp concepts in the test source as I can find to make sure lisp.js is a general purpose lisp. In that respect I'm setting several goals: get lisp-in-lisp working, get Let Over Lambda examples working, ... any other useful lisp constructs that I come across. That's a journey I can only make if I do my own lisp implementation.

tlrobinson 16y ago

So you mean "Lisp for JavaScript"? Why limit it to node.js?

devijvers 16y ago
It might one day become a general lisp for javascript but for now the concurrency-free server-side environment of node.js is assumed.

DTrejo 16y ago

lisp.js will be a lisp-2 (like Common Lisp and unlike Clojure and Scheme) Can you talk more about this decision?

devijvers 16y ago
Macros are important, that's why I seeking guidance in Let Over Lambda. The most suitable lisps for writing macros are lisp-2, hence the preference for a lisp-2 dual namespace.
- pwpwp 16y ago
  Lisp-2's have less potential for trouble with unhygienic macros, but they still have it. Hygienic macro systems work with Lisp-1 as well as with Lisp-2. Apropos, I've written a Lisp->JS compiler a while ago; you might find some of the code useful: http://github.com/manuel/cyberlisp/ http://github.com/manuel/cyberlisp/
  - DTrejo 16y ago
    You might also check out the Racket->JS compiler: http://planet.plt-scheme.org/display.ss?package=moby.plt&owner=dyoo http://planet.plt-scheme.org/display.ss?package=moby.plt&#38... I used it just last year — the developer, dyoo, is super responsive. I'm sure he'd be happy to talk shop with you: http://github.com/dyoo/moby-scheme http://github.com/dyoo/moby-scheme
  - sedachv 16y ago
    In regards to hygienic macros and Hoyte's Let Over Lambda, the type of variable capture hygiene is designed to prevent is actually used as a programming technique in the book; Hoyte calls it free variable injection. Lisp-1 vs Lisp-2 actually has very little to do with macros. Most of the inconvenience comes when writing functions - things like having to name list parameters "lst." The way Common Lisp gets around the variable capture problem is to forbid the rebinding of special forms, functions, and macros defined in the standard.* This is an ugly hack that forces early binding, and obviously does nothing to prevent capture of user-defined functions. The Scheme wiki has a good article on hygiene: http://community.schemewiki.org/?hygiene-versus-gensym http://community.schemewiki.org/?hygiene-versus-gensym * - This wasn't done for the sake of macros. The rationale behind the decision as I heard it is to prevent the hypothetical scenario of some standard function like cdr that's called inside the garbage collector or allocator from being rebound to something that tries to allocate memory and crashes the gc or sends the allocator into an infinite loop. I don't really buy it; it's easy enough to prevent these types of scenarios.