10 ms·

Show HN: A Common Lisp implementation in development, supports ASDF

96 points by andreamonaco 1y ago

Implementation of the standard is still not complete, but breakpoints and stepping work quite well! It also has some support for watchpoints, that no implementation has. Now it ships with ASDF and is capable of loading systems! Let me know if you like it. Support on Patreon or Liberapay is much appreciated

WalterGR 1y ago
> a debugger with stepping, a feature that most free CL implementations lack. I think most free CL implementations have a stepper. Which ones do not?
- andreamonaco 1y ago
  I tried stepping in various free implementations, but I couldn't really follow the source forms and execute them one by one. Also, I couldn't find much information online. Maybe your experience is different?
  - WalterGR 1y ago
    I haven’t used CL recently so I can’t speak from experience. But it looks like: CMU CL, SBCL, and LispWorks have steppers. Clozure does not. (Edit: an answer on https://stackoverflow.com/questions/37754935/what-are-efficient-ways-to-debug-common-lisp-in-emacs-and-slime https://stackoverflow.com/questions/37754935/what-are-effici... suggests it does...) As I understand it, those are the big 4. Clisp, ABCL, and Allegro also appear to have steppers. Always cool to see a new implementation, though!
    - klibertp 1y ago
      In most of those implementations (certainly in SBCL) it's either you break or step; you can't start stepping from a breakpoint. SBCL got some support for that this year, see https://news.ycombinator.com/item?id=43791709 https://news.ycombinator.com/item?id=43791709. It, however, doesn't allow stepping into any functions called after the break. Also, the compilers are allowed to make the code unsteppable in some cases, depending on optimization declaration: generally, debug needs to be >=2 and > speed/compilation-speed/space. In some circumstances, you land in decompiled/macroexpanded code, which is also quite unhelpful. Anyway, it's not that source-level stepping isn't there at all, it's just quirky and somewhat inconvenient. A fresh implementation that does comparatively little optimization and is byte-code based can probably support debuggers better. I hope such support won't go away later when the native code compiler is implemented.
      - WalterGR 1y ago
        Thanks! If I recall correctly, there are macros to control the level of code optimization? And some implementations can turn it off entirely for interactive use? Or am I off-base?
        klibertp 1y ago
        > If I recall correctly, there are macros to control the level of code optimization? Yup, you can either `(proclaim (optimize (debug 3) (speed 1)))` somewhere, which will take effect globally, or you can `(declare (optimize ...))` inside a particular function. It sounds great in theory - and it is great, in some respects - but this granularity makes it harder to ensure all interesting code is steppable when you need it.
eadmund 1y ago
Congratulations! Always good to see another Lisp in the world. Have you thought about writing up your experience?
- andreamonaco 1y ago
  Thanks! Maybe I could do that, if I see that people are interested
- andreamonaco 1y ago
  Also, my Patreon page (https://www.patreon.com/andreamonaco https://www.patreon.com/andreamonaco) has behind-the-scenes posts, some even in the free tier
neonscribe 1y ago
Do you have a goal in mind for this project?
- andreamonaco 1y ago
  Ideally I'd reach ANSI compliance, first with a bytecode compiler and then with a full one
  - neonscribe 1y ago
    Is there some important shortcoming of all the existing Common Lisp implementations that you would like to correct?
    - ngcc_hk 1y ago
      Awaiting answers. Seems stepping is one. Btw, I stick to sbcl as I used vim and so far the script here works for me. Might try this when back to do lisp. https://susam.net/lisp-in-vim.html https://susam.net/lisp-in-vim.html
      - andreamonaco 1y ago
        Yeah, advanced debugging features like watchpoints are very important to me
kazinator 1y ago
[flagged]
- andreamonaco 1y ago
  I'm not sure I understand. What I do in my project is very common practice: generated files (like the configure script) are not part of the repository, but they are put in released tarballs
  - sspiff 1y ago
    This is true and standard for (really) old projects, and dealing with this scripts and their problems used to be the bane of my existence 10 years ago. But I can't say I've encountered any such projects in the last 5 or so years. Either they use a modern programming language (which typically has an included build system, like rust's cargo or simply go build) of they use simple Makefiles. For C/C++ codebases, it seems like CMake has become the dominant build system. All of these are typically better than what GNU autoconf offers, with modern modern features and equally or better flexibility to deal with differences between operating systems, distributions, and/or optional or alternative libraries. I don't really see why anyone would pick autoconf for a modern project.
    - varjag 1y ago
      CMake is really more of a C++ crowd thing, it never won the mindshare with C. > I don't really see why anyone would pick autoconf for a modern project. If you build for your system only and never ever plan to cross compile by all means go with static makefile.
      - fc417fc802 1y ago
        A few of my personal projects cross compile via static makefile. Is there something wrong with that?
        deleted 1y ago
danilor 1y ago
Hey I'm curious as why you chose nongnu to host your project instead of github/gitlab! I don't know much about it, hence my curiosity ;)
- badmonster 1y ago
  +1
- kazinator 1y ago
  I hosted the TXR git on nongnu first, starting at around late 2009 or early 2010 maybe? I abandoned that when I discovered there's no control. I seem to recall having to wait like over a week for someone to enable non-fast-forward pushes. Overly strict and understaffed. I opted for self hosting. I kept the project web page there, though.
- andreamonaco 1y ago
  I don't like sites with heavy Javascript, especially if it's non-free. (Though recently I started using Github for a different project.) Savannah is very basic, perhaps too much, but it's okay for my project.
- volemo 1y ago
  Tbh, this is the first time I see nongnu.org used for something other than Emacs packages (I know that’s on me), so much so that I even thought this was a solution to substitute Emacs Lisp with Common Lisp. :O
badmonster 1y ago
Does alisp plan to eventually support full compilation to native code, or will it mainly stay an interpreter with limited compilation?
- andreamonaco 1y ago
  Yeah, the goal is first bytecode compilation and then full
valorzard 1y ago
I would be excited to use but since it’s using GPLv3 I can’t actually use it for a lot of projects I’d want to make ;-; Is it possible to relicense to LGPL or MPL instead?
- kstrauser 1y ago
  In general, people who license something as GPLv3 probably consider that a feature, not a bug. I mentioned here recently that I released a personal project under the GPLv3. The very first issue someone filed in GitHub was to ask me to relicense it as something more business friendly. I don't think I've been so offended by an issue before. If I'm writing something for fun, I could not possibly be less interested in helping else someone monetize my work. They can play by Free Software rules, or they can write their own version for themselves and license it however they want. I don't owe them the freedom to make it un-Free. The fact that this is hosted on a FSF-managed service indicates the author likely sees it similarly.
  - fc417fc802 1y ago
    I generally agree but it's worth noting that languages are a bit different. Obviously there are GPL'd compilers but those often make an explicit carveout for things like the runtime and standard library. Meanwhile in the Lisp world my impression is that most (but certainly not all) implementations are permissively licensed in part due to concerns that shipping an image file is essentially shipping the entire language implementation verbatim.
    - kstrauser 1y ago
      That totally makes sense and I do appreciate why that would be a problem for some users. And yet, this is a single-user labor of love by one person hosting it on FSF’s servers. I don't know them, and this is pure conjecture, but I suspect they probably couldn't care less if that made it challenging for commercial users. There are plenty of other Lisps for them to choose from.
    - pjmlp 1y ago
whartung 1y ago
At what point does a CL implementation need to be before it can start hoovering up the available library code from other implementations (license permitting). How many LOOP macros does the community need, particularly when bootstrapping an implementation, as an example. Similarly with, arguably 70-80% of the runtime. The CL spec is dominated by the large library which, ideally, should be mostly portable CL, at least I would think.
- klibertp 1y ago
  You're not the first one to think so: https://github.com/robert-strandh/SICL https://github.com/robert-strandh/SICL I'm unsure how complete it is, but it seems to cover much of the standard.
  - v9v 1y ago
    Here's a recently-written summary of the different libraries in SICL (including each library's purpose and status) http://metamodular.com/SICL-related-libraries/sicl-related-libraries.html http://metamodular.com/SICL-related-libraries/sicl-related-l...
- g2963 1y ago
  a hypothetical portable layer exists, but it starts diverging once deployed, because of cleanups, refactoring, or implementation specific hacks. LOOP is a great example, because all loop is just MIT LOOP version 829, originally cleaned up by burke. but nobody can resist deploying their personal architectural touch, so while the basic framework of loop remains identical across impelementations, there's superficial refactoring done by pretty much everyone. if you take SBCL and franz lisp as state of the art in free software and commercial respectivaly, they have equally solid improvements on original loop, that actually produce incompatible behavior in underspecified corners of spec. respective developer communities are very defensive about their incompatible behavior being the correct behavior of course. beach's SICL from sibling comment is the xkcd joke about standards "20 standards? we need a new standard the unifies them all! -- now we have 21 standards" LOOP in this case is a very simple example, but for example CLOS was originally implemented on top of PCL, Portable CommonLoops, an interlisp system, that was massaged into being compliant CLOS over years. for example sbcl uses a ship of theseus PCL, but franz lisp did from scratch rewrite. the hypothetical portability of that layer is significantly trickier than LOOP since clos is is deeply tied to the type system, and the boundary between some hypothetical base system common lisp and its clos layer becomes complicated during system bootstrapping. but that's not all! of course clos has to be deeply tied to the compiler, the type system, all kinds of things, to provide optimizations. discovering the appropriate slicing boundary is difficult to say the least.
CleverLikeAnOx 1y ago
What is ASDF?
- alexjplant 1y ago
  No idea why you're being downvoted for asking a simple question about an acronym. From Wikipedia [1]: > ASDF (Another System Definition Facility) is a package format and a build tool for Common Lisp libraries. It is analogous to tools such as Make and Ant. Contemporary developers using more mainstream languages are likely more familiar with asdf [2], the "Multiple Runtime Version Manager". [1] https://en.wikipedia.org/wiki/Another_System_Definition_Facility https://en.wikipedia.org/wiki/Another_System_Definition_Faci... [2] https://asdf-vm.com/ https://asdf-vm.com/
- mikedelago 1y ago
  ASDF - another system definition facility - is the de facto standard build system for common lisp. https://asdf.common-lisp.dev/ https://asdf.common-lisp.dev/ In common lisp, you don't need a build system at all; you can `(load "file.lisp")` everything and it should generally just work. But of course, build systems are useful tools, so nonetheless ASDF exists and it's nice enough to the degree that nobody has built a better and more widespread common lisp build system. Some good trivial examples are in the lisp cookbook: https://lispcookbook.github.io/cl-cookbook/systems.html https://lispcookbook.github.io/cl-cookbook/systems.html
nexo-v1 1y ago
Out of curiosity, what influenced your decision to prioritize stepping and watchpoints early? Was it driven more by debugging your own compiler code, or anticipating end-user needs
- andreamonaco 1y ago
  I'd say the latter, since I felt the need for debugging tools similar to other languages like C. Watchpoints for example are entirely non-existent in current implementations, as far as I know

96 points by andreamonaco 1y ago

WalterGR 1y ago

> a debugger with stepping, a feature that most free CL implementations lack. I think most free CL implementations have a stepper. Which ones do not?

andreamonaco 1y ago
I tried stepping in various free implementations, but I couldn't really follow the source forms and execute them one by one. Also, I couldn't find much information online. Maybe your experience is different?
- WalterGR 1y ago
  I haven’t used CL recently so I can’t speak from experience. But it looks like: CMU CL, SBCL, and LispWorks have steppers. Clozure does not. (Edit: an answer on https://stackoverflow.com/questions/37754935/what-are-efficient-ways-to-debug-common-lisp-in-emacs-and-slime https://stackoverflow.com/questions/37754935/what-are-effici... suggests it does...) As I understand it, those are the big 4. Clisp, ABCL, and Allegro also appear to have steppers. Always cool to see a new implementation, though!
  - klibertp 1y ago
    In most of those implementations (certainly in SBCL) it's either you break or step; you can't start stepping from a breakpoint. SBCL got some support for that this year, see https://news.ycombinator.com/item?id=43791709 https://news.ycombinator.com/item?id=43791709. It, however, doesn't allow stepping into any functions called after the break. Also, the compilers are allowed to make the code unsteppable in some cases, depending on optimization declaration: generally, debug needs to be >=2 and > speed/compilation-speed/space. In some circumstances, you land in decompiled/macroexpanded code, which is also quite unhelpful. Anyway, it's not that source-level stepping isn't there at all, it's just quirky and somewhat inconvenient. A fresh implementation that does comparatively little optimization and is byte-code based can probably support debuggers better. I hope such support won't go away later when the native code compiler is implemented.
    - WalterGR 1y ago
      Thanks! If I recall correctly, there are macros to control the level of code optimization? And some implementations can turn it off entirely for interactive use? Or am I off-base?
      - klibertp 1y ago
        > If I recall correctly, there are macros to control the level of code optimization? Yup, you can either `(proclaim (optimize (debug 3) (speed 1)))` somewhere, which will take effect globally, or you can `(declare (optimize ...))` inside a particular function. It sounds great in theory - and it is great, in some respects - but this granularity makes it harder to ensure all interesting code is steppable when you need it.

eadmund 1y ago

Congratulations! Always good to see another Lisp in the world. Have you thought about writing up your experience?

andreamonaco 1y ago
Thanks! Maybe I could do that, if I see that people are interested
andreamonaco 1y ago
Also, my Patreon page (https://www.patreon.com/andreamonaco https://www.patreon.com/andreamonaco) has behind-the-scenes posts, some even in the free tier

neonscribe 1y ago

Do you have a goal in mind for this project?

andreamonaco 1y ago
Ideally I'd reach ANSI compliance, first with a bytecode compiler and then with a full one
- neonscribe 1y ago
  Is there some important shortcoming of all the existing Common Lisp implementations that you would like to correct?
  - ngcc_hk 1y ago
    Awaiting answers. Seems stepping is one. Btw, I stick to sbcl as I used vim and so far the script here works for me. Might try this when back to do lisp. https://susam.net/lisp-in-vim.html https://susam.net/lisp-in-vim.html
    - andreamonaco 1y ago
      Yeah, advanced debugging features like watchpoints are very important to me

kazinator 1y ago

[flagged]

andreamonaco 1y ago
I'm not sure I understand. What I do in my project is very common practice: generated files (like the configure script) are not part of the repository, but they are put in released tarballs
- sspiff 1y ago
  This is true and standard for (really) old projects, and dealing with this scripts and their problems used to be the bane of my existence 10 years ago. But I can't say I've encountered any such projects in the last 5 or so years. Either they use a modern programming language (which typically has an included build system, like rust's cargo or simply go build) of they use simple Makefiles. For C/C++ codebases, it seems like CMake has become the dominant build system. All of these are typically better than what GNU autoconf offers, with modern modern features and equally or better flexibility to deal with differences between operating systems, distributions, and/or optional or alternative libraries. I don't really see why anyone would pick autoconf for a modern project.
  - varjag 1y ago
    CMake is really more of a C++ crowd thing, it never won the mindshare with C. > I don't really see why anyone would pick autoconf for a modern project. If you build for your system only and never ever plan to cross compile by all means go with static makefile.
    - fc417fc802 1y ago
      A few of my personal projects cross compile via static makefile. Is there something wrong with that?
      - deleted 1y ago

danilor 1y ago

Hey I'm curious as why you chose nongnu to host your project instead of github/gitlab! I don't know much about it, hence my curiosity ;)

badmonster 1y ago
+1
kazinator 1y ago
I hosted the TXR git on nongnu first, starting at around late 2009 or early 2010 maybe? I abandoned that when I discovered there's no control. I seem to recall having to wait like over a week for someone to enable non-fast-forward pushes. Overly strict and understaffed. I opted for self hosting. I kept the project web page there, though.
andreamonaco 1y ago
I don't like sites with heavy Javascript, especially if it's non-free. (Though recently I started using Github for a different project.) Savannah is very basic, perhaps too much, but it's okay for my project.
volemo 1y ago
Tbh, this is the first time I see nongnu.org used for something other than Emacs packages (I know that’s on me), so much so that I even thought this was a solution to substitute Emacs Lisp with Common Lisp. :O

badmonster 1y ago

Does alisp plan to eventually support full compilation to native code, or will it mainly stay an interpreter with limited compilation?

andreamonaco 1y ago
Yeah, the goal is first bytecode compilation and then full

valorzard 1y ago

I would be excited to use but since it’s using GPLv3 I can’t actually use it for a lot of projects I’d want to make ;-; Is it possible to relicense to LGPL or MPL instead?

kstrauser 1y ago
In general, people who license something as GPLv3 probably consider that a feature, not a bug. I mentioned here recently that I released a personal project under the GPLv3. The very first issue someone filed in GitHub was to ask me to relicense it as something more business friendly. I don't think I've been so offended by an issue before. If I'm writing something for fun, I could not possibly be less interested in helping else someone monetize my work. They can play by Free Software rules, or they can write their own version for themselves and license it however they want. I don't owe them the freedom to make it un-Free. The fact that this is hosted on a FSF-managed service indicates the author likely sees it similarly.
- fc417fc802 1y ago
  I generally agree but it's worth noting that languages are a bit different. Obviously there are GPL'd compilers but those often make an explicit carveout for things like the runtime and standard library. Meanwhile in the Lisp world my impression is that most (but certainly not all) implementations are permissively licensed in part due to concerns that shipping an image file is essentially shipping the entire language implementation verbatim.
  - kstrauser 1y ago
    That totally makes sense and I do appreciate why that would be a problem for some users. And yet, this is a single-user labor of love by one person hosting it on FSF’s servers. I don't know them, and this is pure conjecture, but I suspect they probably couldn't care less if that made it challenging for commercial users. There are plenty of other Lisps for them to choose from.
  - pjmlp 1y ago

whartung 1y ago

At what point does a CL implementation need to be before it can start hoovering up the available library code from other implementations (license permitting). How many LOOP macros does the community need, particularly when bootstrapping an implementation, as an example. Similarly with, arguably 70-80% of the runtime. The CL spec is dominated by the large library which, ideally, should be mostly portable CL, at least I would think.

klibertp 1y ago
You're not the first one to think so: https://github.com/robert-strandh/SICL https://github.com/robert-strandh/SICL I'm unsure how complete it is, but it seems to cover much of the standard.
- v9v 1y ago
  Here's a recently-written summary of the different libraries in SICL (including each library's purpose and status) http://metamodular.com/SICL-related-libraries/sicl-related-libraries.html http://metamodular.com/SICL-related-libraries/sicl-related-l...
g2963 1y ago
a hypothetical portable layer exists, but it starts diverging once deployed, because of cleanups, refactoring, or implementation specific hacks. LOOP is a great example, because all loop is just MIT LOOP version 829, originally cleaned up by burke. but nobody can resist deploying their personal architectural touch, so while the basic framework of loop remains identical across impelementations, there's superficial refactoring done by pretty much everyone. if you take SBCL and franz lisp as state of the art in free software and commercial respectivaly, they have equally solid improvements on original loop, that actually produce incompatible behavior in underspecified corners of spec. respective developer communities are very defensive about their incompatible behavior being the correct behavior of course. beach's SICL from sibling comment is the xkcd joke about standards "20 standards? we need a new standard the unifies them all! -- now we have 21 standards" LOOP in this case is a very simple example, but for example CLOS was originally implemented on top of PCL, Portable CommonLoops, an interlisp system, that was massaged into being compliant CLOS over years. for example sbcl uses a ship of theseus PCL, but franz lisp did from scratch rewrite. the hypothetical portability of that layer is significantly trickier than LOOP since clos is is deeply tied to the type system, and the boundary between some hypothetical base system common lisp and its clos layer becomes complicated during system bootstrapping. but that's not all! of course clos has to be deeply tied to the compiler, the type system, all kinds of things, to provide optimizations. discovering the appropriate slicing boundary is difficult to say the least.

CleverLikeAnOx 1y ago

What is ASDF?

alexjplant 1y ago
No idea why you're being downvoted for asking a simple question about an acronym. From Wikipedia [1]: > ASDF (Another System Definition Facility) is a package format and a build tool for Common Lisp libraries. It is analogous to tools such as Make and Ant. Contemporary developers using more mainstream languages are likely more familiar with asdf [2], the "Multiple Runtime Version Manager". [1] https://en.wikipedia.org/wiki/Another_System_Definition_Facility https://en.wikipedia.org/wiki/Another_System_Definition_Faci... [2] https://asdf-vm.com/ https://asdf-vm.com/
mikedelago 1y ago
ASDF - another system definition facility - is the de facto standard build system for common lisp. https://asdf.common-lisp.dev/ https://asdf.common-lisp.dev/ In common lisp, you don't need a build system at all; you can `(load "file.lisp")` everything and it should generally just work. But of course, build systems are useful tools, so nonetheless ASDF exists and it's nice enough to the degree that nobody has built a better and more widespread common lisp build system. Some good trivial examples are in the lisp cookbook: https://lispcookbook.github.io/cl-cookbook/systems.html https://lispcookbook.github.io/cl-cookbook/systems.html

nexo-v1 1y ago

Out of curiosity, what influenced your decision to prioritize stepping and watchpoints early? Was it driven more by debugging your own compiler code, or anticipating end-user needs

andreamonaco 1y ago
I'd say the latter, since I felt the need for debugging tools similar to other languages like C. Watchpoints for example are entirely non-existent in current implementations, as far as I know