5 ms·

LO – simple WASM native language

221 points by glebbash 2y ago

ledgerdev 2y ago
Any plans to use wasmgc?
- glebbash 2y ago
  I don't see a need for it. Maybe for some higher level scripting, but I don't have any plans for that.
  - azakai 2y ago
    A big advantage of WasmGC is it would allow your binaries to be smaller, since you wouldn't need to bundle an allocator for heap::alloc and free. Kotlin and Java binaries can in some cases be be much smaller than C++ and Rust for that reason.
    - glebbash 2y ago
      Alright, will take a look at it sometime. But the more interesting proposal for me right now is the component model.
- deleted 2y ago
  [deleted]
chubs 2y ago
This looks super impressive! Any chance you could explain how memory management works? Eg is it like rust in that sense? Or is it GC'ed ? Thanks
- glebbash 2y ago
  Absolutely. The memory management is manual like in C, but with addition of defers it is actually quite good to work with. Here is the source for the allocator: https://github.com/glebbash/LO/blob/81f96bb09e3a0b1455b32497da445ff5f71d2907/examples/lib/std.lo#L185 https://github.com/glebbash/LO/blob/81f96bb09e3a0b1455b32497... It also links to where I learned how to build the "proper" allocator from scratch.
glebbash 2y ago
Hi, it's my first time posting here, just wanted to share the language I was working on for some time now. I've been working on integrating it with WASM-4 runtime this past week. And with minimal changes to the compiler I created this demo game: https://rawcdn.githack.com/glebbash/LO/da8305293f5438967619e91c6ecfd472111ccf91/examples/test/demos/wasm4/builds/dark-maze.html https://rawcdn.githack.com/glebbash/LO/da8305293f5438967619e... Game source here: https://github.com/glebbash/LO/blob/main/examples/test/demos/wasm4/src/dark-maze.lo https://github.com/glebbash/LO/blob/main/examples/test/demos...
- JonChesterfield 2y ago
  Straight to wasm is a good choice. You could get something close to a native binary by packaging the result with wasm3 or similar.
- airstrike 2y ago
  you might want to consider changing the title to a "Show HN" https://news.ycombinator.com/show https://news.ycombinator.com/show
- Lerc 2y ago
  Do you have (or plan to support) a Wasm build of the compiler so you can compile from within the browser? Given you have includes I guess you'd need to provide a loadFile(filename) callback to provide it with data.
  - glebbash 2y ago
    Yes. Compiling single files is supported by calling the exported `compile` function, example setup here: https://github.com/glebbash/LO/blob/60b80d1b325136ce1ceaf4dc12073b7faaceb853/utils.mjs#L529 https://github.com/glebbash/LO/blob/60b80d1b325136ce1ceaf4dc.... For includes to work compiler uses WASI which can be implemented in the browser. LO playground and VSCode extension use this approach. Playground link: https://glebbash.deno.dev/lo-playground/ https://glebbash.deno.dev/lo-playground/ and source: https://github.com/glebbash/glebbash.dev/blob/main/static/lo-playground/index.html https://github.com/glebbash/glebbash.dev/blob/main/static/lo...
teucris 2y ago
So far I’m loving this. Would love a language tour doc!
- glebbash 2y ago
  Thanks, I will definitely do that once the feature set will get stable enough. Don't want to update docs constantly :) As for now, you can check the examples folder and explore the compiler sources, tried to keep everything simple.
j-pb 2y ago
Very cute project! I personally wouldn't call anything Rust based "simple" (I'm not trying to hate on Rust, it's my daily driver.), so I have to ask: Wouldn't be the simplest possible wasm language, a small macro system for WAT, that bootstraps a slightly more highlevel lisp dialect? Wasm already seems like an increadibly high-level language, given that it's a low-level compilation target.
- the_duke 2y ago
  I don't see how WASM is that high level. It doesn't have types except numbers (ignoring reference types and the GC proposal), which makes it lower level than LLVM IR. The control flow constructs are also much closer to SSA form than anything viable for humans. If you ever try to write WAT by hand you realize that it's very much a compilation target, not something you'd ever want to manually write.
  - j-pb 2y ago
    Unlike LLVM, control flow is actually structured, no arbitrary jumps, functions, tables, real harvard separation of code and data. The restriction to only number types, is more a byproduct of trying to keep the 1.0 spec as mvp as possible, and reference types are gonna be a big shift in that regard. I mean compare these two. One is a (very low level) language, the other is (high level) assembly. https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/if...else https://developer.mozilla.org/en-US/docs/WebAssembly/Referen... https://mapping-high-level-constructs-to-llvm-ir.readthedocs.io/en/latest/control-structures/if-then-else.html https://mapping-high-level-constructs-to-llvm-ir.readthedocs...
- glebbash 2y ago
  That was how it started actually, I wanted to make a superset of WAT. But then I switched to a more familiar C-like constructs and that gave me a big boost in productivity. I think the superset of WAT idea is very viable and someone more comfortable with s-expressions should explore that for sure.
  - 2y ago
csjh 2y ago
I like that you have a vscode extension already - I'm making a similar language project, and you really don't notice how much you miss even basic syntax highlighting until it's gone
- arijun 2y ago
  Doesn’t tree sitter make it pretty easy to add highlighting?
  - vlakreeh 2y ago
    For editors that support it, yeah. Sadly vscode doesn't natively support tree-sitter.
varispeed 2y ago
This is very nice! What is missing for me is the answer to question "Why?"
- glebbash 2y ago
  Mostly for recreational programming. And then for exploring what the languages with minimal infrastructure could look like.
marianoguerra 2y ago
If you would like to learn how to build your own then check https://wasmgroundup.com/ https://wasmgroundup.com/ Disclaimer: I'm the co-author
- thomasfromcdnjs 2y ago
  How is this project related to the work you do? e.g. instadeq, gloodata Why do you need WASM?
  - marianoguerra 2y ago
    it's not that related, I like to create compilers/transpilers/weird languages, see the "Language Projects" section here: https://marianoguerra.github.io/ https://marianoguerra.github.io/ But there's a relation, two years ago I prototyped a thing that tried to make a headless version of instadeq that could run in the browser and the server, the idea was a dataflow language where the data was represented as apache arrow tables where filters and transformation expressions where compiled to wasm. The prototype kind of worked and I had a lot of fun, I thought about sharing the fun by writing a small book about creating 10 toy languages that compiled to Wasm, I shared the idea with Patrick, he said he was thinking about something similar and the idea evolved into the current book.
  - csjh 2y ago
    In-browser data analysis is a great place to use Wasm. Javascript is great, but larger datasets really benefit from SIMD + tighter memory control/controlled memory layouts. Some interesting data-related Wasm projects: - DuckDB Wasm | https://github.com/duckdb/duckdb-wasm https://github.com/duckdb/duckdb-wasm - Row Zero | https://news.ycombinator.com/item?id=39551064 https://news.ycombinator.com/item?id=39551064 - PGLite | https://github.com/electric-sql/pglite https://github.com/electric-sql/pglite
    - thomasfromcdnjs 2y ago
      Thanks for this answer
kettlecorn 2y ago
Looks very cool! I can see myself using this for some Wasm experiments. Please add a license so it's clear how people can / can't use the project!
- glebbash 2y ago
  Yay a first user :) Added the license.
  - kettlecorn 2y ago
    Fantastic!
niutech 2y ago
How does LO compare with AssemblyScript?
- andrewfromx 2y ago
  AssemblyScript is very strange to me. The whole point of wasm is you can avoid javascript/typescript language and do stuff in a browser in go, rust, or now LO!
  - csjh 2y ago
    I wouldn't say that's the whole point of Wasm. Speed & efficiency are also key goals, which are what AssemblyScript play into, giving access to that speed potential in a format that's easy for web devs
    - eyelidlessness 2y ago
      And the similarities between AS and TS (or especially JS) are very superficial. They share some grammar, and utilize many of the same type system features, but AS is fundamentally a different language with fundamentally different semantics (which are much closer to WASM, and then much closer to those other compile-to-WASM choices).
chai2010 2y ago
Cool project . I am also designing a Wa language for Wasm (https://github.com/wa-lang/wa https://github.com/wa-lang/wa), we are currently improving support for the WebGPU library and LSP extension.
- glebbash 2y ago
  Cool project you too :) It's fun to see how the language of the first compiler affects a lot of the syntax of the languages we build.
SushiHippie 2y ago
FYI the VS Code extension Link in the README is broken, as it links to /lo-extension instead of /vscode-ext And is it intentional that the repository does not have a LICENSE file, or do you intend to open-source (by OSI definition) it?
- glebbash 2y ago
  Fixed the link, forgot that markdown references use double square brackets. Hmm, I haven't thought about it the license actually, I would default to MIT, but wanted to keep the number of files minimal in repo, hehe. Will figure it out asap, thanks.
gsuuon 2y ago
Neat project! I liked the dev log writeups (despite the multiple rickrolls), hope you keep doing these. It was interesting seeing your reasoning. I'm starting a for-fun language project that should compile to wasm and native, and was debating between wasm -> wasm2c or llvm -> emscripten. Your posts about ffi/bindings issues were helpful, I'm probably going to try wasm (or just wat) first and then binaryen for wasm -> wasm optimization.
kukiTsui 2y ago
Looks interesting! Always glad to see more WASM native languages coming. You may take a look at MoonBit(https://github.com/moonbitlang/moonbit-docs/tree/main https://github.com/moonbitlang/moonbit-docs/tree/main). The language is Rust-like with gc support, and we also have a full toolchain like cloud IDE, compiler, build system, and package manager.
cbellew 2y ago
Nice work. Out of curiosity, did you consider using pest (https://pest.rs/ https://pest.rs/) to help build your parser? Or is it too much for what you are doing?
- glebbash 2y ago
  Well the goal for this project is to go dependency free so, yep, it's not a good fit here.
xonix 2y ago
It's funny to watch how often new programming languages resemble visually (and conceptually) the language they are written in. You can put it the other way: just look at a syntax sample and try to guess the implementation language. I can see some problem here. Probably, it means the author is under heavy influence of the implementation language, that limits their thinking and creativity scope to the concepts of that language.
- aniviacat 2y ago
  I'm not sure if that is really a problem. A single individual is likely to miss a lot of edge cases that a larger organisation (like the Rust foundation) has thought of while creating the language. In that sense, I believe following the conventions created by a large and well-known entity is likely to produce better results.
- orphea 2y ago
  I was going to say: I bet this is written in Rust, they use https://docs.rs/syn/latest/syn/ https://docs.rs/syn/latest/syn/ to not write their own parser/lexer, and this is how you get Rust-looking languages... I'm surprised.
- glebbash 2y ago
  It's funny yeah but it makes sense and is not really a problem. You would write a compiler in the language that is the closest to your perfect. Trying to fix the original language by making a "clone" of it and adding/removing features to your liking. That is what I want to change here by making a very simple language that if someone would like they could just fork it and add/remove the needed features instead of building one from scratch. If this idea won't work then at least a very simple compiler would be a good learning resource, so a win win anyways.
  - xonix 2y ago
    Right, this is not the problem par se. To me this is a rather philosophical question. Related to the motivation behind the project. If someone has some particular domain problem to which they need to introduce the new programming language, then, probably, it should look somehow different to all existing languages, otherwise they would be used. Alternatively, the motivation could be: OK I need ExistingLanguageX(-alike) but in the domain/environment where it's not present yet, for example, WASM. The motivation question is always of interest to me since it allows to judje for the long-term perspectives of the project. However don't take this as a discouragement rant, learning case is a good motivation/raison d'être too.
charlietap 2y ago
If you run it in chasm you can run it absolutely everywhere: -JVM -Browser -Mobile -Desktop https://github.com/CharlieTap/chasm https://github.com/CharlieTap/chasm

221 points by glebbash 2y ago

ledgerdev 2y ago

Any plans to use wasmgc?

glebbash 2y ago
I don't see a need for it. Maybe for some higher level scripting, but I don't have any plans for that.
- azakai 2y ago
  A big advantage of WasmGC is it would allow your binaries to be smaller, since you wouldn't need to bundle an allocator for heap::alloc and free. Kotlin and Java binaries can in some cases be be much smaller than C++ and Rust for that reason.
  - glebbash 2y ago
    Alright, will take a look at it sometime. But the more interesting proposal for me right now is the component model.
deleted 2y ago
[deleted]

chubs 2y ago

This looks super impressive! Any chance you could explain how memory management works? Eg is it like rust in that sense? Or is it GC'ed ? Thanks

glebbash 2y ago
Absolutely. The memory management is manual like in C, but with addition of defers it is actually quite good to work with. Here is the source for the allocator: https://github.com/glebbash/LO/blob/81f96bb09e3a0b1455b32497da445ff5f71d2907/examples/lib/std.lo#L185 https://github.com/glebbash/LO/blob/81f96bb09e3a0b1455b32497... It also links to where I learned how to build the "proper" allocator from scratch.

glebbash 2y ago

Hi, it's my first time posting here, just wanted to share the language I was working on for some time now. I've been working on integrating it with WASM-4 runtime this past week. And with minimal changes to the compiler I created this demo game: https://rawcdn.githack.com/glebbash/LO/da8305293f5438967619e91c6ecfd472111ccf91/examples/test/demos/wasm4/builds/dark-maze.html https://rawcdn.githack.com/glebbash/LO/da8305293f5438967619e... Game source here: https://github.com/glebbash/LO/blob/main/examples/test/demos/wasm4/src/dark-maze.lo https://github.com/glebbash/LO/blob/main/examples/test/demos...

JonChesterfield 2y ago
Straight to wasm is a good choice. You could get something close to a native binary by packaging the result with wasm3 or similar.
airstrike 2y ago
you might want to consider changing the title to a "Show HN" https://news.ycombinator.com/show https://news.ycombinator.com/show
Lerc 2y ago
Do you have (or plan to support) a Wasm build of the compiler so you can compile from within the browser? Given you have includes I guess you'd need to provide a loadFile(filename) callback to provide it with data.
- glebbash 2y ago
  Yes. Compiling single files is supported by calling the exported `compile` function, example setup here: https://github.com/glebbash/LO/blob/60b80d1b325136ce1ceaf4dc12073b7faaceb853/utils.mjs#L529 https://github.com/glebbash/LO/blob/60b80d1b325136ce1ceaf4dc.... For includes to work compiler uses WASI which can be implemented in the browser. LO playground and VSCode extension use this approach. Playground link: https://glebbash.deno.dev/lo-playground/ https://glebbash.deno.dev/lo-playground/ and source: https://github.com/glebbash/glebbash.dev/blob/main/static/lo-playground/index.html https://github.com/glebbash/glebbash.dev/blob/main/static/lo...

teucris 2y ago

So far I’m loving this. Would love a language tour doc!

glebbash 2y ago
Thanks, I will definitely do that once the feature set will get stable enough. Don't want to update docs constantly :) As for now, you can check the examples folder and explore the compiler sources, tried to keep everything simple.

j-pb 2y ago

Very cute project! I personally wouldn't call anything Rust based "simple" (I'm not trying to hate on Rust, it's my daily driver.), so I have to ask: Wouldn't be the simplest possible wasm language, a small macro system for WAT, that bootstraps a slightly more highlevel lisp dialect? Wasm already seems like an increadibly high-level language, given that it's a low-level compilation target.

the_duke 2y ago
I don't see how WASM is that high level. It doesn't have types except numbers (ignoring reference types and the GC proposal), which makes it lower level than LLVM IR. The control flow constructs are also much closer to SSA form than anything viable for humans. If you ever try to write WAT by hand you realize that it's very much a compilation target, not something you'd ever want to manually write.
- j-pb 2y ago
  Unlike LLVM, control flow is actually structured, no arbitrary jumps, functions, tables, real harvard separation of code and data. The restriction to only number types, is more a byproduct of trying to keep the 1.0 spec as mvp as possible, and reference types are gonna be a big shift in that regard. I mean compare these two. One is a (very low level) language, the other is (high level) assembly. https://developer.mozilla.org/en-US/docs/WebAssembly/Reference/Control_flow/if...else https://developer.mozilla.org/en-US/docs/WebAssembly/Referen... https://mapping-high-level-constructs-to-llvm-ir.readthedocs.io/en/latest/control-structures/if-then-else.html https://mapping-high-level-constructs-to-llvm-ir.readthedocs...
glebbash 2y ago
That was how it started actually, I wanted to make a superset of WAT. But then I switched to a more familiar C-like constructs and that gave me a big boost in productivity. I think the superset of WAT idea is very viable and someone more comfortable with s-expressions should explore that for sure.
- 2y ago

csjh 2y ago

I like that you have a vscode extension already - I'm making a similar language project, and you really don't notice how much you miss even basic syntax highlighting until it's gone

arijun 2y ago
Doesn’t tree sitter make it pretty easy to add highlighting?
- vlakreeh 2y ago
  For editors that support it, yeah. Sadly vscode doesn't natively support tree-sitter.

varispeed 2y ago

This is very nice! What is missing for me is the answer to question "Why?"

glebbash 2y ago
Mostly for recreational programming. And then for exploring what the languages with minimal infrastructure could look like.

marianoguerra 2y ago

If you would like to learn how to build your own then check https://wasmgroundup.com/ https://wasmgroundup.com/ Disclaimer: I'm the co-author

thomasfromcdnjs 2y ago
How is this project related to the work you do? e.g. instadeq, gloodata Why do you need WASM?
- marianoguerra 2y ago
  it's not that related, I like to create compilers/transpilers/weird languages, see the "Language Projects" section here: https://marianoguerra.github.io/ https://marianoguerra.github.io/ But there's a relation, two years ago I prototyped a thing that tried to make a headless version of instadeq that could run in the browser and the server, the idea was a dataflow language where the data was represented as apache arrow tables where filters and transformation expressions where compiled to wasm. The prototype kind of worked and I had a lot of fun, I thought about sharing the fun by writing a small book about creating 10 toy languages that compiled to Wasm, I shared the idea with Patrick, he said he was thinking about something similar and the idea evolved into the current book.
- csjh 2y ago
  In-browser data analysis is a great place to use Wasm. Javascript is great, but larger datasets really benefit from SIMD + tighter memory control/controlled memory layouts. Some interesting data-related Wasm projects: - DuckDB Wasm | https://github.com/duckdb/duckdb-wasm https://github.com/duckdb/duckdb-wasm - Row Zero | https://news.ycombinator.com/item?id=39551064 https://news.ycombinator.com/item?id=39551064 - PGLite | https://github.com/electric-sql/pglite https://github.com/electric-sql/pglite
  - thomasfromcdnjs 2y ago
    Thanks for this answer

kettlecorn 2y ago

Looks very cool! I can see myself using this for some Wasm experiments. Please add a license so it's clear how people can / can't use the project!

glebbash 2y ago
Yay a first user :) Added the license.
- kettlecorn 2y ago
  Fantastic!

niutech 2y ago

How does LO compare with AssemblyScript?

andrewfromx 2y ago
AssemblyScript is very strange to me. The whole point of wasm is you can avoid javascript/typescript language and do stuff in a browser in go, rust, or now LO!
- csjh 2y ago
  I wouldn't say that's the whole point of Wasm. Speed & efficiency are also key goals, which are what AssemblyScript play into, giving access to that speed potential in a format that's easy for web devs
  - eyelidlessness 2y ago
    And the similarities between AS and TS (or especially JS) are very superficial. They share some grammar, and utilize many of the same type system features, but AS is fundamentally a different language with fundamentally different semantics (which are much closer to WASM, and then much closer to those other compile-to-WASM choices).

chai2010 2y ago

Cool project . I am also designing a Wa language for Wasm (https://github.com/wa-lang/wa https://github.com/wa-lang/wa), we are currently improving support for the WebGPU library and LSP extension.

glebbash 2y ago
Cool project you too :) It's fun to see how the language of the first compiler affects a lot of the syntax of the languages we build.

SushiHippie 2y ago

FYI the VS Code extension Link in the README is broken, as it links to /lo-extension instead of /vscode-ext And is it intentional that the repository does not have a LICENSE file, or do you intend to open-source (by OSI definition) it?

glebbash 2y ago
Fixed the link, forgot that markdown references use double square brackets. Hmm, I haven't thought about it the license actually, I would default to MIT, but wanted to keep the number of files minimal in repo, hehe. Will figure it out asap, thanks.

gsuuon 2y ago

Neat project! I liked the dev log writeups (despite the multiple rickrolls), hope you keep doing these. It was interesting seeing your reasoning. I'm starting a for-fun language project that should compile to wasm and native, and was debating between wasm -> wasm2c or llvm -> emscripten. Your posts about ffi/bindings issues were helpful, I'm probably going to try wasm (or just wat) first and then binaryen for wasm -> wasm optimization.

kukiTsui 2y ago

Looks interesting! Always glad to see more WASM native languages coming. You may take a look at MoonBit(https://github.com/moonbitlang/moonbit-docs/tree/main https://github.com/moonbitlang/moonbit-docs/tree/main). The language is Rust-like with gc support, and we also have a full toolchain like cloud IDE, compiler, build system, and package manager.

cbellew 2y ago

Nice work. Out of curiosity, did you consider using pest (https://pest.rs/ https://pest.rs/) to help build your parser? Or is it too much for what you are doing?

glebbash 2y ago
Well the goal for this project is to go dependency free so, yep, it's not a good fit here.

xonix 2y ago

It's funny to watch how often new programming languages resemble visually (and conceptually) the language they are written in. You can put it the other way: just look at a syntax sample and try to guess the implementation language. I can see some problem here. Probably, it means the author is under heavy influence of the implementation language, that limits their thinking and creativity scope to the concepts of that language.

aniviacat 2y ago
I'm not sure if that is really a problem. A single individual is likely to miss a lot of edge cases that a larger organisation (like the Rust foundation) has thought of while creating the language. In that sense, I believe following the conventions created by a large and well-known entity is likely to produce better results.
orphea 2y ago
I was going to say: I bet this is written in Rust, they use https://docs.rs/syn/latest/syn/ https://docs.rs/syn/latest/syn/ to not write their own parser/lexer, and this is how you get Rust-looking languages... I'm surprised.
glebbash 2y ago
It's funny yeah but it makes sense and is not really a problem. You would write a compiler in the language that is the closest to your perfect. Trying to fix the original language by making a "clone" of it and adding/removing features to your liking. That is what I want to change here by making a very simple language that if someone would like they could just fork it and add/remove the needed features instead of building one from scratch. If this idea won't work then at least a very simple compiler would be a good learning resource, so a win win anyways.
- xonix 2y ago
  Right, this is not the problem par se. To me this is a rather philosophical question. Related to the motivation behind the project. If someone has some particular domain problem to which they need to introduce the new programming language, then, probably, it should look somehow different to all existing languages, otherwise they would be used. Alternatively, the motivation could be: OK I need ExistingLanguageX(-alike) but in the domain/environment where it's not present yet, for example, WASM. The motivation question is always of interest to me since it allows to judje for the long-term perspectives of the project. However don't take this as a discouragement rant, learning case is a good motivation/raison d'être too.

charlietap 2y ago

If you run it in chasm you can run it absolutely everywhere: -JVM -Browser -Mobile -Desktop https://github.com/CharlieTap/chasm https://github.com/CharlieTap/chasm