11 ms·
Org-mode parser in Rust
- kibwen 7y agoThe author is in the following thread if anyone would like to ask questions: https://www.reddit.com/r/rust/comments/bcz7d4/orgrs_org_parser_rewrite_in_rust/ https://www.reddit.com/r/rust/comments/bcz7d4/orgrs_org_pars...
- deleted 7y ago[deleted]
- peatmoss 7y agoI love org-mode as a format and as a literate / notebook programming environment. And I love emacs. But, I think it’s great that more people are looking at org-mode outside emacs. I’d love to see alternative tool chains that push the capabilities and open more people to its magic.
- bluejekyll 7y agoAs someone who only reads about the love of Org, could you breakdown what features you see as improvements over markdown, specially CommonMark? I often see Org and OrgMode conflated, where OrgMode seems to provide much more of an experience with the way Emacs has integrated it into the editor. Could markdown be as integrated and provide similar experience?
- jsilence 7y agoOrg offers much more functionality than MD. Tags, TODOs with start dates and deadlines, an agenda view, capture templates, and a jupyter like notebook format with org-babel that allows for selectively integrating live code snippets into org documents. Been using it for >2 years now and for sure only using a fraction of its capacity.
- dan-robertson 7y agoI think not having to balance headline length with underline length is a good plus. Also not having to memorise the hierarchy of underline characters is a good plus. The format seems better suited to extension too with ways to have annotations or blocks
- Stratoscope 7y agoYou can do headings in Markdown like org mode, just use # instead of *: # Level 1 ## Level 2 ### Level 3 #### Level 4 etc. I rarely see the fiddly format in the wild, probably for the reasons you mentioned, and because it only allows H1 and H2: Level 1 ======= Level 2 ------- Of course there are lots of things org mode does better than Markdown, I'm just addressing the specific point you mentioned.
- juki 7y agoThe big difference between Markdown and the Org-markup is that the latter has syntax for various kinds of metadata, while Markdown is purely content. That metadata is used for things like task management, literate programming, spreadsheets, customizing export behavior, etc. One could, in theory, create an extended Markdown flavor that allows similar metadata, but that wouldn't be compatible with any existing tools anyway, so there's not much point.
- indentit 7y agoPretty sure Markdown with metadata (sometimes referred to as "front matter") already exists and is called MultiMarkdown, no?
- juki 7y agoBy "various kinds of metadata", I don't just mean the document metadata (such as title, author, etc.), but also the metadata applied to specific elements such as the outline items, tables or code blocks. For example, in Org you can add task management data, scheduling information and arbitrary properties to the outline, #+COLUMNS: %ITEM %TODO %ASSIGNED_TO %SCHEDULED %PRIORITY #+PROPERTY: ASSIGNED_TO_ALL juki some other people * TODO [#A] Some task with priority A SCHEDULED: <2019-04-20> :PROPERTIES: :ASSIGNED_TO: juki :END: Some description. This defines a task with a custom property `ASSIGNED_TO` whose value can be one of ["juki", "some", "other", "people"], which can be viewed and edited in a column view. It could also be used by a custom export-backend or some other tool. For another example, you can add formulas to a table, #+CAPTION: A table with formulas. | foo | bar | Row sum | |-----+-----+---------| | 10 | 20 | 30 | | 30 | 40 | 70 | |-----+-----+---------| | 40 | 60 | 100 | #+TBLFM: $3=vsum($1..$2)::@4=vsum(@I..@II) Here the 10, 20, 30, 40 are added manually and the other cells are calculated by the sum-formulas. Since the results are inserted into the table, other tools can just skip over the TBLFM-metadata if they don't support it.
- TeMPOraL 7y agoThe four differences between .org and CommonMark I could spot right now is that the latter lacks metadata[0], macros[1], syntax for specifying language for inline code blocks[2], and for tables[3]. It's hard to separate .org the markup language from org-mode the application, because quite a lot of its syntax is driven by application features. TODO markers, time stamps, schedule/deadline markers, property drawers, log books, tags - all of these are part of Org Mode functionality, but have little use if you're just trying to write a document for export. -- [0] - There's ton of stuff you could optionally add with #+SOMETHING: lines - local configuration of how to work with the file, metadata, exporter settings for any format individually and all of them together, and more. A lot of those can also be specified per-headline. [1] - https://orgmode.org/manual/Macro-replacement.html; https://orgmode.org/manual/Macro-replacement.html; crazy, but when it's needed, it works wonders. [2] - You can make Markdown-style code spans in Org Mode ~like this~, but there's actually a syntax for proper inline source code blocks that's much less known: src_lang{code}, or src_lang[header-args]{code}. For example, src_lisp{(+ 2 2)} or src_lisp[:results output]{(format t "Foobar")}. The reason for the header-args variant is that Org Mode inline code blocks can be executed interactively or on export, just like regular code blocks. See: https://orgmode.org/manual/Structure-of-code-blocks.html https://orgmode.org/manual/Structure-of-code-blocks.html for details on both. [3] - In CommonMark you could technically abuse HTML for that, but Org Mode provides the most readable format for tables... that's also the most convenient to write if you're using Org Mode, and an absolute PITA if you're not. See: https://orgmode.org/manual/Tables.html https://orgmode.org/manual/Tables.html.
- indentit 7y agoI guess these days it is better to compare GitHub Formatted Markdown, a common extension to CommonMark, as opposed to plain CommonMark. It supports code blocks with language hints, tables etc. Also, I am pretty sure Markdown with metadata (sometimes referred to as "front matter") already exists and is called MultiMarkdown, though presumably not as powerful/useful as what you describe here.
- gnomewascool 7y ago> code blocks with language hints CommonMark also supports code blocks with language hints (using the ```lang... syntax). What Org-mode has, in addition to code blocks with language hints, is inline code with language hints, which neither CommonMark nor GFM support. > tables I'm horrified that, indeed, CommonMark doesn't have tables, though you're right that most people now just use GFM.
- harrygeez 7y agoSo they way I see it, `org` file format and `org-mode` are 2 different things. On surface level, `org` files don't bring carry many significant advantages over similar formats like Markdown; rather, it's Emacs's `org-mode` that brings this text file to life. You can have a full-fledged calendar and scheduling system just in a few lines of text. Imagine that.
- conquistadog 7y agoOrg is just a shorthand as far as I know. Emacs Org-Mode works with files typically ending in ".org". I prefer to avoid the confusion by saying "Org-Mode" and "Org-Mode files" fully (and, I believe, properly).
- phaer 7y agoIt's great to have more options, but people might be interested in the org-mode support of pandoc: https://pandoc.org/org.html https://pandoc.org/org.html This enables .e.g. transformation of org-mode files with latex templates, to html slides or even into .odt files and customization via filters over pandocs AST. These filters can be not only written in haskell but in pretty much any language.
- mruts 7y agoI use pandoc + org-mode and it’s perfect. I love being able to convert my notes to LaTeX while correctly handling the inline images and math.
- TeMPOraL 7y agoWhat's the reason to use Pandoc to convert .org files to LaTeX instead of the built-in Org Mode LaTeX exporter? Serious question - I do plenty of .org -> LaTeX like this, and never tried the Pandoc route before.
- branweb 7y agoI don't think there is one, other than personal preference. It's really useful for going the other way though: <some format e.g. html> -> org.
- mruts 7y agoFor some reason, emacs to pdf or latex errors out for some .sty file being missing. I could probably fix it, but pandoc works out of the box so I guess I’ll be sticking with it.
- snicker7 7y agoOrg to PDF via Emacs works great. The main issue is making Emacs a hard dependency of your publishing pipeline.
- TeMPOraL 7y ago
- iBelieve 7y agoThis is great to see. I started my own org-mode library for Rust at the end of last year (https://github.com/iBelieve/orgmode-rs https://github.com/iBelieve/orgmode-rs), with the goal of building a desktop or web app for org-mode, but haven't had much time to work on it.
- 0815test 7y agoYou may want to take a look at this issue https://github.com/ngortheone/org-rs/issues/1 https://github.com/ngortheone/org-rs/issues/1 and perhaps join in - it seems that there are in fact a number of projects at a similar stage of development (or lack thereof, perhaps) so it makes sense to try and coalesce efforts.
- eclipseo76 7y agoNow please write an AsciiDoc parser pretty please. I'd like to use it with a static site generator in Rust.
- amenod 7y agoGenuinly curious: > Why reinventing the wheel when we can just copy it! This project takes the only surefire way to get it right - use the original elisp parser implementation as a blueprint! Emacs Lisp code is under GPLv3 license, is it OK to rewrite it in another language (using original code as blueprint) and publish it under MIT?
- dotancohen 7y agoOf course. That is a derivative work.
- wyldfire 7y agoHuh? Derived works can't be re-licensed arbitrarily. (But yes under the terms of the GPL3 you could release your derived work under GPL3).
- szemet 7y agoAnd if I compile some GNU code (for example written in C) to javascript with a compiler, then change some details here and there can I change the licence to MIT? How derivative is defined? Manual compilation is ok, but automatic is not? But what if I beleive that humans are just fancy machines? ;)
- wyldfire 7y agoIt's somewhat interesting that someone might copyright the output of a compiler. Although in practice it's not terribly different from the source. > How derivative is defined? For the US, it's primarily defined in plain English in the USC. But subtle distinctions are made by court rulings/opinions.
- mikekchar 7y agoThis is actually why GCC has (or at least did at one point) an extra clause to the GPL to specifically allow its output to be used in non-GPL programs. Because the compiler output necessarily has to copy some bits from the compiler, it was the FSF's belief that programs compiled with it would become derived works and hence be forced to use the GPL. They didn't want that, so they added the extra clause. You may think that's a strange interpretation, but before GCC many compilers took that exact same stance. You bought the compiler and you did not have a license to distribute the resultant binary. You had to pay an extra fee for the right to distribute your own compiled code. That pretty much stopped when Borland released a compiler with a mostly sane license. You can imagine why, though, that Cygnus made so much money doing contract work for GCC in the embedded space -- every other compiler required royalties for everything you shipped!
- angleofrepose 7y agoWhat sense does it make to have org mode outside of emacs? I think the entire power of the system would be that it explicitly lives within emacs. From the repo: > Org is probably the best and most complete plain text organizational system known to mankind. It has countless applications like authoring, publishing, task and time tracking, journal, blog, agenda, wiki etc... Yes, absolutely. > Unfortunately Org was originally developed for Emacs and therefore available only inside Emacs. It is a huge limiting factor for Org's development and popularization. Because of that it is not as popular outside of Emacs community as it should be. "Unfortunately"? It is a dynamic tool, I don't understand what it would mean for it available outside of emacs except for if that were also a dynamic tool. Are you going to create a standalone dynamic org mode application? It seems better to make a "stripped down" emacs that only has org mode and literate programming features so as to not have as steep a learning curve as the entirety of emacs all at once. What other reason is there for this work except for avoiding the learning curve of emacs? Emacs has got to be one of the most configurable pieces of software outside of Operating Systems(maybe?) in existence. Org mode is powerful because it is a plaintext system that gives hooks for emacs abilities like transformation (faces, folding, tagging/searching, agenda features) and execution of code with literate programming, and passing values around between code blocks and so on. Why would this want to live outside of emacs? I do think the recently (and again two years ago) extremely popular post on org mode's markup language absolutely misses this point[1]. This post is also linked to from the repo. I'd love to learn about what I'm missing here, and would love to get some answers to my questions or responses to my assumptions! I'm very interested in your roadmap and design decisions, I think your external links (except for that article) are fantastic. I will follow the project for sure. I just think there is some mental disconnect with how org mode is represented and discussed on hn, and those discussions almost all seem to miss the majority of org mode abilities that absolutely depend on living in the style of environment that emacs provides. From reading the top discussions here, one might think org mode is in competition for the same space as markdown while that is not even remotely true [2]. [1]: https://news.ycombinator.com/item?id=19622019 https://news.ycombinator.com/item?id=19622019 [2]: https://hn.algolia.com/?query=org%20mode&sort=byPopularity&prefix=false&page=0&dateRange=all&type=story https://hn.algolia.com/?query=org%20mode&sort=byPopularity&p... PS: I am trying to do a more thorough review of the discussion about org mode on hn, and I will be taking actual notes and comment links down to better explain my ideas here rather that just general takeaways. Thanks for bearing with me, or letting me know if you disagree.
- zebproj 7y agoI've been using org and org-mode for literate programming for the past few months. org-mode provides a really nice interactive environment suitable for literate programming, but tangling code is hilariously slow, to the point where several people [0] (including myself [1]), have written org tanglers. It'd be interesting to see benchmarks against the original org-mode implementation. [0] https://github.com/orgtangle https://github.com/orgtangle [1] https://github.com/OrgTangle/Worgle https://github.com/OrgTangle/Worgle
- Ideabile 7y agoThanks for have making this! Not long ago, I was dreaming about getting an embeddable version with Rust with DSL. And since a couple of days I'm getting more posts about org-mode. I think are nice foundations to continue on.
- westoncb 7y agoI often see things about org-mode showing up on here, and I get the basic idea but I'd like to see a more in depth explanation of why it's so great. Anyone have a recommendation for an article or video or something on the subject?
- Sylos 7y agoI think, the major appeal of it is that it does everything. Lots of "productivity"-applications are either TODO-lists or meant for note-taking or good for writing down concepts or are knowledge bases. But you can't really separate these. Often I'll write something down and later realize that I need to do xyx for that. Or that I want to schedule a reminder for this note. Other times I write something down just to have it written down somewhere and I won't want to see it again until months later. But if I just write it into a random .txt-file, I'll probably never see it again. With it being in the same place as my other notes, todos etc., I will actually find it again. And other times, you might have this dumb idea for a thing. And then you add some more ideas to this idea. And before you know it, you've written down the entire concept for your next software project in your notes-application. When this happens to me with OrgMode, I really don't mind. Its plaintext-format is just as readable as MarkDown for this use-case. I might very well stick this OrgMode-file into the software repository. Also, sometimes I will find an article to something in my software project concept and want to note it down there with a reminder to actually look at it. OrgMode allows me to do that. Eventually this becomes a lot of information in one place, but OrgMode itself has lots of tools to organize that: tags, priorities (from A to whichever letter you like), notebooks (=different files), states (e.g. TODO, DONE, NOTE, and again whatever you like), scheduled times, deadlines, recurring tasks etc. And then there's obviously also parsers like the one in this post. They can do whatever they want with the plaintext you have there. Personally, I mostly use the Android-app Orgzly. It has a widget with a simple DONE-button for my TODO-usage. It can throw notifications at me. And it has a really powerful search-feature, allowing to save specific searches and for example select them as the listing to display in the widget. E.g. I only want to see notes with the state "TODO" that are scheduled within the next three days. I will admit that it's a bit of a rabbit hole, but task/note management in general usually is and this feels better than my previous solutions. Especially also the fact that it is just plaintext-files that I can sync and backup easily.
- burtonator 7y ago"Org-mode parser in Rust" ... I think this is the nerdiest title I've ever seen posted to HN :)
- Lord_Nightmare 7y agoI was excited, but the fact that this is (well, WAS) violating the GPL by being MIT licensed was a major downer. I'd be very interested in an org-mode parser which is NOT GPL-licensed, specifically because it is effectively impossible to integrate GPL code as a plugin (unless your editor is also GPL), which was one of the stated goals of this project. Hence, someone would have to properly clean-room reverse-engineer org-mode in order to achieve 100% compatibility, or make a somewhat-incompatible implementation based on the reference docs instead. As I understand it, Markdown has significant limitations which org-mode doesn't have, but is under a 3BSD-style license. (It also has issues of x+1 slightly incompatible implementations)
- juki 7y agoI believe Gitlab and Github both render Org files with OrgRuby[1], which is MIT. [1]: https://github.com/wallyqs/org-ruby https://github.com/wallyqs/org-ruby
- black-tea 7y agoWhy can't you make a GPL plugin for whatever editor you use?
- Lord_Nightmare 7y agoIt depends on whether it is an essential dll or an optional plugin. I probably could have phrased that better originally.