6 ms·

I just reverse engineered the Supernote note file format with an agent a few weeks ago. For years the community had been asking for a document on the format. An

by philips 25d ago

I just reverse engineered the Supernote note file format with an agent a few weeks ago. For years the community had been asking for a document on the format. And in a few hours the agent, with 20 something file format example fixtures and 30 something prompts, was able to reverse out the format. It would have been completely not worth the effort to do this by hand for a niche device. Now, in a few hours of effort there is working code and a doc. https://github.com/philips/supernote-typescript/blob/main/plans/vector-format-spec.md https://github.com/philips/supernote-typescript/blob/main/pl... https://philips.github.io/supernote-typescript/ https://philips.github.io/supernote-typescript/

zatkin 25d ago
I personally own a Supernote, but I'm not a heavy user of it. For the sake of my own curiosity, what benefits will you get out of having reverse engineered the Supernote note file format? It would be super rad to be able to move my notes between other devices, which is one big plus that comes to my mind.
- philips 25d ago
  I built a management website and plugin for Obsidian. https://supernote.ifup.org/ https://supernote.ifup.org/ https://youtu.be/ihRh_F43-iQ https://youtu.be/ihRh_F43-iQ
frio 25d ago
While it’s impressive work from the LLM and a TS implementation is novel, there’s at least a couple of pre-existing Python REs eg. https://github.com/jya-dev/supernote-tool https://github.com/jya-dev/supernote-tool :)
- fwip 25d ago
  It seems like most of these "an LLM solved this in only X hours! " could have been "I found an open source solution that did what I needed with X minutes of web search." Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art (in either the training set or explicitly in a a web search). But it does seem to be a trend.
  - deleted 25d ago
    [deleted]
  - ShinyLeftPad 25d ago
    > Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art It is definitely the case that people know less and less how to do research themselves though...
  - fragmede 25d ago
    If give an AI the full set of files it needs to RE a file format, and it's running xxd in tool calls in order to document the file format, I don't think it's cheating by copying it off the Internet.
    - fwip 24d ago
      It might be reverse-engineering it, but it also might be kayfabe. (Kayfabe is what professional wrestlers do - the storyline and matches are scripted, but it's presented as real. LLMs excel at doing this.) Similar to how Anthropic showed that their LLMs appear to "think ahead" when writing poetry to come up with a rhyme[1], so too might the knowledge of a file's structure influence how the LLM approaches analysis. As a simplistic example, suppose one section of the file is known by the model to be bzip2 compressed - the LLM may use xxd to scan for common magic numbers that "just so happens" to include 42 5A (Bz). Every step of analysis is like this - what threads to look for, and which ones to pull on. Somebody or something who mostly-remembers the answer is going to find the answer quicker than if they'd gone in blind. [1] https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language...
  - analog_daddy 25d ago
    Ohh, sadly I relate with this feeling too much. For all the agentic loops people seem to have come up with, the research loop or as I call it the “Desperate 10th page on Github’s crappy search results” is still not up to the mark. Either it might be genuine rate limiting these LLM’s face or just that, they are trained to focus on implementing a solution which would be faster and user acceptable solution. (which seems to be a true looking at people pushing LLM generated code as is). At least in my personal experience with niche projects and heck even with well documented and famous libraries, along with fancy mcp’s, llms.txt and skills; RTFM has been more relevant than usual for code that I have asked an agent to generate, since it is too eager to reimplement functionality which already exists, only if it RTFM!!
- philips 25d ago
  I know of the project but it doesn’t actually extract the stroke information. It converts the raster into vector. My reverse engineering extracts each pen stroke directly into a svg vector.
  - psolidgold 25d ago
    This sounds awesome if it works as you say. Will give it a shot tomorrow! Thank you for the tokens to solve this.
    - philips 25d ago
      Checkout this "handwriting demo" I made. It uses the stroke information to animate each individual stroke. https://philips.github.io/supernote-web-component/ https://philips.github.io/supernote-web-component/ Click the "Pen" icon then the "Play" button.
deleted 25d ago
[deleted]

by philips 25d ago

zatkin 25d ago

I personally own a Supernote, but I'm not a heavy user of it. For the sake of my own curiosity, what benefits will you get out of having reverse engineered the Supernote note file format? It would be super rad to be able to move my notes between other devices, which is one big plus that comes to my mind.

philips 25d ago
I built a management website and plugin for Obsidian. https://supernote.ifup.org/ https://supernote.ifup.org/ https://youtu.be/ihRh_F43-iQ https://youtu.be/ihRh_F43-iQ

frio 25d ago

While it’s impressive work from the LLM and a TS implementation is novel, there’s at least a couple of pre-existing Python REs eg. https://github.com/jya-dev/supernote-tool https://github.com/jya-dev/supernote-tool :)

fwip 25d ago
It seems like most of these "an LLM solved this in only X hours! " could have been "I found an open source solution that did what I needed with X minutes of web search." Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art (in either the training set or explicitly in a a web search). But it does seem to be a trend.
- deleted 25d ago
  [deleted]
- ShinyLeftPad 25d ago
  > Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art It is definitely the case that people know less and less how to do research themselves though...
- fragmede 25d ago
  If give an AI the full set of files it needs to RE a file format, and it's running xxd in tool calls in order to document the file format, I don't think it's cheating by copying it off the Internet.
  - fwip 24d ago
    It might be reverse-engineering it, but it also might be kayfabe. (Kayfabe is what professional wrestlers do - the storyline and matches are scripted, but it's presented as real. LLMs excel at doing this.) Similar to how Anthropic showed that their LLMs appear to "think ahead" when writing poetry to come up with a rhyme[1], so too might the knowledge of a file's structure influence how the LLM approaches analysis. As a simplistic example, suppose one section of the file is known by the model to be bzip2 compressed - the LLM may use xxd to scan for common magic numbers that "just so happens" to include 42 5A (Bz). Every step of analysis is like this - what threads to look for, and which ones to pull on. Somebody or something who mostly-remembers the answer is going to find the answer quicker than if they'd gone in blind. [1] https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language...
- analog_daddy 25d ago
  Ohh, sadly I relate with this feeling too much. For all the agentic loops people seem to have come up with, the research loop or as I call it the “Desperate 10th page on Github’s crappy search results” is still not up to the mark. Either it might be genuine rate limiting these LLM’s face or just that, they are trained to focus on implementing a solution which would be faster and user acceptable solution. (which seems to be a true looking at people pushing LLM generated code as is). At least in my personal experience with niche projects and heck even with well documented and famous libraries, along with fancy mcp’s, llms.txt and skills; RTFM has been more relevant than usual for code that I have asked an agent to generate, since it is too eager to reimplement functionality which already exists, only if it RTFM!!
philips 25d ago
I know of the project but it doesn’t actually extract the stroke information. It converts the raster into vector. My reverse engineering extracts each pen stroke directly into a svg vector.
- psolidgold 25d ago
  This sounds awesome if it works as you say. Will give it a shot tomorrow! Thank you for the tokens to solve this.
  - philips 25d ago
    Checkout this "handwriting demo" I made. It uses the stroke information to animate each individual stroke. https://philips.github.io/supernote-web-component/ https://philips.github.io/supernote-web-component/ Click the "Pen" icon then the "Play" button.

deleted 25d ago

[deleted]