5 ms·

While it’s impressive work from the LLM and a TS implementation is novel, there’s at least a couple of pre-existing Python REs eg. https://github.com/jya-dev/su

by frio 24d ago

While it’s impressive work from the LLM and a TS implementation is novel, there’s at least a couple of pre-existing Python REs eg. https://github.com/jya-dev/supernote-tool https://github.com/jya-dev/supernote-tool :)

fwip 24d ago
It seems like most of these "an LLM solved this in only X hours! " could have been "I found an open source solution that did what I needed with X minutes of web search." Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art (in either the training set or explicitly in a a web search). But it does seem to be a trend.
- deleted 24d ago
  [deleted]
- ShinyLeftPad 24d ago
  > Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art It is definitely the case that people know less and less how to do research themselves though...
- fragmede 24d ago
  If give an AI the full set of files it needs to RE a file format, and it's running xxd in tool calls in order to document the file format, I don't think it's cheating by copying it off the Internet.
  - fwip 23d ago
    It might be reverse-engineering it, but it also might be kayfabe. (Kayfabe is what professional wrestlers do - the storyline and matches are scripted, but it's presented as real. LLMs excel at doing this.) Similar to how Anthropic showed that their LLMs appear to "think ahead" when writing poetry to come up with a rhyme[1], so too might the knowledge of a file's structure influence how the LLM approaches analysis. As a simplistic example, suppose one section of the file is known by the model to be bzip2 compressed - the LLM may use xxd to scan for common magic numbers that "just so happens" to include 42 5A (Bz). Every step of analysis is like this - what threads to look for, and which ones to pull on. Somebody or something who mostly-remembers the answer is going to find the answer quicker than if they'd gone in blind. [1] https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language...
- analog_daddy 24d ago
  Ohh, sadly I relate with this feeling too much. For all the agentic loops people seem to have come up with, the research loop or as I call it the “Desperate 10th page on Github’s crappy search results” is still not up to the mark. Either it might be genuine rate limiting these LLM’s face or just that, they are trained to focus on implementing a solution which would be faster and user acceptable solution. (which seems to be a true looking at people pushing LLM generated code as is). At least in my personal experience with niche projects and heck even with well documented and famous libraries, along with fancy mcp’s, llms.txt and skills; RTFM has been more relevant than usual for code that I have asked an agent to generate, since it is too eager to reimplement functionality which already exists, only if it RTFM!!
- Klonoar 23d ago
  Per the author, that open source project is not doing the same things as what they cobbled together.
- inigyou 22d ago
  Someone has to write it though. LLMs make low-quality output in high volumes, and sometimes we find a situation where that's actually good - like this one!
philips 24d ago
I know of the project but it doesn’t actually extract the stroke information. It converts the raster into vector. My reverse engineering extracts each pen stroke directly into a svg vector.
- psolidgold 24d ago
  This sounds awesome if it works as you say. Will give it a shot tomorrow! Thank you for the tokens to solve this.
  - philips 24d ago
    Checkout this "handwriting demo" I made. It uses the stroke information to animate each individual stroke. https://philips.github.io/supernote-web-component/ https://philips.github.io/supernote-web-component/ Click the "Pen" icon then the "Play" button.

by frio 24d ago

fwip 24d ago

It seems like most of these "an LLM solved this in only X hours! " could have been "I found an open source solution that did what I needed with X minutes of web search." Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art (in either the training set or explicitly in a a web search). But it does seem to be a trend.

deleted 24d ago
[deleted]
ShinyLeftPad 24d ago
> Which doesn't mean that the LLM definitely couldn't have accomplished it without the prior art It is definitely the case that people know less and less how to do research themselves though...
fragmede 24d ago
If give an AI the full set of files it needs to RE a file format, and it's running xxd in tool calls in order to document the file format, I don't think it's cheating by copying it off the Internet.
- fwip 23d ago
  It might be reverse-engineering it, but it also might be kayfabe. (Kayfabe is what professional wrestlers do - the storyline and matches are scripted, but it's presented as real. LLMs excel at doing this.) Similar to how Anthropic showed that their LLMs appear to "think ahead" when writing poetry to come up with a rhyme[1], so too might the knowledge of a file's structure influence how the LLM approaches analysis. As a simplistic example, suppose one section of the file is known by the model to be bzip2 compressed - the LLM may use xxd to scan for common magic numbers that "just so happens" to include 42 5A (Bz). Every step of analysis is like this - what threads to look for, and which ones to pull on. Somebody or something who mostly-remembers the answer is going to find the answer quicker than if they'd gone in blind. [1] https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language...
analog_daddy 24d ago
Ohh, sadly I relate with this feeling too much. For all the agentic loops people seem to have come up with, the research loop or as I call it the “Desperate 10th page on Github’s crappy search results” is still not up to the mark. Either it might be genuine rate limiting these LLM’s face or just that, they are trained to focus on implementing a solution which would be faster and user acceptable solution. (which seems to be a true looking at people pushing LLM generated code as is). At least in my personal experience with niche projects and heck even with well documented and famous libraries, along with fancy mcp’s, llms.txt and skills; RTFM has been more relevant than usual for code that I have asked an agent to generate, since it is too eager to reimplement functionality which already exists, only if it RTFM!!
Klonoar 23d ago
Per the author, that open source project is not doing the same things as what they cobbled together.
inigyou 22d ago
Someone has to write it though. LLMs make low-quality output in high volumes, and sometimes we find a situation where that's actually good - like this one!

philips 24d ago

I know of the project but it doesn’t actually extract the stroke information. It converts the raster into vector. My reverse engineering extracts each pen stroke directly into a svg vector.

psolidgold 24d ago
This sounds awesome if it works as you say. Will give it a shot tomorrow! Thank you for the tokens to solve this.
- philips 24d ago
  Checkout this "handwriting demo" I made. It uses the stroke information to animate each individual stroke. https://philips.github.io/supernote-web-component/ https://philips.github.io/supernote-web-component/ Click the "Pen" icon then the "Play" button.