6 ms·

Couldn't the input be automatically described/guessed using a few rows of data and a LLM?

• terrelln 1y ago
  You could have an LLM generate the SDDL description [0] for you, or even have it write a C++ or Python tokenizer. If compression succeeds, then it is guaranteed to round trip, as the LLM-generated logic lives only on the compression side, and the decompressor is agnostic to it. It could be a problem that is well-suited to machine learning, as there is a clear objective function: Did compression succeed, and if so what is the compressed size. [0] https://openzl.org/api/c/graphs/sddl/ https://openzl.org/api/c/graphs/sddl/