26 ms·

And the author specifies Z85 only for 4 byte chunks of data - not all data comes in 32 bit chunks. If the authors want Z85 to be widely used, specifying padding

And the author specifies Z85 only for 4 byte chunks of data - not all data comes in 32 bit chunks. If the authors want Z85 to be widely used, specifying padding would be helpful. Anyway, it all seems a bit https://xkcd.com/927/ https://xkcd.com/927/

dullcrisp 3y ago
It looks like they’re counting on most data being padded to 32 bits. The simplicity due to lack of padding rules seems to be the only real advantage this encoding has over base 64. Otherwise you’re right, offering up a new standard because there are too many competing base 64 standards makes no sense.
- nabla9 3y ago
  >The simplicity due to lack of padding rules seems to be the only real advantage Because there is no padding rule given, it's impossible to use the standard to encode most binary strings. For example, how would you encode/decode these? 0x0000000000000000 0x00000000000000 0x000000000000 0x0000000000 0x00000000 0x000000
  - dullcrisp 3y ago
    You could only encode the ones whose length was a multiple of 8. 0x00000000 would be 00000. 0x0000000000000000 would be 00000 00000. How would you encode binary 000000 in hex?
    - yau8edq12i 3y ago
      "Hex(adecimal)" is not an encoding scheme, it's a number basis. "Binary 00000" as a number is just zero, which is written equivalently as "0", "00"... in hexadecimal. This is different from a chunk of data consisting of six bits that all happen to be zero. And indeed, you need to decide on how to convey that, and simply saying "hex" - or "binary" or "decimal" for that matter - is not enough, that is correct.
  - fwip 3y ago
    If the situation requires decoding strings of unknown length, you simply prefix the length.
    - nabla9 3y ago
      It was rhetorical question. What you describe is not defined in the format. You can't use the format to exchange strings of unknown length. If people would start using this encoding, different users would adopt different solutions padding, length prefixes etc. and it becomes mess.
      - fwip 3y ago
        It's not defined in the format, sure. That doesn't mean the format is bad. The format also doesn't specify how to interpret the binary data, or how to encode multiple strings, or automatically checksum it for you. But you can obviously use the format to exchange strings of variable length, or encode both integers and images, send multiple strings, or add your own checksum. It's not "impossible to encode" that content just because you need to decide how to represent it. It's not "a mess" if some people use fixed-length strings and some use length-prefixed strings. It's just reality for any encoding scheme - you build layers around it, according to how you want to use it. The same way the context of your software determines whether this binary string is supposed to represent a float, a name, a hash, or a pixel-art masterpiece, it will also determine the appropriate serialization.
- karmakaze 3y ago
  All the issues mentioned in these comments is a good indication why this encoding isn't more widely known/used. I myself typically binary compress then base64 encode.
  - bcaxis 3y ago
    For your trouble, you save some extra bytes with 85 vs 64.
    - petre 3y ago
      With the added bonus of funny characters which are a pain to encode in JSON, XML and URLs. I guess I'll keep using using URL safe Base64, thank you.
      - bcaxis 3y ago
        Modifying the base alphabet is simple if there are a couple of problem characters. I use 85 in json just fine without escaping overhead. Never tried for URLs or XML.
  - Varriount 3y ago
    Good to know I'm not the only one who has done this! I once wrote a template generation tool that embedded its input configuration into its output files as base64-encoded gzip'd JSON. Worked quite well at allowing template regeneration.
  - jgalt212 2y ago
    and then the webserver probably recompresses again via gzip.
grodriguez100 3y ago
That xkcd comic was the first thing that came to mind as well. The article explains how Base64 is “problematic because it has more than a dozen variants”. So instead of picking one, let’s invent a new standard.
mnw21cam 3y ago
That XKCD is so old now that the alt-text: > Fortunately, the charging one has been solved now that we've all standardized on mini-USB. Or is it micro-USB? Shit. is hilariously even more true.

dullcrisp 3y ago

It looks like they’re counting on most data being padded to 32 bits. The simplicity due to lack of padding rules seems to be the only real advantage this encoding has over base 64. Otherwise you’re right, offering up a new standard because there are too many competing base 64 standards makes no sense.

nabla9 3y ago
>The simplicity due to lack of padding rules seems to be the only real advantage Because there is no padding rule given, it's impossible to use the standard to encode most binary strings. For example, how would you encode/decode these? 0x0000000000000000 0x00000000000000 0x000000000000 0x0000000000 0x00000000 0x000000
- dullcrisp 3y ago
  You could only encode the ones whose length was a multiple of 8. 0x00000000 would be 00000. 0x0000000000000000 would be 00000 00000. How would you encode binary 000000 in hex?
  - yau8edq12i 3y ago
    "Hex(adecimal)" is not an encoding scheme, it's a number basis. "Binary 00000" as a number is just zero, which is written equivalently as "0", "00"... in hexadecimal. This is different from a chunk of data consisting of six bits that all happen to be zero. And indeed, you need to decide on how to convey that, and simply saying "hex" - or "binary" or "decimal" for that matter - is not enough, that is correct.
- fwip 3y ago
  If the situation requires decoding strings of unknown length, you simply prefix the length.
  - nabla9 3y ago
    It was rhetorical question. What you describe is not defined in the format. You can't use the format to exchange strings of unknown length. If people would start using this encoding, different users would adopt different solutions padding, length prefixes etc. and it becomes mess.
    - fwip 3y ago
      It's not defined in the format, sure. That doesn't mean the format is bad. The format also doesn't specify how to interpret the binary data, or how to encode multiple strings, or automatically checksum it for you. But you can obviously use the format to exchange strings of variable length, or encode both integers and images, send multiple strings, or add your own checksum. It's not "impossible to encode" that content just because you need to decide how to represent it. It's not "a mess" if some people use fixed-length strings and some use length-prefixed strings. It's just reality for any encoding scheme - you build layers around it, according to how you want to use it. The same way the context of your software determines whether this binary string is supposed to represent a float, a name, a hash, or a pixel-art masterpiece, it will also determine the appropriate serialization.
karmakaze 3y ago
All the issues mentioned in these comments is a good indication why this encoding isn't more widely known/used. I myself typically binary compress then base64 encode.
- bcaxis 3y ago
  For your trouble, you save some extra bytes with 85 vs 64.
  - petre 3y ago
    With the added bonus of funny characters which are a pain to encode in JSON, XML and URLs. I guess I'll keep using using URL safe Base64, thank you.
    - bcaxis 3y ago
      Modifying the base alphabet is simple if there are a couple of problem characters. I use 85 in json just fine without escaping overhead. Never tried for URLs or XML.
- Varriount 3y ago
  Good to know I'm not the only one who has done this! I once wrote a template generation tool that embedded its input configuration into its output files as base64-encoded gzip'd JSON. Worked quite well at allowing template regeneration.
- jgalt212 2y ago
  and then the webserver probably recompresses again via gzip.

grodriguez100 3y ago

That xkcd comic was the first thing that came to mind as well. The article explains how Base64 is “problematic because it has more than a dozen variants”. So instead of picking one, let’s invent a new standard.

mnw21cam 3y ago

That XKCD is so old now that the alt-text: > Fortunately, the charging one has been solved now that we've all standardized on mini-USB. Or is it micro-USB? Shit. is hilariously even more true.