5 ms·

Can't store tabs or newlines, odd choice.

romanows 2y ago
Reading quick, it's because the tab is used to indicate nested tabular data in a column. I wonder why not just have a zsv in the zsv?
hafthor 2y ago
yeah, this is a limitation from the TSV format this is based on - there is an extension to the format that supports storing binary blobs - ref: https://github.com/Hafthor/zsvutil?tab=readme-ov-file#nestedbinary-data https://github.com/Hafthor/zsvutil?tab=readme-ov-file#nested...
- alexandreyc 2y ago
  Basically it's the same limitations as CSV. At least you could use something less likely to appear in data as record sepator (like 0x1E) Otherwise it's an interesting idea!
  - tboerstad 2y ago
    0x1E is the record separator, in ASCII precisely for this purpose. Too bad it’s not popular, here we’re stuck with inferior TSV/CSV
    - orf 2y ago
      Strings can contain 0x1E, so it has exactly the same issues as a tab character but with all the downsides of it not being an easy, “simple” character.
    - mattnewton 2y ago
      I can't easily type that out - and once the format can't be read / editing in a simple text editor, I'm starting to lean towards a nice binary format like protobuf.
  - bobbylarrybobby 2y ago
    As far as I know, thanks to quoting it is possible to put basically any data you want in a CSV.
    - layer8 2y ago
      The problem is there is no uniform standard for quoting and escaping in CSV, and different software uses different variants.
      - Dylan16807 2y ago
        There is a standard, and it is very simple and easy to use. Different software uses different variants because we're not allowed to have nice things and devs are too lazy to use something slightly more complicated than .split(',') Though if you're going to ban some common characters anyway like TSV, you might as well use CSV and ban commas, newlines, and quotation marks.
- inimino 2y ago
  (Offtopic, but just FYI) it's tenet (principle) not tenant (building resident).
  - hafthor 2y ago
    doh. fixed. thanks!
- nextaccountic 2y ago
  can't you just do quoting?
  - olejorgenb 2y ago
    https://github.com/Hafthor/zsvutil?tab=readme-ov-file#what-are-some-key-shortcomings-of-zsv https://github.com/Hafthor/zsvutil?tab=readme-ov-file#what-a... > Any escaping or encoding of these characters would make the format less human-readable, harder to parse and could introduce ambiguity and consistency problems. Found the wording of "could introduce ambiguity and consistency problems" a bit odd, but guess they mean that even if things are specified precisely (so there's no ambiguity) not everyone would follow the rules or something? And they want to play nice with other tools following the TSV "standard"
    - 8n4vidtmkvmk 2y ago
      Please. I wrote a csv parser a couple weeks ago in an hour or two. It's not that hard to handle the quoting and edge cases. Yes, maybe different parsers will handle them differently, but just document your choices and that's that. How is ambiguity better than completely disallowing certain chars? That's a non-starter
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]

Reading quick, it's because the tab is used to indicate nested tabular data in a column. I wonder why not just have a zsv in the zsv?

hafthor 2y ago

yeah, this is a limitation from the TSV format this is based on - there is an extension to the format that supports storing binary blobs - ref: https://github.com/Hafthor/zsvutil?tab=readme-ov-file#nestedbinary-data https://github.com/Hafthor/zsvutil?tab=readme-ov-file#nested...

alexandreyc 2y ago
Basically it's the same limitations as CSV. At least you could use something less likely to appear in data as record sepator (like 0x1E) Otherwise it's an interesting idea!
- tboerstad 2y ago
  0x1E is the record separator, in ASCII precisely for this purpose. Too bad it’s not popular, here we’re stuck with inferior TSV/CSV
  - orf 2y ago
    Strings can contain 0x1E, so it has exactly the same issues as a tab character but with all the downsides of it not being an easy, “simple” character.
  - mattnewton 2y ago
    I can't easily type that out - and once the format can't be read / editing in a simple text editor, I'm starting to lean towards a nice binary format like protobuf.
- bobbylarrybobby 2y ago
  As far as I know, thanks to quoting it is possible to put basically any data you want in a CSV.
  - layer8 2y ago
    The problem is there is no uniform standard for quoting and escaping in CSV, and different software uses different variants.
    - Dylan16807 2y ago
      There is a standard, and it is very simple and easy to use. Different software uses different variants because we're not allowed to have nice things and devs are too lazy to use something slightly more complicated than .split(',') Though if you're going to ban some common characters anyway like TSV, you might as well use CSV and ban commas, newlines, and quotation marks.
inimino 2y ago
(Offtopic, but just FYI) it's tenet (principle) not tenant (building resident).
- hafthor 2y ago
  doh. fixed. thanks!
nextaccountic 2y ago
can't you just do quoting?
- olejorgenb 2y ago
  https://github.com/Hafthor/zsvutil?tab=readme-ov-file#what-are-some-key-shortcomings-of-zsv https://github.com/Hafthor/zsvutil?tab=readme-ov-file#what-a... > Any escaping or encoding of these characters would make the format less human-readable, harder to parse and could introduce ambiguity and consistency problems. Found the wording of "could introduce ambiguity and consistency problems" a bit odd, but guess they mean that even if things are specified precisely (so there's no ambiguity) not everyone would follow the rules or something? And they want to play nice with other tools following the TSV "standard"
  - 8n4vidtmkvmk 2y ago
    Please. I wrote a csv parser a couple weeks ago in an hour or two. It's not that hard to handle the quoting and edge cases. Yes, maybe different parsers will handle them differently, but just document your choices and that's that. How is ambiguity better than completely disallowing certain chars? That's a non-starter

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]