10 ms·

Niklaus Wirth has entered the chat.

Tuna-Fish 2y ago
There are a whole bunch of other things I dislike about Pascal, but in this one he was just undeniably correct. Worrying about 1-3 extra bytes (on the 16 and 32 bit platforms where it potentially mattered) was just not worth all the issues that null-terminated strings brought with them. My current favorite string implementations are the various compact string crates for Rust. Generally, you want a string to be able to do at least three things: - Pointer, length, capacity tuple, for heap-allocated strings, 24B on x64. - String inlined into the 24B buffer. - Pointer, length tuple where pointer points to rodata. You can do any of that and the discriminator in 24B, given the healthy assumption that all strings are shorter than 2^63. Sadly, switching costs are massive and every programming language is pretty much struck with the string they started with. Hopefully, whatever comes next can crib from smartstring or compact_str or the like.
- gumby 2y ago
  There was a paper on this in the late 70s from the Cedar group at PARC. This was back when computer science papers were actual scientific papers, so full of analysis of different alogorithms' performance with counted vs delimited strings. Counted strings won hands down on anything but strings so short the length was a large percentage of overall size. Yet...since nobody reads the literature, we have all continued to suffer.
  - ninkendo 2y ago
    The late 70s was way too late for this… null terminated strings were already adopted by C and UNIX by then, and the rest is history.
    - eru 2y ago
      And sadly, we know that the C folks don't read papers; otherwise they wouldn't have come up with Go later.
- IgorPartola 2y ago
  Yes except when no. Imagine you are writing performance sensitive code. You want to get a substring from a string, one that is not going to live outside your hot loop. In standard C you can just reference a part of a string with a pointer offset. All standard functions will continue working and you didn’t have to make any calls outside of your loop, not to the allocator, not to memcopy, nothing. With strings being objects that are prefixed by a header cannot do this. At a minimum you need to allocate a new header, if not the whole string. Yes that’s the safer route but also a lot less performant. Most crucially, you can build the header string implementation on top of C strings. You cannot do the opposite. Realistically though C strings (aka null terminated strings) are just not a great thing because of the null termination. For my money, I would prefer to just use untermianted arrays and a separate size variable, as well as wide character strings for actual display stuff. This way all the interop must include string lengths (or some other way to determine length), and all internal stuff may be just ASCII but must not leave your internal logic and never be shown to the user.
  - BiteCode_dev 2y ago
    Yes but that's the rare case. The rare case should be possible, just not the default. In Rust, you would make custom string handling unsafe for the bottleneck.
    - IgorPartola 2y ago
      Rare for whom? Doing a lot of kernels or embedded development lately?
      - BiteCode_dev 2y ago
        kernels or embedded development is rare compared to web dev, app dev, cli tooling, automation, etc. In fact, it's pretty damn niche. And rust is a general language, so it favors the most common case, but let the niche case be possible.
  - AaronFriel 2y ago
    > Imagine you are writing performance sensitive code. You want to get a substring from a string, one that is not going to live outside your hot loop. In standard C you can just reference a part of a string with a pointer offset. If you want your substring to terminate in the same place as the original, at a null terminator. But that sadly is almost never the case, and as many C practitioners know, references like this are often unsafe and so APIs that substring tend to copy. That's just what they have to do to pass address sanitizer and static analysis checks. If you want arbitrary views on a null terminated string, well, it's no longer null terminated and that's just the start of your problems in C. In languages like Rust and Go, taking a view of a string or array is safe and doesn't copy the underlying data or require an allocation. So if you are writing performance sensitive code where substrings are a major contributor to CPU cycles, best go with those language (or C++) rather than C.
- JonChesterfield 2y ago
  48bit address space and 128bit of return value on systemv make pointer, size, 32bit of capacity-past-the-end attractive on x64. Specifically ptr, size as u64 with the extra capacity stored across the high 16 bits of each of them.
  - Tuna-Fish 2y ago
    What do you do when you have a string that's longer than 2^32 that gets truncated to len=0? Instantly freeing the buffer might not be what the user wants, if they intend to immediately reuse it for an another very long string, for example. I think that's a pretty bad case of premature optimization, especially because the first CPUs with 57 bit support are now hitting mainstream. Just use 3 words, it's not that much extra space.
    - JonChesterfield 2y ago
      Realloc/remap down to 4gb in that case sounds OK to me. > 4gb allocated from a structure which can't do any resizing seems moderately unlikely, but sure, I guess free is also correct in that case. Two 64 bit values can be returned in registers on the systemv x64 abi, three get passed as a pointer to stack memory. It's an optimisation but I think it's a valid one. 57 bit address space has been coming any year now for maybe a decade, I'll worry about that when it happens.
- eru 2y ago
  > Sadly, switching costs are massive and every programming language is pretty much struck with the string they started with. Haskell is _almost_ flexible enough to be able to use a different string than the one it started with. The language itself actually is flexible enough, but many of the libraries are not. The main thing making Haskell flexible enough is that a literal like "foo" can be statically determined to be the right string type that you want to use. (And that happens at compile time, it's not a runtime conversion.)
ThreatSystems 2y ago
Up to
msla 2y ago
Yes, his arrays which have length as an immutable part of their type certainly prevent certain kinds of bugs. Too bad about making it impossible to write generic array-handling subroutines, even if you accept the generally inexpressive type system as a given.

There are a whole bunch of other things I dislike about Pascal, but in this one he was just undeniably correct. Worrying about 1-3 extra bytes (on the 16 and 32 bit platforms where it potentially mattered) was just not worth all the issues that null-terminated strings brought with them. My current favorite string implementations are the various compact string crates for Rust. Generally, you want a string to be able to do at least three things: - Pointer, length, capacity tuple, for heap-allocated strings, 24B on x64. - String inlined into the 24B buffer. - Pointer, length tuple where pointer points to rodata. You can do any of that and the discriminator in 24B, given the healthy assumption that all strings are shorter than 2^63. Sadly, switching costs are massive and every programming language is pretty much struck with the string they started with. Hopefully, whatever comes next can crib from smartstring or compact_str or the like.

gumby 2y ago
There was a paper on this in the late 70s from the Cedar group at PARC. This was back when computer science papers were actual scientific papers, so full of analysis of different alogorithms' performance with counted vs delimited strings. Counted strings won hands down on anything but strings so short the length was a large percentage of overall size. Yet...since nobody reads the literature, we have all continued to suffer.
- ninkendo 2y ago
  The late 70s was way too late for this… null terminated strings were already adopted by C and UNIX by then, and the rest is history.
  - eru 2y ago
    And sadly, we know that the C folks don't read papers; otherwise they wouldn't have come up with Go later.
IgorPartola 2y ago
Yes except when no. Imagine you are writing performance sensitive code. You want to get a substring from a string, one that is not going to live outside your hot loop. In standard C you can just reference a part of a string with a pointer offset. All standard functions will continue working and you didn’t have to make any calls outside of your loop, not to the allocator, not to memcopy, nothing. With strings being objects that are prefixed by a header cannot do this. At a minimum you need to allocate a new header, if not the whole string. Yes that’s the safer route but also a lot less performant. Most crucially, you can build the header string implementation on top of C strings. You cannot do the opposite. Realistically though C strings (aka null terminated strings) are just not a great thing because of the null termination. For my money, I would prefer to just use untermianted arrays and a separate size variable, as well as wide character strings for actual display stuff. This way all the interop must include string lengths (or some other way to determine length), and all internal stuff may be just ASCII but must not leave your internal logic and never be shown to the user.
- BiteCode_dev 2y ago
  Yes but that's the rare case. The rare case should be possible, just not the default. In Rust, you would make custom string handling unsafe for the bottleneck.
  - IgorPartola 2y ago
    Rare for whom? Doing a lot of kernels or embedded development lately?
    - BiteCode_dev 2y ago
      kernels or embedded development is rare compared to web dev, app dev, cli tooling, automation, etc. In fact, it's pretty damn niche. And rust is a general language, so it favors the most common case, but let the niche case be possible.
- AaronFriel 2y ago
  > Imagine you are writing performance sensitive code. You want to get a substring from a string, one that is not going to live outside your hot loop. In standard C you can just reference a part of a string with a pointer offset. If you want your substring to terminate in the same place as the original, at a null terminator. But that sadly is almost never the case, and as many C practitioners know, references like this are often unsafe and so APIs that substring tend to copy. That's just what they have to do to pass address sanitizer and static analysis checks. If you want arbitrary views on a null terminated string, well, it's no longer null terminated and that's just the start of your problems in C. In languages like Rust and Go, taking a view of a string or array is safe and doesn't copy the underlying data or require an allocation. So if you are writing performance sensitive code where substrings are a major contributor to CPU cycles, best go with those language (or C++) rather than C.
JonChesterfield 2y ago
48bit address space and 128bit of return value on systemv make pointer, size, 32bit of capacity-past-the-end attractive on x64. Specifically ptr, size as u64 with the extra capacity stored across the high 16 bits of each of them.
- Tuna-Fish 2y ago
  What do you do when you have a string that's longer than 2^32 that gets truncated to len=0? Instantly freeing the buffer might not be what the user wants, if they intend to immediately reuse it for an another very long string, for example. I think that's a pretty bad case of premature optimization, especially because the first CPUs with 57 bit support are now hitting mainstream. Just use 3 words, it's not that much extra space.
  - JonChesterfield 2y ago
    Realloc/remap down to 4gb in that case sounds OK to me. > 4gb allocated from a structure which can't do any resizing seems moderately unlikely, but sure, I guess free is also correct in that case. Two 64 bit values can be returned in registers on the systemv x64 abi, three get passed as a pointer to stack memory. It's an optimisation but I think it's a valid one. 57 bit address space has been coming any year now for maybe a decade, I'll worry about that when it happens.
eru 2y ago
> Sadly, switching costs are massive and every programming language is pretty much struck with the string they started with. Haskell is _almost_ flexible enough to be able to use a different string than the one it started with. The language itself actually is flexible enough, but many of the libraries are not. The main thing making Haskell flexible enough is that a literal like "foo" can be statically determined to be the right string type that you want to use. (And that happens at compile time, it's not a runtime conversion.)

ThreatSystems 2y ago

Up to

msla 2y ago

Yes, his arrays which have length as an immutable part of their type certainly prevent certain kinds of bugs. Too bad about making it impossible to write generic array-handling subroutines, even if you accept the generally inexpressive type system as a given.