7 ms·

Sorry to say, but these hit close to home for me. A lot of the synchronization paradigms in Go are easy to misuse, but lead the author into thinking it's okay

Sorry to say, but these hit close to home for me. A lot of the synchronization paradigms in Go are easy to misuse, but lead the author into thinking it's okay. the WaitGroup one is particularly poignant for me, since the race detector doesn't catch it. I'll add one other data race goof: atomic.Value. Look at the implementation. Unlike pretty much every other language I've seen, atomic.Value isn't really atomic, since the concrete type can't ever change after being set. This stems from fact that interfaces are two words rather than one, and they can't be (hardware) atomically set. To fix it, Go just documents "hey, don't do that", and then panics if you do.

morelisp 4y ago
> atomic.Value isn't really atomic, since the concrete type can't ever change after being set. How does this mean it's non-atomic? As far as I know you can still never Load() a partial Store(). (Also, even if it was possible, this would never be a good idea...)
- bumper_crop 4y ago
  That's why I opened with "Look at the implementation". Go is unable to store the type and the pointer at the same time, so it warps what "atomic" means. Pretty much every other language has atomic mean "one of these will win, one will lose". Go says "one will win, one will panic and destroy the goroutine. In fact, it's even worse than that. If the Store() caller goes to sleep between setting the type and storing the pointer, it causes every Goroutine that calls Load() to block. They can't make forward progress if the store caller hangs.
  - fastest963 4y ago
    This is why all the examples call Store immediately with a zero value of the type.
    - bumper_crop 4y ago
      https://go.dev/play/p/xolc9oPwA0C https://go.dev/play/p/xolc9oPwA0C Interfaces don't have a zero type, which means that we can't have an atomic.Value which stores Shape. Atomic Value would be much easier to reason about if it had store semantics similar to a regular `var foo Shape = ...`. One of the other comment threads talked about generics helping this, so maybe there is hope.
      - deleted 4y ago
        [deleted]
      - yencabulator 4y ago
        Parent means var bestShape atomic.Value bestShape.Store((*Circle)(nil))
        morelisp 4y ago
        Which will store it as a *Circle, and only allow more *Circles, not Shapes. That part of GP’s claim is correct. It just had nothing to do with atomicity; it means something specific, not just “I like the failure mode.”
        yencabulator 4y ago
        That's pretty easy to workaround: type shapeContainer struct { Shape } The usual way to use atomic.Value is by writing strongly-typed wrappers anyway, so that doesn't affect your codebase beyond about 3 lines.
  - deleted 4y ago
    [deleted]
  - morelisp 4y ago
    Abend is a fairly normal and in many ways best way to "lose" in a race. It's fine, it's atomic.
    - deleted 4y ago
      [deleted]
  - tapirl 4y ago
    > Pretty much every other language has atomic mean "one of these will win, one will lose" Could you elaborate how "much every other language" implement it?
    - deleted 4y ago
      [deleted]
  - yencabulator 4y ago
    > If the Store() caller goes to sleep between setting the type and storing the pointer, it causes every Goroutine that calls Load() to block. Where does this go to sleep: https://cs.opensource.google/go/go/+/refs/tags/go1.18.3:src/sync/atomic/value.go;l=47-83 https://cs.opensource.google/go/go/+/refs/tags/go1.18.3:src/... It looks like a CAS busy loop with preemption disabled, to me.
    - bumper_crop 4y ago
      Make sure to read between the lines. It only looks like a busy loop. Remember, the OS can pause and preempt your thread at any time. This is a real and likely event.
      - mbnull 4y ago
        By reading the lines and not between them, you could read these two lines: runtime_procPin() and runtime_procUnpin(). With explicit comments that these pause preemption.
Groxx 4y ago
The lack of generics has forced all Go concurrency to be intrusive (i.e. implemented by the person using literally any concurrency), and yeah. It's horrifyingly error-prone in my experience. It means everyone needs to be an expert, and lol, everyone is not an expert. Generics might save us from the simple, mechanical flaws. Expect to see `Locker<T>` and `Atomic<T>` types cropping up. And unbounded buffered thread-safe queues backing channels. Etc. I'm very, very much looking forward to it. --- edited to rant more --- I also really wonder where all these "go makes concurrency a first-class concept" claims come from, because I see it quite a few places, and I feel like it's making some very strong implied claims that absolutely do not exist. Go has channels and select. That's neat. But on the other hand it has threads... but no thread handles. It has implicit capturing of closures. It has ambiguous value vs pointer semantics. It (style- and ergonomic-wise) encourages field references, which have no way to enforce mutexes or atomics. It has had crippled lock APIs that effectively force use of channels for... I don't know, philosophical reasons? Go is abnormally dangerous when it comes to concurrency IMO. The race detector does an amazing job helping you discover it, but it's very easy to not use it or not take full advantage of it (i.e. non-parallel tests), and few run their production services with the race detector enabled. Because if they did, it would crash all the time, because there are an absurd amount of races in nearly all of the popular libraries (and in common use of those libraries, because concurrency is not a first-class citizen and you can't tell when it's happening / when it shouldn't happen).
- hexxagone 4y ago
  Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. "go makes concurrency a first-class concept" I think it usually refers to goroutines being built in the language. "Go is abnormally dangerous when it comes to concurrency IMO". Personnally, it has not been my experience with Go concurrency. However I have hit some issues when trying to ocrhestrate tasks via channels and ended up resorting to atomics to do the job.
  - saghm 4y ago
    > Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. This doesn't stop there being "task handles" then, though? I think the point GP was making is that something that in most languages would be simple methods on a handle like "wait for this task to finish" or "stop this task" instead need to be done manually in Go with channels (or potentially `Context` in the latter case, although that was a later addition to the standard library). It doesn't really matter whether you call it a thread or a task; either way, it would be nice to get some return value from spawning some background operation and being able to use it to directly interact with it. I agree with GP that it does seem like an odd omission, since I haven't really heard any actual practical explanation for it.
avgcorrection 4y ago
This is Atomic* * Just don’t be an idiot. Worse is better.
nindalf 4y ago
On atomics in Go, the beta for Go 1.19 was released an hour ago (https://groups.google.com/g/golang-announce/c/SNruPJUSFz0?pli=1 https://groups.google.com/g/golang-announce/c/SNruPJUSFz0?pl...). > The sync/atomic package defines new atomic types Bool, Int32, Int64, Uint32, Uint64, Uintptr, and Pointer. These types hide the underlying values so that all accesses are forced to use the atomic APIs. Pointer also avoids the need to convert to unsafe.Pointer at call sites. Int64 and Uint64 are automatically aligned to 64-bit boundaries in structs and allocated data, even on 32-bit systems. Go 1.19 is expected to release in August.

morelisp 4y ago

> atomic.Value isn't really atomic, since the concrete type can't ever change after being set. How does this mean it's non-atomic? As far as I know you can still never Load() a partial Store(). (Also, even if it was possible, this would never be a good idea...)

bumper_crop 4y ago
That's why I opened with "Look at the implementation". Go is unable to store the type and the pointer at the same time, so it warps what "atomic" means. Pretty much every other language has atomic mean "one of these will win, one will lose". Go says "one will win, one will panic and destroy the goroutine. In fact, it's even worse than that. If the Store() caller goes to sleep between setting the type and storing the pointer, it causes every Goroutine that calls Load() to block. They can't make forward progress if the store caller hangs.
- fastest963 4y ago
  This is why all the examples call Store immediately with a zero value of the type.
  - bumper_crop 4y ago
    https://go.dev/play/p/xolc9oPwA0C https://go.dev/play/p/xolc9oPwA0C Interfaces don't have a zero type, which means that we can't have an atomic.Value which stores Shape. Atomic Value would be much easier to reason about if it had store semantics similar to a regular `var foo Shape = ...`. One of the other comment threads talked about generics helping this, so maybe there is hope.
    - deleted 4y ago
      [deleted]
    - yencabulator 4y ago
      Parent means var bestShape atomic.Value bestShape.Store((*Circle)(nil))
      - morelisp 4y ago
        Which will store it as a *Circle, and only allow more *Circles, not Shapes. That part of GP’s claim is correct. It just had nothing to do with atomicity; it means something specific, not just “I like the failure mode.”
        yencabulator 4y ago
        That's pretty easy to workaround: type shapeContainer struct { Shape } The usual way to use atomic.Value is by writing strongly-typed wrappers anyway, so that doesn't affect your codebase beyond about 3 lines.
- deleted 4y ago
  [deleted]
- morelisp 4y ago
  Abend is a fairly normal and in many ways best way to "lose" in a race. It's fine, it's atomic.
  - deleted 4y ago
    [deleted]
- tapirl 4y ago
  > Pretty much every other language has atomic mean "one of these will win, one will lose" Could you elaborate how "much every other language" implement it?
  - deleted 4y ago
    [deleted]
- yencabulator 4y ago
  > If the Store() caller goes to sleep between setting the type and storing the pointer, it causes every Goroutine that calls Load() to block. Where does this go to sleep: https://cs.opensource.google/go/go/+/refs/tags/go1.18.3:src/sync/atomic/value.go;l=47-83 https://cs.opensource.google/go/go/+/refs/tags/go1.18.3:src/... It looks like a CAS busy loop with preemption disabled, to me.
  - bumper_crop 4y ago
    Make sure to read between the lines. It only looks like a busy loop. Remember, the OS can pause and preempt your thread at any time. This is a real and likely event.
    - mbnull 4y ago
      By reading the lines and not between them, you could read these two lines: runtime_procPin() and runtime_procUnpin(). With explicit comments that these pause preemption.

Groxx 4y ago

The lack of generics has forced all Go concurrency to be intrusive (i.e. implemented by the person using literally any concurrency), and yeah. It's horrifyingly error-prone in my experience. It means everyone needs to be an expert, and lol, everyone is not an expert. Generics might save us from the simple, mechanical flaws. Expect to see `Locker<T>` and `Atomic<T>` types cropping up. And unbounded buffered thread-safe queues backing channels. Etc. I'm very, very much looking forward to it. --- edited to rant more --- I also really wonder where all these "go makes concurrency a first-class concept" claims come from, because I see it quite a few places, and I feel like it's making some very strong implied claims that absolutely do not exist. Go has channels and select. That's neat. But on the other hand it has threads... but no thread handles. It has implicit capturing of closures. It has ambiguous value vs pointer semantics. It (style- and ergonomic-wise) encourages field references, which have no way to enforce mutexes or atomics. It has had crippled lock APIs that effectively force use of channels for... I don't know, philosophical reasons? Go is abnormally dangerous when it comes to concurrency IMO. The race detector does an amazing job helping you discover it, but it's very easy to not use it or not take full advantage of it (i.e. non-parallel tests), and few run their production services with the race detector enabled. Because if they did, it would crash all the time, because there are an absurd amount of races in nearly all of the popular libraries (and in common use of those libraries, because concurrency is not a first-class citizen and you can't tell when it's happening / when it shouldn't happen).

hexxagone 4y ago
Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. "go makes concurrency a first-class concept" I think it usually refers to goroutines being built in the language. "Go is abnormally dangerous when it comes to concurrency IMO". Personnally, it has not been my experience with Go concurrency. However I have hit some issues when trying to ocrhestrate tasks via channels and ended up resorting to atomics to do the job.
- saghm 4y ago
  > Go does not have threads but something like "tasks". The fact that no thread handle is exposed allows for transparently moving these tasks across threads if the scheduler decides so. This doesn't stop there being "task handles" then, though? I think the point GP was making is that something that in most languages would be simple methods on a handle like "wait for this task to finish" or "stop this task" instead need to be done manually in Go with channels (or potentially `Context` in the latter case, although that was a later addition to the standard library). It doesn't really matter whether you call it a thread or a task; either way, it would be nice to get some return value from spawning some background operation and being able to use it to directly interact with it. I agree with GP that it does seem like an odd omission, since I haven't really heard any actual practical explanation for it.

avgcorrection 4y ago

This is Atomic* * Just don’t be an idiot. Worse is better.

nindalf 4y ago

On atomics in Go, the beta for Go 1.19 was released an hour ago (https://groups.google.com/g/golang-announce/c/SNruPJUSFz0?pli=1 https://groups.google.com/g/golang-announce/c/SNruPJUSFz0?pl...). > The sync/atomic package defines new atomic types Bool, Int32, Int64, Uint32, Uint64, Uintptr, and Pointer. These types hide the underlying values so that all accesses are forced to use the atomic APIs. Pointer also avoids the need to convert to unsafe.Pointer at call sites. Int64 and Uint64 are automatically aligned to 64-bit boundaries in structs and allocated data, even on 32-bit systems. Go 1.19 is expected to release in August.