8 ms·

In Go, pointers (mostly) don't go with slices in practice

146 points by deafcalculus 5y ago

friendzis 5y ago
Problems with memory addressing are bound to happen with asynchronous memory manipulation: threads in C/C++ or concurrent GC in go. The biggest issue here is that memory manipulation happens behind the scenes and the runtime does not offer effective tools to synchronise memory state manipulations.
- masklinn 5y ago
  That’s not really the issue here. The issue is that slices are mutation proxies to the backing array, so in a concurrent context you’re sharing mutable state with all that implies. And worse because of the behaviour of append that sharing is not systematic, so it can look like there’s no mutable state sharing… until you append on a slice with leftover capacity and now there is. The GC only operates on “dead” allocations (afaik it remains non-moving) so it’s not a concern for now.
Animats 5y ago
Well, of course slices work that way. Think about what happens if you have a reference to a slice in an array and you shrank the array to 0. You've just created a dangling pointer. In Go, you get a stable version of the old data, and the garbage collector tracks that you still have a reference to it. This is safe, but confuses some people. In Rust, the borrow checker won't let you modify the array while you have a reference to a slice of it. So you can't do this at all. In C++, you get a mention on the Department of Homeland Security's US-CERT site.
- friendzis 5y ago
  > This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference usually means quite an opposite. So it is "safe" in a sense that the pointer points to valid data, but is "incorrect" in a sense that it does wrong thing without warning. Sure, one could argue that value-returning modification functions are a giveaway of invalidated data. But this is not C, go has reference counting and instead of "forcing" underlying array to maintain the same address it just keeps original pointer pointing to dereferenceable, but wrong data.
  - tsimionescu 5y ago
    The problem here is identical to the problem of pointers to array elements in C after a 'realloc', except that Go at least guarantees that you're not going to modify some other object's memory. Of course, since append neither guarantees nor prevents a copy, the semantics of modifying a value through a pointer to a slice element after an append are unspecified, so it is not a useful construct.
    - masklinn 5y ago
      > except that Go at least guarantees that you're not going to modify some other object's memory. Or that you’re way off in UB (UAF) land.
  - foldr 5y ago
    > The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". There’s no need for Go to copy anything in the circumstance the OP described. It just doesn’t shrink the underlying array.
  - heleninboodler 5y ago
    > The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container" No, there is no mention of a "snapshot". You get a reference to the current backing array, which may or may not continue being used by the slice (depending on reallocations). You're pointing to the live slice backing array, and the values in it may change if someone else is manipulating the slice, up to the point where the slice backing array must be reallocated, at which point you'll continue pointing to the old backing array and be keeping it from getting GC'd.
    - db48x 5y ago
      And that’s really the problem with it. If you want to ensure that you have exclusive access to the element(s), then you have to explicitly copy them first or you get silent data corruption. And if you want to ensure that multiple things have access to the elements, then you have to avoid reallocations or you get silent data loss. No matter what you’re doing, a pointer to an element of an array or slice is usually the wrong thing in Go. The language would be better off without them.
- 37ef_ced3 5y ago
  In Go, a []int ("slice of int") is just a C struct like this, passed by value: struct intSlice { int* addr; int len; int cap; }; The memory at addr is not owned by the slice. All the slice operations are simply notation for manipulating the struct. Go's garbage collection makes the whole thing work well. This can be confusing if you're used to C++'s std::vector (which owns the memory) or Python's slices. Go's slices are a shallow pointer/length system exactly like is used in C all the time. For example: void sort(int* addr, int len); becomes func sort(a []int) A Go slice is just a formalization of C's pointer/length idiom, with terse notation for manipulation.
- benibela 5y ago
  In Pascal, there are no slices No slices, no problems If you need to work with a part of a string, you can make two ordinary integer variables for offset and length
  - jayar95 5y ago
    Cool
    - benibela 5y ago
      Actually, Pascal has slices They are just so obscure, I forgot about them and no one uses them. No users, no problems They are not part of the normal type system. You cannot declare a variable of a type slice. Nor a field. But when a parameter of a function is an (open) array, you can call the function with a slice of an existing array That avoids most problems The backing array exists when the function is called, and the function cannot store the slice, so the slice cannot outlive the array. It is like the function borrows the array. Only problem is if the function gets another reference to the array, through a global variable or something, and resizes it
skybrian 5y ago
It would be more accurate to say that pointers don’t work with append() or any other way of growing an array, since they all depend on reallocating it sometimes. Incidentally, this is equally true of creating additional pointers or slices pointing into an growable array. They aren’t safe after the next append(). If you grow an array then you need to refer to its elements using array indices. But if you have a fixed-length array, or between appends, you can use both pointers and slices to point to parts of it, and it will work fine, This all works the same as C if you think of a slice as a glorified pointer. If you’re thinking of a slice as a JavaScript array then you’ll have trouble.
- masklinn 5y ago
  > If you’re thinking of a slice as a JavaScript array then you’ll have trouble. The problem of Go is that it has you uses slices as that as well as actual slices, there is no vector type. So the confusion is very much understandable and to be expected.
xiaq 5y ago
The author seems confused. The following is simply not true: When you take a pointer to a slice, you get a pointer to the current version of this tuple of information for the slice. This pointer may or may not refer to a slice that anyone else is using; for instance: ps := &s s = append(s, 50) At this point, '*ps' may or may not be the same thing as 's', and so it might or might not have the new '50' element at the end. No, *ps will always be the same as s, because ps is a pointer so it carries no information other than the address of s. The author seems to have failed to distinguish the operation of copying a fat pointer (which opens the possibility of divergence) and the operation of the taking the address of a fat pointer (which involves no copying, so divergence is not possible - where would the divergent version be stored?). See this code snippet: https://play.golang.org/p/tdb-O8a6hDN https://play.golang.org/p/tdb-O8a6hDN
- Thorrez 5y ago
  Yep, that immediately stood out to me too. s is a local variable (or global, doesn't matter). ps simply points to that local variable. You can modify the local variable all day long and ps will still point to it, not some old version of it.
  - watt 5y ago
    the line `s = append(s, 50)` redefines what "s" actually is. And after this line `ps` points to some previous version of what "s" used to be.
    - tsimionescu 5y ago
      No, that line modifies the value of the variable s to represent the value of a new slice returned by append (assuming append did need to reallocate). Any pointer to s will point to this new value. A variable in Go always maintains its address after it is allocated. Assignments to that variable copy the assigned value to the original address. Somewhat unhelpfully, this rule is even true for iteration variable - when you write 'for i,v := range arr {...}', i and v get allocated a memory address, and they get successively assigned the indices and values in arr. This implies that each element in arr is copied into the value of v, and that doing &v inside the loop gives you a completely different pointer than &arr[i]. In fact, &v will always point to the last element of arr after the loop is over.
Thorrez 5y ago
>Honestly, this is a strange and peculiar situation, although Go programmers have acclimatized to it. To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Well, it exists, and it "works" in the sense that it yields results and doesn't crash your program, but it doesn't "work" in the sense of doing what you'd actually want. I think it works in the same way as a pointer to std::span in C++. (Or pointer to std::string_view with the exception that std::string_view doesn't allow modification of the elements.) I guess the difference is that std::span doesn't let you append to the backing array through the std::span directly. So with C++ you have to write more code which makes it clearer what's happening.
- edflsafoiewq 5y ago
  C++ has std::vector, which is one level of abstraction above a slice; you push to a vector, and maybe the backing slice changes, but it's still the same vector. Go is unusual in not having a equivalent of vector.
  - pjmlp 5y ago
    Not really, if reallocation takes place. So if you got a pointer to a vector element, it now points to garbage.
    - simiones 5y ago
      I guess the difference is this: C++: std::vector<int> v {1, 2, 3}; void foo(std::vector<int> *ref) { ref.push_back(4); } foo(&v); //v[3] == 4 is true here Go: v := []int{1, 2, 3} func foo(ref *[]int) { append(ref, 4) } foo(&v) //v[3] == 4 may or may not be true here. Pointers to elements in the vector do indeed have the same problems both in Go and C++ (except for memory safety).
      - pjmlp 5y ago
        Ah ok, although for the audience not versed in C++ the correct code is, void foo(std::vector<int> *ref) { ref->push_back(4); } foo(&v); or void foo(std::vector<int> &ref) { ref.push_back(4); } foo(v);
barsonme 5y ago
Outside of a few very specific situations, if you’re working with a pointer to a slice or string you’re doing something very wrong. Slices are “fat” pointers. > To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea Write Go in Go, don’t write C in Go. (Which applies to every language, tbh.)
deleted 5y ago
[deleted]
vanderZwan 5y ago
I get the confusion (to people unfamiliar with pointers) about pointers to elements in reference types, but why would anyone want pointers to reference types? They're basically pointers with extra features
- simiones 5y ago
  There are no such things as "reference types" in Go, though slices do have the extremely odd behavior that they can take the value `nil`, similarly to interfaces and unlike any other non-pointer type in Go. Pointers to "fat pointer" types are sometimes needed, just like you sometimes need pointers-to-pointers.
  - Joker_vD 5y ago
    What about maps? Non-nil maps sure seem like they're "reference types", look at [0]. [0] https://play.golang.org/p/xtY_ASExQzR https://play.golang.org/p/xtY_ASExQzR
    - masklinn 5y ago
      "reference types" is a very specific concept from a specific category of languages: types which are always heap-allocated and sitting behind an invisible (and un-interactible) pointer. But Go doesn't have that distinction, and has actual pointers you can use directly. A map is just a heap-allocated structure sitting behind a pointer. If you create a type which is a pointer to a struct, sure you can say you've built a reference type if you want, but that doesn't actually say much to anyone, because that's not a distinction the language makes, unlike Java or C#.
      - Joker_vD 5y ago
        > types which are always heap-allocated and sitting behind an invisible (and un-interactible) pointer > A map is just a heap-allocated structure sitting behind a pointer. And the difference is?.. Because maps in Go behave exactly as if they were un-referenceable pointers to the hidden, heap-allocated hashtables.
kubb 5y ago
Considering the confusion of the author, it seems like not all junior programmers can understand Go, which makes me wonder: is it simple enough? One pitfall is when getting a slice by value in a function. You cannot be sure that someone is not going to pass you a slice into a buffer that they themselves use, so you have to be careful when appending - someone might be using that buffer and you’ll be writing over it.
- gobookdev 5y ago
  No it's not, it can be even simpler, but it's a great start and evolution
- kansface 5y ago
  I don't feel its a matter of complexity per se, more like bad UX. We could imagine exposing why this doesn't work to end users instead oh hiding behind a leaky abstraction.
- masklinn 5y ago
  > which makes me wonder: is it simple enough? Obviously not given Go was never simple in the first place. Go was built to be easy — for a certain value of easy. Simple tools are often not easy, and simple programming languages are definitely not easy: they tend to be built out of a small set of very powerful concepts which are directly exposed to the language user, said language user has close to the power of the language designer in building abstractions. Lisps, and Smalltalks and Forths are simple, which means they are mind-bending and not only can you build what you want out of them (hello turing equivalence) you can build how you want. And of course the simplest of languages (the turing tarpits) are barely usable at all.
  - kortex 5y ago
    Continuing that thought along the line of "Simple made easy," I think ease has a simplicity all its own. A simple language, e.g. C or lisp, simple in that their grammar is simple, are definitely less easy for the programmer, than say Go. But C is not simple as an experience, since it forces the dev to mentally complect so many concepts in order to get things done: macros, memory management, etc. Lisp is complex in a different way: metaprogamming, DSLs, and deep abstractions are the norm. So simplicity/complexity tends to be something of a whack-a-mole. It's a lower bound, much like the uncertainty principle; you can always add complexity. Go makes a lot of choices that try to really optimize the user_complexity * language_complexity product.
sly010 5y ago
Go-s behavior is the ONLY sensible one in _any_ language that supports pointers. This is a faster and safe(er) way. You simply cannot modify (move or reallocate) a data-structure that has pointers pointing to it without invalidating all pointers. Not in C++, not in any language with pointers (that I know if). This is not "strange and peculiar". What's "strange and peculiar" is that the author thinks that doing this in C++ is a "perfectly decent idea". In fact it's a huge no no and more often than not will crash. Edit: we would both learn something if you offered a counter example instead of downvoted.
- Diggsey 5y ago
  It's clearly not the "only sensible one" given that no other language works the way Go does here. In C++ this is UB, which is bad, but in keeping with the rest of the language. In Rust, the compiler will not allow you to do any operation that would re-allocate the backing store whilst there are outstanding references into it. In most other languages (eg. Java, C#, python, etc.), you can't get a pointer/reference to an array index, only a pointer/reference to the item at that index at the time you looked. Go's decision here is especially weird given that this same thing is seemingly prevented for maps (why the inconsistency?). Given the three goals of memory-safety, "simplicity" and performance, it's true there are not many other options Go could have chosen, but personally I think Go's interpretation of "simplicity" is incredibly warped: it's a kind of superficial simplicity that leads to programs that are much more complicated to reason about.
dilap 5y ago
> To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Ah, I would beg to differ! You should never be taking pointers to a dynamically resizable array, in any language. (Well, caveat, its fine if you do it only for a time period where you know the array won't be growing.) The whole point of a dynamically resizable array is that its addresses can change! If you did this in C++, you'd get undefined behavior. In Go you get "safe" but probably-not-what-you-wanted behavior. In Rust it simply wouldn't be possible (w/o unsafe), and you'd have to use indices (which is the correct thing to do, in any language).
klodolph 5y ago
> Honestly, this is a strange and peculiar situation, although Go programmers have acclimatized to it. To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Well, it exists, and it "works" in the sense that it yields results and doesn't crash your program, but it doesn't "work" in the sense of doing what you'd actually want. In C++, what happens is that the "iterators are invalidated" when you add something to an array. This is CONSTANTLY a source of bugs and frustration for new programmers. In C++, it may yield results or crash your program, and you are never sure quite which will happen. The best you can do (as a senior engineer) is design your software to avoid ever creating this situation in the first place and throw address sanitizer at things to try and catch them when they arise. The difference with Go is that in Go this will never result in a memory error. Strictly speaking, the situation in Go is way better. I will take "incorrect behavior, but not a memory error" over "memory error" any day of the week. We may forget what it's like for new programmers, but for those of us who hang out on Discord channels, Stack Overflow, and Reddit giving people help with programming, simple things like iterator invalidation are a major pain point. "You have a memory error in your program", I say to someone. "Now that you know that you have a memory error, it is probably your highest priority to find and fix this error." And now you start walking someone through the steps of finding and fixing a memory error, which is nontrivial. You'll tell them about Address Sanitizer, GDB, and Valgrind, and you'll wish them luck.
- kevincox 5y ago
  I agree that it is better than C and C++ but not much better. C and C++ are "It is very easy to make this mistake which leads to undefined behaviour and a maybe incorrect program." Go is "It is very easy to make this mistake which leads to a maybe incorrect program." Yes, better! But the problem is still there. I much prefer the Rust solution where there is no common mistake.
  - klodolph 5y ago
    The catch is that code is hard to translate into Rust. "I have this code, you see... and it takes a couple mutable references into an array... how do I translate this into Rust?" There is no one-size-fits-all answer to that question. The code may be correct in C or C++, but the Rust type system may give you one hell of a hard time proving that it is correct to the Rust type system's satisfaction... so you refactor your code completely, or you use integer indexes into arrays rather than references, or you use unsafe code... I've written some amount of Rust code at this point. About half of the time, when I write a project in Rust, there comes a point at which I'm fighting with the type system. I feel like this should stop happening, at some point.
marcus_holmes 5y ago
I love Go, my favourite language. But I've been bitten before by passing slices around and then finding out that they got disassociated and are now pointing at two different backing arrays without telling me. I kinda know enough now to avoid this, but I have to be careful and remind myself it's a possibility. I'd love some built-in method to be able to tell whether altering a value in slice A will also alter the value in slice B (i.e. whether A and B are referring to the same backing array). As far as I'm aware there's no easy way of doing this in Go.
- deleted 5y ago
  [deleted]
Fire-Dragon-DoL 5y ago
s = append(s, elem) I read this immediately as "create a new copy of the original slice with one additional element", so I presumed that was the case. It would actually be shocking the opposite, if I could end up modifying the original one (before the append) with a pointer to the new s, which seems to be the case! Big gotcha there: treat slices as stateful at all time. Since it has an assignment operation, it must be creating something new, otherwise it would have been a method of the slice itself EDIT: I just realized the gotcha is not there at all, Go would consider the first slice to be of N length and the second slice of length N+1. Comparing the two slices would give an error at some point because one is shorter than the other, so the fact that the address changes or not is irrelevant. However I can see this becoming problematic with pointers, which proves the point of the article.

146 points by deafcalculus 5y ago

Problems with memory addressing are bound to happen with asynchronous memory manipulation: threads in C/C++ or concurrent GC in go. The biggest issue here is that memory manipulation happens behind the scenes and the runtime does not offer effective tools to synchronise memory state manipulations.

masklinn 5y ago
That’s not really the issue here. The issue is that slices are mutation proxies to the backing array, so in a concurrent context you’re sharing mutable state with all that implies. And worse because of the behaviour of append that sharing is not systematic, so it can look like there’s no mutable state sharing… until you append on a slice with leftover capacity and now there is. The GC only operates on “dead” allocations (afaik it remains non-moving) so it’s not a concern for now.

Animats 5y ago

Well, of course slices work that way. Think about what happens if you have a reference to a slice in an array and you shrank the array to 0. You've just created a dangling pointer. In Go, you get a stable version of the old data, and the garbage collector tracks that you still have a reference to it. This is safe, but confuses some people. In Rust, the borrow checker won't let you modify the array while you have a reference to a slice of it. So you can't do this at all. In C++, you get a mention on the Department of Homeland Security's US-CERT site.

friendzis 5y ago
> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference usually means quite an opposite. So it is "safe" in a sense that the pointer points to valid data, but is "incorrect" in a sense that it does wrong thing without warning. Sure, one could argue that value-returning modification functions are a giveaway of invalidated data. But this is not C, go has reference counting and instead of "forcing" underlying array to maintain the same address it just keeps original pointer pointing to dereferenceable, but wrong data.
- tsimionescu 5y ago
  The problem here is identical to the problem of pointers to array elements in C after a 'realloc', except that Go at least guarantees that you're not going to modify some other object's memory. Of course, since append neither guarantees nor prevents a copy, the semantics of modifying a value through a pointer to a slice element after an append are unspecified, so it is not a useful construct.
  - masklinn 5y ago
    > except that Go at least guarantees that you're not going to modify some other object's memory. Or that you’re way off in UB (UAF) land.
- foldr 5y ago
  > The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". There’s no need for Go to copy anything in the circumstance the OP described. It just doesn’t shrink the underlying array.
- heleninboodler 5y ago
  > The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container" No, there is no mention of a "snapshot". You get a reference to the current backing array, which may or may not continue being used by the slice (depending on reallocations). You're pointing to the live slice backing array, and the values in it may change if someone else is manipulating the slice, up to the point where the slice backing array must be reallocated, at which point you'll continue pointing to the old backing array and be keeping it from getting GC'd.
  - db48x 5y ago
    And that’s really the problem with it. If you want to ensure that you have exclusive access to the element(s), then you have to explicitly copy them first or you get silent data corruption. And if you want to ensure that multiple things have access to the elements, then you have to avoid reallocations or you get silent data loss. No matter what you’re doing, a pointer to an element of an array or slice is usually the wrong thing in Go. The language would be better off without them.
37ef_ced3 5y ago
In Go, a []int ("slice of int") is just a C struct like this, passed by value: struct intSlice { int* addr; int len; int cap; }; The memory at addr is not owned by the slice. All the slice operations are simply notation for manipulating the struct. Go's garbage collection makes the whole thing work well. This can be confusing if you're used to C++'s std::vector (which owns the memory) or Python's slices. Go's slices are a shallow pointer/length system exactly like is used in C all the time. For example: void sort(int* addr, int len); becomes func sort(a []int) A Go slice is just a formalization of C's pointer/length idiom, with terse notation for manipulation.
benibela 5y ago
In Pascal, there are no slices No slices, no problems If you need to work with a part of a string, you can make two ordinary integer variables for offset and length
- jayar95 5y ago
  Cool
  - benibela 5y ago
    Actually, Pascal has slices They are just so obscure, I forgot about them and no one uses them. No users, no problems They are not part of the normal type system. You cannot declare a variable of a type slice. Nor a field. But when a parameter of a function is an (open) array, you can call the function with a slice of an existing array That avoids most problems The backing array exists when the function is called, and the function cannot store the slice, so the slice cannot outlive the array. It is like the function borrows the array. Only problem is if the function gets another reference to the array, through a global variable or something, and resizes it

skybrian 5y ago

It would be more accurate to say that pointers don’t work with append() or any other way of growing an array, since they all depend on reallocating it sometimes. Incidentally, this is equally true of creating additional pointers or slices pointing into an growable array. They aren’t safe after the next append(). If you grow an array then you need to refer to its elements using array indices. But if you have a fixed-length array, or between appends, you can use both pointers and slices to point to parts of it, and it will work fine, This all works the same as C if you think of a slice as a glorified pointer. If you’re thinking of a slice as a JavaScript array then you’ll have trouble.

masklinn 5y ago
> If you’re thinking of a slice as a JavaScript array then you’ll have trouble. The problem of Go is that it has you uses slices as that as well as actual slices, there is no vector type. So the confusion is very much understandable and to be expected.

xiaq 5y ago

The author seems confused. The following is simply not true: When you take a pointer to a slice, you get a pointer to the current version of this tuple of information for the slice. This pointer may or may not refer to a slice that anyone else is using; for instance: ps := &s s = append(s, 50) At this point, '*ps' may or may not be the same thing as 's', and so it might or might not have the new '50' element at the end. No, *ps will always be the same as s, because ps is a pointer so it carries no information other than the address of s. The author seems to have failed to distinguish the operation of copying a fat pointer (which opens the possibility of divergence) and the operation of the taking the address of a fat pointer (which involves no copying, so divergence is not possible - where would the divergent version be stored?). See this code snippet: https://play.golang.org/p/tdb-O8a6hDN https://play.golang.org/p/tdb-O8a6hDN

Thorrez 5y ago
Yep, that immediately stood out to me too. s is a local variable (or global, doesn't matter). ps simply points to that local variable. You can modify the local variable all day long and ps will still point to it, not some old version of it.
- watt 5y ago
  the line `s = append(s, 50)` redefines what "s" actually is. And after this line `ps` points to some previous version of what "s" used to be.
  - tsimionescu 5y ago
    No, that line modifies the value of the variable s to represent the value of a new slice returned by append (assuming append did need to reallocate). Any pointer to s will point to this new value. A variable in Go always maintains its address after it is allocated. Assignments to that variable copy the assigned value to the original address. Somewhat unhelpfully, this rule is even true for iteration variable - when you write 'for i,v := range arr {...}', i and v get allocated a memory address, and they get successively assigned the indices and values in arr. This implies that each element in arr is copied into the value of v, and that doing &v inside the loop gives you a completely different pointer than &arr[i]. In fact, &v will always point to the last element of arr after the loop is over.

Thorrez 5y ago

edflsafoiewq 5y ago
C++ has std::vector, which is one level of abstraction above a slice; you push to a vector, and maybe the backing slice changes, but it's still the same vector. Go is unusual in not having a equivalent of vector.
- pjmlp 5y ago
  Not really, if reallocation takes place. So if you got a pointer to a vector element, it now points to garbage.
  - simiones 5y ago
    I guess the difference is this: C++: std::vector<int> v {1, 2, 3}; void foo(std::vector<int> *ref) { ref.push_back(4); } foo(&v); //v[3] == 4 is true here Go: v := []int{1, 2, 3} func foo(ref *[]int) { append(ref, 4) } foo(&v) //v[3] == 4 may or may not be true here. Pointers to elements in the vector do indeed have the same problems both in Go and C++ (except for memory safety).
    - pjmlp 5y ago
      Ah ok, although for the audience not versed in C++ the correct code is, void foo(std::vector<int> *ref) { ref->push_back(4); } foo(&v); or void foo(std::vector<int> &ref) { ref.push_back(4); } foo(v);

barsonme 5y ago

Outside of a few very specific situations, if you’re working with a pointer to a slice or string you’re doing something very wrong. Slices are “fat” pointers. > To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea Write Go in Go, don’t write C in Go. (Which applies to every language, tbh.)

deleted 5y ago

[deleted]

vanderZwan 5y ago

I get the confusion (to people unfamiliar with pointers) about pointers to elements in reference types, but why would anyone want pointers to reference types? They're basically pointers with extra features

simiones 5y ago
There are no such things as "reference types" in Go, though slices do have the extremely odd behavior that they can take the value `nil`, similarly to interfaces and unlike any other non-pointer type in Go. Pointers to "fat pointer" types are sometimes needed, just like you sometimes need pointers-to-pointers.
- Joker_vD 5y ago
  What about maps? Non-nil maps sure seem like they're "reference types", look at [0]. [0] https://play.golang.org/p/xtY_ASExQzR https://play.golang.org/p/xtY_ASExQzR
  - masklinn 5y ago
    "reference types" is a very specific concept from a specific category of languages: types which are always heap-allocated and sitting behind an invisible (and un-interactible) pointer. But Go doesn't have that distinction, and has actual pointers you can use directly. A map is just a heap-allocated structure sitting behind a pointer. If you create a type which is a pointer to a struct, sure you can say you've built a reference type if you want, but that doesn't actually say much to anyone, because that's not a distinction the language makes, unlike Java or C#.
    - Joker_vD 5y ago
      > types which are always heap-allocated and sitting behind an invisible (and un-interactible) pointer > A map is just a heap-allocated structure sitting behind a pointer. And the difference is?.. Because maps in Go behave exactly as if they were un-referenceable pointers to the hidden, heap-allocated hashtables.

kubb 5y ago

Considering the confusion of the author, it seems like not all junior programmers can understand Go, which makes me wonder: is it simple enough? One pitfall is when getting a slice by value in a function. You cannot be sure that someone is not going to pass you a slice into a buffer that they themselves use, so you have to be careful when appending - someone might be using that buffer and you’ll be writing over it.

gobookdev 5y ago
No it's not, it can be even simpler, but it's a great start and evolution
kansface 5y ago
I don't feel its a matter of complexity per se, more like bad UX. We could imagine exposing why this doesn't work to end users instead oh hiding behind a leaky abstraction.
masklinn 5y ago
> which makes me wonder: is it simple enough? Obviously not given Go was never simple in the first place. Go was built to be easy — for a certain value of easy. Simple tools are often not easy, and simple programming languages are definitely not easy: they tend to be built out of a small set of very powerful concepts which are directly exposed to the language user, said language user has close to the power of the language designer in building abstractions. Lisps, and Smalltalks and Forths are simple, which means they are mind-bending and not only can you build what you want out of them (hello turing equivalence) you can build how you want. And of course the simplest of languages (the turing tarpits) are barely usable at all.
- kortex 5y ago
  Continuing that thought along the line of "Simple made easy," I think ease has a simplicity all its own. A simple language, e.g. C or lisp, simple in that their grammar is simple, are definitely less easy for the programmer, than say Go. But C is not simple as an experience, since it forces the dev to mentally complect so many concepts in order to get things done: macros, memory management, etc. Lisp is complex in a different way: metaprogamming, DSLs, and deep abstractions are the norm. So simplicity/complexity tends to be something of a whack-a-mole. It's a lower bound, much like the uncertainty principle; you can always add complexity. Go makes a lot of choices that try to really optimize the user_complexity * language_complexity product.

sly010 5y ago

Go-s behavior is the ONLY sensible one in _any_ language that supports pointers. This is a faster and safe(er) way. You simply cannot modify (move or reallocate) a data-structure that has pointers pointing to it without invalidating all pointers. Not in C++, not in any language with pointers (that I know if). This is not "strange and peculiar". What's "strange and peculiar" is that the author thinks that doing this in C++ is a "perfectly decent idea". In fact it's a huge no no and more often than not will crash. Edit: we would both learn something if you offered a counter example instead of downvoted.

Diggsey 5y ago
It's clearly not the "only sensible one" given that no other language works the way Go does here. In C++ this is UB, which is bad, but in keeping with the rest of the language. In Rust, the compiler will not allow you to do any operation that would re-allocate the backing store whilst there are outstanding references into it. In most other languages (eg. Java, C#, python, etc.), you can't get a pointer/reference to an array index, only a pointer/reference to the item at that index at the time you looked. Go's decision here is especially weird given that this same thing is seemingly prevented for maps (why the inconsistency?). Given the three goals of memory-safety, "simplicity" and performance, it's true there are not many other options Go could have chosen, but personally I think Go's interpretation of "simplicity" is incredibly warped: it's a kind of superficial simplicity that leads to programs that are much more complicated to reason about.

dilap 5y ago

> To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Ah, I would beg to differ! You should never be taking pointers to a dynamically resizable array, in any language. (Well, caveat, its fine if you do it only for a time period where you know the array won't be growing.) The whole point of a dynamically resizable array is that its addresses can change! If you did this in C++, you'd get undefined behavior. In Go you get "safe" but probably-not-what-you-wanted behavior. In Rust it simply wouldn't be possible (w/o unsafe), and you'd have to use indices (which is the correct thing to do, in any language).

klodolph 5y ago

> Honestly, this is a strange and peculiar situation, although Go programmers have acclimatized to it. To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Well, it exists, and it "works" in the sense that it yields results and doesn't crash your program, but it doesn't "work" in the sense of doing what you'd actually want. In C++, what happens is that the "iterators are invalidated" when you add something to an array. This is CONSTANTLY a source of bugs and frustration for new programmers. In C++, it may yield results or crash your program, and you are never sure quite which will happen. The best you can do (as a senior engineer) is design your software to avoid ever creating this situation in the first place and throw address sanitizer at things to try and catch them when they arise. The difference with Go is that in Go this will never result in a memory error. Strictly speaking, the situation in Go is way better. I will take "incorrect behavior, but not a memory error" over "memory error" any day of the week. We may forget what it's like for new programmers, but for those of us who hang out on Discord channels, Stack Overflow, and Reddit giving people help with programming, simple things like iterator invalidation are a major pain point. "You have a memory error in your program", I say to someone. "Now that you know that you have a memory error, it is probably your highest priority to find and fix this error." And now you start walking someone through the steps of finding and fixing a memory error, which is nontrivial. You'll tell them about Address Sanitizer, GDB, and Valgrind, and you'll wish them luck.

kevincox 5y ago
I agree that it is better than C and C++ but not much better. C and C++ are "It is very easy to make this mistake which leads to undefined behaviour and a maybe incorrect program." Go is "It is very easy to make this mistake which leads to a maybe incorrect program." Yes, better! But the problem is still there. I much prefer the Rust solution where there is no common mistake.
- klodolph 5y ago
  The catch is that code is hard to translate into Rust. "I have this code, you see... and it takes a couple mutable references into an array... how do I translate this into Rust?" There is no one-size-fits-all answer to that question. The code may be correct in C or C++, but the Rust type system may give you one hell of a hard time proving that it is correct to the Rust type system's satisfaction... so you refactor your code completely, or you use integer indexes into arrays rather than references, or you use unsafe code... I've written some amount of Rust code at this point. About half of the time, when I write a project in Rust, there comes a point at which I'm fighting with the type system. I feel like this should stop happening, at some point.

marcus_holmes 5y ago

I love Go, my favourite language. But I've been bitten before by passing slices around and then finding out that they got disassociated and are now pointing at two different backing arrays without telling me. I kinda know enough now to avoid this, but I have to be careful and remind myself it's a possibility. I'd love some built-in method to be able to tell whether altering a value in slice A will also alter the value in slice B (i.e. whether A and B are referring to the same backing array). As far as I'm aware there's no easy way of doing this in Go.

deleted 5y ago
[deleted]

Fire-Dragon-DoL 5y ago

s = append(s, elem) I read this immediately as "create a new copy of the original slice with one additional element", so I presumed that was the case. It would actually be shocking the opposite, if I could end up modifying the original one (before the append) with a pointer to the new s, which seems to be the case! Big gotcha there: treat slices as stateful at all time. Since it has an assignment operation, it must be creating something new, otherwise it would have been a method of the slice itself EDIT: I just realized the gotcha is not there at all, Go would consider the first slice to be of N length and the second slice of length N+1. Comparing the two slices would give an error at some point because one is shorter than the other, so the fact that the address changes or not is irrelevant. However I can see this becoming problematic with pointers, which proves the point of the article.