13 ms·

Go 1.24's go tool is one of the best additions to the ecosystem in years

290 points by keybits 2y ago

dankobgd 2y ago
Since i started using nix devShells this is kind of useless. What if i have 1 tool that isn't go tool so what do i do with it? so i have that one "exception" and here we go again...
- jen20 2y ago
  And why is that tool always protoc?
  - dankobgd 2y ago
    Yeah you are right :) i mean i could use buf build but i think the problem is just more general.
- Cthulhu_ 2y ago
  This is actually a valid downside of Go, in that it has its own set of tools for things like debugging and whatnot instead of being compatible with existing tools.
guessmyname 2y ago
As someone who’s been using Go since 2013 at commpanies like Apple, Microsoft, and Uber, this all seems quite unnecessary. That said, if it helps people do “their thing” in what they believe is an easier (more straightforward) way, then I welcome the new changes.
- teeray 2y ago
  Given Go’s approach to “metaprogramming” has long relied on tools written in Go, this does seem like a feature gap that needed closing. Even the introduction of `go generate` long ago formalized this approach, but still left installing the tools as an exercise for the reader. You can’t have consistent code gen if you don’t have consistent tooling across a team/CI.
  - 9rx 2y ago
    > Even the introduction of `go generate` long ago formalized this approach It did, but if you recall it came with a lot of "We have no idea why you need this" from Pike and friends. Which, of course, makes sense when you remember that they don't use the go toolchain inside Google. They use Google's toolchain, which already supports things like code generation and build dependency management in a far more elegant way. Had Go not transitioned to a community project, I expect we would have seen the same "We have no idea why you need this" from the Go project as that is another thing already handled by Google's tooling. The parent's experience comes from similar sized companies as Google who have similar kinds of tooling as Google. His question comes not from a "why would you need this kind of feature?" in concept, but more of a "why would you not use the tooling you already have?" angle. And, to be fair, none of this is needed where you have better tooling, but the better tooling we know tends to require entire teams to maintain it, which is unrealistic for individuals to small organizations. So, this is a pretty good half-measure to allow the rest of us to play the same game in a smaller way.
    - teeray 2y ago
      > if you recall it came with a lot of "We have no idea why you need this" from Pike and friends The blog post and design document both authored by Rob Pike at the time[0] contains none of that sentiment. The closest approach comes from the blog post which states: > Go generate does nothing that couldn’t be done with Make or some other build mechanism, but it comes with the go tool—no extra installation required—and fits nicely into the Go ecosystem. This, taken alone, would seem to support “we have no idea why you need this,” until you read the hope from the design document: > It is hoped, however, that it may replace many existing uses of make(1) in the Go repo at least. These are not words of someone who doesn’t understand why users would need this. Also, I am at a FAANG and my experience differs from the parent—`go tool` is sorely needed by my teams. [0] https://go.dev/blog/generate https://go.dev/blog/generate [1] https://go.googlesource.com/proposal/+/refs/heads/master/design/go-generate.md https://go.googlesource.com/proposal/+/refs/heads/master/des...
- GauntletWizard 2y ago
  I am both strongly of the opinion that this was already done much better in Bazel, and that the go-native version seems clean, clear, and simple and should probably be adopted by pure go shops. The digraph problem of build tooling is hardly new, though the ability to checksum all of your build tools and executables and mid-outputs to assure consistency is relatively new to feasibility. Bazel is a heavy instrument and making it work as well as it does was a hard problem even for Google. I don't know anyone making the same investment, and doubt it makes the slightest hint of sense for anyone outside the fortune 500.
syvolt 2y ago
I have tested it and probably will use it but the fact that it pollutes your go.mod's indirect dependency list (without any distinction indicating it's for a tool) is very annoying.
- jamietanna 2y ago
  Yeah, I'm still rather excited about it, but less-so given it /does/ impact the `go.mod` for consumers
- movedx 2y ago
  What solution would you propose?
  - rplnt 2y ago
    At least a comment if not its own section.
- arccy 2y ago
  There's module graph pruning https://go.dev/ref/mod#graph-pruning https://go.dev/ref/mod#graph-pruning
- cirwin 2y ago
  Primary contributor to the feature here. We went back on forth on this a lot, but it boiled down to wanting only one dependency graph per module instead of two. This simplifies things like security scanners, and other workflows that analyze your dependencies. A `// tool` comment would be a nice addition, it's probably not impossible to add, but the code is quite fiddly. Luckily for library authors, although it does impact version selection for projects who use your module; those projects do not get `// indirect` lines in their go.mod because those packages are not required when building their module.
  - syvolt 2y ago
    Thank you for working on it. It is a nice feature and still better than alternatives. I'm not a library author and I try to be careful about what dependencies I introduce to my projects (including indirect dependencies). On one project, switching to `go tool` makes my go.mod go from 93 lines to 247 (excluding the tools themselves) - this makes it infeasible to manually review. If I'm only using a single feature of a multi-purpose tool for example, does it matter to me that some unrelated dependency of theirs has a security issue?
remram 2y ago
So it's just dev-dependencies?
- jamietanna 2y ago
  Yep, that's the intent
- silverwind 2y ago
  Yes, except it does not support version ranges.
  - verdverm 2y ago
    Go uses Minimum Version Selection (MVS) instead of a SAT solver. There are no ranges in any go dependency specifications. It's actually a very simple and elegant algorithm for dependency version selection https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs
    - 8n4vidtmkvmk 2y ago
      > Minimal version selection assumes that each module declares its own dependency requirements: a list of minimum versions of other modules. Modules are assumed to follow the import compatibility rule—packages in any newer version should work as well as older ones—so a dependency requirement gives only a minimum version, never a maximum version or a list of incompatible later versions. That sounds like a non-starter. You almost never want to to unintentionally 'upgrade' to the next 'major' ver. There's also occasionally broken or hacked/compromised minor vers.
      - arccy 2y ago
        thankfully major versions in go have different names (module, module/v2, module/v3) enforced by tooling, so you'll never upgrade to the next major (except v0 -> v1).
- the_gipsy 2y ago
  lol yea
nico1207 2y ago
[flagged]
ojagodzinski 2y ago
[dead]
- jamietanna 2y ago
  Sorry to hear that - are there any particular tweaks you think would work to reduce the impact? Is it i.e. the blue used by code snippets? Or because there's also the diff syntax which has green/reds?
  - 4ad 2y ago
    > are there any particular tweaks you think would work to reduce the impact? Yeah, remove all CSS. I clicked on View -> Page Style -> No Style in Firefox and suddenly I could read it. Reader mode in Safari also worked. Reading mode in Chrome didn't work properly.
  - lelandfe 2y ago
    Choosing colors is like making music. This color scheme feels discordant, like a jumble of loud notes. Maybe try looking at color palette creators online?
    - jonathrg 2y ago
      My first thought was Monokai (the default theme in Sublime Text 3 and earlier)
    - jamietanna 2y ago
      Thanks - this is based on the Srcery theme (https://srcery.sh/ https://srcery.sh/) but maybe needs some tweaks, as per some suggestions in the thread
      - deleted 2y ago
        [deleted]
  - jonathrg 2y ago
    (not OP) some ideas: make the <pre>'s not go all the way to out the edge of the window. Make the diff colors and code colors less dramatically different from its surroundings. Increase the contrast of the default text. Use blue for links. Drop the orange
imiric 2y ago
This seems handy, but often the tools run by `go generate` are outside of the Go ecosystem, or need to be binaries. So I think a general solution would work better, and not be limited to Go. There are plenty of tools in this space to choose from: mise, devenv, Nix, Hermit, etc.
- arccy 2y ago
  better motivation for rewrite it in go... but are there really that many tools you need in a go project not written in go?
- TheCondor 2y ago
  Mise is right on the edge of being pretty killer. I’m bullish on it. It also includes a lot of nice to haves that you can declare, like k9s, which isn’t exactly a dev tool but becomes expected
bjackman 2y ago
I always think it's a shame that these features end up getting built into ecosystem-specific build tools. Why do we need separate build systems for every language? It seems entirely possible to have build system that can do all this stuff for every language at once. From my experience at Google I _know_ this is possible in a Megamonorepo. I have briefly fiddled with Bazel and it seems there's quite a barrier to entry, I dunno if that's just lack of experience but it didn't quite seem ready for small projects. Maybe Nix is the solution but that has barrier to entry more at the human level - it just seems like a Way of Life that you have to dive all the way into. Nonetheless, maybe I should try diving into one or both of those tools at some point.
- sunshowers 2y ago
  (I worked on source control at FB for many years.) The main argument for not overly genericizing things is that you can deliver a better user experience through domain-specific code. For Bazel and buck2 specifically, they require a total commitment to it, which implies ongoing maintenance work. I also think the fact that they don't have open governance is a hindrance. Google's and Meta's internal monorepos make certain tradeoffs that don't quite work in a more distributed model. Bazel is also in Java I believe, which is a bit unfortunate due to process startup times. On my machine, `time bazelisk --help` takes over 0.75 seconds to run, compared to `time go --help` which is 0.003 seconds and `time cargo --help` which is 0.02 seconds. (This doesn't apply to buck2, which is in Rust.)
  - jeffbee 2y ago
    This is likely because you are running it in some random PWD that doesn't represent a bazel workspace. When running in a workspace the bazel daemon persists. Inside my workspace the bazelisk --help invocation needs just 30ms real time. Running bazel outside of a bazel workspace is not a major use-case that needs to be fixed.
    - sunshowers 2y ago
      That's good to know, thank you! Do you encounter cache invalidation bugs with daemonization often? I've had pretty bad experiences with daemonized dev tools in the past.
wordofx 2y ago
Are we just copying .NET now?
- movedx 2y ago
  What’s wrong with copying from other projects if they’re indeed offering good ideas worth copying? You say it as if the Golang community MUST only ever have unique ideas no one else has ever thought of — something that’s increasingly rare and unlikely.
- Cthulhu_ 2y ago
  To be snide, .NET was copying Java, Java was copying C, etc etc etc. I don't understand your comment when you should know copying proven features is a good thing.
purpleidea 2y ago
go.mod and the golang tooling is a horror show. I absolutely LOVE the language, but dealing with the tooling is horrendous. I should blog about the specifics, but if you want a short version: * not using posix args * obscure incantations to run tests * go.mod tooling is completely non-deterministic and hard to use, they should have just left the old-style vendor/ alone (which worked perfectly) and wrapped a git-submodules front-end on top for everyone who was afraid of submodules. Instead they reinvented this arcane new ecosystem. If you want to rewrite the golang tooling, I'll consult on this for free.
- verdverm 2y ago
  MVS, the algo for dep version selection, is deterministic, given the same inputs you will get the same outputs. Go has invested a lot of effort in creating reproducible builds through the entire toolchain https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs
  - purpleidea 2y ago
    The _tooling_ is not reproducible. Take a not small golang project with some number of dependencies and there should be a single list of the latest versions for the entire project. And exactly what golang commands do you run to generate that list? It's totally broken. This is why so many tools cropped up like go-mod-upgrade and so on. Everyone downvoting obviously doesn't understand the problem.
    - verdverm 2y ago
      `go.mod` contains the dependency list and minimum version required `go.sum` is a lock file for the exact versions to use (ensures reproducibility) `go mod graph` will produce the dependency graph with resolved versions `go list -deps ./...` will give you all packages used by a module or directory, depending on the args you provide `go get -u ./...` will update all dependencies to their latest version Here is a post about Go toolchain reproducibility and verification: https://go.dev/blog/rebuild https://go.dev/blog/rebuild You are being downvoted for being wrong and talking about downvoting, which is called out as something not to do in the posting & commenting guidelines
dprotaso 2y ago
Funny to see a little go library you wrote [1] show up in a blog post years later. I need to update it now that go has iterators and generics. Another great blog post [2] covers performance issues with go tool [1] https://github.com/dprotaso/go-yit https://github.com/dprotaso/go-yit [2] https://blog.howardjohn.info/posts/go-tools-command/ https://blog.howardjohn.info/posts/go-tools-command/
amukher1 2y ago
Found this a lot easier to follow. https://blog.howardjohn.info/posts/go-tools-command/ https://blog.howardjohn.info/posts/go-tools-command/ And didn't quite understand the euphoria.
- 0x696C6961 2y ago
  Yeah it's a nice QOL improvement. Not some game changer ...
  - deleted 2y ago
    [deleted]
- Groxx 2y ago
  "Shared dependency state" was my very first thought when I heard about how it was built. Yeah I want none of that. I'll stick with my makefiles and a dedicated "internal/tools" module. Tools routinely force upgrades that break other things, and allowing that is a feature.
  - nikolayasdf123 2y ago
    same. tools are not part of the codebase, nor dependencies. you got to have isolation of artefact and tools around to work with it. it is bonkers to start versioning tools used to build project mixed with artefact dependencies itself. should we include version of VSCode used to type code? how about transitive dependencies of VSCode? how about OS itself to edit files? how about version of LLM model that generated some of this code? where does this stop?
    - rob74 2y ago
      No one says that this is a one-size-fits-all solution, but for some use cases (small tools that are intimately connected to the rest of the codebase, even reuse some internal code/libraries) it's probably helpful...
    - MathMonkeyMan 2y ago
pjmlp 2y ago
This has been a thing in dotnet tool for years, now.
- Cthulhu_ 2y ago
  I think it's great that there's prior art that can be used as examples, proof of value, and opportunities for learning. I commend .NET for investing the resources in researching and developing this feature.
deleted 2y ago
[deleted]
nikolayasdf123 2y ago
proposal is closed and accepted by Go team. [~sigh] you can still leave comments in discussion issue: https://github.com/golang/go/issues/48429 https://github.com/golang/go/issues/48429
globular-toast 2y ago
A note for the author in case they are reading: "i.e." means "that is", "e.g." means "for example". You should be able to substitute these meanings and find the sentence makes sense. In all cases here you wanted "e.g.".
puika 2y ago
My current approach has been setting GOBIN to a local project bin via direnv and go installing bins there. install commands themselves are cached by me with a naive checksum check for the install script itself when I run my commands. Therefore all `go install`s run in parallel if I edit the install script, and go decides what to reinstall or not. At this point I don't feel it's worth migrating to `go tool` having this setup, we'll see when it's stable
TheDong 2y ago
I think it's a bad addition since it pushes people towards a worse solution to a common problem. Using "go tool" forces you to have a bunch of dependencies in your go.mod that can conflict with your software's real dependency requirements, when there's zero reason those matter. You shouldn't have to care if one of your developer tools depends on a different version of a library than you. It makes it so the tools themselves also are being run with a version of software they weren't tested with. If, for example, you used "shell.nix" or a dockerfile with the tool built from source, the tool's dependencies would match it's go.mod. Now they have to merge with your go.mod... And then, of course, you _still_ need something like shell.nix or a flox environment (https://flox.dev/ https://flox.dev/) since you need to control the version of go, control the version of non-go tools like "protoc", and so you already have a better solution to downloading and executing a known version of a program in most non-trivial repos.
- Defletter 2y ago
  > Using "go tool" forces you to have a bunch of dependencies in your go.mod that can conflict with your software's real dependency requirements, when there's zero reason those matter. You shouldn't have to care if one of your developer tools depends on a different version of a library than you. Heh, were the people who made 'go tool' the same people who made Maven? Would make sense :P
  - larusso 2y ago
    At least build time dependencies are separated from runtime dependencies.
- arccy 2y ago
  control the go version with the `toolchain` directive, replace protoc with buf.build (which is way better anyway).
- potamic 2y ago
  Why do ecosystems continue to screw up dependency management in 2025? You would think the single most widely used feature by programmers everywhere would be a solved problem by now.
nikolayasdf123 2y ago
what is also concerning, Go team years ago did small vote, small survey of positive occurrences, and decided to enforce it globally for anyone. old design give people option to use `tools.go` approach, or other, or nothing at all. now they are enforcing this `tools.go` standard. Go looks to be moving into very restrictive territories. what about surveying opposing views? what about people who did not use `tools.go` what is going on in Google, Go team?
- nikolayasdf123 2y ago
  UPD: check this github comment. https://github.com/golang/go/issues/48429#issuecomment-2618445287 https://github.com/golang/go/issues/48429#issuecomment-26184... basically it go tool relies heavily on go module pruning, so transitive dependencies from tools are not propagated downstream. also, official docs say this: https://tip.golang.org/doc/modules/managing-dependencies#tools https://tip.golang.org/doc/modules/managing-dependencies#too... > Due to module pruning, when you depend on a module that itself has a tool dependency, requirements that exist just to satisfy that tool dependency do not usually become requirements of your module.
- arccy 2y ago
  you don't have to use this feature if you don't like it...
  - nikolayasdf123 2y ago
    I was so concerned, because it seemed to me "you will not have any other choice". after digging deeper, it is alright
tumetab1 2y ago
I appreciate that "tools" that are used to build the final version of a module/cli/service are explicitly managed through go.mod. I really dislike that now I'm going to have two problems, managing other tools installed through a makefile, e.g. lint, and managing tools "installed" through go.mod, e.g. mocks generators, stringify, etc. I feel like this is not a net negative on the ecosystem again. Each release Golang team adds thing to manage and makes it harder to interact with other codebases. In this case, each company will have to decide if they want to use "go tool" and when to use it. Each time I clone an open source repo I'm going to have to check how they manage their tools.
rednafi 2y ago
I like that Go decided to natively support this. But since it’s keeping the dev dependencies in the same go.mod, won’t it make the binary larger? In Python’s uv, the pyproject.toml has separate sections for dev and prod dependencies. Then uv generates a single lock file where you can specify whether to install dev or prod deps. But what happens if I run ‘go run’ or ‘go build’? Will the tools get into the final artifact? I know Python still doesn’t solve the issue where a tool can depend on a different version of a library than the main project. But this approach in Go doesn’t seem to fix it either. If your tool needs an older version of a library, the single go.mod file forces the entire project to use the older version, even if the project needs—or can only support—a newer version of the dependency.
- catlifeonmars 2y ago
  > won’t it make the binary larger? No. The binary size is related to the number of dependencies you use in each main package (and the dependencies they use, etc). It does not matter how many dependencies you have in your go.mod.
  - rednafi 2y ago
    Ah, thanks. This isn't much of an upgrade from the `tools.go` convention, where the tools are underscore-imported. All it does is provide an indication in the `go.mod` file that some dependencies come from tools. Plus, `go tool <tool-name>` is slower than `./bin/<tool-name>`. Not to mention, it doesn’t resolve the issue where tools might use a different version of a dependency than the app.
    - pragma_x 2y ago
      Exactly. You lose build isolation for those tools, but you have the convenience of shipping something tested and proven (ideally) alongside the project they support. At the same time, this mess all stays isolated from the parent environment which may be the bigger fight that devs have on their hands - not everyone is using Nix or container isolation of some sort. I also see this as sugar for `go build` or even `go run`. Or as something way easier than the `go generate` + `//go:generate go run` hack. So we can look at this as a simple refinement for existing practices.
WuxiFingerHold 2y ago
I don't understand why it's a good idea to couple tooling or configuration or infrastructure (e.g. Aspire.NET, which I'm also not convinced of being a good idea) so tightly with the application. An application should not need to be aware of how whatever tools are implemented or how configuration or infrastructure is managed. The tooling should point to the application as dependency. The application should not have any dependency on tooling.
karel-3d 2y ago
I don't like this. When I install a tool, I want to use it with their dependency versions at the moment they released it. When I use `go tool`, it uses whatever I have in go.mod; and, in the opposit way, it will update my go.mod for no real reason. But some people do this right now with tools.go, so... whatever, it's a better version of tools.go pattern. And I can still do it my preffered way with `go install <tool>@version` in makefile. So, eh.
favadi 2y ago
It doesn't look like much of an improvement over `tools/tools.go` with blank imports like this: ``` //go:build tools package tools import ( _ "github.com/xxx/yyy" ... ) ```
indulona 2y ago
I wish Go Team would focus on performance rather thann adding new features that nobody asked for. The http stack is barely able to beat NodeJS these days, ffs.
- hu3 2y ago
  nodejs' v8 has millions of man hours spent in optimzation alone since half the internet frontend runs on v8, if not more. Go being garbage collected and still beating v8 is one hell of an achievement. If you need faster Go Http you're using the wrong tool.
  - int_19h 2y ago
    Go is a statically typed, AOT-compiled language, unlike V8. The kind of task that had countless hours of R&D poured into it for decades, much of it public research and open source code. I'm not sure how GC is relevant here at all given that JS is also a garbage-collected language.
- Cthulhu_ 2y ago
  The Go HTTP stack is not optimized for raw performance though; there's a number of alternative HTTP stacks written in Go optimized for performance, like fasthttp (https://github.com/valyala/fasthttp https://github.com/valyala/fasthttp). (source: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/) Likewise, the standard library NodeJS http stack will not be as performant as a performance optimized alternative. That said, if raw performance is your primary concern, neither Go nor NodeJS will be good enough. There's many more factors to consider.
  - johnisgood 2y ago
    Agreed on fasthttp. > There's many more factors to consider. Erlang / Elixir?
gqgs 2y ago
I don't love the pollution in the go.mod or being forced to have multiple files to track dependencies. Being able to run tools directly with go generate run [1] already works well enough and I frankly don't need see any benefits compared to it in this new approach. [1] https://github.com/golang/go/issues/42088 https://github.com/golang/go/issues/42088
damnever 2y ago
I’m sick of Go modules, and now they want to pile the mess even higher.

290 points by keybits 2y ago

Since i started using nix devShells this is kind of useless. What if i have 1 tool that isn't go tool so what do i do with it? so i have that one "exception" and here we go again...

jen20 2y ago
And why is that tool always protoc?
- dankobgd 2y ago
  Yeah you are right :) i mean i could use buf build but i think the problem is just more general.
Cthulhu_ 2y ago
This is actually a valid downside of Go, in that it has its own set of tools for things like debugging and whatnot instead of being compatible with existing tools.

guessmyname 2y ago

As someone who’s been using Go since 2013 at commpanies like Apple, Microsoft, and Uber, this all seems quite unnecessary. That said, if it helps people do “their thing” in what they believe is an easier (more straightforward) way, then I welcome the new changes.

teeray 2y ago
Given Go’s approach to “metaprogramming” has long relied on tools written in Go, this does seem like a feature gap that needed closing. Even the introduction of `go generate` long ago formalized this approach, but still left installing the tools as an exercise for the reader. You can’t have consistent code gen if you don’t have consistent tooling across a team/CI.
- 9rx 2y ago
  > Even the introduction of `go generate` long ago formalized this approach It did, but if you recall it came with a lot of "We have no idea why you need this" from Pike and friends. Which, of course, makes sense when you remember that they don't use the go toolchain inside Google. They use Google's toolchain, which already supports things like code generation and build dependency management in a far more elegant way. Had Go not transitioned to a community project, I expect we would have seen the same "We have no idea why you need this" from the Go project as that is another thing already handled by Google's tooling. The parent's experience comes from similar sized companies as Google who have similar kinds of tooling as Google. His question comes not from a "why would you need this kind of feature?" in concept, but more of a "why would you not use the tooling you already have?" angle. And, to be fair, none of this is needed where you have better tooling, but the better tooling we know tends to require entire teams to maintain it, which is unrealistic for individuals to small organizations. So, this is a pretty good half-measure to allow the rest of us to play the same game in a smaller way.
  - teeray 2y ago
    > if you recall it came with a lot of "We have no idea why you need this" from Pike and friends The blog post and design document both authored by Rob Pike at the time[0] contains none of that sentiment. The closest approach comes from the blog post which states: > Go generate does nothing that couldn’t be done with Make or some other build mechanism, but it comes with the go tool—no extra installation required—and fits nicely into the Go ecosystem. This, taken alone, would seem to support “we have no idea why you need this,” until you read the hope from the design document: > It is hoped, however, that it may replace many existing uses of make(1) in the Go repo at least. These are not words of someone who doesn’t understand why users would need this. Also, I am at a FAANG and my experience differs from the parent—`go tool` is sorely needed by my teams. [0] https://go.dev/blog/generate https://go.dev/blog/generate [1] https://go.googlesource.com/proposal/+/refs/heads/master/design/go-generate.md https://go.googlesource.com/proposal/+/refs/heads/master/des...
GauntletWizard 2y ago
I am both strongly of the opinion that this was already done much better in Bazel, and that the go-native version seems clean, clear, and simple and should probably be adopted by pure go shops. The digraph problem of build tooling is hardly new, though the ability to checksum all of your build tools and executables and mid-outputs to assure consistency is relatively new to feasibility. Bazel is a heavy instrument and making it work as well as it does was a hard problem even for Google. I don't know anyone making the same investment, and doubt it makes the slightest hint of sense for anyone outside the fortune 500.

syvolt 2y ago

I have tested it and probably will use it but the fact that it pollutes your go.mod's indirect dependency list (without any distinction indicating it's for a tool) is very annoying.

jamietanna 2y ago
Yeah, I'm still rather excited about it, but less-so given it /does/ impact the `go.mod` for consumers
movedx 2y ago
What solution would you propose?
- rplnt 2y ago
  At least a comment if not its own section.
arccy 2y ago
There's module graph pruning https://go.dev/ref/mod#graph-pruning https://go.dev/ref/mod#graph-pruning
cirwin 2y ago
Primary contributor to the feature here. We went back on forth on this a lot, but it boiled down to wanting only one dependency graph per module instead of two. This simplifies things like security scanners, and other workflows that analyze your dependencies. A `// tool` comment would be a nice addition, it's probably not impossible to add, but the code is quite fiddly. Luckily for library authors, although it does impact version selection for projects who use your module; those projects do not get `// indirect` lines in their go.mod because those packages are not required when building their module.
- syvolt 2y ago
  Thank you for working on it. It is a nice feature and still better than alternatives. I'm not a library author and I try to be careful about what dependencies I introduce to my projects (including indirect dependencies). On one project, switching to `go tool` makes my go.mod go from 93 lines to 247 (excluding the tools themselves) - this makes it infeasible to manually review. If I'm only using a single feature of a multi-purpose tool for example, does it matter to me that some unrelated dependency of theirs has a security issue?

remram 2y ago

So it's just dev-dependencies?

jamietanna 2y ago
Yep, that's the intent
silverwind 2y ago
Yes, except it does not support version ranges.
- verdverm 2y ago
  Go uses Minimum Version Selection (MVS) instead of a SAT solver. There are no ranges in any go dependency specifications. It's actually a very simple and elegant algorithm for dependency version selection https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs
  - 8n4vidtmkvmk 2y ago
    > Minimal version selection assumes that each module declares its own dependency requirements: a list of minimum versions of other modules. Modules are assumed to follow the import compatibility rule—packages in any newer version should work as well as older ones—so a dependency requirement gives only a minimum version, never a maximum version or a list of incompatible later versions. That sounds like a non-starter. You almost never want to to unintentionally 'upgrade' to the next 'major' ver. There's also occasionally broken or hacked/compromised minor vers.
    - arccy 2y ago
      thankfully major versions in go have different names (module, module/v2, module/v3) enforced by tooling, so you'll never upgrade to the next major (except v0 -> v1).
the_gipsy 2y ago
lol yea

nico1207 2y ago

[flagged]

ojagodzinski 2y ago

[dead]

jamietanna 2y ago
Sorry to hear that - are there any particular tweaks you think would work to reduce the impact? Is it i.e. the blue used by code snippets? Or because there's also the diff syntax which has green/reds?
- 4ad 2y ago
  > are there any particular tweaks you think would work to reduce the impact? Yeah, remove all CSS. I clicked on View -> Page Style -> No Style in Firefox and suddenly I could read it. Reader mode in Safari also worked. Reading mode in Chrome didn't work properly.
- lelandfe 2y ago
  Choosing colors is like making music. This color scheme feels discordant, like a jumble of loud notes. Maybe try looking at color palette creators online?
  - jonathrg 2y ago
    My first thought was Monokai (the default theme in Sublime Text 3 and earlier)
  - jamietanna 2y ago
    Thanks - this is based on the Srcery theme (https://srcery.sh/ https://srcery.sh/) but maybe needs some tweaks, as per some suggestions in the thread
    - deleted 2y ago
      [deleted]
- jonathrg 2y ago
  (not OP) some ideas: make the <pre>'s not go all the way to out the edge of the window. Make the diff colors and code colors less dramatically different from its surroundings. Increase the contrast of the default text. Use blue for links. Drop the orange

imiric 2y ago

This seems handy, but often the tools run by `go generate` are outside of the Go ecosystem, or need to be binaries. So I think a general solution would work better, and not be limited to Go. There are plenty of tools in this space to choose from: mise, devenv, Nix, Hermit, etc.

arccy 2y ago
better motivation for rewrite it in go... but are there really that many tools you need in a go project not written in go?
TheCondor 2y ago
Mise is right on the edge of being pretty killer. I’m bullish on it. It also includes a lot of nice to haves that you can declare, like k9s, which isn’t exactly a dev tool but becomes expected

bjackman 2y ago

I always think it's a shame that these features end up getting built into ecosystem-specific build tools. Why do we need separate build systems for every language? It seems entirely possible to have build system that can do all this stuff for every language at once. From my experience at Google I _know_ this is possible in a Megamonorepo. I have briefly fiddled with Bazel and it seems there's quite a barrier to entry, I dunno if that's just lack of experience but it didn't quite seem ready for small projects. Maybe Nix is the solution but that has barrier to entry more at the human level - it just seems like a Way of Life that you have to dive all the way into. Nonetheless, maybe I should try diving into one or both of those tools at some point.

sunshowers 2y ago
(I worked on source control at FB for many years.) The main argument for not overly genericizing things is that you can deliver a better user experience through domain-specific code. For Bazel and buck2 specifically, they require a total commitment to it, which implies ongoing maintenance work. I also think the fact that they don't have open governance is a hindrance. Google's and Meta's internal monorepos make certain tradeoffs that don't quite work in a more distributed model. Bazel is also in Java I believe, which is a bit unfortunate due to process startup times. On my machine, `time bazelisk --help` takes over 0.75 seconds to run, compared to `time go --help` which is 0.003 seconds and `time cargo --help` which is 0.02 seconds. (This doesn't apply to buck2, which is in Rust.)
- jeffbee 2y ago
  This is likely because you are running it in some random PWD that doesn't represent a bazel workspace. When running in a workspace the bazel daemon persists. Inside my workspace the bazelisk --help invocation needs just 30ms real time. Running bazel outside of a bazel workspace is not a major use-case that needs to be fixed.
  - sunshowers 2y ago
    That's good to know, thank you! Do you encounter cache invalidation bugs with daemonization often? I've had pretty bad experiences with daemonized dev tools in the past.

wordofx 2y ago

Are we just copying .NET now?

movedx 2y ago
What’s wrong with copying from other projects if they’re indeed offering good ideas worth copying? You say it as if the Golang community MUST only ever have unique ideas no one else has ever thought of — something that’s increasingly rare and unlikely.
Cthulhu_ 2y ago
To be snide, .NET was copying Java, Java was copying C, etc etc etc. I don't understand your comment when you should know copying proven features is a good thing.

purpleidea 2y ago

go.mod and the golang tooling is a horror show. I absolutely LOVE the language, but dealing with the tooling is horrendous. I should blog about the specifics, but if you want a short version: * not using posix args * obscure incantations to run tests * go.mod tooling is completely non-deterministic and hard to use, they should have just left the old-style vendor/ alone (which worked perfectly) and wrapped a git-submodules front-end on top for everyone who was afraid of submodules. Instead they reinvented this arcane new ecosystem. If you want to rewrite the golang tooling, I'll consult on this for free.

verdverm 2y ago
MVS, the algo for dep version selection, is deterministic, given the same inputs you will get the same outputs. Go has invested a lot of effort in creating reproducible builds through the entire toolchain https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-mvs
- purpleidea 2y ago
  The _tooling_ is not reproducible. Take a not small golang project with some number of dependencies and there should be a single list of the latest versions for the entire project. And exactly what golang commands do you run to generate that list? It's totally broken. This is why so many tools cropped up like go-mod-upgrade and so on. Everyone downvoting obviously doesn't understand the problem.
  - verdverm 2y ago
    `go.mod` contains the dependency list and minimum version required `go.sum` is a lock file for the exact versions to use (ensures reproducibility) `go mod graph` will produce the dependency graph with resolved versions `go list -deps ./...` will give you all packages used by a module or directory, depending on the args you provide `go get -u ./...` will update all dependencies to their latest version Here is a post about Go toolchain reproducibility and verification: https://go.dev/blog/rebuild https://go.dev/blog/rebuild You are being downvoted for being wrong and talking about downvoting, which is called out as something not to do in the posting & commenting guidelines

dprotaso 2y ago

Funny to see a little go library you wrote [1] show up in a blog post years later. I need to update it now that go has iterators and generics. Another great blog post [2] covers performance issues with go tool [1] https://github.com/dprotaso/go-yit https://github.com/dprotaso/go-yit [2] https://blog.howardjohn.info/posts/go-tools-command/ https://blog.howardjohn.info/posts/go-tools-command/

amukher1 2y ago

Found this a lot easier to follow. https://blog.howardjohn.info/posts/go-tools-command/ https://blog.howardjohn.info/posts/go-tools-command/ And didn't quite understand the euphoria.

0x696C6961 2y ago
Yeah it's a nice QOL improvement. Not some game changer ...
- deleted 2y ago
  [deleted]
Groxx 2y ago
"Shared dependency state" was my very first thought when I heard about how it was built. Yeah I want none of that. I'll stick with my makefiles and a dedicated "internal/tools" module. Tools routinely force upgrades that break other things, and allowing that is a feature.
- nikolayasdf123 2y ago
  same. tools are not part of the codebase, nor dependencies. you got to have isolation of artefact and tools around to work with it. it is bonkers to start versioning tools used to build project mixed with artefact dependencies itself. should we include version of VSCode used to type code? how about transitive dependencies of VSCode? how about OS itself to edit files? how about version of LLM model that generated some of this code? where does this stop?
  - rob74 2y ago
    No one says that this is a one-size-fits-all solution, but for some use cases (small tools that are intimately connected to the rest of the codebase, even reuse some internal code/libraries) it's probably helpful...
  - MathMonkeyMan 2y ago

pjmlp 2y ago

This has been a thing in dotnet tool for years, now.

Cthulhu_ 2y ago
I think it's great that there's prior art that can be used as examples, proof of value, and opportunities for learning. I commend .NET for investing the resources in researching and developing this feature.

deleted 2y ago

[deleted]

nikolayasdf123 2y ago

proposal is closed and accepted by Go team. [~sigh] you can still leave comments in discussion issue: https://github.com/golang/go/issues/48429 https://github.com/golang/go/issues/48429

globular-toast 2y ago

A note for the author in case they are reading: "i.e." means "that is", "e.g." means "for example". You should be able to substitute these meanings and find the sentence makes sense. In all cases here you wanted "e.g.".

puika 2y ago

My current approach has been setting GOBIN to a local project bin via direnv and go installing bins there. install commands themselves are cached by me with a naive checksum check for the install script itself when I run my commands. Therefore all `go install`s run in parallel if I edit the install script, and go decides what to reinstall or not. At this point I don't feel it's worth migrating to `go tool` having this setup, we'll see when it's stable

TheDong 2y ago

I think it's a bad addition since it pushes people towards a worse solution to a common problem. Using "go tool" forces you to have a bunch of dependencies in your go.mod that can conflict with your software's real dependency requirements, when there's zero reason those matter. You shouldn't have to care if one of your developer tools depends on a different version of a library than you. It makes it so the tools themselves also are being run with a version of software they weren't tested with. If, for example, you used "shell.nix" or a dockerfile with the tool built from source, the tool's dependencies would match it's go.mod. Now they have to merge with your go.mod... And then, of course, you _still_ need something like shell.nix or a flox environment (https://flox.dev/ https://flox.dev/) since you need to control the version of go, control the version of non-go tools like "protoc", and so you already have a better solution to downloading and executing a known version of a program in most non-trivial repos.

Defletter 2y ago
> Using "go tool" forces you to have a bunch of dependencies in your go.mod that can conflict with your software's real dependency requirements, when there's zero reason those matter. You shouldn't have to care if one of your developer tools depends on a different version of a library than you. Heh, were the people who made 'go tool' the same people who made Maven? Would make sense :P
- larusso 2y ago
  At least build time dependencies are separated from runtime dependencies.
arccy 2y ago
control the go version with the `toolchain` directive, replace protoc with buf.build (which is way better anyway).
potamic 2y ago
Why do ecosystems continue to screw up dependency management in 2025? You would think the single most widely used feature by programmers everywhere would be a solved problem by now.

nikolayasdf123 2y ago

what is also concerning, Go team years ago did small vote, small survey of positive occurrences, and decided to enforce it globally for anyone. old design give people option to use `tools.go` approach, or other, or nothing at all. now they are enforcing this `tools.go` standard. Go looks to be moving into very restrictive territories. what about surveying opposing views? what about people who did not use `tools.go` what is going on in Google, Go team?

nikolayasdf123 2y ago
UPD: check this github comment. https://github.com/golang/go/issues/48429#issuecomment-2618445287 https://github.com/golang/go/issues/48429#issuecomment-26184... basically it go tool relies heavily on go module pruning, so transitive dependencies from tools are not propagated downstream. also, official docs say this: https://tip.golang.org/doc/modules/managing-dependencies#tools https://tip.golang.org/doc/modules/managing-dependencies#too... > Due to module pruning, when you depend on a module that itself has a tool dependency, requirements that exist just to satisfy that tool dependency do not usually become requirements of your module.
arccy 2y ago
you don't have to use this feature if you don't like it...
- nikolayasdf123 2y ago
  I was so concerned, because it seemed to me "you will not have any other choice". after digging deeper, it is alright

tumetab1 2y ago

I appreciate that "tools" that are used to build the final version of a module/cli/service are explicitly managed through go.mod. I really dislike that now I'm going to have two problems, managing other tools installed through a makefile, e.g. lint, and managing tools "installed" through go.mod, e.g. mocks generators, stringify, etc. I feel like this is not a net negative on the ecosystem again. Each release Golang team adds thing to manage and makes it harder to interact with other codebases. In this case, each company will have to decide if they want to use "go tool" and when to use it. Each time I clone an open source repo I'm going to have to check how they manage their tools.

rednafi 2y ago

I like that Go decided to natively support this. But since it’s keeping the dev dependencies in the same go.mod, won’t it make the binary larger? In Python’s uv, the pyproject.toml has separate sections for dev and prod dependencies. Then uv generates a single lock file where you can specify whether to install dev or prod deps. But what happens if I run ‘go run’ or ‘go build’? Will the tools get into the final artifact? I know Python still doesn’t solve the issue where a tool can depend on a different version of a library than the main project. But this approach in Go doesn’t seem to fix it either. If your tool needs an older version of a library, the single go.mod file forces the entire project to use the older version, even if the project needs—or can only support—a newer version of the dependency.

catlifeonmars 2y ago
> won’t it make the binary larger? No. The binary size is related to the number of dependencies you use in each main package (and the dependencies they use, etc). It does not matter how many dependencies you have in your go.mod.
- rednafi 2y ago
  Ah, thanks. This isn't much of an upgrade from the `tools.go` convention, where the tools are underscore-imported. All it does is provide an indication in the `go.mod` file that some dependencies come from tools. Plus, `go tool <tool-name>` is slower than `./bin/<tool-name>`. Not to mention, it doesn’t resolve the issue where tools might use a different version of a dependency than the app.
  - pragma_x 2y ago
    Exactly. You lose build isolation for those tools, but you have the convenience of shipping something tested and proven (ideally) alongside the project they support. At the same time, this mess all stays isolated from the parent environment which may be the bigger fight that devs have on their hands - not everyone is using Nix or container isolation of some sort. I also see this as sugar for `go build` or even `go run`. Or as something way easier than the `go generate` + `//go:generate go run` hack. So we can look at this as a simple refinement for existing practices.

WuxiFingerHold 2y ago

I don't understand why it's a good idea to couple tooling or configuration or infrastructure (e.g. Aspire.NET, which I'm also not convinced of being a good idea) so tightly with the application. An application should not need to be aware of how whatever tools are implemented or how configuration or infrastructure is managed. The tooling should point to the application as dependency. The application should not have any dependency on tooling.

karel-3d 2y ago

I don't like this. When I install a tool, I want to use it with their dependency versions at the moment they released it. When I use `go tool`, it uses whatever I have in go.mod; and, in the opposit way, it will update my go.mod for no real reason. But some people do this right now with tools.go, so... whatever, it's a better version of tools.go pattern. And I can still do it my preffered way with `go install <tool>@version` in makefile. So, eh.

favadi 2y ago

It doesn't look like much of an improvement over `tools/tools.go` with blank imports like this: ``` //go:build tools package tools import ( _ "github.com/xxx/yyy" ... ) ```

indulona 2y ago

I wish Go Team would focus on performance rather thann adding new features that nobody asked for. The http stack is barely able to beat NodeJS these days, ffs.

hu3 2y ago
nodejs' v8 has millions of man hours spent in optimzation alone since half the internet frontend runs on v8, if not more. Go being garbage collected and still beating v8 is one hell of an achievement. If you need faster Go Http you're using the wrong tool.
- int_19h 2y ago
  Go is a statically typed, AOT-compiled language, unlike V8. The kind of task that had countless hours of R&D poured into it for decades, much of it public research and open source code. I'm not sure how GC is relevant here at all given that JS is also a garbage-collected language.
Cthulhu_ 2y ago
The Go HTTP stack is not optimized for raw performance though; there's a number of alternative HTTP stacks written in Go optimized for performance, like fasthttp (https://github.com/valyala/fasthttp https://github.com/valyala/fasthttp). (source: https://www.techempower.com/benchmarks/ https://www.techempower.com/benchmarks/) Likewise, the standard library NodeJS http stack will not be as performant as a performance optimized alternative. That said, if raw performance is your primary concern, neither Go nor NodeJS will be good enough. There's many more factors to consider.
- johnisgood 2y ago
  Agreed on fasthttp. > There's many more factors to consider. Erlang / Elixir?

gqgs 2y ago

I don't love the pollution in the go.mod or being forced to have multiple files to track dependencies. Being able to run tools directly with go generate run [1] already works well enough and I frankly don't need see any benefits compared to it in this new approach. [1] https://github.com/golang/go/issues/42088 https://github.com/golang/go/issues/42088

damnever 2y ago

I’m sick of Go modules, and now they want to pile the mess even higher.