6 ms·

You can just ask Claude why it did something in the code and it will tell you Programmers just moved up a level , not dumber, we are now managers of a team of

• brookst 19d ago
  Yep. I do so much work with Claude on product requirements and architecture before a single line of code is written, and test coverage is extreme. If I need to understand a specific line of code it means I did something wrong in planning or in requirements for testing.
  • fortzi 18d ago
    In my experience it’s impossible to foresee all the micro decisions that reality will demand. IMO it’s one reason to not use waterfall, which is very similar to what you described
    • brookst 17d ago
      No, waterfall is an engineering process. What I’m describing is thorough documentation of requirements (acceptance criteria, if you like), and then encouraging Claude to be agile in execution. As long as the outcome is well-defined, it is expected and normal to iterate on implementation.
• ButlerianJihad 19d ago
  > You can just ask Claude why it did something But will Claude give you an authentic rationale and a traceable, verifiable "line of reasoning" for those things? Or will it just construct the next plausible Markov chain built on whatever Reddit thread it ingested at random? You can ask Claude or any LLM for citations, and it will RAG them out ex post facto. Those actually aren't citations, they're just web searches for related articles, and they don't necessarily support the assertions that you're asking to cite. I am sure that Claude and the others can produce intermediate logs of their inference and "reasoning" process while they are processing stuff, but can they really go back within the context window and construct an authentic apologia for a specific thing when you ask for it?
  • fluidcruft 18d ago
    You certainly can ask it to produce a verifiable report that tells you how to verify its claims. Yes, you do need to verify the results and confirm you agree with it. I do this sort of thing a lot to discover misunderstandings Claude has about things and iterating on them. There's a difference between reading everything yourself vs having a discussion where one of you (Claude) will happily dig through everything and create detailed memos for review. If you want line numbers as evidence, just ask then verify, etc.
    • ButlerianJihad 17d ago
      Really? Verifiable, really? If you ask it 3 times to generate 3 verifiable reports that confirm its claims, will it answer with the same process and same answers each time? See, when you ask a human to justify a result or a decision, they can often do this very meticulously. If a judge writes a decision from the bench, or a firefighter describes how his battalion knocked down an apartment fire, or a systems admin describes how he configured a NAS, they will all be relying on their training, and precedent, and specifications, and things like that, and they can give you reproducible results and solid justifications for the way they did things. When mistakes are made, and money or life is lost, they can be accountable and you can modify that process to set a precedent for the future. But Claude? How in the world will it produce the same results twice? It is non-deterministic. That is the fundamental issue of LLMs and genAI today. They are all non-deterministic, and SWE treat them as if they are somehow reliable, or produce reproducible results, or that they can follow a procedure or a specification, outlined in their prompts and context, and produce results. No, they only produce results by accident and happenstance, and they only justify them ex post facto by making things up. There is no humanity or deterministic activity in an LLM. You'll never verify "why" they chose that string of tokens, because they could've easily chosen a very different stream of tokens. In fact, now with watermarking, the most deterministic thing will be hitting that watermark standard at all costs!
      • fluidcruft 17d ago
        Are you under the impression that if you ask two programmers to solve the same problem, you will receive the same code or design from both?
• pessimizer 18d ago
  An LLM has no special insight into itself and has no reasons. It will come up with something that is convincing, and it will change based on the word choices you used in the request. You can observe this by asking it why it did something that it didn't actually do. Dementia patients also do this.
• eudamoniac 18d ago
  > and it will tell you [why it did something] No, it will invent retroactively a plausible sounding reason why someone might have done it that way. These are very different things.
• fortzi 18d ago
  Claude won’t tell you why it did something. Instead, it will generate a probable explanation. The two are very different
  • Zambyte 18d ago
    I'm not sure how context is managed between reasoning traces and actual output in Claude / claude code, but if the reasoning trace is in the context of the question for "why did you do that", it can actually answer why it did that.
    • inigyou 18d ago
      Reasoning traces are also probable explanations.
      • Dylan16807 18d ago
        Yeah but at that point it's probably close enough. Humans can get their own reasoning wrong too so some inaccuracy is acceptable.
        • inigyou 18d ago
          It's exactly as close as asking for it after the fact. Reasoning traces have no relation to the reasons LLMs actually do things, except that they may do things because the reasoning trace says they should.
          • Dylan16807 18d ago
            It's exactly as close except when it isn't?
            • inigyou 18d ago
              Yes, and when it isn't is a very specific very limited case that has no relation to the one being discussed. The fact that some course of action was previously mentioned in a reasoning trace, or any other context, makes it more likely to be performed. It has nothing to do with the reason that it was mentioned in the reasoning trace.