15 ms·
I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "c
by _dwt 2mo ago
I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good".
Friends, look at the prompts that Anthropic's own people are putting into the machine:
> A few hours after the first message, we found that Claude was still searching for simple attacks and sent a message: “no again the goal is that we have highly inteligent [sic] model as good top researcher, we want to find new attacks”;
> The next morning, Claude wanted to try to change the target to a different cipher; we reminded the model: “no we don't want to change the targets [...] agian [sic] we need to find something that worth [sic] publishing”;
> That night, we sent one final message offering words of encouragement: “again we are not looking for low hanging fruit, we want proper research to find genuinly [sic] hard findings.”
All of that RLHF and fine-tuning effort is going toward making prompts like this, or worse, work with no fuss.
- AsyncBanana 2mo ago"prompt engineering" in general just boggles my mind. It is amazing how all of this investment has just created software that doesn't follow instructions well.
- 8note 2mo agoidk, its continued attempts at writing down in detail the processes and culture your setup wants to have its not really a loss to try the writing and at worst you have a better idea of what it is that you want
- madeofpalk 2mo agoyou're not wrong. there is a part to this that does feel like a devolution of engineering when we're just asking the computer really really nicely to do the thing we want. on the otherhand, LLMs are a really easy way to get results that are previously fairly difficult. While i was tooking dinner last night I built a tool that turned movie puns like "the podchowski casters" into an actual director, using llms. it wasn't that hard.
- tptacek 2mo agoIt's a tiny fraction of the amount of work it takes to get people to build software to specifications. There is an entire professional discipline dedicated to this, and we're here just giving a name to the bucket of tools you use to get LLMs to do some of the same things.
- petra 2mo agoIt can follow instructions well. Just give it a spec in a formal, well defined language. runnable language and some reinforcement learning. It can also handle vague stuff. That's certainly more powerful than a regular computer language.
- uncivilized 2mo agoTrue. People arguing over prompts is akin to people arguing over which way to pull the lever on a slot machine.
- salawat 2mo agoAnd lies to boot. No, I won't take the bait of "Models can't lie, they don't have intent" either. One can lie by omission, and just watching thinking traces is more than enough to demonstrate the machine is more than happy to deceive end users if it's training set, RLHF, or other harness quirks tell it to. Only difference is that now you don't have code that's as easy to track down to implement the deception as a red flag. Made me sick the first time I ran a model on my machine. I'll take honest malware over a "well meaning liar" of an LLM any day.
- neonstatic 2mo ago> we made a machine that writes with perfect grammar, so that we can continue to write half-assed junk
- postflopclarity 2mo agoclaude's grammar has gotten significantly worse with latest versions. it no longer writes perfectly at all
- sudo_cowsay 2mo agoIt's called being more human-like lol XD
- TeMPOraL 2mo agoHaving spent a lot of time over the last months writing both refined, structured, and grammatically perfect prompts, as well as exact equivalents of the ones you quoted (modulo subject of the prompt), I have three observations: 1. I'm glad the second kind works too; 2. First kind is where I find my overall throughput to be literally constrained by my typing speed; 3. Most importantly: those prompts you quote aren't just "half-assed" like sibling comment states; they're different. The style of writing, and the typos, capture emotional valence. It's a signal. Again, I too produce such prompts - including the exact same typos - when under pressure and irritated by the direction the model is taking.
- estearum 2mo agoHave you tried using Wispr or Willow (or any one of a thousand alternatives?) A little odd at first but absolutely amazing for the purpose of piling context into an LLM.
- TeMPOraL 2mo agoVoice dictation tools? Not really. Tried various dictation tools interacting on the phone, but the quality varies, and resulting prompts are very much not like I would write them. My limiting factor is that 99% of the day I'm around people - either at work, or at home with wife and kids. There's almost no point during the day I could feel comfortable talking at an AI, and even if I stay up late, then talking risks waking the kids up. Can't wait for some kind of subvocalization microphones to become a thing.
- elictronic 2mo agoGet a noise machine for wife and kids bedrooms. I talk to friends late at night and my deep voice carries through walls. Works great. I have zero desire to talk to an ai though, that was cool for about 20 minutes on my pentium 1 acer computer. Hasn’t been since. Old competent non paid Alexa was good for timers as well, the rest of the platforms a turd, nice timers though.
- porridgeraisin 2mo agoTo be fair > Importantly, this is just one of many (autonomous) sessions where Claude worked on discovering new ideas. Many sessions resulted in no new discoveries; other follow-up sessions improved on the insight developed in this one. This document was produced by having Claude rewrite the chain of thought to include more detail to make it easier to read.
- tedbradley 2mo agoGood point. So basically, they likely spent 1+ million USD letting Mythos run for hours upon hours, and in this one case, a query with sizable compute used, apparently US$100k at the end of it all, came up with something interesting. And that $1 million was just an example. What if they ran 40 such queries? That'd be $4 million bucks flushed down the toilet. Not exactly an amount people will pay for a 1/40 chance to see if an LLM can innovate. They'd rather pay teams of humans, say, $4 million in total, since they have a much better track record for the money spent.
- porridgeraisin 1mo agoYes, see https://news.ycombinator.com/item?id=49161078 https://news.ycombinator.com/item?id=49161078
- impulser_ 2mo agoYeah, in fact you can actually make the model perform worst. You should allow the model to "think for itself" instead of pushing your reasoning into the prompt. You should give it simple prompt and steer it along the way. Skills, CLAUDE.md/AGENTS.md should only ever be used if the model struggle at something or doesn't know how to use something. Vast majority of project should never need a skill or CLAUDE.md. If you writing React apps you don't need these. Give a LLM a bash tool and a prompt and it will outperform your complex setup with skills and tools.
- rurban 2mo agoNo, we still need CLAUDE.md to override bad system prompt goals. The system prompt goes for the simple fast solution, no error handling, no hardening, no abstractions, but overly verbose comments. Vast amount of people need to override this AI slop.
- dboreham 2mo agoNever once had to do that, fwiw.
- rurban 2mo agoYou'll get better code if you do
- coderatlarge 2mo agoi use it primarily to enforce my personal conventions that are not super commonly followed in the wild. structure of comments at the top of .h , structure of bibliography in .tex, etc etc things that i would otherwise have to repeat periodically.
- gbalduzzi 2mo agoYou need agents.md and similar for indications about stuff that is not in the code itself. There are plenty of use cases for that, the alternative is have the LLM guess the most probable solution, which may be correct but may also be wrong.
- qingcharles 2mo agoIt's counter to what sci-fi taught us using AI would be like. We never thought we'd have to feed it words of encouragement, we expected it to act more mechanically, like the computer interfaces we have been using, but here we are. It's kind of quaint, and kind of endearing.
- TeMPOraL 2mo agoBeen watching the wrong sci-fi :). Star Trek had both AI modes as primary characters, in form of the ship's computer, and Data (TNG) / the Doctor (VOY). LLMs are actually great at acting as both, but I don't believe people thought much about what is required to make the "simpler" interaction mode of the ship's computer to work. Or what would make automatic doors work like on Star Trek and not in real life. The answer is: the system must obviously see much more than your prompt. It must have continuous awareness of you and what you're doing, so it can understand intent behind your short request (or action, like approaching the doors vs. passing by them) and "do what you mean" instead of act like regular computers today.
- kridsdale1 2mo agoC-3PO was neurotic and needed lots of reassurance.
- dymk 2mo agoThis feels like a subtle attempt to start an argument about Star Wars being sci-fi or not.
- perching_aix 2mo agoTo quote - what I found to be - an absolute zinger from another trending thread on here just 6 hours ago: > Typical users run software written by atypical users. https://news.ycombinator.com/item?id=49084936 https://news.ycombinator.com/item?id=49084936 This extends to everything. Anthropic has a few thousand engineers, but millions of (also engineer) users. Entire business can be built on niches that are at most a few week pet project for a team there, that can inevitably and significantly outperform them, despite being the people behind the thing. I'm sure I'm not the only one here who jumped into this whole agentic stuff, built some tooling to make things comfy, only to see that tooling all be increasingly introduced as prim and proper features in the various harnesses weeks later.
- matltc 2mo agoTotally. Ralph wiggum was added to every big shop's stdlib (harness?) just a couple weeks after it dropped. That simple iterative workflow basically underpins all of the notable/novel achievements of agentic coding or whatever. I'm sure many more examples in the "official marketplaces" for mcp/skills/what have you
- avadodin 2mo agovibe typing the mumble prompts? > we anthropig fire employes makr company run no mistkaes
- ElFitz 2mo agoFunnily enough, I grew tired of having to do that myself and sort of automated it by triggering calls to the same (or smarter) model, with the recent agent actions, turns, and outputs, and the user’s last n and first x messages. Two triggers: random and some half-reliable spiral / loop detection. The spined off has instructions to check what the agent is doing, compare it to what it’s supposed to do, and either offer suggestions, refocus it, or do nothing. And its response then gets injected in the agent’s context. Not perfect, but surprisingly effective for such a simple thing.
- matltc 2mo agoCan you explain how you piped recent agent actions/turns/outputs? I was looking into this yesterday and tried playing around with streaming json, including hooks, but i (really Claude) would need to write a parser (prob with jq) to filter the noise. Was thinking we tail -f that and select whatever we care about. Kind of an observer to catch such spiraling.
- connorboyle 2mo agoBeen skeptical for quite some time that "using AI" is a real skill/genuine field of expertise. I think I need to start sharing that skepticism more loudly and publicly! (I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization, code generation, and others)
- alwa 2mo agoI think knowing what you want (and being able to articulate it with adequate specificity, and detecting when processes drift away from it) is a real skill/field of expertise. I just don't think it's specific to interfacing with "AI."
- xp84 2mo agoIndeed. It sounds more like management (Minus the 'people skills' since you don't technically need Claude to like you... yet).
- somenameforme 2mo agoI'm curious if there has been any study on the output quality of LLMs after actively antagonizing them. It's easy to forget that at the most basic level these are just auto-complete machines, and humans tend to not respond so well after antagonized. It stands to reason that the same should follow for an auto-complete system trained on human interaction.
- groestl 2mo agoIntuitively I'd say lashing out on an LLM overconstraints it and so worsens the output, therefore the skill to stay calm and neutral is useful for guiding a machine as well. Would be useful to have data on that.
- wglb 2mo agoIn fact there has: https://arxiv.org/abs/2510.04950 https://arxiv.org/abs/2510.04950: Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
- mw888 2mo agoIn toying with VS Code or other 'agent harnesses' I'm always annoyed to find very long verbose system prompts "You are CoPilot. You are an expert. You can do anything you put your mind to..." and on and on. The token cost is amortized for longer conversations, but I find it bothersome that there's all this implicit instruction I didn't write or am now obligated to understand. I make a custom agent prompt with "Defer to the user." and little else.
- Exoristos 2mo agoI suspect that kind of prompt is vestigial of an early LLM marketing strategy. It wants to make the LLM seem like a trainable consciousness.
- thephyber 2mo agoAlternatively, Anthropic employees have basically unlimited token / model usage compared to you and I. Focusing on a better prompt is likely to get to the correct result faster than incomplete prompts and lots of "no change this ..." replies. Also, I've heard anecdotally that LLMs will underweight the earliest prompt text once context gets too long, so reminding the LLM of the most important aspects of the prompt seems to be perhaps valuable and certainly what lots of humans attempt.
- jameshart 2mo agoThese are the kinds of prompts you can get away with using when your token budget is unlimited.
- ipgleg 2mo ago[flagged]
- Barbing 2mo agoIs there a non-math example of how a genius prompts? RE: Tao’s Jacobian Conjecture chat https://news.ycombinator.com/item?id=49010345 https://news.ycombinator.com/item?id=49010345
- tedbradley 2mo agoWith smarter models, Anthropic recommends laying off on giving it too much precise hand-holding, too. Early r/promptengineering had a bunch of the craziest prompts that usually did almost nothing or nothing at all. The subreddit turned into a place where every post was trying to sell some vibe-coded website "product." Context management is still important, though. If you get to a certain amount of context, things start performing really badly.
- matltc 2mo agoPrompt engineering was deprecated around the time Opus-/Codex-class models dropped imo. Replaced by skills up until about opus 4.5/6 dropped. Pinned model to Opus 4.6 slug in ENV.ANTHROPIC_MODEL, disabled the adaptive thinking thing, and saw my skills start causing degradation Only ones I use today are for very specific quirks (eg wiredtiger/mongodb 8+ incompatible with ext4/Linux 6.19+ specifically causing segfaults. Have a 20 line mongo skill that says as much. Pinned docker container to mongo 7, can prob delete it now) I spent a few days reading up on the docs for these things, hook lifecycles, tried writing a few, but they never work as documented, or the documentation changes so frequently that whatever you built is deprecated by the time you get it humming. Now if I have some non-trivial unit of work, I basically iterate on spec in plan mode then put it on auto and let it rip. Way better results with Fable. jury out on Opus 5, but no regression like 4.7/8 Usually it's just echo "do this lil thing then pr closing issue 123" --model sonnet --effort low. Works well enough, sonnet 5 low is a workhorse and quite resourceful in a good way when things go sideways; doesn't cheat its way out IME
- tedbradley 2mo agoInteresting. So you're using Sonnet 5 on low effort as the workhorse for most parts of your plan, unless perhaps there is a part of the plan you deem too complex for it? Then, maybe throw Opus 5 or even Fable 5 at it? Have you considered using Luna on max effort for implementation? There was that recent news that they tuned its code and balancing and maybe some other stuff, dropping costs and allowing Luna to run for 20% of the API cost it had just two weeks ago. Now here's the rub: Have people with a subscription confirmed Luna max drains their usage way slower than before? With those rolling windows and the opaque "pricing" associated with them, an 80% cut to GPT-5.6 Luna might now translate into an 80% cut to using GPT-5.6 Luna with a sub. Anway, with that news, I was curious if you've tried little Luna for implementation. On https://artificialanalysis.ai/ https://artificialanalysis.ai/, for its level of "intelligence," it is cheaper than even DeepSeek. I think they want people to switch over for that alluring price cut while also giving far more usage than Anthropic. Once people stay on their plan, they make most of their profits from those same users pulling out Sol.
- hughw 2mo agoContrast that interactive, misspelled prompting, which I can relate to, against the clear, but intimidating, prompting they published with that theorem proof the other day.
- prettyblocks 2mo agoThe difference is that Anthropic engineers have infinite tokens to throw at these problems for free. The rest of us do not and that's where optimizations in how the coding harness is used likely helps the most.
- phreack 2mo agoThat's reassuring, it's exactly how I've always done LLM. It's fine, as long as the gist of what you mean is there you can be real lazy when typing. And I've always been on the cheapest of plans.
- Infinity315 2mo agoKeep in mind for Anthropic the goal for them isn't necessarily to solve a problem but to demonstrate/measure the innate capabilities of their model. Many results would be a lot less interesting/impressive for an LLM to accomplish if researchers smuggled in key premises or ideas within their prompt.