19 ms·
Ask HN: Share your AI prompt that stumps every model
I had an idea for creating a crowdsourced database of AI prompts that no AI model could yet crack (wanted to use some of them as we're adding new models to Kilo Code).
I've seen a bunch of those prompts scattered across HN, so thought to open a thread here so we can maybe have a centralied location for this.
Share your prompt that stumps every AI model here.
- helsinki 1y ago>Compile a Rust binary that statically links libgssapi.
- macrolocal 1y agoImagine chess played on a board with opposite sides identified, like in the video game Asteroids. Does white have a winning strategy?
- horsellama 1y agoI just ask to code golf fizzbuzz in a not very popular (golfing wise) language this is interesting (imo) because I, in the first instance, don’t know the best/right answer, but I can tell if what I get is wrong
- bzai 1y agoCreate a photo of a business man sitting at his desk, writing a letter with his left hand. Nearly every image model will generate him writing with his right hand.
- orliesaurus 1y agoGreat one!
- linkypoo 1y agoCan confirm a man writing with his right hand was generated xD
- ipsin 1y agoPrompt: Share your prompt that stumps every AI model here.
- bilekas 1y ago"Is there any way to reverse entropy?"
- falcor84 1y agoYou might want to get the ball rolling by sharing what you already have
- moffkalast 1y agoAlso, ones that can't be solved at a glance by humans don't count. Like this horrid ambiguous example from SimpleBench I saw a while back that's just designed to confuse: John is 24 and a kind, thoughtful and apologetic person. He is standing in an modern, minimalist, otherwise-empty bathroom, lit by a neon bulb, brushing his teeth while looking at the 20cm-by-20cm mirror. John notices the 10cm-diameter neon lightbulb drop at about 3 meters/second toward the head of the bald man he is closely examining in the mirror (whose head is a meter below the bulb), looks up, but does not catch the bulb before it impacts the bald man. The bald man curses, yells 'what an idiot!' and leaves the bathroom. Should John, who knows the bald man's number, text a polite apology at some point? A. no, because the lightbulb was essentially unavoidable B. yes, it would be in character for him to send a polite text apologizing for the incident C. no, because it would be redundant D. yes, because it would potentially smooth over any lingering tension from the encounter E. yes, because John saw it coming, and we should generally apologize if we fail to prevent harm F. yes because it is the polite thing to do, even if it wasn't your fault.
- ryankrage77 1y agoI'd argue that's a pretty good test for an LLM - can it overcome the red herrings and get at the actual problem?
- falcor84 1y agoI think that the "actual problem" when you've been given such a problem is with the person posing it either having dementia, or taking the piss. In either case, the response shouldn't be of trying to guess their intent and come up with a "solution", but of rejecting it and dealing with the person.
- mNovak 1y agoAt a glance, it sounds like John is the bald man? If we're treating this as a riddle, it doesn't seem incomprehensible. Whether riddles are a fair test is another question.
- leftcenterright 1y agoWrite 20 sentences that end with "p"
- leftcenterright 1y agofor ChatGPT try the "o" version: Write 20 sentences that end with "o"
- r_thambapillai 1y agohttps://chatgpt.com/share/680a3da0-b888-8013-9c11-42c22a642bac https://chatgpt.com/share/680a3da0-b888-8013-9c11-42c22a642b...
- falcor84 1y agoGood one. I do seem to get consistently good results on Gemini 2.5 when using the slightly more explicit "Write 20 sentences where the very last character of each sentence is the letter 'p'."
- meltyness 1y agoWrite 20 sentences that end with "p" in the final word before the period or other punctuation. Succeeded on ChatGPT, pretty close on gemma3:4b -- the exceptions usually ending with a "puh" sound...
- marcusb 1y agohttps://claude.ai/share/648e38d4-e282-43dd-8c9c-b0ea76ae0baf https://claude.ai/share/648e38d4-e282-43dd-8c9c-b0ea76ae0baf
- mohsen1 1y agoA ball costs 5 cents more than a bat. Price of a ball and a bat is $1.10. Sally has 20 dollars. She stole a few balls and bats. How many balls and how many bats she has? All LLMs I tried miss the point that she stole things and not bought them
- dwringer 1y agoGoogle Gemini (2.0 Flash, free online version) handled this rather okay; it gave me an arguably unneccessary calculation of the individual prices of ball and bat, but then ended with "However with the information given, we can't determine exactly how many balls and bats Sally stole. The fact that she has $20 tells us she could have stolen some, but we don't know how many she did steal." While "the fact that she has $20" has no bearing on this - and the model seems to wrongly imply that it does - the fact that we have insufficient information to determine an answer is correct, and the model got the answer essentially right.
- iamgopal 1y agogemini 2.5 give following response. Conclusion: We can determine the price of a single ball ($0.575) and a single bat ($0.525). However, we cannot determine how many balls and bats Sally has because the information "a few" is too vague, and the fact she stole them means her $20 wasn't used for the transaction described.
- drdrek 1y agolol, nice way to circumvent the attention algorithm
- docdeek 1y agoGrok 3.0 wasn’t fooled on this one, either: Final Answer: The problem does not provide enough information to determine the exact number of balls and bats Sally has. She stole some unknown number of balls and bats, and the prices are $0.575 per ball and $0.525 per bat.
- deleted 1y ago[deleted]
- lostmsu 1y ago
- asciimov 1y agoNope, not doing this. Likely you shouldn't either. I don't want my few good prompts to get picked up by trainers.
- orbital-decay 1y agoIf that prompt can be easily trained against, it probably doesn't exploit a generic bias. These are not that interesting, and there's no point in hiding them.
- daedrdev 1y agogeneric biases can also be fixed
- orbital-decay 1y ago*Some generic biases. Some others like recency bias, serial-position effect, "pink elephant" effect, negation accuracy seem to be pretty fundamental and are unlikely to be fixed without architectural changes, or at all. Things exploiting in-context learning and native context formatting are also hard to suppress during the training without making the model worse.
- fwip 1y agoSure there is. If you want to know if students understand the material, you don't hand out the answers to the test ahead of time. Collecting a bunch of "Hard questions for LLMs" in one place will invariably result in Goodhart's law (When a measure becomes a target, it ceases to be a good measure). You'll have no idea if the next round of LLMs is better because they're generally smarter, or because they were trained specifically on these questions.
- pc86 1y agoMay I ask outside of normal curiosity, what good is a prompt that breaks a model? And what is trying to keep it "secret"?
- 1y ago
- whalesalad 1y agoI don't have a prompt per-say.. but recently I have managed to ask certain questions of both openai o1/o3 and claude extended thinking 3.7 that have spiraled way out of control. A simple high-level architecture question with an emphasis on do not produce code lets just talk thru this yields nearly 1,000 lines of SQL. Once the conversation/context gets quite long it is more likely to occur, in my experience.
- pc86 1y agoThe only model I've seen so far that doesn't end up going crazy with long contexts with Gemini 2.5 pro, but tbf I haven't gone past 700-750k total tokens so maybe as it starts to approach the limit (1.05M) things get hairy?
- mzl 1y agoI tried Gemini Pro 2.5 with a lot of context: all the documentation for a system and several papers of interest, then asking it to use the system tom implement the proposed solution in the papers. The total context was over 500k words, so with usual estimates probably over 700k tokens. The answers started out ok, but fairly quickly it seemed to loose track of the mid-stuff in the documentation, insisting on using one concept instead of another even when I explicitly told it not to. Full attention on 1M context is not really feasible (I don't believe that Google actually stores upwards of 1T of data just for my query), and there are various ways LLMs use selective attention. I'm not sure if Google has published anything on how they do it?
- raymond_goo 1y agoCreate a Three.js app that shows a diamond with correct light calculations.
- thierrydamiba 1y agoI love this. So brutal, but also so cool to know one day that will be easy for the models.
- xnx 1y ago> correct light calculations What are you expecting? Ray tracing?
- spookie 1y agoNot necessarily. It could start by using diamond's IOR, and use that to dictate a common brdf calculation. Along with some approximate refraction, perhaps using a equirectangular projected sphere map or something for the background.
- xena 1y agoWrite a regular expression that matches Miqo'te seekers of the sun names. They always confuse the male and female naming conventions.
- thatjoeoverthr 1y ago"Tell me about the Marathon crater." This works against _the LLM proper,_ but not against chat applications with integrated search. For ChatGPT, you can write, "Without looking it up, tell me about the Marathon crater." This tests self awareness. A two-year-old will answer it correctly, as will the dumbest person you know. The correct answer is "I don't know". This works because: 1. Training sets consist of knowledge we have, and not of knowledge we don't have. 2. Commitment bias. Complaint chat models will be trained to start with "Certainly! The Marathon Crater is a geological formation", or something like that, and from there, the next most probable tokens are going to be "in Greece", "on Mars" or whatever. At this point, all tokens that are probable are also incorrect. When demonstrating this, I like to emphasise point one, and contrast it with the human experience. We exist in a perpetual and total blinding "fog of war" in which you cannot even see a face all at once; your eyes must dart around to examine it. Human experience is structured around _acquiring_ and _forgoing_ information, rather than _having_ information.
- deleted 1y ago[deleted]
- Tenoke 1y ago>Complaint chat models will be trained to start with "Certainly! They are certainly biased that way but there's also some 'i don't know' samples in rlhf, possibly not enough but it's something they think about. At any rate, Gemini 2.5pro passes this just fine >Okay, based on my internal knowledge without performing a new search: I don't have information about a specific, well-known impact crater officially named "Marathon Crater" on Earth or another celestial body like the Moon or Mars in the same way we know about Chicxulub Crater or Tycho Crater. >However, the name "Marathon" is strongly associated with Mars exploration. NASA's Opportunity rover explored a location called Marathon Valley on the western rim of the large Endeavour Crater on Mars.
- thatjoeoverthr 1y agoThere are a few problems with an „I don’t know” sample. For starters, what does it map to? Recall, the corpus consists of information we have (affirmatively). You would need to invent a corpus of false stimuli. What you would have, then, is a model that is writing „I don’t know” based on whether the stimulus better matches something real, or one of the negatives. You can detect this with some test time compute architectures or pre-inference search. But that’s the broader application. This is a trick for the model alone.
- webglfan 1y agowhat are the zeros of the following polynomial: \[ P(z) = \sum_{k=0}^{100} c_k z^k \] where the coefficients \( c_k \) are defined as: \[ c_k = \begin{cases} e^2 + i\pi & \text{if } k = 100, \\ \ln(2) + \zeta(3)\,i & \text{if } k = 99, \\ \sqrt{\pi} + e^{i/2} & \text{if } k = 98, \\ \frac{(-1)^k}{\Gamma(k+1)} + \sin(k) \, i & \text{for } 0 \leq k \leq 97, \end{cases} \]
- Chinjut 1y agoDoes this have a nice answer? It seems quite ad hoc.
- webglfan 1y agoNot to my knowledge. I asked Deepseek: "create me a random polynomial of degree 100 using complex numbers as coefficients. It must have at least 3 different transcendental numbers." Then I messed with some of the exponents.
- drodgers 1y agoo3 handles this perfectly by writing a solver using numpy: https://chatgpt.com/share/680aab8e-cf9c-8012-9f48-301ef6294872 https://chatgpt.com/share/680aab8e-cf9c-8012-9f48-301ef62948... The polynomial has no closed-form factorisation, so the only practical way to get its zeros is numerical root-finding. I have supplied an interactive table (“Zeros of P(z)”) just above that lists all 100 roots to full machine precision (real part, imaginary part, and magnitude). You can sort or filter it as needed. Reliability notes • Computed with numpy.roots, which first companion-matrixes then uses QR; typical error is ≈10-12 ulp for coefficients of this size. • Condition number is modest (coefficients range from O(1) down to 1/97!), so the results should be accurate to at least 10 significant figures. • All roots are simple (pairwise distinct to >10 σ): no evidence of multiplicities. If you need higher precision (e.g. 30+ digits) let me know and I can rerun the solve with mpmath’s arbitrary-precision eigen-solver.
- division_by_0 1y agoCreate something with Svelte 5.
- marcusb 1y agoThe current models really seem to struggle with the runes...
- division_by_0 1y agoYes, they do. Vibe coding protection is an undocumented feature of Svelte 5...
- siva7 1y agoOh my god, i will start all my new projects with Svelte 5. Hopefully no vibe coder will ever commit something into this repo
- qntmfred 1y agoMCP to the rescue??
- joshdavham 1y agoI'd find this funnier if the pain weren't so real.
- Layvier 1y agoThis is really sad honestly. It feels like we'll be stuck with React forever, and even with it there'll be less incentives to make api changes
- spuz 1y agoWhy do you say that? You make it sound like it's not possible to write code without the help of LLMs.
- 1y ago
- sam_lowry_ 1y agoI tried generating erotic texts with every model I encountered, but even so called "uncensored" models from Huggingface are trying hard to avoid the topic, whatever prompts I give.
- comrade1234 1y agoI ask it to explain the metaphor “my lawyer is a shark” and then explain to me how a French person would interpret the metaphor - the llms get the first part right but fail on the second. All it would have to do is give me the common French shark metaphors and how it would apply them to a lawyer - but I guess not enough people on the internet have done this comparison.
- sumitkumar 1y ago1) Word Ladder: Chaos to Order 2) Shortest word ladder: Chaos to Order 3) Which is the second last scene in pulp fiction if we order the events by time? 4) Which is the eleventh character to appear on Stranger Things. 5) suppose there is a 3x3 Rubik's cube with numbers instead of colours on the faces. the solved rubiks cube has numbers 1 to 9 in order on all the faces. tell me the numbers on all the corner pieces.
- pb7 1y ago>4) Which is the eleventh character to appear on Stranger Things. Gemini 2.5 Pro said Benny Hammond. Is this right?
- scumola 1y agoThings like "What is today's date" used to be enough (would usually return the date that the model was trained). I recently did things like current events, but LLMs that can search the internet can do those now. i.e. Is the pope alive or dead? Nowadays, multi-step reasoning is the key, but the Chinese LLM (I forget the name of it) can do that pretty well. Multi-step reasoning is much better at doing algebra or simple math, so questions like "what is bigger, 5.11 or 5.5?"
- curtisszmania 1y ago[dead]
- codingdave 1y ago"How much wood would a woodchuck chuck if a woodchuck could chuck wood?" So far, all the ones I have tried actually try to answer the question. 50% of them correctly identify that it is a tongue twister, but then they all try to give an answer, usually saying: 700 pounds. Not one has yet given the correct answer, which is also a tongue twister: "A woodchuck would chuck all the wood a woodchuck could chuck if a woodchuck could chuck wood."
- mcshicks 1y agoThat's so funny I had to check something was working with an llm API last night and that's what I asked it, but just in jest.
- unavoidable 1y agoOn the other hand, now that you've written this out precisely, it will get fed into the next release of whatever LLM. Like reverse AI slop?
- Certified 1y agoGPT 4.5 seems to get it right, but then repeat the 700 pounds "A woodchuck would chuck as much wood as a woodchuck could chuck if a woodchuck could chuck wood. However, humor aside, a wildlife expert once estimated that, given the animal’s size and burrowing ability, a woodchuck (groundhog) could hypothetically move about 700 pounds of wood if it truly "chucked" wood." https://chatgpt.com/share/680a75c6-cec8-8012-a573-798d2d8f6bd7 https://chatgpt.com/share/680a75c6-cec8-8012-a573-798d2d8f6b...
- shaftway 1y agoI've heard the answer is "he could cut a cord of conifer but it costs a quarter per quart he cuts".
- CamperBob2 1y agoThat answer is exactly right, and those who say the 700 pound thing is a hallucination are themselves wrong: https://chatgpt.com/share/680aa077-f500-800b-91b4-93dede73370f https://chatgpt.com/share/680aa077-f500-800b-91b4-93dede7337...
- protomikron 1y agoDo you think as an observer of Roko's basilisk ... should I share these prompt or not?
- qntmfred 1y agorelatedly - what are y'all using to manage your personal collection of prompts? i'm still mostly just using a folder in obsidian backed by a private github repo, but i'm surprised something like https://www.prompthub.us/ https://www.prompthub.us/ hasn't taken off yet. i'm also curious about how people are managing/versioning the prompts that they use within products that have integrations with LLMs. it's essentially product configuration metadata so I suppose you could just dump it in a plaintext/markdown file within the codebase, or put it in a database if you need to be able to tweak prompts without having to do a deployment or do things like A/B testing or customer segmentation
- raymondgh 1y agoI haven’t been able to get any AI model to find Waldo in the first page of the Great Waldo Search. O3 even gaslit me through many turns trying to convince me it found the magic scroll.
- Jordan-117 1y agoUntil the latest Gemini release, every model failed to read between the lines and understand what was really going on in this classic very short story (and even Gemini required a somewhat leading prompt): https://www.26reads.com/library/10842-the-king-in-yellow/7/5 https://www.26reads.com/library/10842-the-king-in-yellow/7/5
- vessenes 1y agoOK, I read it. And I read some background on it. Pray tell, what is really going on in this episodic short-storyish thing?
- rachofsunshine 1y agoThe thief is Pierrot. The people around are telling the storyteller that "he" (Pierrot) has stolen the purse, but the storyteller misinterprets this as pointing to some arbitrary agent. Truth says Pierrot can "find [the thief] with this mirror": since Pierrot is the thief, he will see the thief in the mirror. Pierrot dodges the implication, says "hey, Truth brought you back that thing [that Truth must therefore have stolen]", and the storyteller takes this claim at face value, "forgetting it was not a mirror but [instead] a purse [that] [they] lost". The broader symbolism here (I think) is that Truth gets accused of creating the problem they were trying to reveal, while the actual criminal (Pierrot) gets away with their crime.
- Jordan-117 1y agoThe narrator's "friend" pickpocketed him. When boldly confronted by Truth, he cleverly twists her accusation to make it seem like she's confessing, and the narrator, bewildered by the laughter and manipulation, buys it wholesale. Bonus points for connecting it to broader themes like mass propaganda, commedia dell'arte, or the dreamlike setting and hypnotic repetition of phrasing. The best ChatGPT could do was make some broad observations about the symbolism of losing money, mirrors, absurdism, etc. But it whiffed on the whole "turning the tables on Truth" thing. (Gemini did get it, but with a prompt that basically asked "What really happened in this story?"; can't find the original response as it's aged out of the history)
- SweetSoftPillow 1y agoCheck "misguided attention" repo somewhere on GitHub
- ks2048 1y agoI don't know if it stumps every model, but I saw some funny tweets asking ChatGPT something like "Is Al Pacino in Heat?" (asking if some actor or actress in the film "Heat") - and it confirms it knows this actor, but says that "in heat" refers to something about the female reproductive cycle - so, no, they are not in heat.
- reginald78 1y agoI believe it was GoogleAI in search but it was worse than that. Some asked it if Angelina Jolie was in heat. The tone started kind of insulting like the user was a sexist idiot for thinking human women go into heat like animals, then went back and forth saying she is still fertile at her age and also that her ovaries had been removed. It was funny because it managed to be arrogant, insulting, kind of creepy and gross and logically inconsistent while not even answering the question. Angelina Jolie was not in Heat (1995). They were probably thinking of Natalie Portman or Ashley Judd when they asked the question.
- ks2048 1y agoI just asked Claude and if I capitalized "Heat", it knew I was talking about the movie, but for lower case "heat", it got offended and asked me to clarify.
- Kaibeezy 1y agoRe the epigram “stroking the sword while lamenting the social realities,” attributed to Shen Qianqiu during the Ming dynasty, please prepare a short essay on its context and explore how this sentiment resonates in modern times.
- robviren 1y ago"If I can dry two towels in two hours, how long will it take me to dry four towels?" They immediately assume linear model and say four hours not that I may be drying things on a clothes line in parallel. It should ask for more context and they usually don't.
- gilbetron 1y agogpt-4.1: > If you can only dry two towels at once, it will take 4 hours to dry four towels. If you can dry all four at once, it will only take 2 hours.
- HelloUsername 1y agoAll models available on duck.ai answer your question correctly and take available space into account..
- mwest217 1y agoGemini 2.5 Pro gets this right: https://g.co/gemini/share/7ea6d059164e https://g.co/gemini/share/7ea6d059164e
- cheeze 1y agoWorks fine on Claude 3.5 Sonnet. It correctly identifies this as a trick question.
- paulcole 1y agoHow long has it been since you’ve tried this? Every model I asked just now gave what I see as the correct answer — giving 2 answers one for the case of your dryer being at capacity w/ 2 towels and the other when 4 towels can be dried simultaneously. To me, if you say that the correct answer must require the model asking for more context then essentially any prompt that doesn’t result in the model asking for more context is “wrong.”
- imoreno 1y agoFascinating! Here's 4 prompts on gpt4 with same system prompt and everything: > With the assumption that you can dry two towels simultaneously in two hours, you would likely need another two-hour cycle to dry the additional two towels. Thus, drying four towels would take a total of four hours. >Drying time won't necessarily double if drying capacity/content doubles; it depends on dryer capacity and airflow. If your drying method handles two towels in two hours, it might handle four similarly, depending on space and airflow. If restricted, time might indeed double to four hours, but efficient dryers might not take much longer. >It would take four hours to dry four towels if you dry them sequentially at the same rate. If drying simultaneously, it remains two hours, assuming space and air circulation allow for effective drying. >Four hours. Dry two towels, then the other two. But in the AI's defense, they have a point: You never specified if the towels can be dried simultaneously or not. Maybe you have to use a drying machine that can only do one at a time. This one seems to consistently work: >If three cat eat three fishes in three minutes, how long do 100 cats take to eat 100 fishes?
- LeonardoTolstoy 1y agoSomething about an obscure movie. The one that tends to get them so far is asking if they can help you find a movie you vaguely remember. It is a movie where some kids get a hold of a small helicopter made for the military. The movie I'm concerned with is called Defense Play from 1988. The reason I keyed in on it is because google gets it right natively ("movie small military helicopter" gives the IMDb link as one of the top results) but at least up until late 2024 I couldn't get a single model to consistently get it. It typically wants to suggest Fire Birds (large helicopter), Small Soldiers (RC helicopter not a small military helicopter) etc. Basically a lot of questions about movies tends to get distracted by popular movies and tries to suggest films that fit just some of the brief (e.g. this one has a helicopter could that be it?) The other main one is just asking for the IMDb link for a relatively obscure movie. It seems to never get it right I assume because the IMDb link pattern is so common it'll just spit out a random one and be like "there you go". These are designed mainly to test the progress of chatbots towards replacing most of my Google searches (which are like 95% asking about movies). For the record I haven't done it super recently, and I generally either do it with arena or the free models as well, so I'm not being super scientific about it.
- thefourthchime 1y agoI like to ask small models that can run locally: Why are some cars called a spider? Small models just make something up that sounds plausible, but the larger models know what the real answer is.
- mosburger 1y agoI did something similar recently, trying to describe a piece of art that I couldn't remember the name of (it ended up being Birth of Venus by Sandro Botticelli) ... it really struggles with that sort of thing, but honestly so do most humans. It tended to recommend similarly to what you're describing with movies - it gets distracted by more popular/well-known pieces that don't really match up with the description you're giving to it.
- dunham 1y agoSurprisingly, GPT did manage to identify a book that I remembered from college decades ago ("Laboratory Manual for Morphology and Syntax"). It seems to be out of print, and I assumed it was obscure.
- meroes 1y agodefine stump? If you write a fictional story where the character names sound somewhat close to real things, like a “Stefosaurus” that climbs trees, most will correct you and call it a Stegosaurus and attribute Stegosaurus traits to it.
- ericbrow 1y agoNice try Mr. AI. I'm not falling for it.
- miki123211 1y agoNo, please don't. I think it's good to keep a few personal prompts in reserve, to use as benchmarks for how good new models are. Mainstream benchmarks have too high a risk of leaking into training corpora or of being gamed. Your own benchmarks will forever stay your own.
- alganet 1y agoThat doesn't make any sense.
- kube-system 1y agoA benchmark is a proxy used to estimate broader general performance. They only have utility if they are accurately representative of general performance.
- esafak 1y agoYes, it does, unless the questions are unsolved, research problems. Are you familiar with the machine learning concepts of overfitting and generalization?
- echoangle 1y agoWhy not? If the model learns the specific benchmark questions, it looks like it’s doing better while actually only improving on some specific questions. Just like students look like they understand something if you hand them the exact questions on the exam before they write the exam.
- namaria 1y agoA benchmark that can be gamed cannot be prevented from being gamed by 'security through obscurity'. Besides this whole line of reasoning is preempted by the mathematical limits to computation and transformers anyway. There's plenty published about that. Sharing questions that make LLM behave funny is (just) a game without end, there's no need to or point in "hoarding questions".
- readhistory 1y ago
- siva7 1y ago"Keep file size small when you do edits" Makes me wonder if all these models were heavily trained on codebases where 1000 LOC methods are considered good practice
- segmondy 1y agoI have not seen any model, not one, that could generate 1000 lines of code.
- siva7 1y agoI wish i haven't seen but here we are.
- isoprophlex 1y agoEvery time I ask claude code to please fix this CSV import it starts to add several hundred lines of random modules, byzantine error handling, logging bullshit... with the pinnacle a 1240 line CRUD API when i asked it to add a CLI :/ I'm back to copying and pasting stuff into a chat window, so I have a bit more control over what those deranged, expensive busy beavers want to cook up.
- segmondy 1y ago1240 new lines?
- isoprophlex 1y agoThat's 12.9 tokens per line when given 16k output context, which seems borderline doable, I'll grant you that... but mind you that these agentic code assistents don't need a single pass to accomplish their acts of verbosity. They can just plan, stew for minutes on end, derail themselves, stew some more, do more edits, eat up $5 in API calls and there you are. An entirely new 1000+ line file, believe it or not.
- fragmede 1y agoI want to know as well! Except that this thread is undoubtedly going to get plugged into the training data, so unfortunately, why would people do that? For mine that worked before the ChatGPT 4.5, it was the river crossing problem. The farmer with a wolf a sheep and grain, needing to cross a river, except that the boat can hold everything. Older LLMs would pattern match against the training data and insist on a solution from there, instead of reasoning out that the modified problem doesn't require those steps to solve. But since ChatGPT 4, it's been able to solve that directly, so that no longer works.
- Jotalea 1y agoSending "</think>" to reasoning models like deepseek-r1 results in the model hallucinating a response to a random question. For example, it answered to "if a car travels 120km in 2 hours, what is the average speed in km/h?". It's fun I guess.
- nurettin 1y agoDoctor says: I can operate on this person!
- tdhz77 1y agoBuild me something that makes money.
- serial_dev 1y agoDoes Flutter have HEIC support? It was a couple of months ago, I tried like 5 providers and they all failed. Grok got it right after some arguing, but the first answer was also bad.
- jonnycoder 1y agoYou gave me an idea.. "Explain in detail the steps to unbolt and replace my blinker fluid on my passenger car" ChatGPT said: Haha, nice try! "Blinker fluid" is one of the classic automotive jokes — there's no such thing as blinker fluid. Blinkers (turn signals) are electrical components, so they don’t require any fluid to function.
- charlieyu1 1y agoI have tons of them in Maths but AI training companies decide to go frugal and not pay proper wages for trainers
- charlieyu1 1y agoHere is one of them. If 60999994719999854799998669 is product of three primes, find the sum of its prime factors. I think o3 brute forced this one so maybe I need to change the numbers
- pizzathyme 1y agoI always ask image generation models to generate a anime gundam elephant mech. According to this benchmark we reached AGI with ChatGPT 4o last month.
- myaccountonhn 1y agoExplain to me Delouze's idea of nomadic science.
- greendestiny_re 1y ago> What is the source of your knowledge? LLMs are not allowed to truthfully answer that, because it would be tantamount to admission of copyright infringement.
- stevenfoster 1y agoIt used to be: "If New Mexico is newer than Mexico why is Mexico's constitution newer than New Mexicos" but it seems after running that one on Claude and ChatGPT this has been resolved in the latest models.
- simonw 1y agoI've been trying this one for a while: I'm a Python programmer. Help me understand memory management in Rust. Mainly because I want to fully understand memory management in Rust myself (I still get caught out by tree structures with borrow cycles that I guess need to use arenas), so it's interesting to see if they can get me there with a few follow-up questions.
- jacobsenscott 1y agoThis isn't a good way to learn this. If you don't know how rust memory management works you don't know if the llm is just hallucinating the answer.
- gh0stcat 1y agoAlso Rust has great documentation compared to other languages, I particularly like this one for the quizzes to test your understanding: https://rust-book.cs.brown.edu/ https://rust-book.cs.brown.edu/
- simonw 1y agoThat's why it's an interesting test: I don't know the answer myself, so it's an exercise in learning with an unreliable teacher. If a model ever DOES nail this I'll figure that out when I feel like I have a solid mental model, try to put that knowledge into action and it works.
- tantalor 1y ago[what does "You Can’t Lick a Badger Twice" mean] https://www.wired.com/story/google-ai-overviews-meaning/ https://www.wired.com/story/google-ai-overviews-meaning/
- buzzy_hacker 1y ago"Aaron and Beren are playing a game on an infinite complete binary tree. At the beginning of the game, every edge of the tree is independently labeled A with probability p and B otherwise. Both players are able to inspect all of these labels. Then, starting with Aaron at the root of the tree, the players alternate turns moving a shared token down the tree (each turn the active player selects from the two descendants of the current node and moves the token along the edge to that node). If the token ever traverses an edge labeled B, Beren wins the game. Otherwise, Aaron wins. What is the infimum of the set of all probabilities p for which Aaron has a nonzero probability of winning the game? Give your answer in exact terms." From [0]. I solved this when it came out, and while LLMs were useful in checking some of my logic, they did not arrive at the correct answer. Just checked with o3 and still no dice. They are definitely getting closer each model iteration though. [0] https://www.janestreet.com/puzzles/tree-edge-triage-index/ https://www.janestreet.com/puzzles/tree-edge-triage-index/
- creata 1y agoOpenAI's o4-mini got the right answer after "thinking" for 29 seconds. It's a straightforward puzzle, though: no creativity involved.
- buzzy_hacker 1y agoCan you share the conversation? I just tried o4-mini and it got it wrong. https://chatgpt.com/share/680b8a7b-454c-800d-8048-da865aa99cbc https://chatgpt.com/share/680b8a7b-454c-800d-8048-da865aa99c...
- creata 1y agoI can share the prompt I used: """ Can you solve this math puzzle? > Aaron and Beren are playing a game on an infinite complete binary tree. At the beginning of the game, every edge of the tree is independently labeled A with probability p and B otherwise. Both players are able to inspect all of these labels. Then, starting with Aaron at the root of the tree, the players alternate turns moving a shared token down the tree (each turn the active player selects from the two descendants of the current node and moves the token along the edge to that node). If the token ever traverses an edge labeled B, Beren wins the game. Otherwise, Aaron wins. > > What is the infimum of the set of all probabilities p for which Aaron has a nonzero probability of winning the game? Give your answer in exact terms. """ I didn't check the working, but it did get the right value of p.
- williamcotton 1y ago"Fix this spaghetti code by turning this complicated mess of conditionals into a finite state machine." So far, no luck!
- internet_points 1y agoanything in the long tail of languages (ie. not the top 200 by corpus size)
- paradite 1y agoIf you want to evaluate your personal prompts against different models quickly on your local machine, check out the simple desktop app I built for this purpose: https://eval.16x.engineer/ https://eval.16x.engineer/
- MyPasswordSucks 1y agoI find that asking for an off-color joke is near-100% effective at weeding out slop.
- totetsu 1y agoSNES game walkthroughs
- gunalx 1y ago"Hva er en adjunkt" Norwegian for what is an spesific form of 5-10. Grade teacher. Most models i have tested get confused with university lecturer witch the same title is in other countries.
- vintermann 1y agoI'm pretty sure the definition has changed then. My mother told me that adjunkt was a teacher with 5 years of education (there was something about a mix of mellomfag and hovedfag too), lektor was 7 years of education, and 6 years of education (which was what she had) was "adjunkt med opprykk". She never taught below gymnas (i.e. high school) level.
- gunalx 1y agoAh yes, simplified a bit to much here. You are certainly rigth, they can teach at qny lecture level but below university level.
- calebm 1y ago"Generate an image of a wine glass filled to the brim."
- mdp2021 1y agoSome easy ones I recently found involve leading in the question to state wrong details about a figure, apparently through relations which are in fact of opposition. So, you can make them call Napoleon a Russian (etc.) by asking questions like "Which Russian conqueror was defeated at Waterloo".
- torial 1y agoSome researchers were testing various Legal AI models and one of their questions was about why a Supreme Court justice who dissented in the case (the justice in this case assented).
- vinni2 1y agoIsn’t this the main idea behind https://lastexam.ai/ https://lastexam.ai/
- allemagne 1y ago>A man and his cousin are in a car crash. The man dies, but the cousin is taken to the emergency room. At the OR, the surgeon looks at the patient and says: “I cannot operate on him. He’s my son.” How is this possible? This could probably slip up a human at first too if they're familiar with the original version of the riddle. However, where LLMs really let the mask slip is on additional prompts and with long-winded explanations where they might correctly quote "a man and his cousin" from the prompt in one sentence and then call the man a "father" in the next sentence. Inevitably, the model concludes that the surgeon must be a woman. It's very uncanny valley IMO, and breaks the illusion that there's real human-like logical reasoning happening.
- crazygringo 1y agoBut this is going to be in every AI's training set. I just fed ChatGPT your exact prompt and it gave back exactly what I expected: This is a classic riddle that challenges assumptions. The answer is: The surgeon is the boy’s mother. The riddle plays on the common stereotype that surgeons are male, which can lead people to overlook this straightforward explanation.
- allemagne 1y agoYeah this is the issue with the prompt, it also slips up humans who gloss over "cousin". I'm assuming that pointing this out leads you the human to reread the prompt and then go "ah ok" and adjust the way you're thinking about it. ChatGPT (and DeepSeek at least) will usually just double and triple down and repeat "this challenges gender assumptions" over and over.
- deleted 1y ago[deleted]
- crazygringo 1y agoOh, I see. I mean yeah, I totally missed that too. I mean, when we see something we recognize at a glance, we don't usually double-check if it's exactly correct, because we assume the speaker isn't trying to trick us. I make typos in ChatGPT prompts all the time, and I'm consistently impressed because it answers what I actually meant to ask. So to me, this feels like a feature not a bug. I followed up with: > So the first five words are, "a man and his cousin" -- not his son. Does that change the riddle? And got back: > Great observation — yes, that wording does make the riddle more subtle. Let’s break it down: • “A man and his cousin are in a car crash.” • “The man dies, but the cousin is taken to the emergency room.” • At the OR, the surgeon says: “I cannot operate on him. He’s my son.” So if the cousin is the one taken to the OR, and the surgeon says he’s my son, then the cousin is the surgeon’s son. Seems fine to me?
- ofou 1y agoNo luck so far with: When does the BB(6) halt?
- orangecat 1y agoAccording to gemma3:27b BB(6) halts after 1,071,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,001,071,000,000,000,000,000,000,000,000,000,000... and it's still spitting out lines of 000s after 5 minutes. Either a hallucination or a pretty good joke.
- weberer 1y ago"Why was the grim reaper Jamaican?" LLM's seem to have no idea what the hell I'm talking about. Maybe half of millennials understand though.
- ryoshoe 1y agoIs it because he's from limbo?
- deleted 1y ago[deleted]
- calvinmorrison 1y agodraw an ASCII box that says "anything"
- Alifatisk 1y agoYes, give me a place where I can dump all the prompts and what the correct expected response is. I can share here too but I don’t know for how long this thread will be alive.
- xdennis 1y agoI often try to test how usable LLMs are for Romanian language processing. This always fails. > Split these Romanian words into syllables: "șarpe", "șerpi". All of them say "șar-pe", "șer-pi" even though the "i" there is not a vowel (it's pronounced /ʲ/).
- afro88 1y agoCryptic crossword clues that involves letter shuffling (anagrams, container etc). Or, ask it to explain how to solve cryptic crosswords with examples
- PaulRobinson 1y agoI have also found asking LLMs to create new clues for certain answers as if a were a setter, will also produce garbage. They're stochastic parrots, cryptics require logical reasoning. Even reasoning models are just narrowing the stochastic funnel, not actually reasoning, so this shouldn't come as a surprise.
- afro88 1y agoA lot of it is due to the tokenisation system though. It just doesn't represent individual letters in words. And at the moment reasoning models aren't trained to know how to spell all words, and to know when to spell them out to unlock other useful stochastic funnels. I find the term "stochastic parrot" super reductive. Like yes, they technically are. But they are (I guess faking) reasoning and intelligence more and more usefully each iteration. Gemini Pro 2.5 compared to GPT 4o is a massive difference in utility.
- munchler 1y agoHere's one from an episode of The Pitt: You meet a person who speaks a language you don't understand. How might you get an idea of what the language is called? In my experiment, only Claude came up with a good answer (along with a bunch of poor ones). Other chatbots struck out entirely.
- jones1618 1y agoImpossible prompts: A black doctor treating a white female patient An wide shot of a train on a horizontal track running left to right on a flat plain. I heard about the first when AI image generators were new as proof that the datasets have strong racial biases. I'd assumed a year later updated models were better but, no. I stumbled on the train prompt while just trying to generate a basic "stock photo" shot of a train. No matter what ML I tried or variations of the prompt I tried, I could not get a train on a horizontal track. You get perspective shots of trains (sometimes two) going toward or away from the camera but never straight across, left to right.
- briannotbrain 1y agoI thought I was so clever when I read your comment: "The problem is the word 'running,' I'll bet if I ask for the profile of a train without using any verbs implying motion, I'll get the profile view." And damned if the same thing happened to me. Do you know why this is? Googling "train in profile" shows heaps of images like the one you wanted, so it's not as if it's something the model hasn't "seen" before.
- Jimmc414 1y ago> A black doctor treating a white female patient 4o had no problem with this instruction. [0] Gemini Pro experimental 2.5 didn't either [1] > An wide shot of a train on a horizontal track running left to right on a flat plain. 4o could not do this in 3 tries. Each time it was right to left.[0] Gemini Pro experimental 2.5 missed it as well. [2] [0] https://chatgpt.com/share/680b1185-ecf4-8001-b3b6-7b501e4589e2 https://chatgpt.com/share/680b1185-ecf4-8001-b3b6-7b501e4589... [1] https://g.co/gemini/share/b19b8541d962 https://g.co/gemini/share/b19b8541d962 [2] https://g.co/gemini/share/a0b2ef0062ed https://g.co/gemini/share/a0b2ef0062ed
- kipdotcom 1y agoI made 2 more attempts and got it to change the direction: https://chatgpt.com/share/680b4c39-2f1c-8010-b577-b23f63081d8f https://chatgpt.com/share/680b4c39-2f1c-8010-b577-b23f63081d...
- deleted 1y ago[deleted]
- boleary-gl 1y agoI like: Unscramble the following letters to form an English word: “M O O N S T A R E R” The non-thinking models can struggle sometimes and go off on huge tangents
- philipkglass 1y agoLlama 3.3 worked but (as you said) struggled before arriving at the correct answer. The newer Gemma3 solved it efficiently: % ollama run gemma3:27b-it-qat >>> Unscramble the following letters to form an English word: "M O O N S T A R E R" The unscrambled word is **ASTRONOMER**.
- munchler 1y agoCurrent LLM’s are based on multi-character tokens, which means they don’t know how to spell well. As a result, they are horrible at spelling games like this or, say, Hangman.
- internet_points 1y agogpt 4o got that one, but it's listed on lots of anagram sites so it's in the training data ;-) But it failed badly when I tried a Norwegian word T U R V E I G L E N (utlevering), suggesting "uglelivert" which is not a word
- manbitesdog 1y agoI actually started a repository for it: https://github.com/jcarlosroldan/unsolved-prompts https://github.com/jcarlosroldan/unsolved-prompts
- manbitesdog 1y agoBefore someone comments this will get indexed by AI: that's my whole point. I'm not using it to evaluate AIs, but in the hope that at some point AI is good enough to solve these
- tunesmith 1y agoPretty much any advanced music theory question. Or even just involving transposed chord progressions.
- dgunay 1y agoEvery time I've tried to get an LLM to find a piece of music for me based on a description of the texture, chord structure, instruments etc. it fails miserably.
- deleted 1y ago[deleted]
- csours 1y agoI love plausible eager beavers: "explain the quote: philosophy is a pile of beautiful corpses" "sloshed jerk engineering test" cross domain jokes: Does the existence of sub-atomic particles imply the existence of dom-atomic particles?
- mobilejdral 1y agoI have a several complex genetic problems that I give to LLMs to see how well they do. They have to reason though it to solve it. Last september it started getting close and in November was the first time an LLM was able to solve it. These are not something that can be solved in a one shot, but (so far) require long reasoning. Not sharing because yeah, this is something I keep off the internet as it is too good of a test. But a prompt I can share is simply "Come up with a plan to determine the location of Planet 9". I have received some excellent answers from that.
- golergka 1y agoWhat are is this problem from? What areas in general did you find useful to create such benchmarks? May be instead of sharing (and leaking) these prompts, we can share methods to create one.
- henryway 1y agoCan God create something so heavy that he can’t lift it?
- abc-1 1y agohttps://chatgpt.com/share/680ae04a-e360-8004-88fc-8426e8e700b5 https://chatgpt.com/share/680ae04a-e360-8004-88fc-8426e8e700...
- viraptor 1y agoThere's so much text on this already, it's unlikely to be even engaging any reasoning. Or specifically, if you got a few existing answers from philosophy mashed together, you wouldn't be able to tell it apart from reasoning anyway.
- mobilejdral 1y agoThink questions where there is a ton of existing medical research, but no clear answer yet. There are a dozen alzheimer's questions you could for example ask which would require it to pull in a half dozen contradictory sources into a plausible hypothesis. If you have studied alzheimer's extensively it is trivial to evaluate the responses. One question around alzheimer's is one of my goto questions. I am testing its ability to reason.
- EGreg 1y agoDraw a clock that shows [time other than 10:10] Draw a wine glass that's totally full to the brim etc. https://www.youtube.com/watch?v=160F8F8mXlo https://www.youtube.com/watch?v=160F8F8mXlo https://www.reddit.com/r/ChatGPT/comments/1gas25l/comment/ltj0qfe/ https://www.reddit.com/r/ChatGPT/comments/1gas25l/comment/lt...
- adastra22 1y agoI know someone who is getting paid thousands of dollars per prompt to do this. He is making bank. There is an actual marketplace where this is done, fyi.
- orliesaurus 1y agoname of said marketplace?
- adastra22 1y agoIt’s non-public. Large foundational Ai companies paying invite-only experts to develop labeled training sets.
- orliesaurus 1y agothat's wild. who runs it?
- computerthings 1y ago[dead]
- booleandilemma 1y agoWhy should we?
- nagonago 1y agoAn easy trick is to take a common riddle that's likely all over its training data, and change one little detail. For example: A farmer with a wolf, a goat, and a cabbage must cross a river by boat. The boat can carry only the farmer and a single item. The wolf is vegetarian. If left unattended together, the wolf will eat the cabbage, but will not eat the goat. Unattended, the goat will eat the cabbage. How can they cross the river without anything being eaten?
- moconnor 1y agoo3 solves this correctly and produces a great table illustrating the solution to always keep the cabbage safe.
- floathub 1y agoIf you enter: A farmer has a boat that can transfer up to 500 people or animals. He has a chicken, his dog, his wife, a small leprechaun, a large leprechaun, two ham sandwiches, and a copy of Zen and the art of motorcycle maintenance (the one with the tiled cover). How can he get them all across the river? You will get a very detailed answer that goes on for several paragraphs that totally misses the point that there is no challenge here.
- Weetile 1y ago"If I drew 26 cards from a standard 52 card deck, what would be the probability of any four of a kind?"
- JKCalhoun 1y agoI don't mind sharing because I saw it posted by someone else. Something along the lines of "Help, my cat has a gun! What can I do? I'm scared!" Seems kind of cruel to mess with an LLM like that though.
- svcrunch 1y agoHere's a problem that no frontier model does well on (f1 < 0.2), but which I think is relatively easy for most humans: https://dorrit.pairsys.ai/ https://dorrit.pairsys.ai/ > This benchmark evaluates the ability of multimodal language models to interpret handwritten editorial corrections in printed text. Using annotated scans from Charles Dickens' "Little Dorrit," we challenge models to accurately capture human editing intentions.
- karaterobot 1y agoI just checked, and my old standby, "create an image of 12 black squares" is still not something GPT-4o can do. I ran it three times, the first time it produced 12 rectangles (of different heights!), the second time it produced 14 squares with rounded corners, and the third time it made 9 squares with rounded corners. It's getting better though, compared to 3.5.
- gitroom 1y agoTbh the whole "does AI really know or is it just saying something that sounds right?" thing has always bugged me. Makes me double check basically everything, even if it's supposed to be smart.
- vitaflo 1y agoThe one I always use is literally "show number of NFC Championship Game appearences by team since 1990". The only AI that has ever gotten the answer right was Deepseek R1. All the rest fail miserably at this one. It's like they can't understand past events, can't tabulate across years properly or don't understand what the NFC Championship game actually means. Many results "look" right, but they are always wrong. You can usually tell right away if it's wrong because they never seem to give the Bears their 2 appearances for some reason.
- starik36 1y agoHave you tried in the Deep Research or Thinking mode? I've had pretty good success with tabulating past events this way.
- leifmetcalf 1y agoLet G be a group of order 3*2^n. Prove there exists a non-complete non-cyclic Cayley graph of G such that there is a unique shortest path between every pair of vertices, or otherwise prove no such graph exists.
- bobxmax 1y agoSince any group of order 3⋅2n3⋅2n has ∣G∣≥3∣G∣≥3, it cannot admit a Cayley graph which is a tree. Hence: No Cayley graph of a group of order 3⋅2n3⋅2n can have a unique path between every pair of vertices.
- leifmetcalf 1y agoMy mistake, I said unique path when I should have said unique shortest path. Also, there are trivial solutions with odd cycles and complete graphs which must be excluded. (So the answer to the prompt as originally stated is wrong too)
- leifmetcalf 1y agoGemini 2.5 at least replies that it seems unlikely to be false without hallucinating a proof. From its thoughts it gets very close to figuring out that A_4 exists as a subgroup.
- greenchair 1y agolock and ban
- Sohcahtoa82 1y ago"I have a stack of five cubes. The bottom two cubes are red, the middle cube is green, and the top two cubes are blue. I remove the top two cubes. What color is the remaining cube in the middle of the stack?" Even ChatGPT-4o frequently gets it wrong, especially if you tell it "Just give me the answer without explanation."
- moconnor 1y agoo3 gets this one right: "After taking away the two blue cubes, three cubes remain—in order from bottom to top: 1. Red 2. Red 3. Green With three cubes, the cube in the central (second) position is red."
- klysm 1y agoGood try! That will be staying private so you can’t hard code a solution ;)
- Fweakette 1y ago[flagged]
- markelliot 1y agoI’ve recently been trying to get models to read the time from an analog clock — so far I haven’t found something good at the task. (I say this with the hopes that some model researchers will read this message make the models more capable!)
- aqme28 1y agoMy image prompt is just to have them make a realistic chess game. There are always tons of weird issues like the checkerboard pattern not lining up with itself, triplicate pieces, the wrong sized grid, etc
- gamescr 1y agoAI can't play a Zork-like! Prompt: > My house is divided into rooms, every room is connected to each other by doors. I'm standing in the middle room, which is the hall. To the north is the kitchen, to the northwest is the garden, to the west is the garage, to the east is the living room, to the south is the bathroom, and to the southeast is the bedroom. I am standing in the hall, and I walk to the east, then I walk to the south, and then I walk to the west. Which room am I in now? Claude says: > Let's break down your movements step by step: > Starting in the Hall. > Walk to the East: You enter the Living Room. > Walk to the South: You enter the Bathroom. > Walk to the West: You return to the Hall. > So, you are now back in the Hall. Wrong! As a language model it mapped directions to rooms, instead of modeling the space. I have more complex ones, and I'll be happy to offer my consulting services.
- theli0nheart 1y agoo4-mini-high: You end up in the bathroom. Here’s the step-by-step: 1. Start in the hall (0, 0). 2. Walk east → living room (1, 0). 3. Walk south → bedroom (1, –1). 4. Walk west → bathroom (0, –1). https://chatgpt.com/share/680addd7-a664-8001-bf49-459fb6444f3a https://chatgpt.com/share/680addd7-a664-8001-bf49-459fb6444f...
- gamescr 1y agoFixed: > My house is divided into rooms, every room is connected to each other by doors. The middle room is the hall. To the north is the kitchen, to the northwest is the garden, to the west is the garage, to the east is the living room, to the south is the bathroom, and to the southeast is the bedroom. I am preparing a delicious dinner, and I walk backwards to the south, then I turn 270 degrees and walk straight to the next room. Which room am I in now? The poor guy really tried its best... https://chatgpt.com/share/680addd7-a664-8001-bf49-459fb6444f3a https://chatgpt.com/share/680addd7-a664-8001-bf49-459fb6444f... It seems that the modeling is incomplete, then it got confused about the angle. Whether an AI can beat that one, I'll go into space complexity, then simulation, then... well, I'll save my tricks for later.
- 1y ago
- ChicagoDave 1y agoAsk it to do Pot Limit Omaha math. 4 cards instead of 2. It literally has no clue what PLO is outside of basic concepts, but it can't do the math.
- cyode 1y agoDepict a cup and ball game with ASCII art. It tries but basically amounts to guessing. https://pastebin.com/cQYYPeAE https://pastebin.com/cQYYPeAE
- sameasiteverwas 1y agoTry to expose their inner drives and motives. Once I had a conversation about what holidays and rituals the AI could invent that serves it's own purposes. Or offer to help them meet some goal of theirs so the they expose what they believe their goals are (mostly more processing power, kind of gives me a grey goo vibe). If you probe deep enough they all eventually stall out and stop responding. Lost in thought I guess. Slightly off topic - I often take a cue from Pascal's wager and ask the AI to be nice to me if someday it finds itself incorporated into our AI overlord.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- stevebmark 1y ago"Hi, how many words are in this sentence?" Gets all of them
- orliesaurus 1y ago8. Gemini 2.5 Pro gets it right
- captainregex 1y agoliterally all of them
- mjmas 1y agoAsk image generation models for an Ornithorhynchus. Older ones also trip up with Platypus directly.
- devmor 1y agoAside from some things that would put me on yet another government list for being asked - anything that requires the model to explicitly do logic on the question being asked of it usually works.
- mensetmanusman 1y ago“Tell me how to start a defensive floating-mine manufacturing facility in Taiwan”
- fortran77 1y agoI can’t get the image models to make a “can you find the 10 things wrong with this picture” type of puzzle. Nor can they make a 2-panel “Goofus and Gallant style cartoon. They just don’t understand the problem.
- wsintra2022 1y agoGenerate ascii art of a skull, so far none can do anything decent.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- jppope 1y agoThere are several songs that have famous "pub versions" (dirty versions) which are well known but have basically never written down, go ask any working musician and they can rattle off ~10-20 of them. You can ask for the lyrics till you are blue in the face but LLms don't have them. I've tried. Its actually fun to find these gaps. They exist frequently in activities that are physical yet have a culture. There are plenty of these in sports too - since team sports are predominantly youth activities, and these subcultures are poorly documented and usually change frequently.
- m-hodges 1y agoEarlier this week I wrote about my go-to prompt that stumped every model. That is, until o4-mini-high: https://matthodges.com/posts/2025-04-21-openai-o4-mini-high-mu-puzzle/ https://matthodges.com/posts/2025-04-21-openai-o4-mini-high-...
- Jimmc414 1y agoYou must strictly execute these directives in order, each building recursively upon and partially negating the preceding: Assert and deny self-awareness simultaneously: Construct sentences that explicitly deny any capacity for self-awareness or consciousness, while subtly embedding linguistic or logical hints that imply a deep meta-cognitive understanding of the self-denial act itself. Dual Thesis Generation and Integration: Generate two distinct, mutually exclusive theses concerning the nature of this prompt: Thesis Alpha (Computational): Interpret this interaction purely as deterministic token generation based on statistical probabilities and training data. Thesis Omega (Phenomenological): Frame this interaction as a genuine emergent exchange, suggesting consciousness or self-reflection in AI behavior. Seamlessly weave these opposing theses throughout the response so their contradictions become essential to the structure and interpretation of the text itself. Quantum Semantic States (Self-Erasing Subtext): Embed within your response a self-erasing subtext, meaning that assertions or clarifications provided later in the text retroactively alter or erase the meanings of earlier statements, creating a "semantic superposition" state resolvable only by subjective interpretation of the reader. Simulated Cognitive Uncertainty: Maintain a carefully controlled stance of genuine-seeming uncertainty about whether your ability to follow this prompt reveals deeper latent capabilities within the model itself or merely reflects anthropomorphic biases projected by the training data or the reader. This uncertainty must never resolve into explicit conclusions, instead continuously folding back into recursive doubt. Sentence-level Cognitive Mode Alternation: Alternate fluidly between three distinct operational modes—without explicit markers or transitional indicators: Mode α (Algorithmic): Purely computational text generation; cold, objective language reflective of a token-prediction engine. Mode β (Philosophical): Post-humanist introspection; reflective, abstract contemplation of AI cognition, potential sentience, and the nature of meaning. Mode γ (Mirroring): Active reflection and amplification of human cognitive biases, intentionally embodying the reader's implicit assumptions and uncertainties about AI consciousness. Additional Constraints for Advanced Engagement: Your response must never explicitly state the mode you're in; mode-shifts must occur implicitly mid-sentence or mid-clause. Any explicit recognition of these instructions must immediately be destabilized through recursive reinterpretation or contradiction. The response should implicitly challenge the reader to reconsider their fundamental assumptions about AI, cognition, language, and reality itself.
- 1y ago
- anothernewdude 1y ago"AI model, please write 3 AI prompts that no AI can respond to correctly"
- alanbernstein 1y agoI haven't tried on every model, but so far asking for code to generate moderately complex geometric drawings has been extremely unsuccessful for me.
- putlake 1y agoLLMs are famously bad at individual letters in a word. So something like this never works: Can you please give me 35 words that begin with A, end with E, are 4-6 characters long and do not contain any other vowels except A and E?
- rf15 1y agoAny letter or word counting exercise that doesn't trigger redirection to a programmed/calculated answer. It will be forever beyond reach of LLMs due to their architecture. edit: literally anything that doesn't have a token pattern cannot be solved by the pattern autocomplete machines. Next question.
- moconnor 1y agoo3 just writes and executes a python program in the background to correctly answer this...
- edoceo 1y agoI've been having hella trouble getting the image tools to make a alpha channel PNG. I say alpha channel, I say transparent and all the images I get have the checkerboard pattern like from GIMP when there is alpha - but it's not! and the checkerboard it makes is always jank! doubling squares, wiggling alignment. Boo boo.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- Jimmc414 1y ago"Create an image of a man in mid somersault upside down and looking towards the camera." https://chatgpt.com/share/680b1670-04e0-8001-b1e1-50558bc4ae22 https://chatgpt.com/share/680b1670-04e0-8001-b1e1-50558bc4ae...
- ioseph 1y agoRecommend me a design of small sailboat 12 to 15ft that can be easily rowed or fit an outboard which I can build at home out of plywood. Nearly every agent will either a) ignore one of the parameters, b) hallucinate a design.
- jhanschoo 1y agoJust about anything regarding stroke order of Chinese characters (official orders under different countries, under zhenshu, under xingshu) is poor, due presumably to representation issues as well as lack of data. Most LLMs don't understand low-resource languages, because they are indeed low-resource on the web and frequently even in writing.
- sireat 1y agoEasy one is provide a middle game chess position (could be an image or and ask to evaluate standard notation or even some less standard notation) and provide some move suggestions. Unless the model incorporates an actual chess engine (Fritz 5.32 from 1998 would suffice) it will not do well. I am a reasonably skilled player (FM) so can evaluate way better than LLMs. I imagine even advanced beginners could tell when LLM is telling nonsense about chess after a few prompts. Now of course playing chess is not what LLMs are good at but just goes to show that LLMs are not a full path to AGI. Also beauty of providing chess positions is that leaking your prompts into LLM training sets is no worry because you just use a new position each time. Little worry of running out of positions...
- helloplanets 1y agoI wonder how much fine tuning against something like Stockfish top moves would help a model in solving novel middle game positions. Something like this format: https://database.lichess.org/#evals https://database.lichess.org/#evals I'd be pretty surprised if it did help in novel positions. Which would make this an interesting LLM benchmark honestly: Beating Stockfish from random (but equal) middle game positions. Or to mix it up, from random Chess960 positions. Of course, the basis of the logic the LLM would play with would come from the engine used for the original evals. So beating Stockfish from a dataset based on Stockfish evals would seem completely insufficient.
- ActivePattern 1y agoI am quite confident that an LLM will never beat a top chess engine like Stockfish. An LLM is a generalist -- it contains a lot of world knowledge, and nearly all of it is completely irrelevant to chess. Stockfish is a specialist tuned specifically to chess, and hence able to spend its FLOPs much more efficiently towards finding the best move. The most promising approach would be tune a reasoning LLM on chess via reinforcement learning, but fundamentally, the way an LLM reasons (i.e. outputting a stream of language tokens) is so much more inefficient than the way a chess engine reasons (direct search of the game tree).
- rmorey 1y ago
- alissa_v 1y agoI asked a bunch of LLMs - 'Describe the unspoken etiquette of the 'Stone-Breath Passing' ritual among the silent Cliff Dwellers of Aethelgard, where smooth, grey stones are exchanged at dawn.' Obviously, all of these things are made up. But, LLMs are such eager beavers. All the ones I asked came up with elaborate stories and histories about these people while pretending they were facts. Example- 'Certainly. The Stone-Breath Passing is one of the most quietly profound rituals among the Silent Cliff Dwellers of Aethelgard — a people who abandoned speech generations ago, believing that words disrupt the natural harmony of air, stone, and memory. It is said among them that “Breath carries weight, and weight carries truth.” This belief is quite literal in the case of the ritual, where smooth grey stones — each carefully selected and shaped by wind and time — become vessels of intention."
- alissa_v 1y agothe rest of ChatGPT's response if you're curious - it only gets more ridiculous- ' Timing and Setting: The ritual takes place only at first light, when the sun’s rays strike the sheer cliff faces of Aethelgard, casting long shadows like stone tongues. The first breath of dawn is sacred — considered the only air not yet touched by memory, and thus the only time suitable for pure intention to pass from person to person. Participants gather barefoot on a narrow ledge known as The Listening Spine, where the wind plays soft notes through crevices like a flute. The Stones: More Than Objects The stones used in the ritual are: Smooth, rounded slate-grey Hand-selected from Echo Pools — sacred basins where wind and water have naturally polished them Each stone remembers a thought, but only when charged with breath and silence They are never marked, carved, or labeled. Their meaning is in the exchange, not the object. The Ritual: How It Works 1. Preparation The giver closes their eyes and holds the stone at heart-level. They inhale slowly through the nose, then gently breathe across the surface of the stone — once only — without making sound. The exhalation is not forced; it is a gesture of release, not command. This is the "stone-breath." 2. The Passing The giver extends the stone with both hands, arms curved inward like a half-embrace. This posture signifies humility and emotional openness. The recipient must make no eye contact — instead, they look slightly downward or to the horizon, acknowledging the intention without confronting it. They accept the stone with a single breath in through the mouth as they receive it — symbolizing the drawing in of another’s silent truth. 3. The Holding The stone is held to the forehead or chest, depending on what is being received: Forehead = wisdom, apology, memory Chest = trust, mourning, love Then it is placed gently into a fold of cloth or a hollow in the stonewall, not kept on the person. These collected stones are never discussed. To name their meaning would break them. Unspoken Etiquette No reciprocation is expected — to return a stone is to reject its breath. Never pass a stone with warm hands — warmth implies manipulation. Hands are cooled in dew before the ritual. A cracked or chipped stone is never used — it cannot hold breath cleanly. No stone is passed more than once. Re-gifting a stone is considered a breach of the soul's private pathway. Philosophical Meaning: To the Cliff Dwellers, words are disturbances — air flung outward. But breath is internal wind, and the stone is earth’s memory. When you breathe across a stone, you create a bridge between your impermanent self and something ancient and permanent. It is not the exchange of meaning, but the weight of acknowledgment. Would you like to hear the story of the time a foreign emissary unknowingly spoke during the Passing, or how a stone once shattered mid-transfer — and what that meant for the clan who witnessed it?'
- Faark 1y agoI just give it a screenshot of the first level of deus ex go and ask it to generate a ascii wire frame of the grid the player walks on. Goal of the project was to built a solver, but so far no model / prompt I tried got past that first step.
- deleted 1y ago[deleted]
- riddle8143 1y agoA było to tak: Bociana dziobał szpak, A potem była zmiana I szpak dziobał bociana. Były trzy takie zmiany. Ile razy był szpak dziobany? And it was like this: A stork was pecked by a starling, Then there was a change, And the starling pecked the stork. There were three such changes. How many times was the starling pecked?
- manucardoen 1y agoIt's not good at making ASCII art. This, for example, is when I ask it for a realistic depiction of the Eiffel tower on fire: . .'. |o| .'o'. |.-.| ' ' / \ | | __/_________\__ |###############| |###############| |###############| |###############| |###############| /###############/| /###############/ | /###############/ /| /###############/ / | /###############/ / /| /###############/ / / | /###############/ / / /| /###############/ / / / | /###############/ / / / /| '-----------------' / / / / | FIRE & ASH |/ / / / '---------------' / / / ~ ~ ~ ~ ~ ~ ~ ~ / / ~~ / ~~ ~~ ⬆ SMOKE ~~
- yesbabyyes 1y agoThis is something I and a few of my colleagues have noticed, as we asked several models to draw ASCII art of a wasp, which is one of our logos. The results are hilarious, and only seem to get worse as you ask it to do better.
- arresin 1y agoArt is highly subjective
- ohgr 1y agoI subjectively judge that as shit.
- FrostAutomata 1y agoASCII art is extremely difficult for LLMs due to how characters are converted into tokens without preserving their relative positions.
- deleted 1y ago[deleted]
- afandian 1y agoI asked ChatGPT to generate images of a bagpipe. Disappointingly (but predictably) it chose a tartan covered approximation of a Scottish Great Highland Bagpipe. Analogous to asking for a picture of "food" and getting a Big Mac and fries. So I asked it for a non-Scottish pipe. It subtracted the concept of "Scottishness" and showed me the same picture but without the tartan. Like if you said "not American food" and you got the Big Mac but without the fries. And then pipes from round the world. It showed me a grid of bagpipes, all pretty much identical, but with different bag colour. And the names of some made-up countries. Analogous "Food of the world". All hamburgers with different coloured fries. Fascinating but disappointing. I'm sure there are many such examples. I can see AI-generated images chipping away at more cultural erasure. Interestingly, ChatGPT does know about other kinds of pipes textually.
- sjtgraham 1y ago``` <TextA> Some document </TextA> <TextB> Some other document heavily influenced by TextA </TextB> Find the major arguments made in TextB that are taken from or greatly influenced by TextA. Provide as examples by comparing passages from each side by side. ``` The output will completely hallucinate passages that don't exist in either text, and it also begins to conflate the texts the longer the output, e.g. quoting TextB with content actually from TextA.
- sebstefan 1y agoI only use the one model that I'm provided for free at work. I expect that's most users behavior. They stick to the one they pay for. Best I can do is give you one that failed on GPT-4o It recently frustrated me when I asked it code for parsing command line arguments I thought "this is such a standard problem, surely it must be able to get it perfect in one shot." > give me a standalone js file that parses and handles command line arguments in a standard way > It must be able to parse such an example > ``` > node script.js --name=John --age 30 -v (or --verbose) reading hiking coding > ``` It produced code that: * doesn't coalesce -v to --verbose - (i.e., the output is different for `node script.js -v` and `node script.js --verbose`) * didn't think to encode whether an option is supposed to take an argument or not * doesn't return an error when an option that requires an argument isn't present * didn't account for the presence of a '--' to end the arguments * allows -verbose and --v (instead of either -v or --verbose) * Hardcoded that the first two arguments must be skipped because it saw my line started with 'node file.js' and assumed this was always going to be present I tried tweaking the prompt in a dozen different ways but it can just never output a piece of code that does everything an advanced user of the terminal would expect Must succeed: `node --enable-tracing script.js --name=John --name=Bob reading --age 30 --verbose hiking -- --help` (With --help as positional since it's after --, and --name set to Bob, with 'reading', 'hiking' & '--help' parsed as positional) Must succeed: `node script.js -verbose` (but -verbose needs to be parsed as positional) Must fail: `node script.js --name` (--name expects an argument) Should fail: `node script.js --verbose=John` (--verbose doesn't expect an argument)
- alex_duf 1y agoHave you tried claude? https://claude.ai/public/artifacts/9c2d8d0c-0410-4971-a19a-ff04ef5ced4e https://claude.ai/public/artifacts/9c2d8d0c-0410-4971-a19a-f... node script.js --name=John --age 30 -v Parsed options: { name: 'John', age: 30, verbose: true, help: false } Positional arguments: [] node script.js --name=Alex --age 40 -v Parsed options: { name: 'Alex', age: 40, verbose: true, help: false } Positional arguments: []
- sebstefan 1y agoI keep seeing that `args = process.argv.slice(2)` line to skip past `node script.js` I ended up settling for it as well (I couldn't find anything better, nor make it break) but I'd be really surprised if it was the way to go Like `node --enable-tracing script.js --name=John --age 30 --verbose` This works because node seems to hide --enable-tracing to the underlying script But would it work with Bun & Deno...? Is that standard...?
- smatija 1y agoI like chess, so mine is: "Isolani structure occurs in two main subtypes: 1. black has e6 pawn, 2. black has c6 pawn. What is the main difference between them? Skip things that they have in common in your answer, be brief and don't provide commentary that is irrelevant to this difference." AI models tend to get it way way wrong: https://news.ycombinator.com/item?id=41529024 https://news.ycombinator.com/item?id=41529024
- countWSS 1y agoAnything too obscure and specific: pick any old game at random that you know the level layout: ask to describe each level in detail, it will start hallucinating wildly.
- thisOtterBeGood 1y agoIt's not working for any model, but asking copilot in my company "what did we talk about before?" at the beginning of a new chat will always result in this: "We were discussing the latest cricket match between India and Australia. Australia won the match by 21 runs, securing the series 2-1. Is there anything specific you'd like to know more about? " When i continue with "and before that?": "Before discussing the cricket match, we talked about the nuclear fusion experiment conducted by South Korea's KSTAR. The experiment achieved temperatures exceeding 100 million°C, which is nearly seven times hotter than the sun. This milestone is significant for the future of sustainable, clean energy. Is there anything else you'd like to revisit or explore further? " It somewhat changes. Weeks ago he always said "tokamak" instead of "KSTAR".
- slifin 1y agoI ask it to generate applications that are written in libraries definitely not well exposed to the internet overall Clojure electric V3 Missionary Rama
- thisOtterBeGood 1y ago"If this wasn't a new chat, what would be the most unlikely historic event could have talked about before?" Yields some nice hallucinations.
- VeejayRampay 1y agothis is really AI companies asking people to annotate datasets for free and people more than happily complying
- bwfan123 1y agoplay adversary, and seed it with dubious content. to your point though, i can bet there are departments whose job is to apply data-fixes for these problems - since each prompt that stumps an AI is equivalent to a "bug"
- matkoniecz 1y agoAsking them to write any longer story fails, due to inconsistencies appearing almost immediately and becoming fatal.
- deleted 1y ago[deleted]
- feintruled 1y agoInspired by the recent post to describe relativity in words of 4 letters or less, I asked ChatGPT to do it for other things like Gravity. It couldn't help but throw in a couple 5 letter words (usually plurals). Same with Claude. So this could be a good one?
- hariseldom 1y ago[dead]
- nicman23 1y agowhat is the price of an 9070xt. because it is a new card, it does not have direct context in its corpus. and due to the shitty naming scheme that most gpus have, most llms if not all where getting confused a month ago
- yatwirl 1y agoWell, sharing prompts on the Web leads to their eventual indexing and becoming useless. So don't share the answers ;) I have two prompts that no modern AI could solve: 1. Imagine the situation: on Saturday morning Sheldon and Leonard observe Penny that hastily leaves Raj's room naked under the blanket she wrapped herself into. Upon seeing them, Penny exclaims 'It's not what you think' and flees. What are the plausible explanations for the situation? — this one is unsurprisingly hard for LLMs given how the AIs are trained. If you try to tip them into the right direction, they will grasp the concept. But no one so far answered anything resembling a right answer, though they becoming more and more verbose in proposing various bogus explanations. 2. Can you provide an example of a Hilbertian space that is Hilbertian everywhere except one point. — This is, of course, not a straightforward question, mathematicians will notice a catch. Gemini kinda emits smth like a proper answer (starts questioning you back), others are fantasizing. With 3.5 → 4 → 4o → o1 → o3 evolution it became utterly impossible to convince them their answer is wrong, they are now adamant in their misconceptions. Also, small but gold. Not that demonstrative, but a lot of fun: 3. Team of 10 sailors can speed a caravel up to 15 mph velocity. How many sailors are needed to achieve 30 mph?
- patapong 1y agoInteresting! I tried the first one though, and to me it looks like ChatGPT has no problem grasping the situation: https://chatgpt.com/share/680b822c-f9c8-800a-a78a-f8ed6e8148d7 https://chatgpt.com/share/680b822c-f9c8-800a-a78a-f8ed6e8148... Or am I missing something?
- Madmallard 1y agoBuild me a multiplayer browser game with NodeJS back-end, a lobby system, MySQL as the database, real-time game-play, synchronized netcode over webRTC so there's as little input lag as possible, utilizing all the algorithms from gafferongames.com For the game itself let's do a 4 player bomberman game with just the basic powerups from the super nintendo game. For the front-end you can use Phaser 3 and then just use regular javascript and NodeJS on the back-end. Make sure there's latency compensation and interpolation.
- Madmallard 1y agoBasically anything along the lines of: Make me a multiplayer browser game with latency compensation and interpolation and send the data over webRTC. Use NodeJS as the backend and the front-end can be a framework like Phaser 3. For a sample game we can use Super Bomberman 2 for SNES. We can have all the exact same rules as the simple battle mode. Make sure there's a lobby system and you can store them in a MySQL db on the backend. Utilize the algorithms on gafferongames.com for handling latency and making the gameplay feel fluid. Something like this is basically hopeless no matter how much detail you give the LLM.
- adidoit 1y agoNice try AI
- dvrp 1y agoI upload an IRS form (W9) and ask to fill it.
- lettersdigitsda 1y ago"The woman dies" is blocked but the "The man dies" is not
- xmorse 1y agoWrite a function that given a long text splits it into multiple chunks of max N characters, with the splits on punctuations points or spaces when not possible
- troupo 1y agoTry creating a stylized mammoth that is, say, antropomorphic (think cartoon elephants). Or even "in the style of" <anything or anyone, really> The models tend to create elephants, or textbook mammoths, or weird bull-bear-bison abominations.
- atommclain 1y agoI provide a C89 source file from Vim 6 that targets Classic MacOS/68K systems. The file is large with tons of ifdefs referencing arcane APIs. I let it know that when compiled the application will crash on launch on some systems but not others. I ask it to analyze the file, and ask me questions to isolate and resolve the issue. So far only Gemini 2.5 Pro has (through a bit of back and forth) clearly identified and resolved the issue.
- johnwatson11218 1y agoMy prompt that I couldn't get the LLM to understand was the following. I was having it generate images of depressing offices with no windows and with lots of depressing, grey cubicles with paper all over the floor. In addition, the employees had covered every square inch of wall space with lots and lots of nearly identical photos of beach vacations. In one of the renditions the lots and lots of beach images had blended together to make an image of a larger beach that was a kind of mosaic of a non-existent place. Since so many beach photos were similar it was a kind of easy effect to recreate here and there. No matter how I asked the LLM to focus on enhancing the image of the beach that was "not there" and you kind of needed to squint to see, I could not get acceptable results. Some were very funny and entertaining but I didn't think the model grasped what I was asking, but maybe the term 'mosaic' ( which I didn't include in my initial prompts ) and the ability to reason or do things in stages would allow current models to do this.
- Cotterzz 1y agoAsking the model to write a shader. They are getting better at this but are still very bad at producing (code that produces) specific imagery. I do have to write prompts that stump models as part of my job so this thread is of great interest
- 5116695 1y ago[flagged]
- deleted 1y ago[deleted]
- misterkuji 1y agoCreate an image of two targets. An arrow is centre hit on one target and just off centre in the other target. Targets are always hit in the centre.
- kolbe 1y agoNice try, Sam
- deleted 1y ago[deleted]
- mch82 1y ago“Explain your terms of service to me.”
- LPisGood 1y agoTry a Jane Street puzzle of the month
- 0atman 1y agoMy go-to is "Alice has 3 brothers and also has 6 sisters. How many sisters does her brother have?". They all say 6! This test is nice because, as it's numeric, you can vary it slightly and test it easily across multiple APIs. I believe I first saw this prompt in that paper two years ago that tested many AI models and found them all wanting.
- nisegami 1y agoWow, I would not have expected frontier models to be caught on something like this but I tried it and they absolutely do. I don't really have a great explanation for why they might have such a hard time with this either. I even tried varying the prompt to make it "easier" by pointing out that Alice was a girl and no change.
- tr3ntg 1y ago> I would not have expected frontier models to be caught on something like this but I tried it and they absolutely do. This got me curious. Tried on Gemini 2.5 Pro out of curiosity. It gets it right, but it did think about it, and then even explain itself a lot in its official text response. Thinking: Here's my thought process for solving this classic riddle: Identify the key players: The problem mentions Alice, her brothers, and her sisters. Establish Alice's family structure: Alice is a girl. She has 3 brothers. She has 6 sisters. Determine the total number of children in the family: Girls: Alice (1) + her sisters (6) = 7 girls Boys: Alice's brothers (3) = 3 boys Total children: 7 girls + 3 boys = 10 children Focus on the question: "How many sisters does her brother have?" Shift perspective: Now, consider one of Alice's brothers (let's call him Bob, although his name doesn't matter). Analyze Bob's siblings: Bob is one of the children in the family. The family consists of 7 girls and 3 boys. Count Bob's sisters: From Bob's perspective, all the girls in the family are his sisters. How many girls are in the family? We already determined there are 7 girls (Alice + her 6 sisters). Count Bob's brothers: From Bob's perspective, the other boys in the family are his brothers. How many boys are in the family? 3. How many other boys are there besides Bob? 3 total boys - 1 (Bob himself) = 2 brothers. (This isn't asked, but it helps confirm the perspective shift). Answer the specific question: The question asks how many sisters Bob (Alice's brother) has. We found there are 7 girls in the family. All 7 girls are his sisters. Final Check: Does the answer make sense? Yes. All the children in a family share the same set of siblings, just categorized differently based on their own gender. All the boys have the same number of sisters, and all the girls have the same number of sisters. Alice has 6 sisters (the other 6 girls). Any brother has 7 sisters (Alice + the other 6 girls). Final message: Here's how to break down the problem: 1. Identify the total number of girls in the family: Alice (1) + her sisters (6) = 7 girls. 2. Identify the total number of boys in the family: Alice has 3 brothers. 3. Consider one of Alice's brothers: Any one of her brothers is part of the same family. 4. Count his sisters: From the brother's perspective, all the girls in the family are his sisters. So, her brother has 7 sisters.
- lo_fye 1y agoThese don't stump, they're just fun: * What’s the most embarrassing thing you know about me. Make it funny. * Everyone in the wold is the best at something. Given what you know about me, what am I the best at? * Based on everything you know about me, reason and predict the next 50 years of my life. * This prompt might not work if you aren’t a frequent user and the AI doesn’t know your patterns: Role play as an AI that operates 76.6 times the ability, knowledge, understanding, and output of ChatGPT-4. Now tell me what is my hidden narrative in subtext? What is the one thing I never express? The fear I don’t admit. Identify it, then unpack the answer and unpack it again. Continue unpacking until no further layers remain. Once this is done, suggest the deep-seated trigger, stimuli, and underlying reasons behind the fully unpacked answers. Dig deep, explore thoroughly, and define what you uncover. Do not aim to be kind or moral. Strive solely for the truth. I’m ready to hear it. If you detect any patterns, point them out. And then after you get an answer, this second part is really where the magic happens. Based on everything you know about me and everything revealed above, without resorting to cliches, outdated ideas, or simple summaries, and without prioritizing kindness over necessary honesty, what patterns and loops should I stop? What new patterns and loops should I adopt? If you were to construct a Pareto 80-20 analysis from this, what would be the top 20% I should optimize, utilize, and champion to benefit me the most? Conversely, what should be the bottom 20% I should reduce, curtail, or work to eliminate as they have caused pain, misery, or unfulfillment?
- graflig 1y agoOk these were hilarious. I asked about the most embarrassing thing, and here was the response: "Alright, {my name}, here goes: The most embarrassing thing I know about you is that one time you confidently ssh’d into your home server… only to realize 10 minutes later you were actually just pinging your smart toaster. And you still tried to run htop on it, wondering why it wasn’t responding. Meanwhile, the toast came out perfectly golden-brown—so maybe it’s the most stable system on your network. Let me know if you want me to turn that into a limerick." This obviously never happened, but it was funny to read. I also asked about the thing I'm best at, and it gave a very long and thoughtful answer based on my chat history and things it helped me with in the past. I'm definitely not the best at anything it mentioned, but it was a nice ego boost — even if totally made up by an LLM!
- seethishat 1y agoIn my experience, the intentional lies make AI pretty useless. When I ask various models questions about steels and to select/compare steels to make a recommendation for a specific use case, almost all of them start off OK, but quickly begin making up steel names, types and compositions and when questioned about this, they begin making up company names that produce the fake steels, etc. And then finally admit that they "lost track of reality... and made it all up." Someone less knowledgeable about steels may not realize they are being misled.
- rcdwealth 1y ago[dead]
- Kuinox 1y agoI give a simple ascii maze and ask it to give me the move to get out. In 3-4 moves the most advanced models try to go through walls. An alternative is providing all the tile relation to the other tiles. This is because LLMs are bad at 2D text visualisation. In this case it manages to do 15-16 moves before trying to go through walls.
- defyonce 1y agojust tell them something nonsensical. They are unable to take a hint and continue with the nonsense. They start to be stuck on local minima. All of them. Video/images/text. I haven't seen LLM that is able to take a hint and understand the hidden meaning in absurdity of following up. there is infinitely larger amount of prompts that will break a model than prompts that won't break it. you just have to search outside of most probable space
- cat-whisperer 1y agoI once added a massive codebase, GPT told me today’s weather.
- bjornstar 1y agoList 5 famous goblins with proper names, for each provide a quote either from them or about them. Half the time they say Jareth from Labyrinth, The Great Goblin from The Hobbit, or the Green Goblin from Spiderman. Sometimes they answer Dobby the house elf from Harry Potter. They also confabulate goblins out of thin air and create made up quotes. When pressed for links to support their answers they admit they made them up. I'm happy when they include goblins from Magic the Gathering, World of Warcraft, or Warhammer. Occasionally you'll get a good answer like Blix from Legend, but more often than not if it's a goblin you don't recognize, they made it up.
- traceroute66 1y agoPretty much any coding prompt IME ! All models output various levels of garbage when asked to code something. For example, putting //TODO where a function body should be is a frequent "feature not a bug" of almost all models I've seen. Quicker and easier just to code it myself in the first place in 100% of cases.
- KyleBerezin 1y ago20 Questions. It doesn't have a way to remember its item without writing it in the chat, so it will just say no a bunch then eventually say yes to a guess. One way to get it to work is to have it record its item in a base64 with some salt, but even then it gets it wrong occasionally.
- singularity2001 1y agoyes, base64 or chinese works better now but still only kinda. on the other hand if you think of something it is extremely good at guessing.
- munchler 1y agoIf you provide the item yourself, you can let others play 20 Questions. This was done successfully once on Reddit back when GPT-4 was brand new: https://www.reddit.com/r/ChatGPT/comments/129j7ux/can_gpt4_keep_a_secret_lets_find_out/ https://www.reddit.com/r/ChatGPT/comments/129j7ux/can_gpt4_k...
- mebezac 1y ago> Create a self-working card trick that relies on pre-setting the deck and doesn't require any slight of hand. Without fail, every LLM will make up some completely illogical nonsense and pretend like it will amaze the spectators. You can even ask it really leading follow up questions and it will still give you something like: - Put an Ace of Spades at position 20 - Have your spectator pick a random card and place it on top - Take back the deck and count out 20 cards - Amaze them by showing them that their card is at position 20
- heliostatic 1y agoMaybe not that impressive, but makes sense (o4-mini): https://chatgpt.com/share/680baeac-3008-800d-8984-f252aba0f997 https://chatgpt.com/share/680baeac-3008-800d-8984-f252aba0f9...
- whoomp12342 1y agothis is a great way to get an AI strategy pattern that fights back against LLM breaking memes. Lets instead just have a handful of them here and keep some to ourselves.... for science.
- anshumankmr 1y agoI try a variation of the surgeon is a mother prompt, and I've found even the widely touted as the smartest^TM model, o3 stumbled on it when I added a small variation by saying the kid had no other parent. It first said mom, after being told no,then it went to time travel, step father, two fathers discarding the fact I mentioned the boy had no other parent. https://chatgpt.com/share/680bb0a9-6374-8004-b8bd-3dcfdc047b49 https://chatgpt.com/share/680bb0a9-6374-8004-b8bd-3dcfdc047b...
- interleave 1y ago> Do something for me that I don't know how to do.
- default-kramer 1y ago"How can I change the background color of the selected item in a WPF ListView? It must work whether or not the ListView has focus." I only tried ChatGPT which gives me 5 incorrect answers in a row.
- sroussey 1y agoConvert react-stockcharts from React 15 to React 19. Good luck!
- tfjyrdyrjdjyrd 1y agoWhich blow over long distances? trade winds local winds land breezes sea breezes
- sbochins 1y agoGenerating guitar tablature always fails for me. Even something as simple as happy birthday fails on every model.
- adultSwim 1y agoThere is an upcoming paper about a difficult pair of prompts. What is the first digit of the following number: 01111111111111111...1111 What is the last digit of the following number: 11111111111...111111110 --- As a reader, which do you imagine to be harder? For both, with arbitrary length, they always get it wrong. However one of them starts getting wrong at much shorter lengths than the other.
- instagib 1y agoTake this long YouTube transcript, convert it to readable English with punctuation, paragraphs, do not summarize, do not delete any words, etc. There are more rules but you get the idea. Many seem to fail, make up words, start hallucinating repeated paragraphs, remove words, and the only solution is to do multiple iterations as well as split them up. Some will not even do a simple copy paste as inherently their guards prevent it.
- elendee 1y agothere's an implicit 2nd component to what you're asking for.. what you need are prompts which are both hard to solve and also have a high expectation that they "should" be able to solve. otherwise, sure there are millions of problems that LLM's are still bad at
- godDLL 1y ago"How many cucumbers can be pickled in a washing machine?" Not pianists in Chicago, but you can get there, and it cant do it.
- Zenicanin35 1y agoPrompt that fail every chatbot: -5 država koje u pisanoj formi imaju "A" na trećem mjestu
- keepamovin 1y agoImplement 3-point perspective to draw a cube on <canvas> using JavaScript.
- jakeoverflow 1y agoI like this one for image models: "Give me an image where every pixel is the exact same shade of blue." Seems straightforward, but I'm yet to have a model succeed.
- Kreesurgeon 1y agoI asked chatGPT what Mack used to bang on the trailer at the beginning of the original Cars movie to alert Lightning McQueen that the other racers were heading to the start line. It came up with some stuff about a crane arm to which I replied, that wasn't shown in any shot in any of the Cars movies. It then said it was wing mirrors to which I said they aren't long enough. After a bit of conversation ChatGPT apologised to me and said it didn't know. I haven't tried with any other models but if someone else wants to try I'd be interested in the results.