5 ms·

Great post. Funny, I just posted this to X 2025 GenAI challenge Create a 5x5 crossword puzzle with two distinct solutions. Each clue must work for both solut

Great post. Funny, I just posted this to X 2025 GenAI challenge Create a 5x5 crossword puzzle with two distinct solutions. Each clue must work for both solutions. Do not use the same word in both solutions. No black squares. I try with each new model that lands. Still can’t get it.

alberto_balsam 2y ago
Do you know if there is a solution to this by humans? I'd be interested in seeing it.
- korymath 2y ago
  I've not found a solution at any NxN size made by human or machine.
  - quuxplusone 2y ago
    You might get a little closer by tweaking the prompt — you're asking the LLM to "figure out" that the first step is to create two 5x5 word squares with no repeated words, and then the second step is to solve ten requests of the form, "Give me a crossword-style clue that could be reasonably solved by either the words OPERA or the word TENET" (for each of the ten word-pairs in your square-pairs). However, LLMs are based on tokens, and thus fundamentally don't "understand" that words are made out of letters — that's why we have memes about their inability to count the number of "r"s in "strawberry" and so on. So we shouldn't expect an LLM to be able to perform Step 1 at all, really. And Step 2 requires wordplay and/or lateral thinking, which LLMs are again bad at. (They can easily do "Give me a crossword-style clue that could be solved by the word OPERA," because there are databases of such things on the web which form part of every LLM's dataset. But there's no such database for double-solution clues.) Generating a 5x5 word square (with different words across and down, so not of the "Sator Arepo" variety) is already really hard for a human. I plugged the Wordle target word list into https://github.com/Quuxplusone/xword/blob/master/src/xword-fill.c https://github.com/Quuxplusone/xword/blob/master/src/xword-f... to get a bunch of plausible squares like this: SCALD POLAR ARTSY CEASE ERROR But you want two word squares that can plausibly be clued together, which is (not impossible, but) difficult if matching entries aren't the same part of speech. For example, cluing "POLAR" together with "ARTSY" (both adjectives) seems likely more doable than cluing "POLAR" together with "LASSO" (noun or verb). Anyway, here's my attempt at a human solution, using the grid above — and another grid, which I'll challenge you to find from these clues. Hint: All but two of the ten pairs match, part-of-speech-wise. 1A. Remove the outer layer of, perhaps 2A. Region on a globe 3A. Like some movie theaters 4A. Command to a lawbreaker 5A. Rhyme for Tom Lehrer? 1D. ____yard (sometime sci-fi setting) 2D. It goes something like this: Ꮎ 3D. Feature of liturgy, often 4D. It's vacuous, in a sense 5D. Fino, vis-a-vis Pedro Ximénez
echelon 2y ago
That's algorithmically hard. Ask the LLM to generate a program to solve the problem.
- korymath 2y ago
  I've tried that, as recently as today with latest Gemini, Claude, and o1 ... none have been successful.
abstractbill 2y ago
Thanks! That's a wonderfully hard problem, I'd love to see it get solved.

alberto_balsam 2y ago

Do you know if there is a solution to this by humans? I'd be interested in seeing it.

korymath 2y ago
I've not found a solution at any NxN size made by human or machine.
- quuxplusone 2y ago
  You might get a little closer by tweaking the prompt — you're asking the LLM to "figure out" that the first step is to create two 5x5 word squares with no repeated words, and then the second step is to solve ten requests of the form, "Give me a crossword-style clue that could be reasonably solved by either the words OPERA or the word TENET" (for each of the ten word-pairs in your square-pairs). However, LLMs are based on tokens, and thus fundamentally don't "understand" that words are made out of letters — that's why we have memes about their inability to count the number of "r"s in "strawberry" and so on. So we shouldn't expect an LLM to be able to perform Step 1 at all, really. And Step 2 requires wordplay and/or lateral thinking, which LLMs are again bad at. (They can easily do "Give me a crossword-style clue that could be solved by the word OPERA," because there are databases of such things on the web which form part of every LLM's dataset. But there's no such database for double-solution clues.) Generating a 5x5 word square (with different words across and down, so not of the "Sator Arepo" variety) is already really hard for a human. I plugged the Wordle target word list into https://github.com/Quuxplusone/xword/blob/master/src/xword-fill.c https://github.com/Quuxplusone/xword/blob/master/src/xword-f... to get a bunch of plausible squares like this: SCALD POLAR ARTSY CEASE ERROR But you want two word squares that can plausibly be clued together, which is (not impossible, but) difficult if matching entries aren't the same part of speech. For example, cluing "POLAR" together with "ARTSY" (both adjectives) seems likely more doable than cluing "POLAR" together with "LASSO" (noun or verb). Anyway, here's my attempt at a human solution, using the grid above — and another grid, which I'll challenge you to find from these clues. Hint: All but two of the ten pairs match, part-of-speech-wise. 1A. Remove the outer layer of, perhaps 2A. Region on a globe 3A. Like some movie theaters 4A. Command to a lawbreaker 5A. Rhyme for Tom Lehrer? 1D. ____yard (sometime sci-fi setting) 2D. It goes something like this: Ꮎ 3D. Feature of liturgy, often 4D. It's vacuous, in a sense 5D. Fino, vis-a-vis Pedro Ximénez

echelon 2y ago

That's algorithmically hard. Ask the LLM to generate a program to solve the problem.

korymath 2y ago
I've tried that, as recently as today with latest Gemini, Claude, and o1 ... none have been successful.

abstractbill 2y ago

Thanks! That's a wonderfully hard problem, I'd love to see it get solved.