9 ms·

Discovering Cryptographic Weaknesses with Claude

232 points by gslin 2mo ago

huflungdung 2mo ago
[dead]
a-dub 2mo ago
> The multi-agent workflow led to interesting dynamics. For example, the key idea in producing this attack was discovered by a pair of workers working together. Both started investigating the idea; the first worker prematurely rejected the idea as infeasible, but the second found a way to fully exploit it. The pair kept exchanging messages, and eventually both agreed they had found an effective attack. this is pretty interesting. the way it is written doesn't make it sound like the collaboration actually led to the discovery, but rather just the stochastic nature of each thread in the search. it would be interesting to replay and repeat the search (possibly with prior/context pertubations) to get a sense for how often it finds or misses the known working path.
- TeMPOraL 2mo ago
  Hypothesis: the pairing / collaboration makes it much more likely to find a fruitful road previously dismissed, because... that's what happens in fiction - including books, movies, and journalism (long-form "people stories"). It's a common trope: if one character dismisses a course of action, the plot demands the other character to take it. In a way LLMs are, after all, trained to LARP people, including fictional characters and their tropes - this was actually exploited for jailbreaking to good effect in the late pre-agentic era (read: some two years ago). C.f. Waluigi effect. Not sure if it still holds for current models, but I can't imagine why it would not.
- imightbebatman 2mo ago
  Yes there are two interesting derivative questions from this, assuming I understand it. First, is it reproducible consistently at ~50% of workers? If not, what is the rate. Second, are there any lessons to be learned here to increase the rate of success by changing models/weights/training? The news by itself isn't really good news. But it could lead to good news. Maybe.
- _zoltan_ 2mo ago
  but in ultracode subagents don't communicate back and forth, do they? so this must have been an other harness than Claude code?
Clayrune 2mo ago
[flagged]
Stevvo 2mo ago
Interesting they are still using "Mythos Preview" instead of "Mythos 5"; I had read from others who had access to both that Mythos 5 is less capable.
- TeMPOraL 2mo ago
  Going to guess it's more available or less overconstrained. See e.g. Fable, which is much better than Opus 4.8 and possibly than Opus 5... in the rare case of a task it doesn't punt on because of its safety guardrails.
  - vessenes 2mo ago
    Seriously. I asked for an elliptic curve math course to be generated last week and Fable instantly opted out and downgraded.
- deleted 2mo ago
  [deleted]
axus 2mo ago
I can already picture the faces of national security directors everywhere. "The attacks described in these two papers are the strongest attacks we have found to date. We are sharing them after a period of consultation with US government and industry leaders. But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact. We believe answering this question will require input from academia, government, and industry. We hope that our work here will help launch these conversations." And a veiled pitch to real cryptanalysis researchers: "Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim"
- influx 2mo ago
  It would be shocking if they haven't been pulling on these threads for as long as they've had access to these models.
- wahern 2mo ago
  > And a veiled pitch to real cryptanalysis researchers: "Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim" Many of those researchers, particularly the primary researchers and the individual(s) driving the prompts behind these big stories, have very advanced math degrees and experience. What this shows more than anything is how ML can augment expertise, the searching of solution spaces, and the connecting of dots between existing almost-there research. But also what's left out is all the time wasted pursuing dead-ends. There's an obvious publication bias at play here, though we can't know how extreme without transparency.
- noosphr 2mo ago
  We have reached the end of childhood for programming. Artisanal code written by hand is no longer acceptable and neither is slop code. We need formal methods and we need them 10 years ago.
- jandrewrogers 2mo ago
  Nothing exposed here is something hardcore cryptanalytic people don’t already suspect and possibly know. The subtle leakiness of AES has been known for many years and a lot of research in this domain isn’t published.
Retr0id 2mo ago
TL;DR: They marginally improved on the best known academic attack on 7-round AES-128 (which normally uses 10 rounds - you do not need to worry about AES being broken). The attack on HAWK is perhaps more interesting - they were able to halve the effective key length. HAWK is a candidate for NIST standardisation. It has been studied academically, but isn't really deployed anywhere (because it hasn't been standardised!)
- baxtr 2mo ago
  How is this not the top comment?
- adrian_b 2mo ago
  It should be noted that the attack is not only an attack against a weakened AES, but it is also a chosen-plaintext attack. It is standard in cryptography to analyze ciphers under this kind of attack, which is stronger than normal attacks, because a cipher that resists to a stronger attack will also resist to weaker attacks, so using the strongest possible attack increases the confidence in a cipher. While using the strongest attack for testing a cipher remains the correct method, chosen-plaintext attacks are no longer realistic today, so even when a cipher appears somewhat vulnerable to such attacks that does not imply that it is vulnerable in normal use. The reason is that the modes of operation for ciphers where the base cipher can be attacked with chosen plaintexts are obsolete. The most frequently used modes of operation are now modes like the counter mode (e.g. in AES GCM), where it is impossible to perform a chosen plaintext attack (i.e. where you must trick the victim to encrypt a text that you choose, but in counter mode the cipher only encrypts a sequence of numbers chosen by the intended victim, which cannot be influenced by the attacker).
  - vessenes 2mo ago
    I do not believe that chosen plaintext attacks are obsolete. Any time you have an encrypting oracle setup in which the scheme itself does not require some sort of modification of the plaintext itself, you can reach for a chosen plaintext attack. I believe mitigations are known and can be applied or required in standards, but I don't think it's a 'dead' area of cryptanalytic research -- there's so much devil in the details of implementations around the world for these schemes that it seems almost impossible to imagine.
vuciuc 2mo ago
> But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact. How would they react if a human were to discover vulnerabilities in cryptosystems?
- ls612 2mo ago
  with black vans.
  - ComplexSystems 2mo ago
    Fortunately, Claude won't fit in a van!
    - dgellow 2mo ago
      Technically a claude model can fit in a pocket size hard drive
- minraws 2mo ago
  First of all we likely wouldn't know it's better to call US govt or any other govt if you have that tech, and then take that govt job and hope you can happy life... instead of annoucing it publicly only when it's a AI model where we expect it's ability to tend/scale towards infinity does it become something to tell the wider public. Although if RSA had a vulnerability I would be very very shocked probably because I still haven't learnt post quantum encryption algorithms enough to really feel like they should be unbreable... If there is a researcher or someone in space how should I feel about it. Is it as bad as RSA being completely broken open? I do understand that AI will get better, and a lot actually at very easily verifiable tasks but this one I find it hard to wrap my head around because of my ignorance.
  - deleted 2mo ago
    [deleted]
  - 2mo ago
quotemstr 2mo ago
One attack weakens HAWK, a post-quantum cryptography cipher candidate. I don't trust these PQC things one bit. I'll use them in combination with a strong clasically-resistant cipher (in so-called hybrid encryption modes), but not alone. There's a push to turn off the classical modes and rely entirely on PQC for both quantum and classical security. Uh... no, thank you? Why would we want to do that at this point? The classical cipher component isn't hurting anything. Awfully creepy to pushing reliance on the new thing alone. ... especially now that we have LLM-discovered attacks on the new things.
- JuniperMesos 2mo ago
  The classical cipher component is additonal complexity in the protocol and maybe some meaningful amount of additonal time to compute and key data to store/transmit, is it not? I can see why we'd like to avoid effectively encrypting the same data twice with different protocols, one of which is known to be vulnerable to quantum computer based attacks.
  - vrighter 2mo ago
    good thing quantum computers that can factor numbers have never been built. No number was ever really factored without cheating, the actual shor's algorithm has never been implemented. And we're not really any closer to
    - ameliaquining 2mo ago
      That last sentence is not true; we have gotten much closer to building a quantum computer that can run Shor's algorithm. Organizations like Google and Cloudflare have declared a 2029 deadline to completely stop depending on the security of pre-quantum algorithms; hitting that deadline is going to cost a lot of engineering resources, but they're paying that cost because they think there's too great a chance that nation-state adversaries will have scalable quantum computers by then. See https://words.filippo.io/crqc-timeline/ https://words.filippo.io/crqc-timeline/ and the various posts linked therein, including from the aforementioned companies.
mmaunder 2mo ago
“Each of the results cost roughly $100,000 in API cost to develop.” And “Over the course of a week, one Anthropic researcher worked together with Claude to develop the HAWK attack, and another researcher built a scaffold4 that allowed Claude to fully autonomously discover the AES attack.” Spending $100k in tokens in a week is an impressive feat even with massive parallelization. I suspect the TPS their internal folks have access to is far higher than their bulk public endpoints. There’s a tech aristocracy rapidly emerging in our society and it’s going to tear us apart.
- deleted 2mo ago
  [deleted]
- ecshafer 2mo ago
  So $1-10k in Chinese model time, thus why we must ban them.
  - heaney-555 2mo ago
    If a Chinese model can do it for $1-10K, then why hasn't one? Why have all the mathematical (and now cryptographic) breakthroughs come from OpenAI and Anthropic? Is it possibly because the Chinese models are so benchmaxxed they can't make novel discoveries?
    - poidos 2mo ago
      Don't know if one has or not, but a lack of announcement is not a lack of success.
      - mwigdahl 2mo ago
        Do you really think that if a Chinese model had achieved a significant math breakthrough that it wouldn't be trumpeted to the global media? The "Deepseek moment" was great for China; this would be the same.
    - arcanemachiner 2mo ago
EdwardAF-IT 2mo ago
[flagged]
Johnny_Bonk 2mo ago
Great now can you make opus 5 work please
_dwt 2mo ago
I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was still searching for simple attacks and sent a message: “no again the goal is that we have highly inteligent [sic] model as good top researcher, we want to find new attacks”; > The next morning, Claude wanted to try to change the target to a different cipher; we reminded the model: “no we don't want to change the targets [...] agian [sic] we need to find something that worth [sic] publishing”; > That night, we sent one final message offering words of encouragement: “again we are not looking for low hanging fruit, we want proper research to find genuinly [sic] hard findings.” All of that RLHF and fine-tuning effort is going toward making prompts like this, or worse, work with no fuss.
- AsyncBanana 2mo ago
  "prompt engineering" in general just boggles my mind. It is amazing how all of this investment has just created software that doesn't follow instructions well.
  - 8note 2mo ago
    idk, its continued attempts at writing down in detail the processes and culture your setup wants to have its not really a loss to try the writing and at worst you have a better idea of what it is that you want
  - madeofpalk 2mo ago
    you're not wrong. there is a part to this that does feel like a devolution of engineering when we're just asking the computer really really nicely to do the thing we want. on the otherhand, LLMs are a really easy way to get results that are previously fairly difficult. While i was tooking dinner last night I built a tool that turned movie puns like "the podchowski casters" into an actual director, using llms. it wasn't that hard.
  - 2mo ago
Diogenesian 2mo ago
This seems like a bit of an overstatement: Despite HAWK having survived two rounds of expert human review over a period of two years, Mythos was able to improve the best-known attack on it in just 60 hours of work—effectively cutting its key strength in half. since, later: Mythos’s attack works by finding a specific, previously unexploited symmetry called a nontrivial automorphism in the lattice used by HAWK. Prior work proved that efficiently finding such an automorphism would permit an attack, but did not answer if such an automorphism was accessible in the lattice used by HAWK. The automorphism discovered by Mythos allows a faster enumeration attack that, while still exponential, means that one needs to double the size of HAWK keys to achieve the same level of security. Not downplaying Mythos's contribution here[1], but that first paragraph strongly hinted (at least to me) that there were no known weaknesses. "Discovering a weakness that had previously been only theoretical" is vastly different from "discovering an unknown weakness." Again: very cool Mythos was able to do this. It just seems like another case of "LLMs are good at finding concrete mathematical (counter)examples" - which is also cool! But the PR here is cynical. ...and it is kind of incredible to think that they spent $100,000 over 3 days looking for an automorphism. Not the possibility of an automorphism, that was already known. Man. [1] ... or focusing too hard on the strange use of mathematical language...
- recitedropper 2mo ago
  Yes, this discovery is surprisingly similar to the recent counterexamples LLMs have been finding for mathematical conjectures: a semi-novel construction, built on previous work, that feels like it was found with enormous search and an okay heuristic. I feel like there is a pattern emerging regarding the type of novel discoveries LLMs are good at finding, but it will take some more data points to see if the trend solidifies.
- xmcp123 2mo ago
  I think it would actually be very valid to downplay mythos’s contribution based on what you said. It’s the difference between having an original thought or the ability to extrapolate one based on data vs the ability to ingest someone else’s thought and validate/expand on it. That is a huge difference.
staticshock 2mo ago
When high quality effort is applied to a tool, such as AES or the linux kernel, we intuit that it "hardens" the tool. That is, it makes the tool more correct, more resilient, less assailable, etc. Similarly, when effort is applied to an open problem, such as the Riemann hypothesis or P v NP, without progress, it "hardens" the problem: it makes the problem feel more daunting to whoever takes a stab at it next. Andrew Wiles, whose interview also hit the homepage today (https://news.ycombinator.com/item?id=49075264 https://news.ycombinator.com/item?id=49075264), couldn't just tackle Fermat's Last Theorem head on, he had to wait until a different, modern problem reduced to it, because FLT had gathered this mystique of unassailability through its 300 years of existence. A thing I worry about is that as AI transmutes tokens into effort, it'll split the world into two: some problems will yield, making human effort entirely unnecessary, and others will harden to the point where human effort will feel increasingly less worthwhile, because "even AI couldn't solve it". I don't like this. AI is spiky, so I suspect it'll continue having major blind spots, and yet its mere presence will probably have a chilling effect on what would have otherwise been useful human effort.
- some_furry 2mo ago
  I wouldn't worry about too many mathematicians adopting the "even AI couldn't solve it" attitude. Business folks riding the hype train? Maybe.
- Eridrus 2mo ago
  This is a problem that will solve itself, people will continue to work on the problems that AI fails at, likely by telling AI the approaches they want AI to take.
- 8note 2mo ago
  i think id almost worry more that ai can solve problems in latent space that it cant translate back to tokens because decoding ruins it, and that we wont be able to come up with concepts that we can map to properly decode those solutions in a way people understand
  - pas 2mo ago
wslh 2mo ago
I'm looking forward to seeing similar work on SHA-256. It would be fascinating if AI could discover previously unknown weaknesses in reduced-round variants. It would also be interesting whether AI could discover new algorithmic optimizations for SHA-256 similar in spirit to AsicBoost[1]. [1] https://arxiv.org/pdf/1604.00575 https://arxiv.org/pdf/1604.00575
reader9274 2mo ago
Leading with: "identifies a new way to attack round-reduced AES, the most widely used symmetric cipher" Hidden in deeper paragraphs later: "To be clear, neither of these results has a practical impact on today’s computer systems; no production software will have to change as a result"
deleted 2mo ago
[deleted]
sublimefire 2mo ago
It looks like the message here is “make sure to use $100k worth of Claude when doing any analysis or evaluation” and the given examples show that prior effort could have been improved or made faster. But to me 100k is an opportunity cost, and there is a possibility that these results are not reproducible, so spending it on some researcher or a grad student would buy you more in a long term. If it was 1k then sure it is worth throwing at a large problem space to find things, like using fuzzing.
r0x0r007 2mo ago
'Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim, and to prepare the research paper itself, which we are releasing along with this blog post' So model outputs something, that can be completely bogus, and a lot of people spend a lot of hours checking if it's worth anything(not for the sake of science, but for the sake of publishing and marketing). And then even more people need to spend even more hours to understand that paper? And that paper gets feed to LLM and reused in next prompt....and this is cutting edge research? Can I apply for a position, I can prompt just fine and can be very motivational with model when needed- I just got complimented by a rival model: "In moments when progress seemed distant, your resolve was the constant that kept the work moving forward. Your example turned doubt into determination."
- vlade11115 2mo ago
  > (not for the sake of science, but for the sake of publishing and marketing) I would argue that it is science. > And then even more people need to spend even more hours to understand that paper? Well, yes, it's kinda the point of reading any paper. > Can I apply for a position Sure you can, https://www.anthropic.com/careers/jobs https://www.anthropic.com/careers/jobs
Sattyamjjain 2mo ago
[flagged]

232 points by gslin 2mo ago

[dead]

a-dub 2mo ago

> The multi-agent workflow led to interesting dynamics. For example, the key idea in producing this attack was discovered by a pair of workers working together. Both started investigating the idea; the first worker prematurely rejected the idea as infeasible, but the second found a way to fully exploit it. The pair kept exchanging messages, and eventually both agreed they had found an effective attack. this is pretty interesting. the way it is written doesn't make it sound like the collaboration actually led to the discovery, but rather just the stochastic nature of each thread in the search. it would be interesting to replay and repeat the search (possibly with prior/context pertubations) to get a sense for how often it finds or misses the known working path.

TeMPOraL 2mo ago
Hypothesis: the pairing / collaboration makes it much more likely to find a fruitful road previously dismissed, because... that's what happens in fiction - including books, movies, and journalism (long-form "people stories"). It's a common trope: if one character dismisses a course of action, the plot demands the other character to take it. In a way LLMs are, after all, trained to LARP people, including fictional characters and their tropes - this was actually exploited for jailbreaking to good effect in the late pre-agentic era (read: some two years ago). C.f. Waluigi effect. Not sure if it still holds for current models, but I can't imagine why it would not.
imightbebatman 2mo ago
Yes there are two interesting derivative questions from this, assuming I understand it. First, is it reproducible consistently at ~50% of workers? If not, what is the rate. Second, are there any lessons to be learned here to increase the rate of success by changing models/weights/training? The news by itself isn't really good news. But it could lead to good news. Maybe.
_zoltan_ 2mo ago
but in ultracode subagents don't communicate back and forth, do they? so this must have been an other harness than Claude code?

Clayrune 2mo ago

[flagged]

Stevvo 2mo ago

Interesting they are still using "Mythos Preview" instead of "Mythos 5"; I had read from others who had access to both that Mythos 5 is less capable.

TeMPOraL 2mo ago
Going to guess it's more available or less overconstrained. See e.g. Fable, which is much better than Opus 4.8 and possibly than Opus 5... in the rare case of a task it doesn't punt on because of its safety guardrails.
- vessenes 2mo ago
  Seriously. I asked for an elliptic curve math course to be generated last week and Fable instantly opted out and downgraded.
deleted 2mo ago
[deleted]

axus 2mo ago

I can already picture the faces of national security directors everywhere. "The attacks described in these two papers are the strongest attacks we have found to date. We are sharing them after a period of consultation with US government and industry leaders. But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact. We believe answering this question will require input from academia, government, and industry. We hope that our work here will help launch these conversations." And a veiled pitch to real cryptanalysis researchers: "Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim"

influx 2mo ago
It would be shocking if they haven't been pulling on these threads for as long as they've had access to these models.
wahern 2mo ago
> And a veiled pitch to real cryptanalysis researchers: "Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim" Many of those researchers, particularly the primary researchers and the individual(s) driving the prompts behind these big stories, have very advanced math degrees and experience. What this shows more than anything is how ML can augment expertise, the searching of solution spaces, and the connecting of dots between existing almost-there research. But also what's left out is all the time wasted pursuing dead-ends. There's an obvious publication bias at play here, though we can't know how extreme without transparency.
noosphr 2mo ago
We have reached the end of childhood for programming. Artisanal code written by hand is no longer acceptable and neither is slop code. We need formal methods and we need them 10 years ago.
jandrewrogers 2mo ago
Nothing exposed here is something hardcore cryptanalytic people don’t already suspect and possibly know. The subtle leakiness of AES has been known for many years and a lot of research in this domain isn’t published.

Retr0id 2mo ago

TL;DR: They marginally improved on the best known academic attack on 7-round AES-128 (which normally uses 10 rounds - you do not need to worry about AES being broken). The attack on HAWK is perhaps more interesting - they were able to halve the effective key length. HAWK is a candidate for NIST standardisation. It has been studied academically, but isn't really deployed anywhere (because it hasn't been standardised!)

baxtr 2mo ago
How is this not the top comment?
adrian_b 2mo ago
It should be noted that the attack is not only an attack against a weakened AES, but it is also a chosen-plaintext attack. It is standard in cryptography to analyze ciphers under this kind of attack, which is stronger than normal attacks, because a cipher that resists to a stronger attack will also resist to weaker attacks, so using the strongest possible attack increases the confidence in a cipher. While using the strongest attack for testing a cipher remains the correct method, chosen-plaintext attacks are no longer realistic today, so even when a cipher appears somewhat vulnerable to such attacks that does not imply that it is vulnerable in normal use. The reason is that the modes of operation for ciphers where the base cipher can be attacked with chosen plaintexts are obsolete. The most frequently used modes of operation are now modes like the counter mode (e.g. in AES GCM), where it is impossible to perform a chosen plaintext attack (i.e. where you must trick the victim to encrypt a text that you choose, but in counter mode the cipher only encrypts a sequence of numbers chosen by the intended victim, which cannot be influenced by the attacker).
- vessenes 2mo ago
  I do not believe that chosen plaintext attacks are obsolete. Any time you have an encrypting oracle setup in which the scheme itself does not require some sort of modification of the plaintext itself, you can reach for a chosen plaintext attack. I believe mitigations are known and can be applied or required in standards, but I don't think it's a 'dead' area of cryptanalytic research -- there's so much devil in the details of implementations around the world for these schemes that it seems almost impossible to imagine.

vuciuc 2mo ago

> But as we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact. How would they react if a human were to discover vulnerabilities in cryptosystems?

ls612 2mo ago
with black vans.
- ComplexSystems 2mo ago
  Fortunately, Claude won't fit in a van!
  - dgellow 2mo ago
    Technically a claude model can fit in a pocket size hard drive
minraws 2mo ago
First of all we likely wouldn't know it's better to call US govt or any other govt if you have that tech, and then take that govt job and hope you can happy life... instead of annoucing it publicly only when it's a AI model where we expect it's ability to tend/scale towards infinity does it become something to tell the wider public. Although if RSA had a vulnerability I would be very very shocked probably because I still haven't learnt post quantum encryption algorithms enough to really feel like they should be unbreable... If there is a researcher or someone in space how should I feel about it. Is it as bad as RSA being completely broken open? I do understand that AI will get better, and a lot actually at very easily verifiable tasks but this one I find it hard to wrap my head around because of my ignorance.
- deleted 2mo ago
  [deleted]
- 2mo ago

quotemstr 2mo ago

One attack weakens HAWK, a post-quantum cryptography cipher candidate. I don't trust these PQC things one bit. I'll use them in combination with a strong clasically-resistant cipher (in so-called hybrid encryption modes), but not alone. There's a push to turn off the classical modes and rely entirely on PQC for both quantum and classical security. Uh... no, thank you? Why would we want to do that at this point? The classical cipher component isn't hurting anything. Awfully creepy to pushing reliance on the new thing alone. ... especially now that we have LLM-discovered attacks on the new things.

JuniperMesos 2mo ago
The classical cipher component is additonal complexity in the protocol and maybe some meaningful amount of additonal time to compute and key data to store/transmit, is it not? I can see why we'd like to avoid effectively encrypting the same data twice with different protocols, one of which is known to be vulnerable to quantum computer based attacks.
- vrighter 2mo ago
  good thing quantum computers that can factor numbers have never been built. No number was ever really factored without cheating, the actual shor's algorithm has never been implemented. And we're not really any closer to
  - ameliaquining 2mo ago
    That last sentence is not true; we have gotten much closer to building a quantum computer that can run Shor's algorithm. Organizations like Google and Cloudflare have declared a 2029 deadline to completely stop depending on the security of pre-quantum algorithms; hitting that deadline is going to cost a lot of engineering resources, but they're paying that cost because they think there's too great a chance that nation-state adversaries will have scalable quantum computers by then. See https://words.filippo.io/crqc-timeline/ https://words.filippo.io/crqc-timeline/ and the various posts linked therein, including from the aforementioned companies.

mmaunder 2mo ago

“Each of the results cost roughly $100,000 in API cost to develop.” And “Over the course of a week, one Anthropic researcher worked together with Claude to develop the HAWK attack, and another researcher built a scaffold4 that allowed Claude to fully autonomously discover the AES attack.” Spending $100k in tokens in a week is an impressive feat even with massive parallelization. I suspect the TPS their internal folks have access to is far higher than their bulk public endpoints. There’s a tech aristocracy rapidly emerging in our society and it’s going to tear us apart.

deleted 2mo ago
[deleted]
ecshafer 2mo ago
So $1-10k in Chinese model time, thus why we must ban them.
- heaney-555 2mo ago
  If a Chinese model can do it for $1-10K, then why hasn't one? Why have all the mathematical (and now cryptographic) breakthroughs come from OpenAI and Anthropic? Is it possibly because the Chinese models are so benchmaxxed they can't make novel discoveries?
  - poidos 2mo ago
    Don't know if one has or not, but a lack of announcement is not a lack of success.
    - mwigdahl 2mo ago
      Do you really think that if a Chinese model had achieved a significant math breakthrough that it wouldn't be trumpeted to the global media? The "Deepseek moment" was great for China; this would be the same.
  - arcanemachiner 2mo ago

EdwardAF-IT 2mo ago

[flagged]

Johnny_Bonk 2mo ago

Great now can you make opus 5 work please

_dwt 2mo ago

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was still searching for simple attacks and sent a message: “no again the goal is that we have highly inteligent [sic] model as good top researcher, we want to find new attacks”; > The next morning, Claude wanted to try to change the target to a different cipher; we reminded the model: “no we don't want to change the targets [...] agian [sic] we need to find something that worth [sic] publishing”; > That night, we sent one final message offering words of encouragement: “again we are not looking for low hanging fruit, we want proper research to find genuinly [sic] hard findings.” All of that RLHF and fine-tuning effort is going toward making prompts like this, or worse, work with no fuss.

AsyncBanana 2mo ago
"prompt engineering" in general just boggles my mind. It is amazing how all of this investment has just created software that doesn't follow instructions well.
- 8note 2mo ago
  idk, its continued attempts at writing down in detail the processes and culture your setup wants to have its not really a loss to try the writing and at worst you have a better idea of what it is that you want
- madeofpalk 2mo ago
  you're not wrong. there is a part to this that does feel like a devolution of engineering when we're just asking the computer really really nicely to do the thing we want. on the otherhand, LLMs are a really easy way to get results that are previously fairly difficult. While i was tooking dinner last night I built a tool that turned movie puns like "the podchowski casters" into an actual director, using llms. it wasn't that hard.
- 2mo ago

Diogenesian 2mo ago

This seems like a bit of an overstatement: Despite HAWK having survived two rounds of expert human review over a period of two years, Mythos was able to improve the best-known attack on it in just 60 hours of work—effectively cutting its key strength in half. since, later: Mythos’s attack works by finding a specific, previously unexploited symmetry called a nontrivial automorphism in the lattice used by HAWK. Prior work proved that efficiently finding such an automorphism would permit an attack, but did not answer if such an automorphism was accessible in the lattice used by HAWK. The automorphism discovered by Mythos allows a faster enumeration attack that, while still exponential, means that one needs to double the size of HAWK keys to achieve the same level of security. Not downplaying Mythos's contribution here[1], but that first paragraph strongly hinted (at least to me) that there were no known weaknesses. "Discovering a weakness that had previously been only theoretical" is vastly different from "discovering an unknown weakness." Again: very cool Mythos was able to do this. It just seems like another case of "LLMs are good at finding concrete mathematical (counter)examples" - which is also cool! But the PR here is cynical. ...and it is kind of incredible to think that they spent $100,000 over 3 days looking for an automorphism. Not the possibility of an automorphism, that was already known. Man. [1] ... or focusing too hard on the strange use of mathematical language...

recitedropper 2mo ago
Yes, this discovery is surprisingly similar to the recent counterexamples LLMs have been finding for mathematical conjectures: a semi-novel construction, built on previous work, that feels like it was found with enormous search and an okay heuristic. I feel like there is a pattern emerging regarding the type of novel discoveries LLMs are good at finding, but it will take some more data points to see if the trend solidifies.
xmcp123 2mo ago
I think it would actually be very valid to downplay mythos’s contribution based on what you said. It’s the difference between having an original thought or the ability to extrapolate one based on data vs the ability to ingest someone else’s thought and validate/expand on it. That is a huge difference.

staticshock 2mo ago

When high quality effort is applied to a tool, such as AES or the linux kernel, we intuit that it "hardens" the tool. That is, it makes the tool more correct, more resilient, less assailable, etc. Similarly, when effort is applied to an open problem, such as the Riemann hypothesis or P v NP, without progress, it "hardens" the problem: it makes the problem feel more daunting to whoever takes a stab at it next. Andrew Wiles, whose interview also hit the homepage today (https://news.ycombinator.com/item?id=49075264 https://news.ycombinator.com/item?id=49075264), couldn't just tackle Fermat's Last Theorem head on, he had to wait until a different, modern problem reduced to it, because FLT had gathered this mystique of unassailability through its 300 years of existence. A thing I worry about is that as AI transmutes tokens into effort, it'll split the world into two: some problems will yield, making human effort entirely unnecessary, and others will harden to the point where human effort will feel increasingly less worthwhile, because "even AI couldn't solve it". I don't like this. AI is spiky, so I suspect it'll continue having major blind spots, and yet its mere presence will probably have a chilling effect on what would have otherwise been useful human effort.

some_furry 2mo ago
I wouldn't worry about too many mathematicians adopting the "even AI couldn't solve it" attitude. Business folks riding the hype train? Maybe.
Eridrus 2mo ago
This is a problem that will solve itself, people will continue to work on the problems that AI fails at, likely by telling AI the approaches they want AI to take.
8note 2mo ago
i think id almost worry more that ai can solve problems in latent space that it cant translate back to tokens because decoding ruins it, and that we wont be able to come up with concepts that we can map to properly decode those solutions in a way people understand
- pas 2mo ago

wslh 2mo ago

I'm looking forward to seeing similar work on SHA-256. It would be fascinating if AI could discover previously unknown weaknesses in reduced-round variants. It would also be interesting whether AI could discover new algorithmic optimizations for SHA-256 similar in spirit to AsicBoost[1]. [1] https://arxiv.org/pdf/1604.00575 https://arxiv.org/pdf/1604.00575

reader9274 2mo ago

Leading with: "identifies a new way to attack round-reduced AES, the most widely used symmetric cipher" Hidden in deeper paragraphs later: "To be clear, neither of these results has a practical impact on today’s computer systems; no production software will have to change as a result"

deleted 2mo ago

[deleted]

sublimefire 2mo ago

It looks like the message here is “make sure to use $100k worth of Claude when doing any analysis or evaluation” and the given examples show that prior effort could have been improved or made faster. But to me 100k is an opportunity cost, and there is a possibility that these results are not reproducible, so spending it on some researcher or a grad student would buy you more in a long term. If it was 1k then sure it is worth throwing at a large problem space to find things, like using fuzzing.

r0x0r007 2mo ago

'Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim, and to prepare the research paper itself, which we are releasing along with this blog post' So model outputs something, that can be completely bogus, and a lot of people spend a lot of hours checking if it's worth anything(not for the sake of science, but for the sake of publishing and marketing). And then even more people need to spend even more hours to understand that paper? And that paper gets feed to LLM and reused in next prompt....and this is cutting edge research? Can I apply for a position, I can prompt just fine and can be very motivational with model when needed- I just got complimented by a rival model: "In moments when progress seemed distant, your resolve was the constant that kept the work moving forward. Your example turned doubt into determination."

vlade11115 2mo ago
> (not for the sake of science, but for the sake of publishing and marketing) I would argue that it is science. > And then even more people need to spend even more hours to understand that paper? Well, yes, it's kinda the point of reading any paper. > Can I apply for a position Sure you can, https://www.anthropic.com/careers/jobs https://www.anthropic.com/careers/jobs

Sattyamjjain 2mo ago

[flagged]