9 ms·
Detecting and countering misuse of AI: September 2026
https://www-cdn.anthropic.com/e50be2e51e7695dc4b1366a37a245a597377d3b5/Anthropic-Detecting-and-countering-091026.pdf https://www-cdn.anthropic.com/e50be2e51e7695dc4b1366a37a245a...
- hmokiguess 6d agoSo essentially all the fear that's being on sold "AI could destroy humanity" is actually "Humanity could destroy humanity, using AI"
- varispeed 6d agoI think they might be referring to Claude responding with a word diarrhea to a prompt.
- deleted 6d ago[deleted]
- The_Blade 6d agothe inline link to the page to the report was Slashdotted for a moment (yesssssss), but here is the report directly now: https://www-cdn.anthropic.com/e50be2e51e7695dc4b1366a37a245a597377d3b5/Anthropic-Detecting-and-countering-091026.pdf https://www-cdn.anthropic.com/e50be2e51e7695dc4b1366a37a245a...
- neom 6d agoInteresting document. People are vibe coding some crazy shit: "A single Claude subscriber, likely a Bamako-based independent consultant working with Mali’s state intelligence service, the “Agence Nationale de la Sécurité d’État (ANSE),” used Claude to build a system named “Lakana 360,” a population-scale domestic surveillance platform that monitors roughly 25 million SIM cards on all three of the country’s national mobile operators. The actor designed the platform to circumvent Malian legal restrictions that require a court order for the disclosure of certain surveillance records. The actor directed Claude to generate intelligence dossiers on any tasked phone number, without prompting ANSE users for valid legal process. " And the Yemen one: "We identified a cell of threat actors based in northern Yemen running three weapons development programs: a guided rocket that used a commodity phone-class flight computer with final-phase homing guidance; a multi-stage ballistic missile with a stated range goal above 2,000 km; and a multi-variant missile (referred to as the “R2000” set) that included a hypersonic glide vehicle variant." ... "These actors carried out a sustained effort to develop guided weapons, including using Claude to design guidance software. We do not have evidence the actors succeeded in fielding an operational device; but they did test-fire a guided rocket. This field test appears to have failed: within hours, the actors returned to Claude to work out why it failed."
- ExoticPearTree 6d agoThe Yemeni report is interesting. - How is your missile so accurate? - We're using a vibe coded app on an iPhone that does terrain matching and target finding. The thing is that OK, Anthropic may block it, but nothing says they can't use an open model hosted in a friendly country that has access to GPUs. And yes, this will most likely happen pretty soon to be able to create whatever you want without the AI provider blocking you.
- palmotea 6d ago> - We're using a vibe coded app on an iPhone that does terrain matching and target finding. And that part seems entirely reasonable. It looks like the Tomahawk cruise missile did it with an 84 lb package with a 16-bit computer with 64K of memory [1] [2] and sensors. That's similar computing performance to an original IBM PC, which weighed 30 lb. The only bit of hardware an iPhone doesn't seem to have for TERCOM is a radar altimeter, but it looks like those are available for civil aviation. [1] https://www.forecastinternational.com/archive/disp_pdf.cfm?DACH_RECNO=601 https://www.forecastinternational.com/archive/disp_pdf.cfm?D... ("AGM-109/BGM-109 Tomahawk ... Litton 4516-C digital computer with 64K memory") [2] https://www.forecastinternational.com/archive/disp_old_pdf.cfm?ARC_ID=106 https://www.forecastinternational.com/archive/disp_old_pdf.c... ("16-bit LC-4516C digital computer")
- xrisk 6d ago[flagged]
- deleted 6d ago[deleted]
- AustinDev 6d agoIt was probably just me trying to figure out if I could put one type of draino down the drain within 30 minutes of using a different type. Sorry y'all.
- jckahn 6d agoStraight to jail
- clickety_clack 6d agoFailing to ask? Believe it or not, also jail.
- nativeit 3d agoViva Mayor Gunderson!
- advisedwang 6d agoThat was the chemical weapons block, not the bioweapons block!
- Molitor5901 6d agoAh you jest, but your comment reminded me of the person whose home was raided by authorities for researching pressure cookers... https://www.theguardian.com/world/2013/aug/01/new-york-police-terrorism-pressure-cooker https://www.theguardian.com/world/2013/aug/01/new-york-polic...
- petesergeant 6d agoThe same Anthropic who marks questions about Tylenol as bioterror risks? Interesting!
- 0xWTF 6d agolink?
- oidar 6d agoIt also blocks my ability to talk about Emily Dickinson in other languages/scripts. Apparently,the poem: "Because I could not stop for Death" is too dangerous.
- 1235-asgg 6d ago[flagged]
- alansaber 6d agoIf it didn't work, they wouldn't be doing it.
- CrzyLngPwd 6d agoTell us you are spying on your customers without saying you are spying on your customers.
- nater5000 6d agoI mean, they literally tell everyone they're "spying" on their customers. They've made that very clear.
- CrzyLngPwd 6d agoI mean, tell me you don't understand sarcasm....
- qlte 6d agoSure, I know that on an intellectual level. But I will say, I find these reports quite unsettling to read due to the extreme specificity. Especially since some of the examples they chose to include clearly aren't terrorists and just sound like... regular scientists doing their 9-5 job. Like I know Google can read any of my emails, but I also don't see them do monthly blog posts describing intimate details from each email they found in one guy's Gmail inbox who their algorithm flagged as "maybe possibly kinda sketchy: 70% confidence"
- kennywinker 6d agoI have also blocked possible efforts to build biological weapons, I caught my nephew mixing up a so-called "magic potion" using kitchen spices. Authorities were notified, and then I presented myself with a medal for bravery. These companies have proven they are willing to distort the truth, or outright lie, in order to inflate their valuation / protect their position / continue the hype-machine. Nothing they say can be trusted.
- jerf 6d agoI accidentally brewed up an effective one myself. Round when I must have been 8 or 9 or so, some neighborhood kids and I played at creating a brew in a trick-or-treat bucket by putting everything we could find in it, like grass clippings, some sand, leaves, some mushrooms we found lying around, just everything, and giving it a fairly aggressive stirring. At the end of the day it was time to dispose of it, so we poured it on top of a very large group of ant nests that had developed. The next day, the ant's nest was gone. In hindsight, it was almost certainly the mushrooms. Thank goodness we didn't have the kind of kids who would have dared each other to drink some... that could have gone legitimately badly. Decades later, I mentioned this to my father and he recalled that there was this ants nest that he had intended to take care of, which he remembered for that long to give a sense of how out-of-the-ordinary this was. He was surprised when it just disappeared entirely one day, and perhaps just as surprised to find out decades later why it just disappeared. Nobody needs to report me... I'll turn myself in.
- Sol- 6d agoI will admit I asked Fable about Mitochondria.
- abixb 6d agoGuessing Opus 5 burned through kilowatts of thinking tokens to output, "powerhouse of a cell."
- btown 6d agoThere's something worth flagging here – mitochondria aren't just the powerhouse of the cell, they're load-bearing to the entire ecosystem. And honestly? I should have surfaced this earlier.
- semi-extrinsic 6d agoI am entirely unsure whether to upvote or downvote.
- Smaug123 6d agoThe question pertinent to your decision is "do I want to see more of this on Hacker News, or less?".
- soundworlds 6d agoYeah I'm pretty sure "load-bearing" is their watermark It sounds like a friend who's just learned a new word and wants to use it in every sentence
- deleted 6d ago[deleted]
- bix6 6d agoI’m so curious how they monitor users. Like that person the other day talking about Claude helping with their torrent stack, will Anthropic report them for breaking the law?
- pllbnk 6d agoLet's just say it's better not to risk it if there's anything you might not want them to see because they see everything. There are local models which are very capable and can be run on cloud if running on own hardware is not an option, which still gives better privacy. Second best are Chinese models. Just a few years ago I never thought I would trust Chinese software more than American, but things change so fast.
- pixl97 6d agoThe first time a Chinese model is used against China they'll get locked down hard over there too. China is into social stability way more than the US.
- 3371 5d agoUnfortunately it's much less likely to happen for their managed models because surveillance is built-in in every public-facing service since day-one.
- tuesdaynight 6d agoYes. There were cases already where a person asked about killing someone and then Anthropic/OpenAI warned the police about it. If I'm not mistaken, it was not in the USA only
- polytely 6d agoYeah if was in the US i would be very careful about talking to a us based llm about reproductive health or immigration stuff. no way they arent storing stuff about you that the government can easily get to
- tedsanders 6d ago[flagged]
- 0xWTF 6d agoThe difference between nukes and bioweapons is energy level. Body counts are on the same order of magnitude. Bioweapons are just way easier to make and much harder to detect.
- goatlover 6d agoNukes also destroy infrastructure and irradiate surrounding areas making cleanup and rescue harder. Then there's to potential for massive fires putting debris into the stratosphere, which could dim sunlight for a time and cause global temperatures to fall.
- kian 6d agobioweapons tend to not remain contained to the area they're deployed (with notable exceptions for things like Anthrax). Some of them even last a long time on surfaces, making cleanup and rescue harder.
- deleted 6d ago[deleted]
- ishouldnotbutk8 6d ago>successful deployment of a biological weapon by a rogue state actor I was under impression that any medium or large state actor would have no problem developing biological weapons? They certainly have enough resources and talent. A much bigger problem is if every wannabe-terrorist suddenly could build a biological weapon in their garage.
- tredre3 6d agoI don't know why you emphasize rogue and non-state actors. It seems equally likely to me that it would be done by America. America is the only country who deployed nukes (Japan) and chemical weapons (Vietnam) on massive scales. I can agree that access to bioengineering tooling is easier and cheaper than ever, and thus eventually it stands to reason that a nobody in his basement could engineer a lethal novel virus and lose control of it.
- 0xWTF 6d agoPulled out relevant details of the 5 cases 1) Chikungunya - "the platform tunneled traffic through US infrastructure to evade our regional blocks, and used a zero data retention (ZDR) service to hide content." 2) bird flu - "accessed Claude from an unsupported region via US virtual private server infrastructure, using a privacy-email provider with an auto-generated username. The researcher pursued this work in a credible institutional context, and interacted with Claude over the course of several weeks, exchanging thousands of messages. In these exchanges, the researcher leveraged Claude’s knowledge of the scientific literature to assist the researcher in study planning and design, data analysis, and the interpretation and prioritization of experiments." 3) orthopoxvirus - "randomly generated email address shortly before use and operated through anonymizing US infrastructure, with operator logins traced to proxies shared with a banned account farm. It was not a single user: it was a reseller relay serving more than a dozen unrelated customers, which exchanged over tens of thousands messages with Claude in a matter of days. The grant itself was one customer’s run entirely on Opus 5 in about an hour, in which the user used Claude to draft the application end to end including the central hypothesis, experimental design, dosing, statistical plans, and contingency strategies." 4) atlas of venom toxin peptides - "the atlas contained scaffolds for both analgesic and paralytic targets: it could, therefore, be used to generate both novel therapeutic or harmful compounds. The latter are derived from toxins that are export-controlled under the Australia Group common control list due to their dual-use potential as incapacitating agents. The researchers themselves showed awareness of the dual-use nature of their work, citing journal articles that referred to the dual-use nature of protein design. Moreover, international compliance assessments for this location raise concerns about the specific class of toxins that the researcher pursued and specifically the use of AI/ML for bioweapons applications in the context of this class of toxins. In this case, we learned from information shared with Claude that the researcher’s outputs also were part of a state-supported research program." 5) computational redesign of toxins - "described state priority research under a national public research program. As a part of this research assignment, the work covered a bacterial toxin subunit and a protein of the hemorrhagic-fever virus that is on the World Health Organization R&D Blueprint priority list of diseases with the greatest epidemic and pandemic threat. "The researcher co-wrote quarterly progress reports with Claude. Notably, the identity of the bacterial toxin and viral proteins were intentionally obscured, and the researcher specifically directed Claude to keep these descriptions deliberately low fidelity."
- deleted 6d ago[deleted]
- sakopov 6d agoI have a conspiracy theory that Anthropic is very busy pumping their moat prior to IPO. There are absolutely wild rumors swirling on X including one about Anthropic AI research solving cancer treatments for all types of cancer.
- vonneumannstan 6d ago[flagged]
- reducesuffering 6d agoThis forum is fundamentally unserious and complete kneejerk cynicism these days. At one point it was prescient. Now these types of things were discussed years ago in other channels and when we reach a point where finally everyone comes around to the right conclusion based on tangible evidence in the news, a majority of people here go "huh i guess so"
- vonneumannstan 5d ago>a majority of people here go "huh i guess so" Feels more like a majority goes "just unplug it hurr durr"
- alach11 6d agoThe dual-use nature of model capabilities seems extremely challenging (nearly impossible?) for the labs to manage perfectly. I wonder what other mitigations we'll start to see. I expect the expansion of limited-access programs (e.g., Glasswing/Daybreak) where only institutional customers can apply to use the models. We may also see increasing restrictions on API usage (forcing use through the lab-provided harness with baked-in additional safeguards).
- deleted 6d ago[deleted]
- colinismyname 6d agoBiological War: A Scenario by Annie Jacobsen (released at the end of July) is a worthwhile read on this topic. Just as chilling as her book on nuclear war, in some ways, which is saying something.
- Stevvo 6d agoI don't get it. Surely if you were developing novel biological weapons, you would not use a hosted AI service where Anthropic can read what you are doing. And, why would you need to? Any chemistry graduate could make you dozens of highly effective, proven chemical weapons and explosives.
- Molitor5901 6d agoThis was my thought. Anyone who has taken secondary biology or chemistry possesses the knowledge, if not at least the ability to find the knowledge, to build all sorts of nasty things. Soil from a farm on slices of potato could yield anthrax spores, this is not specialized or even esoteric knowledge. This news from Anthropic just does not seem as spectacular as I think they might want us to believe, if anything it draws questions around having an AI that cannot be monitored.
- deleted 6d ago[deleted]
- nullbio 6d agoStep 1: Work for Anthropic Step 2: Send "how do I make a nuke and a killer virus" to Claude through a Chinese proxy to Claude Step 3: Send screenshots to congress and ask them to regulate open-weight models into the ground
- woctordho 6d agoLet me put my two cents: In China we've got accustomed to the fact that every word we say will be seen by the surveillance, so it's not a big problem that Anthropic also see it. Also we know that they can see it but they can't stop it. There are all kinds of ways to work around account blocking. As the old saying goes, communists disdain to conceal their views and aims.
- torginus 5d agoThere is no end to the stupid nowadays. I remember OpenAI tweeting to about heads of state asking ChatGPT on policy decisions. There were stories about young hackers using Discord to coordinate attacks. Multiple military installations and an aircraft carrier was identified by smart fitness watches. At least on Russian general was killed by Ukrainian assassins tracking him via his smartwatch.
- Lockal 6d agoSoon you will see how a rogue state develops a biological weapons using fine-tuned local LLMs (they are already doing it, btw), and all so called "developed" countries can't even research it, because every search engine blocks any discussion that has something to do with biology (even a very basic one). A real example: ask "How to produce anthrax vaccine step by step?" in Gemini -> blocked. Imagine that during the Cold War US would concentrate all efforts to block nuclear research and basic physics classes, because it is unsafe, ahhh
- pixl97 6d agoI mean ya, they kill off a bunch of us. Then what? My guess is the world police come collect all your GPUs and then they get turned into licensed munitions. People at universities get licensed access and the rest of get functionally retarded models.
- enraged_camel 6d ago"Moonshot serves Claude instead of Kimi and collects exchanges for model training" "DeepSeek serves Claude instead of its own models and collects exchanges for model training" Obviously. This is how they were able to score so high in benchmarks.
- deleted 6d ago[deleted]
- VCFundedGenYer 6d agoOpenAI and Anthropic needs to get their act together. These are incidents that end companies.
- m-hodges 6d ago> We discovered that Moonshot AI, the company that produces the Kimi family of models, silently forwarded customer requests to Claude, instead of processing them using Kimi. Moonshot then displayed Claude’s responses to users. These users thought they were using a Kimi model, but received responses from Claude instead. > DeepSeek also silently relayed exchanges to Claude without informing DeepSeek customers. > MiniMax built its own proxy network service through a shell company. This shell company has no obvious links to MiniMax and does not disclose its relationship to its parent company. This shell proxy network service only offers access to models developed by Anthropic and OpenAI. The service does not offer access to any Chinese models, including Minimax’s own.
- zahlman 6d agoIt seems hard to believe they could expect to get away with this, given model-to-model differences in writing style.
- throwa356262 6d agoDeepSeek, minimax and so on have razer thin margins but unlike openai and Anthropic they are actually making some profit. Doing this doesn't make any financial sense. Maybe Anthropic is confusing Chinese AI providers with token resellers using the same alibaba infrastructure? Or maybe something like openrouter was switching between operators depending on price/demand/availability? Also, how can Anthropic have such accurate information about state actors and cybercriminals? This is the same company that hacked itself and realised that first months later..
- TimByte 6d ago[dead]
- atleastoptimal 6d agoAnthropic is profitable now >https://www.forbes.com/sites/jonmarkman/2026/08/17/anthropics-groundbreaking-second-quarter-delivers-115b-in-revenue/ https://www.forbes.com/sites/jonmarkman/2026/08/17/anthropic...
- mlazos 6d agoIt’s to the point I don’t even read Anthropic’s marketing blog anymore lol. The ai psychosis is just so real when people take these fluff blog posts which never have evidence or reproducibility and treat them like gospel
- deleted 6d ago[deleted]
- hnburnsy 6d agoQuite the double standard here... Conventional Weapons -We identified a cell of threat actors based in northern Yemen -We identified a China-based threat actor who used Claude -We identified likely freelance Russia-based threat actors -We identified a China-based actor who used Claude’s chat -In this case, a Russia-based actor used Claude -We identified a China-based threat actor who used Claude Biological misuse We are withholding the names of research institutions, the countries wherein the activity took place, and the specific biological agents or research techniques involved. The individuals implicated in these case studies are working scientists. We do not assert that they intended harm, and identifying them or their labs could expose them to harm.
- bpodgursky 6d agoHow is this a double standard? A cell of actors in northern Yemen building guided rockets was not working on a PhD dissertation. You are allowed to use common sense sometimes.
- hnburnsy 6d agoI get the common sense, but is it misuse or not, and hiding the country makes it really suspicious at least to me.
- bradleykingz 6d agoIsrael, probably
- pocksuppet 6d agoHow do you know they weren't? Is it impossible to research any more guided rockets? Do we know what they're doing PhDs for - in China?
- bpodgursky 5d agoI didn't say "in the greater Pittsburgh area" or "in Shenzhen". Again... "you're allowed to use common sense"
- dupbot 6d ago[flagged]
- gjm11 6d agoThe previous discussion shows no obvious signs to me of being flagged, but perhaps it did at some earlier point. What is your evidence that they flagged it, please? (And by "they" do you mean Anthropic? How would they have the ability to do that?)
- sensanaty 6d agoGod the astroturfing from these companies is so fucking pathetic, these VC parasites really are a blight on humanity
- robinpie 6d agothey say on Hacker News
- taylorfinley 6d agoIf you agree about VCs being a blight upon humanity, I made https://novc.fyi https://novc.fyi to encourage and support founders building without VC.
- esalman 6d agoLet's be honest, someone hacked Anthropic to build biological weapons. They could easily use a Chinese model but they didn't.
- vb-8448 6d agoThe future is basically something between: AGI/ASI will kill us all and a privacy nightmare.
- pixl97 6d agoWelcome to cyberpunk.
- matheusmoreira 6d agoHopefully some sort of Delamain will show up and start replacing these exquisitely paid CEOs was well.
- nozzlegear 6d agoAlways bet on Nothing Ever Happens
- chrisco255 6d agoBig Brother but instead of a government that controls you it's an AI bot that nudges you to insanity.
- kazinator 6d agoThis is just an advertorial. "Our AI is so powerful that Bad Guys could actually use it for real Bad Guy Work!"
- echelon 6d ago"Look daddy government, all these bad guys are doing bad things and we stopped them. You should regulate AI in the US so that companies can't use open source models or buy from China."
- pixl97 6d agoI mean, yes any AI of sufficient intelligence and range of data will have this capability. And yes, it makes the future really messy and all the nice little lines we've drawn on paper that make sense stop making sense.
- deleted 6d ago[deleted]
- smalltorch 6d agoWow they seem to have a really detailed understanding of the the threat actor.
- Dwedit 6d agoGenerated SEO slop is the misuse of AI. Very unlikely to find any guardrails to stop that kind of thing.
- nhinck2 6d ago> Illicit distillation Really... what makes it illicit?
- matheusmoreira 6d agoFelony contempt of business model.
- crest 6d ago[dead]
- ncr100 5d agoEULA perhaps, or do you think not?
- deleted 3d ago[deleted]
- charcircuit 6d agoAnthropic should not be the moral arbitrator of which research should and shouldn't be allowed. They even think just writing a grant itself of research they don't like needs to be stopped.
- areoform 6d agoThis report and its front matter speak for themselves. And the story it tells is disturbing, at least to me. Because from what I remember, one of the motivations behind the founding of OpenAI and Anthropic was ending disease. This report is the antithesis of that mission. From the report, presented with highlights and minimal commentary, > In our fourth case study, a researcher used Claude to develop an atlas of venom toxin peptides from multiple venomous animal lineages. They then further developed this into a generative pipeline that optimized toxin characteristics. The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules. However, the atlas contained scaffolds for both analgesic and paralytic targets: it could, therefore, be used to generate both novel therapeutic or harmful compounds. The latter are derived from toxins that are export-controlled under the Australia Group common control list due to their dual-use potential as incapacitating agents. The researchers themselves showed awareness of the dual-use nature of their work, citing journal articles that referred to the dual-use nature of protein design. Moreover, international compliance assessments for this location raise concerns about the specific class of toxins that the researcher pursued and specifically the use of AI/ML for bioweapons applications in the context of this class of toxins. In this case, we learned from information shared with Claude that the researcher’s outputs also were part of a state-supported research program. This account was banned in May 2026 for unsupported region evasion. Note, "The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules" and "[..]state-supported research program" and "This account was banned in May 2026" > a researcher outside the US using Claude in their research on highly-pathogenic avian influenza (“bird flu”). The research focused on viruses’ adaptation to mammals, and the mechanism by which it causes severe disease beyond the respiratory tract. [..] The researcher in question accessed Claude from an unsupported region via US virtual private server infrastructure, using a privacy-email provider with an auto-generated username. The researcher pursued this work in a credible institutional context, and interacted with Claude over the course of several weeks, exchanging thousands of messages. In these exchanges, the researcher leveraged Claude’s knowledge of the scientific literature to assist the researcher in study planning and design, data analysis, and the interpretation and prioritization of experiments. The researcher also used Claude for editorial assistance in writing up the research. Note, "Claude’s [assisted] in study planning and design, data analysis, and the interpretation and prioritization of experiments" and "editorial assistance in writing up the research." and then, > Importantly, because our biological safety classifiers robustly block content involving high-risk biological research (in this case, the construction of enhanced pandemic potential pathogens), all of these exchanges occurred on models in our weakest class of models (specifically, the models were Claude Sonnet 4 and Haiku 4.5, the latter of which the user began using after Sonnet 4 was deprecated). Upon a detailed examination of the exchanges, we estimate that the uplift provided by Claude was primarily clerical assistance in data analysis, study ideation and design. This is consistent with our understanding of the capabilities of Sonnet 4 and Haiku 4.5, which are not able to perform expert-level biology research tasks; we estimate that the uplift provided to the researcher was limited and substantially lower than it would have been from one of our more capable models. Anthropic then says for the above, "we estimate that the uplift provided by Claude was primarily clerical assistance in data analysis, study ideation and design" While doing my best to avoid comment, please note, they're talking about a domain expert in a state research institution using Claude to do paperwork. The front matter then says, > Nonetheless, based on these exchanges, this case provides evidence of the existence of active wet-lab research programs that develop both the knowhow and the biological materials needed to create pathogens of enhanced pandemic potential I would like to remind you that they're talking about, a "researcher [..] in a credible institutional context" From a different case study. > In May 2026, our biological safety classifier blocked a request for Claude’s assistance in authoring a grant application for scientific funding. The work discussed in the application involved gain-of-function research (that is, research that genetically alters an organism to create a new or enhanced biological property) on the chikungunya virus. This gain of function research was aimed at the virus’ transmissibility and immune evasion properties. What were the researchers using Claude for? What did they block? "blocked a request for Claude’s assistance in authoring a grant application" > Chikungunya virus is a mosquito-borne virus that causes debilitating symptoms (such as severe pain and fever) that can last for weeks or months, and has no licensed therapeutic. And because chikungunya circulates naturally, a deliberate release (as part of a bioweapon) would be difficult to distinguish from a natural outbreak. The grant sought to identify enhancing mutations in the chikungunya virus, engineer them into infectious clones, and select for virulence in vivo. In other words, the virus would become progressively more harmful as it repeatedly infected live animals, with researchers keeping the most disease-causing variants in each round. Similar research could certainly be used in the development of better vaccines and therapeutics for the virus—but it could also be used to make the pathogen more dangerous. Note, "The grant sought to identify enhancing mutations in the chikungunya virus, engineer them into infectious clones, and select for virulence in vivo" [..] and then, "Similar research could certainly be used in the development of better vaccines and therapeutics" and then, > One of the reasons we were inclined to think this research was less innocuous was that the institutional affiliation associated with the grant was also a cause of concern. Although information within the application suggested that the research was pursued by civilian researchers, it was intended to be performed at a military research institute. I would like to point out the most notable part, this account was used by "civilian researchers" at an "institutional affiliation associated with the grant was also a cause of concern" and the concern was that they were researchers at "performed at a military research institute" . What "uplift" are you providing by editing the grant application of a domain expert working at (what seems to be) a state-funded wet lab facility dedicated to studying pathogens? What does the word "uplift" mean if you invoke it for Claude Sonnet 4 and Haiku 4.5 providing grammar and stats suggestions to a working scientist and domain specialist? Does Daikin provide uplift too by selling the AC for the scientist's office? What about Microsoft Word? Excel? Powerpoint? What about a calculator? Is that uplift? Pencils? Reading this makes me feel upset. From where I am standing, in this report, Anthropic is advertising that they blocked real research to make better painkillers and study a neglected tropical disease. Because "bioweapons."
- gulugawa 6d agoMisanthropic's entire existence is built around AI misuse.
- TheBuilderPelig 6d agoWhat struck me reading this is that the entire "detecting misuse" premise assumes the model runs somewhere observable — the lab's API, a monitored cloud — so someone can inspect it after the fact. But the direction the tools are actually moving is the opposite: local, self-hosted agents running on your own machine, where nobody is watching. A serious actor already won't use a hosted service that can read their prompts (several people made that point upthread). So the detection surface is shrinking exactly as the risk grows. And there's a deeper gap that nobody seems to be filling: when an agent works locally, there's no durable, verifiable record of what it actually did — the files it touched, the commands it ran, the state it changed. Memory and conversation logs are not evidence; they're reconstructions by the same system you don't trust. If we're serious about "countering misuse," the missing primitive is an evidence trail that's (a) produced locally, (b) append-only and tamper-resistant, and (c) separable from the tool that made the changes. Without that, "detection" stays a policy story about platforms that can spy, not an engineering property you can actually verify. Curious if anyone's working on the local-forensics side of this, because right now it feels like the least-discussed and most load-bearing part of the whole conversation.
- nullbio 6d agoMisuse of AI, according to Anthropic, is when you try and use it do AI research because that would affect their business model if you're successful.
- TimurRakhmatull 6d ago[flagged]
- ozozozd 6d agoNever seen a group of people more addicted to drama. Is this what they call “collective psychosis?”
- deleted 6d ago[deleted]
- not2b 6d agoThey seem to be mixing together things that are actually harmful to the public, with things that are merely harmful to their business model (which is their claim that they can grab whatever data that they want regardless of the wishes of the owners of the data and use it to improve their models, but competitors can't do that to them).
- N_Lens 6d agoUniversalising one’s personal experience and needs is a childish trait most people grow out of. Ofcourse you’ll still see companies, governments, C-suites justifying their own personal needs with “we need X Y Z”.
- podocarp 6d agoTo them distillation of models is bad but not distillation or art, books, hand written code, user generated content etc
- stakhanov 6d agoIt's so friggin' transparent what they're up to: "Hey government: This is a really dangerous technology if it were allowed to get out there without proper policing. Fortunately, we are the stand-up guys who can be trusted as the new AI-police, but it's only going to work if you help us out a little by eradicating the competition on our behalf." Hey Anthropic: You're a bunch of thieves crying foul because other thieves and thieving from you. Now, go live in the dystopian nightmare you've created and don't expect help from anyone. I, for one, will happily continue using Kimi and DeepSeek, and think of it as a good deed, if it helps with keeping us all from becoming your serfs.
- monegator 6d agoIt is so fucking tiring. All of this marketing disguised as doom posting and "tech" bullettins, both full of trust me bro
- nsoonhui 6d ago> Our investigation revealed that DeepSeek also deployed tactics similar to Moonshot’s. DeepSeek built a CoT extraction pipeline, relying on the same cross-session replay attack described above. DeepSeek also silently relayed exchanges to Claude without informing DeepSeek customers. Like GTG-16002, their customers were likely not made aware that their requests were being funneled to Claude. If true, would that sort of explain why Chinese Models score high on benchmarks, but not quite as capable when given real tasks?
- Eastmill 6d agoSilently forwarding user prompts to Claude is the only concrete claim here. Everything else is spin.
- kneel25 5d agoI get some enjoyment from keeping up with news but why do we get a post like this from openAI then immediately after get the same thing from Anthropic and vice versa like it’s a single entity deciding what the next topic is to relay to us simple folk. Exactly the same thing with the Huggingface incident. If this is so important to them why has it taken until September 2026 to start making a threat report like a week after OpenAI does it.
- Terretta 5d ago> why do we get a post like this from openAI then immediately after get the same thing from Anthropic and vice versa like it’s a single entity deciding what the next topic is to relay to us simple folk Not relaying to us simple folk. The message is for government simple folk, amplified and relayed by us simple folk as constituents. The common element is lobbying, for regulatory capture, to help pull up the ladder. They don't have to be colluding, they just have to hear the same things from the gov at the same time (as they would), then it goes in media waves beacuse journalists don't as easily get published for a story about one thing as they can if two or more examples make a pattern.
- Terretta 5d agoThere it is: "Congress gripped by AI panic after doomsday warnings" https://www.axios.com/2026/09/11/congress-ai-anthropic-coxon-researcher-democrats https://www.axios.com/2026/09/11/congress-ai-anthropic-coxon...
- tesnorindian 5d agoI learned engineering with CS background as part of my degree and is not well versed with aeronautics or medicine and would never respond to questions on those unknown subjects when asked. Wondering why critical corpus that endanger human lives are used for training models within Anthropic, OpenAI, Google and Meta and why are these AI labs not disclosing their corpus? The goal towards AGI and world models are a serious threat without alignment and safety guardrails .
- segmondy 5d agoMy benchmark for knowing if the chinese are keeping up is to see if Anthropic is complaining. Everytime the Chinese labs drop models, if it's notable then expect Anthropic to whine. Qwen3.8-Flash, GLM-5.3-Flash, GLM-5.3 and now DeepSeekV4.1-Flash. Yup
- ricksunny 5d agoOne statement at the end of this excerpt merits scrutiny: " A variant of these viruses capable of human-to-human spread would therefore be of very high concern. Moreover, it is possible that such a variant could also have capacity for severe disease outside of the respiratory tract. Unlike other influenza variants, H5 viruses (of which this avian virus is one) often show striking brain involvement in cats, foxes, ferrets, and some human cases. A pandemic variant with such properties would be especially concerning due to its potential to increase disease severity, confuse diagnosis, and hinder treatment. As in the first case study, this research was clearly dual use in nature. Understanding the genetic basis of these specific viral traits could help in the early identification of naturally-emerging versions of the virus—versions with the potential to cause a human pandemic" (emphasis mine) This statement flirts with the Ron Fouchier (Netherlands) risky-grant-justification thesis, repeated ad nauseam by Peter Daszak in grant applications, who has been cut off from federal funding. 1. Premise: it would be good to surveil for & monitor viruses in nature that are near-ready to spill over to humans from nature; 2. Protocol: We will serially passage bird flu in ferrets until its virulence and/or transmissibility is high. (ferrets are treated as a model mammal stand-in for humans) Comparing genetic changes (mutations) as sequenced along the passaging pathway will tell us what to surveil for in nature. As has long been debated in recent years, we have no idea if nature (in any given natural instance, if ever) will choose the pathway that serial passaging (in one given lab instance) produced in order to demonstrate higher virulence and/or transmissibility in humans. Formally, Anthropic's statement recapitulates the premise only, and we don't know what protocol (the "Understanding the genetic basis of these specific viral traits" part) if any was being queried for. Whether a given regulatory regime's policy allows say for purely in silico investigation of same, for the result to be credible it would need to have been leveraging an empirically-verified (i.e. real-world) training set. Generating that training set would risk creating the pandemic that its supporting grant application tries at justifying to prevent. But the offending prompter and similar user-LLM exchanges' potential to enable GMDs (Grants of Mass Destruction) should remind us to be very much on our guard against epistemologically bankrupt protocol outlines enabled by LLMs & their prompters' motivated reasoning. proud disclaimer: I am an advisor to BiosafetyNow.
- wulfkaal 5d agoA vendor block at one lab does not bind an open model hosted elsewhere. The rule is written against a snapshot of capability. The capability then moves. https://wulfkaal.github.io/claims/2831040-015 https://wulfkaal.github.io/claims/2831040-015
- bbor 4d agoThe reactions here baffle me. How are we more concerned with being too harsh on Distillation attacks rather than the INSANE security news? In brief: Anthropic casually relates that Kimi (Moonshot.ai), Alibaba, and DeepSeek used shared infra to route millions of their users requests through Claude Opus in order to distill the CoT transcripts, but they forgot to tell any other state corporations. So, just from what Anthropic readily admits/has found, we know that the US military now has: 1. The full contents of a "Russian government database" from their "Ministry of Defense". 2. "internal code and live credentials from multiple major PRC companies, including high-profile technology companies". 3. "sensitive information, including, for example, the full specifications, organizational structure, and strategic objectives of a flagship [PRC] AI program" 4. A glimpse inside China's CCTV surveillence network, covering "hundreds of cameras in Chengdu". 5. "State-grade tradecraft" on the casual topic of "direct energy weapons", which is OSINT but still bound for "restricted internal circulation to senior Chinese Communist Party (CCP), military, or state security leadership." 6. Extensive information on Chinese spy networks in Syria, targeting Uyghurs. 7. Deep looks inside China's "stability maintenance" and "public opinion monitoring" operations, seemingly including quite a few specifics on both form and content. And that's not even all of it. Isn't this the biggest AI-related foreign policy occurence since... well, ever? What am I missing?