6 ms·

Anthropic AI

128 points by vulkd 5y ago

chetan_v 5y ago
Looking at the team seems to be all ex-openai employees and one of the cofounders worked on building gpt3. Will be exciting to see what they are working on and if it will be similar work to openai but more commercialized.
- ilaksh 5y ago
  I wonder why they left Open AI? Maybe they didn't like Microsoft's influence, or they felt that Open AI wasn't really open, or that it didn't have enough focus on safety? Or they just didn't like their boss that much?
gavanwilhite 5y ago
This looks quite promising!
- mark_l_watson 5y ago
  I agree! Love the public benefit aspect of Anthropic.
ansk 5y ago
I can't find any mention of who currently comprises the core research team. It mentions Dario Amodei as CEO, and their listed prior work suggests some others from OpenAI may be tagging along. However, the success of this group is going to be highly dependent on the caliber of the research team, and I was hoping to see at least a few prominent researchers listed. I believe OpenAI launched with four or five notable researchers as well as close ties to academia via the AI group at Berkeley. Does anyone have further info on the research team?
- phreeza 5y ago
  Chris Olah posted that he is involved.
- chetan_v 5y ago
  Seems you can see some of them on their company linkedin page : https://www.linkedin.com/company/anthropicresearch/about/ https://www.linkedin.com/company/anthropicresearch/about/
  - ansk 5y ago
    LinkedIn authwall, we meet again. Could someone list the researchers (if there are any, and assuming there are only a few). Frankly, it's not a great sign that the Anthropic site isn't touting the research team itself and LinkedIn sleuthing is even necessary.
    - Qworg 5y ago
      Current list (in LI order): * Dario Amodei * Benjamin Mann * Kamal Ndousse * Daniela Amodei * Sam McCandlish * Tom Henighan * Catherine Olsson * Nicholas Joseph * Andrew Jones
      - ansk 5y ago
        Thank you.
      - polynomial 5y ago
        Co-founder Tom Brown
    - deleted 5y ago
      [deleted]
strin 5y ago
https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-research-outfit-from-openais-dario-amodei-and-it-has-124m-to-burn https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-re...
n1g3Jude 5y ago
Complete waste of money.... Better to burn cash directly cause that at least generates heat... This will generate nothing
- etaioinshrdlu 5y ago
  Since this is Hacker News, I'll point out that training on GPUs produces plenty of heat.
  - andyxor 5y ago
    they should create some GPU-based crypto instead of wasting all that heat talking about AI safety when there is no AI in sight is like arguing how many angels can dance on the head of a pin, time is better spent actually building models of (natural) intelligence
    - jwilber 5y ago
      Always seems like the strongest opinions on AI come from those with the weakest backgrounds in the subject.
      - andyxor 5y ago
        you're not wrong, most of the talk about dangers of AI comes from opinionated AI dilettantes like Musk who then throw millions of dollars on pointless "AI safety" initiatives. I just hope some of that money would trickle down to fundamental research into inner workings of common sense.
- m4t3june 5y ago
  That's not true, they might generate some heat with the GPU training
joe_the_user 5y ago
Looks like an interesting project. The thing is, I don't think ideal qualities like "reliable, interpretable, and steerable" can really be simply added "on top of" existing deep learning systems and methods. Much is made of GPT-3's ability to sometimes do logic or even arithmetic. But that ability is unreliable and even more spread through the whole giant model. Extracting a particular piece of specifically logical reasoning from the model is hard problem. You can do it - N-times the cost of the model. And in general, you can add extras to the basic functionality of deep neural nets (few-shot, generational, etc) but with a cost of, again, N-times the base (plus decreased reliability). But the "full" qualities mentioned initially would many-many extras-equivalent to one-shot and need to have them happen on the fly. (And one-shot is fairly easy seeming. Take a system that recognizes images by label ("red", "vehicle", etc). Show it thing X - it uses the categories thing X activates to decide whether other things are similar to thing X. Simple but there's still lots of tuning to do here). Just to emphasize, I think they'll need something extra in the basic approach.
- Der_Einzige 5y ago
  Go check out the entire project of captum for pytorch. I assure you that gradient based explanations can be simply added to existing deep learning systems...
  - joe_the_user 5y ago
    All sorts of explanation scheme can and have be added to existing processes. They just tend to fail to be what an ordinary human would take as an explanation. Note - I never argued that "extras" (including formal "explanations") can't be added to deep learning system. My point is you absolutely can add some steps at generally high cost. The argument is those sequence of small steps won't get you to the ideal of broad flexibility that the OP landing page outlines.
andreyk 5y ago
Excited for this! While OpenAI has generated plenty of overhyped results (imo as an AI researcher), their focus on large scale empirical research is pretty different from most of the field and had yielded some great discoveries. And with this being started by many of the safety and policy people from OpenAI, I am pretty optimistic for it.
- deleted 5y ago
  [deleted]
Animats 5y ago
Their paper "Concrete problems in AI safety"[1] is interesting. Could be more concrete. They're run into the "common sense" problem, which I sometimes define, for robots, as "getting through the next 30 seconds without screwing up". They're trying to address it by playing with the weighting in goal functions for machine learning. They write "Yet intuitively it seems like it should often be possible to predict which actions are dangerous and explore in a way that avoids them, even when we don’t have that much information about the environment." For humans, yes. None of the tweaks on machine learning they suggest do that, though. If your constraints are in the objective function, the objective function needs to contain the model of "don't do that". Which means you've just moved the common sense problem to the objective function. Important problem to work on, even though nobody has made much progress on it in decades. [1] https://arxiv.org/pdf/1606.06565.pdf https://arxiv.org/pdf/1606.06565.pdf
- joe_the_user 5y ago
  So they're on safety, explainability, steerability and so-forth. It seems like you could call these "holistic" properties and they have always been hard to add to add to a system - indeed, it seems like in a conventionally constructed system they tend to have to be engineered-in from the start. The success of deep learning might be something of a curse - it's go enough success that creating a safe system seems to automatically be modifying a neural net to be safe despite it not having the "engineered from the start" quality.
  - Animats 5y ago
    The success of deep learning might be something of a curse Possibly. In AI, someone has a good idea, there's great enthusiasm, people in the field predict Strong AI Real Soon Now, the good idea hits a ceiling, and then people are stuck for a while. AI has been through four cycles of that. The ceiling of the current cycle may be in sight. The next big problem is, as they say, "safety", or "common sense". Nobody really has a handle on how to do that yet. Checking proposed explorations against a simulation of some kind works if you can simulate the world well enough. Outside that niche, it's hard. Collecting vast amounts of data to predict what can go wrong without an underlying model runs into combinatorial limits. More things can go wrong than you are likely to be able to observe. Good that there are people thinking about this.
rm445 5y ago
Is it appropriate to ask an on-topic but naive question? It seems as a layman that safer and more legible AI would come from putting neural networks inside of a more 'old-fashioned AI' framework - layers of control loops, decision trees etc based on the output of the black box AI. So at least if your robot crashes or whatever, the computer vision output is in some legible intermediate form and the motion is tractable to traditional analysis. This can't be an original thought, it must already be done to a large extent. But I get the impression that the way things are going is to get the trained model to 'drive' the output, for whatever the application is. Can someone with current industry knowledge comment?
deleted 5y ago
[deleted]

128 points by vulkd 5y ago

Looking at the team seems to be all ex-openai employees and one of the cofounders worked on building gpt3. Will be exciting to see what they are working on and if it will be similar work to openai but more commercialized.

ilaksh 5y ago
I wonder why they left Open AI? Maybe they didn't like Microsoft's influence, or they felt that Open AI wasn't really open, or that it didn't have enough focus on safety? Or they just didn't like their boss that much?

gavanwilhite 5y ago

This looks quite promising!

mark_l_watson 5y ago
I agree! Love the public benefit aspect of Anthropic.

ansk 5y ago

I can't find any mention of who currently comprises the core research team. It mentions Dario Amodei as CEO, and their listed prior work suggests some others from OpenAI may be tagging along. However, the success of this group is going to be highly dependent on the caliber of the research team, and I was hoping to see at least a few prominent researchers listed. I believe OpenAI launched with four or five notable researchers as well as close ties to academia via the AI group at Berkeley. Does anyone have further info on the research team?

phreeza 5y ago
Chris Olah posted that he is involved.
chetan_v 5y ago
Seems you can see some of them on their company linkedin page : https://www.linkedin.com/company/anthropicresearch/about/ https://www.linkedin.com/company/anthropicresearch/about/
- ansk 5y ago
  LinkedIn authwall, we meet again. Could someone list the researchers (if there are any, and assuming there are only a few). Frankly, it's not a great sign that the Anthropic site isn't touting the research team itself and LinkedIn sleuthing is even necessary.
  - Qworg 5y ago
    Current list (in LI order): * Dario Amodei * Benjamin Mann * Kamal Ndousse * Daniela Amodei * Sam McCandlish * Tom Henighan * Catherine Olsson * Nicholas Joseph * Andrew Jones
    - ansk 5y ago
      Thank you.
    - polynomial 5y ago
      Co-founder Tom Brown
  - deleted 5y ago
    [deleted]

strin 5y ago

https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-research-outfit-from-openais-dario-amodei-and-it-has-124m-to-burn https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-re...

n1g3Jude 5y ago

Complete waste of money.... Better to burn cash directly cause that at least generates heat... This will generate nothing

etaioinshrdlu 5y ago
Since this is Hacker News, I'll point out that training on GPUs produces plenty of heat.
- andyxor 5y ago
  they should create some GPU-based crypto instead of wasting all that heat talking about AI safety when there is no AI in sight is like arguing how many angels can dance on the head of a pin, time is better spent actually building models of (natural) intelligence
  - jwilber 5y ago
    Always seems like the strongest opinions on AI come from those with the weakest backgrounds in the subject.
    - andyxor 5y ago
      you're not wrong, most of the talk about dangers of AI comes from opinionated AI dilettantes like Musk who then throw millions of dollars on pointless "AI safety" initiatives. I just hope some of that money would trickle down to fundamental research into inner workings of common sense.
m4t3june 5y ago
That's not true, they might generate some heat with the GPU training

joe_the_user 5y ago

Looks like an interesting project. The thing is, I don't think ideal qualities like "reliable, interpretable, and steerable" can really be simply added "on top of" existing deep learning systems and methods. Much is made of GPT-3's ability to sometimes do logic or even arithmetic. But that ability is unreliable and even more spread through the whole giant model. Extracting a particular piece of specifically logical reasoning from the model is hard problem. You can do it - N-times the cost of the model. And in general, you can add extras to the basic functionality of deep neural nets (few-shot, generational, etc) but with a cost of, again, N-times the base (plus decreased reliability). But the "full" qualities mentioned initially would many-many extras-equivalent to one-shot and need to have them happen on the fly. (And one-shot is fairly easy seeming. Take a system that recognizes images by label ("red", "vehicle", etc). Show it thing X - it uses the categories thing X activates to decide whether other things are similar to thing X. Simple but there's still lots of tuning to do here). Just to emphasize, I think they'll need something extra in the basic approach.

Der_Einzige 5y ago
Go check out the entire project of captum for pytorch. I assure you that gradient based explanations can be simply added to existing deep learning systems...
- joe_the_user 5y ago
  All sorts of explanation scheme can and have be added to existing processes. They just tend to fail to be what an ordinary human would take as an explanation. Note - I never argued that "extras" (including formal "explanations") can't be added to deep learning system. My point is you absolutely can add some steps at generally high cost. The argument is those sequence of small steps won't get you to the ideal of broad flexibility that the OP landing page outlines.

andreyk 5y ago

Excited for this! While OpenAI has generated plenty of overhyped results (imo as an AI researcher), their focus on large scale empirical research is pretty different from most of the field and had yielded some great discoveries. And with this being started by many of the safety and policy people from OpenAI, I am pretty optimistic for it.

deleted 5y ago
[deleted]

Animats 5y ago

Their paper "Concrete problems in AI safety"[1] is interesting. Could be more concrete. They're run into the "common sense" problem, which I sometimes define, for robots, as "getting through the next 30 seconds without screwing up". They're trying to address it by playing with the weighting in goal functions for machine learning. They write "Yet intuitively it seems like it should often be possible to predict which actions are dangerous and explore in a way that avoids them, even when we don’t have that much information about the environment." For humans, yes. None of the tweaks on machine learning they suggest do that, though. If your constraints are in the objective function, the objective function needs to contain the model of "don't do that". Which means you've just moved the common sense problem to the objective function. Important problem to work on, even though nobody has made much progress on it in decades. [1] https://arxiv.org/pdf/1606.06565.pdf https://arxiv.org/pdf/1606.06565.pdf

joe_the_user 5y ago
So they're on safety, explainability, steerability and so-forth. It seems like you could call these "holistic" properties and they have always been hard to add to add to a system - indeed, it seems like in a conventionally constructed system they tend to have to be engineered-in from the start. The success of deep learning might be something of a curse - it's go enough success that creating a safe system seems to automatically be modifying a neural net to be safe despite it not having the "engineered from the start" quality.
- Animats 5y ago
  The success of deep learning might be something of a curse Possibly. In AI, someone has a good idea, there's great enthusiasm, people in the field predict Strong AI Real Soon Now, the good idea hits a ceiling, and then people are stuck for a while. AI has been through four cycles of that. The ceiling of the current cycle may be in sight. The next big problem is, as they say, "safety", or "common sense". Nobody really has a handle on how to do that yet. Checking proposed explorations against a simulation of some kind works if you can simulate the world well enough. Outside that niche, it's hard. Collecting vast amounts of data to predict what can go wrong without an underlying model runs into combinatorial limits. More things can go wrong than you are likely to be able to observe. Good that there are people thinking about this.

rm445 5y ago

Is it appropriate to ask an on-topic but naive question? It seems as a layman that safer and more legible AI would come from putting neural networks inside of a more 'old-fashioned AI' framework - layers of control loops, decision trees etc based on the output of the black box AI. So at least if your robot crashes or whatever, the computer vision output is in some legible intermediate form and the motion is tractable to traditional analysis. This can't be an original thought, it must already be done to a large extent. But I get the impression that the way things are going is to get the trained model to 'drive' the output, for whatever the application is. Can someone with current industry knowledge comment?

deleted 5y ago

[deleted]