7 ms·
EleutherAI announces it has become a non-profit
- eachro 4y agoHow do they pay for their compute?
- mperham 4y agoWith money. A non-profit can still have plenty of revenue.
- ronsor 4y agoDonations and sponsors such as Stability AI, based on the announcement in their Discord server
- stellaathena 4y agoWe have a number of donors including Hugging Face, Stability AI, Nat Friedman, Lambda Labs, and Canva that make our work possible. We also have some orgs that provide sponsorship for computing resources specifically: Stability AI, CoreWeave, and Google Research.
- ftxbro 4y agoare you bottlenecked by engineering
- stellaathena 4y agoYes, we are currently bottlenecked primarily by engineering manpower.
- mcbits 4y agoHave you reached the point of dogfooding your own models to augment work that would normally be considered engineering? If so, I wonder about the implications of black-box models helping to study and develop solutions for interpretability. It gives me "reflections on trusting trust" vibes.
- ENGNR 4y agoI can only imagine the more successful that activity was, the worse the bottleneck would become
- Taek 4y agoNot access to GPUs for training? What would it take to get a 65B model out of eleuther?
- stellaathena 4y agoA cr q
- stellaathena 4y agoOur policy is to not comment on timelines for future models, as our ability to meet those timelines is heavily influenced by factors outside of our control and we don’t want to lead people on.
- jackblemming 4y agoThey're the real OpenAI! Great team.
- behnamoh 4y agoIf only they had chosen a better name... "Eleuther" is hard to pronounce correctly for someone who doesn't know the word.
- ftxbro 4y agohow do you pronounce it correctly
- danuker 4y agoI couldn't find the IPA even in Wiktionary, but I found out where it comes from: https://en.wikipedia.org/wiki/Eleutheromania https://en.wikipedia.org/wiki/Eleutheromania
- stellaathena 4y agoActually the origin is this word (with the last two letters transposed), and the Wikipedia article gives IPA https://en.wikipedia.org/wiki/Eleutheria https://en.wikipedia.org/wiki/Eleutheria
- spapas82 4y agoIf it's from the greek word for freedom (ελευθερία) then i guess elefther: ele (like electric) f the (like thesis) r.
- akavi 4y agoWhere are you getting an /f/ sound from? Edit: Ah, it seems the digraph "ευ" is pronounced /ef/ in Modern Greek, despite (by convention) being transliterated "eu".
- victorbjorklund 4y agoAwesome. Remember your great work in GTP-Neo
- whitten 4y agoWhat was special about GPT-nano ?
- v64 4y agonot nano, Neo, and its successor NeoX, one of the largest public GPT models available https://blog.eleuther.ai/announcing-20b/ https://blog.eleuther.ai/announcing-20b/
- victorbjorklund 4y agoVery big LLM. Was almost as good as GTP-3. And it was open source.
- return_to_monke 4y agosorry for killing the mood. great article and great progress. but. didn't openai start as a nonprofit, too?
- loa_in_ 4y agoOpenAI has some complicated setup about this. Part of it is non-profit and part of it isn't and there's some rules they laid out on how to keep it non profit first from what I've gathered
- infinityio 4y agoto my knowledge - started 100% non-profit, the split into partial profit came later when they started having a chance to make money
- dvt 4y agoNot sure why the AI community has a weird obsession with being non-profit (or not). It's not like non-profits can't be evil, and it's not like OpenAI has done anything particularly egregious, either. What we do need is more weights being released in the public domain (hard to find even on Huggingface), easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts, and so on. My big gripe, and for obvious reasons, is that we need to step away from cloud-based inference, and it doesn't seem like anyone's working on that.
- vikp 4y agoIf you want more open research and weights, you should be happy about this announcement. Incorporating as a nonprofit doesn't guarantee that an organization will act ethically, but it does make it more likely. Nonprofits have more restrictions around how they can spend their money. They also pay less, so the teams is more likely to be mission-aligned. The profit motive pushes organizations to keep their work secret (which is what has happened with OpenAI).
- fl0ps 4y agoI'm with you there on hoping for more concentration on local training and inference, especially on something like the new Orin (open to donations, NVidia!). I think there's still something of a financial disincentive to promote selfhost over cloud capabilities for just about every party involved except the selfhoster. NVidia loses out if they're selling a few Nano or Orin platforms relative to the much pricier datacenter cards sold in huge lots. The cloud hosters are making up for the cost of hosting in terms of end-user pricing, snarfing all that lovely customer data and likely selling analytics, and probably other measures I can't understand yet. And the large companies that fund research and initial model development want to know what's being done with their models so as to gain any possible competitive advantage. They can't necesarily guarentee that intel from a self-hoster. Almost nobody is willing to spend the time explicitly to make it easier for the individual dev at a small lab or at home to do this because that's essentially a donation and not a business expense that might yeild obvious returns.
- warent 4y ago
- hinkley 4y agoNow I have Lenny Kravitz songs stuck in my head.
- whitten 4y agoAs a three letter acronym (TLA) aficionado, the page uses TPU wwhich isn’t explained.
- inconceivable 4y agotensor processing unit. a processor designed to do large, fast matrix operations.
- archgoon 4y agoTPU - Tensor Processing Unit. They're essentially Google GPU's.
- quartzbox 4y agoEleuther has been at the forefront of AI, without them none of this would be possible ,thanks and kudos
- rahidz 4y agoI hope Eleuther will continue to focus on the development of truly open-source models, which seems to be where the big gap is. The quality gap between Neo-20X and even the original GPT3 is still rather large, and making headway there would be amazing.
- riku_iki 4y agothere will always be gap, since corps can grab open model, and add resources to improvements on top of it.
- Taek 4y agoThis is not the case with Stable Diffusion. The open source community is moving faster and innovating faster, and the open models are by far the best. Midjourney is a contender, but it doesn't hold a candle to the best open models
- wokwokwok 4y agoLet’s be honest though; the number of people willing to devote their time to tireless effort generating porn (or suggestive images) is >> the number who will ever be willing to do it for text models. Add the size barrier (an order of magnitude > SD for text models) and it’s pretty questionable the open source text models ecosystem is going to go anywhere. Hardly anyone has done anything interesting with any of the existing ones and they’ve been out for longer than SD has. Text models just aren’t that interesting to people.
- Taek 4y ago9 days later and its now possible to run a 65B parameter LLM on a 64GB macbook pro, significantly outperforming GPT-3, OpenAssistant has announced that they got more than 100,000 samples of RLHF to replicate the 'Chat' component, Kobold Horde now supports Llama-33B for anyone to try in a playground with no censorship, and at least 4 other independent projects are spun up to develop at-home LLM tooling. People want good text models at home.
- valzam 4y agoEasy to become a non-profit if you've never made any money...
- kobalsky 4y agobold claim to make on a site linked to growing companies by burning money.
- bilsbie 4y agoI followed along on their discord for a while but I wasn’t sure how to gradually increase my engagement to get involved.
- victor9000 4y agoI really appreciate how the work of EAI has truly focused on making models accessible to the general public.
- supermatt 4y agoWhile I appreciate what Eleuther have done - why are they highlighting "research" consisting of removing a bit of whitespace from some source code... is it April 1st already? Heres my addition! Or subtraction, i guess... (-4 chars by changing call to AdaptiveAvgPool2d): from torch.nn import* def c(h,d,k,p,n):S,C,A=Sequential,Conv2d,lambda x:S(x,GELU(),BatchNorm2d(h));R=type('',(S,),{'forward':lambda s,x:s0+x});return S(A(C(3,h,p,p)),*[S(R(A(C(h,h,k,1,k//2,1,h))),A(C(h,h,1)))for _ in[0]*d],AdaptiveAvgPool2d(1),Flatten(),Linear(h,n))
- deleted 4y ago[deleted]
- jamilton 4y agoJust having a bit of fun.