9 ms·
Correct me if I’m wrong, but this is a big bet on a company that just had their lunch eaten on another product by open source, no? Didn’t see much of Dall-E aft
by fintechjock 4y ago
Correct me if I’m wrong, but this is a big bet on a company that just had their lunch eaten on another product by open source, no? Didn’t see much of Dall-E after Stable Diffusion release
- ghilston 4y agoIt would be incredible if a chat gpt competitor came into the self host able, open source, niche like stable diffusion. The options I've seen today are no where near as good as chat gpt
- mustacheemperor 4y agoThere are open source / OS model-based projects competing with OpenAI’s text generation. At least one, NovelAI, also has a Stable Diffusion fork incorporated into their paid product. At least so far none of them are close to what’s been achieved with ChatGPT, and that could in itself be driving some of the hype. ChatGPT was probably more impressive to me on the first impression being familiar with the limitations of existing AI text gen projects.
- lossolo 4y agoI think the problem is time. To replicate the OpenAI RLHF architecture, they need their own high-quality dataset, which takes time to create. Without details about the hyperparameters, RL architecture, and omitted steps, they need to test a lot of things, which takes time and money. It requires more resources than it took SD to replicate DALL-E 2, which took months and was an easier task.
- visarga 4y agoCheck out Constitutional AI from Anthropic. They automated "RLHF" by simply writing a few rules.
- lossolo 4y agoThanks, just skimmed the paper, I think that "they automated RLHF" statement is maybe too strong here, there is still manual process but it seems like they optimized away a lot of manual labeling work.
- coolspot 4y agoThere is Open Assistant by LAION (WiP): https://github.com/LAION-AI/Open-Assistant https://github.com/LAION-AI/Open-Assistant