8 ms·
This is leaps and bounds beyond anything out there, including both public models like SVD 1.1 and Pika Labs' / Runway's models. Incredible.
by treesciencebot 3y ago
This is leaps and bounds beyond anything out there, including both public models like SVD 1.1 and Pika Labs' / Runway's models. Incredible.
- jasonjmcghee 3y agoI agree in terms of raw generation, but runway especially is creating fantastic tooling too.
- Zelphyr 3y agoAgreed. It's amazing how much of a head start OpenAI appears to have over everyone else. Even Microsoft who has access to everything OpenAI is doing. Only Microsoft could be given the keys to the kingdom and still not figure out how to open any doors with them.
- SeanAnderson 3y agoEh. MSFT owns 49% of OpenAI. Doesn't really seem like they need to do much except support them.
- Zelphyr 3y agoExcept they keep trying to shove AI into everything they own. CoPilot Studio is an example of how laughably bad at it they are. I honestly don't understand why they don't contract out to OpenAI to help them do some of these integrations.
- SeanAnderson 3y agoEvery company is trying to shove AI into everything they own. It's what investors currently demand. OpenAI is likely limited by how fast they are able to scale their hiring. They had 778 FTEs when all the board drama occurred, up 100% YoY. Microsoft has 221,000. It seems difficult to delegate enough headcount to all the exploratory projects of MSFT and it's hard to scale headcount quicker while preserving some semblance of culture.
- frabcus 3y agoThey don't own 49% of OpenAI. They have capped rights to 49% of OpenAI's profits.
- SeanAnderson 3y agoApparently all the rumors weren't true then, my mistake. I don't think what you're saying is correct though, either. All the early news outlets reported 49% ownership: https://en.wikipedia.org/wiki/OpenAI#:~:text=Rumors%20of%20this%20deal%20suggested%20that%20Microsoft%20may%20receive%2075%25%20of%20OpenAI%27s%20profits%20until%20it%20secures%20its%20investment%20return%20and%20a%2049%25%20stake%20in%20the%20company.%5B https://en.wikipedia.org/wiki/OpenAI#:~:text=Rumors%20of%20t... https://www.theverge.com/2023/1/23/23567448/microsoft-openai-partnership-extension-ai https://www.theverge.com/2023/1/23/23567448/microsoft-openai... https://www.reuters.com/world/uk/uk-antitrust-regulator-considering-microsoft-openai-partnership-2023-12-08/ https://www.reuters.com/world/uk/uk-antitrust-regulator-cons... https://techcrunch.com/2023/01/23/microsoft-invests-billions-more-dollars-in-openai-extends-partnership/?guccounter=1 https://techcrunch.com/2023/01/23/microsoft-invests-billions... The only official statement from Micorosft is "While details of our agreement remain confidential, it is important to note that Microsoft does not own any portion of OpenAI and is simply entitled to share of profit distributions," said company spokesman Frank Shaw. No numbers, though. Do you have a better source for numbers?
- sailingparrot 3y agoMicrosoft doesn’t have access to OpenAI’s research, this was part of the deal. They only have access to the weights and inference code of production models and even then who has access to that inside MS is extremely gated and only a few employees have access to this based on absolute need to actually run the service. AI researcher at MSFT barely have more insights about OpenAI than you do reading HN.
- Zelphyr 3y agoI didn't realize that. Thank you for the clarification.
- toneyG 3y agoThis is not true. Microsoft have a perpetual license to all of OpenAI's IP. If they really wanted to they could get their hands on it.
- 93po 3y agoYeah but what's in the license? It's not public so we have no way of knowing
- Jensson 3y agoMicrosoft said that they could continue OpenAI's research with no slowdown if OpenAI cut them off by hiring all OpenAI's people, so from that statement it sounds like they have access.
- costcofries 3y agoI promise you this isn't true.
- vitorgrs 3y agoNo. They have early access. Example: MSFT was using Dall-e Exp (early 3 version) in PUBLIC, since February of 2023. In the same month, they were also using GPT4 in public - before OpenAI. And they had access to GPT4 in 2022 (which was when they decided to create Bing Chat, now called Copilot). All the current GPT4 models at MSFT are also finetuned versions (literally Creative and Precise mode runs different finetuned versions of GPT4). It runs finetuned versions since launch even...
- DANI-HACKER 3y ago[flagged]
- pcbro141 3y agoMany people say the same about Google/DeepMind.
- davidbarker 3y agoI'm almost speechless. I've been keeping an eye on the text-to-video models, and if these example videos are truly indicative of the model, this is an order of magnitude better than anything currently available. In particular, looking at the video titled "Borneo wildlife on the Kinabatangan River" (number 7 in the third group), the accurate parallax of the tree stood out to me. I'm so curious to learn how this is working. [Direct link to the video: https://player.vimeo.com/video/913130937?h=469b1c8a45 https://player.vimeo.com/video/913130937?h=469b1c8a45]
- calgoo 3y agoThe video of the gold rush town just makes me think of what games like Red Dead and GTA could look like.
- 93po 3y agoholy cow, is that the future of gaming? instead of 3D renders it's real-time video generation, complete with audio and music and dialog and intelligent AI conversations and it's a unique experience no one else has ever played. gameplay mechanics could even change on the fly
- gdubs 3y agoThat's why NVIDIA's CEO said recently that in the future every pixel will be generated — not rendered.
- Keyframe 3y agofive years ago: https://www.youtube.com/watch?v=ayPqjPekn7g https://www.youtube.com/watch?v=ayPqjPekn7g I'm eager to see an updated version.
- 7734128 3y agoDLSS is essentially this, isn't it? It uses a low quality render from the game and then increases the fidelity with something very similar to a diffusion model.
- mizzao 3y agoMust be intimidating to be on the Pika team at the moment...
- alokjnv10 3y agoyou nailed it
- jsheard 3y agoWhere is the training material for this coming from? The only resource I can think of that's broad enough for a general purpose video model is YouTube, but I can't imagine Google would allow a third party to scrape all of YT without putting up a fight.
- Zetobal 3y agoIt's movies the shots are way to deliberate to have random YouTube crap in the dataset.
- cma 3y agoYou can still have a broad dataset and use RLHF to steer it more towards the aesthetic like midjourney and SDXL did through discord feedback. I think there was still some aesthetic selection in the dataset as well but it still included a lot of crap.
- xnx 3y agoIt's very good. Unclear how far ahead of Lumiere it is (https://lumiere-video.github.io/ https://lumiere-video.github.io/) or if its more of a difference in prompting/setttings.
- ehsankia 3y agoIn terms of following the prompt and generating visually interesting results, I think they're comparable. But the resolution for Sora seems so far ahead. Worth noting that Google also has Phenaki [0] and VideoPoet [1] and Imagen Video [2] [0] https://sites.research.google/phenaki/ https://sites.research.google/phenaki/ [1] https://sites.research.google/videopoet/ https://sites.research.google/videopoet/ [2] https://imagen.research.google/video/ https://imagen.research.google/video/
- vunderba 3y agoThe big stand out to me beyond almost any other text video solution is that the video duration is tremendously longer (minute+). Everything else that I've seen can't get beyond 15 to 20 seconds at the absolute maximum.
- drdaeman 3y agoLet's hold our breath. Those are specifically crafted hand-picked good videos, where there wasn't any requirement but "write a generic prompt and pick something that looks good", with no particular requirements. Which is very different from the actual process where you have a very specific idea and want the machine to make it happen. DALL-E presentation also looked cool and everyone was stoked about it. Now that we know of its limitations and oddities? YMMV, but I'd say not so much - Stable Diffusion is still the go-to solution. I strongly suspect the same thing with Sora.
- treesciencebot 3y agoThe examples are most certainly cherry-picked. But the problem is there are 50 of them. And even if you gave me 24 hour full access to SVD1.1/Pika/Runway (anything out there that I can use), I won't be able to get 5 examples that match these in quality (~temporal consistency/motions/prompt following) and more importantly in the length. Maybe I am overly optimistic, but this seems too good.
- xnx 3y agoCredit to OpenAI for including some videos with failures (extra limbs, etc.). I also wonder how closely any of these videos might match one from the training set. Maybe they chose prompts that lined up pretty closely with a few videos that were already in there.
- htrp 3y agohttps://twitter.com/sama/status/1758200420344955288 https://twitter.com/sama/status/1758200420344955288 They're literally taking requests and doing them in 15 minutes.
- iLoveOncall 3y agoIt is incredible indeed, but I remember there was a humongous gap between the demoed pictures for DALL-E and what most prompts would generate. Don't get overly excited until you can actually use the technology.
- sschueller 3y agoYes, but I am stuck in their (American) view of what is consider appropriate. Not what is legal, but what they determine to be OK to produce. Good luck generating anything similar to an 80s action movie. The violence and light nudity will prevent you from generating anything.
- Xirgil 3y agoI suspect it's less about being puritanical about violence and nudity in and of themself, and more a blanket ban to make up for the inability to prevent the generation of actually controversial material (nude images of pop stars, violence against politicians, hate speech)
- wilg 3y agoNo, it's America's fault.
- SamBam 3y agoPut like that, it's a bit like the Chumra in Judaism [1]. The fence, or moat, around the law that extends even further than the law itself, to prevent you from accidentally commiting a sin. 1. https://en.m.wikipedia.org/wiki/Chumra_(Judaism) https://en.m.wikipedia.org/wiki/Chumra_(Judaism)
- UberFly 3y agoNa. It's more like what he said: Cover your ass legally for the real problems this could cause.
- TulliusCicero 3y agoIt's not a particularly American attitude to be opposed to violence in media though, American media has plenty of violence. They're trying to be all-around uncontroversial.
- throwitaway222 3y agoI am guessing a movie studio will get different access with controls dropped. Of course, that does mean they need to be VERY careful when editing, and making sure not to release a vagina that appears for 1 or 2 frames when a woman is picking up a cat in some random scene.
- jug 3y agoYup, it's been even several months! ;) But now we finally have another quantum leap in AI.
- deleted 3y ago[deleted]
- JKCalhoun 3y agoI know it's Runway (and has all manner of those dream-like AI artifacts) but I like what this person is doing with just a bunch 4 second clips and an awesome soundtrack: https://youtu.be/JClloSKh_dk https://youtu.be/JClloSKh_dk https://youtu.be/upCyXbTWKvQ https://youtu.be/upCyXbTWKvQ
- rvz 3y agoAll those startups have been squeezed in the middle. Pika, Runway, etc might as well open source their models. Or Meta will do it for them.