7 ms·
Weird A.I. Yankovic: a cursed deep dive into the world of voice cloning
- simonw 3y agoI did not know about this: "The center of the A.I. cover songs community is a massive 500,000+ member Discord called A.I. Hub, where members trade new tips, tools, techniques, and links to their original and cover songs."
- codetrotter 3y agoMe neither. That’s what’s so weird about the internet. Imagine half a million people out in the streets together. You’d definitely notice that. Meanwhile, we can have these massive online communities and you’d never know unless you accidentally stumbled across it or someone told you about it.
- evan_ 3y agomore accurate to say that, while 500,000 people joined the discord by clicking a link, some much, much smaller number are actually active on any sort of a regular basis
- dylan604 3y agothis sounds like the description of most "new" social platforms. we see immediate interest, and then a sudden loss of that interest
- LordDragonfang 3y agoYeah, one of the "worst" (good for metrics, bad for legibility) parts of the trend of moving to discord for any sort of online community is that you have to "join" the community to even view any of the resources ensconced within. Meaning it's poorly indexed (discord search is okay, but not great) and not available at all to external crawlers.
- throwaway290 3y agoIf this community was available for crawling then LLM would crawl it and there would be no value in participating in the community because you can just ask the LLM about all that, no?
- LordDragonfang 3y agoIf the value your community provides is low enough that it can be effectively replaced by a general purpose LLM, then it should be. The value of a community should be pushing the boundaries of knowledge, not gatekeeping it. C'mon, this is hacker news, what happened to "information should be free"?
- deleted 3y ago[deleted]
- throwaway290 3y ago"Information should be free" doesn't work when you have Microsoft inserting itself as a middleman of information. The community is not gatekeeping knowledge, anyone can join. It merely tries to keep certain corporations out...
- BeFlatXIII 3y ago> C'mon, this is hacker news, what happened to "information should be free"? We've had an infestation of "pay me or I won't share" types.
- throwaway290 3y agoMore like an infestation of "open = I must be able to exploit and abuse for profit" entitled SV VC types.
- thomastjeffery 3y agoSo to continue the analogy comparison, 500,000 people walked in that street at some point. Some unknown percentage of that number is made of unrecognized duplicates (same person new username).
- lmm 3y ago> Imagine half a million people out in the streets together. You’d definitely notice that. In the streets, sure. Meeting up at out of town conference centers a few times a year, probably not. Most real communities have always been "dark matter" to those outside them; Discord working the same way feels more authentic than most of the internet.
- jrm4 3y agoI poked around there for a while, and my takeaway was "sub-par" all around, which might be the reason for it's relative obscurity? The thing is, I can't tell to what extent it's the tech, and to what extent it's just "very uninteresting source material." Like, there's a whole lot of "classic song done by presently popular rapper," and I'll be the first to insist that there is nearly nothing vocally interesting at all coming from todays popular hip-hop artists (and I say this as an extreme long-time hip-hop aficionado)
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- joenot443 3y agoSomething I think we're slowly coming to terms with is that the current generation of techies (the ones who can afford to spend hours upon hours tweaking models and sharing results) really prefer Discord over our Web 2.0 forum type communities like this one. Even reddit on, which is lagging in popularity amongst Gen-Z when compared to Discord or TikTok, you can immediately tell upon reading /r/LocalLLMs that a really big chunk of this community are underaged. To be clear, I think this is a good thing! There was a generation that preferred mailing lists. There was a generation that preferred IRC and BBS, and "my" generation which likes forums and lengthy comment threads. One would be naiive to think this style (the one we're engaging in here) would last forever. There are definitely very real criticisms of Discord, searchability and discoverability being the most common, but at this point I think the die has been cast. Young people have made their choice.
- BandButcher 3y agoAgree, im in my early 30s and jump through most platforms, but very little with tiktok/discord. but i have to admit a lot of newer content (and tech framework support) has migrated to discord channels. Even some YouTube sports talk shows have their own discord for call ins, etc... These big teleconference apps are usually hit or miss but discord seems to be the winner currently for actual "social networking", also add in its trend in the gaming community
- ThrowawayTestr 3y agoAre we so out of touch? No, it's the children who are wrong.
- tavavex 3y agoI kind of disagree? I am gen Z myself, and have used reddit extensively. While I like Discord a lot, I strongly disagree with using it to host content, essentially gating non-members from getting what they want (which is what leads to these communities with ludicrously inflated member counts). And this sentiment definitely isn't just me, a lot of the techie "CS major" people I know lean towards using slightly older services - which is also probably why the aforementioned /r/localllama community still has more than 60 thousand members. That being said, Discord does have some advantages over older forum-type communities - it's usually way better for cultivating smaller communities, and its no-effort-required chat systems means that you can always hop on and discuss things that are on the cutting edge. This is quite important in a field like AI, where it feels like something revolutionary happens every other week. (Also, I don't know if that implication was intentional, but gen Z and "underaged" haven't meant the same thing for many years now)
- mecredis 3y agoIt's kind of wild that these tools just transfer a copy of these models every time they're spun up (whether it's to a Google Colab notebook or a local machine.) This must mean Hugging Face's bandwidth bill must be crazy, or am I missing something (maybe they have a peering agreement? heavily caching things?)
- satertek 3y agoTheir Python module caches the downloads, which is checked before downloading them again...but you're probably not wrong on the crazy bandwidth bill. Looks like they have crazy VC money though, considering the current climate.
- minimaxir 3y agoThe Colab notebooks are a fresh and independent session with no caching.
- infinityio 3y agoGoogle might cache further up the chain, which could help
- civilitty 3y agoUnmetered 10+ gigabit connections were on the order of $1/mbit/mo wholesale over a decade ago when I priced out a custom CDN so for the cost of 100 TB of data transfer out of AWS you could get a 24/7 sustained 10gbit/s (>3 PB per month at 100% utilization). Bandwidth has always been crazy cheap.
- colechristensen 3y agoIndeed. If you're not using a cloud provider bandwidth is extremely cheap. In fact locally I can get a 10 gbps home internet unmetered connection for $300/mo. I'm not sure how they'd react if I transferred 1 PB/mo though :)
- mckirk 3y agoMy absolute favorite application of this tech so far is The Beach Boys singing 'Hurt'. It's the first time I seriously didn't notice any artifacts, and it just works so well even though it really shouldn't. Enjoy: https://youtu.be/gmNSFqyg_Z8 https://youtu.be/gmNSFqyg_Z8
- dwringer 3y agoI don't know what I was expecting but that isn't Hurt, it's Surfin' USA with Hurt's lyrics that sound extremely jittery and grainy. I'm curious though if some AI soon could in fact synthesize the Beach Boys' style with the actual chords and melody from the NIN song, possibly with some of the pathos of Johnny Cash as well.
- sumtechguy 3y agoThe one I found fun was the matrix ice ice baby mashup. That was sort of janky but good enough to be fun.
- darkerside 3y agoYeah, I hate it to the point of being personally offended. It has nothing to do with Johnny Cash's rendition. I'd probably feel a bit better, but not much, if it were advertised as a NIN mashup.
- cm2012 3y agoIt's definitely in the realm of "soulless"
- mock-possum 3y agoYeah that’s kind of the theme of the YouTube channel - I think it’s hilarious honestly, but maybe you have to go into it knowing what to expect.
- darkerside 3y agoYeah, based on the parent, and the genius of the musicians involved, I was expecting something more than the sum of its parts. Hurt is an incredibly powerful song, and the Cash rendition imbues it with another beautiful layer. As a joke, I can see it being funny, but it was a jarring way to experience it.
- causi 3y agoAI song covers are incredible, from Goku singing "Don't Stop Me Now" to the cast of Spongebob singing "Ocean Man".
- ssalka 3y agoMy favorite is the Mr. Krabs cover of "Billie Jean" https://www.youtube.com/watch?v=CkQ-44PvTs8 https://www.youtube.com/watch?v=CkQ-44PvTs8
- civilitty 3y agoMr Krabs rapping Lose Yourself by Eminem [1] is all the evidence I've ever needed that Clancy Brown should have been a rapper. [1] https://www.youtube.com/watch?v=d7N6jOziN4E https://www.youtube.com/watch?v=d7N6jOziN4E
- tetris11 3y agoThose high notes really kind of overflow to baritone there
- all2 3y agoThis is actually good. Hysterically so.
- lostlogin 3y agohttps://m.youtube.com/watch?v=XzqbhDqAEtw https://m.youtube.com/watch?v=XzqbhDqAEtw
- cm2012 3y agoWould have strongly preferred DBZA goku :)
- matheusmoreira 3y ago... Muffin button.
- satvikpendem 3y agoWhat's the best open source text to speech? Eleven Labs and others are interesting but closed source. I want to use them mainly for audiobooks as I have a lot of ePubs and I'm just using the basic Google text to speech voices on my Android, via Moon+ Reader. It works fine but it's still more robotic than state of the art.
- artninja1988 3y agoWould also like to know this. Can't seem to find an open source tts engine that works on mobile to read muh books
- modeless 3y agoI've tried a few, not an expert, but I think Coqui's new XTTS models are decent performance and quality wise (just in terms of how the speech sounds, can't speak to the voice cloning fidelity as I don't care about that). Open source code but non-commercial license for the model. They also have a bunch of models with more permissive licenses that aren't as good. I doubt they're better than Google's TTS though.
- lhl 3y agoFor neutral sounding very fast/efficient voices, I find Coqui TTS VITS models to be very good. For slower, more expressive voice or voice cloning I think the Coqui TTS XTTS is good (or you can look at the mrq/tortoise-tts). I'm still awaiting a StyleTTS2 implementation. The audio samples sound top notch: https://styletts2.github.io/ https://styletts2.github.io/
- modeless 3y agoYou're in luck, the code dropped 6 hours ago :) https://github.com/yl4579/StyleTTS2 https://github.com/yl4579/StyleTTS2 Looks promising, I'm going to check it out too! MIT license, even! If it's fast enough for real time, it could be the new best option. The paper claims faster inference than VITS...
- lhl 3y ago
- minimaxir 3y agoThis article only covers the musical aspects of AI voice cloning, but there's another dynamic to AI voice cloning that's more complicated: replacing general voice actors in movies/video games/anime (example: https://www.axios.com/2023/07/24/ai-voice-actors-victoria-atkin-assassins-creed https://www.axios.com/2023/07/24/ai-voice-actors-victoria-at... ) Unlike musicians who can't be replaced without significant postprocessing, have enough money to not be impacted by competition, and have legal muscle, voice over artists: - Can be reproduced with good-enough results from out-of-the-box voice cloning settings on ElevenLabs or an open source equivalent (Bark, VALL-E X) - Are already underpaid for their work as-is - Have no legal ownership of their voice since they are contractors, and their voicework is owned by their clients who may not be as incentivised in protecting the VO. I want to write a blog post about it but I suspect most people on Hacker News won't be interested in a treatise on the cultural impacts of the voicework in Persona 5 and Genshin Impact.
- aaroninsf 3y ago<raises hand> I am
- supriyo-biswas 3y agoHN isn't the only community to write for. While most people here seem to be unsympathetic to such job concerns, unconventional articles do hit the front page from time to time. I'd like to read it, in any case.
- distantsounds 3y agoThe sampled voices sound neither like Michael Jackson nor Weird Al. A good effort, but a professional impersonator could likely do better on either front.
- nemo44x 3y agoIt sounds like Weird Al trying to be Michael Jackson trying to be Weird Al.
- Reventlov 3y agoAs a non native speaker, it does sound a bit like Michael Jackson imo…
- hinkley 3y agoSometimes I’ll watch a movie with voiceover work, where some character has a very specific accent, and I’ll be watching along for twenty minutes and the VA will let slip just a couple syllables of their real voice and my ears will prick up and I’ll think, hey I know this guy. Isn’t that… oh the guy from the thing. From <wrong movie>, no wait I mean <other movie>? Yes, it is. That’s what this sounds like. Five syllables of Michael Jackson while he’s trying to be Action Hero or Big Villain, or Funny Sidekick (a problem Eddie Murphy has never had, all evidence from Coming to America notwithstanding).
- hinkley 3y agoThe best Michael Jackson interpreter in a town of 50,000 could do better than this. It’s… this is bad.
- code_runner 3y agoI know what you mean. Its more noticeable (imo) on the Michael one.... but its definitely in there. I think the pitch correction is to blame for a bit of the weirdness.
- hinkley 3y ago> Artifacts aside, it sounds like Michael Jackson doing a Weird Al impression?! Every line has a distinctly “white and nerdy” vibe: it loses any seriousness and edge, exaggerating words for comic effect and enunciating lyrics really clearly so the punchlines can be heard. No, it sounds like someone doing doing an impression of Weird Al doing an impression of Michael Jackson. Someone whose mom told them they were special and they believed it. These examples are standing on a ridge line, surveying the uncanny valley and looking for the best way to cross.
- deleted 3y ago[deleted]
- blagie 3y ago... they're good enough. I have an accent. If not for that, I'd be a great presenter. If I could translate my voice into a poor Neil deGrasse Tyson, a poor Patrick Steward, a poor Carl Sagan, a poor Morgan Freeman, etc., my presentations would be... better.
- totetsu 3y agoFinally a way to not have to fix societies Prejudices just give everybody the tools to emulate the ideal of perfection no matter what color their skin or what their accent sounds like.
- hinkley 3y agoIf it makes you more comfortable and confident, that is helping you. This isn't autotune for the spoken word, though. It's not fixing pacing or vocabulary, and in the audio above it isn't even fixing intonation. Yes, a thick German accent will give you away as being of German extraction. But you're also using the word 'since' when Brits and Americans would use 'for', and it's not going to fix that. Any more than it'll fix my french when I make the exact same mistake going the other direction (for=duration vs for=purpose vs for=interval). If I hear 'since one month' you're likely German or Indian. If you ask how long I've been in Marseille you'll know I'm American in about half that time.
- smath 3y agoRelated article from 1 year ago on Darth Vader’s voice being AI generated going forward: https://arstechnica.com/information-technology/2022/09/james-earl-jones-signed-darth-vader-voice-rights-to-disney-for-ai-use/amp/ https://arstechnica.com/information-technology/2022/09/james...
- Calamitous 3y agoKey takeaway: > No current artificial intelligence is powerful enough to hide the weirdness of Weird Al.
- deleted 3y ago[deleted]
- mito88 3y ago"celebrity voices impersonated" Watch Light My Fire on YouTube Music https://music.youtube.com/watch?v=lN3v3EfA6_A&si=_hcG3WjakxdagLYf https://music.youtube.com/watch?v=lN3v3EfA6_A&si=_hcG3Wjakxd...
- dreamcompiler 3y ago> ... Tom Waits, LeBron James, Knuckles, and, uh, Adolf Hitler. I can't figure out if this is an example of Godwin's Law or not.
- deleted 3y ago[deleted]
- ddmf 3y agoThe most recent episode of Tacoma FD covered something similar to this mixed with a messed up Christmas Carol.
- RecycledEle 3y agoWow. I just realized any one of us could redo Weird Al's songs with his lyrics, but with the original singer's voice. We could be listening to Michael Jackson singing "Just Eat It" by lunchtime. I am constantly amazed at how the new AI tech can be used. Of course this would be illegal under most countries copyright laws.
- RecycledEle 3y agoAnd...they already did it.
- unnah 3y agoThere's also a Weird Al piece "I think I'm a clone now", for which an AI clone voice performance would definitely be fitting. (The original song was "I think we're alone now" by Tommy James and the Shondells, but it seems Weird Al was parodying the cover by Tiffany in the 1980's.) While Weird Al himself asks for permission, it's well established that parody is not copyright infringement. There should be room for parody performances by AI voices as well, especially if argued by a good lawyer.
- mbg721 3y agoAl is very self-aware (that second character is a lower-case ell), he's less concerned with legal entities than with his relationships with musicians.
- greenhearth 3y agoHow would this be amazing? It just sounds stupid and a waste of time.