9 ms·

Show HN: A singing synthesizer for the browser with automatic 3-part harmony

208 points by zebproj 2y ago

ldenoue 2y ago
This thing is amazing can you add recording? Perhaps some samples you or visitors create? Then add a little sampler for beat and it’s a fantastic tool
_nivlac_ 2y ago
I've also enjoyed Google's blob opera, for something similar! https://artsandculture.google.com/experiment/blob-opera/AAHWrq360NcGbw?hl=en&cp=e30 https://artsandculture.google.com/experiment/blob-opera/AAHW...
- modeless 2y ago
  And the original singing mouths made by the same guy: https://www.adultswim.com/etcetera/choir/ https://www.adultswim.com/etcetera/choir/
  - jcmeyrignac 2y ago
    More experiments here: http://david.li/ http://david.li/
- Bluestein 2y ago
  Not to nitpick, but, it being "Arts & Culture" and all, they could've at least said opera aria ... ... not song.-
  - chpatrick 2y ago
    Not to nitpick but an aria is for one voice.
    - Bluestein 2y ago
      Point.- PS. The rest are ... what, the choir? A concertante if your fingers are fast enough, I guess :)
- smusamashah 2y ago
  Also pink trombone https://dood.al/pinktrombone/ https://dood.al/pinktrombone/
  - bravura 2y ago
    For years, I've wished someone would port Pink Trombone to a VST or something I could control through code.
    - whilenot-dev 2y ago
      Only thing I could find is a Eurorack module: https://blog.bela.io/salt-a-bela-powered-programmable-synth-module/#the-pink-trombone https://blog.bela.io/salt-a-bela-powered-programmable-synth-... I thought there was some repo public with C++ code, but I can't find it anymore... (https://github.com/belaPlatform https://github.com/belaPlatform) EDIT: stupid me, it's linked in the article: https://github.com/giuliomoro/pink-trombone https://github.com/giuliomoro/pink-trombone
deleted 2y ago
[deleted]
blaufast 2y ago
Neat!
brunorsini 2y ago
Fun! I'd love the option to increase (double?) range. It would sacrifice precision somewhat, so a toggle would be ideal. A toggle for quickly visualizing notes would also be useful, as well as keyboard shortcuts — perhaps while one clicks the mouse on a given note, pressing the keyboard shortcut for another specific note gradually moves pitch in that direction, with programmable speed.
- brunorsini 2y ago
  Oh, and clicking multiple times at the leftmost portion of the instrument (around 1/4th from the top) sounds exactly like Laurie Anderson's classic "O Superman" :)
varunneal 2y ago
so beautiful. I can imagine using this a lot if some buttons are added (start/stop harmony, view note/resonance, etc). great job
ycombiredd 2y ago
in Chrome, on my iphone, I can get a cool effect that looks like a glitch, but I wonder if it is a feature. If I hold a note a certain way, where the browser does a “select all”, and then drag my thumb, it drags a thumbnail of the page. When I release it, it keeps the first note and starts a new one using the new position of my thumb. I can create my own harmonies this way instead of the default parts that appear when I hold the note in place.
fhackenberger 2y ago
This is such a great way to practice! Thank you for creating it! Could you perhaps add pitch indicators on the side and multi touch support? Or maybe it's open source?
butz 2y ago
Add "display:block;" to canvas to fix the issue with vertical scroll appearing. Also, you might want to consider looking into vh and vw units to fill canvas to viewport without using javascript.
- skykooler 2y ago
  Is there a reason to avoid using javascript for canvas layout considering you need javascript to draw on the canvas in the first place?
- zebproj 2y ago
  Thanks, I'll look into it
MarkusWandel 2y ago
Does anyone make a USB Midi Theremin? That would totally be the right front end for this one!
- vunderba 2y ago
  Moog does - I've combined it with a midi translator app to customize midi (CC, etc) so that I can use it as an expression input to SWAM physically modeled instruments like violin. https://www.moogmusic.com/products/etherwave-theremins/theremini https://www.moogmusic.com/products/etherwave-theremins/there...
kitshunt 2y ago
Source code is here: https://github.com/PaulBatchelor/Trio https://github.com/PaulBatchelor/Trio It's written in JavaScript and Rust (compiled to WASM).
smusamashah 2y ago
This is awesome fun. Reminds of Google Blog Opera and Pink Trombone. Can we have multiple voices when using multiple fingers please? Edit: I can't sing or play music but this is making me feel like I am making someone sing nicely, it's relatively lot easy to play with.
vunderba 2y ago
Very neat. Side note, but one of the more fun aspects of Garageband on the iPad is that you can load string instruments such as an Erhu and play it by physically sliding your finger along the string in a very similar way.
makapuf 2y ago
I'm sorry but I can see a small keyboard with selected notes, can some one describe what makes it 3 parts harmony ? Is it a song thing ? Some chord thing ? There is no description on the site or the github repo.
hunter2_ 2y ago
95% of the time, I can only get 1 voice at a time. 5% of the time, I can get 2 or 3 voices at a time (1 variable pitch, the others fixed pitch) but I have no idea what input causes this rare behavior. Can you add some help text explaining what various input sequences will achieve? Android Chrome with touch input (Pixel 8 Pro).
- zebproj 2y ago
  A tutorial would be helpful. Holding down a note and waiting will cause a second, then a third not to appear. When you move your held, note to another pitch, the other pitches will follow, but with a bit of delay. This produces what is known as staggered voice leading, and produces interesting "in-between" chords.
  - hunter2_ 2y ago
    Aha! In hindsight, I'm amazed that I managed to not figure this out on my own.
zebproj 2y ago
Thanks everyone for the suggestions and kind words. Some details: The source code for this project can be found on github [0]. I am using an AudioWorklet node with custom DSP using Rust/WebAssembly. Graphics are just done with the Canvas API. The voice leading is done algorithmically using a state machine with some heuristics. The underlying DSP algorithm is a physical model of the human voice, similar to the model you'd find in Pink Trombone [1], but with some added improvements. The DSP code for that is a small crate [2] I've been working on just for singing synthesizers based on previous work I've done. 0: https://github.com/paulBatchelor/trio https://github.com/paulBatchelor/trio 1: https://dood.al/pinktrombone/ https://dood.al/pinktrombone/ 2: https://github.com/PaulBatchelor/voxbox https://github.com/PaulBatchelor/voxbox
- atech4826 2y ago
  Dear zebproj, Apologies for the late comment, but I had a query I wanted to share. Would it be possible for you to create a tool that allows users to mimic human emotional sounds directly in the browser? I’m thinking of sounds like realistic coughs, sighs, gasps, and other vocal expressions like shouting or crying etc . It would be amazing if the tool could optionally incorporate TTS, but even without it, the functionality would be very valuable for content creators or people who need custom sound effects. The idea is to let users customize these sounds by adjusting parameters such as intensity, pitch, and duration. It could also include variations for emotional contexts, like a sad sigh, a relieved sigh, a startled gasp, or a soft cough. An intuitive interface with sliders and buttons to tweak and preview sounds in real-time would make it super user-friendly, with options to save or export the generated audio much like the project pinktrombone I’m quite new to this field and only have basic experience with HTML, CSS, and JavaScript. However, I am very much interested in this area and I was wondering if this is something that could be achieved using tools like CursorAI or similar AI-based solutions? Or better yet, is it possible for you to create something like this for people like me who aren’t very tech-savvy? Thank you so much
  - zebproj 2y ago
    Greetings, What a beautiful idea. Sadly, I do not think I currently have the skills required to build such a tool. The underlying algorithms and vocal models I'm using here are just good enough to get some singing vowels working. You'd need a far more complex model to simulate the turbulent airflow required for a cough. If you suspend disbelief and allow for more abstract sounds, I believe you can craft sounds that have similar emotional impact. A few years ago, I made some non-verbal goblin sounds [0] from very simple synthesizer components and some well-placed control curves. Even though they don't sound realistic, character definitely comes through. 0: https://pbat.ch/gestlings/goblins https://pbat.ch/gestlings/goblins
zellyn 2y ago
This is so fun. One note: on iOS Safari on my iPhone, it really believes I’m trying to select everything, do drag and drop, etc. I’m sure someone here knows the proper CSS incantation.
joshuaheard 2y ago
This is awesome! It would be great if you could record a snippet then layer your recordings.

208 points by zebproj 2y ago

This thing is amazing can you add recording? Perhaps some samples you or visitors create? Then add a little sampler for beat and it’s a fantastic tool

_nivlac_ 2y ago

I've also enjoyed Google's blob opera, for something similar! https://artsandculture.google.com/experiment/blob-opera/AAHWrq360NcGbw?hl=en&cp=e30 https://artsandculture.google.com/experiment/blob-opera/AAHW...

modeless 2y ago
And the original singing mouths made by the same guy: https://www.adultswim.com/etcetera/choir/ https://www.adultswim.com/etcetera/choir/
- jcmeyrignac 2y ago
  More experiments here: http://david.li/ http://david.li/
Bluestein 2y ago
Not to nitpick, but, it being "Arts & Culture" and all, they could've at least said opera aria ... ... not song.-
- chpatrick 2y ago
  Not to nitpick but an aria is for one voice.
  - Bluestein 2y ago
    Point.- PS. The rest are ... what, the choir? A concertante if your fingers are fast enough, I guess :)
smusamashah 2y ago
Also pink trombone https://dood.al/pinktrombone/ https://dood.al/pinktrombone/
- bravura 2y ago
  For years, I've wished someone would port Pink Trombone to a VST or something I could control through code.
  - whilenot-dev 2y ago
    Only thing I could find is a Eurorack module: https://blog.bela.io/salt-a-bela-powered-programmable-synth-module/#the-pink-trombone https://blog.bela.io/salt-a-bela-powered-programmable-synth-... I thought there was some repo public with C++ code, but I can't find it anymore... (https://github.com/belaPlatform https://github.com/belaPlatform) EDIT: stupid me, it's linked in the article: https://github.com/giuliomoro/pink-trombone https://github.com/giuliomoro/pink-trombone

deleted 2y ago

[deleted]

blaufast 2y ago

Neat!

brunorsini 2y ago

Fun! I'd love the option to increase (double?) range. It would sacrifice precision somewhat, so a toggle would be ideal. A toggle for quickly visualizing notes would also be useful, as well as keyboard shortcuts — perhaps while one clicks the mouse on a given note, pressing the keyboard shortcut for another specific note gradually moves pitch in that direction, with programmable speed.

brunorsini 2y ago
Oh, and clicking multiple times at the leftmost portion of the instrument (around 1/4th from the top) sounds exactly like Laurie Anderson's classic "O Superman" :)

varunneal 2y ago

so beautiful. I can imagine using this a lot if some buttons are added (start/stop harmony, view note/resonance, etc). great job

ycombiredd 2y ago

in Chrome, on my iphone, I can get a cool effect that looks like a glitch, but I wonder if it is a feature. If I hold a note a certain way, where the browser does a “select all”, and then drag my thumb, it drags a thumbnail of the page. When I release it, it keeps the first note and starts a new one using the new position of my thumb. I can create my own harmonies this way instead of the default parts that appear when I hold the note in place.

fhackenberger 2y ago

This is such a great way to practice! Thank you for creating it! Could you perhaps add pitch indicators on the side and multi touch support? Or maybe it's open source?

butz 2y ago

Add "display:block;" to canvas to fix the issue with vertical scroll appearing. Also, you might want to consider looking into vh and vw units to fill canvas to viewport without using javascript.

skykooler 2y ago
Is there a reason to avoid using javascript for canvas layout considering you need javascript to draw on the canvas in the first place?
zebproj 2y ago
Thanks, I'll look into it

MarkusWandel 2y ago

Does anyone make a USB Midi Theremin? That would totally be the right front end for this one!

vunderba 2y ago
Moog does - I've combined it with a midi translator app to customize midi (CC, etc) so that I can use it as an expression input to SWAM physically modeled instruments like violin. https://www.moogmusic.com/products/etherwave-theremins/theremini https://www.moogmusic.com/products/etherwave-theremins/there...

kitshunt 2y ago

Source code is here: https://github.com/PaulBatchelor/Trio https://github.com/PaulBatchelor/Trio It's written in JavaScript and Rust (compiled to WASM).

smusamashah 2y ago

This is awesome fun. Reminds of Google Blog Opera and Pink Trombone. Can we have multiple voices when using multiple fingers please? Edit: I can't sing or play music but this is making me feel like I am making someone sing nicely, it's relatively lot easy to play with.

vunderba 2y ago

Very neat. Side note, but one of the more fun aspects of Garageband on the iPad is that you can load string instruments such as an Erhu and play it by physically sliding your finger along the string in a very similar way.

makapuf 2y ago

I'm sorry but I can see a small keyboard with selected notes, can some one describe what makes it 3 parts harmony ? Is it a song thing ? Some chord thing ? There is no description on the site or the github repo.

hunter2_ 2y ago

95% of the time, I can only get 1 voice at a time. 5% of the time, I can get 2 or 3 voices at a time (1 variable pitch, the others fixed pitch) but I have no idea what input causes this rare behavior. Can you add some help text explaining what various input sequences will achieve? Android Chrome with touch input (Pixel 8 Pro).

zebproj 2y ago
A tutorial would be helpful. Holding down a note and waiting will cause a second, then a third not to appear. When you move your held, note to another pitch, the other pitches will follow, but with a bit of delay. This produces what is known as staggered voice leading, and produces interesting "in-between" chords.
- hunter2_ 2y ago
  Aha! In hindsight, I'm amazed that I managed to not figure this out on my own.

zebproj 2y ago

Thanks everyone for the suggestions and kind words. Some details: The source code for this project can be found on github [0]. I am using an AudioWorklet node with custom DSP using Rust/WebAssembly. Graphics are just done with the Canvas API. The voice leading is done algorithmically using a state machine with some heuristics. The underlying DSP algorithm is a physical model of the human voice, similar to the model you'd find in Pink Trombone [1], but with some added improvements. The DSP code for that is a small crate [2] I've been working on just for singing synthesizers based on previous work I've done. 0: https://github.com/paulBatchelor/trio https://github.com/paulBatchelor/trio 1: https://dood.al/pinktrombone/ https://dood.al/pinktrombone/ 2: https://github.com/PaulBatchelor/voxbox https://github.com/PaulBatchelor/voxbox

atech4826 2y ago
Dear zebproj, Apologies for the late comment, but I had a query I wanted to share. Would it be possible for you to create a tool that allows users to mimic human emotional sounds directly in the browser? I’m thinking of sounds like realistic coughs, sighs, gasps, and other vocal expressions like shouting or crying etc . It would be amazing if the tool could optionally incorporate TTS, but even without it, the functionality would be very valuable for content creators or people who need custom sound effects. The idea is to let users customize these sounds by adjusting parameters such as intensity, pitch, and duration. It could also include variations for emotional contexts, like a sad sigh, a relieved sigh, a startled gasp, or a soft cough. An intuitive interface with sliders and buttons to tweak and preview sounds in real-time would make it super user-friendly, with options to save or export the generated audio much like the project pinktrombone I’m quite new to this field and only have basic experience with HTML, CSS, and JavaScript. However, I am very much interested in this area and I was wondering if this is something that could be achieved using tools like CursorAI or similar AI-based solutions? Or better yet, is it possible for you to create something like this for people like me who aren’t very tech-savvy? Thank you so much
- zebproj 2y ago
  Greetings, What a beautiful idea. Sadly, I do not think I currently have the skills required to build such a tool. The underlying algorithms and vocal models I'm using here are just good enough to get some singing vowels working. You'd need a far more complex model to simulate the turbulent airflow required for a cough. If you suspend disbelief and allow for more abstract sounds, I believe you can craft sounds that have similar emotional impact. A few years ago, I made some non-verbal goblin sounds [0] from very simple synthesizer components and some well-placed control curves. Even though they don't sound realistic, character definitely comes through. 0: https://pbat.ch/gestlings/goblins https://pbat.ch/gestlings/goblins

zellyn 2y ago

This is so fun. One note: on iOS Safari on my iPhone, it really believes I’m trying to select everything, do drag and drop, etc. I’m sure someone here knows the proper CSS incantation.

joshuaheard 2y ago

This is awesome! It would be great if you could record a snippet then layer your recordings.