8 ms·
Show HN: Pi-C.A.R.D, a Raspberry Pi Voice Assistant
Pi-card is an AI powered voice assistant running locally on a Raspberry Pi. It is capable of doing anything a standard LLM (like ChatGPT) can do in a conversational setting. In addition, if there is a camera equipped, you can also ask Pi-card to take a photo, describe what it sees, and then ask questions about that.
It uses distributed models so latency is something I'm working on, but I am curious on where this could go, if anywhere.
Very much a WIP. Feedback welcome :-)
- harwoodr 2y agoI see that a speaker is in the hardware list - does this speak back?
- nkaz123 2y agoYes! I'm currently using https://espeak.sourceforge.net/ https://espeak.sourceforge.net/, so it isn't especially fun to listen to though. Additionally, since I'm streaming the LLM response, it won't take long to get your reply. Since it does it a chunk at a time, there's occasionally only parts of words that are said momentarily. Also of course depends on what model you use or what the context size is for how long you need to wait.
- t-vi 2y agoWhen I did a similar thing (but with less LLM) I liked https://github.com/coqui-ai/TTS https://github.com/coqui-ai/TTS but back then I needed to cut out the conversion step from tensor to a list of numbers to make it work really nicely.
- stereosteve 2y agoWhy does Picard always have to specify temperature preference for his Earl Gray tea? Shouldn’t the super smart AI have learned his preference by now?
- dragonwriter 2y ago> Why does Picard always have to specify temperature preference for his Earl Gray tea? Completely OT, but, most likely, he doesn’t. Lots of people in the show direct with the replicators more fluidly; “Tea, Earl Grey, Hot” seems like a Picard quirk, possibly developed with more primitive food/beverage units than the replicators on the Enterprise-D.
- deleted 2y ago[deleted]
- lttlrck 2y agoPerhaps he must be specific to override a hard, lawsuit proof, default that is too tepid for his tastes. Will there still be lawsuits in the post-scarcity world? Probably.
- pkaye 2y agoWell the one time he just said "Tea, Earl Grey" the computer assumed, "Tea, Earl Grey, luke warm".
- TeMPOraL 2y agoForce of habit? Most of Starfleet folks seem to not know how to use their replicators well anyway. For all the smarts they have, they use it like a mundane appliance they never bothered to read the manual for, miss 90% of its functionality, and then complain that replicated food tastes bad.
- cheeseomlit 2y agoIf anything he's not being specific enough https://i.redd.it/hluqexh3oqc91.jpg https://i.redd.it/hluqexh3oqc91.jpg
- zenkalia 2y agoHow hard is it to make an offline model like this learn? The readme mentions a memory that lasts as long as each conversation which seems like such a hard limitation to live with.
- aci_12 2y agohow does wake word work? Does it keep listening and ignore if the last few seconds does not have the wake word/phrase?
- knodi123 2y agothat's the general idea, yes. Or rather, store several chunks of audio, and discard the oldest. aka "Rolling window".
- eddieroger 2y ago> Why Pi-card? > Raspberry Pi - Camera Audio Recognition Device. Missed opportunity for LCARS - LLM Camera Audio Recognition Service, responding to the keyword "computer," naturally. I guess if this ran elsewhere from a Pi, it could be LCARS.
- rkagerer 2y agoPi-C.A.R.D is perfect. Read it 100% as Picard, and more recognizable that LCARS.
- orthecreedence 2y agoJust configure it to respond to "Computer" and you're good to go.
- nkaz123 2y agoThe wake word detection is an interesting problem here. As you can see in the repo, I have a lot of mis-heard versions of the wake word in place, in this case being "Raspberry". Since the system heats up fast you need a fan, and with the microphone directly on a USB port next to the fan, I needed something distinct, and computer wasn't cutting it for this. Changing the transcription model to something a bit better or moving the mic away from the fan could help this happen.
- jhbruhn 2y agoHave a look at the openWakeWord model which is especially built for detecting wakewords in a stream of speech.
- thesnide 2y ago"Number One" would be my code word...
- pimeys 2y agoAnd finally saying "make it so" to make the command happen.
- rkagerer 2y agoI wanted to create a voice assistant that is completely offline and doesn't require any internet connection. This is because I wanted to ensure that the user's privacy is protected and that the user's data is not being sent to any third party servers. Props, and thank you for this.
- ornornor 2y agoDitto!
- pyaamb 2y agoI would love for Apple/Google to introduce some tech that would make it provable/verifiable that the camera/mic on the device can only be captured when the indicator is on and that it isn't possible for apps or even higher layers of OS to spoof this
- herval 2y agoThat’s allegedly the case in iOS (not the provable part, but I wonder if anyone managed to disprove it yet?)
- pyaamb 2y agoI'm thinking perhaps a standardized open design circuit that can you can view by opening up back cover and zooming in with a microscope. feel like privacy tech like this that seemed wildly overkill for everyday users becomes necessary as the value of collecting data and profiling humans goes through the roof
- sneak 2y agoThe value of the data you willingly transmit (both to data brokers, as well as in terms of the harm that it could do to you) via the use of apps (that are upfront about their transmission of your private data) and websites is far, far greater than the audio stream from inside your house. If you don’t want your private information transmitted, worry about the things that are verifiably and obviously transmitting your private information instead of pointlessly fretting over things that are verifiably behaving as they claim. Do you have the Instagram or Facebook apps on your phone? Are you logged in to Google? These are much bigger threats to your privacy than a mic. The sum total of all of your Telegram, Discord, and iMessage DMs (all of which are effectively provided to the service provider without end to end encryption) is way more interesting than 86400 images of you sitting in front of your computer with your face scrunched up, or WAVs of you yelling at your kids. One you knowingly provide to the service provider. The other never leaves your house.
- cmcconomy 2y agoFunny, I just picked up a device for use with https://heywillow.io https://heywillow.io for similar reasons
- knodi123 2y agome too, but I bricked mine when flashing the bios. just a fluke, nothing to be done about it.
- nkaz123 2y agoI watched the demo, to be honest if I saw it sooner I probably would have tried to start this as a fork from there. Any idea what the issue was?
- knodi123 2y agoNo, but the core concept had changed so much, between the firmware version when I bought it, and what it is now, that I'm not surprised if the upgrade is a buggy process. It's a shame, because I only wanted a tenth of what it could do- I just wanted it to send text to a REST server. That's all! And I had it working! But I saw there was a major firmware update, and to make a long story short, KABOOM.
- kkielhofner 2y agoI'm the founder of Willow. Early versions sent requests directly from devices and we found that to be problematic/inflexible for a variety of reasons. Our new architecture uses the Willow Application Server (WAS) with a standard generic protocol to devices and handles the subtleties of the various command endpoints (Home Assistant, REST, OpenHAB, etc). This can be activated in WAS with the "WAS Command Endpoint" configuration option. This approach also enables a feature we call Willow Auto Correct (WAC) that our users have found to be a game-changer in the open source voice assistant space. I don't want to seem like I'm selling or shilling Willow, I just have a lot of experience with this application and many hard-learned lessons over the past two decades.
- dasl 2y agoWhat latency do you get? I'd be interested in seeing a demo video.
- nkaz123 2y agoFully depends on the model, how much conversational context you provide, but if you keep things to a bare minimum, ~< 5 seconds from message received to starting the response using Llama 3 8B. I'm also using a vision language model, https://moondream.ai/ https://moondream.ai/, but that takes around 45 seconds so the next idea is to take a more basic image captioning model and insert it's output into context and try to cut that time down even more. I also tried using Vulkan, which is supposedly faster, but the times were a bit slower than normal CPU for Llama CPP.
- deleted 2y ago[deleted]
- pawelduda 2y agoAll I need is a voice assistant that: - RPi 4 can handle, - I can integrate with HomeAssistant, - is offline only, and doesn't send my data anywhere. This project seems to be ticking most, if not all of the boxes, compared to anything else I've seen. Good job! While at it, can someone drop a recommendation for a Rpi-compatible mic for Alexa-like usecase?
- 8xeh 2y agohttps://www.robotshop.com/products/respeaker-usb-microphone-array https://www.robotshop.com/products/respeaker-usb-microphone-...
- baobun 2y agoCheck out Rhasspy. You won't get anything practically useful running LLMs on the 4B but you also don't strictly need LLM-based models. In the Rhasspy community, a common pattern is to do (cheap and lightweight) wake-word detection locally on mic-attached satellites (here 4B should be sufficient) and then stream the actual recording (more computational resources for better results) over the local network to a central hub.
- shaan7 2y agoI've been a happy Rhasspy user and now even more excited for the future because I'm hoping the conversational style (similar to what OpenAI demo'd yesterday) will eventually come to the offline world as well. It is okay if I gotta buy a GPU to make that happen, or maybe we get luckier and it wont? Ok, maybe I'm getting too optimistic now
- nirav72 2y agoSomeone posted this on the HA sub yesterday - https://www.youtube.com/watch?v=4W3_vHIoVgg https://www.youtube.com/watch?v=4W3_vHIoVgg He's using GPT 4o. Although not the stuff that was demo'ed yesterday, since it hasn't been widely rolled out yet. But I think its entirely possible in the near future.
- knodi123 2y agoIs it possible to run this on a generic linux box? Or if not, are you aware of a similar project that can? I've googled it before, but the space is crowded and the caveats are subtle.
- CaptainOfCoit 2y agoRaspberry-pi is very much like a generic Linux box, biggest difference being that it's ARM rather than Intel/AMD CPU which makes things slightly less widely supported. But overall, Pi-C.A.R.D seems to be using Python and cpp so shouldn't be any issues whatsoever to run this on whatever Python and cpp can be run/compiled on.
- MH15 2y agoI tried to build this on an early gen RPI 4 about three years ago- but ran in to limitations in the hardware (and in my own knowledge). Super cool to see it happening now!
- nickthegreek 2y agoShow HN:
- ddingus 2y agoThank you! This is a high value effort!
- kazinator 2y agoI would have hidden the name behind a bit of indirection by calling it Jean Luc or something.
- ethagnawl 2y agoI'm looking forward to trying this. Hopefully this gains traction, as (AFAIK) an open, reliable, flexible, privacy focused voice assistant is still sorely needed. About a year ago, my family was really keen on getting an Alexa. I don't want Bezos spy devices in our home, so I convinced them to let me try making our own. I went with Mycroft on a Pi 4 and it did not go well. The wake word detection was inconsistent, the integrations were lacking and I think it'd been effectively abandoned by that point. I'd intended to contribute to the project and some of the integrations I was struggling with but life intervened and I never got back to it. Also, thankfully, my family forgot about the Alexa.
- genewitch 2y agosome "maker" products that were sold at target had a cardboard box with an arcade RGB-LED button on top, a speaker, and 4 microphones on a "hat" for a rpi... nano? pico? whatever the one that is roughly the size of a sodimm. It didn't have a wakeword, you'd push the white-lit button, it would change color twice, once to acknowledge the press, and again to indicate it was listening. It would change color once you finished speaking and then it would speak back to you. It used some google thing on the backend, and it was really frustrating to get set up and keep working - but it did work. i have two of those devices, so i've been waiting for something to come that would let me self-host something similar.
- nkaz123 2y agoI'm really inspired reading this and hope it can help! I'm planning to put more work into this. I have a few rough demos of it in action on youtube (https://www.youtube.com/watch?v=OryGVbh5JZE https://www.youtube.com/watch?v=OryGVbh5JZE) which should give you an idea of the quality of it at the moment.
- deleted 2y ago[deleted]
- 8mobile 2y agoI really like having a voice assistant that focuses on privacy first. I will definitely try it. always add a video to show how it works. Thank you
- yumong 2y ago[dead]
- nl 2y ago> It uses distributed models so latency is something I'm working on, I think it uses local models, right?
- robbyiq999 2y agoIt would be cool to see some raspi hats you could plug a GPU into, though unsure of how practical or feasible that would be. Todays graphics cards are tomorrows e-waste, perhaps they could get a second life beefing up a diy raspi project like this
- piltdownman 2y agoMost of the USP, outside of the ecosystem developed around the single platform, is to do with form and power draw. Adding in the GPU/adapter/PSU to leverage cheap CUDA cores probably works out worse power/price/form wise than going for a better SoC or x86 NUC solution.
- genewitch 2y agofor crypto-mining you'd convert a single PCIe slot into 4 x1 PCIe slots - or just have a board with 12+ x1 PCIe slots. I'm not sure what magic goes in to PCIe, but i do know that at least one commodity board had "exposed" PCIe interface - the Atomic Pi. anyhow the GPU would sit on a small PCB that would connect to an even smaller PCB in the PCIe slot on the motherboard, via USB3 cable. My point here is merely that whatever PCIe is, it can be transported to a GPU to do work via USB3 cables.
- ghnws 2y agoI didn't see a mention of languages in the readme. Does this understand languages other than english?
- timendum 2y agoThe suggested model for vision capabilities is english only.
- nkaz123 2y agoThere are two models at use here, whisper tiny for transcribing audio, and then llama 3 for responding. Whisper tiny is multi lingual (though I am using the english specific variant) and I believe llama 3 is technically capable of multi-lingual, but not sure of any benchmarks. I think it could be made better, but for now focus is english. I'll add this to the readme though. Thanks!
- wwryuryrf 2y ago[flagged]
- alexalx666 2y agoCan't wait until I can buy a licensed lieutenant Data voice pack inside chatGPT