sipjca

8 ms·

1.
by sipjca 8d ago
I understand. And inference optimizations are great. It's just that it's not a new model, and that's what I was excited about. I work on cross-platform transcription inference and was hoping for something more than just a re-
2.
by sipjca 8d ago
seems to be…
3.
by sipjca 8d ago
at first i got very excited about a new fast transcription model (voz) but turns out its just parakeet v3 with some new inference code which is macOS/iOS specific
4.
by sipjca 18d ago
it's unbelievable this was not pointed out earlier in this thread, long live tom 7
5.
by sipjca 25d ago
what do you mean cheaper/faster is not really coming? the cost of the same level of intelligence steadily decreases year over year. computer hardware also advances at the same time enabling cheaper and faster serving (or move to local)
6.
by sipjca 1mo ago
people are lazy people don't want to maintain infra (ie adding new models all the time) people want their queries to work without thinking if a provider goes down, openrouter queries dont (ideally) it's really not that complex to
7.
by sipjca 1mo ago
Agree, I was not talking about a business context. But if people are submitting garbage in a business context you got other problems you need to address.
8.
by sipjca 1mo ago
As a maintainer just saying no and closing PRs is largely the solution
9.
by sipjca 1mo ago
Unfortunately you are wrong. Time marches on and progress in silicon does too. You may not like the timeline, but it will happen. While not exactly an “SSD”, High Bandwidth Flash will make it to market and make big sparse MOE models even mo
10.
by sipjca 2mo ago
all day yesterday, Taiwan time Sol was having significant issues. “Overloaded”, maybe half of requests or more wouldn’t go through
11.
by sipjca 2mo ago
It's not that much extra work to compile, the extra work comes from the maintenance and feature requests. By not shipping the binary directly I am defending my time until other contributors want to step up and maintain things. I am one
12.
by sipjca 2mo ago
we have wtype and ydotool support as well in Handy, but it's not exactly a great out of the box experience always, but some of it is on me for poor linux packaging
13.
by sipjca 2mo ago
Right now I intend to maintain this as a library. The examples are just that, examples for programmers/agents. If someone in the community wants to step up to maintaining release binaries I will gladly have that support, it's just
14.
by sipjca 2mo ago
Wow, it's amazing to hear that even though I released this so recently people are already using it properly! Thanks! Please let me know any issues you run into
15.
by sipjca 2mo ago
yes
16.
by sipjca 2mo ago
word boosting will probably come on a much longer time horizon, but streaming is here! I'm really hoping someone either contributes a good server example to the codebase (and is willing to help with issues) or use transcribe.cpp or the
17.
by sipjca 2mo ago
I’m on a train right now but off the top of my head the audio pipeline may have to be modified slightly to emit partial text segments as they come in from the transcription engine. And then calling the appropriate paste method the user has
18.
by sipjca 2mo ago
Eventually I would like to be more fully drop in compatible, right now some feature support is a bit sparse. And whisper has so much work done to it over the years so it’s hard to support every possible thing. Right now it’s a more bog stan
19.
by sipjca 2mo ago
For sure, it was not initially a target because I didn’t need it for Handy but I do understand the importance in the broader context
20.
by sipjca 2mo ago
Testers by far as the most needed thing, I do maintain a list of per platform people who help to test so if you drop a GitHub username (or email me) I will add you to the list and ping for help Basically the biggest blocker is me being the
21.
by sipjca 2mo ago
Every model with open weights has some code which can be used to inference it. So we download the published weights and run against inference library they suggest, be it transformers, Nemo, etc
22.
by sipjca 2mo ago
Largely this is out of scope for the library, mainly because I’m not aware of many models supporting this. but if there are models which support this would be happy to support
23.
by sipjca 2mo ago
Yep, could do simple things like literal regex or all the way up to LLM cleanup, tons of options
24.
by sipjca 2mo ago
Yeah I’m working on it, Linux is a big pain point especially Wayland Once things are more or less ironed out on MacOS and Windows a lot of attention will be turned towards Linux I know a lot of Linux PRs are open it just takes me so long to
25.
by sipjca 2mo ago
TensorRT and CUDA is effectively the same speed as CPU for the speech to text models I was testing via ONNX at a huge binary bloat penalty. WGPU is hard to ship and also equivalent speed or slower. This may not be the case for LLM or other
26.
by sipjca 2mo ago
Not in the library itself, it’s pure inference. Some models have this trained out of them anyhow. Otherwise this is a post processing task which is not really inference
27.
by sipjca 2mo ago
Out of the box no probably not, but if people are interested there’s probably ways forward
28.
by sipjca 2mo ago
You can fairly easily modify [Handy]( https://handy.computer ) to do this if you want I’m planning on having it as a first class feature of the app too just too many other issues to work on first
29.
by sipjca 2mo ago
Parakeet unified for me no longer does this and it’s also a streaming transcription model! But the answer largely depends on you, the languages you speak, and personal preference. Whisper is still excellent and supported in transcribe.cpp C
30.
by sipjca 2mo ago
Contributions are always welcome! There’s a WIP diarization PR rn, and after it’s merged would love to have support if it fits well into the interface. And if not would love to figure out a good interface for it

More ›