6 ms·
> except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the res
by ricketycricket 2y ago
> except if the tokenizer or whatever doesn't follow a particular format but in that case you just upload it to some free web service and make a PR with the result and reference that version hash specifically and it'll work.
May I ask to which service you are referring?
- cess11 2y agoThis one: https://jonatanklosko-bumblebee-tools.hf.space/apps/tokenizer-generator https://jonatanklosko-bumblebee-tools.hf.space/apps/tokenize... It's linked in the Bumblebee README. Seems broken at the moment, maybe the PR it made is more informative: https://huggingface.co/Neprox/STT-Swedish-Whisper/discussions/3 https://huggingface.co/Neprox/STT-Swedish-Whisper/discussion...
- cess11 2y agoThe tokenizer generator has risen again.