Local models need to be tuned to work well so this looks useful. Seems to be for general purpose model serving. I’ve been using https://github.com/adrianco/reto

Local models need to be tuned to work well so this looks useful. Seems to be for general purpose model serving. I’ve been using https://github.com/adrianco/retort https://github.com/adrianco/retort to run experiments for coding models across 13 different programming languages to see which frontier and local models work.