That is actually possible. For example, someone wrote python code to do this for the massive open source model BLOOM. However, it's still slow as tar. When I w

That is actually possible. For example, someone wrote python code to do this for the massive open source model BLOOM. However, it's still slow as tar. When I was running the BLOOM model I think my inference time was 1 token / m. See: https://towardsdatascience.com/run-bloom-the-largest-open-access-ai-model-on-your-desktop-computer-f48e1e2a9a32 https://towardsdatascience.com/run-bloom-the-largest-open-ac...