I did it for story events in my grand strategy game (Fall of an Empire) with a set of finetuned local models. It doesn't affect performance (for most people) si

I did it for story events in my grand strategy game (Fall of an Empire) with a set of finetuned local models. It doesn't affect performance (for most people) since the game is bottlenecked on other things. Players seem to like it, there are some issues since the models are so small they are more prone to hallucination, and you also have to give them preset topics or they go on the same track again and again. I haven't been able to use it for more than events since the tokens per second aren't high enough - I would like to use the models to e.g. generate actions for characters based on their history of interactions with other characters, but that ends up taking so much time that the player never gets any events. I also did a demo of an LLM-powered NPC, I think the issue here is that you need more than just the LLM in e.g. a 3D RPG where you can actually go up and talk to the character. You can generate the text, but from there you also need to generate speech, from that you need to dynamically generate realistic facial and body animations that match up with it - this means you need 3 or 4 models chained together in a pipeline.