7 ms·

Here's the template I'm using - https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd0d4c1d5a2d4a70b49166/litellm/llms/prompt_templates/factory.py#L5 htt

Here's the template I'm using - https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd0d4c1d5a2d4a70b49166/litellm/llms/prompt_templates/factory.py#L5 https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd... Anything I'm doing incorrectly?

YetAnotherNick 3y ago
The format is really weird for chat models. In your code post message is `<</SYS>>\n [/INST]` which is wrong. Instead the overall format is `[INST] <<SYS>>\n{system}\n<</SYS>>\n\n{instruction} [/INST]` So both the system message and first instruction are in same [INST] block.