6 ms·
5.6 class models have a brand new caching architecture. Make sure you were using the prompt cache breakpoints and cache keys correctly - it's much different tha
by MitziMoto 2mo ago
5.6 class models have a brand new caching architecture. Make sure you were using the prompt cache breakpoints and cache keys correctly - it's much different than < 5.5 and makes a huge difference for many production workloads.