4 ms·
It would be really cool if they could expose this information to customers somehow. Imagine: - having a log of the most prominent J-space tokens during your
by meatmanek 2mo ago
It would be really cool if they could expose this information to customers somehow. Imagine:
- having a log of the most prominent J-space tokens during your customer support chatbot's interactions with a user, so you can have more introspection into why a particular outcome happened
- being able to detect certain thoughts associated with undesirable behavior (hallucinations, overstepping authority, lying, etc.) and trigger some sort of remediation (e.g. upgrading to a better model, redirecting to a human, forcing tool calls)
- dofm 2mo agoPresumably the rationale for the decision to abridge the thinking traces will ensure that they don’t; if this is real (and there’s no good reason to trust that it is yet) then it is the secret sauce.
- charcircuit 2mo agoAnthropic aren't even willing to expose the CoT of their models. You will have to rely on them to build those sorts of things into dedicated signals.
- throw310822 2mo agoAnthropic won't do it, but they published the j-lens to introspect the model- from what I understand it's roughly simply feeding a chosen layer straight into the final layers of the LLM for decoding into language: https://github.com/anthropics/jacobian-lens https://github.com/anthropics/jacobian-lens Looks like it should be easy to use on open weights models.