5 ms·
I don't even need to tiptoe ! Not being there and not prompting anything is enough to trigger safeguards. Having not asked a single security question it will w
by kay_o 24d ago
I don't even need to tiptoe ! Not being there and not prompting anything is enough to trigger safeguards.
Having not asked a single security question it will write wildly vulnerable code, go back and fix it, and guardrail itself out of existence after charging me a large sum with no refunds for no output and having not fixed it because that might be secuirty adjacents.
And if it doesn't do this you end up with code that has such holes, store xss , no authz ... if it does not go back and notice it has written bad code.
Since they hide thinking and reasoning from the user (who is also paying for those tokens) it is a black box what is triggering it, has the LLM this time thought of "Oh, this has XSS" and used a bad dangerous word such as XSS, while the previous conversation did not ?