9 ms·

> it is about attacking ... via its http interface Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token

• kzrdude 25d ago
  That reads to me as if the GP confabulated a different exploit target than what the article mentioned (or the article changed), but it's also a threat that should be taken seriously.
• angry_octet 25d ago
  How do you think the tokens get to the vLLM instance? Telepathy?
  • hobofan 25d ago
    > the parser that ingests token sequences
  • angry_octet 25d ago
    @hobofan How would you access that parser?
    • hobofan 24d ago
      _you_ wouldn't. The blog post is about vulnerabilities that happen inside the inference engine, and not via external interfaces accessible via HTTP. The token parser is involved both on the way in (where yes, you would as user supply it likely via HTTP), as well as the way out of the LLM (where there are very few, less hardened layers when interfacing with the parser). e.g. for a vulnerability exploit in the parser, an LLM can be trained with a specific input sequence that would output a fixed malicious payload to output eval-able content, as outlined in the article.
      • angry_octet 24d ago
        The attacker supplies the context that triggers these bugs via the http interface. There is no other way to supply it with context. Vulnerable code could exist in token generation and in hook recognition and tool call parsing [1]. There is also significant scope for mischief in tool ID mapping between models and harnesses, as these are done by untyped numeric IDs, with varying schema[2]. Routers also introduce vulnerability paths as they inspect these tokenized (json) sequences and act in them, e.g. to match models with stricter call signature regex. [3] Parallel tool calling is also an interesting surface for exploits. [1] https://docs.vllm.ai/en/stable/api/vllm/tool_parsers/#vllm.tool_parsers.ToolParserManager https://docs.vllm.ai/en/stable/api/vllm/tool_parsers/#vllm.t... [2] https://docs.mistral.ai/resources/cookbooks/concept-deep-dive-tokenization-tool_calling https://docs.mistral.ai/resources/cookbooks/concept-deep-div... [3] https://docs.litellm.ai/docs/completion/function_call https://docs.litellm.ai/docs/completion/function_call
        • hobofan 24d ago
          I'm not saying that the HTTP interface is not an attack surface, but that securing that attack surface (as one may assume in a simple security model) is not enough to protect against that class of attacks, as no level of introspecting the input will help predict and secure against malicous output.
          • angry_octet 23d ago
            This is misunderstanding the attack. The potentially malicious output tokens are processed entirely by classical text processing techniques, i.e. parsing, in order to enable tool calling on the client and via MCP servers. Garden variety vulnerabilities can occur on the inference server, but are amenable to modest levels of software engineering competency. Basically, it's a regex, don't fuck it up.
• j16sdiz 25d ago
  > ... the parser that ingests token sequences. which is exposed via http