5 ms·

Thank you for this release. I believe your library is a key component to unlocking the potential of LLMs without the limitations/restricitions of existing clien

Thank you for this release. I believe your library is a key component to unlocking the potential of LLMs without the limitations/restricitions of existing clients. Since you released version 0.26 alpha, I’ve been trying to create a plugin to interact with a some MCP server, but it’s a bit too challenging for me. So far, I’ve managed to connect and dynamically retrieve and use tools, but I’m not yet able to pass parameters.

simonw 1y ago
Yeah I had a bit of an experiment with MCP this morning, to see if I could get a quick plugin demo out for it. It's a bit tricky! The official mcp Python library really wants you to run asyncio and connect to the server and introspect the available tools.
- mihau 1y ago
  Hi Simon! I'm a heavy user of the llm tool, so as soon as I saw your post, I started tinkering with MCP. I’ve just published an alpha version that works with stdio-based MCP servers (tested with @modelcontextprotocol/server-filesystem) - https://github.com/Virtuslab/llm-tools-mcp https://github.com/Virtuslab/llm-tools-mcp. Very early stage, so please make sure to use with --ta option (Manually approve every tool execution). The code is still messy and there are a couple of TODOs in the README.md, but I plan to work on it full-time until the end of the week. Some questions: Where do you think mcp.json should be stored? Also, it might be a bit inconvenient to specify tools one by one with -T. Do you think adding a --all-tools flag or supporting glob patterns like -T name-prefix* in llm would be a good idea?
  - simonw 1y ago
    OK this looks like a very promising start! You're using function-based tools at the moment, hence why you have to register each one individually. The alternative to doing that is to use what I call a "toolbox", described here: https://llm.datasette.io/en/stable/python-api.html#python-api-toolbox https://llm.datasette.io/en/stable/python-api.html#python-ap... Those get you two things you need: 1. A single class can have multiple tool methods in it, you just have to specify it once 2. Toolboxes can take configuration With a Toolbox, your plugin could work like this: llm -T 'MCP("path/to/mcp.json")' ... You might even be able to design it such that you don't need a mcp.json at all, and everything gets passed to that constructor. There's one catch: currently you would have to dynamically create the class with methods for each tool, which is possible in Python but a bit messy. I have an open issue to make that better here: https://github.com/simonw/llm/issues/1111 https://github.com/simonw/llm/issues/1111
    - mihau 1y ago
      Thanks for feedback! Ah, I saw "llm.Toolbox" but I thought it's just for plugin developer convenience. I'll take a look at the issue you posted (#1111). Maybe I can contribute somehow :).

simonw 1y ago

Yeah I had a bit of an experiment with MCP this morning, to see if I could get a quick plugin demo out for it. It's a bit tricky! The official mcp Python library really wants you to run asyncio and connect to the server and introspect the available tools.

mihau 1y ago
Hi Simon! I'm a heavy user of the llm tool, so as soon as I saw your post, I started tinkering with MCP. I’ve just published an alpha version that works with stdio-based MCP servers (tested with @modelcontextprotocol/server-filesystem) - https://github.com/Virtuslab/llm-tools-mcp https://github.com/Virtuslab/llm-tools-mcp. Very early stage, so please make sure to use with --ta option (Manually approve every tool execution). The code is still messy and there are a couple of TODOs in the README.md, but I plan to work on it full-time until the end of the week. Some questions: Where do you think mcp.json should be stored? Also, it might be a bit inconvenient to specify tools one by one with -T. Do you think adding a --all-tools flag or supporting glob patterns like -T name-prefix* in llm would be a good idea?
- simonw 1y ago
  OK this looks like a very promising start! You're using function-based tools at the moment, hence why you have to register each one individually. The alternative to doing that is to use what I call a "toolbox", described here: https://llm.datasette.io/en/stable/python-api.html#python-api-toolbox https://llm.datasette.io/en/stable/python-api.html#python-ap... Those get you two things you need: 1. A single class can have multiple tool methods in it, you just have to specify it once 2. Toolboxes can take configuration With a Toolbox, your plugin could work like this: llm -T 'MCP("path/to/mcp.json")' ... You might even be able to design it such that you don't need a mcp.json at all, and everything gets passed to that constructor. There's one catch: currently you would have to dynamically create the class with methods for each tool, which is possible in Python but a bit messy. I have an open issue to make that better here: https://github.com/simonw/llm/issues/1111 https://github.com/simonw/llm/issues/1111
  - mihau 1y ago
    Thanks for feedback! Ah, I saw "llm.Toolbox" but I thought it's just for plugin developer convenience. I'll take a look at the issue you posted (#1111). Maybe I can contribute somehow :).