7 ms·

Honest question: has anyone found skills that fundamentally changed their workflow vs. ones that are just ‘nice to have’? Curious what the actual power-user sta

Honest question: has anyone found skills that fundamentally changed their workflow vs. ones that are just ‘nice to have’? Curious what the actual power-user stack looks like. Anyways, great work on this btw, the agent-agnostic approach is the right call

rudedogg 8mo ago
My experience with them is limited, but I’m having issues with the LLMs ignoring the skills content. I guess it makes sense, it’s like any other piece of context. But it’s put a damper in my dream of constraining them with well crafted skills, and producing high quality output.
- amadeuswoo 8mo ago
  Yeah, the context window is a blunt instrument, everything competes for attention. I get better luck with shorter, more opinionated skills that front-load the key constraints vs. comprehensive docs that get diluted. Also explicitly invoking them (use the X skill) seems to help vs hoping they get picked up automatically
  - deleted 8mo ago
    [deleted]
  - SOLAR_FIELDS 8mo ago
    Yes, unfortunately the most reliable way is to inject them into the user prompt at a fresh session. My guess is that biasing towards checking for the tools availability too much affects performance, which might explain why it is quite rarer that I see it just choose to use a skill without previous prompting.
- KingMob 8mo ago
  Yeah, I'm still trying to figure out why some skills are used every day, while others are constantly ignored. I suspect partially overlapping skill areas might confuse it. I've added a UserPromptSubmit hook that does basic regex matches on requests, and tries to interject a tool suggestion, but it's still not foolproof.
NitpickLawyer 8mo ago
One simple but useful flow is to ask cc to review a session and find miss-matches between initial skills / agent.md and current session, and propose an edit. I then skim over it and add it. It feels like it helps, but I don't have quantitative data yet.
flwi 8mo ago
Define "fundamental", but I added skills to run complicated evaluation procedures for my ML research. This way I can open 5 CC instances, let them run and iterate on research without intervention. After that, I can do the final review.
esperent 8mo ago
> power-user stack Try installing the Claude Superpowers skills - you can install them one by one from here, but it's easier to install the superpowers plugin. Try using it for a couple of sessions and see how it works for you. For a full test, try starting with the brainstorming one which then guides you from brainstorming though planning, development etc. I've been using it for a few days and I would say it's enhanced my workflows at least.
zby 8mo ago
I would double - do skills reliably work for you? I mean are they reliably injected when there is a need, as opposed to being actively called for (which in my opinion defeats the purpose of skills - because I can always ask the llm to read a document and then do something with the new knowledge). I have a feeling that codex still does not do it reliably - so I still have normal README files which it loads quite intelligently and it works better than the discovery via skills.
theshrike79 8mo ago
Basically skills are /commands that can have attached scripts, that's about it. If your "skill" doesn't come with scripts/executables, it's just a fancy slash command. I've had success with code quality analysis skills that use PEP723 Python scripts with tree-sitter to actually analyse the code structure as an AST. The script output and switches are specifically optimised for LLM use, no extra fluff - just condensed content with the exact data needed.

rudedogg 8mo ago

My experience with them is limited, but I’m having issues with the LLMs ignoring the skills content. I guess it makes sense, it’s like any other piece of context. But it’s put a damper in my dream of constraining them with well crafted skills, and producing high quality output.

amadeuswoo 8mo ago
Yeah, the context window is a blunt instrument, everything competes for attention. I get better luck with shorter, more opinionated skills that front-load the key constraints vs. comprehensive docs that get diluted. Also explicitly invoking them (use the X skill) seems to help vs hoping they get picked up automatically
- deleted 8mo ago
  [deleted]
- SOLAR_FIELDS 8mo ago
  Yes, unfortunately the most reliable way is to inject them into the user prompt at a fresh session. My guess is that biasing towards checking for the tools availability too much affects performance, which might explain why it is quite rarer that I see it just choose to use a skill without previous prompting.
KingMob 8mo ago
Yeah, I'm still trying to figure out why some skills are used every day, while others are constantly ignored. I suspect partially overlapping skill areas might confuse it. I've added a UserPromptSubmit hook that does basic regex matches on requests, and tries to interject a tool suggestion, but it's still not foolproof.

NitpickLawyer 8mo ago

One simple but useful flow is to ask cc to review a session and find miss-matches between initial skills / agent.md and current session, and propose an edit. I then skim over it and add it. It feels like it helps, but I don't have quantitative data yet.

flwi 8mo ago

Define "fundamental", but I added skills to run complicated evaluation procedures for my ML research. This way I can open 5 CC instances, let them run and iterate on research without intervention. After that, I can do the final review.

esperent 8mo ago

> power-user stack Try installing the Claude Superpowers skills - you can install them one by one from here, but it's easier to install the superpowers plugin. Try using it for a couple of sessions and see how it works for you. For a full test, try starting with the brainstorming one which then guides you from brainstorming though planning, development etc. I've been using it for a few days and I would say it's enhanced my workflows at least.

zby 8mo ago

I would double - do skills reliably work for you? I mean are they reliably injected when there is a need, as opposed to being actively called for (which in my opinion defeats the purpose of skills - because I can always ask the llm to read a document and then do something with the new knowledge). I have a feeling that codex still does not do it reliably - so I still have normal README files which it loads quite intelligently and it works better than the discovery via skills.

theshrike79 8mo ago

Basically skills are /commands that can have attached scripts, that's about it. If your "skill" doesn't come with scripts/executables, it's just a fancy slash command. I've had success with code quality analysis skills that use PEP723 Python scripts with tree-sitter to actually analyse the code structure as an AST. The script output and switches are specifically optimised for LLM use, no extra fluff - just condensed content with the exact data needed.