patcher99

5 ms·

1.
by patcher99 1y ago
Very cool, would this also support moderations api for self-hosted LLMs by any chance?
2.
by patcher99 2y ago
Hi HN! I'm preparing to relaunch my AI App development tool, OpenLIT, on Product Hunt. During our first attempt, we made it to #10 on Product of the Day. While that was a solid outcome, there's room for improvement, and we can aim
3.
by patcher99 2y ago
Love this blog and really glad that you found OpenLIT useful for your blog.
4.
Show HN: How I made monitoring and debugging self-hosted LLMs way easier(docs.openlit.io)
1 points by patcher99 2y ago|0 comments
5.
Show HN: All you need is Prometheus and Jaeger for LLM Observability(docs.openlit.io)
3 points by patcher99 2y ago|0 comments
6.
Should I add CLA to my Open-source project?(docs.openlit.io)
1 points by patcher99 2y ago|1 comments
7.
by patcher99 2y ago
Last week we showcased our open-source project, OpenLIT ( https://github.com/openlit/openlit ), here, and thanks to this incredible community, we hit 300 stars in just a couple of days! One of my mentors, a core lead on
8.
by patcher99 2y ago
We do Otel metrics for the main things needed in dashboarding 1. Requests (Counter) 2. Tokens (Counter) (Seperate metric for Prompt Tokens, Completion and Total Tokens) 3. Cost (Histogram) I did attach a Grafana dashboard too (Works for Gra
9.
by patcher99 2y ago
@xyst https://discord.gg/RbNPvG54 Thanks for the suggestion on community (Not Slack). I Created a Discord server now, Feel free to spam questions, suggestions! Thanks for the feedback!
10.
by patcher99 2y ago
Yeah, 100%, I couldn't agree more on this! Though if you'd like to see something more, Lemme know! Thanks!
11.
by patcher99 2y ago
Yup, You are on point! We do see the UI/dashnoarding layer to find other usage a bit more as we add more tools(Compare/Evaluate) to help the overall LLM Developer Experience. Also, I totally get the existing tools like Grafana are
12.
by patcher99 2y ago
Yup, The Dashboard is entirely powered by OTEL traces and yeah, you can see full prompts and responses including cost and request metadata. The UI is pretty easy to run, Its one single image. Lemme know if you need any help on instrumentati
13.
by patcher99 2y ago
Yup thats correct, Love as more tools come up! We though tried to follow the actual OpenTelemetry's Semantic Convention for GenAI, the others dont. Additionally none of these libraries "then"(Can check again) seemed to send t
14.
by patcher99 2y ago
Hey! On the metrics front, we do basic metrics like requests, tokens and cost for now as all of the usefull information about llm is included directly in the Span Attributes of the Trace. So what a lot of users have told me is the want to
15.
Show HN: OpenLIT – Open-Source LLM Observability with OpenTelemetry(github.com)
1 points by patcher99 2y ago|0 comments
16.
by patcher99 2y ago
Hey! Kudos to them firstly! Its the same problem statement but different solutions. They seem to use tracing and tracing generally adds a lot of overhead on the application. Our method tries to avoid that plus added latency is ~0.01s in our
17.
Show HN: Open-Source LLM Observability and Export to Grafana, Datadog etc.(github.com)
3 points by patcher99 2y ago|2 comments