marcklingen

6 ms·

1.
How we run untrusted code in Langfuse (OSS) and why we picked lambda(langfuse.com)
4 points by marcklingen 3mo ago|3 comments
2.
Scaling ClickHouse to petabytes of AI observability data(langfuse.com)
5 points by marcklingen 6mo ago|0 comments
3.
by marcklingen 8mo ago
Let me know what features are missing in prompt management
4.
by marcklingen 8mo ago
(thank you!)
5.
Ask HN: Who/what is the top-educator/content on how to use ChatGPT/Claude well?
1 points by marcklingen 1y ago|0 comments
6.
Open Source LLMOps Stack(oss-llmops-stack.com)
64 points by marcklingen 2y ago|7 comments
7.
by marcklingen 2y ago
The Innovators, Walter Isaacson It’s interesting to read how many individuals contributed in all sorts of important ways in the history of computing.
8.
by marcklingen 2y ago
This is interesting! Do you plan to add support for distinguishing between blocking and non-blocking/async migrations? This seems to be a common challenge when using Clickhouse and needing to change the ordering of existing tables. Con
9.
by marcklingen 2y ago
The Langfuse data model is closely inspired by OpenTelemetry and we plan to add a Collector soonish. Until now, the OTel-semantics for LLMs have not been very stable and exhaustive while LLM capabilities are changing frequently (think promp
10.
by marcklingen 2y ago
Thanks! IMO, Discord is good, but GitHub Discussions is the better option for building a growing open-source community. It is indexed and makes it easier to revisit conversations weeks later. Currently we use both but have a strong preferen
11.
by marcklingen 2y ago
All core features are fully open-source and identical to those in Langfuse Cloud, with no limitations on capabilities or scalability (e.g. all v3 infrastructure changes). We also offer some optional commercial add-on features that can help
12.
by marcklingen 2y ago
Congrats on the Launch!
13.
by marcklingen 2y ago
As you mentioned, this was a significant trade-off. We faced two choices: (1) Stick with a single Docker container and Postgres. This option is simple to self-host, operate, and iterate on, but it suffers from poor performance at scale, esp
14.
by marcklingen 2y ago
Thanks! Really enjoyed working with you maintainers of other projects to help them offer more native LLM observability and evaluation to their users/communities. There is a lot that goes into making the observability/eval part sca
15.
by marcklingen 2y ago
Thank you! Working with your team has been great. I love seeing you ship LLM-powered features and appreciate the feedback you have shared along the way.
16.
by marcklingen 2y ago
thank you! if you have any ideas for improvements after having used Langfuse for a while, please contribute them via github discussions: https://langfuse.com/ideas
17.
by marcklingen 2y ago
Thanks for the feedback, glad that you find Langfuse useful! Can you create an issue with more details on the API performance problems? We monitor strict SLOs on the public API for Langfuse Cloud and are not aware of any ongoing issues, wou
18.
by marcklingen 2y ago
appreciate your constructive feedback! > i wonder if there are new ops solutions for the realtime apis popping up This is something we have spent quite some time on already, both on designs internally and talking to teams using Langfuse
19.
by marcklingen 2y ago
Thanks for the feedback. Being unopinionated and API-first has been a core design decision. We want to build the building blocks that everyone needs while acknowledging that most Langfuse users are very sophisticated teams that have a clear
20.
by marcklingen 2y ago
Thanks AJ, feedback on GitHub/Discord (like yours) has been very helpful to evolve prompt management from a quick addition of the core platform to one of the most-used features -- for which we then actually needed to change a lot of in
21.
by marcklingen 2y ago
This is a good long-list of projects, although it is not narrowly scoped to tracing/evals/prompt-management: https://github.com/tensorchord/Awesome-LLMOps?tab=readme-ov-...
22.
by marcklingen 2y ago
Thank you for the kind words! Let us know if you have any questions or feedback regarding the self-hosting documentation and experience. We collaborate with many teams that have diverse security needs, including HIPAA, PCI, and on-premises
23.
by marcklingen 2y ago
If you use an OSS Gateway already, some (e.g. LiteLLM) can natively forward logs to Langfuse: https://docs.litellm.ai/docs/proxy/logging#langfuse We are looking into adding an Otel Collector as OTel-semantics are
24.
by marcklingen 2y ago
Positioning/roadmap differs between the different project in the space. We summarized what we strongly believe in here: https://langfuse.com/why Tldr: open apis, self-hostable, LLM/cloud/model/framework-
25.
by marcklingen 2y ago
This is an optional instrumentation of the OpenAI SDK which simplifies getting started, tracking token counts, model parameters and streaming latencies. Langfuse is not in the critical path, this just helps with instrumentation. You can use
26.
by marcklingen 2y ago
-- langfuse founder/maintainer here After working with thousands of teams building agents and complex LLM applications, I'd conclude that: - all frameworks have their strengths and weaknesses, Crew and LangGraph are the popular op
27.
by marcklingen 2y ago
disclosure: founder/maintainer of Langfuse (OSS LLM application observability) I believe proxy-based implementations like Velvet are excellent for getting started and solve for the immediate debugging use case; simply changing the base
28.
by marcklingen 2y ago
Langfuse co-founder/maintainer here, while the underlying architecture will change in v3 (thanks for referencing the thread), the docker-composed based self-hosting setup will remain as simple as shown in the video verdverm, thanks for
29.
by marcklingen 2y ago
Big fan of LiteLLM Proxy and LiteLLM Python SDK to connect to various local models. Might be helpful here as well
30.
OSS Decorator to Trace LLM Apps – Integrated with LlamaIndex, LangChain, OAI SDK(langfuse.com)
6 points by marcklingen 2y ago|0 comments

More ›