roh26it

6 ms·

1.
by roh26it 4mo ago
What do you use for the onboarding guide?
2.
by roh26it 5mo ago
Well written, but does this all change in a claude-ified world?
3.
Why do you need an MCP Gateway?(portkey.ai)
1 points by roh26it 8mo ago|0 comments
4.
Show HN: Fast Semantic Tool-filtering for MCP servers(github.com)
1 points by roh26it 10mo ago|0 comments
5.
Everything we know about Claude Code Limits(portkey.ai)
4 points by roh26it 1y ago|0 comments
6.
Portkey's Prompt Engineering Studio – The IDE for Prompt Engineers
2 points by roh26it 2y ago|0 comments
7.
Ask HN: Have you used v0, lovable or bolt for anything useful?
2 points by roh26it 2y ago|1 comments
8.
We analysed 4B requests on our AI Gateway(portkey.ai)
5 points by roh26it 2y ago|0 comments
9.
by roh26it 2y ago
Looks solid, going to try it out. I'm going to be __that guy__, but just ask - is the functionality set similar to llamaparse or is this llamaparse + llm?
10.
by roh26it 2y ago
So, while time to first token is lower, throughput might also be lower in most cases?
11.
by roh26it 2y ago
What are the trade-offs you've made to achieve this?
12.
Come down a rabbit hole with OpenAI o1 system card(portkey.ai)
1 points by roh26it 2y ago|1 comments
13.
by roh26it 2y ago
The most interesting findings (at least from my perspective) - OpenAI measured "honesty" that could become an interesting model eval others should adopt. - Using instruction heirarchy to _avoid_ prompt injections - When pushed to
14.
Ask HN: Is function patching in Python acceptable for a library to do?
3 points by roh26it 2y ago|2 comments
15.
by roh26it 2y ago
I've been thinking about AI guardrails in production for almost 7 months now, and I'm convinced that using them to "control", "secure", or "block" LLM outputs is not the best use for them . Using them
16.
by roh26it 2y ago
Also wondering if this would become expensive very fast if it ends up using S3 with a large number of PUT calls
17.
Show HN: Created a notebook to compare the top LMSYS vision models easily(github.com)
2 points by roh26it 2y ago|0 comments
18.
by roh26it 2y ago
thanks for the support!
19.
by roh26it 2y ago
We use a bunch of caching mechanisms on the LLM requests themselves and extend the same to guardrails now. So there's 2 levels of cache - the LLM request itself might be cached (simple and semantic) and the guardrail response can be ca
20.
Show HN: An open-source AI Gateway with integrated guardrails(github.com)
21 points by roh26it 2y ago|5 comments
21.
by roh26it 2y ago
Being one of the most downloaded datasets on Huggingface, I was a little bit surprised by how dirty this dataset was. Plus it had very limited information and some incorrect classifications as well. For an internal experiment on building a
22.
I added context data to the TruthfulQA dataset(huggingface.co)
1 points by roh26it 2y ago|1 comments
23.
by roh26it 2y ago
Here's a mega guide on keeping costs low with LLMs - https://portkey.ai/blog/implementing-frugalgpt-smarter-llm-u... tl;dr: - Keep prompts short, combine prompts or make more detailed prompts but go to a smaller m
24.
by roh26it 2y ago
At Portkey, this is a problem we deal with quite a bit. Also the reason that Datadog and the traditional observability vendors did not work for LLM use cases since they're not built to handle large volumes of data. We've done this
25.
Implementing FrugalGPT in Production(portkey.ai)
2 points by roh26it 2y ago|0 comments
26.
Streaming LLM requests need this Nginx config(serverfault.com)
1 points by roh26it 2y ago|0 comments
27.
by roh26it 2y ago
Openrouter packages APIs and most companies prefer having individual relationships with AI vendors. Choosing an AI gateway might be another way to go
28.
by roh26it 3y ago
Individual subreddits have usually been my go-to place to discuss and understand the nuances of academic papers. Love LocalLLama for discussions on generative AI papers.
29.
Adding a streaming run function to the Assistants API(github.com)
1 points by roh26it 3y ago|0 comments
30.
Caching stream responses across LLMs is hard(twitter.com)
2 points by roh26it 3y ago|0 comments

More ›