ankitmathur

5 ms·

1.
by ankitmathur 1mo ago
Hey! Thanks for the feedback! I work on many of these things at Databricks, so figured I'd chime in on this. Firstly, while routing is important, simple things like observability into the token costs of various features, which can driv
2.
by ankitmathur 1mo ago
We're going to do a followup blog detailing our routing approach soon! In short, the router takes in the task description and infers what models and harnesses are available and makes a recommendation up-front. So essentially the routin
3.
by ankitmathur 1mo ago
Something underlying a lot of this is that pricing models for enterprise coding tools have changed from seat-based to consumption-based pretty quickly, as AI usage has exploded. For months, engineers were able to use unlimited AI for no mar
4.
by ankitmathur 1mo ago
Omnigent and OpenRouter are different in the sense that OpenRouter is where you can go to call the actual model but Omnigent is intended to be the place where you go describe the high level task to be done, and work is farmed out to various
5.
by ankitmathur 1mo ago
100% - there's a lot to learn from traces from real-life sessions with coding tools! For example, I found it pretty eye-opening to see how wide the distribution of tasks truly is. There's also subtle things like how a poorly desig
6.
by ankitmathur 3y ago
I’ve never used Paperspace, so I’ll try to give it a try this weekend. How much RAM do you have attached to the compute. We don’t think it should be any harder to run this via HF pipelines than other similarly sized models, but I’ll look in
7.
by ankitmathur 3y ago
We'd love to help you all deploy this! 1. We just released a couple models that are much smaller ( https://huggingface.co/databricks/dolly-v2-6-9b ), and these should be much easier to run on commodity hardware in a
8.
by ankitmathur 3y ago
Augmenting the answer to address your followup: parameters are any trainable variable in a model's definition. Model training is a process where you basically tweak the parameters in your model and then re-evaluate the model on a metri
9.
by ankitmathur 3y ago
Hey there! I'm one of the folks working on Dolly - Dolly-V2 is based on the GPT-NeoX architecture. llama.cpp is a really cool library that was built to optimize the execution of the Llama architecture from Facebook on CPUs, and as such
10.
by ankitmathur 3y ago
Sure! I'll try to briefly summarize though almost certainly will oversimplify. There are a couple of open source language models trained by Eleuther AI - the first one was called GPT-J, and it used some newer model architecture concept
11.
by ankitmathur 3y ago
Hey! Worked on this here at Databricks: the blog post goes into the dataset collection design a bit ( https://www.databricks.com/blog/2023/04/12/dolly-first-open-... ). In summary, you're right - brai
12.
by ankitmathur 3y ago
Out of curiosity: what's an example of a metric that you would use to evaluate the ability of the model? For example, just looking qualitatively, asking a prompt like "How do I tie a tie?" to Pythia produces content that isn&
13.
by ankitmathur 3y ago
Hey there! I worked on Dolly, and I work on Model Serving at Databricks. DollyV1 is GPT-J-based, so it'll run easily on llama.cpp. DollyV2 is Pythia-based, which is built with the GPT-NeoX library GPT-NeoX is not that different than GP
14.
by ankitmathur 6y ago
While this is true, I'm pretty sure the referenced poll was conducted well before that was announced. In fact, I've heard internal criticisms of the opposite direction. The prospect of a full time shift to remote was not communi