joshhart

6 ms·

1.
by joshhart 20d ago
An impressive amount of gall. Meta will only pay the full settlement if their competitors ALSO pay a fine. From the article: The remaining 30% (approximately $5.3 billion) will be released only after two specific conditions are met. YouTube
2.
by joshhart 1mo ago
Discriminative has a meaning in machine learning that I think is relevant here. There are "generative" models like LLMs that are learning joint probabilities P(X, Y) and "discriminative" models like logistic regression t
3.
by joshhart 3mo ago
According to the article, the major effect is they are catching and prosecuting cheaters with a secondary effect that students are over-relying on them for homework or practice, and are unprepared for exams. The article also talks about the
4.
by joshhart 4mo ago
Fireworks will serve them for $1.74 / $0.14 / $3.48. That's input / cached input / output. https://fireworks.ai/models/deepseek-ai/deepseek-v4-pro . Call it about a third the price of Sonn
5.
by joshhart 7mo ago
If you have good ideas that have a nice return on investment and leverage existing skills, sure. If you don’t have good opportunity laying around, best for the business to switch to maintenance mode, which means cutting staff. Or maybe cut
6.
by joshhart 8mo ago
I thought this wasn't viable due to cooling requirements - how do you cool massive amounts of compute when the only option is to radiate it into space - nothing to convect it with? Also, the incredible amount of grift here with the lef
7.
by joshhart 8mo ago
Vitamin D toxicity is absolutely real, causes hypercalcemia, and can occur even at the 4,000 IU dose. I would really recommend you be getting regular bloodwork done if you go beyond that. Here’s a fun podcast on a case study. https:/&
8.
by joshhart 8mo ago
Huh. The standard in your case is to measure waist circumference if BMI is high. Did no doctor do that? As long as you are below 40” or 37” if Asian you are considered good to go.
9.
by joshhart 10mo ago
This is super awesome, but how in the world did they come up with a name "Nano Banana Pro"? It sounds like an April Fools joke.
10.
by joshhart 11mo ago
I believe it's intended to convince the audience they are experts, that this type of thing is dangerous to a business, and they are the ones doing the most to prevent it. There is no explicit statement to this effect, but I get the sen
11.
by joshhart 1y ago
So the inference speed at low to medium usage is memory bandwidth bound, not compute bound. By “forecasting” into the future you do not increase the memory bandwidth pressure much but you use more compute. The compute is checking each poten
12.
by joshhart 1y ago
A single node with GPUs has a lot of FLOPs and very high memory bandwidth. When only processing a few requests at a time, the GPUs are mostly waiting on the model weights to stream from the GPU ram to the processing units. When batching req
13.
by joshhart 2y ago
This is pretty exciting. Now an organization could produce an open weights mixture of experts model that has 8-15b active parameters but could still be 500b+ parameters and it could be run locally with INT4 quantization with very fast perfo
14.
by joshhart 2y ago
Hi, Yes you can. The community creates quantized variants of these that can run on consumer GPUs. A 4-bit quantization of LLAMA 70b works pretty well on Macbook pros, the neural engine with unified CPU memory is quite solid for these. GPUs
15.
by joshhart 2y ago
The benchmarks compare it favorably to GPT-4-turbo but not GPT-4o. The latest versions of GPT-4o are much higher in quality than GPT-4-turbo. The HN title here does not reflect what the article is saying. That said the conclusion that it&#x
16.
by joshhart 2y ago
Hi, I run the model serving team at Databricks. Usually you run regex filters, LLAMA Guard, etc on chunks at a time so you are still streaming but it's in batches of tokens rather than single tokens at a time. Hope that helps! You coul
17.
by joshhart 3y ago
I spent 12 years at LinkedIn. Sadly, it's not even close to the engineering org it used to be. The era where Kevin Scott led engineering was a really good one in comparison.
18.
by joshhart 3y ago
At Databricks we have an LLM that is fine-tuned to do the problem you raise - https://www.databricks.com/blog/announcing-public-preview-ai... Many customers like it a lot. Although perhaps in your case if there are man
19.
by joshhart 3y ago
Makes sense, CPUs and memory sizes aren’t growing that fast anymore. But I’m sure they are spending a ton on TPUs/GPUs, the article is clear on very high capex
20.
by joshhart 3y ago
If you are making many requests in batch this works ok because you can shuffle the next layer in while the current one is processing a set of matrix multiplies. This takes it from being a memory bound problem to a flops bound problem. This
21.
by joshhart 3y ago
LinkedIn was already very FCF positive. They tightly managed margins to get to net income positive (account for dilution and so on) but it took maybe 2 years after the acquisition.
22.
by joshhart 3y ago
This was cancelled over a year ago - which the articles notes and is old news. It was clear the effort would have needed a very significant push that would have required a large halt in product development and management wasn't willing
23.
by joshhart 3y ago
I left LinkedIn 1.5 years ago. I was there 12 years. I saw the revenue & profitability growth that occurred post acquisition. I am very very confident LinkedIn would be worth north of $100B on public markets today and Microsoft made the
24.
by joshhart 3y ago
Will be a long time before a large number of these are available in clouds though, not until summer at least.
25.
by joshhart 3y ago
Trainium and Inferentia sadly aren't going this way, they have their own approach https://github.com/aws-neuron/transformers-neuronx . I think the best case scenario is some middleware like Triton https:/&#x
26.
by joshhart 3y ago
This is a big cluster, definitely large enough to pretrain 100B+ parameter LLMs in months. Source - I work at Databricks in the ML platform. I don’t know much about AV processing, that’s highly customized to only a few customers but I’d exp
27.
by joshhart 3y ago
I think these are good points, I am hugely in favor of expanding food stamps and child tax credits for this reason. One estimate is that every $1 spend on food stamps expands GDP by $1.50, so this is really good for the overall economy. I h
28.
by joshhart 3y ago
The article suggests several causes: 1. Bias in population - these students all have families where at least one parent has a stable job, which isn't true elsewhere. There could also be other factors, for instance maybe people who ente
29.
Deploy Private LLMs Using Databricks Model Serving(databricks.com)
2 points by joshhart 3y ago|0 comments
30.
by joshhart 3y ago
Former director at LinkedIn. There might be that many internal URL endpoints (I am skeptical though). There are far fewer services than that though, each service supports many APIs.

More ›