sshumaker

6 ms·

1.
by sshumaker 3mo ago
There’s both medication (which takes a while to get dosage right but can be extremely helpful for some) and also techniques you can use to help compensate.
2.
by sshumaker 4mo ago
Are you serious? It’s multiple nodes to run a frontier model (a node is 8x GPUs), and they aren’t running on H100s. You are looking at 32+ GPUs.
3.
by sshumaker 2y ago
It’s how LLMs work - they are effectively recursive at inference time, after each token is sampled, you feed it back in. You will end up with the same model state (not including noise) as if that had been the original input prompt.
4.
by sshumaker 2y ago
It depends on how large the input prompt (previous context) is. Also, if you can keep cache on GPU with a LRU mechanism, for certain workloads it's very efficient. You can also design an API optimized for batch workloads (say the same
5.
by sshumaker 2y ago
They are almost certainly doing this internally for their own chat products. The simple version of this just involves saving off the KV cache in the attention layers, and restore it back instead of recomputing. It only requires small change
6.
by sshumaker 2y ago
This is a pretty standard technique if you're running the models yourself. e.g. ChatGPT almost certainly does this. There's even work that is more sophisticated in this domain that allows 'template' style partial caching
7.
by sshumaker 2y ago
Stealth startup | ML Engineer (edge inferencing) | Bay Area or LA (hybrid) | Full-time We're a stealth startup building something unbelievably ambitious in the AI space that blends AI and gaming tech - co-founders are Andy Gavin [1] (c
8.
by sshumaker 2y ago
I had the pleasure of working closely with Jeff Slutter early in my career. Was the first really fantastic engineer I worked with - he’s at Santa Monica Studios (God of War) these days. I think Matt Toschlog is there too now. Glad to see th
9.
by sshumaker 2y ago
Monarchies usually result in long-term thinking? I’m sure we haven’t read the same history books. Most monarchs throughout history have been very self-interested and their decisions have been focused on their own personal interest - the def
10.
by sshumaker 2y ago
Hamas gained a ton of support for their cause by the tragedy inflicted in the counterattack. I believe they celebrate the deaths of Palestinian innocents as much as they do the Israeli ones so they can extract propaganda wins. This has bee
11.
by sshumaker 3y ago
It seems like rather than adding a random amount to each sample (which lets them compute a mean by recreating the same audio and extracting out the differences), Safari could instead add randomness that is based on a key that rotates every
12.
by sshumaker 3y ago
Stealth startup | Lead mobile engineer, AI Engineer | Bay Area or LA (hybrid) | Full-time We're a stealth startup building something unbelievably ambitious in the AI space that blends AI and gaming tech - co-founders are Andy Gavin [1]
13.
by sshumaker 3y ago
You’re being downvoted because this is a hot take that isn’t supported by evidence. I just tried exactly that with dalle-3 and it worked well. More to the point, it’s pretty clear LLMs do form a model of the world, that’s exactly how they r
14.
by sshumaker 3y ago
You can also look at Bertopic which has this functionality as an open source library: https://maartengr.github.io/BERTopic/index.html
15.
by sshumaker 3y ago
Sometimes you can use a heuristic to estimate K, or use a variant that terminates at some distance threshold. That said, something like hdbscan doesn’t suffer from this problem.
16.
by sshumaker 3y ago
I have an eightsleep, which cools the bed down dramatically (circulates cold water). Makes a huge difference in my ability up sleep.
17.
by sshumaker 3y ago
I talked to Pavel about taking a role in my team earlier this year (I ended up leaving Microsoft myself since). He was passionate about making the developer experience excellent - code quality, clean APIs, etc. Thats a tall order for some p
18.
by sshumaker 3y ago
I don’t know about [1]. I asked an example from the paper above to GPT-4: “[If you had to guess] how many thumbs did Lincoln’s maternal grandmother have?” Response: There is no widely available historical information to suggest that Abraham
19.
by sshumaker 3y ago
If you ask it properly it gets it right. From a pure measurement standpoint, could Jupiter fit in the space between the earth and moon? The average distance from the Earth to the Moon is about 238,855 miles (384,400 kilometers). Jupiter, th
20.
by sshumaker 3y ago
I’m fairly confident this is untrue. At Microsoft at least, it’s a big deal when there is a privacy issue, even a small localized one on a single product - and creates a small firestorm. We’ll get engineers working long hours focused on it,
21.
by sshumaker 3y ago
I had the same issue, but support kept disconnecting me and I had to start all over again going through the entire “try all of the brain dead stuff like resetting” 3 times, before I finally gave up and just ordered a new Nest and ate the co
22.
by sshumaker 3y ago
> They will instead pay a lot for MSFTs cloud service offering, which of course comes with the crucial promise that their data is safe and secured and handled in a way that is compliant with all privacy laws. Which of course isn't t
23.
by sshumaker 3y ago
This is really a lot less necessary with GPT-4. What required careful prompting in 3.5 often you can give it something slapdash in 4 and it can do a great job figuring out intent.
24.
by sshumaker 4y ago
Just use the Azure hosted solution, which has all of Azure's stronger guarantees around compliance. I'm sure it will update with GPT-4 pricing shortly. https://azure.microsoft.com/en-us/products/cognitive
25.
by sshumaker 4y ago
Lots of people have way more than that due to retirement accounts (which sometimes need more liquidity or people want to get out of the market so they have more cash on hand). Or even if you have a down payment on a house in most of the coa
26.
by sshumaker 4y ago
That’s how PageRank effectively worked, and people created deep networks of pages voting for each other. To solve this problem for real you either need Sybil protection or to make it economically infeasible (impose a tax for making content
27.
by sshumaker 4y ago
In a leadership role. I would love this - particularly to help make me aware of when I should be holding more space for others. But the fact that the meeting data (audio/video) gets transmitted to a third party is a complete deal kille
28.
by sshumaker 4y ago
Revisionist history? The Jews who converted to avoid expulsion and (some of whom) were practicing their religion in secret would like a word. https://www.chabad.org/library/article_cdo/aid/2430792/jewis..
29.
by sshumaker 4y ago
It did use one time pads: Two paper tape readers are mounted in front of the 700 Unit at the top. One is a 5-level tape reader, which is used to read the plaintext (when used offline). The other one is a 6-level reader which is used to read
30.
by sshumaker 4y ago
This would be easily exploitable by a competitor. For example, search engines (used to) rank back links - that is other domains pointing to your domain. Some bad actors took advantage of this by creating rings of sites that voted each other

More ›