mirker

15 ms·

1.
by mirker 3y ago
One thing to point out is that the threshold of predictor complexity is dependent on the execution pipeline. A very speculative and deep architecture has a bigger need for better predictors, since it has a massive penalty when there is a mi
2.
by mirker 3y ago
Does anyone have experience using these open source models in production?
3.
by mirker 3y ago
There is a ton of value. OpenAI having proprietary LLMs single handedly pivoted the entire field to LLMs. A random GitHub repository doesn’t come close to impact.
4.
by mirker 3y ago
Because the authors don’t get a large reward for open sourcing the work and they stand to lose future value by lowering the gate to competition. You may want the code, but Google will not care (or it might dislike it). Look at GPT-3+, OpenA
5.
by mirker 3y ago
They asked Watson of course.
6.
by mirker 3y ago
Agreed. Here’s the thing: the authors of that paper got early access to GPT-4 and ran a bunch of tests on it. The important bit is that MSR does not see into OpenAI’s sausage making. Now imagine if you were a peasant from 1000 AD who was g
7.
by mirker 3y ago
The APIs were messed up early on, which is a reason TF2 happened. Every team started making their own random implementations of stuff. You had the TF Slim API, you had Keras, etc. The API just got fatter and fatter and then libraries would
8.
by mirker 3y ago
Nah, TF has had dynamic execution since TF2 and it’s still losing users, it seems. The execution model and API is simply more complicated. What’s a session, placeholder, constant, tensor, …? PyTorch was sold as numpy with GPU support and it
9.
by mirker 3y ago
PyTorch examples were also cleaner. torchvision had ResNet training batteries included while TF had role your own or clone some weird Keras repository.
10.
by mirker 3y ago
I don’t understand what you mean. Here’s how many applied ML papers work: create a new dataset for a novel problem, download a PyTorch model, point model at dataset directory. Is it novel? By construction. Is the ML technique novel? No.
11.
by mirker 3y ago
The story I’ve heard is the economics undergrads can’t get into economics grad school. This is just a rumor but the sentiment is that undergrads get taught a watered down version of economics theory. Economics theory is potentially very tec
12.
by mirker 3y ago
I agree with you but does anyone even recognize the last category outside blue-sky research? People have a tendency to bin other people into buckets. Being a master at 2 things means you can’t be easily placed in a typical team structure.
13.
by mirker 3y ago
But it’s more thoughtful. The purchaser thought long and hard about what place the purchasee can use it in.
14.
by mirker 3y ago
When people say you need the third hardware revision to get functional performance, and it still doesn’t work, you should conclude it’s poorly designed. The fact that these chips were being sold with old hardware revisions also seems anti-c
15.
by mirker 3y ago
Yeah the issue is you can generate data, but it won’t be good data. Training over random strings won’t make you learn language, but it’s technically data.
16.
by mirker 3y ago
The OPs point is that it’s likely impossible to do what is claimed here in general. Imagine the LLM says something like Fermat’s Last Theorem. To verify it, you’d have to either 1) have a proof assistant powerful enough to construct a proof
17.
by mirker 3y ago
Many of those people left though.
18.
by mirker 3y ago
There’s actually a few papers already on constrained decoding. I won’t link them but if you go on arxiv and really look you will find a couple in the past year.
19.
by mirker 3y ago
Sure, I agree they are useful. My objection is it’s more in the tool category than science, while Alphafold is both. There isn’t convincing evidence that GPTs are pushing what we know; rather, they make it easier to process/search what
20.
by mirker 3y ago
Alphafold is open and seems fundamentally transformative in the science space. GPT is nice but it’s a smart meme-generator at the moment. I don’t disagree with the impact on G’s bottom line, though.
21.
by mirker 3y ago
OS needs less than 100GB, though, so mixed drives seems more cost effective.
22.
by mirker 3y ago
The image is nearly unreadable for me on mobile.
23.
by mirker 3y ago
DOTA, Rocket League, CSGO, etc. are e-sports friendly but don’t have this population problem. It’s not about being “hardcore”; it’s about having broad appeal to hit a wide market segment. RTS has struggled with that historically. Tower defe
24.
by mirker 3y ago
Hardcore “competitive” RTS (StarCraft/Age of Empires) can’t since it’s not competitive enough if you can’t hit a meaningful 300+ actions per minute, which is impossible without hotkeys. Even IF they could, I am skeptical you’d get conv
25.
by mirker 3y ago
It looks interesting, honestly. Shame the RTS market is relatively small.
26.
by mirker 3y ago
I think one of the most glaring problems with the argument is HPC maintains tons of legacy simulations which may not even have the original authors around. Adding Spark support wouldn’t make this easier.
27.
by mirker 3y ago
Isn’t the simpler example data parallel training? That’s a reduce and broadcast.
28.
by mirker 3y ago
1 GPU in a NAS server is all you need. Pretty simple.
29.
by mirker 3y ago
Integrating an accelerator into a dishwasher is also expensive. How many inferences are you going to get out of it before the hardware is outdated? Once a day for 5 years may not be enough to justify the cost. Compare that to having a singl
30.
by mirker 3y ago
Privacy doesn’t matter unless the IoT devices are secure. Often times, they’re not.

More ›