confuseshrink

7 ms·

1.
by confuseshrink 4y ago
Starcraft in the form of Alphastar worked in the sense that it could beat humans, at least in the short term. The problem with the whole technique is that they had to tether it to the human examples they had gathered in the form of a diverg
2.
by confuseshrink 6y ago
Interesting point. Nvidia have been improving the int performance for quantized inference on their GPUs a lot. It might be a lot of work but could it be possible to scale up this NNUE approach to the point where it would be worthwhile to ru
3.
by confuseshrink 6y ago
The published hyperparameters are usually ridiculously conservative, for the simple games like breakout and pong you can usually converge in far fewer frames than in the papers.
4.
by confuseshrink 6y ago
> We're getting a bit off-topic here, but the #1 target by far in reducing greenhouse emissions is power generation. I readily admit I don't know any of the numbers associated with carbon production and my comment was solely ba
5.
by confuseshrink 6y ago
This is a very valid argument but it's hard to know what scaling a transformer will really do without trying (looking at you GPT-3). This is probably an issue for ML in general at this point. I think a more nuanced conversation around
6.
by confuseshrink 6y ago
Yes it's something I often see ignored as "common knowledge" dictates that in ML inference is way cheaper than training. But if you're running a model in production at google with loads of google searches hitting it ever
7.
by confuseshrink 6y ago
The Strubell paper which is the origin of this "5 cars" number isn't even in the right ballpark for this stuff. What they did was take desktop GPU power consumption running the model in fp32, extrapolate to a 240x GPU (P100)
8.
by confuseshrink 6y ago
Yann is a consistently sober voice in this world of AI hype. I find it quite refreshing. Personally I see little evidence that this "just scale a transformer until sentience" hype-train is going to take us anywhere interesting or
9.
by confuseshrink 6y ago
> So what changed? We aren't sure, but the speculation is that in the process of training, GPT-3 found that the best strategy to correctly predicting the continuation of arithmetic expressions was to figure out the rules of basic ar
10.
by confuseshrink 6y ago
I would start with David Silvers (DeepMind) youtube series to get an idea of what's possible or not. Running an already trained reinforcement learning agent is relatively cheap (unless your model is massive). I suspect the reason peopl
11.
by confuseshrink 6y ago
It depends on how intensive your pre-processing pipeline is. With a really fast accelerator you can quite easily start to be bottlenecked by your CPU.
12.
by confuseshrink 6y ago
Vectorization: I'm not an expert in this area so I can only tell you what I've personally found difficult in dealing with vectorization. Usually it all comes down to alignment and vector lanes. To utilize the vector instructions y
13.
by confuseshrink 6y ago
Bias is just a scalar term that is added. You can learn it via backpropagation like all the other weights.
14.
by confuseshrink 6y ago
Since you are right that I have no idea what you are talking about, could you explain what you are talking about?
15.
by confuseshrink 6y ago
I find it very surprising that someone would rely on unvalidated mathematical models for this, that goes for the Imperial College people as well as Sweden. Are they even able to fit the parameters in retrospect? Anyone with a background in
16.
by confuseshrink 6y ago
That article doesn't state that 14% of cases were asymptomatic.
17.
by confuseshrink 6y ago
I'm not a mathematician but coming from the software world if one guy wrote a massive program (I'm assuming 600 pages is massive) in "an impenetrable, idiosyncratic style" you could virtually guarantee it would not be co