eiz

5 ms·

1.
by eiz 3y ago
https://arxiv.org/pdf/2302.13971.pdf table 15. 1770394 A100-80GB hours to train the entire model suite at the going rate for cloud 8xA100-80GBs (~$12/hr if you could actually get capacity) is ~$2.6M, under extreme
2.
by eiz 3y ago
> Where is the connection between computational details and the model's high-level behavior? Do we even know? This is an active area of study ("mechanistic interpretability") and it's very early days. For instance her
3.
by eiz 3y ago
> 4. Describing positional embeddings as multiplicative. They are generally (and very counterintuitively to me, but nevertheless) additive with token embeddings. Worth noting that rotary position embeddings, used in many recent architect
4.
by eiz 3y ago
Fair enough. In the GH100 architecture doc https://resources.nvidia.com/en-us-tensor-core/gtc22-whitepa... (page 18) they do mention retaining 2 graphics-capable TPCs but it's clearly not the focus.
5.
by eiz 3y ago
> They do not have any graphics rendering circuitry. What? Not having a display output is not the same as not having graphics rendering circuitry. Here's vulkaninfo from an A100 box: https://gist.github.com/eiz/
6.
by eiz 3y ago
Here's a couple more quotes from Yann LeCun, their Chief AI Scientist: https://twitter.com/ylecun/status/1643945742850031616 https://twitter.com/ylecun/status/1644503237699969026 pr
7.
by eiz 4y ago
I tried this: > Solve the following word problem step by step: > "John is in a train moving west at ten miles per hour. Jill is in a train moving north at twenty miles per hour. How fast are John and Jill moving apart from each o
8.
by eiz 4y ago
I've done this a few times too. > There’s no way to pre-register the pet. At least on Delta, you can. You need to call the airline and arrange it. > Oh, and to go through security, you have to take a nervous and scared cat out of
9.
Preparing for the first winter without Russian gas(bruegel.org)
38 points by eiz 5y ago|6 comments
10.
by eiz 5y ago
> This means no closures, which K implementers consider a feature (I don't). having not touched K in about 15 years, when did this change? in k3: K 3.2 2004-09-23 Copyright (C) 1993-2004 Kx Systems LIN32 16CPU 15985MB ubun
11.
by eiz 6y ago
> If you think Chat apps are winning, let me know when you can buy an item online without an account linked to an Email. I'd love to see an example. https://www.amazon.com/gp/help/customer/display.html
12.
by eiz 6y ago
We can do full (permanent) remote.
13.
by eiz 8y ago
> It's an HDMI cable! The video signal has CRC in it and is packetized, it's either going to make it or it isn't. I don't disagree with your main point, but this actually isn't quite true. The HDMI signal is spli
14.
by eiz 9y ago
Last time I checked the "free" provisioning profiles it generates are valid for something like 7 days. It's not really convenient for anything but experimenting with iOS development.
15.
by eiz 10y ago
The calculator you linked to assumes a maximum TCP window size of 64KB for everything but "replication". TCP window scaling has been on by default in every major OS for 10 years or more, allowing much greater throughput. It's
16.
by eiz 11y ago
Relevant: https://www.youtube.com/watch?v=Q0-51IkWpFE
17.
by eiz 11y ago
house_arrest used to allow direct access to the documents and container of any app. It's used by the Xcode "installed applications" list, to allow you to download and upload container contents. In 8.3 they changed it to only