w1nk

6 ms·

1.
by w1nk 2y ago
For a more informed opinion than folks on the internet, here's some work from Microsoft with early/internal access to gpt4: https://arxiv.org/abs/2303.12712 . I don't think people close by these systems
2.
by w1nk 2y ago
These projects are awesome to see, there are similar efforts for everquest. Is anyone aware of anyone trying to create different clients/renderers for these MMOs? A VR client for any of these worlds would instantly be amazing.
3.
by w1nk 3y ago
Just to further this, it's not just 'big names' that feel this way. Read this paper from a team at Microsoft Research: https://arxiv.org/abs/2303.12712 . These folks spent months studying properties of
4.
by w1nk 3y ago
This seems like (clearly?) bad journalism. RFID tags on their own are for sure not taking active sensor readings, that'd be left for something closer to TPMS style systems.
5.
by w1nk 3y ago
Yes, it did: https://chasethedevil.github.io/post/the_neural_network_in_y...
6.
by w1nk 3y ago
No, your OP is mistaken. The model weights have to all be accessed for the forward pass. What has happened is that using mmap changes where the memory is consumed (kernel vs process) and so it was being incorrectly interpreted. There are
7.
by w1nk 3y ago
Ahh, this would do it, thanks :).
8.
by w1nk 3y ago
That shouldn't be the case. 30B is a number that directly represents the size of the model, not the size of the other components.
9.
by w1nk 3y ago
I also have this question, yes it should be. The forward pass should require accessing all the weights AFAIK.
10.
by w1nk 3y ago
Does anyone know how/why this change decreases memory consumption (and isn't a bug in the inference code)? From my understanding of the issue, mmap'ing the file is showing that inference is only accessing a fraction of the we
11.
by w1nk 4y ago
So can we at least take a second to appreciate the orders of magnitude of growth between BBS' and tiktok? With data structures and algorithms we're very willing to accept that orders of magnitudes shifting can change our basic as
12.
by w1nk 4y ago
This thread is exposing one of the dirty realities of modern SEO. Most of the time, SEO folks will measure their success based on having some way of determining their ranking for a given search query. The problem is that in 2022, google pe
13.
by w1nk 4y ago
It's mostly a curiosity. Occasionally things like this occur: https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i... . TL;DR exploited an accidentally turing complete file format to execute instruc
14.
by w1nk 4y ago
For big organizations that have the capacity and data, online advertising becomes a ROI optimization game, and one that they perform quite well at. For a random business that wants to advertise online, without the infrastructure and data ca
15.
by w1nk 4y ago
Given the usual markers of attractiveness, wouldn't you expect age and weight to be strong predictors usually?
16.
by w1nk 4y ago
There's nothing 1960s about it, that's just not well reasoned. These computation constructs/tools just don't currently have better abstractions while maintaining the desired computational performance. It's a strang
17.
by w1nk 4y ago
What? CUDA is intentionally granular and low level, why do you feel that is a negative thing at this level of abstraction? Are you suggesting the tools should be better so that doesn't have to be the case? I can't figure out wh
18.
by w1nk 4y ago
You're not wrong that the dataset and compute are important, and if you browse the author's previous work, you'll see there are datasets available. The reproduction of DALL-E 2 required a dataset of similar size to the one i
19.
by w1nk 4y ago
To expand a bit for the grandparent, if you check out this authors other repos you'll notice they have a thing for implementing these papers (multiple DALLE-2 implementations for instance). You should expect to see an implementation t
20.
by w1nk 4y ago
Why does everyone assume this guy has zero business attempting this? If you read his credentials, he should be every bit as qualified as you to attempt this kind of work while understanding the pitfalls. According to his CV he's been
21.
by w1nk 4y ago
So that's actually one of the awful implications of this paper. It's probably actually happening at a rate higher than would be noticed by humans. If a given piece of silicon is hosing up a GEMM (matrix multiply), in graphics sce
22.
by w1nk 4y ago
It's boggling to me that everyone is so entirely convinced of this fact. Check out this paper from google where they observe this exact process occurring in their CPUs at scale: https://sigops.org/s/conferences&#x
23.
by w1nk 4y ago
Yes, but for different reasons. The grandparent is probably incorrect though, there is emerging evidence that silicon is actually changing/failing over time. See this paper from google on their CPU cores where they have practical evi
24.
by w1nk 4y ago
Actually there's reasonable evidence to believe this isn't true. This paper from google: https://sigops.org/s/conferences/hotos/2021/papers/hotos21-s... outlines specific scenarios where
25.
by w1nk 4y ago
Lua definitely caught a stray here. The reference VM is pretty fast, the JIT is just way faster. No idea how Lua ended up wrapped in the conversation with Python...
26.
by w1nk 4y ago
So if you want someone to answer precisely how they'll extract hundreds of millions of dollars from an emerging market, I have to imagine this isn't the correct forum to expect such answers. Enabling the general software community
27.
by w1nk 4y ago
So this question can only come from a place where you have no idea what they do in their field. For every news article or arxiv post that you see talking about how this amazing new GPT-N model has broken all sorts of language benchmark sco
28.
by w1nk 4y ago
> There’s this interesting dissonance where all the second year CS students and their professors agree it’s the wrong tool for the job yet the most successful products in the world did it anyway. > I guess you could interpret that to
29.
by w1nk 4y ago
Hello whomever from clubhouse is promoting this blog post on the internet. Looking forward, please consider adding more technical depth to the content. This is meant to be a technical blog post and the only technical description of the sys
30.
by w1nk 4y ago
Have you ever seen the OSINT where people figured out that the badges of certain military programs gave away the reason for their existence? https://www.nytimes.com/2008/04/01/world/americas/01iht-p

More ›