Analog24

8 ms·

1.
by Analog24 6mo ago
So the "E2B" and "E4B" models are actually 5B and 8B parameters. Are we really going to start referring to the "effective" parameter count of dense models by not including the embeddings? These models are impre
2.
by Analog24 2y ago
They had to put together ~14 hours of visuals, more than 5x what U2 had to do. There's nothing low effort about that. Some visuals were not as good as others as a result but I can promise you that the audience had their jaws on the flo
3.
by Analog24 2y ago
As someone else mentioned in this thread, it is not possible to understand how everything you purchase works. That is also incredibly infeasible. What you are asking is to effectively revert back two hundred years of technological progress
4.
by Analog24 2y ago
I think the expectation that the entire consumer market (or even just a majority) is going to collectively become universally informed about all their purchases and shift the market for the better is far less likely then a government interv
5.
by Analog24 2y ago
Internships are definitely not a waste of time. First of all, they pay well in and of themselves (at least in tech). Second of all, most internships are filled by students on their summer break. What better use of that time than getting an
6.
by Analog24 3y ago
That's not true at all. At Meta your RSUs are based off the average share price for the month before your start date
7.
by Analog24 3y ago
"neat future" is very ambiguous. At the moment there is nothing even close to transformers in terms of performance. I suspect you are right in general but I'm not sure about the "near future" part, there needs to be
8.
by Analog24 3y ago
This is the reason why they're not going to move on device anytime soon. You can use compression techniques, sure, but you're not going to get anywhere near the level of performance of GPT-4 at a size that can fit on most consumer
9.
by Analog24 3y ago
"It's six years later and Siri, Alexa, and Google Home are still nearly as dumb as they were back then". You can't even keep a coherent discussion and you are delusional about the significance of your work. You shared a
10.
by Analog24 3y ago
The fact that you think creating a writing assistant plugged into Word is equivalent to building a general purpose, always-on voice assistant tells me all I need to know.
11.
by Analog24 3y ago
Do you have any publications to back up your claims about your work? They seem more than a bit grandiose. If you're ideas are as novel and useful as you say then you should publish them. And I'm sorry, but you're completely w
12.
by Analog24 3y ago
Scaling up an LM from 2017 would not achieve what GPT-4 does. It's nowhere near that simple. Of course companies saw the potential of natural language interfaces, there has been billions spent on it over the years and a lot of progress
13.
by Analog24 3y ago
It fails at _some_ arithmetic. Humans also fail at arithmetic... In any case, is that the defining characteristic of having a good enough "world model"? What distinguishes your ability understand the world vs. an LLM? From my pers
14.
by Analog24 3y ago
What is your intent?
15.
by Analog24 3y ago
If you read some of the studies of these new LLMs you'll find pretty compelling evidence that they do have a world model. They still get things wrong but they can also correctly identify relationships and real world concepts with start
16.
by Analog24 3y ago
You can't just wave your hand and tell someone that words are broken up into sub-word tokens that are then transformed into a numerical representation to feed to a transformer and expect people to understand what is happening. How is a
17.
by Analog24 3y ago
Embeddings and their relationship to each other are definitely relevant to transformers. Why do you think that's not the case?
18.
by Analog24 3y ago
What would be the differentiating factor(s) for true AI/intelligence in your opinion?
19.
by Analog24 3y ago
Another hyperbolic analogy. What a great argument, I'm convinced.
20.
by Analog24 3y ago
No it wasn't. You have the misconception that an AI system has to achieve anthropomorphic qualities to become dangerous. ML algorithms are goal-oriented, they are optimized to maximize a specific goal. If that goal is not aligned with
21.
by Analog24 3y ago
That analogy is nonsense. We're actually making measurable gains in AI and the pace of progress is only increasing. Belief in this isn't based on faith, it is based on empirical evidence.
22.
by Analog24 3y ago
> > > Should we develop nonhuman minds > The embarrassing part. Preaching a religion under professional pretenses. There are quite a few companies with the explicitly stated goal of developing AGI. You can debate whether or not
23.
by Analog24 3y ago
I had the same thought after reading this and I have to say my gut reaction to that thought was: "I hope so".
24.
by Analog24 3y ago
You can make a pretty compelling argument that those narco states developed because of the war on drugs
25.
by Analog24 4y ago
What developer working on anything meaningful does not rely on documentation? You certainly have to make the documentation available similar to how you would have to make it "available" to an LLM. I think you might be missing the
26.
by Analog24 4y ago
Because there is a huge selection bias that influences the sample of people that comment on these types of posts. I will back up the claim that, in my experience, a lot of this thread is hyperbole. There is some truth to it, sure, but don&#
27.
by Analog24 4y ago
The idea of connecting CV to audio via spectrograms pre dates Jeremy Howard's course by quite a bit. That's not really the interesting part here though. The fact that a simple extension of an image generation pipeline produces suc
28.
by Analog24 4y ago
Neutrons aren't that hard to capture. They are certainly harder to capture than charged particles but there are plenty of materials that are dense enough to reliably capture neutrons. This is how heat is extracted from the reaction to
29.
by Analog24 4y ago
Could you elaborate on your point a bit more? If you're talking about utilizing the weak force vs. the residual strong force then I'm not sure this argument holds up. Also, when comparing to renewable+storage you have to consider
30.
by Analog24 4y ago
You have it backwards. The fission component is just the "trigger" for the fusion element, which produces the vast majority of the energy release.

More ›