zingelshuher

6 ms·

1.
by zingelshuher 2y ago
> Google knowingly made their search results shittier and shittier for years unfortunately this extends to youtube too. now they have a new shitty trick. you click on the link and they randomly give you a completely different video.
2.
by zingelshuher 2y ago
> It is unlikely people are going to switch en mass to open source models It depends on the task at hands. For complex tasks no way personal computer can compete with giants data centers. But, as soon as software becomes available, user
3.
by zingelshuher 2y ago
> poorly designed government intervention due to misunderstanding of the dynamics behind the process (homelessness) Major drive is easy to understand, just cross the border and you are homeless on full support. Millions did with the hel
4.
by zingelshuher 2y ago
had to upvote this
5.
by zingelshuher 2y ago
Only if it does nothing. In fact Google is one of the major players in LLM field. The winner is hard to predict, chip makers likely ;) Everybody jumped on bandwagon, Amazon is jumping...
6.
by zingelshuher 2y ago
I often use ChatGPT4 for technical info. It's easier then scrolling through pages whet it works. But.. the accuracy is inconsistent, to put it mildly. Sometimes it gets stuck on wrong idea. Interesting how far LLMs can get? Looks like
7.
by zingelshuher 2y ago
It's impossible. Meta itself cannot reproduce the model. Because training is randomized and that info is lost. First samples a coming at random. Second there are often drop-out layers, they generate random pattern which exists only on
8.
by zingelshuher 2y ago
If we can keep unlimited memory, but use only a selected relevant subset in each chat session. This should help. Of course the key is 'selected', it's another big problem. Like short memory. Probably we can make summaries fro
9.
by zingelshuher 2y ago
It's a different animal. In general you cannot reproduce the model even having all the training data. There are too many random factors and nobody keeps track of them. Just pushing the training data is done at random from the dataset.
10.
by zingelshuher 2y ago
It's a matter of opinion how much open model should be to be called 'open source'. Looks like some believe they have the right to define it for everybody else to use. Like for software. Have to disagree. Why don't we int
11.
by zingelshuher 2y ago
> I'll bet Adobe or similiar will buy this 0,5 mil May be that was the business plan, or 'plan B'
12.
by zingelshuher 2y ago
Yes. And there are many forgotten accounts on youtube, their owners now have another way to monetarize. Not sure why Adobe didn't talk to Youtube directly.
13.
by zingelshuher 2y ago
As you mentioned models need _random_ videos. Just 'walk outside' will produce more or less the same. My guess Adobe is more interested in 'family' sort of videos, with humans. This is what most users will be asking for.
14.
by zingelshuher 2y ago
It doesn't affect creators life as they guarantee (likely) it will not be made public. So it's just side income. Likely creators make a lot of fragments which don't get into the final version. The problem for Adobe is most of
15.
by zingelshuher 2y ago
It reflects the fact that Amazon in serious about AI. If fact they are well positioned with their datacenters and a lot of ways to apply, starting with smarter Alexa.
16.
by zingelshuher 2y ago
Expect sh*t load of AI hallucinations. As if Wiki isn't bad enough with BS some intentionally posting.
17.
by zingelshuher 2y ago
"the bigger you make epsilon "... " thus slower the training progress will be" Sounds like variable epsilon is optimal, that's instead of learning rate, or both together. Would be nice if this can somehow be algorit
18.
by zingelshuher 2y ago
Intuitively looks like models should be close enough, or sparse enough for merge to work. I wonder if MoE experts can be merged(?)
19.
by zingelshuher 2y ago
Good luck with that ;) Actually you _can_ learn juggling this way, just couple of minutes at a time.
20.
by zingelshuher 2y ago
There are unstable cases when static learning rate doesn't work. Solution starts wobbling too much after some time and explodes. Using too small LR from the beginning leads to local minima. Making it stable _is_ possible, but it's
21.
by zingelshuher 2y ago
Have to disagree with this. The major limiting factor is the software and lack of applications. If there was a killer application it would be much easier to sell. Then the numbers would drive the prices down.
22.
by zingelshuher 2y ago
Why isn't he identified personally? Very likely he is 'contributing' to other projects under different accounts.
23.
by zingelshuher 2y ago
Question, have you seen the improvement after adding the noise? I mean in practice. Asking because intuition sometimes doesn't work.
24.
by zingelshuher 2y ago
I run some tests. Single model of the same size is better than MoE. Single expert out of N is better than model of the same size (i.e. same as expert). 2 experts are better than one. That was on small LLM, not sure if it scales.
25.
by zingelshuher 2y ago
It was inspired by Mixtral 8x7B, of course. I think the same approach, soft to hard MoE, can be used in other domains. Like video/image processing. Would be interesting to take it to extreme, like 4 experts out of 100.
26.
by zingelshuher 2y ago
Similar MoE implementation was on GitHub for a while, since Jan 2024 https://github.com/zxaall/moegpt
27.
by zingelshuher 3y ago
We've seen how it (didn't) with net neutrality. No reasons to think it'll work this time. Especially when current administration is looking in any way to shift attention from migrants packing in US. They will 'show stren
28.
by zingelshuher 3y ago
If grandmother had balls she would be a grandfather.
29.
by zingelshuher 3y ago
Agree, it's better to be rich and healthy than poor and sick.
30.
by zingelshuher 3y ago
Some were even payed for university. Those were times when almost nobody actually could afford to pay, socialism, <beep>

More ›