slashcom

4 ms·

1.
by slashcom 7y ago
There’s lots of work on distillation, smaller models, approximations, etc. People already have simpler forms of these running on smartphones. Models seem to be growing faster than we can make them small though :D
2.
by slashcom 7y ago
A DGX-2 will do just fine.
3.
by slashcom 7y ago
Quite a few of my colleagues came back Monday the 27th. I imagine many stay longer since the trip is onerous.
4.
by slashcom 7y ago
http://nsaphra.github.io/post/hands/ This one perhaps
5.
by slashcom 7y ago
fp16 saves a lot of memory and is worth doing. I've not had trouble fine tuning all these models with fp16.
6.
by slashcom 7y ago
Zipf’s law
7.
by slashcom 7y ago
Links to papers: https://arxiv.org/abs/1811.00207 - Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset https://arxiv.org/abs/1811.01241 - Wizard of Wikipedia: Knowledg
8.
by slashcom 7y ago
From the paper: "$50,000 was divided among the human participants based on their performance to incentivize them to play their best. Each player was guaranteed a minimum of $0.40 per hand for participating, but this could increase to a
9.
by slashcom 7y ago
That should tell you how grad students are paid.
10.
by slashcom 7y ago
I mean, that’s the thing though. Undergrad in state tuition plus R&B for NCSU and UTexas (programs I got in state tuition for) is at 9k and 10k per year with 5% interest for undergraduate, and more like 6-8% for MS students. You can be
11.
by slashcom 7y ago
Indeed it may somewhat be a result of financial illiteracy, but today's young people are significantly better educated than previous generations ( https://en.wikipedia.org/wiki/Educational_attainment_in_the_... ). W
12.
by slashcom 7y ago
That is the actual plot of the SheWork episode in Broad City.
13.
by slashcom 7y ago
Yann LeCun did not, otherwise he’d be a coauthor. As it is, this was a collaboration between NYU and Facebook AI Research, with multiple authors working at both institutions.
14.
by slashcom 7y ago
I don’t know man, 20% of your annual income would be seen as a sizeable fine. That’s 20% of their yearly profits, and it wipes out most of their earnings for Q1.
15.
by slashcom 7y ago
And replaced by residual connections in transformers, which are absolutely dominating LSTMs now.
16.
by slashcom 8y ago
Interestingly, we have 3 in North America: East, West and Texas. https://en.m.wikipedia.org/wiki/Texas_Interconnection
17.
by slashcom 8y ago
It’s indeed a very high trials to success ratio. Again though, there’s enough papers preceding this one that you could have good confidence in the effort. Another thing that helps is orgs like OpenAI have their own servers, rather than rent
18.
by slashcom 8y ago
There’s a natural way to parallelize these models so that using 128 GPUs is the same as a 128x batch size. You can similarly simulate 128x batch size by accumulated gradients before backpropping. So you can test on just one or a few GPUs be
19.
by slashcom 8y ago
Top of the document says use a ballpoint pen, which doesn't smear and can't be erased like a pencil.
20.
by slashcom 8y ago
An infinitely sized 2 layer NN is universal in the same way a Turing machine is universal — sure you can write any program; God help you if you try.
21.
by slashcom 8y ago
So Russia ays they were forced to develop it after Trump withdrew from the nuclear treaty. 2 months is some impressive R&D time, huh?
22.
by slashcom 8y ago
An easier way to understand it is in the context morphology: word prefixes and suffixes mean things, and words have common roots. For example, polymorphism could be decomposed into poly-morph-ism. Antidisestablishmentarianism, which is unli
23.
by slashcom 8y ago
Picking which items to show you in your newsfeed; making recommendations for places based on your friends; automatically generating blind-friendly captions for uploaded photos (read the hover text for some of your friends’ photos); suggesti
24.
by slashcom 8y ago
Buy a dumb terminal. A very cheap, lightweight laptop that can handle a web browser, ssh, and ideally drive an external monitor or two. Then rent time on a cloud service of your choice. Deep learning is very power hungry. Pretending you can
25.
by slashcom 8y ago
Either this was a distraction/rogue American sympathizer, or the Trump Administration has jumped the shark in its “America first” behaviors.
26.
by slashcom 8y ago
For the record, good old fashioned bag of words representations (tf-idf, LDA, LSA) still provide useful representations for documents. Obviously we hope to do better, but recently people act like there's no way of turning a document in
27.
by slashcom 8y ago
My understanding was that Amazon, until recently, intentionally hasn’t turned any profit. You don’t have to pay taxes if you don’t make any money...
28.
by slashcom 9y ago
This is a subfield called anomaly detection.
29.
by slashcom 9y ago
Wait but, the batch size is 8x bigger for the TPU? That's not a fair comparison; increasing batch size always speeds things up...
30.
by slashcom 9y ago
Alas, the "secret sauce" is already freely available: http://waifu2x.udp.jp/ . The only real difference is the (very, very appreciable) effort to make it commercialized software. Anybody who cares enough to disasse

More ›