apl

7 ms·

1.
by apl 4y ago
Neural architecture search (NAS) is a thing! But it's almost exclusively based on meta-gradients. Again, wouldn't put my money on GAs ever outperforming gradient-based methods again.
2.
by apl 4y ago
An AlexNet/ResNet-type moment may be in the cards for GAs, but I wouldn't put any money on it. They're typically only one better than brute-force. This can be good enough (and is certainly easy to implement), but if you can g
3.
by apl 5y ago
Several hints here are severely outdated. For instance, never train a model in end-to-end FP16. Use mixed precision, either via native TF/PyTorch or as a freebie when using TF32 on A100s. This’ll ensure that only suitable ops are run w
4.
by apl 6y ago
Hard disagree. V100s are a perfectly valid comparison point. They're usually what's available at scale (on AWS, in private clusters, etc.) because nobody's rolled out enough A100s at this point. If you look at any paper from
5.
by apl 6y ago
You can almost 1:1 translate this by swapping "tf" and "torch". No need to use nn.Conv2d -- there's a functional API for all these layers: https://pytorch.org/docs/master/nn.functional.html
6.
by apl 6y ago
> gradient descent no longer has to be written by hand Nobody's been writing derivatives by hand for 5+ years. All major frameworks (PyTorch, Tensorflow, MXNet, autodiff, Chainer, Theano, etc.) have decent to great automatic differe
7.
by apl 7y ago
Mainly because it is genuinely exhausting for any medical practitioner. That lots of patients "enjoy" googling symptoms and coming up with far-fetched self-diagnoses is a given. But couple that with the perceived intellectual supe
8.
by apl 7y ago
For this particular problem, Mask R-CNN would have been the way to go -- it spits out instances as opposed to just deciding, for each pixel, to which class it belongs. Or an SSD (if we don't care about the mask at all).
9.
by apl 7y ago
There's many perspectives on everything. Deep ConvNets, for instance, can be expressed as a continuously evolving ODE. Here's a fantastic paper on this view: https://papers.nips.cc/paper/7892-neural-ordinary-d
10.
by apl 7y ago
Yeah, that's right. Doing too little proofreading with HN comments...
11.
by apl 7y ago
Their applications are tabular data (for which MLPs have never been the method of choice) and MNIST (which I could classify at 85% with a rusty nail), so it's not super impressive. NNs and the associated toolkit shine with structured h
12.
by apl 7y ago
This represents the absolute worst kind of science journalism, completely devoid of context and domain knowledge. Virtually every definitive statement in here is wrong. Their explanation of spiking alone is a complete disaster. Out of all t
13.
by apl 7y ago
Running Windows is perfectly fine; the major libraries for GPU-accelerated autodiff and networks (CUDNN with Pytorch or Tensorflow) have great support nowadays. It's the AMD GPU that remains essentially useless, as of 2019. If you want
14.
by apl 8y ago
That's not quite how the process works. These papers go through multiple (> 2) revisions. At any iteration, there's ample opportunity for updating references. This applies double given how long the Cueva/Wei work has been
15.
by apl 8y ago
It's certainly an interesting paper, but there's a bit of publication weirdness at play here. In October '17, Cueva & Wei put out a(n anonymous) paper that recapitulates the core result almost exactly -- that training a r
16.
by apl 9y ago
> I've noticed that when I get a blink/flinch response from > something (usually some sand or a bug hitting my face when > I'm on the bike), it feels like I blink just a split second > before the thing hit me. We
17.
by apl 9y ago
As someone who works in visual neuroscience, this article's a tough read. Lots of statements that are semi-accurate at best. 1) Eyes don't work like cameras; there's no real "exposure" phase as such (even though the
18.
by apl 9y ago
Thanks for coding this up. The code in the blog post is, at best, incredibly misleading.
19.
by apl 9y ago
I only skimmed the notebook, but the code looks fairly inadequate. The pandas portion, for instance, treats the data frame as a dumb array and critically ignores grouping functionality which should offer a tremendous speedup. Moreover, for
20.
by apl 9y ago
We didn't have to reverse-engineer the CPU. Also, CPUs are rationally designed; we have zero guarantees that an evolved brain follows any principles at all. Every brain region may be highly specialised to a single algorithm or task wit
21.
by apl 10y ago
This came out two days ago and uses what they call intelligent synapses to improve multi-task learning: https://arxiv.org/abs/1703.04200 Seems closely related.
22.
by apl 10y ago
Try this: https://www.ncbi.nlm.nih.gov/pubmed/26234212 I guess the key lesson is -- don't rely on a single approach, because its limitations may well lead you astray. Applies to connectomics, physiology, modelling
23.
by apl 10y ago
Even in these apparently simple feedforward sensory networks, connectomics haven't been the anticipated panacea. There's been a flurry of follow-up papers to Takemura et al., essentially refuting the suggested model. Turns out, ev
24.
by apl 10y ago
Cortical layers are in no way equivalent to DNN layers.
25.
by apl 10y ago
That'd be interesting, but once you reach an orchestra you're generally beyond technical training. Players are expected to know how to play; individual training is rare and focuses on style aspects more than technical things.
26.
by apl 11y ago
The key bottleneck with all connectome projects is still reconstruction. There's on the order of a hundred billion neurones in the human brain; they're connected via 1,000-10,000 times the number of synapses. Current machine learn
27.
by apl 11y ago
Examples of this type are not a good guideline for policy decisions, almost by definition. For every Marshall, there's a million cranks whose ideas are rejected by the establishment and rightfully so . At the outset, all these ideas a
28.
by apl 11y ago
A simple explanation for this: cutting carbs is tricky and requires a lot of careful planning etc., increasing the likelihood that people actually stay under their maintenance caloric intake. It doesn't really matter how you do it. A
29.
by apl 11y ago
The Julia FFI for Python is absolutely excellent -- calling a particular Python library from Julia takes very, very little effort. As a language for scientific programming, Julia is way ahead of Python. So I'm not sure if this particul
30.
by apl 11y ago
People are trying to accomplish precisely that: it's called eLife. Wellcome Trust, HHMI and the Max-Planck-Society are backing it; adoption is moderately successful. So there's hope!

More ›