jimfleming

9 ms·

1.
by jimfleming 6y ago
From the tree docs: > tree has originally been part of TensorFlow and is available as tf.nest. The tf.nest docs can be found here and may be more useful for now: https://www.tensorflow.org/api_docs/python/tf&#x2
2.
by jimfleming 6y ago
I'm aware of that and I've done quite a bit of work on both spiking neural networks and modern deep learning. My point is that those complexities are not required to implement many important functional aspects of the brain: most b
3.
by jimfleming 6y ago
> That needs to work before moving to more complexity. It really depends on what level of abstraction you care to simulate. OpenWorm is working at the physics and cellular level, far below the concept level as in most deep learning resea
4.
by jimfleming 6y ago
This functionality is built into OpenCV[0]. If you're using a reference image (or if you know the lens properties) it doesn't require ML. It's mostly just a matrix transform. [0] https://docs.opencv.org/2.4&#x
5.
by jimfleming 6y ago
I've noticed that many JAX libraries (including those from Google) seem to adopt an object-oriented style more similar to Torch/Keras rather than JAX's functional style demonstrated in modules like jax.experimental.stax. This
6.
by jimfleming 7y ago
I think your conclusions are accurate. For many problems LightGBM or xgboost can often yield decent results in short amounts of time and for many problems that’s sufficient. A lot of the work we do is about pushing the results as far as we
7.
by jimfleming 7y ago
It’s really very problem dependent. I allude to a few low-hanging things in my post above: e.g. feature engineering. Just because neural networks have an easier time learning non-linear feature transformations doesn’t mean its good to ignor
8.
by jimfleming 7y ago
It takes time and a lot of hands-on experience. Many ML teams tend to work on one or just a few tightly coupled project for years. By contrast, we’ve worked on a lot of unique projects with real-world constraints so it gives us a different
9.
by jimfleming 7y ago
More anecdata: we consistently outperform lightgbm, xgboost, random forests, linear models, etc. using neural networks even on smaller datasets. This applies whether we implemented the other algorithms ourselves or simply compared to someo
10.
by jimfleming 7y ago
> Seems indie frameworks in AI can't survive? AI frameworks are enormously complex pieces of software—a mixed bag of GPU acceleration, math utilities, low and high-level implementations of state-of-the-art components, and a (hopeful
11.
by jimfleming 7y ago
The article's demonstration of a counting model is horribly inaccurate to the point where I'm not sure why it was included. Most people see "AI" as being either good at something or not. There's little nuance such a
12.
by jimfleming 7y ago
To those that are upvoting this and previous Swift + TF announcements: What are you excited about, specifically? Why Swift? Why not Julia? Is it the syntax? Types? Compilation? Performance? Community? I like Swift and all but our ML/DL
13.
by jimfleming 7y ago
This is sort of true. Birds are significantly more energy efficient than planes for some kinds of flight. Birds are also able to perform maneuvers that planes cannot such as landing on a branch. For transportation we just don't care ab
14.
by jimfleming 7y ago
My point is that DQN is pretty far removed from the biological equivalent. It's impressive and useful but the main reason it succeeded was not because of some deep insight from neuroscience but because it scaled well (or at least bette
15.
by jimfleming 7y ago
Hyperparameter optimization (including architectures) is not really meta-learning. Meta-learning, also known as "learning to learn", is more like MAML[0], RL2[1], L2RL[2], etc. 0. https://arxiv.org/abs/1703.03
16.
by jimfleming 7y ago
> I believe his thinking was already proven, and going for emulating and integrating the known algorithms that the brain uses is the fastest way to reach AGI. That's a stretch considering AGI has not yet been created by DeepMind or
17.
by jimfleming 7y ago
> Has this been put into practice in any AI to date? Meta-learning is an active subfield of research in machine learning. Gödel machines may overlap with program synthesis as well which is another subfield. > Does TensorFlow count as
18.
by jimfleming 7y ago
Highway connections are actually used heavily in NLP in combination with LSTMs.
19.
by jimfleming 8y ago
The paper[0] is less vague than the article. To put this in terms of reinforcement learning: 1. Sample actions from a random policy distribution. 2. Fit an inverse model with supervised learning from this data. Inverse models learn to map c
20.
by jimfleming 8y ago
A common misconception is that self-driving car companies (outside of a few smaller startups) are using RL to drive the car. They are not. They use deep learning for perception systems which produce tangible outputs that can be processed by
21.
by jimfleming 8y ago
Numenta is probably the most direct / well-known example of what you're asking. They get mixed reactions from the community (some fair, some not). DeepMind occasionally publishes crossover papers from neuroscience (e.g. grid cells
22.
by jimfleming 8y ago
Hopefully this leads to better temporal priors for machine learning models. Sequence encoding frequently uses position coding[0] but the typical approaches (e.g. low frequency sin waves) are fairly naive. In reinforcement learning we have (
23.
by jimfleming 8y ago
In addition, multiple works[0][1] have discovered that grid cell representations arise from regularized recurrent networks when provided relative inputs to predict absolute outputs. [0] https://arxiv.org/abs/1803.07770
24.
by jimfleming 8y ago
That's really cool and interesting. There may be a deeper meaning to what you saw but it would be a rather surprising result if there was conscious awareness of these structures since there does not appear to be any mechanisms for that
25.
by jimfleming 8y ago
I have experience designing these kinds of computer vision systems for other applications. What you describe is very doable. I'd start by isolating each component you want to identify and track. Then break that list down into multiple
26.
by jimfleming 8y ago
Yep, people post long-form content to Twitter because there's engagement there. Literally no other reason because it is a terrible medium for reading and writing long-form content like this. But that level of engagement is rare on prop
27.
by jimfleming 8y ago
I agree with your point and that's probably not the primary audience for VR. A more likely audience is the millions of people who, for example, played Call of Duty together last night. Most of those players were connected remotely via
28.
by jimfleming 8y ago
From the article: > A good estimation of p(x) makes it possible to efficiently complete many downstream tasks: sample unobserved but realistic new data points (data generation), predict the rareness of future events (density estimation),
29.
by jimfleming 8y ago
The previous version has a PDF[0] but you're right, the current version shows none. [0] https://arxiv.org/pdf/1805.11851v1.pdf
30.
by jimfleming 8y ago
I think you're referring to this paper: "Emergence of grid-like representations by training recurrent neural networks to perform spatial localization". https://arxiv.org/abs/1803.07770 It appears to be f

More ›