atschantz

7 ms·

1.
by atschantz 8y ago
It's remarkable how much this still rings true today. The idea that we should try and understand life as a battle against the second law of thermodynamics has had a significant influence on many of today's great thinkers - such as
2.
by atschantz 8y ago
I previously tried to sign up via a Google account but it required a 'company Google account' - I was slightly confused about the reasoning behind this.
3.
by atschantz 8y ago
I used Notion for a significant period but ended up switching to Nuclino [1] - which is identical in many respects, but without the various add-ons that are unnecessary if you're working with text/images. I've found it to be
4.
by atschantz 8y ago
I mean it's difficult to 'observe' gradient descent, there are no characteristic properties that you can identify without specifying the relative objective function. But most of the process theories from computational neurosc
5.
by atschantz 8y ago
As a general answer, the theory suggests that organisms maximize a quantity known as model evidence, which is just a way of saying 'how much evidence does some data provide for my model of the world?' There are two complementary w
6.
by atschantz 8y ago
I think you might be right, a quote from Friston on the relationship (in reference to belief propagation): "We turn to the equivalent message passing for continuous variables, which transpires to be predictive coding [...]" It cou
7.
by atschantz 8y ago
Some attempts have been made in the form of Bayesian model reduction [1]. The idea is to 'carve' out the structure of your model using free energy minimization. [1] https://arxiv.org/abs/1805.07092
8.
by atschantz 8y ago
The idea is that you learn a model by calculating the derivative of free energy with respect to your model parameters.
9.
by atschantz 8y ago
It's worth noting that 'free energy' is just the 'evidence lower bound' that is optimized by a large portion of today's machine learning algorithms (i.e. variational auto-encoders). It's also worth noting
10.
by atschantz 8y ago
The particular study cited in the article is [1], however for a more general review of the links to reinforcement learning [2]. [1] https://www.biologicalpsychiatrycnni.org/article/S2451-9022(... [2] https:/&#x2
11.
by atschantz 8y ago
In terms of the free energy 'principle', it makes no predictions about how free energy minimized. But there have been multiple process theories suggested, most notably predictive coding (which is a dominant paradigm in neuroscienc
12.
by atschantz 8y ago
Well, a significant portion of empirical neuroscience works under the assumption that parts of the brain operate according to a predictive coding scheme, and there are countless studies that support this notion. As predictive coding is a fo
13.
by atschantz 8y ago
There is a large overlap, for instance, the popular VIME exploration algorithm [1] uses part of the free energy objective function. However, free energy isn't a theory of curiosity per se, its posed as description of self-organisation.
14.
by atschantz 8y ago
The 'revolutionary' aspect is the suggestion that a single celled organism is also doing variational inference. Or, more accurately, can be described as such.