4 ms·

This example makes it really clear what's going on. Could someone translate it to do reverse automatic differentiation? That's the one I never quite understand.

Animats 11y ago
You mean integration? That's much harder, but has been done automatically. Symbolic differentiation is easy, because you can just keep blindly applying a set of rewrite rules until none of them apply. That process converges on a unique result. Symbolic integration doesn't converge in that way. More strategy is required, and you're not guaranteed a closed form solution. Mathematica has a good symbolic integrator.
- cperciva 11y ago
  Symbolic integration doesn't converge in that way. More strategy is required, and you're not guaranteed a closed form solution. However, if a closed-form solution exists which can be expressed in terms of the operations + - * / exp log, then it is guaranteed to be found.
- tome 11y ago
  Reverse mode AD is another algorithm for calculating derivatives https://en.wikipedia.org/wiki/Automatic_differentiation#Reverse_accumulation
kxyvr 11y ago
The short answer is no, for a couple of reasons. First, you have to distinguish whether you want the antiderivative or the integral. In the first case, the problem with finding the antiderivative is that it's not unique. For example, the antiderivative of x can be 0.5 * x^2 or 0.5 * x^2 + 1 or 0.5 * x^2 + 2, etc. Basically, taking derivatives can lose information and we don't know how to put that information back. In the second case, we have the integral, but that creates problems because the integral really needs to have a domain defined to make sense. Basically, over what values are we integrating? In 1-D we can specify this with a bunch of intervals, but in 2 and higher dimensions, it becomes difficult. In fact, this is partially why we have meshing tools like Cubit or gmesh. Yes, we use them with finite element methods, but really, at it's core, we need them to break up a domain because we want to integrate the pieces. Anyway, with enough information or on a simple enough setup, both can actually be done, but the general case isn't super well defined, which is why the simple answer is no.
- dllthomas 11y ago
  I read the parent (as does 'tome in a neighboring thread) to be asking about reverse-mode automatic differentiation (where what is explained in TFA is forward-mode), not about computing antiderivatives.
  - kxyvr 11y ago
    Well, crap, you're right. The other comment screwed me up. Apologies!

Animats 11y ago

You mean integration? That's much harder, but has been done automatically. Symbolic differentiation is easy, because you can just keep blindly applying a set of rewrite rules until none of them apply. That process converges on a unique result. Symbolic integration doesn't converge in that way. More strategy is required, and you're not guaranteed a closed form solution. Mathematica has a good symbolic integrator.

cperciva 11y ago
Symbolic integration doesn't converge in that way. More strategy is required, and you're not guaranteed a closed form solution. However, if a closed-form solution exists which can be expressed in terms of the operations + - * / exp log, then it is guaranteed to be found.
tome 11y ago
Reverse mode AD is another algorithm for calculating derivatives https://en.wikipedia.org/wiki/Automatic_differentiation#Reverse_accumulation

kxyvr 11y ago

The short answer is no, for a couple of reasons. First, you have to distinguish whether you want the antiderivative or the integral. In the first case, the problem with finding the antiderivative is that it's not unique. For example, the antiderivative of x can be 0.5 * x^2 or 0.5 * x^2 + 1 or 0.5 * x^2 + 2, etc. Basically, taking derivatives can lose information and we don't know how to put that information back. In the second case, we have the integral, but that creates problems because the integral really needs to have a domain defined to make sense. Basically, over what values are we integrating? In 1-D we can specify this with a bunch of intervals, but in 2 and higher dimensions, it becomes difficult. In fact, this is partially why we have meshing tools like Cubit or gmesh. Yes, we use them with finite element methods, but really, at it's core, we need them to break up a domain because we want to integrate the pieces. Anyway, with enough information or on a simple enough setup, both can actually be done, but the general case isn't super well defined, which is why the simple answer is no.

dllthomas 11y ago
I read the parent (as does 'tome in a neighboring thread) to be asking about reverse-mode automatic differentiation (where what is explained in TFA is forward-mode), not about computing antiderivatives.
- kxyvr 11y ago
  Well, crap, you're right. The other comment screwed me up. Apologies!