6 ms·
Hard disagree, to the point where I'm not sure there's not some confusion at play here. I find using indices and summations generally confusing and hard to rea
by meken 4d ago
Hard disagree, to the point where I'm not sure there's not some confusion at play here.
I find using indices and summations generally confusing and hard to read/follow/understand. To me, it's so much simpler to draw out the computational graph, derive the gradient with a single element (so I'm in agreement with you guys here?), see how that applies the same to every element the same, then scale up to the vectorized/matrix version.
> and you need to memorize a bunch of new rules to apply it
There's nothing to memorize though..?
Is there something called "matrix calculus" that's different than just drawing the computational graph and deriving the gradient in the way I described..?
Like, is this "matrix calculus"?
- https://cs231n.github.io/optimization-2/ https://cs231n.github.io/optimization-2/
If so, what is there to memorize here?
- srean 4d agoNot memorize but one may need to derive the result. Consider differentiating the matrix logarithm of a complex matrix.
- pdhborges 4d agoOP was talking about this: https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf Look into some examples of the section 2.4 "Derivatives of Matrices, Vectors and Scalar Forms"