mcabbott

18 ms·

1.
by mcabbott 1y ago
Covariant and contravariant indices would be the formal terms. I'm not really sure whether I've seen "upstairs" written down. Sub/superscript... strike me as the typographical terms, not the meaning? Like $x_\mathrm
2.
by mcabbott 1y ago
This doesn't really help with programming, but in physics it's traditional to use up- and down-stairs indices, which makes the distinction you want very clear. If input x has components xⁿ, and output f(x) components fᵐ, then the
3.
by mcabbott 2y ago
That's what Reactant.jl is aiming to do: Take the LLVM code from Julia and pipe to to XLA, where it can benefit from all the investment which make Jax fast. Same author as the more mature Enzyme.jl, which is AD done at the LLVM level.
4.
by mcabbott 2y ago
This is also how it works in Julia, where macros digest notation for einsum-like operations before compile-time. In fact the linked file's explanatory comment: (einsum (A i j) (B i k) (C k j)) results in the the updates
5.
by mcabbott 3y ago
using Unitful: m; foo(2.0m) With the above definition, this will give DimensionError: 2.0 m and 1.0 are not dimensionally compatible. Probably this means you should define `foo(x::Number) = x + oneunit(x)` to respect the Number in
6.
by mcabbott 3y ago
Yes, it obeys enough algebraic laws that calling it a derivative is useful. But I don't think there's any underlying notion of small changes to something continuous. It is not the slope of some smooth function.
7.
by mcabbott 3y ago
Or perhaps better "foreigner", legal alien, someone from over the border. The alien derivative is an operation which moves you from one sector to another. A bit like crossing a branch cut.
8.
by mcabbott 4y ago
I presume so. Note that this pseudo-code is very nearly valid Julia code (with my package): @tullio (min) C[i,j] := A[i,k] + B[k,j]
9.
by mcabbott 4y ago
Yes, although that seems like the easy half of this, making sure `struct NewNum <: AbstractFloat` defines everything. There aren't yet tools for this but they are easy to imagine. And missing methods do give errors. The hard half se
10.
by mcabbott 4y ago
The unicode epsilon isn't in the public API, it's describing the 3rd positional argument. This was added recently, and for some reason the PR (1840) didn't fix the docs, which is bad. The Optimisers.jl version has an explanat
11.
by mcabbott 4y ago
The struct's field name is `eta`, but this is an internal detail. Its constructor takes a positional-only argument, no public name. The greek letter is used in the documentation. And the reason is that every optimiser's documentat
12.
by mcabbott 4y ago
> > Learning rate (η): Amount by which gradients are discounted before updating the weights. > so this is already explicit to anyone who reads the documentation. The quibble in the post is about the named parameter. As far as I can
13.
by mcabbott 4y ago
On the GPU, operations which look like matrix multiplication are indeed quite slow. As you say there are many tricks, and Tullio doesn't (right now) know them. For operations which are more like broadcasting it does much better. On the
14.
by mcabbott 5y ago
There is a discussion here: https://discourse.julialang.org/t/rust-julia-comparison-post... So far, a bug fix improves it 4700 min -> 150 min, changing the type stored -> 20 min, further optimisations -> 15 mi
15.
by mcabbott 5y ago
Thanks for the references. We're trying to write up a paper about all this work, and (to us) it seems obvious that differential geometry is the right framework. What reverse mode AD talks informally about as "gradients" are c
16.
by mcabbott 5y ago
Some points have simple answers: * Support for using @copied s.x = 1 where s is an immutable struct. I believe the standard package for this is this: https://github.com/jw3126/Setfield.jl#usage * Currently macros have
17.
by mcabbott 5y ago
The big Alcoa press is 50_000 tons, not lbs. There are slightly larger ones now, 60, 75ktons. (22 tons is, of course, force that any rail yard could produce a century ago.)
18.
by mcabbott 6y ago
The Julia translations of these might look as follows: using OffsetArrays cash_flows = zeros(0:max_time) altitude = zeros(Bool, -100:100) for I in eachindex(cash_flows) some_function(cash_flows[I])
19.
by mcabbott 6y ago
The goal of Tullio.jl is certainly not to beat MKL at its own game. But it can still be 4-5 times faster on some permutedims + matmul operations, by fusing them. Besides handling operations which aren't standard kernels, handling weird
20.
by mcabbott 6y ago
Sure! The desire for fancier notation is from more complicated examples, the kind where you write elaborate comments to explain why you're permuting 3rd & 4th dim of B to line up with C and part of D.
21.
by mcabbott 6y ago
> until next time you have to read that But how will this tool help? If you put the sample you used in a comment, then your confusion is wondering if that's in sync... and whether the input is like what you wrote. IMO the sweet spot
22.
by mcabbott 6y ago
Right, NamedDims.jl is the one closest what the article describes. Very simple and light-weight, plays well with others. There is a small zoo of packages attaching also labels along the indices, alla python's xarray. Perhaps it's
23.
by mcabbott 6y ago
You can do this: @_ images |> filter(size(_,1)>100, __) .|> resize(__, 100,100) .|> crop(__, 25,25,75,75) with Underscores.jl. Not exactly `images.first.resize(100, 100).crop(25, 25, 75, 75)` anymore, to point o
24.
by mcabbott 6y ago
At 2:39 it's saying that "doing good" and "doing well" are both correct, but mean different things: superman is doing good deeds, and scrooge is doing very well. But what to make of "I am good" isn't
25.
by mcabbott 7y ago
Those are all good synonyms for "got" in the context of shooting at things. But none of the others already has a strong meaning in the context of aircraft, and this other meaning does create some confusion, which is why many speak
26.
by mcabbott 7y ago
There are a bunch of packages exploring ideas around labelled dimensions, although we are still far from the vision of caring as little as we do about registers: * https://github.com/JuliaArrays/AxisArrays.jl * https:
27.
by mcabbott 7y ago
That seems strange, if I'm reading it correctly. In the author's packages, integers which label dimensions etc. have gradient `nothing`, but arrays which happen to contain integers do not signal anything: julia> Zygote.gr
28.
by mcabbott 7y ago
A tensor product of two matrices is just this, with 4 loose ends: ---A--- ---B--- A_ab B_cd = (A⊗B)_acbd. If a,c=1...N then you can think of (ac) as being an index with N^2 possible values, which is what the double line now mea
29.
by mcabbott 7y ago
The thing about RPN is that it worked well as a sort-of "spoken not written" language. It's a clear and usable way to give instructions one-at-a-time to a calculator with a 1-line display. But written out on paper, it's
30.
by mcabbott 8y ago
Maybe I phrased that poorly: you can indeed write the loops yourself (without a speed penalty) as advertised, to do things which are hard/ugly/impossible in numpy. But very often you can do this without dropping all the way down t

More ›