frankling_

6 ms·

1.
by frankling_ 1mo ago
A question out of honest curiosity: Would you say that it is unknown whether any humans have subjective experiences, or just any humans outside yourself? I've seen the former view being expressed previously and can't wrap my min
2.
by frankling_ 2mo ago
Mere mortals can only aspire to this blissful and perfect alignment between normative stance and immediate self-interest.
3.
by frankling_ 4mo ago
There are also these somewhat classic-looking bitmap terminal fonts large enough for modern displays: https://github.com/B2HDPI/B2HDPI
4.
by frankling_ 5mo ago
Here are a few made by upscaling and then manually cleaning up classic fonts: https://github.com/B2HDPI/B2HDPI The glyph coverage is enough for most programming languages; missing glyphs just fall back to a pixelized l
5.
by frankling_ 6mo ago
The recent announcement to reject review articles and position papers already smelled like a shift towards a more "opinionated" stance, and this move smells worse. The vacuum that arXiv originally filled was one of a glorified PDF
6.
by frankling_ 8mo ago
Wow, they finally figured out that it's actually not A, it's B—at least if C.
7.
by frankling_ 2y ago
If you're able to predict the future with 50% accuracy, you should start filling out some lottery tickets.
8.
by frankling_ 2y ago
The convolution is approximated via a form of sampling with additional bookkeeping at each encountered branch. How well that scales for deeply branching programs depends on the probabilities of the branches and the diversity in the output o
9.
by frankling_ 2y ago
In DiscoGrad, smoothing would be applied by adding Gaussian noise with some configurable variance to x and running the program on those x's. The gradient would then be calculated based on the branch condition's derivative wrt. x (
10.
by frankling_ 2y ago
Yeah, those tricks are highly related to what we do, the main difference being that we don't require a priori information about the distributions involved in the program. Instead, we compute a density estimation of the distribution of
11.
by frankling_ 2y ago
Well, the most common ML problems can be expressed as optimization over smooth functions (or reformulated that way manually). We might have to convince the ML world that branches do matter :) On the other hand, there are gradient-free appro
12.
by frankling_ 2y ago
Great point, the sigmoid approximation works well for certain problems and that's in fact what I used in the exploratory papers that lead to this work. The downsides are the lack of a clear interpretation how the original program and i
13.
by frankling_ 2y ago
That's right, plain autodiff just ignores branches. Our canonical "why is this even needed" example is a program like "if (x >= 0) return 1; else return 0", x being the input. The autodiff derivative of this is z
14.
by frankling_ 2y ago
Not super closely related: the polytope model (to the degree I'm familiar with it) is used as a representation that facilitates optimization of loop nests. That's optimization in the sense of finding an efficient program. DiscoGra
15.
by frankling_ 2y ago
We're doing something less expensive: essentially, the overall gradient is computed based on certain statistics based on the branch condition and its derivatives when a branch is encountered. We mention neural networks because DiscoGra
16.
by frankling_ 2y ago
We actually did some preliminary experiments with Taichi hoping to benefit from the GPU parallelization. I think generally, the world of autodiff tooling is in very good shape. For anything non-exotic, we just use JAX or Torch to get things
17.
by frankling_ 2y ago
Enzyme is traditional, but super duper optimized, autodiff, that is, it returns the partial derivatives for one path taken through the program, ignoring other branches. DiscoGrad captures the effects of alternative branches. What's spe
18.
by frankling_ 2y ago
Thanks for the kind words! We'd be super happy if this work gets picked up, whether in a commercial context or not. We were thinking of some disco ball-based logo (among some other designs). With this encouragement, there'll proba
19.
by frankling_ 2y ago
The key point is that Ceres requires derivatives, which can come from manually derived formulae, approximations via finite differences, or autodiff ( http://ceres-solver.org/derivatives.html ). DiscoGrad doesn't do the o
20.
by frankling_ 2y ago
I agree with that intuition. In our experience, it's easiest to see gains over other optimization techniques when the program is "branch-wise smooth and non-constant". Then, we get the full benefits of exact autodiff gradient
21.
by frankling_ 2y ago
Yep, that's exactly it. The smoothness can either come from randomness in the program itself (then the objective function is asymptotically smooth and DiscoGrad estimates the gradient of that smooth function), or the smoothness can be
22.
Show HN: Boldly go where Gradient Descent has never gone before with DiscoGrad(github.com)
232 points by frankling_ 2y ago|66 comments
23.
Show HN: DiscoGrad – Automatically differentiate across branches in C++ programs(github.com)
2 points by frankling_ 2y ago|0 comments
24.
by frankling_ 3y ago
That "Bach style generation" is on point. In fact, it's "O Haupt voll Blut und Wunden": https://www.youtube.com/watch?v=Fpqd1gCzLN4
25.
by frankling_ 4y ago
This is the problem dealt with by the field of parallel and distributed simulation [1]. The two basic approaches are to either block the receiver until you can guarantee that a received message will not be displaced by one with earlier time
26.
by frankling_ 4y ago
What funding programme are you thinking about? The most common type of project proposal is funded at much lower rates: https://www.dfg.de/en/dfg_profile/facts_figures/statistics/p...
27.
by frankling_ 5y ago
True, I guess my perspective is biased towards CS. In that field, what I've seen in terms of editing on the part of a publisher amounted to requests for minor formatting changes (to be done by the authors) and insertion of a copyright
28.
by frankling_ 5y ago
1) For the status of being, e.g., editor-in-chief of a well-known journal. 2) I'd expect that they would be opposed, but firing their editors for it would make for interesting headlines.
29.
by frankling_ 5y ago
That's true, and I'm not sure how these fees can be justified.
30.
by frankling_ 5y ago
That's a good point, editors and reviewers come and go. Ideally, the newcomers would by default also be attached to "Independent Open Nature", until it becomes irrational to submit to Nature itself.

More ›