6 ms·
This is actually a really exciting development to me. (Note, what is exciting is the "optometrist algorithm" from the paper [1] not necessarily googles involvem
by abefetterman 9y ago
This is actually a really exciting development to me. (Note, what is exciting is the "optometrist algorithm" from the paper [1] not necessarily googles involvement as pitched in the guardian). Typically a day of shots would need to be programmed out in advance, typically scanning over one dimension (out of hundreds) at a time. It would then take at least a week to analyze the results and create an updated research plan. The result is poor utilization of each experiment in optimizing performance. The 50% reduction in losses is a big deal for Tri Alpha.
I can see this being coupled with simulations as well to understand sources of systematic errors, create better simulations which can then be used as a stronger source of truth for "offline" (computation-only) experiments.
The biggest challenge of course becomes interpreting the results. So you got better performance, what parameters really made a difference and why? But that is at least a more tractable problem than "how do we make this better in the first place?"
[1] http://www.nature.com/articles/s41598-017-06645-7 http://www.nature.com/articles/s41598-017-06645-7
- jlarocco 9y agoAs a complete outsider, I don't understand what's special about the "optometrist algorithm." As described in the Nature article it's just hill climbing using humans as the evaluation function. Isn't it basically the same thing they were already doing but more granular?
- abefetterman 9y agoBasically nobody was using automated gradient descent / etc because of the proclivity of these algorithms to get stuck on a boundary. The problem is the boundaries are not well defined. One example might be a catastrophic instability. If it gets triggered it has the potential to damage the machine. But the exact parameters in which the instability occurs are not well known. So with this algorithm you mix the best of both worlds: the human can guide away from the areas where we think instabilities are, the machine can do it's optimization thing. It's pretty simple overall but enables a big shift in how experiments are run. Edit to add: these instabilities often look just like better performance on a shot-to-shot basis, which makes the algos especially tricky. Using a human we could say "this parameter change is just feeding the instability" vs "oh this is interesting go here"
- hyperbovine 9y agoTo be clear: there are no gradients here (right?) This is just 0th order hill-climbing with a human assist.
- ouid 9y agohow does one climb a hill with no gradient? [serious question]
- Govindae 9y agoThe naive way to do calculus. Use secants to approximate the tangent. I think it's called finite difference.
- dmurray 9y agoYou can climb a hill without knowing the gradient, so long as you can compare two points in terms of height. You randomly move in some direction, then compare the new point to the old point, go to whichever of them is higher, and repeat. This sounds like what the experimenters are doing. Perhaps the GP was alluding to "first order hill climbing" as evaluating the gradient in every direction and climbing the steepest one, but the "0th order" version is also usually considered hill climbing and is better for some classes of problem.
- noobermin 9y agoThat is exactly what they're doing. See the section on Exploratory Technique, second to last paragraph. As I said above, the possible innovation here is they can change midstream the criteria one uses to decide what is a "better shot".
- ouid 9y agois it picking a new configuration at random, or does it still have to be "close" to the last configuration?
- 9y ago
- amelius 9y agoPerhaps a stupid question, but why can't the whole experiment be run as a simulation?
- noobermin 9y agoThe system is fundamentally 6^N dimensional with N~10^23.
- sixdimensional 9y agoYeah, 6^N dimensions are fun! ;)
- semi-extrinsic 9y agoI suppose you meant 6*N? Which is a lot better, but still intractable. And anyway, we don't exactly resolve molecules in e.g. turbulent flow simulations, yet they still take tens, even hundreds of millions of CPU-hours.
- noobermin 9y ago6*N, yes. Pretty bad mistake there. But yes, even if you don't model every particle and restrict yourself to "parcels" of fluid like in most simulations, you still have a very difficult problem.
- igravious 9y agoOkay, this is really showing my ignorance but why 6? You start off with 4 (3 space plus one time (ignoring 11-dimensional space-time)) and add which dimensions exactly? Can the individual interactions between wave/particles be reduced to 2 dimensions? Aren't they going to interact along the whole range of forces they exert: gravitational, weak, electromagnetic, strong?
- kilotaras 9y ago3 dimensions for position + 3 dimensions for speed
- marco_salvatori 9y agoThough this work may seem exciting, there is an existing, respected body of work available on how to mathematically structure a search over a large parameter space and how to mathematically interpret experimental responses. That body of work is a subset of applied statistics called design of experiments. It helps scientists avoid the common failures that result from doing exactly what was done here, random space exploration and non rigourous evaluation of results. For this to be exciting I would expect some indication as to how this method extends and enhances the existing science of experimental methods and the trade offs involved with using their method. I dont see that.
- Libbum 9y agoInteresting. Are you able to provide some links to decent resources on this topic?
- adrianratnapala 9y agoIt would not surprise me if high-tech companies are inventing new, useful things in this field. In my career as first a scientist and then an engineer, I've found very few practical users of highly technical experimental design theory, and all of them were in industry. These algorithms move about intelligently along all dimensions of some search space, whereas in the lab we prefered to turn just one knob at a time. One reason is that the algorithms are optimally seraching for "known unknowns" -- that is they assume they roughly understand the problem. The lab is a world of unkown unknowns where the more plodding, understandable protocols tend to be safer. But in industry, some problems are of the known-unknowns type. And experiment runs can burn up seriously expensive hardware time. So it makes sense for fusion researchers and cloud-computing giants a like to invent new practical ways to optimise searches. Besides, optimising searches is what Googlers are for.