simonblanke

4 ms·

1.
Show HN: Optimization Strategies for Deep Learning with Hyperactive(nbviewer.jupyter.org)
3 points by simonblanke 5y ago|0 comments
2.
by simonblanke 6y ago
I am glad you like it. Hyperband relies on early stopping of the model. So it is something you would do inside the objective function by yourself. I have plans to add some helper functions for early stopping in the future.
3.
by simonblanke 6y ago
Thank you very much :-) yes you can put anything you want into the search space. Even pandas dataframes, numpy arrays or classes. Here is an example: https://github.com/SimonBlanke/Hyperactive/blob/master&#x2
4.
by simonblanke 6y ago
Check out the Neural Architecture Search Tutorial here: https://nbviewer.jupyter.org/github/SimonBlanke/hyperactive-... Neural Architecture Search is just one of many optimization applications you can work on with
5.
Hyperactive – Easy Neural Architecture Search for Deep Learning in Python(github.com)
13 points by simonblanke 6y ago|6 comments
6.
by simonblanke 6y ago
Yes it is quite easy to switch algorithms via the "gpr" parameter. You just have to write a wrapper class. I am currently working on a repository that discusses how to do that in detail: https://github.com/SimonBla
7.
by simonblanke 6y ago
Yes Hill Climbing + some of its variants are featured in this package.
8.
by simonblanke 6y ago
Unfortunately NLopt cannot be included into Gradient-Free-Optimizers. But i would like to implement multiple algorithms that are also preset in NLopt. I think it will help my understanding of optimization algorithms if i implement them myse
9.
by simonblanke 6y ago
Your parallel computing approach sounds intriguing! Could you provide an example script? I would like to look into this. If you like you could open an issue as a feature request and provide a code snipped there.
10.
by simonblanke 6y ago
I thought about a table in the readme that shows some kind of metric for each optimizer that describes its performance. I will look into that.
11.
by simonblanke 6y ago
I will look into this algorithm. Thanks for the suggestion. I have some basic explanations of the optimization techniques and their parameters in a separate repository: https://github.com/SimonBlanke/optimization-tutori
12.
by simonblanke 6y ago
Two very interesting questions! I should work soon on a comparison of Hyperactive and GFO to other packages. If some important features are missing, maybe i could add them. I will also look into Dlib. If you like you can open an official is
13.
by simonblanke 6y ago
Gradient-Free-Optimizers has a type of genetic algorithm: Evolution Strategy
14.
by simonblanke 6y ago
Evolution strategy is included. The "Covariance matrix adaptation" is for making this algorithm work for continuous search spaces. But gradient-free-optimizers has discrete search space.
15.
by simonblanke 6y ago
Yeah this is a warning from sklearns gaussian process regressor. Sklearn probably wants you to increase the "n_restarts_optimizer"-parameter of the gpr, but from my experience this warning does not correlate with bad results from
16.
by simonblanke 6y ago
You could be right. I must confess, that i have a (probably) very narrow understanding of typical optimization problems. Most of the objective functions i optimize have machine learning algorithms in it (to optimizer hyperparameters). Depen
17.
by simonblanke 6y ago
I am currently working on the Nelder-Mead algorithm. I did not realize that it is that popular. This gives me motivation to implement it soon ;-) If there are more "must have"-algorithms you could open an issue in Gradient-Free-Op
18.
by simonblanke 6y ago
Okay that is interesting. You could realize that by making a restricted area in the search space by returning np.nan in the objective function for those cases. Gradient-Free-Optimizers can handle np.nan and np.inf just fine. Maybe you could
19.
by simonblanke 6y ago
I had the same excitement for tpot! That project was the reason i started creating mine.
20.
by simonblanke 6y ago
Gradient-Free-Optimizers is a lightweight optimization package that serves as a backend for Hyperactive: https://github.com/SimonBlanke/Hyperactive Hyperactive can do parallel computing with multiprocessing or joblib,
21.
by simonblanke 6y ago
Also: Gradient-Free-Optimizers is basically just the optimization-backend for a much larger project of mine: https://github.com/SimonBlanke/Hyperactive
22.
by simonblanke 6y ago
I think Random Restart Hill Climbing is good if your objective function evaluates fast (simple functions). It does not get stuck in local optima and also does local search very well. Bayesian Optimization is very good if your objective func
23.
by simonblanke 6y ago
I am currently working on the Nelder-Mead optimiser. I will also look into "BOBYQA". I am always searching for interesting algorithms that my users need. If you have more suggestions you can open an issue in the repository.
24.
by simonblanke 6y ago
I am glad you like my project. The search space is not continuous. But you can make the steps as small as you want. Like: np.arange(0, 10, 0.00001) I am not sure i understand your second question. The entire search space is always a N-dimen
25.
by simonblanke 6y ago
I just saw, that my project got a lot of new stars on github. A surprise to be sure, but a welcome one.
26.
Show HN: Neural Architecture Search via Bayesian Optimization(nbviewer.jupyter.org)
2 points by simonblanke 6y ago|0 comments
27.
Optimize any Python function with modern algorithms in numerical search spaces(github.com)
6 points by simonblanke 6y ago|0 comments