6 ms·

What is a good algorithm-to-purpose map for ML beginners? Looking for something like "Algo X is good for making predictions when your data looks like Y," etc.

by ehayes 4y ago

remram 4y ago
This maybe? https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html https://scikit-learn.org/stable/tutorial/machine_learning_ma...
- bagels 4y ago
  This is what I would have replied with too.
sandGorgon 4y ago
xgboost. always xgboost. it will scale all the way from college kaggle problems (where it is the top performer almost always) to cloud scale. xgboost is one of the few frameworks supported by Sagemaker, etc
xdfgh1112 4y ago
Vowpal Wabbit is not the best anymore, but it is incredibly simple. You train it by piping text files in, then pipe your input into it for predictions.
- FreakLegion 4y ago
  Tsk to whoever downvoted this. Simple linear models are indeed the right starting point for most new projects while you come to grips with your data. In some cases you can stop there or apply a quick nonlinearization like Fastfood to get good, snappy, and generally debuggable results for very little RAM. In other cases you move on to decision tree ensembles or neural networks, depending on whether you already have features or need those to be learned, too. Either way this ratchets up the complexity and resource requirements. Decision trees in particular tend to have bloated implementations. I still use XGBoost or Scikit for training, but wrote my own library to translate the models into a more efficient format (~95% smaller than Scikit) and have thread-safe inference.
  - xdfgh1112 4y ago
    Thanks for the reply! What is Fastfood though? I can't find anything on Google.
    - FreakLegion 4y ago
      Of course. The paper is at https://arxiv.org/abs/1408.3060 https://arxiv.org/abs/1408.3060. > Our method applies to any translation invariant and any dot-product kernel, such as the popular RBF kernels and polynomial kernels. We prove that the approximation is unbiased and has low variance. Experiments show that we achieve similar accuracy to full kernel expansions and Random Kitchen Sinks while being 100x faster and using 1000x less memory. These improvements, especially in terms of memory usage, make kernel methods more practical for applications that have large training sets and/or require real-time prediction. Sadly Fastfood didn't quite make it into Scikit[1], but did land in scikit-learn-extra[2]. 1. https://github.com/scikit-learn/scikit-learn/pull/3665 https://github.com/scikit-learn/scikit-learn/pull/3665. A shame, Scikit's equivalents scale very poorly. 2. https://scikit-learn-extra.readthedocs.io/en/stable/generated/sklearn_extra.kernel_approximation.Fastfood.html https://scikit-learn-extra.readthedocs.io/en/stable/generate...

What is a good algorithm-to-purpose map for ML beginners? Looking for something like "Algo X is good for making predictions when your data looks like Y," etc.

by ehayes 4y ago

What is a good algorithm-to-purpose map for ML beginners? Looking for something like "Algo X is good for making predictions when your data looks like Y," etc.

remram 4y ago

This maybe? https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html https://scikit-learn.org/stable/tutorial/machine_learning_ma...

bagels 4y ago
This is what I would have replied with too.

sandGorgon 4y ago

xgboost. always xgboost. it will scale all the way from college kaggle problems (where it is the top performer almost always) to cloud scale. xgboost is one of the few frameworks supported by Sagemaker, etc

xdfgh1112 4y ago

Vowpal Wabbit is not the best anymore, but it is incredibly simple. You train it by piping text files in, then pipe your input into it for predictions.

FreakLegion 4y ago
Tsk to whoever downvoted this. Simple linear models are indeed the right starting point for most new projects while you come to grips with your data. In some cases you can stop there or apply a quick nonlinearization like Fastfood to get good, snappy, and generally debuggable results for very little RAM. In other cases you move on to decision tree ensembles or neural networks, depending on whether you already have features or need those to be learned, too. Either way this ratchets up the complexity and resource requirements. Decision trees in particular tend to have bloated implementations. I still use XGBoost or Scikit for training, but wrote my own library to translate the models into a more efficient format (~95% smaller than Scikit) and have thread-safe inference.
- xdfgh1112 4y ago
  Thanks for the reply! What is Fastfood though? I can't find anything on Google.
  - FreakLegion 4y ago
    Of course. The paper is at https://arxiv.org/abs/1408.3060 https://arxiv.org/abs/1408.3060. > Our method applies to any translation invariant and any dot-product kernel, such as the popular RBF kernels and polynomial kernels. We prove that the approximation is unbiased and has low variance. Experiments show that we achieve similar accuracy to full kernel expansions and Random Kitchen Sinks while being 100x faster and using 1000x less memory. These improvements, especially in terms of memory usage, make kernel methods more practical for applications that have large training sets and/or require real-time prediction. Sadly Fastfood didn't quite make it into Scikit[1], but did land in scikit-learn-extra[2]. 1. https://github.com/scikit-learn/scikit-learn/pull/3665 https://github.com/scikit-learn/scikit-learn/pull/3665. A shame, Scikit's equivalents scale very poorly. 2. https://scikit-learn-extra.readthedocs.io/en/stable/generated/sklearn_extra.kernel_approximation.Fastfood.html https://scikit-learn-extra.readthedocs.io/en/stable/generate...