6 ms·

Machine Learning Library for C++

76 points by optiminimalist 13y ago

JacobiX 13y ago
The library supports some very useful algorithms. Both supervised and unsupervised ones. But I can't use it for our commercial non GPL projects.
- aeroevan 13y ago
  If LGPL works for you, you may want to look into http://www.mlpack.org/ http://www.mlpack.org/ I'm not really sure how they compare since I haven't really used either library, but there does seem to be some overlap.
  - JacobiX 13y ago
    Thanks, it's not as complete as Shark. But it is nonetheless very interesting.
- mindcrime 13y ago
  Could you have the ML part done as a standalone process and not actually linked into your other project code? That is, process some data "out of band" so to speak, and write it out to a file, which is then used by "your stuff"? Or communicate with the ML part over a socket? Either approach would let you use this code without requiring the other parts of your project to be licensed under the GPL. Just a thought...
  - JacobiX 13y ago
    Yes I think It is possible to use the library as a standalone process. It will introduce some little overhead. More importantly the code will be harder to debug and maintain.
- jhartmann 13y ago
  Flame Suit On / Rant Mode On I actually really don't understand why anyone uses GPL for a library. I've been doing open source for a long long time, and love the GPL. I have code in the Linux kernel, and believe free software AND open source software are great solutions to very real problems in software engineering. Having open code just gives people more options, and I firmly believe it will win over time as far as quality is concerned. I just think only providing libraries to other GPL code is stupid. It just limits the usefulness of the software. LGPL is great here, you get the core changes contributed back to your library from a greater group of people and everyone wins. Limiting a library to GPL means a large population can not use your code, those writing applications that can't be licensed under the GPL. Limiting choice is BAD. The whole reason you should be creating and using free software and OSS is to not weld the hood shut. GPL should be for applications, LGPL just limits choices for libraries. Down with the GPL for libraries!!! Flame Suit Off / Rant Mode Off
  - kruhft 13y ago
    Some people believe in software freedom more than you do.
    - forrestthewoods 13y ago
      Some people define "freedom" with 10 pages of restrictions of which the exact interpretation is continuously debated years after release. Other people consider that the opposite of freedom.
      - ddfu 13y ago
        Some wouldn't mind their open-source software integrated into high-frequency trading systems and military munitions, and others either object on moral terms or wish to be paid fairly for such lucrative use. Do you think programmers shouldn't have the choice to decide whether their software may be used to kill people or cause the next flash crash? GPLv3 allows programmers to share their software while making sure big corporations and defense contractors steer clear.
  - 13y ago
hackinthebochs 13y ago
This looks awesome. I've been itching to try out some ideas I have after having gone through Bishop's book, but I've been hesitant to write the algorithms from scratch. Now I'll have to decide between learning matlab or a library such as this.
- c0g 13y ago
  Speaking as a PhD student in machine learning- Implement the algorithm yourself, first, in Python+Numpy. The only reason I feel comfortable with Gaussian Processes and SVMs is due to writing code to solve them manually. Once you're happy with the basics, and can test your ideas with code you intimately understand, optimise for speed by using a library like this.
  - misiti3780 13y ago
    Implementing the SVM from scratch was time consuming - no?
    - iskander 13y ago
      The only tricky part would be writing a quadratic solver. Alternatives: either solve a linear SVM using gradient descent (simpler to write), or offload the core of the algorithm to an existing solver like cvxopt. edit: For an example of using cvxopt, check out http://www.mblondel.org/journal/2010/09/19/support-vector-machines-in-python/ http://www.mblondel.org/journal/2010/09/19/support-vector-ma...
      - droz 13y ago
        Another approach is to implement Platt's SMO: http://en.wikipedia.org/wiki/Sequential_minimal_optimization http://en.wikipedia.org/wiki/Sequential_minimal_optimization
      - misiti3780 13y ago
        cool - thanks
    - 13y ago
swalsh 13y ago
Does anyone know if there's a good ML library written in C# or F#
- danieldk 13y ago
  libsvm and liblinear are nice libraries for training SVM and linear classifiers. There seem to be two ports of liblinear for C# (I haven't tried them): http://www.csie.ntu.edu.tw/~cjlin/libsvm/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/
- profquail 13y ago
  I don't know of any written in F#...maybe I'll have to write one :) You might also ask on the fsharp-opensource mailing list, maybe someone has an F# ML library I don't know about: https://groups.google.com/forum/?fromgroups#!forum/fsharp-opensource https://groups.google.com/forum/?fromgroups#!forum/fsharp-op...
- jnazario 13y ago
  how about numl? http://numl.net/ http://numl.net/ folks on SO also like WEKA run through IKVM (a Java to .NET converter): http://stackoverflow.com/questions/1624060/machine-learning-libraries-in-c-sharp http://stackoverflow.com/questions/1624060/machine-learning-...
pilooch 13y ago
Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 https://news.ycombinator.com/item?id=5613797 ). Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to maintain. This makes me skeptical of performances and maintenance of the code, but it would be fairer to try it first. Still very impressed.
- pmelendez 13y ago
  May I ask how do you become a C++ ML programmer? It is not an usual position but one I would definitely be interested in pursuing
  - pilooch 13y ago
    Sure, I started as a researcher 14 years ago, then drifted to what I thought was a sweet spot then, half-research / half-programmer. I say 'sweet spot' because many applications did require both the academic and the applied background at the time, so for the sake of thrilling applications, it was worth 'downgrading' to pure engineering work when needed. Now I believe the game has changed a bit, coursera and others are infusing the minds of engineers with highly technical knowledge far more rapidly than before. Typically I am astonished at the number of implementations of deep learning techniques (Shark does include some, AFAIK). My past experience is that I had write many AI algorithms myself because I could not find any suitable, free and/or open implementations (or other researchers would not share theirs ;) ).
    - pmelendez 13y ago
      Thanks! That's very inspiring :) I worked in a similar position for a short time long time ago but after that I hadn't been able to find something similar. Good thing to see that people can find those sweet spots! :D It's very true what you said about all the implementations available now. Although, for most algorithms I tend to try to implement them myself as a learning experience, maybe the biggest exception is standard SVM since it is kinda tricky but even for that there are some online algorithms that are easy to implement. Thanks for sharing your experience!

The library supports some very useful algorithms. Both supervised and unsupervised ones. But I can't use it for our commercial non GPL projects.

aeroevan 13y ago
If LGPL works for you, you may want to look into http://www.mlpack.org/ http://www.mlpack.org/ I'm not really sure how they compare since I haven't really used either library, but there does seem to be some overlap.
- JacobiX 13y ago
  Thanks, it's not as complete as Shark. But it is nonetheless very interesting.
mindcrime 13y ago
Could you have the ML part done as a standalone process and not actually linked into your other project code? That is, process some data "out of band" so to speak, and write it out to a file, which is then used by "your stuff"? Or communicate with the ML part over a socket? Either approach would let you use this code without requiring the other parts of your project to be licensed under the GPL. Just a thought...
- JacobiX 13y ago
  Yes I think It is possible to use the library as a standalone process. It will introduce some little overhead. More importantly the code will be harder to debug and maintain.
jhartmann 13y ago
Flame Suit On / Rant Mode On I actually really don't understand why anyone uses GPL for a library. I've been doing open source for a long long time, and love the GPL. I have code in the Linux kernel, and believe free software AND open source software are great solutions to very real problems in software engineering. Having open code just gives people more options, and I firmly believe it will win over time as far as quality is concerned. I just think only providing libraries to other GPL code is stupid. It just limits the usefulness of the software. LGPL is great here, you get the core changes contributed back to your library from a greater group of people and everyone wins. Limiting a library to GPL means a large population can not use your code, those writing applications that can't be licensed under the GPL. Limiting choice is BAD. The whole reason you should be creating and using free software and OSS is to not weld the hood shut. GPL should be for applications, LGPL just limits choices for libraries. Down with the GPL for libraries!!! Flame Suit Off / Rant Mode Off
- kruhft 13y ago
  Some people believe in software freedom more than you do.
  - forrestthewoods 13y ago
    Some people define "freedom" with 10 pages of restrictions of which the exact interpretation is continuously debated years after release. Other people consider that the opposite of freedom.
    - ddfu 13y ago
      Some wouldn't mind their open-source software integrated into high-frequency trading systems and military munitions, and others either object on moral terms or wish to be paid fairly for such lucrative use. Do you think programmers shouldn't have the choice to decide whether their software may be used to kill people or cause the next flash crash? GPLv3 allows programmers to share their software while making sure big corporations and defense contractors steer clear.
- 13y ago

hackinthebochs 13y ago

This looks awesome. I've been itching to try out some ideas I have after having gone through Bishop's book, but I've been hesitant to write the algorithms from scratch. Now I'll have to decide between learning matlab or a library such as this.

c0g 13y ago
Speaking as a PhD student in machine learning- Implement the algorithm yourself, first, in Python+Numpy. The only reason I feel comfortable with Gaussian Processes and SVMs is due to writing code to solve them manually. Once you're happy with the basics, and can test your ideas with code you intimately understand, optimise for speed by using a library like this.
- misiti3780 13y ago
  Implementing the SVM from scratch was time consuming - no?
  - iskander 13y ago
    The only tricky part would be writing a quadratic solver. Alternatives: either solve a linear SVM using gradient descent (simpler to write), or offload the core of the algorithm to an existing solver like cvxopt. edit: For an example of using cvxopt, check out http://www.mblondel.org/journal/2010/09/19/support-vector-machines-in-python/ http://www.mblondel.org/journal/2010/09/19/support-vector-ma...
    - droz 13y ago
      Another approach is to implement Platt's SMO: http://en.wikipedia.org/wiki/Sequential_minimal_optimization http://en.wikipedia.org/wiki/Sequential_minimal_optimization
    - misiti3780 13y ago
      cool - thanks
  - 13y ago

swalsh 13y ago

Does anyone know if there's a good ML library written in C# or F#

danieldk 13y ago
libsvm and liblinear are nice libraries for training SVM and linear classifiers. There seem to be two ports of liblinear for C# (I haven't tried them): http://www.csie.ntu.edu.tw/~cjlin/libsvm/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/
profquail 13y ago
I don't know of any written in F#...maybe I'll have to write one :) You might also ask on the fsharp-opensource mailing list, maybe someone has an F# ML library I don't know about: https://groups.google.com/forum/?fromgroups#!forum/fsharp-opensource https://groups.google.com/forum/?fromgroups#!forum/fsharp-op...
jnazario 13y ago
how about numl? http://numl.net/ http://numl.net/ folks on SO also like WEKA run through IKVM (a Java to .NET converter): http://stackoverflow.com/questions/1624060/machine-learning-libraries-in-c-sharp http://stackoverflow.com/questions/1624060/machine-learning-...

pilooch 13y ago

Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the pain (someone explains why, see comment https://news.ycombinator.com/item?id=5613797 https://news.ycombinator.com/item?id=5613797 ). Second, it contains a version of Blas and ublas as well as LBFGS and more, much more, coded from scratch as it seems. This seems too much for an ML library, and a lot to maintain. This makes me skeptical of performances and maintenance of the code, but it would be fairer to try it first. Still very impressed.

pmelendez 13y ago
May I ask how do you become a C++ ML programmer? It is not an usual position but one I would definitely be interested in pursuing
- pilooch 13y ago
  Sure, I started as a researcher 14 years ago, then drifted to what I thought was a sweet spot then, half-research / half-programmer. I say 'sweet spot' because many applications did require both the academic and the applied background at the time, so for the sake of thrilling applications, it was worth 'downgrading' to pure engineering work when needed. Now I believe the game has changed a bit, coursera and others are infusing the minds of engineers with highly technical knowledge far more rapidly than before. Typically I am astonished at the number of implementations of deep learning techniques (Shark does include some, AFAIK). My past experience is that I had write many AI algorithms myself because I could not find any suitable, free and/or open implementations (or other researchers would not share theirs ;) ).
  - pmelendez 13y ago
    Thanks! That's very inspiring :) I worked in a similar position for a short time long time ago but after that I hadn't been able to find something similar. Good thing to see that people can find those sweet spots! :D It's very true what you said about all the implementations available now. Although, for most algorithms I tend to try to implement them myself as a learning experience, maybe the biggest exception is standard SVM since it is kinda tricky but even for that there are some online algorithms that are easy to implement. Thanks for sharing your experience!