5 ms·

Show HN: FUSE filesystem for Google Drive written in OCaml

86 points by astrada 13y ago

bsaul 13y ago
OCaml is pretty big in french universities, but could anyone from other countries say how popular this language is in their country (education and corporate sectors) ? I'm quite surprised to see a news about this language on the front page of HN.
- BMarkmann 13y ago
  Probably the most known group using it in the US is Jane Street. https://ocaml.janestreet.com/ https://ocaml.janestreet.com/
- fabriceleal 13y ago
  In Portugal I would say it doesn't exist... But I have knowledge of its usage on some projects by big corporations. It seems that janestreet [1] takes great pride in using OCaml. They have some interesting open source projects in github [2] and bitbucket [3]. And facebook has pfff [4], an "Ocaml API to write static analysis on source code" (such as PHP code, not only OCaml code). (I'm not a facebook or janestreet employee) [1]: http://janestreet.com/ http://janestreet.com/ [2]: https://github.com/janestreet https://github.com/janestreet [3]: https://bitbucket.org/janestreet https://bitbucket.org/janestreet [4]: https://github.com/facebook/pfff https://github.com/facebook/pfff
  - icebraining 13y ago
    We were taught OCaml in the Software Engineering undergraduate course at FCT-UNL, in Portugal, along with Prolog, C and Java.
    - fabriceleal 13y ago
      My bad ... I wasn't aware. Do you know any Portuguese company that uses OCaml?
      - icebraining 13y ago
        No, not really, it seems almost all use Java or .NET languages. Thankfully, I found a Python startup here in Lisbon.
- benesch 13y ago
  I can't speak to other undergraduate programs, but it's taught as the second course in Computer Science (Intro II) at Harvard College [0]. That said, it was chosen because it was less popular than other functional languages (namely Haskell or a Lisp flavor), so everyone was likely to be starting on "equal footing." Take a look at the OCaml success stories [1]. Unison [2] in particular stands out—it's essentially the only two-way rsync tool for Unix, and it's written entirely in OCaml. The only corporation I know of that uses OCaml is Jane Street Capital [3][4]. They've been huge proponents since they switched over, but the movement doesn't appear to have caught on among many other US-based companies. (More complete list [5].) Jane Street recruits fairly heavily from Harvard for internships—presumably because very few schools teach OCaml. [0] http://cs51.seas.harvard.edu/ http://cs51.seas.harvard.edu/ [1] http://caml.inria.fr/about/successes.en.html http://caml.inria.fr/about/successes.en.html [2] http://www.cis.upenn.edu/~bcpierce/unison/ http://www.cis.upenn.edu/~bcpierce/unison/ [3] http://janestreet.com/technology/ http://janestreet.com/technology/ [4] http://janestreet.github.io/ http://janestreet.github.io/ [5] http://ocaml.org/companies.html http://ocaml.org/companies.html
- astrada 13y ago
  In Italy it isn't popular at all, but a couple of friends of mine use OCaml, mostly for hobby projects. And I like the language. It is more advanced than main stream languages I know (C#, Javascript), but, being strict, it is easier (for me) to reason about than Haskell.
- timsally 13y ago
  OCaml sees a lot of use in the applied program analysis community, which has applications to automated reverse engineering and formal verification. A company in this area recently started by CMU assistant professor David Brumley is: http://www.forallsecure.com/mayhem.html http://www.forallsecure.com/mayhem.html.
- munin 13y ago
  you learn it in the PL class at UMD (that everyone takes) and the compilers class (which is an elective) uses it for everything.
- jahewson 13y ago
  F# is a superset of OCaml and both are descendants of ML (ML was created here in Edinburgh). Having worked with Ocaml extensively I would say that the ecosystem around the language is pretty stale: the standard library sucks, as does the documentation for it and important libraries such as "batteries included" are under the GPL, which rules out many uses. Usability is poor: stack traces can be a nightmare to get working properly, the compiler errors are the worst I have ever encountered anywhere (including C++ templates!), there's no standard build chain, the list goes on... I switched to Scala and haven't looked back. Haskell is very popular here in Edinburgh also.
  - jon_smark 13y ago
    You should take a new/better look at OCaml, as most of the information you mentioned is wrong and/or out-of-date. For instance: * F# is not a superset of OCaml. OCaml has a number of powerful features missing from F#, like functors, polymorphic variants, and first-class modules. * I disagree that the ecosystem is "stale". There's lots of recent activity around the language, and you can find libraries for almost everything.
    - gtani 13y ago
      also, there's Real World Ocaml beta, publically available. In this case, beta means manuscript has been commented on by many qualified reviewers already, http://anil.recoil.org/2013/06/17/real-world-ocaml-beta-available.html http://anil.recoil.org/2013/06/17/real-world-ocaml-beta-avai...
- zhemao 13y ago
  I think it's relatively popular in University courses on compilers and programming languages. We use it for our Programming Languages and Translators course (i.e. the course the Dragon Book was written for) here at Columbia University.
- tbrownaw 13y ago
  University of Illinois uses it for CS421 "programming languages and compilers" (or at least they used to, and I'm assuming it won't have changed). We don't use it at work, and I haven't heard any mention of it (or any other functional languages) at the various local programmer groups I recently started going to.
- bfung 13y ago
  University of Illinois teaches it in the Programming Languages course: http://courses.engr.illinois.edu/cs421/mps/index.html http://courses.engr.illinois.edu/cs421/mps/index.html Back in when it was CS 321, the course used bison and ocaml to build a smaller ML language, which helped in understanding ocaml itself!
- mhd 13y ago
  It is a compiled language with some decent enough unix integration, so this isn't the first system-level program I've seen written in it. The rather popular unison file synchronizer is Ocaml, too, if I'm not mistaken. For a while the most popular Linux client for the edonkey peer-to-peer file sharing network was mldonkey. And just recently I stumbled across a minimal PDF viewer called llpp that turned out to be Ocaml, too. You can easily distribute packages without huge dependencies, it's not any worse at concurrency than your usual Unix go-to language C, and the FFI ain't half bad, either. It's also relatively unopinionated about IO, which certainly helps writing servers and system utilities. There's also parsers and some web stuff, but I'll let others talk about this. Just wanted to say a bit about this somewhat surprising niche where the language managed to eke out a bit of a marketing share.
- bcuccioli 13y ago
  It's used in Cornell's intermediate-level, required, notoriously difficult functional programming course.
  - agumonkey 13y ago
    It looks larger than the one I had in France. Happy to have read your comment. offtopic ps: for a minute the staff pictures were CV segmented which was quite original.
- georgemcbay 13y ago
  It isn't widely used here in the USA, in my experience and outside of people who are big fans of ML and derivatives, or those who had some OCaml exposure in college courses, most programmers here seem to be largely unaware of it. I'm aware of it primarily through my exposure to Nicolas Cannasse's Haxe language, the compiler for which is written in OCaml.
- apaprocki 13y ago
  On my team at Bloomberg, we built a non-trivial (roughly 20k loc) DSL -> C++ "transpiler" in OCaml that is in the critical path responsible for building core parts of the infrastructure.
chrissnell 13y ago
I've never used OCaml so I started randomly clicking through your source. I was looking at this file: https://github.com/astrada/google-drive-ocamlfuse/blob/master/src/gaeProxy.ml https://github.com/astrada/google-drive-ocamlfuse/blob/maste... It seems like it's using some GAE node (presumably owned by the author) for something to do with authentication. Would the author care to explain what's going on here?
- astrada 13y ago
  That is a proxy on GAE that makes it easier to complete the OAuth2 flow. The source code of the proxy is here: https://github.com/astrada/gd-ocaml-auth https://github.com/astrada/gd-ocaml-auth. More info about the authorization process: https://github.com/astrada/google-drive-ocamlfuse/wiki/Authorization https://github.com/astrada/google-drive-ocamlfuse/wiki/Autho.... Let me know if you need further info.
  - chrissnell 13y ago
    Fair enough. I can't imagine that most users will want to pass access tokens through your proxy, despite the hassle it saves. I would suggest making proxied auth /not/ the default or at least divulging the use of the proxy in a more prominent fashion in your documentation. Just a suggestion. People get sensitive when it comes to their personal cloud storage.
    - astrada 13y ago
      I see your point. Thanks for your feedback.
rca 13y ago
That's seems like a neat app! How did you handle the concurrent remote-access on the files? Is there a way for the user applications to be warned when you receive an operational transformation? I'm implementing a fuse filesystem for a similar webapp and I thought it would be impossible for an editor that hasn't be designed for that to deal with the concurrent modifications, even if the filesystem can handle it.
- astrada 13y ago
  Conflict resolution is not very sofisticated. You can choose to always take server side version (the default) or client side version (see https://github.com/astrada/google-drive-ocamlfuse/wiki/Configuration https://github.com/astrada/google-drive-ocamlfuse/wiki/Confi...). My main use case is a single user working on the client or on the server. So the scenario is not of a collaborative app.
gregwebs 13y ago
I am using the google drive app for Windows (in a shared folder that my Linux VM sees). When I save a file with Vim it creates a duplicate copy on the GDrive server. I couldn't figure out how to turn off Vim's file saving behaviour, so I will give this a shot and see if it works better.
- gregwebs 13y ago
  works fine with Vim

86 points by astrada 13y ago

bsaul 13y ago

OCaml is pretty big in french universities, but could anyone from other countries say how popular this language is in their country (education and corporate sectors) ? I'm quite surprised to see a news about this language on the front page of HN.

BMarkmann 13y ago
Probably the most known group using it in the US is Jane Street. https://ocaml.janestreet.com/ https://ocaml.janestreet.com/
fabriceleal 13y ago
In Portugal I would say it doesn't exist... But I have knowledge of its usage on some projects by big corporations. It seems that janestreet [1] takes great pride in using OCaml. They have some interesting open source projects in github [2] and bitbucket [3]. And facebook has pfff [4], an "Ocaml API to write static analysis on source code" (such as PHP code, not only OCaml code). (I'm not a facebook or janestreet employee) [1]: http://janestreet.com/ http://janestreet.com/ [2]: https://github.com/janestreet https://github.com/janestreet [3]: https://bitbucket.org/janestreet https://bitbucket.org/janestreet [4]: https://github.com/facebook/pfff https://github.com/facebook/pfff
- icebraining 13y ago
  We were taught OCaml in the Software Engineering undergraduate course at FCT-UNL, in Portugal, along with Prolog, C and Java.
  - fabriceleal 13y ago
    My bad ... I wasn't aware. Do you know any Portuguese company that uses OCaml?
    - icebraining 13y ago
      No, not really, it seems almost all use Java or .NET languages. Thankfully, I found a Python startup here in Lisbon.
benesch 13y ago
I can't speak to other undergraduate programs, but it's taught as the second course in Computer Science (Intro II) at Harvard College [0]. That said, it was chosen because it was less popular than other functional languages (namely Haskell or a Lisp flavor), so everyone was likely to be starting on "equal footing." Take a look at the OCaml success stories [1]. Unison [2] in particular stands out—it's essentially the only two-way rsync tool for Unix, and it's written entirely in OCaml. The only corporation I know of that uses OCaml is Jane Street Capital [3][4]. They've been huge proponents since they switched over, but the movement doesn't appear to have caught on among many other US-based companies. (More complete list [5].) Jane Street recruits fairly heavily from Harvard for internships—presumably because very few schools teach OCaml. [0] http://cs51.seas.harvard.edu/ http://cs51.seas.harvard.edu/ [1] http://caml.inria.fr/about/successes.en.html http://caml.inria.fr/about/successes.en.html [2] http://www.cis.upenn.edu/~bcpierce/unison/ http://www.cis.upenn.edu/~bcpierce/unison/ [3] http://janestreet.com/technology/ http://janestreet.com/technology/ [4] http://janestreet.github.io/ http://janestreet.github.io/ [5] http://ocaml.org/companies.html http://ocaml.org/companies.html
astrada 13y ago
In Italy it isn't popular at all, but a couple of friends of mine use OCaml, mostly for hobby projects. And I like the language. It is more advanced than main stream languages I know (C#, Javascript), but, being strict, it is easier (for me) to reason about than Haskell.
timsally 13y ago
OCaml sees a lot of use in the applied program analysis community, which has applications to automated reverse engineering and formal verification. A company in this area recently started by CMU assistant professor David Brumley is: http://www.forallsecure.com/mayhem.html http://www.forallsecure.com/mayhem.html.
munin 13y ago
you learn it in the PL class at UMD (that everyone takes) and the compilers class (which is an elective) uses it for everything.
jahewson 13y ago
F# is a superset of OCaml and both are descendants of ML (ML was created here in Edinburgh). Having worked with Ocaml extensively I would say that the ecosystem around the language is pretty stale: the standard library sucks, as does the documentation for it and important libraries such as "batteries included" are under the GPL, which rules out many uses. Usability is poor: stack traces can be a nightmare to get working properly, the compiler errors are the worst I have ever encountered anywhere (including C++ templates!), there's no standard build chain, the list goes on... I switched to Scala and haven't looked back. Haskell is very popular here in Edinburgh also.
- jon_smark 13y ago
  You should take a new/better look at OCaml, as most of the information you mentioned is wrong and/or out-of-date. For instance: * F# is not a superset of OCaml. OCaml has a number of powerful features missing from F#, like functors, polymorphic variants, and first-class modules. * I disagree that the ecosystem is "stale". There's lots of recent activity around the language, and you can find libraries for almost everything.
  - gtani 13y ago
    also, there's Real World Ocaml beta, publically available. In this case, beta means manuscript has been commented on by many qualified reviewers already, http://anil.recoil.org/2013/06/17/real-world-ocaml-beta-available.html http://anil.recoil.org/2013/06/17/real-world-ocaml-beta-avai...
zhemao 13y ago
I think it's relatively popular in University courses on compilers and programming languages. We use it for our Programming Languages and Translators course (i.e. the course the Dragon Book was written for) here at Columbia University.
tbrownaw 13y ago
University of Illinois uses it for CS421 "programming languages and compilers" (or at least they used to, and I'm assuming it won't have changed). We don't use it at work, and I haven't heard any mention of it (or any other functional languages) at the various local programmer groups I recently started going to.
bfung 13y ago
University of Illinois teaches it in the Programming Languages course: http://courses.engr.illinois.edu/cs421/mps/index.html http://courses.engr.illinois.edu/cs421/mps/index.html Back in when it was CS 321, the course used bison and ocaml to build a smaller ML language, which helped in understanding ocaml itself!
mhd 13y ago
It is a compiled language with some decent enough unix integration, so this isn't the first system-level program I've seen written in it. The rather popular unison file synchronizer is Ocaml, too, if I'm not mistaken. For a while the most popular Linux client for the edonkey peer-to-peer file sharing network was mldonkey. And just recently I stumbled across a minimal PDF viewer called llpp that turned out to be Ocaml, too. You can easily distribute packages without huge dependencies, it's not any worse at concurrency than your usual Unix go-to language C, and the FFI ain't half bad, either. It's also relatively unopinionated about IO, which certainly helps writing servers and system utilities. There's also parsers and some web stuff, but I'll let others talk about this. Just wanted to say a bit about this somewhat surprising niche where the language managed to eke out a bit of a marketing share.
bcuccioli 13y ago
It's used in Cornell's intermediate-level, required, notoriously difficult functional programming course.
- agumonkey 13y ago
  It looks larger than the one I had in France. Happy to have read your comment. offtopic ps: for a minute the staff pictures were CV segmented which was quite original.
georgemcbay 13y ago
It isn't widely used here in the USA, in my experience and outside of people who are big fans of ML and derivatives, or those who had some OCaml exposure in college courses, most programmers here seem to be largely unaware of it. I'm aware of it primarily through my exposure to Nicolas Cannasse's Haxe language, the compiler for which is written in OCaml.
apaprocki 13y ago
On my team at Bloomberg, we built a non-trivial (roughly 20k loc) DSL -> C++ "transpiler" in OCaml that is in the critical path responsible for building core parts of the infrastructure.

chrissnell 13y ago

I've never used OCaml so I started randomly clicking through your source. I was looking at this file: https://github.com/astrada/google-drive-ocamlfuse/blob/master/src/gaeProxy.ml https://github.com/astrada/google-drive-ocamlfuse/blob/maste... It seems like it's using some GAE node (presumably owned by the author) for something to do with authentication. Would the author care to explain what's going on here?

astrada 13y ago
That is a proxy on GAE that makes it easier to complete the OAuth2 flow. The source code of the proxy is here: https://github.com/astrada/gd-ocaml-auth https://github.com/astrada/gd-ocaml-auth. More info about the authorization process: https://github.com/astrada/google-drive-ocamlfuse/wiki/Authorization https://github.com/astrada/google-drive-ocamlfuse/wiki/Autho.... Let me know if you need further info.
- chrissnell 13y ago
  Fair enough. I can't imagine that most users will want to pass access tokens through your proxy, despite the hassle it saves. I would suggest making proxied auth /not/ the default or at least divulging the use of the proxy in a more prominent fashion in your documentation. Just a suggestion. People get sensitive when it comes to their personal cloud storage.
  - astrada 13y ago
    I see your point. Thanks for your feedback.

rca 13y ago

That's seems like a neat app! How did you handle the concurrent remote-access on the files? Is there a way for the user applications to be warned when you receive an operational transformation? I'm implementing a fuse filesystem for a similar webapp and I thought it would be impossible for an editor that hasn't be designed for that to deal with the concurrent modifications, even if the filesystem can handle it.

astrada 13y ago
Conflict resolution is not very sofisticated. You can choose to always take server side version (the default) or client side version (see https://github.com/astrada/google-drive-ocamlfuse/wiki/Configuration https://github.com/astrada/google-drive-ocamlfuse/wiki/Confi...). My main use case is a single user working on the client or on the server. So the scenario is not of a collaborative app.

gregwebs 13y ago

I am using the google drive app for Windows (in a shared folder that my Linux VM sees). When I save a file with Vim it creates a duplicate copy on the GDrive server. I couldn't figure out how to turn off Vim's file saving behaviour, so I will give this a shot and see if it works better.

gregwebs 13y ago
works fine with Vim