8 ms·
Facebook open-sources deep-learning modules
- UXDork 12y agoDoes anyone know what Facebook is using the AI for?
- raverbashing 12y agoBasically everything in FB has one main goal: Ads Secondary goals might be: Spam detection and extracting info from the graph/pictures, etc FB identify people from your Social Graph in Pictures, for example
- pkfrank 12y agoAlso, identifying trends in users engaging in predatory behavior, etc.
- WillNotDownvote 12y ago"Basically everything in FB has one main goal: Ads" Is this actually true, or a glib answer? Ads certainly don't feel like much of the FB experience, nowhere near the level of Google, or even Twitter. They're negligible, really. Whereas shaping the feed and identifying people in pics, etc, is central to the FB experience.
- raverbashing 12y agoNot necessarily Ads in the traditional sense, but FB also sells content reach (Promoted Content) and targeting Shaping the feed is certainly one part of it. But I guess that part of the success of FB is that they don't make it, as you said, not make the ads feel as much part of the experience.
- kylestlb 12y agoSomeone with more context can answer this better but I would guess that selling behavioral data to ad agencies is a thing?
- samolang 12y agoAgreed. You can't show ads unless you have users. Optimizing advertisements is definitely second fiddle to optimizing the user experience.
- dragonwriter 12y ago> Ads certainly don't feel like much of the FB experience, nowhere near the level of Google, or even Twitter. Facebook creates an artificial distinction between labeled ads (which are, of course, obvious ads) and posts with paid reach (which are also ads, but presented as normal content), exactly for the purpose of maximizing the quantity of ads while minimizing the impression of content being dominated by ads.
- WillNotDownvote 12y agoRight, so ads don't feel like much of the FB experience. My point is that FB seems much, much more focused on creating an engaging user experience than it does on ads, so it seems wrong to say that their deep learning efforts have ads as their one main goal.
- dragonwriter 12y ago> Right, so ads don't feel like much of the FB experience. Right, but the issue was the claim that with FB everything is about ads, not the UX feels like it is about ads. Making ads not feel like as much of the FB experience as they are is, itself, about ads -- and, particularly, is about maximizing the degree to which the FB experience is actually about ads.
- WillNotDownvote 12y ago"Making ads not feel like as much of the FB experience as they are is, itself, about ads..." Well, we just disagree here. I see it from exactly the other end. That the main FB goal is to engage users, and minimizing the (necessary, but grudgingly so) ads is about that.
- unclebucknasty 12y agoI think the answer is both-and vs. either-or. FB's bait is user engagement. So, applying machine learning to the goal of engaging users makes sense. But, to monetize that user-engagement they, of course, use ads. Applying machine learning directly there makes sense too. Beyond that, it's just semantics. Yes, one could accurately say that everything FB does is about ads because, ultimately, they are a for-profit company which derives its revenue from advertising. In that way, the ultimate goal of all of its activity is aimed at generating ad revenue. But, that doesn't mean that every single activity or bit of technology that supports that activity is directly tied to driving ad revenue. That is to say, that if FB featured no advertising but, instead, charged users, they would still likely deploy machine-learning to user-engagement and, thus, customer-retention.
- deleted 12y ago[deleted]
- deleted 12y ago[deleted]
- choppaface 12y agoCategorization is critical to boosting precision in information retrieval tasks. Deep networks for cropped object recognition are an important source of (inferred) content categories (especially when other context, e.g. photo captions, are not useful). While a good image categorizer would be very useful for ad targeting and auctions, the technology can also boost precision for news feed, graph search, etc., which improves user experience.
- amelius 12y agoThe software runs on Nvidia hardware. Not sure if all their servers are fitted with one or more GPU cards... I think it is more likely that this is just research software.
- raverbashing 12y agoWell, it depends, you can definitely have servers with NVidia hardware, and you don't need to have all of them with it. (Amazon offers GPGPU servers on EC2 for example)
- bhouston 12y agoAmazon only offers very costly Tesla and Quadro cards on their GPGPUs. Super expensive. For scale, if you do not need the extra memory on the Tesla cards, what you want is commodity NVIDIA GeForce cards that are a fraction of the cost.
- polskibus 12y agoStandard cards are handicapped for double precision operations though (1/8 speed afaik).
- p1esk 12y agoYou don't need double precision for typical DL tasks, such as training a convnet.
- raverbashing 12y agoDouble precision is overrated :) (at least for ML)
- Houshalter 12y agoDeep neural networks are usually trained on GPUs which can provide a huge speed up, and is much cheaper. However they can easily be run on CPUs after being trained. I don't see why they wouldn't have access to GPUs in production though.
- anantzoid 12y agoTo further personalise the user feed.
- vonnik 12y agoThis is a recruiting move. Facebook is building its DL team, and to do that, they need to grow the community using their tools.
- ghayes 12y agoAnd yet we should applaud the move. I will always appreciate a company moving proprietary technology into the public sphere.
- kylebrown 12y agoNow they just need to release the datasets as well. The datasets are far more important than the algos.
- publicfig 12y agoI see no issue with open sourcing tools to recruit developers. Honestly, that's one of the best methods I can think of.
- vonnik 12y agoI'm not against it. It's a great way to recruit devs. Just wanted to point out that they would likely be recruiting from the community.
- contingencies 12y agoPaper with proper summary: http://arxiv.org/abs/1412.7580 http://arxiv.org/abs/1412.7580
- ajtulloch 12y agoThis paper describes one part of the fbcunn release (the fast convolution layers implemented via FFT, with the source available at https://github.com/facebook/fbcunn/tree/master/src/cuda/fft https://github.com/facebook/fbcunn/tree/master/src/cuda/fft). There's a lot more in fbcunn if you want to check it out.
- Hydraulix989 12y agoIn my experience, the fully connected layers are the bottleneck. The other issue was the alternating compute-heavy convolution and the IO-heavy pooling. I'm curious how this FFT implementation stacks up against cuDNN (what's the speedup like for just the convolutional layers? and then what's the overall speedup like?).
- ajtulloch 12y agohttp://arxiv.org/pdf/1412.7580v2.pdf http://arxiv.org/pdf/1412.7580v2.pdf compares the convolutional implementation with the cuDNN layers. For the FC layers, it's just CuBLAS `sgemm`.
- nightski 12y agoToo bad it is Torch. Working in the lua environment is not enjoyable at all. Every error becomes a long procedure of looking deep into the source of the framework since there is no type information or stack traces to go off whatsoever. You are constantly guessing about the shape of the data or what parameters are supposed to look like.
- smhx 12y agothere's clear and well stack-traces at the lua level. there's great debugger (in fblualib, called fb.debugger). there's an inbuilt profiler into LuaJIT. too bad you've had a bad experience.
- nightski 12y agoThanks I am always willing to give it another shot with more tools.
- vonnik 12y agotry http://deeplearning4j.org/ http://deeplearning4j.org/ they just partnered with canonical: http://insights.ubuntu.com/?p=9373 http://insights.ubuntu.com/?p=9373
- frik 12y agoWow great. I dreamed of an AI platform based on LuaJIT a few years ago. Torch and FB's deep-learning module sound amazing!
- technologia 12y agoWe've been using Torch for a while at work, really appreciate Facebook doing this. Some of us had a hard time adjusting to Lua, but it was well worth it.
- chubot 12y agoI'm curious what your team experience was with Lua. IIRC, Lua is even more liberal than Python in that if you mistype a variable name, it will silently give you 'nil' rather than throwing an exception. Also, the prototype system is a little wonky. It seems overly flexible and makes you do too much work yourself, which would make it hard on a large team all getting started at once. If Torch gets popular I wonder if something like TypeScript or Facebook's own Flow would start to make sense.
- psykotic 12y ago> IIRC, Lua is even more liberal than Python in that if you mistype a variable name, it will silently give you 'nil' rather than throwing an exception. Like most things in Lua, that is easy to customize. You can just add an __index metamethod to environment tables. There are various 'strict' modules which do just that.
- technologia 12y agoThats what we love about Lua, it gives us control of the side effects.
- technologia 12y agoSo some of our team were a bit disoriented with the lack of reference counting and for others it was their first foray into functional programming. Our team first picked up the Little Schemer and then jumped directly to lua.
- muser 12y agoCould you provide some steps how you could bring everyone one board? We are still having python/R migration issues.
- DannyBee 12y agoSadly, this has facebook's new, horrible patent clause. For those who aren't aware, it says 1. If facebook sues you, and you counterclaim over patents (whether about software or not), you will lose rights under all these patent grants. So essentially you can't defend yourself. This is different than the typical apache style patent grant, which instead would say "if you sue me over patents in apache licensed software x, you lose rights to software x" (IE it's limited to software, and limited to the thing you sued over) 2. It terminates if you challenge the validity of any facebook patent in any way. So no shitty software patent busting!
- deleted 12y ago[deleted]
- jawns 12y agoHere's a link to the patent clause: https://github.com/facebook/fbcunn/blob/master/PATENTS https://github.com/facebook/fbcunn/blob/master/PATENTS DannyBee, I'm having a hard time envisioning what sort of scenario might trigger this, and what the repercussions might be. Any chance you could explain?
- j_s 12y agoThe most interesting part to me is whether or not such claim is related to the Software - so any patent suit against any software, product or service of Facebook or any of its subsidiaries or affiliates terminates patent license to this software... seems like they went for the nuclear defense!
- SixSigma 12y ago> The license granted hereunder will terminate, automatically and without notice, for anyone that makes any claim ... by ... assertion or other action ... alleging .. that any right in any patent claim of Facebook is invalid or unenforceable. That sounds pretty chilling considering that could mean that anything any of your employees say about any other Facebook patent could wreck your business, if you choose to use Facebook code from Github.
- DannyBee 12y ago
- bpodgursky 12y agoI appreciate that Facebook open-sources their libraries, but they have an awful habit of dumping source code on Github and then continuing to develop it internally without pushing those changes back out. For example, Facebook open-sourced Thrift, then proceeded to develop it internally, and recently released out a fork of thrift, not merging anything back into the original tree. Likewise, they open-sourced Corona a few years ago, their Hadoop fork, and I'm pretty sure that branch has been abandoned too. So I'm really hesitant to start using any projects out of Facebook unless they develop some processes to actually maintain these projects.
- amelius 12y agoFrom the article: > We hope that these high-quality code releases will be a catalyst to the research community and we will continue to update them from time to time.
- deleted 12y ago[deleted]
- peterhunt 12y agoThat was years ago. Things have changed around there: https://github.com/facebook https://github.com/facebook
- choppaface 12y agoAgree that their stewardship has been lacking (especially in comparison to say Apache, Docker, etc), but for research code this is a pretty nice release in terms of the scope of problems solved. For research it's more important to have an implementation of what worked (and what /didn't/) than a polished solution + well-kept community because a lot of the current code will probably get thrown away eventually anyways. The service here is research done rather than software provided.
- tdaltonc 12y agoThey've done a good job keeping up PlanOut. https://github.com/facebook/planout https://github.com/facebook/planout