16 ms·

Why Swift for TensorFlow?

212 points by nnd 8y ago

deleted 8y ago
[deleted]
deleted 8y ago
[deleted]
adamnemecek 8y ago
I can imagine swift really taking off in this space. It’s going to be a battle between Julia and Swift for who does the best automatic differentiation.
- FridgeSeal 8y ago
  I’m happy for Swift, but I really, really want Julia to win out here. There’s some pretty impressive ML frameworks in Julia and the language can do some really cool things, so I’m hoping that gives it the edge. Plus, I found tensorflow exceedingly painful to use, so hopefully something else prevails.
- pjmlp 8y ago
  Google would need to make Swift a first class citzen on Windows, currently Julia is winning.
  - skohan 8y ago
    That's one of the outcomes I am hoping for in this. I would love for Swift to be a first-class language. I believe the Swift for TensorFlow team is currently hiring for this.
- make3 8y ago
  how can you not mention Python when it's currently what is used 99% of the time, the other 1% being R
elpakal 8y ago
I also wonder how much of this coincidentally lines up with Chris Lattner landing at Google. As Chris will admit, and as was left out of this analysis, Swift has also been given the humble goal of achieving world domination. All joking aside I'm very thrilled about this and have enjoyed tremendously watching the Swift language mature since its launch due in large part to the open source community and the Swift team's admirable commitment. Onward!
- zozbot123 8y ago
  > ...Swift has also been given the humble goal of achieving world domination Unfortunately it's too late for Swift - the Rust Evangelism Strike Force (aided by the Rewrite-It-In-Rust Task Force) is well on the way towards achieving this goal for Rust, all by themselves! There is no place remaining for Swift, D or any other language in the same niche.
- tempdeadbeef 8y ago
  http://nondot.org/sabre/Resume.html http://nondot.org/sabre/Resume.html "Swift for TensorFlow rethinks machine learning development ... I imagined, advocated for, coded the initial prototype and many of the subsystems after that; recruited, hired and trained an exceptional engineering team; we drove it to an open source launch and are continuing to build out and iterate on infrastructure."
- DannyBee 8y ago
  What do you mean? This was not in process before Chris came, it was a project he suggested and started pushing on? What is there to be coincidental?
  - insulanus 8y ago
    I think they mean the exact opposite of "coincidental", in the sense of "hmm... is this a coincidence" (no, it isn't)
    - DannyBee 8y ago
sjwright 8y ago
I have no idea what TensorFlow is (other than the basics) but I enjoyed reading that entire document because it did such a wonderful job of explaining a complex and potentially contentious decision. It’s fascinating to see Swift feature so strongly in a pragmatic analysis that doesn’t explicitly favour Apple platform interop.
- skwb 8y ago
  I am a bit ignorant on the topic, but is swift available for Windows/Ubuntu? Most of the deep learning scientists I know and work with use either of the two setups. I know there technically exists CUDA GPU support for Apple, but I have frankly never even attempted to mess with it.
  - rudedogg 8y ago
    Ubuntu is supported (see https://swift.org/download/ https://swift.org/download/), but Windows is pretty early I think. See https://github.com/apple/swift/blob/master/docs/Windows.md https://github.com/apple/swift/blob/master/docs/Windows.md and https://forums.swift.org/t/windows-nightlies/19174 https://forums.swift.org/t/windows-nightlies/19174 for more info.
    - pjmlp 8y ago
      Ubuntu is "supported". The compiler might be available, but there are hardly many libraries available that would compile outside Apple platforms.
      - skohan 8y ago
        That's no longer true. The Foundation framework is basically complete on Linux, and the vast majority 3rd party libraries which are not iOS specific will work on Ubuntu. Even many of Apple's own libraries (i.e. SwiftNIO, a low-level, high performance networking library for things like implementing web-servers) are cross-platform.
m0zg 8y ago
Is there a "standard" way of running Swift on Ubuntu LTS nowadays? A while back I looked into it, and ran into some hokey and unsatisfying solutions. I used Swift on iOS, and I like it a lot, but if they care about adoption, someone needs to reduce friction of getting up and running to approximately zero. A snap package a-la Go or per-user script based installation a-la Rust would be quite OK, as long as it's just one, easy to discover command.
- rudedogg 8y ago
  It looks pretty straightforward, see the Linux section of https://swift.org/download/#using-downloads https://swift.org/download/#using-downloads
  - m0zg 8y ago
    Yeah, going through two screenfuls of text every time I want to upgrade is not "straightforward".
    - rudedogg 8y ago
      Sorry, I just didn't want to give people the impression that it's more difficult than other languages. To upgrade you'd just have to remove the original install directory and untar the new release. I'm not familiar with Snap, but I did find https://snapcraft.io/swift https://snapcraft.io/swift Also, upgrading will be less common than Rust since almost everyone uses the latest release/toolchain. There's not really a reason to use the daily builds unless you're contributing to the Swift project.
      - m0zg 8y ago
        For comparison, for Go, it's "sudo snap refresh go". For Rust it's "rustup update stable". I mean, how hard would it be to properly package this stuff, and why should tens of thousands of users deal with all this manual downloading and unpacking? Assuming, of course, that Swift folks don't deliberately want to make the language unpolular, like Haskell.
igotsideas 8y ago
Well written explanation! I really enjoy Swift but it's not as accessible as some of the other languages mentioned. I have a 2011 Macbook Pro and wanted to use the latest and greatest new Swift features. Unfortunately, my machine is too old to upgrade to Mojave which means I can't download the latest version of xcode, which means no new version of Swift. I'm not mad at Apple in the least bit. I just wish I could use Swift 4 on my machine.
- gthippo 8y ago
  You might be interested in checking out Swift on Google Colab (e.g., https://colab.research.google.com/github/tensorflow/swift-tutorials/blob/master/iris/swift_tensorflow_tutorial.ipynb https://colab.research.google.com/github/tensorflow/swift-tu...)
- coldtea 8y ago
  Isn't Swift open source and available to build anyway, regardless of Xcode and OS X version? Even on Linux etc?
- delinka 8y ago
  Have you considered trying to build Swift from source? It's a bit time-consuming the first time, but subsequent updates less so - and you'd have Swift 4 at your disposal.
- stephencanon 8y ago
  FWIW, you can install Xcode 10.1 on High Sierra, and use the Swift.org 5.0 toolchain (https://swift.org/download/#snapshots https://swift.org/download/#snapshots) or build swift from source. You can't ship App Store apps this way, but it works great for experiments.
  - igotsideas 8y ago
    My machine is too old to upgrade to High Sierra.
    - 8y ago
gthippo 8y ago
Swift is also now supported in Colab (Google-hosted Jupyter notebooks) and there's a nice tutorial on some of the features of Swift for Tensorflow at https://colab.research.google.com/github/tensorflow/swift-tutorials/blob/master/iris/swift_tensorflow_tutorial.ipynb https://colab.research.google.com/github/tensorflow/swift-tu...
temp_qqq 8y ago
I feel bad for the Julia community. They must be sad that they didn't get picked over Swift.
- pjmlp 8y ago
  So far I can easily use Julia on Windows, Swift on the other hand....
- FridgeSeal 8y ago
  I don't think it's over yet really. Tensorflow might be moving their code base to Swift, but there's other frameworks and TF isn't the be-all-and-end-all of ML frameworks. Having used it, I'd really hope it isn't, because it's incredibly painful to use. I'm personally excited for the likes of Julia's Flux framework to get a bit more production ready, I think that's got serious legs.
  - wallnuss 8y ago
    I particularly enjoy that Flux.jl is an AD framework and a couple lines of Julia code defining convenience functions for ML, and it is all written in Julia so it invites exploration. Tim Besard gave a talk at a Tensorflow meetup recently that walked through the Julia stack (GPU&ML) that I would highly recommend: https://docs.google.com/presentation/d/1y93Kg8ZizvabKAGs-zmAr88xlOi4ST-UIycnWtETkGo/edit?usp=sharing https://docs.google.com/presentation/d/1y93Kg8ZizvabKAGs-zmA...
aaaaaaaaaaab 8y ago
Why not Rust? Edit: I wonder if Swift could be replaced with Rust for iOS development?
- deleted 8y ago
  [deleted]
- criddell 8y ago
  From the article: We believe that Rust supports all the ingredients necessary to implement the techniques in this paper: it has a strong static side, and its traits system supports zero-cost abstractions which can be provably eliminated by the compiler. It has a great pointer aliasing model, a suitable mid-level IR, a vibrant and engaging community, and a great open language evolution process. A concern with using Rust is that a strong goal of this project is to appeal to the entire TensorFlow community, which is currently pervasively Python based. We love Rust, but it has a steep learning curve that may exclude data scientists and other non-expert programmers who frequently use TensorFlow. The ownership model is really great, but mostly irrelevant to the problems faced by today’s machine learning code implemented in Python.
  - coder543 8y ago
    As I pointed out in two lengthy comments on day one[1][2], that reasoning is nonsense. If Chris wants to use the language he created in this new endeavor for machine learning simply because he made it, that's totally fine and completely his prerogative, but he should just say so, rather than trying (and failing) to convince people that other languages aren't better suited for this task. From my point of view, a weak justification is worse than no justification in cases like this. Rust is much better suited to this task than Swift from a technical point of view. The far superior platform support for Windows and Linux is ample reasoning to say Rust is better suited for this task, since very few data scientists will be training models on macOS. However, that's only one of several areas where Swift has shortcomings for a project like this. Swift is great for iOS and macOS development, of course, since it was designed for that. I don't think Swift is a bad language by any means, and with enough effort, it can be reshaped to be good for Tensorflow... the GitHub document just provides zero useful justification for the work required to make it good for Tensorflow. EDIT: to some of the replies talking about Rust's learning curve, that mostly applies when you start trying to design efficient, interlinked data structures involving ownership. For most applications of machine learning, this simply wouldn't be a problem. The library would provide the data structures, you just have to use them. Rust can provide simple interfaces to complicated things.[3] The compiler's error messages are usually incredibly helpful. The learning curve of Rust should not be relevant here, compared to Swift, which is also full of idiosyncrasies. Swift and Rust both have a large learning curve for someone coming from Python. This is because they're statically typed languages that are just different from a scripting language. For an application like this, I would say those learning curves are roughly equal at the language level, but as I pointed out in my comments, Swift has an enormous learning curve of requiring many data scientists to either install and learn Linux, or throw out their current computer, buy a Mac, and learn macOS. My point here is not that Rust is the most suitable language for Tensorflow (although it could be), but rather I'm making the point that Rust is more suitable than Swift for a project like this, and therefore this document is just annoying. It would be better for them to delete this document and just say "we're using Swift because our team has a lot of experience with it and because the creator of Swift is leading this project, so we would lack enthusiasm and momentum if we were using something else, even if it were more suitable." Julia would be really interesting to see explored further, since it would appeal much better to many existing data scientists who would be transitioning from Python. The times that I've played with Julia, I was amazed at how slow the JIT is for even tiny scripts. LLVM is powerful stuff, but it is painfully slow at everything. It would be nice if Julia offered an alternative backend for rapid development. [1]: https://github.com/tensorflow/swift/issues/3#issuecomment-384823808 https://github.com/tensorflow/swift/issues/3#issuecomment-38... [2]: https://github.com/tensorflow/swift/issues/3#issuecomment-384828003 https://github.com/tensorflow/swift/issues/3#issuecomment-38... [3]: http://kiss3d.org/ http://kiss3d.org/
hooloovoo_zoo 8y ago
https://xkcd.com/927/ https://xkcd.com/927/
- adamnemecek 8y ago
  This can be used as an argument against just about anything.
  - shuoli84 8y ago
    noop, some solution will dominate. Which will make 14 => 1 or 2.
skwb 8y ago
Forgive me for my ignorance, but does swift have any good plotting and interactive "notebook" ability? Specifically the ability to plot images such as matplotlib. I ask this because the number 1 reason my deep learning research group chose python was because of the extensive and interactive scientific plotting ability that's built into python jupyter notebooks. While our volume of analysis isn't on the scale of say a google/fb (primarily biomedical image analysis), the ability to easily visually debug the results is much more important for developing robust models.
- dynamicwebpaige 8y ago
  Yes! Swift is supported in Google Colab, and as a Jupyter kernel: https://github.com/google/swift-jupyter https://github.com/google/swift-jupyter.
  - skwb 8y ago
    What is the plotting experience like though? As I previously mentioned, plotting is one of the main reasons our group uses python. Another reason now that I think about it, is the number of scientific libraries that I can just "pip install" without much thought (such as scipy/opencv).
    - marcrasi 8y ago
      You can call out to matplotlib (or any other python libraries installed on your system), using the python interop feature (https://github.com/tensorflow/swift/blob/master/docs/PythonInteroperability.md https://github.com/tensorflow/swift/blob/master/docs/PythonI...)! https://github.com/google/swift-jupyter#rich-output https://github.com/google/swift-jupyter#rich-output has an example with screenshots.
      - Redoubts 8y ago
        Oof
- eschaton 8y ago
3jckd 8y ago
It's interesting how it's going to play out. On one hand side, Swift is a pleasant language to work with (despite its infancy). But on the other, having a Tensorflow API doesn't suddenly give it a bunch of libraries for statistics, comp. vision, modeling, visualisation, etc. that Python/R/Julia coughMATLABcough have. Nowadays, it's difficult enough to convince people to drop e.g. MATLAB for R or Python for Julia (let's assume that there's some merit to it), despite them having excellent counterparts for almost everything. Swift's success in this domain depends solely on the adoption by developers/researchers/engineers. Unless they're just going to mostly use it internally (as Google is known to). Which brings me to the last point - why on Earth would they pick Swift (apart from Christ Lattner being involved) when Julia was on the table? It ticks all their boxes and has more mature ecosystem for all things "data". Provided rationale is hardly convincing.
- marmaduke 8y ago
  Julia doesn't tick the "compile to .o/.h" box. As far as I can tell, the use case for AOT Julia is avoiding package compilation overhead, not delivery of standalone code objects. _edit_ seems JuliaC does support this sort of thing: https://juliacomputing.com/blog/2016/02/09/static-julia.html https://juliacomputing.com/blog/2016/02/09/static-julia.html
  - StefanKarpinski 8y ago
    Julia does already support this kind of thing. Moreover with a minuscule fraction of the money that’s being poured into making Swift usable for data science and machine learning, truly top notch support for generating standalone binaries from Julia could readily be developed. Which is kind of frustrating but what can you do?
  - celrod 8y ago
    If you'd like to track the latest developments: https://github.com/JuliaLang/PackageCompiler.jl https://github.com/JuliaLang/PackageCompiler.jl
- aschampion 8y ago
  Yeah, I don't buy the justification versus Julia because of community size either, given most of Swift's community has little to do with data science. The document even says as much, contradicting that rationale, later on. As someone who uses TF heavily, I would be much more excited about this project if they'd chosen Julia. Swift's tooling isn't great, and I already have a foot in one language with an immature data science ecosystem (Rust).
victor106 8y ago
You might agree/disagree with their decision but this is one of the most honest and comprehensive evaluation for using a language I've read.
kodablah 8y ago
Strange to see a requirement for choosing one language over another is supposed ease of adoption and then they choose the one not easily adopted across every platform. That easy to write syntax takes precedence over general easy to write/run is unfortunate.
byt143 8y ago
Julia would have been a much better and more cost effective choice in my opinion. It's a superior platform to on which develop this sort of thing, and further along at that. Also easier to use.
- siproprio 8y ago
  Julia doesn't have a debugger... They specifically claimed this is a very important thing. In my experience, Julia has also inscrutable scoping rules, a slow REPL, and it's only fast if you don't count the "startup time" of having to precompile everything.
  - byt143 8y ago
    Re: Debugger, fair enough but it's in the works Re: Scoping rules, these are being evaled. Re; startup time, already better in 1.1, and will soon be marginalized from two ends: Better static compilation and better tiered compilation.
mamcx 8y ago
I vaguely know tensor flow as the most(?) popular lib of his kind, but I wonder how is the history of swift on non-apple platforms and its impact of the actual users. Is TensorFlow "huge" in linux, windows, android? Because I also evaluate swift for my use case (https://www.reddit.com/r/swift/comments/8zb9y1/state_of_swift_for_androidlinuxwindows_on_2018/ https://www.reddit.com/r/swift/comments/8zb9y1/state_of_swif...) and decide instead on use rust mainly because the lack of solid support on non-apple platforms. However, after use rust for months now I still consider swift a better contender between performance/ergonomics than rust (rust is damm hard sometimes, and suddenly you could hit a trouble with above-average complications. I don't see how put this burden in a library to be used for more "regular" folks could work)
- skohan 8y ago
  The story of Swift on Linux is now quite good. Windows is less far along, but recently a contributor got nightly builds started on Azure, and it appears there is serious work on this front. In any case, it's already possible to run Swift for Tesorflow on Windows using WSL and Docker.
  - mamcx 8y ago
    Ok, that is for swift... But is not tensor flow popular on windows? Because then build on top of swift will mean: - Put swift on a fast track to be decent on windows, linux, android(?) - Ignore the windows users and let them battle a bad dependency?
5stospace 8y ago
I still don't understand why they would choose Swift over C#? They complain about C#/Java having "highly dynamic constructs" but correct me if I'm wrong but isn't swift also a GC/OOP like Java and C#? I don't think Swift has any inherent objective advantages over c#. I think it would have been a better decision to go with C# over Swift as Microsoft has a clear roadmap with the language and it is already supported on linux/mac/windows.
- barbecue_sauce 8y ago
  While I do think the choice of Swift is kind of weird, you are wrong about Swift being garbage collected (it uses Automatic Reference Counting). It also compiles to actual machine code (rather than an intermediate representation for use in a VM).
  - pjmlp 8y ago
    Reference counting is a garbage collection algorithm as per CS literature, you are mixing it up with tracing garbage collection algorithms. Swift makes use of SIL and LLVM bitcode before the final binary is produced. Likewise C# can be AOT compiled to actual machine code via NGEN, .NET Native, CoreRT and Mono/Xamarin.
- FridgeSeal 8y ago
  I would rather claw my eyes out than write any ML stuff in C#. It's a fine enterprise language, but good lord writing data science and machine learning stuff in it would be an right pain. It's also not super high performance, and when you're doing a lot of maths heavy operations, high performance is absolutely crucial. I had great difficulty establishing whether SIMD/vectorisation was even supported, and then even more difficulty getting it to work. Julia would have been a far, far superior choice than Swift.
  - pjmlp 8y ago
    SIMD is supported already for quite some time on RyuJIT. Quite easy to find out when searching the MSDN .NET Blog. Its performance is good enough for doing medical digital imagining as presented by Siemens at FOSDEM 2019. It is a matter to properly use the features that the language gives us.
tanilama 8y ago
Fail to see the point of this project. Swift for Tensorflow might work if the scope is to create a client side model definition loader natively for various TF models. Nobody use Swift seriously for server side training, there is no point in doing so except to add swift to the list of language that claim to do deep learning but in reality nobody will consider them.
moocowtruck 8y ago
why? because i made the language, thats why...no real good reason
- FridgeSeal 8y ago
  And here's 3 pages of very vague, half-justifications as to why we didn't choose anything else to head off any complaints.
solidsnack9000 8y ago
Setting aside for a moment the appropriateness of Swift for TensorFlow, this is a very impressive example of using an embedded DSL to work with a component that is a full programming system in its own right. On the one hand, we do want full access to the programming model exposed by the component -- its control structures, abstractions, everything else. One the other hand, these are mostly duplicated by our host programming language: it's going to have variable bindings, operators, iteration, conditionals and everything else. Doing an embedding like this is a way to expose most of the component's facilities without introducing a ton of "new syntax" in the form of combinators or having programs where a lot of the critical code is escaped in strings. This same problem shows up in programming interfaces to RDBMSes. LINQ is a good example of the same embedding technique.
tylerwhipple 8y ago
I am surprised Dart is not mentioned at all (maybe implied under the OOP languages?). While Flutter and Tensorflow are very different usecases, I am surprised there is nothing in the document on why Dart specifically would be a good choice. I believe if they used Dart for Tensorflow as well, the community would be able to get behind the idea that will not be an abandoned language.
masha_sb 8y ago
I'll wait for the PyTorch version.
xiaodai 8y ago
Haven't we seen this before?
- skohan 8y ago
  It's been in development for about a year, and I have seen several posts about it here. It's still not quite feature-complete or ready for real use.
Hacker_Support 8y ago
HAVE YOU LOST YOUR HARD EARNED MONEY TO A BINARY OPTION SCAM?? DO YOU WANT TO RECOVER IT ALL? DO NOT GIVE UP, I HAVE GOOD NEWS FOR YOU!!! First of all, in my opinion, Binary options trading is dead — though few will mourn it, but recovery is 100% possible and many fall for scams other than legit hackers. Dearly beloved readers. We are gathered here today to celebrate the passing of binary options, the much-hated financial instrument. Traded by risk-hungry speculators, binaries brought riches to the few and hefty losses to the many. They inhabited that tantalising grey area between financial investment and gambling, luring us in with promises of mega returns and pictures of people on Twitter with Lamborghinis. Instead, most were left red-faced with empty pockets. Many hundreds of people around the world are targeted each day by disreputable fraudulent brokers. They deliberate target novice investors in contravention of the trading regulations. In many cases the investors who are targeted are completely unfamiliar with the markets and do not recognise that they are dealing with a bogus trading platform. The fraudulent brokers rely on this lack of knowledge to extract as much money as possible before closing down the account, cutting all contact and disappearing. The hapless investor then begins to suspect that they have been scammed. The targeted individuals have frequently lost considerable sums of money and their beginner status means that they have little idea of where to turn for help. Now, investors like you run to an unknown hacker who are secretly scammers to help recorver your fund of which you will be disappointed at the end. We are a group of hackers called HackerOne. We consist of top skilled individual hackers coming together to render services to as many people out there on the common web, we all have operated in the dark web and have carried out classified job so we're bringing our skills here to the common web cause we know how difficult it is to access a service of a real hacker out here. HackerOne has a track record of recovery in relation to financial fraud, with many strategies and tactics to compel the fraudulent broker to restore funds to their former clients, then extract your files and documents, Decrypt your Transaction Details and some Technical Hacking Procedures follows then you have your money recovered in Bitcoins. You Can Also Contact us for other Technical Hacking Services you desire Such as: * WEBSITE HACKING * PHONE HACKING (giving you Unnoticeable access to everything Happening on the Target’s Phone) * LOCATION TRACKING * CLEARING OF CRIMINAL RECORDS * SOCIAL MEDIA ACCOUNTS HACKING etc For further information, please contact us on our Email address bellow: >> hacktech@hackermail.com Cheers!!
sometimesijust 8y ago
https://github.com/malmaud/TensorFlow.jl/blob/master/docs/src/why_julia.md https://github.com/malmaud/TensorFlow.jl/blob/master/docs/sr...
ramoz 8y ago
Is there an official release roadmap?
- skohan 8y ago
  Last I heard the goal for "initial adoption" is set for Spring 2019.
midgetjones 8y ago
I wonder what this means for iOS apps themselves
- skohan 8y ago
  At the moment not much. Swift for TensorFlow is a fork of the language, with language-level support for some features which are useful for data science, for instance automatic differentiation and dynamically-callable objects. Some of those features are making their way into the main branch, but at the moment you could not import the TensorFlow library into an iOS project and use it. Swift for TensorFlow needs to be built using a separate toolchain.
mikkelam 8y ago
As as python machine learning practitioner and previous iOS engineer I have for while come to miss using swift and type safety for that matter. I really like the language and wish great success for the TF team with swift. Side note, does anyone know the effort required to get various python based libraries running on swift? i.e. numpy, scipy, pandas and so on?
- physicsyogi 8y ago
  The Swift for Tensorflow team has added some python interop to Swift. So you’ll be able to, for instance, do an “import numpy” in your Swift code.
lovasoa 8y ago
Reading the document really gives a feeling the author is not being honest on why they chose Swift. The lack of windows support is addressed in just two lines. Julia being an already established language in the domain of data science does not seem to be especially important to them. I think the most honest part of the document is: > because we were more familiar with its [Swift's] internal implementation details

212 points by nnd 8y ago

deleted 8y ago

[deleted]

deleted 8y ago

[deleted]

adamnemecek 8y ago

I can imagine swift really taking off in this space. It’s going to be a battle between Julia and Swift for who does the best automatic differentiation.

FridgeSeal 8y ago
I’m happy for Swift, but I really, really want Julia to win out here. There’s some pretty impressive ML frameworks in Julia and the language can do some really cool things, so I’m hoping that gives it the edge. Plus, I found tensorflow exceedingly painful to use, so hopefully something else prevails.
pjmlp 8y ago
Google would need to make Swift a first class citzen on Windows, currently Julia is winning.
- skohan 8y ago
  That's one of the outcomes I am hoping for in this. I would love for Swift to be a first-class language. I believe the Swift for TensorFlow team is currently hiring for this.
make3 8y ago
how can you not mention Python when it's currently what is used 99% of the time, the other 1% being R

elpakal 8y ago

I also wonder how much of this coincidentally lines up with Chris Lattner landing at Google. As Chris will admit, and as was left out of this analysis, Swift has also been given the humble goal of achieving world domination. All joking aside I'm very thrilled about this and have enjoyed tremendously watching the Swift language mature since its launch due in large part to the open source community and the Swift team's admirable commitment. Onward!

zozbot123 8y ago
> ...Swift has also been given the humble goal of achieving world domination Unfortunately it's too late for Swift - the Rust Evangelism Strike Force (aided by the Rewrite-It-In-Rust Task Force) is well on the way towards achieving this goal for Rust, all by themselves! There is no place remaining for Swift, D or any other language in the same niche.
tempdeadbeef 8y ago
http://nondot.org/sabre/Resume.html http://nondot.org/sabre/Resume.html "Swift for TensorFlow rethinks machine learning development ... I imagined, advocated for, coded the initial prototype and many of the subsystems after that; recruited, hired and trained an exceptional engineering team; we drove it to an open source launch and are continuing to build out and iterate on infrastructure."
DannyBee 8y ago
What do you mean? This was not in process before Chris came, it was a project he suggested and started pushing on? What is there to be coincidental?
- insulanus 8y ago
  I think they mean the exact opposite of "coincidental", in the sense of "hmm... is this a coincidence" (no, it isn't)
  - DannyBee 8y ago

sjwright 8y ago

I have no idea what TensorFlow is (other than the basics) but I enjoyed reading that entire document because it did such a wonderful job of explaining a complex and potentially contentious decision. It’s fascinating to see Swift feature so strongly in a pragmatic analysis that doesn’t explicitly favour Apple platform interop.

skwb 8y ago
I am a bit ignorant on the topic, but is swift available for Windows/Ubuntu? Most of the deep learning scientists I know and work with use either of the two setups. I know there technically exists CUDA GPU support for Apple, but I have frankly never even attempted to mess with it.
- rudedogg 8y ago
  Ubuntu is supported (see https://swift.org/download/ https://swift.org/download/), but Windows is pretty early I think. See https://github.com/apple/swift/blob/master/docs/Windows.md https://github.com/apple/swift/blob/master/docs/Windows.md and https://forums.swift.org/t/windows-nightlies/19174 https://forums.swift.org/t/windows-nightlies/19174 for more info.
  - pjmlp 8y ago
    Ubuntu is "supported". The compiler might be available, but there are hardly many libraries available that would compile outside Apple platforms.
    - skohan 8y ago
      That's no longer true. The Foundation framework is basically complete on Linux, and the vast majority 3rd party libraries which are not iOS specific will work on Ubuntu. Even many of Apple's own libraries (i.e. SwiftNIO, a low-level, high performance networking library for things like implementing web-servers) are cross-platform.

m0zg 8y ago

Is there a "standard" way of running Swift on Ubuntu LTS nowadays? A while back I looked into it, and ran into some hokey and unsatisfying solutions. I used Swift on iOS, and I like it a lot, but if they care about adoption, someone needs to reduce friction of getting up and running to approximately zero. A snap package a-la Go or per-user script based installation a-la Rust would be quite OK, as long as it's just one, easy to discover command.

rudedogg 8y ago
It looks pretty straightforward, see the Linux section of https://swift.org/download/#using-downloads https://swift.org/download/#using-downloads
- m0zg 8y ago
  Yeah, going through two screenfuls of text every time I want to upgrade is not "straightforward".
  - rudedogg 8y ago
    Sorry, I just didn't want to give people the impression that it's more difficult than other languages. To upgrade you'd just have to remove the original install directory and untar the new release. I'm not familiar with Snap, but I did find https://snapcraft.io/swift https://snapcraft.io/swift Also, upgrading will be less common than Rust since almost everyone uses the latest release/toolchain. There's not really a reason to use the daily builds unless you're contributing to the Swift project.
    - m0zg 8y ago
      For comparison, for Go, it's "sudo snap refresh go". For Rust it's "rustup update stable". I mean, how hard would it be to properly package this stuff, and why should tens of thousands of users deal with all this manual downloading and unpacking? Assuming, of course, that Swift folks don't deliberately want to make the language unpolular, like Haskell.

igotsideas 8y ago

Well written explanation! I really enjoy Swift but it's not as accessible as some of the other languages mentioned. I have a 2011 Macbook Pro and wanted to use the latest and greatest new Swift features. Unfortunately, my machine is too old to upgrade to Mojave which means I can't download the latest version of xcode, which means no new version of Swift. I'm not mad at Apple in the least bit. I just wish I could use Swift 4 on my machine.

gthippo 8y ago
You might be interested in checking out Swift on Google Colab (e.g., https://colab.research.google.com/github/tensorflow/swift-tutorials/blob/master/iris/swift_tensorflow_tutorial.ipynb https://colab.research.google.com/github/tensorflow/swift-tu...)
coldtea 8y ago
Isn't Swift open source and available to build anyway, regardless of Xcode and OS X version? Even on Linux etc?
delinka 8y ago
Have you considered trying to build Swift from source? It's a bit time-consuming the first time, but subsequent updates less so - and you'd have Swift 4 at your disposal.
stephencanon 8y ago
FWIW, you can install Xcode 10.1 on High Sierra, and use the Swift.org 5.0 toolchain (https://swift.org/download/#snapshots https://swift.org/download/#snapshots) or build swift from source. You can't ship App Store apps this way, but it works great for experiments.
- igotsideas 8y ago
  My machine is too old to upgrade to High Sierra.
  - 8y ago

gthippo 8y ago

Swift is also now supported in Colab (Google-hosted Jupyter notebooks) and there's a nice tutorial on some of the features of Swift for Tensorflow at https://colab.research.google.com/github/tensorflow/swift-tutorials/blob/master/iris/swift_tensorflow_tutorial.ipynb https://colab.research.google.com/github/tensorflow/swift-tu...

temp_qqq 8y ago

I feel bad for the Julia community. They must be sad that they didn't get picked over Swift.

pjmlp 8y ago
So far I can easily use Julia on Windows, Swift on the other hand....
FridgeSeal 8y ago
I don't think it's over yet really. Tensorflow might be moving their code base to Swift, but there's other frameworks and TF isn't the be-all-and-end-all of ML frameworks. Having used it, I'd really hope it isn't, because it's incredibly painful to use. I'm personally excited for the likes of Julia's Flux framework to get a bit more production ready, I think that's got serious legs.
- wallnuss 8y ago
  I particularly enjoy that Flux.jl is an AD framework and a couple lines of Julia code defining convenience functions for ML, and it is all written in Julia so it invites exploration. Tim Besard gave a talk at a Tensorflow meetup recently that walked through the Julia stack (GPU&ML) that I would highly recommend: https://docs.google.com/presentation/d/1y93Kg8ZizvabKAGs-zmAr88xlOi4ST-UIycnWtETkGo/edit?usp=sharing https://docs.google.com/presentation/d/1y93Kg8ZizvabKAGs-zmA...

aaaaaaaaaaab 8y ago

Why not Rust? Edit: I wonder if Swift could be replaced with Rust for iOS development?

deleted 8y ago
[deleted]
criddell 8y ago
From the article: We believe that Rust supports all the ingredients necessary to implement the techniques in this paper: it has a strong static side, and its traits system supports zero-cost abstractions which can be provably eliminated by the compiler. It has a great pointer aliasing model, a suitable mid-level IR, a vibrant and engaging community, and a great open language evolution process. A concern with using Rust is that a strong goal of this project is to appeal to the entire TensorFlow community, which is currently pervasively Python based. We love Rust, but it has a steep learning curve that may exclude data scientists and other non-expert programmers who frequently use TensorFlow. The ownership model is really great, but mostly irrelevant to the problems faced by today’s machine learning code implemented in Python.
- coder543 8y ago
  As I pointed out in two lengthy comments on day one[1][2], that reasoning is nonsense. If Chris wants to use the language he created in this new endeavor for machine learning simply because he made it, that's totally fine and completely his prerogative, but he should just say so, rather than trying (and failing) to convince people that other languages aren't better suited for this task. From my point of view, a weak justification is worse than no justification in cases like this. Rust is much better suited to this task than Swift from a technical point of view. The far superior platform support for Windows and Linux is ample reasoning to say Rust is better suited for this task, since very few data scientists will be training models on macOS. However, that's only one of several areas where Swift has shortcomings for a project like this. Swift is great for iOS and macOS development, of course, since it was designed for that. I don't think Swift is a bad language by any means, and with enough effort, it can be reshaped to be good for Tensorflow... the GitHub document just provides zero useful justification for the work required to make it good for Tensorflow. EDIT: to some of the replies talking about Rust's learning curve, that mostly applies when you start trying to design efficient, interlinked data structures involving ownership. For most applications of machine learning, this simply wouldn't be a problem. The library would provide the data structures, you just have to use them. Rust can provide simple interfaces to complicated things.[3] The compiler's error messages are usually incredibly helpful. The learning curve of Rust should not be relevant here, compared to Swift, which is also full of idiosyncrasies. Swift and Rust both have a large learning curve for someone coming from Python. This is because they're statically typed languages that are just different from a scripting language. For an application like this, I would say those learning curves are roughly equal at the language level, but as I pointed out in my comments, Swift has an enormous learning curve of requiring many data scientists to either install and learn Linux, or throw out their current computer, buy a Mac, and learn macOS. My point here is not that Rust is the most suitable language for Tensorflow (although it could be), but rather I'm making the point that Rust is more suitable than Swift for a project like this, and therefore this document is just annoying. It would be better for them to delete this document and just say "we're using Swift because our team has a lot of experience with it and because the creator of Swift is leading this project, so we would lack enthusiasm and momentum if we were using something else, even if it were more suitable." Julia would be really interesting to see explored further, since it would appeal much better to many existing data scientists who would be transitioning from Python. The times that I've played with Julia, I was amazed at how slow the JIT is for even tiny scripts. LLVM is powerful stuff, but it is painfully slow at everything. It would be nice if Julia offered an alternative backend for rapid development. [1]: https://github.com/tensorflow/swift/issues/3#issuecomment-384823808 https://github.com/tensorflow/swift/issues/3#issuecomment-38... [2]: https://github.com/tensorflow/swift/issues/3#issuecomment-384828003 https://github.com/tensorflow/swift/issues/3#issuecomment-38... [3]: http://kiss3d.org/ http://kiss3d.org/

hooloovoo_zoo 8y ago

https://xkcd.com/927/ https://xkcd.com/927/

adamnemecek 8y ago
This can be used as an argument against just about anything.
- shuoli84 8y ago
  noop, some solution will dominate. Which will make 14 => 1 or 2.

skwb 8y ago

Forgive me for my ignorance, but does swift have any good plotting and interactive "notebook" ability? Specifically the ability to plot images such as matplotlib. I ask this because the number 1 reason my deep learning research group chose python was because of the extensive and interactive scientific plotting ability that's built into python jupyter notebooks. While our volume of analysis isn't on the scale of say a google/fb (primarily biomedical image analysis), the ability to easily visually debug the results is much more important for developing robust models.

dynamicwebpaige 8y ago
Yes! Swift is supported in Google Colab, and as a Jupyter kernel: https://github.com/google/swift-jupyter https://github.com/google/swift-jupyter.
- skwb 8y ago
  What is the plotting experience like though? As I previously mentioned, plotting is one of the main reasons our group uses python. Another reason now that I think about it, is the number of scientific libraries that I can just "pip install" without much thought (such as scipy/opencv).
  - marcrasi 8y ago
    You can call out to matplotlib (or any other python libraries installed on your system), using the python interop feature (https://github.com/tensorflow/swift/blob/master/docs/PythonInteroperability.md https://github.com/tensorflow/swift/blob/master/docs/PythonI...)! https://github.com/google/swift-jupyter#rich-output https://github.com/google/swift-jupyter#rich-output has an example with screenshots.
    - Redoubts 8y ago
      Oof
eschaton 8y ago

3jckd 8y ago

It's interesting how it's going to play out. On one hand side, Swift is a pleasant language to work with (despite its infancy). But on the other, having a Tensorflow API doesn't suddenly give it a bunch of libraries for statistics, comp. vision, modeling, visualisation, etc. that Python/R/Julia coughMATLABcough have. Nowadays, it's difficult enough to convince people to drop e.g. MATLAB for R or Python for Julia (let's assume that there's some merit to it), despite them having excellent counterparts for almost everything. Swift's success in this domain depends solely on the adoption by developers/researchers/engineers. Unless they're just going to mostly use it internally (as Google is known to). Which brings me to the last point - why on Earth would they pick Swift (apart from Christ Lattner being involved) when Julia was on the table? It ticks all their boxes and has more mature ecosystem for all things "data". Provided rationale is hardly convincing.

marmaduke 8y ago
Julia doesn't tick the "compile to .o/.h" box. As far as I can tell, the use case for AOT Julia is avoiding package compilation overhead, not delivery of standalone code objects. _edit_ seems JuliaC does support this sort of thing: https://juliacomputing.com/blog/2016/02/09/static-julia.html https://juliacomputing.com/blog/2016/02/09/static-julia.html
- StefanKarpinski 8y ago
  Julia does already support this kind of thing. Moreover with a minuscule fraction of the money that’s being poured into making Swift usable for data science and machine learning, truly top notch support for generating standalone binaries from Julia could readily be developed. Which is kind of frustrating but what can you do?
- celrod 8y ago
  If you'd like to track the latest developments: https://github.com/JuliaLang/PackageCompiler.jl https://github.com/JuliaLang/PackageCompiler.jl
aschampion 8y ago
Yeah, I don't buy the justification versus Julia because of community size either, given most of Swift's community has little to do with data science. The document even says as much, contradicting that rationale, later on. As someone who uses TF heavily, I would be much more excited about this project if they'd chosen Julia. Swift's tooling isn't great, and I already have a foot in one language with an immature data science ecosystem (Rust).

victor106 8y ago

You might agree/disagree with their decision but this is one of the most honest and comprehensive evaluation for using a language I've read.

kodablah 8y ago

Strange to see a requirement for choosing one language over another is supposed ease of adoption and then they choose the one not easily adopted across every platform. That easy to write syntax takes precedence over general easy to write/run is unfortunate.

byt143 8y ago

Julia would have been a much better and more cost effective choice in my opinion. It's a superior platform to on which develop this sort of thing, and further along at that. Also easier to use.

siproprio 8y ago
Julia doesn't have a debugger... They specifically claimed this is a very important thing. In my experience, Julia has also inscrutable scoping rules, a slow REPL, and it's only fast if you don't count the "startup time" of having to precompile everything.
- byt143 8y ago
  Re: Debugger, fair enough but it's in the works Re: Scoping rules, these are being evaled. Re; startup time, already better in 1.1, and will soon be marginalized from two ends: Better static compilation and better tiered compilation.

mamcx 8y ago

I vaguely know tensor flow as the most(?) popular lib of his kind, but I wonder how is the history of swift on non-apple platforms and its impact of the actual users. Is TensorFlow "huge" in linux, windows, android? Because I also evaluate swift for my use case (https://www.reddit.com/r/swift/comments/8zb9y1/state_of_swift_for_androidlinuxwindows_on_2018/ https://www.reddit.com/r/swift/comments/8zb9y1/state_of_swif...) and decide instead on use rust mainly because the lack of solid support on non-apple platforms. However, after use rust for months now I still consider swift a better contender between performance/ergonomics than rust (rust is damm hard sometimes, and suddenly you could hit a trouble with above-average complications. I don't see how put this burden in a library to be used for more "regular" folks could work)

skohan 8y ago
The story of Swift on Linux is now quite good. Windows is less far along, but recently a contributor got nightly builds started on Azure, and it appears there is serious work on this front. In any case, it's already possible to run Swift for Tesorflow on Windows using WSL and Docker.
- mamcx 8y ago
  Ok, that is for swift... But is not tensor flow popular on windows? Because then build on top of swift will mean: - Put swift on a fast track to be decent on windows, linux, android(?) - Ignore the windows users and let them battle a bad dependency?

5stospace 8y ago

I still don't understand why they would choose Swift over C#? They complain about C#/Java having "highly dynamic constructs" but correct me if I'm wrong but isn't swift also a GC/OOP like Java and C#? I don't think Swift has any inherent objective advantages over c#. I think it would have been a better decision to go with C# over Swift as Microsoft has a clear roadmap with the language and it is already supported on linux/mac/windows.

barbecue_sauce 8y ago
While I do think the choice of Swift is kind of weird, you are wrong about Swift being garbage collected (it uses Automatic Reference Counting). It also compiles to actual machine code (rather than an intermediate representation for use in a VM).
- pjmlp 8y ago
  Reference counting is a garbage collection algorithm as per CS literature, you are mixing it up with tracing garbage collection algorithms. Swift makes use of SIL and LLVM bitcode before the final binary is produced. Likewise C# can be AOT compiled to actual machine code via NGEN, .NET Native, CoreRT and Mono/Xamarin.
FridgeSeal 8y ago
I would rather claw my eyes out than write any ML stuff in C#. It's a fine enterprise language, but good lord writing data science and machine learning stuff in it would be an right pain. It's also not super high performance, and when you're doing a lot of maths heavy operations, high performance is absolutely crucial. I had great difficulty establishing whether SIMD/vectorisation was even supported, and then even more difficulty getting it to work. Julia would have been a far, far superior choice than Swift.
- pjmlp 8y ago
  SIMD is supported already for quite some time on RyuJIT. Quite easy to find out when searching the MSDN .NET Blog. Its performance is good enough for doing medical digital imagining as presented by Siemens at FOSDEM 2019. It is a matter to properly use the features that the language gives us.

tanilama 8y ago

Fail to see the point of this project. Swift for Tensorflow might work if the scope is to create a client side model definition loader natively for various TF models. Nobody use Swift seriously for server side training, there is no point in doing so except to add swift to the list of language that claim to do deep learning but in reality nobody will consider them.

moocowtruck 8y ago

why? because i made the language, thats why...no real good reason

FridgeSeal 8y ago
And here's 3 pages of very vague, half-justifications as to why we didn't choose anything else to head off any complaints.

solidsnack9000 8y ago

Setting aside for a moment the appropriateness of Swift for TensorFlow, this is a very impressive example of using an embedded DSL to work with a component that is a full programming system in its own right. On the one hand, we do want full access to the programming model exposed by the component -- its control structures, abstractions, everything else. One the other hand, these are mostly duplicated by our host programming language: it's going to have variable bindings, operators, iteration, conditionals and everything else. Doing an embedding like this is a way to expose most of the component's facilities without introducing a ton of "new syntax" in the form of combinators or having programs where a lot of the critical code is escaped in strings. This same problem shows up in programming interfaces to RDBMSes. LINQ is a good example of the same embedding technique.

tylerwhipple 8y ago

I am surprised Dart is not mentioned at all (maybe implied under the OOP languages?). While Flutter and Tensorflow are very different usecases, I am surprised there is nothing in the document on why Dart specifically would be a good choice. I believe if they used Dart for Tensorflow as well, the community would be able to get behind the idea that will not be an abandoned language.

masha_sb 8y ago

I'll wait for the PyTorch version.

xiaodai 8y ago

Haven't we seen this before?

skohan 8y ago
It's been in development for about a year, and I have seen several posts about it here. It's still not quite feature-complete or ready for real use.

Hacker_Support 8y ago

HAVE YOU LOST YOUR HARD EARNED MONEY TO A BINARY OPTION SCAM?? DO YOU WANT TO RECOVER IT ALL? DO NOT GIVE UP, I HAVE GOOD NEWS FOR YOU!!! First of all, in my opinion, Binary options trading is dead — though few will mourn it, but recovery is 100% possible and many fall for scams other than legit hackers. Dearly beloved readers. We are gathered here today to celebrate the passing of binary options, the much-hated financial instrument. Traded by risk-hungry speculators, binaries brought riches to the few and hefty losses to the many. They inhabited that tantalising grey area between financial investment and gambling, luring us in with promises of mega returns and pictures of people on Twitter with Lamborghinis. Instead, most were left red-faced with empty pockets. Many hundreds of people around the world are targeted each day by disreputable fraudulent brokers. They deliberate target novice investors in contravention of the trading regulations. In many cases the investors who are targeted are completely unfamiliar with the markets and do not recognise that they are dealing with a bogus trading platform. The fraudulent brokers rely on this lack of knowledge to extract as much money as possible before closing down the account, cutting all contact and disappearing. The hapless investor then begins to suspect that they have been scammed. The targeted individuals have frequently lost considerable sums of money and their beginner status means that they have little idea of where to turn for help. Now, investors like you run to an unknown hacker who are secretly scammers to help recorver your fund of which you will be disappointed at the end. We are a group of hackers called HackerOne. We consist of top skilled individual hackers coming together to render services to as many people out there on the common web, we all have operated in the dark web and have carried out classified job so we're bringing our skills here to the common web cause we know how difficult it is to access a service of a real hacker out here. HackerOne has a track record of recovery in relation to financial fraud, with many strategies and tactics to compel the fraudulent broker to restore funds to their former clients, then extract your files and documents, Decrypt your Transaction Details and some Technical Hacking Procedures follows then you have your money recovered in Bitcoins. You Can Also Contact us for other Technical Hacking Services you desire Such as: * WEBSITE HACKING * PHONE HACKING (giving you Unnoticeable access to everything Happening on the Target’s Phone) * LOCATION TRACKING * CLEARING OF CRIMINAL RECORDS * SOCIAL MEDIA ACCOUNTS HACKING etc For further information, please contact us on our Email address bellow: >> hacktech@hackermail.com Cheers!!

sometimesijust 8y ago

https://github.com/malmaud/TensorFlow.jl/blob/master/docs/src/why_julia.md https://github.com/malmaud/TensorFlow.jl/blob/master/docs/sr...

ramoz 8y ago

Is there an official release roadmap?

skohan 8y ago
Last I heard the goal for "initial adoption" is set for Spring 2019.

midgetjones 8y ago

I wonder what this means for iOS apps themselves

skohan 8y ago
At the moment not much. Swift for TensorFlow is a fork of the language, with language-level support for some features which are useful for data science, for instance automatic differentiation and dynamically-callable objects. Some of those features are making their way into the main branch, but at the moment you could not import the TensorFlow library into an iOS project and use it. Swift for TensorFlow needs to be built using a separate toolchain.

mikkelam 8y ago

As as python machine learning practitioner and previous iOS engineer I have for while come to miss using swift and type safety for that matter. I really like the language and wish great success for the TF team with swift. Side note, does anyone know the effort required to get various python based libraries running on swift? i.e. numpy, scipy, pandas and so on?

physicsyogi 8y ago
The Swift for Tensorflow team has added some python interop to Swift. So you’ll be able to, for instance, do an “import numpy” in your Swift code.

lovasoa 8y ago

Reading the document really gives a feeling the author is not being honest on why they chose Swift. The lack of windows support is addressed in just two lines. Julia being an already established language in the domain of data science does not seem to be especially important to them. I think the most honest part of the document is: > because we were more familiar with its [Swift's] internal implementation details