8 ms·

Taking ML to production with Rust

195 points by LukeMathWalker 7y ago

xiaodai 7y ago
Rust will never work except in niche areas because it doesn't allow for easy experimentation and does not provide an iterative workflow like Python, R, or Julia. I think the future is Julia, as it's as easy to write as Python, is interactive, and can be compiled for fast speeds! It will just takes time for Julia to have as many libraries, but at some point the tipping point will be reached, and Julia will just eat Python's lunch. Python was dying before the latest wave of data science and machine learning craze saved it anyway. There are pretty good reasons to believe Python (and Ruby) will slowly wither away. I am careful not to savy that they will die out, because they won't as some computing problems do not require so much speed and hence Python and the like can still pass mustard. But in a post-Moore's-law-world, data is getting bigger, the demand for computing is larger, but programs aren't getting faster, and there is a limit to how much you can optimise Python. Eventually, a crop of speedy and easy to write programming languages will rise up and eat Python's lunch. The early signs are there with tools like Julia.
- Ygg2 7y ago
  > Rust will never work except in niche because It warms my heart to see people don't read TFA. Like at all. From TFA: > But it was important to clean the air of any possible misunderstanding before getting started. I don’t believe Rust will replace Python as the language of choice for ML - it just isn’t happening, it’s not a thing, neither today or tomorrow.
  - hu3 7y ago
    What's wrong in expanding on a topic that was barely mentioned in the article? Your parent wrote detailed reasoning that's not present in the article. Also this type of uncharitable comment is against Hacker News guidelines: "Please don't comment on whether someone read an article". http://news.ycombinator.com/newsguidelines.html http://news.ycombinator.com/newsguidelines.html
    - fnord123 7y ago
      There can be no discussion with GGP since he obviously didn't read the article. They think the article will claim Rust supplants Python. They think there is a performance advantage in using Julia instead of Python (but the Python code is just calling C, C++, and Rust as described in the article). GGP thinks there will be a tipping point where Julia has more libraries than Python but this is also addressed in the article. There is no expanding on topics barely mentioned in the article. As far as GGP is concerned there was just a headline and they Kanye'd with "Julia is the best". As far as I care, mods can delete the comment and subtree.
- MayeulC 7y ago
  That may be true in data science, but the tradeoffs are not necessarily the same in other fields. I do not see Python fading away in mine, for instance. If anything, it looks more and more encroached, and even if the Python code becomes legacy code, interfaces will have to be maintained. I still use TCL quite often in design automation. Python is and will still be used to script Blender, Kodi, Inkscape, Klayout, sigrok, and countless others, for instance. But your week-end project you need to write a bit of code for and want to practice a new language with/just want to code as quick as possible? Sure, the languages will come and go. Likewise with Rust. Maybe not for iterative design phases, but I can certainly see it being used in production, especially if robustness/speed are needed.
- mirekrusin 7y ago
  Yes, Julia makes some impressive stunts - supercomputing [0], ml where code is very close to mathematical description [1], code reuse [2] etc. [0] https://juliacomputing.com/blog/2019/04/12/Supercomputing-julia.html https://juliacomputing.com/blog/2019/04/12/Supercomputing-ju... [1] https://www.youtube.com/watch?time_continue=188&v=9KBaRS2gy-U&feature=emb_logo https://www.youtube.com/watch?time_continue=188&v=9KBaRS2gy-... [2] https://www.youtube.com/watch?v=kc9HwsxE1OY&t=130s https://www.youtube.com/watch?v=kc9HwsxE1OY&t=130s
- deleted 7y ago
  [deleted]
- pjmlp 7y ago
  I am also betting on Julia.
  - ilteris 7y ago
    What do you think about Swift
    - pjmlp 7y ago
      It won't go anywhere, Tensorflow for Swift only exists because of Chris Lattner. Many researchers work on Windows, Swift has zero support there and when asked about it, they just kind of invite the Swift community to provide support. So zero interest from either Apple or Google to make it work on Windows. This is the latest status of Swift on Windows, https://www.youtube.com/watch?v=Zjlxa1NIfJc https://www.youtube.com/watch?v=Zjlxa1NIfJc Then on Linux, Foundation support is still flaky, for basic stuff one still needs to do conditional compilation and directly import either Darwin or GLibc modules. Swift is just like Objective-C, yes you can kind of use it outside Apple's ecosytem, but at expense of productivity and lack of adequate tooling. Really I don't expect Tensorflow for Swift to ever reach mainstream adoption, maybe Google can sell it as Mac only and thus announce the project has having been a success.
    - xiaodai 7y ago
      Too verbose. Does not even support Windows. Too closely related to iOS at least in image. MLIR sounds like a nice idea but isn't really Swift specific. I don't understand why they decided to do Swift for Tensorflow, should have invested in Julia for Tensorflow
- Certhas 7y ago
  I like Julia, but let's be real. We are a long way away from Julia being anywhere near Python when it comes to usability. Error messages, code structure and readability and documentation are afterthoughts rather than a central focus of the community. Because the core is so brilliant at what it does I am sure that Julia will get there eventually, but it will be a painful way. For crying out loud, we still don't have a sory for people to save their data...
  - wtetzner 7y ago
    > For crying out loud, we still don't have a sory for people to save their data... Can you elaborate on what you mean by this? Does Julia not support writing files?
    - Certhas 7y ago
      It does, of course, but serialization is problematic (with the combination of Type inference and dynamicism this is maybe to be expected). Beyond that there is a bewildering array of packages, and not really a clear consensus for which to use: https://discourse.julialang.org/t/what-is-the-preferred-way-to-save-variables/21918 https://discourse.julialang.org/t/what-is-the-preferred-way-... JLD2, which for a while seemed like an emerging consensus option has caused massive dataloss for a number of people: https://github.com/JuliaIO/JLD2.jl/issues/55 https://github.com/JuliaIO/JLD2.jl/issues/55 The issue is also that Julia packages often use their own container types for convenient analysis. For example the solution type of DiffEq has a number of important features like interpolation built in. If I want to reliably save the data I got from the numerical integration, and don't want to rely on brittle/broken serialization packages, I have to extract the underlying arrays though. That would be fine if not for the fact that there is no documented way (that I've found) to initialize the solution objects from this raw data, so now I have lost access to the convenience functions. Don't get me wrong, this is all growing pains. I am sure this will be sorted out eventually. But it's important to point out how the intersection of features that enable Julia to do things that are unheard of in other languages (e.g. DiffEqFlux, Zygote) also mean there are some things that still need to be figured out for this language (serialization, useful error messages, interfaces).
- cwyers 7y ago
  It's not like Python's community is done writing libraries and improving libraries. If there is going to be a tipping point it's going to have to start happening before the Julia package ecosystem catches up to Python's.
psv1 7y ago
Rust can never replace Python as a front-end because it's compiled and not all that flexible or user-friendly. Rust can't currently replace C or C++ at the back-end because it doesn't have good enough support for GPU computing yet.
- excerionsforte 7y ago
  "The Rust ecosystem is indeed rich in ML crates - just take a look at what a quick search for machine learning on crates.io returns. No need to go and rewrite everything from scratch: I picture linfa as a meta-package, a collection of curated algorithm implementations from the Rust ecosystem. The first stop for your ML needs, as scikit-learn for Python." Rust doesn't have to replace anything. It can be used to create complementary packages especially ones that use C/C++. The point of introducing Rust into your ecosystem is for safety + performance which is hard to achieve without discipline.
  - psv1 7y ago
    > Rust doesn't have to replace anything. It can be used to create complementary packages especially ones that use C/C++. I'm not sure that I understand correctly - are you suggesting that people use Rust as an additional layer between Python and C++?
    - eden_h 7y ago
      I've heard this suggestion fairly frequently - that people use Rust as a wrapper over C++ bindings to add in safety to the C++ underneath. Mostly regarding using BLAS etc in Rust rather than C++. It seems a little pointless as the unsafe nature is within the underlying code, so it only santises input/output.
      - pjmlp 7y ago
        For that use case it is more productive to actually use static analysis tooling for C++.
      - 7y ago
wdroz 7y ago
The 25x speedup is because the gRPC python implementation is very slow [0]. [0] -- https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5636470266134528 https://performance-dot-grpc-testing.appspot.com/explore?das...
- kbumsik 7y ago
  Yeah, by skimming the article, it sound like making gRPC microservices 25x faster, not ML.
  - nevi-me 7y ago
    gRPC would account for very little time relative to the k-means algorithm. I think what results in the large speedup is the "embarrassing" parallelism that an async Rust server (gRPC in this case) would give. Knowing the CPU time during the tests could confirm this. I'll try run the benchmarks on my machine this week to verify my assumption.
    - bpicolo 7y ago
      I've hit some degenerate cases on gRPC serialization. The golang grpc/proto library accounted for something like 40% of cpu time doing serialization on mapping types for an API I developed under load test. Changing that to a list of key/value pairs dropped that 40% to <1%. The gRPC python sever implementation does a lot of work using python builtins, and doesn't use recent asyncio primitives. It uses built-in python thread-pool execution, which means you also have the GIL to worry about: https://grpc.io/docs/tutorials/basic/python/#starting-the-server https://grpc.io/docs/tutorials/basic/python/#starting-the-se... Given that, I would expect its connection / request management to be pretty darn slow. gRPC has better support for Python as the client. The thread pool execution model also prevents you from doing a number of things you might want to do with gRPC servers (long-lived request streaming is out of the picture). There are a couple third party packages out there trying to make this better (e.g. https://github.com/vmagamedov/grpclib https://github.com/vmagamedov/grpclib) but I hit compatibility issues for a few cases when trying to use them for some prototypes.
  - 7y ago
captain_price7 7y ago
I was a little surprised to see that instead of directly benchmarking prediction performance on a local machine, author chose to go through that complicated web-server route. I saw author saying this is nothing more than a remote RFC, still, a benchmark should try to remove unrelated factors as much as possible. I would be interested to see how the results compare when this web-service layer is removed.
- cwyers 7y ago
  Serving model predictions via an API is a pretty common usecase for putting data science models in production. If Python's web service layer is slower than Rust's, that's a real concern even if it's not directly related to model prediction.
sandGorgon 7y ago
Although I am a python guy, if I had to bet on an alternative language here - its going to be Swift. The amount of evangelism money that Google is spending on Tensorflow is staggering. Its not just t-shirts, but an entire ecosystem - TensorBoard.dev, Tensorflow Privacy, TF Enterprise, Explainable AI Beta, free TPU on Google Colab etc. And all of them will soon be backed using Tensorflow Swift (https://www.tensorflow.org/swift https://www.tensorflow.org/swift). This is MNIST for Swift Tensorflow - https://github.com/tensorflow/swift-models/blob/master/Datasets/MNIST/MNIST.swift https://github.com/tensorflow/swift-models/blob/master/Datas... . Mind you this is runnable on Google Colab using TPU hardware acceleration in swift. Swift vs Rust is a very political thing...but previous discussions on HN have been tilted in favor of Swift [1] And we havent even touched on the elephant in the Room - Swift may soon hook into OSX Metal (which is arguably the only way to do GPU AI on macbooks now that the nvidia-Apple CUDA divorce is official [2] ). You can already call Metal from Swift...but not write shaders directly IMHO. [1] https://cloud.google.com/explainable-ai/ https://cloud.google.com/explainable-ai/ [2] https://gizmodo.com/apple-and-nvidia-are-over-1840015246 https://gizmodo.com/apple-and-nvidia-are-over-1840015246
- Roritharr 7y ago
  This is really new to me, I always thought of Swift as the iOS thing to look into when doing something there. Very interesting that Google is pushing Swift for their Tensorflow Platform. Any ideas why Swift is of interest for Google?
  - jorgemf 7y ago
    They hired one of the Swift leaders for the tensor flow team
  - bpicolo 7y ago
    Swift playgrounds seem very much aligned with notebook style ML development. They're in many ways a much more powerful version of notebooks. https://www.apple.com/swift/playgrounds/ https://www.apple.com/swift/playgrounds/
  - Abishek_Muthian 7y ago
k__ 7y ago
Good! I think Rust will eat more and more of the stack. It has similar performance as C/C++, but feels more higher level. It's a drop-in replacement for C and is developed by a company that devs like.
- ryuukk_ 7y ago
  rust feels like low level C++, its syntax speaks for itself
- madmax96 7y ago
  Rust is not a drop-in replacement for C, it’s a drop-in replacement for C++.
  - pnako 7y ago
    It's a drop-in replacement for neither since it's an entirely different language. C++ went to some great pain to be a drop-in replacement for C; you can "drop in" a C++ compiler in your toolchain and still compile your C code (mostly).
    - littlestymaar 7y ago
      This was kind of true till C99, but not so much anymore. restrict is the most obvious offender but far from the only one[1]. [1]: https://en.m.wikipedia.org/wiki/Compatibility_of_C_and_C%2B%2B https://en.m.wikipedia.org/wiki/Compatibility_of_C_and_C%2B%...
- zabzonk 7y ago
  If it is a "drop-in replacement" the rust compiler should be able to compile C code - it can't.
lewq 7y ago
Nice article!
yubozhao 7y ago
I think it is also worth examining what type of production load and its importance within the business. For mission critical production usage, I think to use high performance systems/languages is pretty good starting point. With the assumptions that you update your model conservatively(not often), there are enough engineering resources to maintain and 'port' models from python, since most data scientists are trained with python. I think in this type of workload and context, it makes sense use Rust for deployment. Another type of workload I think it is much more common. It is the experimental projects that data scientists are trying to discover does those have enough ROI to be part of the production system. Those projects and deployments are requiring a quick turn around on iteration cycle. I am not sure Rust or even Swift are good tools, when typical data scientists are not well versed in those. Not to mention, usually in this setting, they don't have a lot of engineering resources they can use. Python is still the go to option for this type of work. I think the article has the right intention, speed up ML production. For the experimental work setting, I think we can have the cake and eat it too. Data scientists, still use python and generate production ready deployment service without help from engineers. We create an open source python lib/platform called BentoML(www.github.com/bentoml/bentoml). BentoML makes it easy to serving and deploying ML models in the cloud, from ML model to production API endpoint with few lines of code. You can try it out at this Google Colab notebook(https://colab.research.google.com/github/bentoml/BentoML/blob/master/guides/quick-start/bentoml-quick-start-guide.ipynb https://colab.research.google.com/github/bentoml/BentoML/blo...) BentoML works with multiply ML framworks(Tensorflow/fastai/pytorch/etc) and could generate different distribution formats (docker/AWS Lambda/CLI/Spark UDF) for your serving need. We also support custom runtime backend. Feel free to ping me or ask questions in our slack channel. We are pretty active there.

Rust will never work except in niche areas because it doesn't allow for easy experimentation and does not provide an iterative workflow like Python, R, or Julia. I think the future is Julia, as it's as easy to write as Python, is interactive, and can be compiled for fast speeds! It will just takes time for Julia to have as many libraries, but at some point the tipping point will be reached, and Julia will just eat Python's lunch. Python was dying before the latest wave of data science and machine learning craze saved it anyway. There are pretty good reasons to believe Python (and Ruby) will slowly wither away. I am careful not to savy that they will die out, because they won't as some computing problems do not require so much speed and hence Python and the like can still pass mustard. But in a post-Moore's-law-world, data is getting bigger, the demand for computing is larger, but programs aren't getting faster, and there is a limit to how much you can optimise Python. Eventually, a crop of speedy and easy to write programming languages will rise up and eat Python's lunch. The early signs are there with tools like Julia.

Ygg2 7y ago
> Rust will never work except in niche because It warms my heart to see people don't read TFA. Like at all. From TFA: > But it was important to clean the air of any possible misunderstanding before getting started. I don’t believe Rust will replace Python as the language of choice for ML - it just isn’t happening, it’s not a thing, neither today or tomorrow.
- hu3 7y ago
  What's wrong in expanding on a topic that was barely mentioned in the article? Your parent wrote detailed reasoning that's not present in the article. Also this type of uncharitable comment is against Hacker News guidelines: "Please don't comment on whether someone read an article". http://news.ycombinator.com/newsguidelines.html http://news.ycombinator.com/newsguidelines.html
  - fnord123 7y ago
    There can be no discussion with GGP since he obviously didn't read the article. They think the article will claim Rust supplants Python. They think there is a performance advantage in using Julia instead of Python (but the Python code is just calling C, C++, and Rust as described in the article). GGP thinks there will be a tipping point where Julia has more libraries than Python but this is also addressed in the article. There is no expanding on topics barely mentioned in the article. As far as GGP is concerned there was just a headline and they Kanye'd with "Julia is the best". As far as I care, mods can delete the comment and subtree.
MayeulC 7y ago
That may be true in data science, but the tradeoffs are not necessarily the same in other fields. I do not see Python fading away in mine, for instance. If anything, it looks more and more encroached, and even if the Python code becomes legacy code, interfaces will have to be maintained. I still use TCL quite often in design automation. Python is and will still be used to script Blender, Kodi, Inkscape, Klayout, sigrok, and countless others, for instance. But your week-end project you need to write a bit of code for and want to practice a new language with/just want to code as quick as possible? Sure, the languages will come and go. Likewise with Rust. Maybe not for iterative design phases, but I can certainly see it being used in production, especially if robustness/speed are needed.
mirekrusin 7y ago
Yes, Julia makes some impressive stunts - supercomputing [0], ml where code is very close to mathematical description [1], code reuse [2] etc. [0] https://juliacomputing.com/blog/2019/04/12/Supercomputing-julia.html https://juliacomputing.com/blog/2019/04/12/Supercomputing-ju... [1] https://www.youtube.com/watch?time_continue=188&v=9KBaRS2gy-U&feature=emb_logo https://www.youtube.com/watch?time_continue=188&v=9KBaRS2gy-... [2] https://www.youtube.com/watch?v=kc9HwsxE1OY&t=130s https://www.youtube.com/watch?v=kc9HwsxE1OY&t=130s
deleted 7y ago
[deleted]
pjmlp 7y ago
I am also betting on Julia.
- ilteris 7y ago
  What do you think about Swift
  - pjmlp 7y ago
    It won't go anywhere, Tensorflow for Swift only exists because of Chris Lattner. Many researchers work on Windows, Swift has zero support there and when asked about it, they just kind of invite the Swift community to provide support. So zero interest from either Apple or Google to make it work on Windows. This is the latest status of Swift on Windows, https://www.youtube.com/watch?v=Zjlxa1NIfJc https://www.youtube.com/watch?v=Zjlxa1NIfJc Then on Linux, Foundation support is still flaky, for basic stuff one still needs to do conditional compilation and directly import either Darwin or GLibc modules. Swift is just like Objective-C, yes you can kind of use it outside Apple's ecosytem, but at expense of productivity and lack of adequate tooling. Really I don't expect Tensorflow for Swift to ever reach mainstream adoption, maybe Google can sell it as Mac only and thus announce the project has having been a success.
  - xiaodai 7y ago
    Too verbose. Does not even support Windows. Too closely related to iOS at least in image. MLIR sounds like a nice idea but isn't really Swift specific. I don't understand why they decided to do Swift for Tensorflow, should have invested in Julia for Tensorflow
Certhas 7y ago
I like Julia, but let's be real. We are a long way away from Julia being anywhere near Python when it comes to usability. Error messages, code structure and readability and documentation are afterthoughts rather than a central focus of the community. Because the core is so brilliant at what it does I am sure that Julia will get there eventually, but it will be a painful way. For crying out loud, we still don't have a sory for people to save their data...
- wtetzner 7y ago
  > For crying out loud, we still don't have a sory for people to save their data... Can you elaborate on what you mean by this? Does Julia not support writing files?
  - Certhas 7y ago
    It does, of course, but serialization is problematic (with the combination of Type inference and dynamicism this is maybe to be expected). Beyond that there is a bewildering array of packages, and not really a clear consensus for which to use: https://discourse.julialang.org/t/what-is-the-preferred-way-to-save-variables/21918 https://discourse.julialang.org/t/what-is-the-preferred-way-... JLD2, which for a while seemed like an emerging consensus option has caused massive dataloss for a number of people: https://github.com/JuliaIO/JLD2.jl/issues/55 https://github.com/JuliaIO/JLD2.jl/issues/55 The issue is also that Julia packages often use their own container types for convenient analysis. For example the solution type of DiffEq has a number of important features like interpolation built in. If I want to reliably save the data I got from the numerical integration, and don't want to rely on brittle/broken serialization packages, I have to extract the underlying arrays though. That would be fine if not for the fact that there is no documented way (that I've found) to initialize the solution objects from this raw data, so now I have lost access to the convenience functions. Don't get me wrong, this is all growing pains. I am sure this will be sorted out eventually. But it's important to point out how the intersection of features that enable Julia to do things that are unheard of in other languages (e.g. DiffEqFlux, Zygote) also mean there are some things that still need to be figured out for this language (serialization, useful error messages, interfaces).
cwyers 7y ago
It's not like Python's community is done writing libraries and improving libraries. If there is going to be a tipping point it's going to have to start happening before the Julia package ecosystem catches up to Python's.

psv1 7y ago

Rust can never replace Python as a front-end because it's compiled and not all that flexible or user-friendly. Rust can't currently replace C or C++ at the back-end because it doesn't have good enough support for GPU computing yet.

excerionsforte 7y ago
"The Rust ecosystem is indeed rich in ML crates - just take a look at what a quick search for machine learning on crates.io returns. No need to go and rewrite everything from scratch: I picture linfa as a meta-package, a collection of curated algorithm implementations from the Rust ecosystem. The first stop for your ML needs, as scikit-learn for Python." Rust doesn't have to replace anything. It can be used to create complementary packages especially ones that use C/C++. The point of introducing Rust into your ecosystem is for safety + performance which is hard to achieve without discipline.
- psv1 7y ago
  > Rust doesn't have to replace anything. It can be used to create complementary packages especially ones that use C/C++. I'm not sure that I understand correctly - are you suggesting that people use Rust as an additional layer between Python and C++?
  - eden_h 7y ago
    I've heard this suggestion fairly frequently - that people use Rust as a wrapper over C++ bindings to add in safety to the C++ underneath. Mostly regarding using BLAS etc in Rust rather than C++. It seems a little pointless as the unsafe nature is within the underlying code, so it only santises input/output.
    - pjmlp 7y ago
      For that use case it is more productive to actually use static analysis tooling for C++.
    - 7y ago

wdroz 7y ago

The 25x speedup is because the gRPC python implementation is very slow [0]. [0] -- https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5636470266134528 https://performance-dot-grpc-testing.appspot.com/explore?das...

kbumsik 7y ago
Yeah, by skimming the article, it sound like making gRPC microservices 25x faster, not ML.
- nevi-me 7y ago
  gRPC would account for very little time relative to the k-means algorithm. I think what results in the large speedup is the "embarrassing" parallelism that an async Rust server (gRPC in this case) would give. Knowing the CPU time during the tests could confirm this. I'll try run the benchmarks on my machine this week to verify my assumption.
  - bpicolo 7y ago
    I've hit some degenerate cases on gRPC serialization. The golang grpc/proto library accounted for something like 40% of cpu time doing serialization on mapping types for an API I developed under load test. Changing that to a list of key/value pairs dropped that 40% to <1%. The gRPC python sever implementation does a lot of work using python builtins, and doesn't use recent asyncio primitives. It uses built-in python thread-pool execution, which means you also have the GIL to worry about: https://grpc.io/docs/tutorials/basic/python/#starting-the-server https://grpc.io/docs/tutorials/basic/python/#starting-the-se... Given that, I would expect its connection / request management to be pretty darn slow. gRPC has better support for Python as the client. The thread pool execution model also prevents you from doing a number of things you might want to do with gRPC servers (long-lived request streaming is out of the picture). There are a couple third party packages out there trying to make this better (e.g. https://github.com/vmagamedov/grpclib https://github.com/vmagamedov/grpclib) but I hit compatibility issues for a few cases when trying to use them for some prototypes.
- 7y ago

captain_price7 7y ago

I was a little surprised to see that instead of directly benchmarking prediction performance on a local machine, author chose to go through that complicated web-server route. I saw author saying this is nothing more than a remote RFC, still, a benchmark should try to remove unrelated factors as much as possible. I would be interested to see how the results compare when this web-service layer is removed.

cwyers 7y ago
Serving model predictions via an API is a pretty common usecase for putting data science models in production. If Python's web service layer is slower than Rust's, that's a real concern even if it's not directly related to model prediction.

sandGorgon 7y ago

Although I am a python guy, if I had to bet on an alternative language here - its going to be Swift. The amount of evangelism money that Google is spending on Tensorflow is staggering. Its not just t-shirts, but an entire ecosystem - TensorBoard.dev, Tensorflow Privacy, TF Enterprise, Explainable AI Beta, free TPU on Google Colab etc. And all of them will soon be backed using Tensorflow Swift (https://www.tensorflow.org/swift https://www.tensorflow.org/swift). This is MNIST for Swift Tensorflow - https://github.com/tensorflow/swift-models/blob/master/Datasets/MNIST/MNIST.swift https://github.com/tensorflow/swift-models/blob/master/Datas... . Mind you this is runnable on Google Colab using TPU hardware acceleration in swift. Swift vs Rust is a very political thing...but previous discussions on HN have been tilted in favor of Swift [1] And we havent even touched on the elephant in the Room - Swift may soon hook into OSX Metal (which is arguably the only way to do GPU AI on macbooks now that the nvidia-Apple CUDA divorce is official [2] ). You can already call Metal from Swift...but not write shaders directly IMHO. [1] https://cloud.google.com/explainable-ai/ https://cloud.google.com/explainable-ai/ [2] https://gizmodo.com/apple-and-nvidia-are-over-1840015246 https://gizmodo.com/apple-and-nvidia-are-over-1840015246

Roritharr 7y ago
This is really new to me, I always thought of Swift as the iOS thing to look into when doing something there. Very interesting that Google is pushing Swift for their Tensorflow Platform. Any ideas why Swift is of interest for Google?
- jorgemf 7y ago
  They hired one of the Swift leaders for the tensor flow team
- bpicolo 7y ago
  Swift playgrounds seem very much aligned with notebook style ML development. They're in many ways a much more powerful version of notebooks. https://www.apple.com/swift/playgrounds/ https://www.apple.com/swift/playgrounds/
- Abishek_Muthian 7y ago

k__ 7y ago

Good! I think Rust will eat more and more of the stack. It has similar performance as C/C++, but feels more higher level. It's a drop-in replacement for C and is developed by a company that devs like.

ryuukk_ 7y ago
rust feels like low level C++, its syntax speaks for itself
madmax96 7y ago
Rust is not a drop-in replacement for C, it’s a drop-in replacement for C++.
- pnako 7y ago
  It's a drop-in replacement for neither since it's an entirely different language. C++ went to some great pain to be a drop-in replacement for C; you can "drop in" a C++ compiler in your toolchain and still compile your C code (mostly).
  - littlestymaar 7y ago
    This was kind of true till C99, but not so much anymore. restrict is the most obvious offender but far from the only one[1]. [1]: https://en.m.wikipedia.org/wiki/Compatibility_of_C_and_C%2B%2B https://en.m.wikipedia.org/wiki/Compatibility_of_C_and_C%2B%...
zabzonk 7y ago
If it is a "drop-in replacement" the rust compiler should be able to compile C code - it can't.

lewq 7y ago

Nice article!

yubozhao 7y ago

I think it is also worth examining what type of production load and its importance within the business. For mission critical production usage, I think to use high performance systems/languages is pretty good starting point. With the assumptions that you update your model conservatively(not often), there are enough engineering resources to maintain and 'port' models from python, since most data scientists are trained with python. I think in this type of workload and context, it makes sense use Rust for deployment. Another type of workload I think it is much more common. It is the experimental projects that data scientists are trying to discover does those have enough ROI to be part of the production system. Those projects and deployments are requiring a quick turn around on iteration cycle. I am not sure Rust or even Swift are good tools, when typical data scientists are not well versed in those. Not to mention, usually in this setting, they don't have a lot of engineering resources they can use. Python is still the go to option for this type of work. I think the article has the right intention, speed up ML production. For the experimental work setting, I think we can have the cake and eat it too. Data scientists, still use python and generate production ready deployment service without help from engineers. We create an open source python lib/platform called BentoML(www.github.com/bentoml/bentoml). BentoML makes it easy to serving and deploying ML models in the cloud, from ML model to production API endpoint with few lines of code. You can try it out at this Google Colab notebook(https://colab.research.google.com/github/bentoml/BentoML/blob/master/guides/quick-start/bentoml-quick-start-guide.ipynb https://colab.research.google.com/github/bentoml/BentoML/blo...) BentoML works with multiply ML framworks(Tensorflow/fastai/pytorch/etc) and could generate different distribution formats (docker/AWS Lambda/CLI/Spark UDF) for your serving need. We also support custom runtime backend. Feel free to ping me or ask questions in our slack channel. We are pretty active there.