ospillinger

6 ms·

1.
by ospillinger 7y ago
That makes sense. Programmatically updating cortex.yaml is a common use case especially when you're thinking about continuous deployment. We also have a Python client which can replace the cortex.yaml file ( https://www.corte
2.
by ospillinger 7y ago
Hey Aaron, I work on Cortex which is a tool for continuously deploying models as HTTP endpoints on AWS. Under the hood we use Kubernetes instead of Lambda to avoid cold starts, enable more flexibility with customizing compute and memory usa
3.
by ospillinger 7y ago
From the MLflow Models docs: "An MLflow Model is a standard format for packaging machine learning models that can be used in a variety of downstream tools—for example, real-time serving through a REST API or batch inference on Apache S
4.
by ospillinger 7y ago
Each model is loaded into a Docker container, along with any Python packages and request handling code. The cluster runs on EKS on your AWS account. Cortex takes the declarative configuration from 'cortex.yaml' and creates it ever
5.
by ospillinger 7y ago
Yes, Cortex uses ONNX Runtime ( https://github.com/microsoft/onnxruntime ) under the hood so any model that can be exported to ONNX can be deployed.
6.
by ospillinger 7y ago
Good idea, I definitely think it's doable.
7.
by ospillinger 7y ago
The Terraform provider idea is interesting, I'll think about it more carefully. Almost all of our deployment configuration under the hood is done with Kubernetes (which is focused on the declaration of deployment state). We modeled our
8.
by ospillinger 7y ago
My understanding is that Seldon and Kubeflow are more geared towards infrastructure engineers. Our goal is to hide the infrastructure tooling so that Kuberentes, Docker, or AWS expertise isn’t required. Cortex installs with one command, mod
9.
by ospillinger 7y ago
Thanks for the feedback! We aren't trying to invent another infrastructure provisioning language, and I agree that Terraform would be the right choice if that was the case. Our YAML is more similar to the configuration of deployment to
10.
by ospillinger 7y ago
We solve a similar problem to SageMaker but we are focused on developer experience and flexibility. - Deployments are defined with declarative configuration and no custom Docker images are required (although you can use your own if you want
11.
Cortex – An ML model deployment platform that runs in your AWS account(github.com)
65 points by ospillinger 7y ago|24 comments