nicoinstrument

18 ms·

1. 1.▲
   by nicoinstrument 4mo ago

   Thanks for the suggestion! Have you found that Ray Serve’s built-in autoscaling plays nicely with custom SLO-based concurrency limits, or do you usually let Ray handle the load balancing entirely?"
2. 2.▲
   by nicoinstrument 4mo ago

   I'm learning about inference by running vLLM on a k8s cluster (EKS), building a gateway to keep a <2s TTFT SLO. Most recent ha-ha moment: I kept wondering if it was normal that my cluster was only able to process 4 requests per seco
3. 3.▲
   by nicoinstrument 1y ago

   Chime | Software Engineer, Infrastructure | Full Time | REMOTE (US) We’re looking for talented engineers who: • Have hands-on experience in infrastructure or platform engineering. • Enjoy working on distributed systems, scaling, automatio
4. 4.▲
   by nicoinstrument 3y ago

   Thank you! 1. I think it can be compared to a process of rendering the charts on your local machine to preview the changes and possibly pasting them in your PR for the reviewer. Sort of like mani-diffy by fully manual, yuck. 2. The other o
5. 5.▲
   by nicoinstrument 3y ago

   you can watch it full length (~1hr) with a San Francisco library account : https://sfpl.bibliocommons.com/v2/record/S93C4825237