robertnishihara

8 ms·

1.
High Performance Distributed Inference with Ray Serve LLM(anyscale.com)
3 points by robertnishihara 3mo ago|0 comments
2.
Data Processing Is Becoming a GPU Workload(anyscale.com)
2 points by robertnishihara 3mo ago|0 comments
3.
67% Cost Savings with PD Disaggregation Using Ray and vLLM on AMD MI325X(anyscale.com)
4 points by robertnishihara 3mo ago|0 comments
4.
Major upgrades to Ray Serve: 88% lower latency and 11.1x higher throughput(anyscale.com)
2 points by robertnishihara 6mo ago|1 comments
5.
SkyRL brings Tinker to your GPUs (2025)(novasky-ai.notion.site)
24 points by robertnishihara 7mo ago|5 comments
6.
vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep(blog.vllm.ai)
147 points by robertnishihara 8mo ago|54 comments
7.
Massively Parallel Agentic Simulations with Ray(anyscale.com)
2 points by robertnishihara 1y ago|0 comments
8.
Deploy DeepSeek‑R1 with VLLM and Ray Serve on Kubernetes(anyscale.com)
1 points by robertnishihara 1y ago|0 comments
9.
An Open Source Stack for AI Compute: Kubernetes and Ray and PyTorch and VLLM(anyscale.com)
1 points by robertnishihara 1y ago|0 comments
10.
Native LLM APIs in Ray Data and Ray Serve(anyscale.com)
2 points by robertnishihara 1y ago|0 comments
11.
Joins and Hash-Shuffle in Ray Data(anyscale.com)
3 points by robertnishihara 1y ago|0 comments
12.
AsyncFlow: An Asynchronous Streaming RL Framework for LLM Post-Training(arxiv.org)
4 points by robertnishihara 1y ago|0 comments
13.
Open Source RL Libraries for LLMs(anyscale.com)
1 points by robertnishihara 1y ago|0 comments
14.
Large-Scale Deployment of Ray in Tencent's Weixin AI Infrastructure(anyscale.com)
2 points by robertnishihara 1y ago|0 comments
15.
Uv and Ray: Pain-Free Python Dependencies in Clusters(anyscale.com)
44 points by robertnishihara 1y ago|10 comments
16.
Roll: Reinforcement Learning Optimization for Large-Scale Learning(github.com)
1 points by robertnishihara 1y ago|0 comments
17.
An Open Source Stack for AI Compute: Kubernetes and Ray and PyTorch and VLLM(anyscale.com)
1 points by robertnishihara 1y ago|0 comments
18.
Uv and Ray: Pain-Free Python Dependencies in Clusters(anyscale.com)
1 points by robertnishihara 2y ago|0 comments
19.
Ray Batch Inference at Pinterest (Part 3)(medium.com)
1 points by robertnishihara 2y ago|0 comments
20.
Direct Preference Optimization with Synthetic Data on Anyscale(anyscale.com)
1 points by robertnishihara 2y ago|0 comments
21.
by robertnishihara 2y ago
To clarify, what I mean is that working with "exabytes" is atypical. Most use cases are at a slightly smaller scale :) Data processing workloads are quite common on Ray, especially with unstructured data. Also, I work on Ray, whic
22.
by robertnishihara 2y ago
Other folks have built data processing libraries on top of Ray: Modin and Daft come to mind. But I'm not aware of anything exactly like what you're referring to!
23.
by robertnishihara 2y ago
Multi-threaded libraries (e.g., numpy and PyTorch on CPUs come to mind) are well supported. In scenarios where many processes are each running heavily multi-threaded computations, it can help to pin specific processes to specific cores (e.g
24.
by robertnishihara 2y ago
I'm glad you find it exciting! Our intention from the start was for Ray to be general purpose. And the core Ray APIs are quite general (basically just scheduling a Python function somewhere in a cluster or instantiating a Python class
25.
by robertnishihara 2y ago
Yeah, mmap, I think this is the relevant line [1]. Fun fact, very early on, we used to create one mmapped file per serialized object, but that very quickly broke down. Then we switched to mmapping one large file at the start and storing all
26.
by robertnishihara 2y ago
Your right that the serialization / deserialization overhead can quickly exceed the compute time. To avoid this you have to get a lot of small things right. And given our focus on ML workloads, this is particularly important when shari
27.
by robertnishihara 2y ago
I'm one of the creators of Ray. A few thoughts :) 1. This is truly impressive work from AWS. Patrick Ames began speaking about this a couple years ago, though at this point the blog post is probably the best reference. https:/&#x
28.
Building an LLM Router for High-Quality and Cost-Effective Responses(anyscale.com)
1 points by robertnishihara 2y ago|0 comments
29.
Ray Infrastructure at Pinterest(medium.com)
1 points by robertnishihara 2y ago|0 comments
30.
Lessons from training a Stable Diffusion model on 2B images(anyscale.com)
5 points by robertnishihara 2y ago|0 comments

More ›