ben_s

17 ms·

1.
by ben_s 9mo ago
Once you oversubscribe GPU memory, performance usually collapses. Frameworks like vLLM can explicitly offload things like the KV cache to CPU memory, but that's an application-level tradeoff, not transparent GPU virtual memory.
2.
by ben_s 9mo ago
We didn't focus on vGPU and largely avoided it on purpose. Instead, we focused on whole-GPU and NVSwitch-partitioned passthrough (Shared NVSwitch Multitenancy Mode), which is a better fit for the workloads we care about.
3.
by ben_s 9mo ago
In this article, we're primarily concerned with whole-GPU or multi-GPU partitions that preserve NVLink bandwidth, rather than finer-grained fractional sharing of a single GPU.
4.
by ben_s 9mo ago
We haven't looked deeply at inter-machine communication yet. NVLink/NVSwitch (which this post focuses on) are intra-node, so InfiniBand is mostly orthogonal I think and comes down to NIC passthrough, NUMA/PCIe placement, and
5.
by ben_s 9mo ago
Thanks for the comment! You're right that a lot of the mechanics apply more generally. On point (3) specifically: we handle this by allocating at the IOMMU-group level rather than individual devices. Our allocator selects an IOMMU grou
6.
by ben_s 9mo ago
Fabric Manager itself is not open source. It's NVIDIA-provided software, and today it's required to bring up and manage the NVLink/NVSwitch fabric on HGX systems. What we meant by "open" is that everything around it
7.
by ben_s 9mo ago
Thanks! I haven't looked deeply into slicing up a single GPU. My understanding is that vGPU (which we briefly mention in the post) can partition memory but time-shares compute, while MIG is the only mechanism that provides partitioning
8.
by ben_s 9mo ago
(author of the blog post here) For me, the hardest part was virtualizing GPUs with NVLink in the mix. It complicates isolation while trying to preserve performance. AMA if you want to dig into any of the details.
9.
Virtualizing Nvidia HGX B200 GPUs with Open Source(ubicloud.com)
116 points by ben_s 9mo ago|28 comments
10.
Fifty problems with standard web APIs in 2025(zerotrickpony.com)
2 points by ben_s 9mo ago|0 comments
11.
These Aren't the Tools You're Looking For: The Hidden Dangers of MCP(ainativedev.io)
1 points by ben_s 9mo ago|0 comments
12.
How to review AI generated PRs(thoughtbot.com)
1 points by ben_s 9mo ago|1 comments
13.
Snowflake: Building a Data Cloud(medium.com)
1 points by ben_s 2y ago|0 comments
14.
Finding Stuff in the Terminal(flakm.com)
1 points by ben_s 2y ago|0 comments
15.
The What, Why and How of Containers(annwan.me)
226 points by ben_s 2y ago|128 comments
16.
PostgreSQL Heap-Only-Tuple or Hot Update Optimization(vladmihalcea.com)
1 points by ben_s 2y ago|0 comments
17.
KeepYourMouthShut – A Python program to auto-generate Podcasts(github.com)
2 points by ben_s 2y ago|0 comments
18.
Container Loading in AWS Lambda(brooker.co.za)
2 points by ben_s 2y ago|0 comments
19.
Equations That Changed the World – Rewritten in JavaScript(runjs.app)
3 points by ben_s 2y ago|0 comments
20.
Logging: The Unsung Hero in Developer Security – Here's Why and How(withstandsecurity.com)
5 points by ben_s 2y ago|0 comments
21.
CloudZip: List/read from remote zip files without downloading them(github.com)
2 points by ben_s 2y ago|0 comments
22.
Exploring Gleam with Genetic Algorithms(silasmarvin.dev)
1 points by ben_s 2y ago|0 comments
23.
DuckDB Walks to the Beat of Its Own Analytics Drum(datanami.com)
2 points by ben_s 2y ago|0 comments
24.
The Tragic Race Condition(codereliant.io)
1 points by ben_s 2y ago|0 comments
25.
How to keep your product alive: poor man's SRE(pixeljets.com)
2 points by ben_s 2y ago|0 comments
26.
Relational Data at the Edge: How Cloudflare Operates Distributed Postgres(infoq.com)
1 points by ben_s 2y ago|0 comments
27.
When Postgres Is Not Enough? Perf Evaluation of Postgres vs. Distributed DBMSs(blog.ydb.tech)
2 points by ben_s 2y ago|0 comments
28.
Postgres Performance Boost: Hot Updates and Fill Factor(crunchydata.com)
1 points by ben_s 2y ago|0 comments
29.
Distributed Queries for Pgvector(jkatz05.com)
1 points by ben_s 2y ago|0 comments
30.
A Comprehensive Comparison for Vector Search in PostgreSQL(blog.pgvecto.rs)
2 points by ben_s 2y ago|0 comments

More ›