shivanshvij

6 ms·

1.
by shivanshvij 4mo ago
We started an Ollama container on a MacBook. There's no NVIDIA GPU, no CUDA toolkit, and macOS doesn't even have CUDA drivers. Ollama found an NVIDIA GPU anyway: A 128 GB Blackwell GPU on a DGX Spark across the network. Our anppro
2.
Ollama Doesn't Know Its GPU Is on Another Machine(loopholelabs.io)
5 points by shivanshvij 4mo ago|1 comments
3.
by shivanshvij 8mo ago
Founder of loophole labs here! Cilium is definitely the gold standard if you’re working with Kubernetes clusters and need a full CNI, but if you want to extend CNI functionality without replacing it, then this approach is the only option.
4.
by shivanshvij 11mo ago
Sorry about that! It's fixed!
5.
by shivanshvij 11mo ago
Sorry about that, we've fixed it!
6.
by shivanshvij 11mo ago
That's a great point. This was one of my iPerf3 runs at one point: `iperf3 -c 172.31.45.187 -p 5201 -P 128 -t 5 -b 512M -u -l 1448 --bidir | grep "\[SUM\]\[\(TX\|RX\)-C\].*receiver"` We're also looking at using packet ge
7.
by shivanshvij 11mo ago
It is! We demo'd it on stage at KubeCon last hear: https://loophole.sh/kc2024
8.
by shivanshvij 11mo ago
Wow thank for the recording! We were able to push out a fix, should be workin g now!
9.
by shivanshvij 11mo ago
No this was multiple connections - and we tried with both `iperf2` and `iperf3`, UDP and TCP traffic. UDP actually does much worse on `iptables` than TCP, and I'm not sure why just yet.
10.
by shivanshvij 11mo ago
We absolutely ran into these issues. A couple notes that help quite a bit: 1. Always build the eBPF programs in a container - this is great for reproducibility of course, but also makes DevX on MacOS better for those who prefer to use that.
11.
by shivanshvij 11mo ago
Oh no we absolutely have to also correct the IPv4 header checksum!
12.
by shivanshvij 11mo ago
Awesome we’ll be looking forward to it!
13.
by shivanshvij 11mo ago
Actually the latest benchmarks were ran on a Fedora 43 host, which as far as I can tell uses the nftables backend for iptables!
14.
by shivanshvij 11mo ago
As far as we can tell, it’s a mixture of a lot of things. One of the questions I got asked was how useful this is if you have a smaller performance requirement than 200Gbps (or, maybe a better way to put it, what if your host is small and c
15.
by shivanshvij 11mo ago
Hi HN, Shivansh (founder) here, happy to answer any questions folks might have about the implementation and the benchmarks!
16.
An eBPF loophole: Using XDP for egress traffic(loopholelabs.io)
243 points by shivanshvij 11mo ago|77 comments
17.
by shivanshvij 11mo ago
XDP (eXpress Data Path) is the fastest packet processing framework in linux - but it only works for incoming (ingress) traffic. We discovered how to use it for outgoing (egress) traffic by exploiting a loophole in how the linux kernel deter
18.
by shivanshvij 4y ago
Hey Everyone, Founder of Loophole Labs here! The team and I are happy to answer any questions you might have about Scale Functions or Loophole Labs in general!
19.
by shivanshvij 4y ago
We wrote fRPC because we really liked the DevX and tooling around the proto3 syntax, but we needed the generated code to be significantly more performant than what gRPC provides. We also needed the ability to extend the RPC framework with o
20.
by shivanshvij 4y ago
Hey Everyone, Founder of Loophole Labs here. The team and I are happy to answer any questions you might have about fRPC, Frisbee, or Loophole in general!