olokobayusuf

33 ms·

1.
Making PyTorch –> Qualcomm NPUs less treacherous(muna.ai)
1 points by olokobayusuf 7mo ago|1 comments
2.
by olokobayusuf 7mo ago
There are over 2.5 billion Qualcomm processors in the world today (PC, mobile, automotive, etc). But the process for bringing AI models to run on Qcom processors is a (massive) pain. Their 2GB+ SDK is an encyclopedia's worth of informa
3.
Show HN: Our command line tool to transpile AI Inference from Python to C++(github.com)
4 points by olokobayusuf 8mo ago|0 comments
4.
by olokobayusuf 11mo ago
Congrats on the launch!
5.
by olokobayusuf 1y ago
We should collab! We prefer to be the underlying infrastructure behind the scenes, and have a pretty holistic approach towards hardware coverage and performance optimization. Read more: - https://blog.codingconfessions.com/p
6.
by olokobayusuf 1y ago
We're building something closer to this at Muna: https://docs.muna.ai . Check us out and let me know what you think!
7.
by olokobayusuf 1y ago
This is super interesting! I'm the founder of Muna ( https://docs.muna.ai ) with much of the same underlying philosophy, but a different approach: We're building a general purpose compiler for Python. Once compiled, deve
8.
by olokobayusuf 1y ago
Yup that's true. We do benefit from massive efficiencies though, thanks to LLM codegen.
9.
by olokobayusuf 1y ago
Our primary use case is cross-platform AI inference (unsurprising), and for that use case we're already in production by startups to larger co's. It's kind of funny: our compiler currently doesn't support classes, but we
10.
by olokobayusuf 1y ago
I'm founding a company that is building an AOT compiler for Python (Python -> C++ -> object code) and it works by propagating type information through a Python function. That type propagation process is seeded by type hints on th
11.
Compiling Python to Run Anywhere(blog.codingconfessions.com)
4 points by olokobayusuf 1y ago|0 comments
12.
by olokobayusuf 1y ago
Function ( https://fxn.ai ) | Remote (US) We're building native code generation for AI developers. We generate high-performance C++/Rust to power open-source and on-device AI for our customers. We have customers ranging
13.
by olokobayusuf 1y ago
Link to original article that kickstarted all of this: https://medium.com/@eugeniyoz/powering-angular-with-rust-was...
14.
Show HN: Powering React with Python (WASM)(medium.com)
3 points by olokobayusuf 1y ago|1 comments
15.
by olokobayusuf 1y ago
When we trace Python code, devs have to explicitly opt-in dependency modules to tracing. Specifically, the `@compile` decorator has a `trace_modules` parameter which is a `list[types.ModuleType]`. With this in place, when we trace through a
16.
by olokobayusuf 1y ago
Not entirely sure what you mean by having to deal with C-API and having a language spec. We're also not competing with LLMs at all--we use LLMs for said conversion (under strict verification requirements).
17.
by olokobayusuf 1y ago
Actually we're currently implementing Numpy (and PyTorch) support, and will cover a few other core scientific computing libraries like scipy. See docs: https://docs.fxn.ai/predictors/requirements#library-coverage
18.
by olokobayusuf 1y ago
Not quite. First, Function is designed to be truly cross-platform but libraries like Numpy aren't compiled for say WebAssembly. Second, the native libraries are usually built around CPython interop (i.e. the C API expects to interact w
19.
by olokobayusuf 1y ago
Spot on! The majority of the innovation here is in building enough rails (specifically around lowering Python's language features to native code) so that LLM codegen can help you transform any Python code into equivalent native code (C
20.
by olokobayusuf 1y ago
I think a more pedantic way to describe what I mean is: "What if we could compile Python into raw native code *without having a Python interpreter*?" The key distinguishing feature of this compiler is being able to make standalone
21.
by olokobayusuf 1y ago
Yes, we upload user code to a cloud sandbox in order to run our symbolic tracing and code generation algorithm. Beyond that, we also compile the generated native code in the cloud so that devs don't have to have a cross-compiler instal
22.
by olokobayusuf 1y ago
Way ahead of you: https://github.com/olokobayusuf/python-vs-rust/blob/main/Car... I've clarified that this is not designed to be a rigorous benchmark. We've got rigorous benchmarks coming for i
23.
by olokobayusuf 1y ago
More serious reply: us and Mojo have similar visions of where the world is going. The key difference between us is that Mojo is itself a new programming language. Sure, it supports Python, but it doesn't actually compile Python. It sim
24.
by olokobayusuf 1y ago
We use LLVM! We're not translating Python directly to LLVM IR (I think I've seen other projects do this). We translate Python to C++/Rust first, where we have rigorous unit tests for every operation we support translating. We
25.
by olokobayusuf 1y ago
There's actually no overhead of "still using Python" because we don't use Python. The overhead in Function exists solely because we have a bunch of sugar added to create a unified interface for calling different kinds of
26.
by olokobayusuf 1y ago
https://x.com/OlokobaYusuf/status/1908538983874810303
27.
by olokobayusuf 1y ago
We're focused specifically on on-device AI inference (and related compute-bound algorithms, like computer vision or scientific computing). We want devs to find or develop inference code (always in Python); decorate it with Function&#x2
28.
by olokobayusuf 1y ago
Yup but you're skilled enough to write--and more importantly, maintain--the required C/C++ code. Most devs and companies we talk to just want to make something people want; they don't care for the added complexity of writing
29.
by olokobayusuf 1y ago
No need to be snarky. The choice to default to FP32 is inspired by the fact that most typical use cases don't need double-precision (GPU shader languages and game engines do this all the time). This in turn allows us to vectorize code
30.
by olokobayusuf 1y ago
Thank you <3

More ›