ag2718

7 ms·

1.
by ag2718 3mo ago
This is a simplification in the blog post: each activation doesn't map one-to-one onto a physical FPGA LUT primitive, but is instead represented as a "logical LUT" (L-LUT) that Vivado synthesizes into distributed RAM. These L
2.
by ag2718 3mo ago
Our end-to-end implementation can be found here! https://github.com/Duchstf/KANELE
3.
by ag2718 3mo ago
Some very cool applications of small models! It seems that this scale of models tends to be sufficient when doing simpler classification, anomaly detection, signal processing, etc. as compared to generative modeling (where larger models are
4.
by ag2718 3mo ago
That is a really cool application of FPGA-based machine learning that I would not have thought of :)
5.
by ag2718 3mo ago
This is definitely true: one could imagine a model with a mix of the two layers or a simple linear / MLP-like kernel doing "preprocessing" before KAN layers. Other work that explores task performances for KANs and MLPs genera
6.
by ag2718 3mo ago
Thank you :)
7.
by ag2718 3mo ago
Ah I see, that's an interesting point about higher depth potentially having other benefits. For our work on smaller models (e.g. generally <5 layers), this might not have been as relevant but I would definitely be interested to see
8.
by ag2718 3mo ago
A key benefit of KANs is expressivity, as each layer is significantly more expressive than an MLP layer. This can be seen in our benchmarks: KAN networks need fewer layers than MLPs to match or beat their performance, even in software. Ho
9.
by ag2718 3mo ago
There is definitely a precision-performance tradeoff to consider. We explored this through ablation studies on bitwidth precision / resource usage in our work (Figure 6a in https://arxiv.org/pdf/2512.12850 , Figure
10.
by ag2718 3mo ago
You're correct that this work is not very applicable for LLMs and that the focus here is primarily on latency.
11.
by ag2718 3mo ago
Yes, definitely: this type of work is applicable in domains where software run on general-purpose processors cannot meet latency or power requirements.
12.
by ag2718 3mo ago
Hmm the post is still up for me?
13.
by ag2718 3mo ago
Yes, this work is focused on accelerating very small models, typically for real-time systems that require extremely low power or low latency. One primary application of this work is in high-energy physics ( https://home.cern/
14.
Ultrafast machine learning on FPGAs via Kolmogorov-Arnold Networks(aarushgupta.io)
283 points by ag2718 3mo ago|52 comments