mukel

6 ms·

1.
Show HN: Jinfer – AI inference engine for the JVM. AI in a jar(qxotic.ai)
3 points by mukel 3d ago|0 comments
2.
Gemma4.java: Run Gemma 4 in pure Java (no Python, no JNI)(github.com)
4 points by mukel 5mo ago|1 comments
3.
by mukel 5mo ago
I built llama3.java in the past, this is a follow-up: Gemma 4 running entirely on the JVM. No Python. No JNI. No native code. Just Java. It’s (mostly) a single Java file implementing the full stack: GGUF parsing, tokenization, Gemma 4 trans
4.
by mukel 2y ago
Code: https://github.com/mukel/llama3.java
5.
Fast Llama inference in pure, modern Java(youtube.com)
3 points by mukel 2y ago|3 comments
6.
by mukel 2y ago
Features: - Single file, no dependencies - GGUF format parser - Llama 3 tokenizer - Support Llama 3, 3.1 (ad-hoc RoPE scaling) and 3.2 (tie word embeddings) - Fast matrix-vector multiplication routines for Q4_0 and Q8_0 quantized tenso
7.
by mukel 2y ago
Llama3.java: featuring .GGUF file format support, Q8_0 and Q4_0 quantizations, fast matrix/vector multiplication routines using Java's Vector API; served by a simple CLI with a --chat mode to interact with the Llama 3 models.
8.
Practical Llama 3 inference in Java(github.com)
4 points by mukel 2y ago|1 comments
9.
by mukel 3y ago
This will boost adoption at so many levels: - Importing a Truffle language as a regular Maven dependency - Ease integration with mainstream package managers - Ability to update to the latest language version, independently of the JVM used I
10.
by mukel 3y ago
Author here: I implemented several versions of matmul with different unrolling schemes using the Vector API and I got a ~4X speedup with a single thread, but the speedup fades the more threads you add. I think that performance is constraine
11.
by mukel 3y ago
A Java port of llama2.c that performs very close to C on large models. Llama 2 7B runs at a whooping 1.6 tokens/s.
12.
Llama2.java: Karpathy's llama2.c ported to Java(github.com)
33 points by mukel 3y ago|18 comments
13.
by mukel 4y ago
GraalVM team member here. Implementing any mainstream language is indeed a challenge, more so if you have to maintain bug-compatibility and cope with all the bits of bad design that went through the cracks in the de-facto implementation. Tr
14.
by mukel 6y ago
The goal is not to compete, but rather complement HotSpot/GraalVM with: - (Polyglot) scripting with Java - Augmenting native images e.g. native javac with instant startup + annotation processors (very dynamic) running on Espresso - A
15.
by mukel 6y ago
It's a Linux-only trick: https://github.com/kt97679/tetris/blob/52dfb3a703e4dd5b37990...
16.
by mukel 6y ago
Java on Truffle is a normal Java application, it can run on a vanilla OpenJDK, doesn't need the bootstrap step. Currently Graal/Truffle implements the first Futamura projection: you give it an interpreter written in Java, it autom
17.
by mukel 6y ago
Running on HotSpot is an advantage for the development of Java on Truffle itself; the tooling is amazing, debugger, inspectability, you can debug the VM as a normal Java application... GraalVM can compile an AOT version of Java on Truffle,
18.
A meta-circular Java bytecode interpreter for GraalVM(graalvm.org)
3 points by mukel 6y ago|1 comments
19.
by mukel 6y ago
Java on Truffle is an implementation of the Java Virtual Machine Specification, Java SE 8 and Java SE 11, built upon GraalVM as a Truffle interpreter. It is a minified Java VM that includes all core components of a VM, implements the same A
20.
by mukel 7y ago
Google is going downhill, the day I finished my internship I sweared I'll never in my life work on any money-making, no-challenge project. A bunch of engineers do enjoy what they do, they work on the cool projects, that's enough t