craigacp

8 ms·

1.
by craigacp 11mo ago
SIMT - Single Instruction, Multiple Threads
2.
by craigacp 1y ago
The same operations in the same order is a tough constraint in an environment where core count is increasing and clock speeds/IPC are not. It's hard to rewrite some of these algorithms to use a parallel decomposition that's t
3.
by craigacp 2y ago
Some of the gradients are present in the TF C API (and thus the C++ API), but it's hit & miss which are in C and which are in Python. There was an attempt years ago to port more gradients to C, but this petered out in the way that
4.
by craigacp 2y ago
The encoder's embedding is contextual, it depends on all the tokens. If you pull out the embedding layer from a decoder only model then that is a fixed embedding where each token's representation doesn't depend on the other t
5.
by craigacp 2y ago
There are a bunch of these things in a word2vec space. I had a blog post years ago on my group's blog which trained word2vec on a bunch of wikias so we could find out who is the Han Solo of Doctor Who (which I think somewhat inexplicab
6.
by craigacp 2y ago
Java has had the ability to run a single code file directly by running `java <path-to-file.java>` since Java 11 - https://openjdk.org/jeps/330 . And it's had a Java REPL since Java 9.
7.
by craigacp 3y ago
There's usually a two or three step training procedure, first training to predict the next word on a huge corpus of text (billions or trillions of words), then possibly some instruction tuning (giving the model question & answer pa
8.
by craigacp 3y ago
I did. It depends what you want, for an overview of how ONNX Runtime works then Microsoft have a bunch of things on https://onnxruntime.ai , but the Java content is a bit lacking on there as I've not had time to write much.
9.
by craigacp 3y ago
Both GPU support and SIMD CPU support are being worked on in OpenJDK projects, there are talks on them from last year's JVM language summit - https://youtube.com/playlist?list=PLX8CzqL3ArzW90jKUCf4H6xCK...
10.
by craigacp 3y ago
ONNX Runtime needed a good demo application in Java and this was fun to do (I maintain the Java API for ONNX Runtime and wrote this SD implementation). I've added SDv2 and SDXL support (and the turbo variants thereof) after the initial
11.
by craigacp 3y ago
There's a correction to that tweet, larger vocab means fewer tokens for any given sequence (usually, assuming it's not to add other languages or character sets).
12.
by craigacp 3y ago
As ONNX models are protobufs you can edit them at a Python or Java REPL (or other language but I've personally used those two). Dumping them out as text seems like a lot more work, and a lot less typesafe.
13.
by craigacp 4y ago
We built model & data provenance into our open source ML library, though it's admittedly not the W3C PROV standard. There were a few gaps in it until we built an automated reproducibility system on top of it, but now it's pret
14.
by craigacp 4y ago
A conventional neural network, i.e. one using a stack of dense layers, can't unroll across a sequence in the way the transformer does. So while it could compute the relative importance and interaction of the features it sees it wouldn&
15.
by craigacp 4y ago
Exactly so, there are a bunch of Java interfaces to native ML libraries that have some messy bit hacking to convert between fp16 and Java floats. This API along with the intrinsification ( https://github.com/openjdk/jdk&
16.
by craigacp 5y ago
Probabilistic programming can be done via MCMC approaches, but you can also infer the necessary quantities by using variational inference (which approximates the distribution described by your program with something that's simpler and
17.
by craigacp 5y ago
There is extensive discussion of that topic in the valhalla-dev and valhalla-spec-experts mailing lists, but I think the details are still being worked out.
18.
by craigacp 5y ago
There are JEPs out for parts of it now - Primitive Objects (which is the new name for value types) https://openjdk.java.net/jeps/401 and Unify the basic primitives with objects https://openjdk.java.net/
19.
by craigacp 6y ago
XGBoost also has a binary file format which is neither pickle nor json. It's shared across their Python, Java and R frontends and parsed by the C++ library.
20.
by craigacp 6y ago
The reviews for published NeurIPS papers for at least the past 5 years are available from each paper's page on http://papers.neurips.cc .
21.
by craigacp 6y ago
It's weird to me that people build libraries on top of the ML stack to track provenance, when it's really the ML library's job to do that for its inputs. However it is a right pain building it into the ML library as it affect
22.
by craigacp 6y ago
You can with some hoop jumping in the current release (e.g. using 1.14 you need to define the model and optimizer in Python - https://github.com/oracle/tribuo/blob/main/Interop/Tensorflo... ). With t
23.
by craigacp 6y ago
The Tensorflow Java project is alive and well. The next version of the API based on TF 2 is coming out soon. (Full disclosure, I'm a member of the SIG that's building it). I'm a firm believer in the utility of type systems fo
24.
by craigacp 6y ago
Actually my group in Oracle Labs launched a Java ML library last week - https://tribuo.org .
25.
by craigacp 6y ago
We're implementing the extra trees algorithm at the moment, an isolation forest is only a small amount of code from there.
26.
by craigacp 6y ago
We have a strong focus on provenance, Tribuo models capture their input and output domains, along with the necessary configuration to rebuild a model. Tribuo's also more object oriented, nothing returns a bare float or int, you always
27.
by craigacp 6y ago
- Short term roadmap is here - https://github.com/oracle/tribuo/blob/main/docs/Roadmap.md , longer term we'd like to see what the community wants. - Yep. - There are various efforts on the JVM t
28.
by craigacp 6y ago
We're thinking about the visualisation story. At the moment we use notebooks with the IJava kernel to explore things (our tutorials are jupyter notebooks), but we're still working on the plotting angle.
29.
by craigacp 6y ago
The announcement blog talks a little about why we built this library, and where we think it sits in the ML ecosystem - https://blogs.oracle.com/datascience/tribuo-java-machine-lea...
30.
by craigacp 6y ago
I'm the lead developer on Tribuo, happy to answer any questions. The announcement blogpost goes into more detail about why we built it and where it fits into the ML ecosystem - https://blogs.oracle.com/datascience/

More ›