tuned

8 ms·

1.
New Frontier of Sound: Generative Learned Transformers
1 points by tuned 2mo ago|0 comments
2.
Syntethic Intelligence: from IT to AI, a semantic revolution(amazon.com)
2 points by tuned 2mo ago|0 comments
3.
by tuned 2mo ago
trying to generate images and sounds from eigenvectors as harmonic basis
4.
by tuned 6mo ago
Measuring structural information. Check out the paper, notebooks and Python library
5.
Graph Wiring: speed, accuracy, RAG-focused(tuned.org.uk)
2 points by tuned 7mo ago|0 comments
6.
by tuned 8mo ago
ok, thanks. I am taking it slow then
7.
by tuned 8mo ago
no, from my point of view is being more domain-focused instead of going full-orthogonal.
8.
by tuned 8mo ago
right. this is a proposal that needs to be tested. I started testing it on 30M parameters then I will move to a 100M and evaluate the generation on domain-specific assisting tasks
9.
by tuned 8mo ago
> This is obviously not powerful enough to express non-linear relationships - like graph relationships. the distance metrics used is based on energy-informed graphs that encode energy relations in a distribution called taumode, see my pr
10.
by tuned 8mo ago
also: precomputing a sparse Laplacian for N vectors at dimension D (NxD) is infinitely cheaper (if using `arrowspace`, my previous paper) than computing distances on the same full dense vectors billions of times. There are published tests t
11.
by tuned 8mo ago
if you have a corpus of code snippets to train the manifold (Laplacian) on (and a good embedding model), it is definitely possible to try something like this.
12.
by tuned 8mo ago
it made sense to me as it is a very simple idea I guess: causal self-attention compute QKV distances computing on the full vectors for Q,K and V; the topological transformer can provide the same computation using Q, scalar K and V. Instead
13.
by tuned 8mo ago
it most-likely will in terms of performance as it uses 50% less memory (for sure it will at inference time that is the most used operation on web services), because it can leverage longer T and D if the design is confirmed and the quality o
14.
by tuned 8mo ago
Thanks to all that have read. I would be glad to answer further scoped questions on the content of the post and the paper. I answered some comments that may clarify the ideas from the redesign.
15.
by tuned 8mo ago
the idea is to have a lot of "narrow" models to work with RAG instead of one model for all the knowledge domains or also distil the metadata that is currently in enterprise Knowledge Graphs
16.
by tuned 8mo ago
exactly, that is the current objective. To proove that generation for a specific domain is on-par with causal attention models
17.
by tuned 8mo ago
comparisons will be run when the quality of generation will be on pair with other available models. It is useless to have preformance if the quality is not at lease on par. The paper runs a bench (code and bench in the paper) to compare the
18.
by tuned 8mo ago
This is a novel re-interpretation of the Transformer, based on my previous research made with a library called `arrowspace`. It is somehow what is called a "Grassmann-like flow" but without the Plucker embedding, or also similar t
19.
by tuned 8mo ago
thanks for linking. Yes the paper compares the new architecture (that is also a fork of my implementation of nanoGPT) with Karpathy's nanoGPT. There are also links to the code and bench used.
20.
by tuned 8mo ago
thanks for reading. I cannot retrain an existing model as the self-attention mechanism has been completely redesigned. The Keys and Values in self-attention are stored as scalars, so a latent space with traditional weights does not make sen
21.
Starting from scratch: Training a 30M Topological Transformer(tuned.org.uk)
144 points by tuned 8mo ago|58 comments
22.
Show HN: A concept implementation for a faster Transformer [pdf](github.com)
2 points by tuned 9mo ago|0 comments
23.
Metamaterials-Inspired Software Engineering(economyoftime.net)
1 points by tuned 10mo ago|0 comments
24.
A Rust+Burn Implementation for Nanochat(github.com)
2 points by tuned 11mo ago|1 comments
25.
by tuned 11mo ago
Model Architecture (gpt.rs) Multi-layer Transformer: N stacked decoder blocks with pre-norm residual connections Rotary Position Embeddings (RoPE): Replaces learned positional encodings with rotary embeddings for better length generalizatio
26.
Show HN: Spectral Indexing, from concept to paper to alpha in 45 days(tuned.org.uk)
2 points by tuned 11mo ago|0 comments
27.
DeepSeek-OCR Compression Meets Energy Search(tuned.org.uk)
3 points by tuned 11mo ago|1 comments
28.
by tuned 11mo ago
In this post, I demonstrate how DeepSeek's optical compression approach—treating rendered text as a visual medium—has been replicated in Rust using `burn.dev`, and how this compression primitive unlocks a new search paradigm in arrowsp
29.
DeepSeek-OCR compression in readable Rust(crates.io)
3 points by tuned 11mo ago|1 comments
30.
Show HN: Beyond geometric similarity in vector databases(tuned.org.uk)
1 points by tuned 11mo ago|0 comments

More ›