jncraton

5 ms·

1.
How dangerous is Mythos, Anthropic's new AI model?(economist.com)
3 points by jncraton 5mo ago|0 comments
2.
by jncraton 7mo ago
You are right that the concept of "safe" is nebulous, but the goal here is specifically to be XSS-safe [1]. Elements or properties that could allow scripts to execute are removed. This functionality lives in the user agent and pre
3.
by jncraton 7mo ago
2 and 3 bit is where quality typically starts to really drop off. MXFP4 or another 4-bit quantization is often the sweet spot.
4.
by jncraton 8mo ago
That's great. Here's "me" implementing a JS version of that library in one shot using Github Copilot and a 1 sentence prompt: > Implement when.js as a simple, zero-dependency js library following SPEC.md exactly. htt
5.
by jncraton 9mo ago
I've adjusted or removed those sentences in the article.
6.
by jncraton 2y ago
Pyodide supports numpy and scipy. https://pyodide.org/en/stable/usage/packages-in-pyodide.html
7.
by jncraton 2y ago
The languagemodels[1] package that I maintain might meet your needs. My primary use case is education, as myself and others use this for short student projects[2] related to LLMs, but there's nothing preventing this package from being
8.
by jncraton 2y ago
It would be nice to see the Phind Instant weights released under a permissive license. It looks like it could be a useful tool in the local-only code model toolbox.
9.
by jncraton 2y ago
The speedup would not be that high in practice for folks already using speculative decoding[1]. ANPD is similar but uses a simpler and faster drafting approach. These two enhancements can't be meaningfully stacked. Here's how the
10.
by jncraton 3y ago
You might be interested in "Text Embeddings Reveal (Almost) As Much As Text": > We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important persona
11.
by jncraton 3y ago
Google released the T5 paper about 5 years ago: https://arxiv.org/abs/1910.10683 This included full model weights along with a detailed description of the dataset, training process, and ablations that led them to that
12.
by jncraton 3y ago
> PNG uses deflate. General byte-level patterns. It does not do bespoke image-specific stuff. That's not quite the whole story. PNG does include simple filters to represent a line as a difference from the line above, and that may be
13.
by jncraton 3y ago
There is a large amount of theoretical research on the subject of energy limits in computing. For example, Landauer's principle states any irreversible change in information requires some amount of dissipated heat, and therefore some e
14.
by jncraton 3y ago
You might be interested in OpenWorm: https://openworm.org/ This paper might be helpful for understanding the nervous system in particular: https://royalsocietypublishing.org/doi/10.1098/rstb.2017.0
15.
Details emerge of surprise board coup that ousted CEO Sam Altman at OpenAI(arstechnica.com)
581 points by jncraton 3y ago|722 comments
16.
by jncraton 3y ago
This is great to see. It looks like the size of the embedding vector is half the size of text-embedding-ada-002 (768 vs 1536) while providing competitive performance. This will save space in databases and make lookups somewhat faster. For t
17.
by jncraton 3y ago
You might be interested in TinyStories: https://arxiv.org/abs/2305.07759 > In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually
18.
by jncraton 3y ago
OpenLLaMA models up to 13B parameters have now been trained on 1T tokens: https://github.com/openlm-research/open_llama
19.
by jncraton 3y ago
Thanks for pointing that out. Classification is half-baked at the moment. It should ultimately be restricting output to only appropriate labels, but right now it is simply sampling.
20.
by jncraton 3y ago
You can actually get these models to do this, but you have to ask: >>> lm.do(f"Answer from the context: What is YCombinator? {lm.get_wiki('Python')}") 'The context does not provide information abo
21.
by jncraton 3y ago
That's correct. The current base model is an int8 quantization of LaMini-Flan-T5-248M described here: https://github.com/mbzuai-nlp/lamini-lm I shared more details over on Reddit: https://www.reddit.com
22.
Show HN: Explore large language models with 512MB of RAM(github.com)
138 points by jncraton 3y ago|33 comments
23.
OpenLLaMA 7B Training Completed to 1T Tokens(huggingface.co)
58 points by jncraton 3y ago|3 comments
24.
by jncraton 3y ago
I appreciated Richard's take on AI-assisted programming: > It doesn’t remove the need to communicate precisely. ... In a sense, that’s almost the definition of what makes a programming language a programming language, as opposed to
25.
by jncraton 3y ago
> There’s usually no need to go beyond 16-bit accuracy, and most of the time when you go to 8-bit accuracy there is too much loss of resolution. I'm not sure this is accurate. From what I have seen, 8-bit quantization is usually fin
26.
Development notes from Xkcd’s “Gravity” and “Escape Speed”(chromakode.com)
7 points by jncraton 3y ago|0 comments
27.
by jncraton 3y ago
> could camera manufacturers add a new tag to the EXIF data that is cryptographically signed by the camera They could, and some already do, but this doesn't fully solve the problem. You'd still be left with the analog loophole[
28.
by jncraton 3y ago
Embeddings can be trained specifically to cause questions and content including their answers to have similar representations in latent space. This has been used this to create QA retrieval systems. Here's one commonly used example: h
29.
by jncraton 3y ago
What is wrong with the CodeGen model that they are using? It is a reasonably large model (up to 16B params) that has already been trained on both natural language and code. I would expect it to underperform larger models, including GPT-3.5
30.
by jncraton 4y ago
This can certainly be done. Here's one example from 2021 demonstrating training an LLM to use a scratchpad ("talking to itself") to greatly improve accuracy on arithmetic problems: https://arxiv.org/pdf/2

More ›