svcrunch

5 ms·

1.
Pulley is shutting down and will cease all operations and services on Dec 8th
10 points by svcrunch 2d ago|6 comments
2.
A $37 GLM 5.3 red team: the Alloy-modeled auth layer held, but two bugs outside(goodmem.ai)
1 points by svcrunch 8d ago|0 comments
3.
GPT-6 Astra soundly defeats Fable 5.1 on recognizing handwritten corrections(dorrit.pairsys.ai)
3 points by svcrunch 11d ago|1 comments
4.
by svcrunch 11d ago
The [Pelican Benchmark]( https://github.com/simonw/pelican-bicycle ) is in the LLM's training data and probably not a useful indicator of improving LLM capabilities any more. For the past two years, I've run th
5.
by svcrunch 2mo ago
For over a year, I've run the Little Dorrit benchmark. It's a vision task that converts handwritten editor marks on a printed page into structured JSON output. I hadn't updated the benchmark in several months, but there are s
6.
Handwritten-edit benchmark: Fable 5 is #1, Opus 4.8 regresses 55% on miscounting(dorrit.pairsys.ai)
4 points by svcrunch 2mo ago|1 comments
7.
by svcrunch 5mo ago
The grandparent is definitely wrong on (3). Yes, coding is a killer product, I agree with you. On (2), I agree with you for local models. BUT , there are also the open source Chinese models accessible via open-router. Your argument ("
8.
by svcrunch 7mo ago
I generally don't waste time with patents. I think most patents in deep learning can be overturned by prior art. My current approach to IP is trade secrets. If we publish, we are careful to avoid details that would make the techniques
9.
GPT-5.4 Scores 0.62 F1 on Understanding Handwritten Edits in Dickens(dorrit.pairsys.ai)
2 points by svcrunch 7mo ago|0 comments
10.
by svcrunch 8mo ago
Thanks for your interest. The rerankers are external, GoodMem is a unified API layer that calls out to various providers. There's no model running inside the database or the GoodMem server. We support both commercial APIs and self-host
11.
by svcrunch 8mo ago
Hi there, thanks for writing and sharing your experiences. I'm one of the builders of GoodMem ( https://goodmem.ai/ ), which is infra to simplify end-to-end RAG/agentic memory systems like the one you built. It&#x27
12.
by svcrunch 1y ago
Here's a problem that no frontier model does well on (f1 < 0.2), but which I think is relatively easy for most humans: https://dorrit.pairsys.ai/ > This benchmark evaluates the ability of multimodal language mode
13.
by svcrunch 1y ago
This is really cool.
14.
Can GPT-4o Accurately Read Handwritten Proofreading Marks?(dorrit.pairsys.ai)
1 points by svcrunch 1y ago|2 comments
15.
by svcrunch 1y ago
Various frontier LLMs were evaluated on their ability to interpret handwritten proofreading marks in printed literary text, using a small benchmark based on Charles Dickens's "Little Dorrit". Results are modest at best, and s
16.
by svcrunch 3y ago
No. But to your point, note that in 2020 neuroscientists introduced the Tolman-Eichenbaum Machine (TEM) [1], a mathematical model of the hippocampus that bears a striking resemblance to transformer architecture. Artem Kirsanov has a very ni
17.
by svcrunch 3y ago
While in Google Research, I worked with two of the authors of the "Attention is All you Need" paper, including the gentleman who chose that title. As others have pointed out, self-attention was already a known concept in the resea
18.
by svcrunch 3y ago
Take a look at the BEIR benchmark, which has served as one of the main drivers for development of neural IR systems since its introduction in 2020. BM25 presents a challenging cross-domain benchmark, and it wasn't till ~2022 that neura
19.
by svcrunch 3y ago
> but HNSW is the best 99% of the time for both performance and latency, and is implemented in almost every modern major vector store. In my experience, HNSW indexes are very expensive to build, relative to indexes like IVF. They also ha
20.
by svcrunch 3y ago
I think your comment is accurate, but regarding your last point: "However, fine-tuning on relevant, high quality, knowledge-rich question/answer pairs seems dominant, when such examples are available or can be generated." How
21.
by svcrunch 3y ago
While transformer-based AI is very powerful, and its potential uses in the business world nearly limitless, the issue of hallucination is holding back adoption. Here, Simon Hughes of Vectara introduces an open-source model, HEM, that can au
22.
by svcrunch 3y ago
The metrics presented in the blog post are those of our production model. When designing Boomerang, we tried to balance latency and search relevance in a manner that strikes the right balance for most use cases. On the other hand, GTR-XXL i
23.
by svcrunch 3y ago
I believe that retrieval-augmented generation is the right path to generative AI within organizations, at least for the next few years. Trying to directly fine-tune an LLM on your data also runs into issues with enforcing access permissions
24.
by svcrunch 3y ago
I'm curious, is HSNW the only option? Do you support IVF-style indexes? Also, FAISS is nice because it supports a pluggable storage layer. Is this something that's easily supported in USearch? Great work, and thank you for your co
25.
by svcrunch 3y ago
Thank you for this! I know that Cruikshank was the original illustrator of many of Dickens's novels, but I prefer the artwork of James Mahoney. As a point of comparison, the same scene by both artists: 1. "Oliver Rather Astonishes
26.
by svcrunch 3y ago
This guide reviews 13 prominent instruction-following LLMs given constraints like commercial or non-commercial usage, and self-hosted versus API access.
27.
Stop Stopping: Why stop word lists are problematic in search(vectara.com)
1 points by svcrunch 4y ago|1 comments
28.
by svcrunch 4y ago
The rise of neural retrieval systems has diminished the need to define stopword lists.
29.
by svcrunch 4y ago
This is an area that's dear to me, I'm the cofounder of Vectara and have been working with embedding-based semantic search, aka neural search or neural IR, since 2017. To whether Google uses semantic search, the answer is yes, ver
30.
by svcrunch 4y ago
JDBC definitely. I didn't mention it because I didn't think of it as part of the Java EE spec (JDBC preceded Java EE if I'm not mistaken). As far as Jetty handlers, they might be a good alternative to servlets. I admit I&#x27

More ›