olliepro

9 ms·

1.
by olliepro 3mo ago
The authors have some inconsistencies with training token length… Most errors are probably responses that didn’t finish before their 3K token limit. They’ve measured how well RL is able to shorten the response to their limit.
2.
by olliepro 3mo ago
This is the classic pattern of LLM generated MCQs.
3.
by olliepro 4mo ago
With super high res onboard camera footage too.
4.
by olliepro 5mo ago
They do quite a lot of distillation. As we've seen from the American open weight models from AI2 (OLMo series of models). They have a lot of incentive to distill beyond just copying, they're much more compute constrained, so open
5.
by olliepro 5mo ago
A lot of distillation happens. E.g. OLMo models have a completely open dataset and they are heavily distilled. It only makes sense to try to absorb behaviors from the best models out there. That said, I think the open weight juggernaughts a
6.
by olliepro 5mo ago
decentralized training makes a lot more sense when the required hardware isn't a $40K GPU...
7.
by olliepro 5mo ago
This would likely only get used for small finetuning jobs. It’s too slow for the scale of pretraining.
8.
by olliepro 7mo ago
I bet they lack good long context training data and need to start a flywheel of collecting it via their api (from willing customers)
9.
by olliepro 7mo ago
Tensors are in no shortage nowadays. I did read this a tensors though and got a good laugh.
10.
by olliepro 7mo ago
There’s a section of I-15 in Utah’s Salt Lake County which reliably has a crash on weekdays at 6pm. It was unfortunately at a pinch point in the mountains with no good alternate route… very annoying. In a similar way that Google Maps shows
11.
by olliepro 8mo ago
Much of the scientific medical literature is behind paywalls. They have tapped into that datasource (whereas ChatGPT doesn't have access to that data). I suspect that were the medical journals to make a deal with OpenAI to open up the
12.
by olliepro 8mo ago
It depends on your thing. If the marathon was just the motivation, your thing is running... if the marathon was the bucketlist item, it is the thing.
13.
by olliepro 8mo ago
Getting everyone to fall in love with the thing is not doing the thing... learned this as a data scientist brought in to work on a project which ended soon thereafter. A team of 20 people spent 1.5 years getting people to love an idea which
14.
by olliepro 8mo ago
Everyone's threshold is different. I aspire to "move fast and break things", but more often than not, I obsess over the rough edges.
15.
by olliepro 8mo ago
The more I use AI to do the thing, the more it feels like I didn't do the thing.
16.
by olliepro 8mo ago
What abstraction levels do you expect will remain only in the Human domain? The progression from basic arithmetic, to complex ratios and basic algebra, graphing, geometry, trig, calculus, linear algebra, differential equations… all along th
17.
by olliepro 8mo ago
I made a skill that reflects on past conversations via parallel headless codex sessions. Its great for context building. Repo: https://github.com/olliepro/Codex-Reflect-Skill
18.
by olliepro 8mo ago
I was thinking about something like this, but I don't have codex running on a server. Keep me posted on how it goes!
19.
Show HN: Codex Self-Reflect Skill and CLI to run subagents on past Codex convos(github.com)
3 points by olliepro 8mo ago|2 comments
20.
by olliepro 8mo ago
I believe the idea is that it “files away” the files into folders.
21.
by olliepro 8mo ago
Lol
22.
by olliepro 8mo ago
Can Claude code jump through the hoops for you?
23.
by olliepro 9mo ago
Three things that shook me awake to the idea that the information barrage of the internet is a tranquilizer/red herring: - Bad Mental Health: At the start of the war in Ukraine I read/listened to the news every day. I’d frequently
24.
by olliepro 9mo ago
Although there are many examples of troubling sycophantic responses confirming or encouraging delusions, this document is the original complaint (the initial filing) in a lawsuit against OpenAI. Because it is an initial legal complaint, it
25.
by olliepro 9mo ago
It feels like this should work, but the breadth of knowledge in these models is so vast. Everyone knows how to taste, but not everyone knows physics, biology, math, every language… poetry, etc. Enumerating the breadth of valuable human task
26.
by olliepro 9mo ago
Do you have a better way to measure LLMs? Measurement implies quantitative evaluation... which is the same as benchmarks.
27.
by olliepro 10mo ago
A more sound approach would have been to do a monte carlo simulation where you have 100 portfolios of each model and look at average performance.
28.
Show HN: ND Loss Optimizer Arena(nd-optimizer-arena.vercel.app)
1 points by olliepro 10mo ago|0 comments
29.
by olliepro 11mo ago
Ohio bill in motion to deny AI legal personhood: https://www.legislature.ohio.gov/legislation/136/hb469
30.
by olliepro 11mo ago
Descript has some nice video/screen recording tools to help beginners make this kind of thing look moderately professional. I've used it for various walkthroughs in my previous place of employment.

More ›