LinchZhang

7 ms·

1.
by LinchZhang 11d ago
As I said multiple times in the linked post, everybody knows that CoT monitoring is imperfect. The question is whether the benefits are high enough that you should be happy to drop an imperfect safety mechanism in the hopes that some day, o
2.
by LinchZhang 11d ago
> Not everyone in the AI space approves of these people or their doomerish. "Not everyone in the automobiles space approve of consumer safety advocates" What an understatement!
3.
by LinchZhang 11d ago
It's very hard in practice. Even without optimization against it, it's about as hard as understanding an activation layer today and will probably get harder in the future. There's also nothing stopping the secondary LLM from
4.
by LinchZhang 11d ago
I agree in the long run whitebox interpretability would be better than CoT monitoring but the technology is very much not ready for it today (and it's not clear we'd solve enough of interpretability before the AIs have actually sc
5.
by LinchZhang 11d ago
I agree CoT monitoring is imperfect but it's okay in practice and helps us get defense-in-depth re: model intent. We absolutely do not have a singular safety mechanism in place that's sufficiently good that we can use it in exclus
6.
You're Absolutely Right(lesswrong.com)
5 points by LinchZhang 1mo ago|1 comments
7.
by LinchZhang 1mo ago
"Phoebus keeps taking screenshots of our latest model’s thoughts. It’s getting kind of embarrassing."
8.
by LinchZhang 10mo ago
Thank you for your kind words and empathy! I appreciate it. Writing to the void is hard, and while I care a lot about improving and not being wrong, I also appreciate it when people realize that the writer on the other side of the screen is
9.
by LinchZhang 10mo ago
Thanks, appreciate the anecdote!
10.
by LinchZhang 10mo ago
If you have suggestions on a better title, please let me know! I tried pretty hard to come up with different ones, giving time constraints, and this was the best one I had. I'm really bad at titles and this is an area of active growth
11.
by LinchZhang 10mo ago
Why fixate on a specific word rather than the overall idea? If the idea is clear enough I don't think it's worth fighting over semantics.
12.
by LinchZhang 10mo ago
I agree I used the word substantially more expansively than some other people use it. That's why I defined it in the beginning so people can understand the local scoping of the relevant word! :) (That said "salespeople" are i
13.
Middlemen Are Eating the World (and That's Good, Actually)(inchpin.substack.com)
14 points by LinchZhang 10mo ago|27 comments
14.
by LinchZhang 10mo ago
I think many people have some intuition that work can be separated between “real work“ (farming, say, or building trains) and “middlemen” (e.g. accounting, salespeople, lawyers, bureaucrats, DEI strategists). “Bullshit jobs” by David Graebe
15.
OpenAI subpoena'd various nonprofits to get them to shut up on SB 53(twitter.com)
6 points by LinchZhang 11mo ago|1 comments
16.
by LinchZhang 11mo ago
The supposed excuse was Elon Musk's lawsuit, but they interpreted the discovery process very broadly.
17.
Intellectual Jokes That Teach(linch.substack.com)
3 points by LinchZhang 1y ago|1 comments
18.
by LinchZhang 1y ago
I compiled a list of my favorite intellectual jokes, as well as offer a short treatise on why intellectual jokes aren't just "jokes about smart people" In xenosociology class we learned about a planet full of people who belie
19.
by LinchZhang 1y ago
I have heard approximately equal number of comments that Chiang's writings are clearly hard sci-fi and that his stories are clearly science fantasy. I find my own ontology to be useful for myself and others, but I don't begrudge o
20.
by LinchZhang 1y ago
If you like explorations of the deep impacts of parallel universes which have bridges that lets you transport bits (but not matter), you might enjoy https://brainchip.thecomicseries.com/ I like some of Yudkowsky's shor
21.
by LinchZhang 1y ago
For this article, I wrote 4000+ words in the first draft and asked AIs to help suggest places where I repeated myself, different sections or other things to cut, etc, so the final article is tightly focused. I tend naturally to be longwinde
22.
by LinchZhang 1y ago
I'm so glad my review had an effect on people! I hope the book can bring you half as much joy as it brought me <3 <3 <3
23.
by LinchZhang 1y ago
checking logprobs doesn't seem weird to me, it was the priming that was weird.
24.
by LinchZhang 1y ago
It wasn't one of the short stories I reread for the review. I thought he simplified the thermodynamics element to make the story work, but multiple people have corrected me by now. Note the specific wording was "appear to" be
25.
by LinchZhang 1y ago
This seems like a weird way to check if something's AI? a) Like presumably AIs are much more likely to make mistakes of a certain form if there are more such mistakes in the training data (or similar ones) b) to figure out whether some
26.
by LinchZhang 1y ago
It's really cool that you ask 10 people their favorite chiang story, and chances are, you'd get 11 answers. And he didn't even write that many more than 10 stories! Really tells you both how talented he is, and how different
27.
by LinchZhang 1y ago
Well, feel free to send my review to anybody cool living in SF or East Bay, especially people new to the area! Maybe they'd read the review and think they'd vibe well with me :)
28.
by LinchZhang 1y ago
OMG I'm so glad this review might have an impact! Please do check out Story of Your Life and then read the other stories! Without giving too many spoilers away, the short story's plot is simultaneously extremely similar to and
29.
by LinchZhang 1y ago
I appreciate the nitpicks! Re #1 It's been several years since I read up on that area of philosophy. I'll need to reread some stuff to decide whether I think the definition I used is a fine enough simplification for sci-fi readers
30.
by LinchZhang 1y ago
The Merchant and the Alchemist's Gate is SO good. I tear up every time I reread it. If you haven't already done so, check out The Truth of Fact, The Truth of Feeling.

More ›