5 ms·

> I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude! I didn't realize Clau

by jdnier 7mo ago

> I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude! I didn't realize Claude was named after Claude Shannon! https://en.wikipedia.org/wiki/Claude_Shannon https://en.wikipedia.org/wiki/Claude_Shannon

NitpickLawyer 7mo ago
Wait till you hear about nvidia and their GPU architecture naming scheme :)
bread-wood 7mo ago
Here I was assuming it was named after https://en.wikipedia.org/wiki/Claude_(alligator) https://en.wikipedia.org/wiki/Claude_(alligator)
- deleted 7mo ago
  [deleted]
tzumaoli 7mo ago
Trivia: Claude Shannon proposed the idea of predicting the next token (letter) using statistics/probabilities in the training data corpus in 1950: "Prediction and Entropy of Printed English" https://languagelog.ldc.upenn.edu/myl/Shannon1950.pdf https://languagelog.ldc.upenn.edu/myl/Shannon1950.pdf
- Anon84 7mo ago
  It goes back a bit further than that. His 1948 “Mathematical theory of communication” [1] already has (what we would now call) a Markov chain language model, page 7 onwards. AFAIK, this was based on his classified WWII work so it was probably a few years older than that [1] https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf https://people.math.harvard.edu/~ctm/home/text/others/shanno...
  - aix1 7mo ago
    I was just reading Norbert Wiener's "The Human Use of Human Beings" (1950) and this quote gave me a good chuckle: "One may get a remarkable semblance of a language like English by taking a sequence of words, or pairs of words, or triads of words, according to the statistical frequency with which they occur in the language, and the gibberish thus obtained will have a remarkably persuasive similarity to good English."
- Trinicode 7mo ago
  A letter is not a token, is it? Redundancy could hit 75% in long sentences, but Shannon was not predicting tokens or words, he was predicting letters (characters).
pfdietz 7mo ago
It's like the diesel engine, which is named after Rudolf Engine.
- ai_critic 7mo ago
  :|
- roer 7mo ago
  Is this a joke I don't get? His name was Rudolf Diesel, right?
  - stavros 7mo ago
    Yes, it is a fantastic joke and I laughed for ages, well played.
SenorKimchi 7mo ago
And Claude had a collection of cycles, unicycles. Unfortunately the article is about something else altogether.
teekert 7mo ago
Last time I asked Claude itself also didn’t know.

by jdnier 7mo ago

NitpickLawyer 7mo ago

Wait till you hear about nvidia and their GPU architecture naming scheme :)

bread-wood 7mo ago

Here I was assuming it was named after https://en.wikipedia.org/wiki/Claude_(alligator) https://en.wikipedia.org/wiki/Claude_(alligator)

deleted 7mo ago
[deleted]

tzumaoli 7mo ago

Trivia: Claude Shannon proposed the idea of predicting the next token (letter) using statistics/probabilities in the training data corpus in 1950: "Prediction and Entropy of Printed English" https://languagelog.ldc.upenn.edu/myl/Shannon1950.pdf https://languagelog.ldc.upenn.edu/myl/Shannon1950.pdf

Anon84 7mo ago
It goes back a bit further than that. His 1948 “Mathematical theory of communication” [1] already has (what we would now call) a Markov chain language model, page 7 onwards. AFAIK, this was based on his classified WWII work so it was probably a few years older than that [1] https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf https://people.math.harvard.edu/~ctm/home/text/others/shanno...
- aix1 7mo ago
  I was just reading Norbert Wiener's "The Human Use of Human Beings" (1950) and this quote gave me a good chuckle: "One may get a remarkable semblance of a language like English by taking a sequence of words, or pairs of words, or triads of words, according to the statistical frequency with which they occur in the language, and the gibberish thus obtained will have a remarkably persuasive similarity to good English."
Trinicode 7mo ago
A letter is not a token, is it? Redundancy could hit 75% in long sentences, but Shannon was not predicting tokens or words, he was predicting letters (characters).

pfdietz 7mo ago

It's like the diesel engine, which is named after Rudolf Engine.

ai_critic 7mo ago
:|
roer 7mo ago
Is this a joke I don't get? His name was Rudolf Diesel, right?
- stavros 7mo ago
  Yes, it is a fantastic joke and I laughed for ages, well played.

SenorKimchi 7mo ago

And Claude had a collection of cycles, unicycles. Unfortunately the article is about something else altogether.

teekert 7mo ago

Last time I asked Claude itself also didn’t know.