5 ms·

For sure. Great argument + the experiments may already be in the dataset so it’s really testing if it remembers pop psychology

• a123b456c 2y ago
  Yes. A stronger test would be guessing the results of as-yet-unpublished experiments.
  • jslezak 2y ago
    They did this. Read the paper
    • abcgde 2y ago
      Well, they looked at papers that weren't published as of the original model release. But GPT very likely had unannounced model updates. Is it not possible that many of the post 2021 papers were in the version of GPT they actually worked with?