7 ms·

GPT4 can pass the the neurosurgical medical boards, most of the people laughing at it are typically too dumb to note the difference between 3.5 and 4. >pushing

GPT4 can pass the the neurosurgical medical boards, most of the people laughing at it are typically too dumb to note the difference between 3.5 and 4. >pushing the bottom out of the labor market and actively making the lives of the poorest worse in order to line their own pockets. This makes zero sense. GPT4 has little effect on a janitor or truck driver. It doesn't pick fruit, or wash cars.

pclmulqdq 3y ago
FWIW, as a non-pathologist with a pathologist for a father, I can almost pass the pathology boards when taken as a test in isolation. Most of these tests are very easy for professionals in their fields, and are just a Jacksonian barrier to entry. Being allowed to sit for the test is the hard part, not the test itself. As far as I know, the exception to this is the bar exam, which GPT-4 can also pass, but that exam plays into GPT-4's strengths much more than other professional exams.
- Quarrel 3y ago
  > the exception to this is the bar exam FWIW, this is more true for CA than most states.
- defen 3y ago
  What is a Jacksonian barrier to entry? I can't find the phrase "Jacksonian barrier" anywhere else on the internet except in one journal article that talks about barriers against women's participation in the public sphere in Columbia County NY during Andrew Jackson's presidency.
  - pclmulqdq 3y ago
    I may have gotten the president wrong (I was 95% sure it's named after Jackson until I Googled it), but the word "Jacksonian" was meant to refer to the addition of bureaucracy to a process to make it cost more to do it, and thus discourage people. I guess I should have said "red tape" instead... Either it's a really obscure usage of the word or I got the president wrong.
    - CamperBob2 3y ago
      "It's difficult to attribute the addition of bureaucracy or increased costs to a specific U.S. president, as many presidents have overseen the growth of the federal government and its bureaucracy throughout American history. However, it is worth mentioning that Lyndon B. Johnson's administration, during the 1960s, saw a significant expansion of the federal government and the creation of many new agencies and programs as part of his "Great Society" initiative. This expansion led to increased bureaucracy, which some argue made certain processes more expensive and inefficient. But it's important to note that the intentions of these initiatives were to address issues such as poverty, education, and civil rights, rather than to intentionally make processes more costly or discourage people. Signed, Guess Who"
      - pclmulqdq 3y ago
        Yeah, I asked ChatGPT about it too, and unsurprisingly (and unhelpfully) got answers that pointed to every president.
        CamperBob2 3y ago
        Hmm, which version did you ask? GPT4 went immediately to LBJ, whom I assume you had in mind originally. That is not an instance of passing a standardized academic test through "autocompletion" or "regurgitation." It's rudimentary synthetic thought. If it had named a different president, I could have argued with it, which is what I find especially interesting.
- BlackSwanMan 3y ago
  lmao do you have any idea how much time medical students spend studying the STEP exams(prior to them becoming P/F)
quickthrower2 3y ago
Exams are designed to be challenging to humans because most of us don’t have photographic memories or RAM based memory, so passing the test is a good predictor of knowing your stuff, i.e. deep comprehension. Making GPT sit it is like getting someone with no knowledge but a computer full of past questions and answers and a search button to sit the exam. It has metaphorical written it’s answers on it’s arm.
- entropicdrifter 3y ago
  This is essentially true. I explained it to my friends like this: It knows a lot of stuff, but it can't do much thinking, so the minute your problem and its solution are far enough off the well-trodden path, its logic falls apart. Likewise, it's not especially good at math. It's great at understanding your question and replying with a good plain-english answer, but it's not actually thinking
  - zamnos 3y ago
    That's a disservice to your friends, unless you spend a bunch of time defining thinking first, and even then, it's not clear that it, with what it knows and the computing power it has access to, doesn't "think". It totally does a bunch of problem solving; fails on some, succeeds on others (just like a human that thinks); GPT-4's better than GPT-3. It's quite successful at simple reasoning (eg https://sharegpt.com/c/SCeRkT7 https://sharegpt.com/c/SCeRkT7 and moderately successful at difficult reasoning (eg getting a solution to the puzzle question about the man, the fox, the chicken, and the grain trying to cross the river. GPT-3 fails if you substitute in different animals, but GPT-4 seems to be able to handle that. GPT-4's passed the bar exam, which has a whole section on logic puzzles (sample test questions from '07: https://www.trainertestprep.com/lsat/blog/sample-lsat-logic-games https://www.trainertestprep.com/lsat/blog/sample-lsat-logic-... ). It's able to define new concepts and new words. It's masters have gone to great lengths to prevent it from writing out particular types of judgements (eg https://sharegpt.com/c/uPztFv1 https://sharegpt.com/c/uPztFv1). Hell, it's got a great imagination if you look at all the hallucinations it produces. All of that sum up to many thinking-adjacent things, if not actual thinking! It all really hinges on your definition of thinking.

pclmulqdq 3y ago

FWIW, as a non-pathologist with a pathologist for a father, I can almost pass the pathology boards when taken as a test in isolation. Most of these tests are very easy for professionals in their fields, and are just a Jacksonian barrier to entry. Being allowed to sit for the test is the hard part, not the test itself. As far as I know, the exception to this is the bar exam, which GPT-4 can also pass, but that exam plays into GPT-4's strengths much more than other professional exams.

Quarrel 3y ago
> the exception to this is the bar exam FWIW, this is more true for CA than most states.
defen 3y ago
What is a Jacksonian barrier to entry? I can't find the phrase "Jacksonian barrier" anywhere else on the internet except in one journal article that talks about barriers against women's participation in the public sphere in Columbia County NY during Andrew Jackson's presidency.
- pclmulqdq 3y ago
  I may have gotten the president wrong (I was 95% sure it's named after Jackson until I Googled it), but the word "Jacksonian" was meant to refer to the addition of bureaucracy to a process to make it cost more to do it, and thus discourage people. I guess I should have said "red tape" instead... Either it's a really obscure usage of the word or I got the president wrong.
  - CamperBob2 3y ago
    "It's difficult to attribute the addition of bureaucracy or increased costs to a specific U.S. president, as many presidents have overseen the growth of the federal government and its bureaucracy throughout American history. However, it is worth mentioning that Lyndon B. Johnson's administration, during the 1960s, saw a significant expansion of the federal government and the creation of many new agencies and programs as part of his "Great Society" initiative. This expansion led to increased bureaucracy, which some argue made certain processes more expensive and inefficient. But it's important to note that the intentions of these initiatives were to address issues such as poverty, education, and civil rights, rather than to intentionally make processes more costly or discourage people. Signed, Guess Who"
    - pclmulqdq 3y ago
      Yeah, I asked ChatGPT about it too, and unsurprisingly (and unhelpfully) got answers that pointed to every president.
      - CamperBob2 3y ago
        Hmm, which version did you ask? GPT4 went immediately to LBJ, whom I assume you had in mind originally. That is not an instance of passing a standardized academic test through "autocompletion" or "regurgitation." It's rudimentary synthetic thought. If it had named a different president, I could have argued with it, which is what I find especially interesting.
BlackSwanMan 3y ago
lmao do you have any idea how much time medical students spend studying the STEP exams(prior to them becoming P/F)

quickthrower2 3y ago

Exams are designed to be challenging to humans because most of us don’t have photographic memories or RAM based memory, so passing the test is a good predictor of knowing your stuff, i.e. deep comprehension. Making GPT sit it is like getting someone with no knowledge but a computer full of past questions and answers and a search button to sit the exam. It has metaphorical written it’s answers on it’s arm.

entropicdrifter 3y ago
This is essentially true. I explained it to my friends like this: It knows a lot of stuff, but it can't do much thinking, so the minute your problem and its solution are far enough off the well-trodden path, its logic falls apart. Likewise, it's not especially good at math. It's great at understanding your question and replying with a good plain-english answer, but it's not actually thinking
- zamnos 3y ago
  That's a disservice to your friends, unless you spend a bunch of time defining thinking first, and even then, it's not clear that it, with what it knows and the computing power it has access to, doesn't "think". It totally does a bunch of problem solving; fails on some, succeeds on others (just like a human that thinks); GPT-4's better than GPT-3. It's quite successful at simple reasoning (eg https://sharegpt.com/c/SCeRkT7 https://sharegpt.com/c/SCeRkT7 and moderately successful at difficult reasoning (eg getting a solution to the puzzle question about the man, the fox, the chicken, and the grain trying to cross the river. GPT-3 fails if you substitute in different animals, but GPT-4 seems to be able to handle that. GPT-4's passed the bar exam, which has a whole section on logic puzzles (sample test questions from '07: https://www.trainertestprep.com/lsat/blog/sample-lsat-logic-games https://www.trainertestprep.com/lsat/blog/sample-lsat-logic-... ). It's able to define new concepts and new words. It's masters have gone to great lengths to prevent it from writing out particular types of judgements (eg https://sharegpt.com/c/uPztFv1 https://sharegpt.com/c/uPztFv1). Hell, it's got a great imagination if you look at all the hallucinations it produces. All of that sum up to many thinking-adjacent things, if not actual thinking! It all really hinges on your definition of thinking.