7 ms·

There's a wonderful exchange in the movie Interstellar that speaks to this: Cooper: You're ruling my son out for college now? The kid's fifteen. Princi

by kxyvr 2mo ago

There's a wonderful exchange in the movie Interstellar that speaks to this: Cooper: You're ruling my son out for college now? The kid's fifteen. Principal: Tom's score simply isn't high enough. Cooper: What's your waistline? 32? With, what, a 33 inseam? Principal: I'm not sure I see what you're getting at. Cooper: You're telling me it takes two numbers to measure your own ass but only one to measure my son's future? The point being is that a person, and their future, should not be distilled into a single number like an SAT score because people are far more complex than a single number. I would also contend that splitting them into a few numbers, such as by subject area, doesn't help either.

triceratops 2mo ago
The SAT has 2 components with separate scores. The SAT score is usually the sum of both, but there's no reason colleges can't weight one section more than the other. If you really care about multiple dimensions, add more types of standardized tests - general science (or specialized - separate physics, chem and bio sections), history, geography, whatever. Or tell students to take the standardized tests that most closely map to their preferred major.
- kxyvr 2mo ago
  I still contend strongly that it is wrong to distill a person into a collection of numbers. They are not. My mother was a special ed teacher who eventually became a diagnostician whose job it was to determine eligibility for services. Part of this job was to give a standardized test to a child to see if they met state set criteria. That said, this was only part of an assessment and was used along with parent preference, teacher recommendations, and other people involved with the process at the school. Outside of meeting state criteria, the numbers in the test were used to identify areas of strength and weakness to help understand a child and put this in perspective of a broader story. For example, if a child struggled with math, was it because they couldn't read well, struggled with the math itself, was distractable? There were also other instances where the test scores themselves were misleading. For example, did a child perform poorly because they were sick, or the AC of the building broke, or they set a test date on a school holiday when the child knew the other kids had a day off? My point in telling this story is that the raw score in a standarized test can be helpful, but it does not tell a complete story. It is part of a broader assessment. The SAT and college admission is not different. Purely using a test score does provide an objective score to rank a collection of applicants, but I still contend strongly that it misses the point. Depending on your point of view, college is meant to educate a populace, provide job training, give economic opportunity, or a whole host of other goals. One can use test scores a proxy to determine which applicants best approximate these goals, but I think we should consider whether they really help achieve this. I'll also mention that using tests scores is not entirely "fair" either. I'd a friend who dated a guy who worked for Kaplan, one of the test prep companies. They had very good data about how much their test prep improved test scores. They were also very expensive. As a result, money and wealth can be used pretty directly to improve one's score. It's difficult for a university to fully address wealth inequality in society. However, since tests can be gamed so easily, I would caution against their over use.
  - triceratops 2mo ago
    > I still contend strongly that it is wrong to distill a person into a collection of numbers No one's saying a person is a collection of numbers. > My point in telling this story is that the raw score in a standarized test can be helpful, but it does not tell a complete story. It is part of a broader assessment And that's very noble and well-meaning. But it can also be used to discriminate and deny opportunity. Raw scores, on the other hand, are hard data. You either have them or you don't. Do factors outside of one's control go into them? Absolutely. Those factors are all-pervasive and will influence anything you might use to determine merit. > Kaplan, one of the test prep companies. They had very good data about how much their test prep improved test scores. Of course they would say that. Even assuming it's true, Kaplan test prep books are only around $50 on Amazon. You could also get them for free from a public or school library (or the high seas, if that's your thing). As I understand it, Kaplan has apps and dashboards and more material if you're a subscriber. Is there data about the marginal score improvement using that versus just the books? When you get right down to it, everyone's starting point is different. Reducing the number of variables improves opportunity IMO.
    - kxyvr 2mo ago
      My continued assertion is that "hard data" can be misleading, wrong, and discriminatory. It requires judgement to understand what it tells us and whether there is error in the collection of the data itself. As far as Kaplan, I don't have metrics to share, but what I saw shocked me at the time. It appeared as though they collected data as to how much their test prep improved scores and it was pretty dramatic. And, they were smart enough to distinguish between their test prep books and their in person tutoring. In person tutoring worked better and by a lot, but it all helped. As far as tests, I've never contended that they shouldn't be done, but that these scores need to be part of an overall portfolio. I have in fact sat on admittance boards for graduate school and the two dominant factors other reviewers used were GPA and where they went to undergrad. Both of these were hard metrics and full of bias. I'll also contend that they weren't particularly effective in identifying students who successfully completed the program. Something to consider, I do think test scores can help screen candidates. For example, if one is going to admit into a math program, then their quantitative score should probably be above a certain level. After that, I personally don't think it matters. In that way, one could just set minimum criteria to be accepted, which includes GPA, tests scores, research, jobs, whatever, and then randomly select from that group. That would certainly eliminate a huge amount bias, though not all, but most people hate this approach because it contradicts this belief that we live mostly in a meritocracy. They want rankings and they want to know who number 1 is. My continued argument is that it's impossible to distill a person down to a hard ranking in a reasonable matter, especially with test scores, and it's not necessary for college admission since one can select a group of applicants for acceptance without ordering them.

by kxyvr 2mo ago

triceratops 2mo ago

The SAT has 2 components with separate scores. The SAT score is usually the sum of both, but there's no reason colleges can't weight one section more than the other. If you really care about multiple dimensions, add more types of standardized tests - general science (or specialized - separate physics, chem and bio sections), history, geography, whatever. Or tell students to take the standardized tests that most closely map to their preferred major.

kxyvr 2mo ago
I still contend strongly that it is wrong to distill a person into a collection of numbers. They are not. My mother was a special ed teacher who eventually became a diagnostician whose job it was to determine eligibility for services. Part of this job was to give a standardized test to a child to see if they met state set criteria. That said, this was only part of an assessment and was used along with parent preference, teacher recommendations, and other people involved with the process at the school. Outside of meeting state criteria, the numbers in the test were used to identify areas of strength and weakness to help understand a child and put this in perspective of a broader story. For example, if a child struggled with math, was it because they couldn't read well, struggled with the math itself, was distractable? There were also other instances where the test scores themselves were misleading. For example, did a child perform poorly because they were sick, or the AC of the building broke, or they set a test date on a school holiday when the child knew the other kids had a day off? My point in telling this story is that the raw score in a standarized test can be helpful, but it does not tell a complete story. It is part of a broader assessment. The SAT and college admission is not different. Purely using a test score does provide an objective score to rank a collection of applicants, but I still contend strongly that it misses the point. Depending on your point of view, college is meant to educate a populace, provide job training, give economic opportunity, or a whole host of other goals. One can use test scores a proxy to determine which applicants best approximate these goals, but I think we should consider whether they really help achieve this. I'll also mention that using tests scores is not entirely "fair" either. I'd a friend who dated a guy who worked for Kaplan, one of the test prep companies. They had very good data about how much their test prep improved test scores. They were also very expensive. As a result, money and wealth can be used pretty directly to improve one's score. It's difficult for a university to fully address wealth inequality in society. However, since tests can be gamed so easily, I would caution against their over use.
- triceratops 2mo ago
  > I still contend strongly that it is wrong to distill a person into a collection of numbers No one's saying a person is a collection of numbers. > My point in telling this story is that the raw score in a standarized test can be helpful, but it does not tell a complete story. It is part of a broader assessment And that's very noble and well-meaning. But it can also be used to discriminate and deny opportunity. Raw scores, on the other hand, are hard data. You either have them or you don't. Do factors outside of one's control go into them? Absolutely. Those factors are all-pervasive and will influence anything you might use to determine merit. > Kaplan, one of the test prep companies. They had very good data about how much their test prep improved test scores. Of course they would say that. Even assuming it's true, Kaplan test prep books are only around $50 on Amazon. You could also get them for free from a public or school library (or the high seas, if that's your thing). As I understand it, Kaplan has apps and dashboards and more material if you're a subscriber. Is there data about the marginal score improvement using that versus just the books? When you get right down to it, everyone's starting point is different. Reducing the number of variables improves opportunity IMO.
  - kxyvr 2mo ago
    My continued assertion is that "hard data" can be misleading, wrong, and discriminatory. It requires judgement to understand what it tells us and whether there is error in the collection of the data itself. As far as Kaplan, I don't have metrics to share, but what I saw shocked me at the time. It appeared as though they collected data as to how much their test prep improved scores and it was pretty dramatic. And, they were smart enough to distinguish between their test prep books and their in person tutoring. In person tutoring worked better and by a lot, but it all helped. As far as tests, I've never contended that they shouldn't be done, but that these scores need to be part of an overall portfolio. I have in fact sat on admittance boards for graduate school and the two dominant factors other reviewers used were GPA and where they went to undergrad. Both of these were hard metrics and full of bias. I'll also contend that they weren't particularly effective in identifying students who successfully completed the program. Something to consider, I do think test scores can help screen candidates. For example, if one is going to admit into a math program, then their quantitative score should probably be above a certain level. After that, I personally don't think it matters. In that way, one could just set minimum criteria to be accepted, which includes GPA, tests scores, research, jobs, whatever, and then randomly select from that group. That would certainly eliminate a huge amount bias, though not all, but most people hate this approach because it contradicts this belief that we live mostly in a meritocracy. They want rankings and they want to know who number 1 is. My continued argument is that it's impossible to distill a person down to a hard ranking in a reasonable matter, especially with test scores, and it's not necessary for college admission since one can select a group of applicants for acceptance without ordering them.