5 ms·

Standardized assessments are an inadequate way to measure outcomes at the higher ed level; the methodology discussed in the article is terminally flawed. I pro

• edtechdev 12y ago
  I'd read up on the Collegiate Learning Assessment (CLA). It addresses the concerns you raised, and it's not aiming to measure some of the things you assert it is measuring. It's not the same thing as the SAT or the GRE. Formative (as opposed to summative) assessments like the CLA are helpful in reforming and improving education. When a high percentage of students are still making the same basic grammatical and writing errors they were as freshmen, that's useful to know. The same thing with mathematical and other skills. It helps convince folks at universities that teaching needs to be reformed, improved to go beyond the standard lecture, homework, and test formula which is ineffective for learning: http://news.sciencemag.org/education/2014/05/lectures-arent-just-boring-theyre-ineffective-too-study-finds http://news.sciencemag.org/education/2014/05/lectures-arent-... See also concept inventories which are tests which are similarly helping measure and assess student conceptual understanding (and misconceptions) in various disciplines (including computer science and mathematics). There are also surveys of student attitudes and engagement and beliefs which are helping improve student learning and retention (like NSSE - the National Survey of Student Engagemen). Then there are non-standard assessments which are helpful for assessing and improving and demonstrating student learning, such as e-portfolios. Learning can be measured. It can be assessed. (As can motivation, beliefs, etc.) And there are some skills we try to help students learn that are indeed major-agnostic. The ability to write and communicate effectively. The ability to critically think about and evaluate information. The ability to work with others. The ability to properly use and apply mathematical and computational tools. Even digital humanities folks are using computational tools to assist their work. Employers are complaining about some college graduates having poor communication skills, having poor programming skills or conceptual understanding of fundamentals (fizz buzz test), etc. There are things that can and should be improved at the college level - at all levels.
  • nmrm 12y ago
    (this is a living comment until the first reply. Sorry.) I have four major points: 1. You can assess even major-agnostic, general skills in content-specific ways. Such assessments are far less likely to have the negative externalities of more generic tests (e.g. first or last semester "Literature" courses which are basically CLA et al. boosters. These most definitely exist). But even still, I think this is a bad idea and too ripe for abuse. 2. Generic "Critical Thinking" is often not all it's cracked up to be, and some experts claim it is not statistically correlated with the sort of critical thinking purported to be mastered in science and mathematics courses. (Oh, CLA really likes hedge words. They mean you're smart... I mean... that "you've likely acquired transferable critical thinking skills" [1].) 3. Higher Ed shouldn't be responsible for filling in gaps in High School and even Elementary education. Also, grammar!? I rest my case! 4. The CLA is computer-graded (no, really, it is [1]). -- 1. A Problem with Generic Critical Thinking Generic "critical thinking" is often nothing more than "the right kind of sophistry". The author of [1] agrees with this sentiment w.r.t the CLA (although, it should be stated, not in general. See the source; what follows are my own opinions, not those of the author of [1]). In addition to pointing out that any answer along the lines of "IDK" is treated as sufficient, he also notes that the model answers provided for the CLA are full of logical fallacies. But the author of [1] really hits the nail on the head in this paragraph: This is exactly the kind of instruction and practice students receive in a dedicated critical-thinking or informal logic course. Critical-thinking skills are not statistically significantly enhanced by content-specific courses like introduction to philosophy or chemistry, or by content-independent courses such as symbolic logic. I argue that the what the CLA measures is not what colleges and universities are or should ever be intended to teach. Instead, it measures a tortured form of "critical thinking" that is somehow not statistically significantly improved even by courses in the hard sciences and mathematics. Not even by courses in formal logic! Only by courses in informal logic. If science and math don't improve "critical thinking", I'm not sure what "critical thinking" is supposed to mean. Except, of course, the right kind of sophistry+. Ergo, it's likely that if performance on the CLA significantly improves, it will be because colleges and universities are teaching to the test in the guise of "Critical Thinking and Informal Logic" (although I'm sure universities will have better names for their mandatory freshman CLA courses). I'm very concerned that the true humanities -- one place where actual critical thinking is taught -- will be gutted in order to cover the costs of such instruction (I already know of one instance where this has happened). 2. Higher Education Should Assume the Basics As for grammar and mechanics, things are even simpler. Colleges and universities are absolutely not responsible for those sorts of basic skills. [1] http://www.assessmentupdate.com/sample-articles/a-fatal-flaw-in-the-collegiate-learning-assessment-test.aspx http://www.assessmentupdate.com/sample-articles/a-fatal-flaw... + My use of the word sophistry is definitely intentional. It's worth noting that the sort of persuasion rooted in a combination of informal logic and careful rhetoric ("always use hedge words" is a modern equivalent) is exactly the sort of thing that the Sophists were known for! Disclaimer: I'm a huge fan of assessment-driven education. I just don't think it's right for institution-level assessment at the higher education level. Not unless it's content-specific.
• mathattack 12y ago
  When I see people argue "You can't measure the learning that's going on" I have to disagree. It's only based on my personal experience, but in undergrad I didn't see a lot of learning going on. My (large public Midwestern) school had CS and math majors that studied and learned. But there were a lot of people in other majors doing just enough to graduate. And professors didn't require much. I don't know if it's 10%, 20% or 30% that learned, but it wasn't 50%. I see this in NYC. The mayor's belief is that we are doing a little too much testing, and closing too many schools. Their reaction to bad schools is "We just might not be measuring the learning." In this case, they seem to be doing an accurate job of measuring the bullying at least.
  • nmrm 12y ago
    Maybe I should have been more specific. Major and content agnostic standardized tests are useless at the college level. This should be fairly obvious; I think you have to do some work to believe it's possible that it isn't true. How can you measure how well a university prepared a student if you don't even know what the student majored in? On thing you can measure some fairly rudimentary stuff they should have had coming in (grammar, basic math, etc.). But this is answering the wrong question; universities shouldn't be responsible for catching students up. It's a disservice to people who actually wanted to take the course in which the review happens. The other thing you can do is measure generic "critical thinking". But I think this sort of test is ripe for abuse. I don't trust college administrators to not abuse it. And besides, generic "critical thinking" isn't. See the link I posted in my other response here.
    • mathattack 12y ago
      I agree that content (17th century European history, for exam) and general thinking (How to think critically) are very hard to separate. I don't know of a good answer, but I think with all the money that gets thrown at schools, it's worth asking how they are improving the students.