Sunday, July 5, 2020
Sleight of hand an illustration of PSAT score inflation
A couple of posts back, I wrote about a recentà Washington Post article in which a tutor named Ned Johnson pointed out that the College Board might be giving students an exaggeratedly rosy picture of their performance on the PSAT by creating two score percentiles: a user percentile based on the group of students who actually took the test; and a national percentile based on how the student would rank if every 11th (or 10th) grader in the United States took the test a percentile almost guaranteed to be higher than the national percentile.à When I read Johnsonsà analysis, Ià assumed that both percentiles would be listed on the score report. But actually, theres an additional layer of distortion not mentioned in the article.à I stumbled on ità quite by accident. Id seen a PDF-form PSAT score report, andà although I only recalled seeing one set of percentiles listed, I assumed that the other set must be on the report somewhere and that Ià simply hadnt noticed them. A few days ago, however, aà longtime reader of this blog was kind enough to offer me access to her sons PSAT so that I could see the actual test. Since ità hasnt been released in booklet form, the easiest way to give me access was simply to let me log in to her sons account (its amazing what strangers trust me with!). When I logged in, I did in fact see the two sets of percentiles, with the national, higher percentile of course listedà first. à But then I noticed the download report button, and something occurred to me.à The earlier PDF report Id seenà absolutely did not present the two sets of percentiles as clearly as the online report didà of that I was positive. So I downloaded a report, and sure enough,à only the national percentiles were listed. The user percentile the rankingà based on the group students who actually took the test wasà completely absent. I looked over every inch of that report, as well as the earlier report Id seen, and I could not find the user percentile anywhere. Unfortunately (well, fortunately for him, unfortunately for me), the student in question had scored extremely well, so the discrepancy between the twoà percentiles wasà barely noticeable. For a student with a score 200à points lower, the gap would be more pronounced. Nevertheless, Im posting the two images here (with permission) to illustrate theà difference in how the percentiles are reported on the different reports. Somehow I didnt think the College Board would be quite so brazen in its attempt to mislead students, but apparently I underestimated how dirty theyre willing to play. Giving two percentiles is one thing, but omitting the lower one entirely from the report format that most people will actually pay attention toà is really a new low.à Ive been hearing tutors comment that theyve never seen so many students obtain reading scoresà in the 99th percentile, whichà apparently extends all the way down to 680/760 for the national percentile, and 700/760 for the user percentile. Wellâ⬠¦thats what happens when a curve is designed to inflate scores. But hey, if it makes students and their parents happy, andà boostsà market share, thats all that counts, right? Shareholders must be appeased.à Incidentally, the college readiness benchmark for 11th grade reading is now set at 390. 390. In contrast, the I confess:à I tried to figure out what thatà à corresponds to on the old test, but looking at the concordance chart gave me such a headache that I gave up. (If anyone wants to explain it me, youre welcome to do so.) At any rate, itsà still shockinglyà low the benchmark on the old test was 550 as well as a whopping 110 points lower than the math benchmark. Theres also an approaching readiness category, which further extends the wiggle room.à A few months back, before any of this had been released, I wrote that the College Board would create a curve to supportà the desired narrative. If the primary goal was to pave the way for a further set of reforms, then scores would fall; if the primary goal was to recapture market share, then scores would rise. I guess its clear now which wayà they decided to go.
Subscribe to:
Posts (Atom)