Test Prep Professionals

From (The Chronicle of Higher Education) The poet John Godfrey Saxe famously suggested that if you like law and sausage, you shouldn’t watch them being made. He said that in 1869, when graduate school was in its infancy in the United States. If he had waited, Saxe might have added graduate admissions to that list.

The admissions process is where graduate school starts. Everyone goes through it, and it’s the first point in a prospective student’s career when disciplinary norms and requirements exert their force. But admissions is a black box. The process is conducted in secret, like a papal conclave: Only the results are made public.

Julie R. Posselt, an assistant professor of higher education at the University of Michigan at Ann Arbor, went into that box. She personally observed the workings of graduate admissions committees in six departments at three universities, and emerged to tell the tale in her excellent new book, Inside Graduate Admissions: Merit, Diversity, and Faculty Gatekeeping,just published by Harvard University Press.

Posselt made some disturbing discoveries. We know that graduate admissions is bound to be subjective, but Posselt showed how strikingly personal it is. She found that while faculty may claim to be guided by ethical goals like rewarding merit and representing diversity, their admissions practice instead upholds the status quo. They engage in homophily, or love of same — meaning that they choose people just like themselves.

Posselt and I recently had a conversation via email about her book. I’ve edited it slightly for length and clarity.

How did you come to your topic?

Posselt: Three things drew me to research on graduate admissions. The first was a problem of practice. I worked for four years at the University of Northern Colorado, and one of my responsibilities was advising college seniors as they applied to graduate school. Although my advisees consistently got in, I found the admissions process to be incredibly opaque and unpredictable. Students don’t know what is expected or desired, and the result is a system that’s difficult to navigate, even for those with inside knowledge.

I was also drawn to the topic because very little research has been published about graduate admissions, especially from an institutional or faculty point of view. Much has been written about undergraduate admissions, but the graduate side was unexplored ground.

Finally, I wanted to understand an apparent contradiction: We say in higher education that we’re committed to diversity, and we know about gender and racial disparities in doctoral enrollment and the professoriate, but nationally, two of the strongest predictors of admission to graduate programs are very high GRE scores and attending a highly selective college or university. When we rely heavily on those criteria — as many departments do — we undermine the goals of equity and diversity.

Could you describe your overall findings concerning the GRE?

Posselt: I found pervasive misuse of GRE scores, at least by the standards that the Educational Testing Service advises. It was common practice to apply score or percentile thresholds, and to use scores as a primary criterion — sometimes the primary one — for cutting applicants in the first round of review. Just as important, I wanted to understand why faculty use scores that way. To me, changing these practices starts with understanding them from the professors’ own perspectives.

For example, I learned that admissions committees often recognize the false precision of GRE scores, but they were also drawn toward anything that would speed up the process. Applying GRE cutoffs was just one of several strategies to quantify applicant quality and minimize conflict.Participants also relied on the GRE because they associated scores with qualities that they thought had been central to their own success, such as intelligence and a sense of belonging in elite intellectual communities. They understandably wanted to uphold the rigor of their programs and disciplines, and that started with admitting students whose qualities resemble the professors’ own.

I recall how one committee ended up in a prolonged debate over whether to admit a student with a strong scholastic record and a compelling personal background, but very low GRE scores. By that point in the meeting, it was almost 8 p.m. and dark outside, with the only light in the room coming from their laptops and a projector. I noticed that the tables were littered with empty pizza boxes and red Solo cups. The space felt like a war room. These conditions — paired with a fundamental disagreement about what GRE scores mean and how they should be used — made a difficult decision even harder. Meetings like that brought to life what a human activity admissions evaluation can be.

That said, there were also a few committees who acknowledged that so-called uncomfortable conversations — both about the GRE and other aspects of admissions — might be just what is needed. The way those committees saw it, friendly debate was the best way to correct misperceptions, increase transparency, and hold each other accountable.

Could you describe your methods and fieldwork? How did you choose the universities and departments?

Posselt: I collected the data over two years through extended fieldwork at three well-known research universities. The institutions were natural choices because they had a large number of highly ranked, selective doctoral programs in core academic disciplines. I focused on 10 programs, interviewing members of the admissions committees and other program faculty. Six committees kindly permitted me to observe — but not record — their deliberations, interviews with prospective students, and other events.

I assumed that in programs where most applicants are rejected, the multiple demands of selection would come into sharper relief. Organizational sociologists have also found that we obtain insight into the direction that an entire system is headed by studying powerful organizations within that system. Generally, less powerful organizations tend to mimic the more powerful rather than the other way around.

That’s certainly true of graduate programs! Was it difficult to gain access?

More departments wanted to take part than I expected, but there were a couple that I had to let go because they didn’t have enough willing participants.

I attributed faculty members’ willingness to participate in the study to a combination of factors: a sincere desire to improve admissions and enrollment outcomes in their own departments, a lot of careful work on my part to protect confidentiality, and a healthy dose of divine intervention.

How did the faculty react to having you in the room?

Posselt: No doubt my presence encouraged some self-monitoring, but I don’t think it undercuts the findings. Here’s why: In the couple of departments where I perceived a little reticence at the start of meetings — or nervous laughter when the admissions chair introduced me — I saw it quickly dissipate and the deliberations resemble the character of other admissions discussions I observed. … They aimed for careful consideration, but there were also rude comments, unfounded assumptions, jokes at applicants’ expense, and a few comments of questionable legality.

For example, to appease a difficult colleague, one committee rescinded its decision to reject an applicant whose ability to succeed had been seriously questioned. I saw another committee look hard for reasons to accept one applicant but not another with a very similar academic and personal profile — right down to their upbringings in a rural part of the same faraway state — because they wanted geographic diversity in their cohort. Interestingly, the fine line in that case ended up being a couple of words in one recommendation letter. It said that he was “intellectually zealous,” which they interpreted to mean “competitive.” And because they worried that a competitive personality would be toxic in their small department, they rejected him. I heard everything from hairstyles to hometowns mentioned, and though they might not have explicitly attached value judgments to these details, once on the table, they became part of the fabric of their judgment.

Driving home after one meeting, I thought to myself, “If this is how they acted with me in the room, what must it be like without me there?”

What do you think your most important findings are?

Posselt: First, there is no singular idea of merit that faculty everywhere should use. It is context-specific, and it needs to be context-specific because programs and disciplines vary in their missions and the work that it takes to be successful. What programs can do is be clear with themselves and with prospective students about their mission and priorities.

If merit is context-specific, then it seems inevitable that professors will update their ideas about it to reflect changes in the population and changes in the labor market. What’s less clear is how long that will take. Faculty are notoriously loath to change. Committees spent most of their face time deliberating borderline cases. Where faculty drew that line said as much about how they viewed themselves, the future of their fields, and one another as it did about how they saw students or the careers for which they wanted to prepare students.

I also found several important aversions affecting the outcomes — risk aversion, conflict aversion, and ambiguity aversion, as well as aversion to change. Risk aversion was a common rationale for expecting very high GRE scores and very high grades from elite colleges and universities. Conflict aversion helps explain why faculty don’t more openly debate their ideals of merit.

Finally, I saw how inequalities can become locked in as the “normal” state of affairs when departments run admissions according to apparently innocent understandings or standard operating procedures that have a disparate impact on some groups.

Let me provide a couple of examples of this: When programs oriented themselves toward rankings, it became a powerful filter that could justify preferences that outsiders would view as exclusionary. I also observed a tendency to conflate admission to an educational program with hiring research assistants for specific grants. That’s a real tension for professors who have work that needs to be done, but it’s often used to justify a preference for applicants who need less mentoring, and those candidates are more likely to fall within privileged demographics.

What contributions to practice do you hope your research makes?

Posselt: Admissions is a process with many moving parts. Other research has made clear the importance of attending to implicit bias and affirmative action. I think my work reveals that we also need to revisit inherited assumptions and procedures that affect prospects for greater diversity — the kind that are so routine that we take them for granted. It’s a systemic challenge that requires a systemic response.

I hope my research adds a helpful perspective to the broader national conversation about race and gender consciousness in colleges and universities. We are in the middle of an extremely important historical moment, and doctoral admissions is just one spoke in the wheel of institutional practices that we need to examine — the sooner the better.