It may seem strange for a money and business website like Expensivity.com to present this interview about college rankings and the problems they raise. But these academic rankings are big business, and the actual school ranking lists that they produce have significant business and financial implications for the schools listed, especially depending on where they land (or don’t land) on these ranking lists. A recent stark example of the financial implications of college and university rankings is the case of Moshe Porat, former dean of Temple University’s Fox Business School. Porat was convicted on November 29, 2021 of engaging in a fraudulent scheme to move the business school to the top of U.S. News & World Report’s national rankings. “He, along with two of his subordinates,” writes Paul Caron, “had for years knowingly embellished the data they were sending on Fox’s students to the magazine U.S. News & World Report, allowing its online MBA program to achieve its No. 1 ranking for four straight years. The distinction helped Fox more than double its enrollment for the program between 2014 and 2017, raking in millions in tuition payments from students and donor dollars.” College and university rankings are therefore not just of academic interest. They are serious business!
Bruce Gordon: It’s great to be interviewing you, Prof. Stake, and you, Prof. Macosko about your thoughts on the way various websites and magazines rank universities. Can you tell us a little bit about what you each bring to the table when it comes to understanding these rankings?
Jeff Stake: I’m a professor of law at Indiana University, Bloomington, and I have served as the chair of our admissions committee, so I have seen some of the effects of the US News ranking of law schools. Troubled by these effects and the method used by US News, I created a website called “The Ranking Game” to highlight just some of the inadequacies in US News and other ranking sites. I hoped that by playing the Ranking Game, others would appreciate the degree to which rankings are built on arbitrary decisions about what factors to include and how much weight to allocate to each factor.
Jed Macosko: Like Jeff, I’m a professor of physics at Wake Forest University who normally wouldn’t have much expertise in rankings if it wasn’t for the fact that I helped start a school ranking website. Unlike Jeff, I don’t have educational research in my DNA. That perhaps cryptic statement refers to something that Jeff’s brief introduction just now didn’t mention: his father was and is a card-carrying research professor in the field of education. I think that’s pretty interesting and don’t want Jeff to let that fact go unmentioned.
JS: I wondered whether we would get around to talking about that part of my backstory. Now that you brought it up, yes, my father and mother were both involved in education and the evaluation of education. My father, Robert Stake, started out measuring the efficacy of educational programs using statistical methods, including pretests, posttests, and the kinds of standardized tests that we assume to be part of education today. The really interesting thing about his long career in that field was that, as good as he was at the math, he became disillusioned with the simplistic way efficacy was being evaluated. I remember him wearing a t-shirt that said, “See how each datum differs.” The tests and statistics were too simple, too thin a representation relative to all that happens in teaching environments. So, he broke away from many of his colleagues, following a much less travelled path along with a few intrepid colleagues, becoming more of an anthropologist of educational settings. He viewed his responsibilities as extending to all of the stakeholders involved in schools: the teachers, the students, and the parents along with the administrators who had engaged his services.
BG: That’s fascinating! Do you feel like you were following in your father’s footsteps when you became disillusioned by the US News rankings of law schools?
JS: To a large degree, yes. I don’t think my initial foray into rankings was anything like my dad’s radical shift in how he was approaching his career. For me, starting down the rabbit hole of rankings was very much a frolic and detour while I continued to spend most of my time teaching and writing about property law. In my dad’s case, his whole career shifted direction. I think that must have taken a lot of courage (even with the freedom of inquiry that is supported by our tenure system). Still, there is definitely an element of wanting to address some of the same questions that motivated my dad, such as, “What makes one school a better place than another for a given student?” and “To whom are the teachers and administrators responsible?” and things like that.
BG: How about you, Jed, what motivated you to help start the AcademicInfluence.com ranking site?
JM: Like Jeff, I wanted to answer questions about what makes a school a good school. With five kids of my own heading off to college, I was particularly interested in what makes a good university a good university. Consulting the US News rankings is a natural place for any parent or high school junior to begin, and I certainly had those rankings firmly in mind when I started working alongside the people who were writing the algorithms that eventually became AcademicInfluence.com. I remember, in my second year at Wake Forest University, we had a planning session that all faculty were invited to attend. I was in a small group with our provost and said something naïve. I think it was, “If we can just do such and such, then our university will really shoot up in the US News rankings.” The provost gently set me straight and told me that the top-30 spot that Wake Forest held among so-called national universities was something that we could all be grateful for but not something that could improve much by our efforts. At the time, I didn’t really understand what he was saying, but, since he was so much older and wiser than me, I knew that what he was saying must be correct. It was only later, probably after working with AcademicInfluence.com for a year or two, that I understood what he meant.
BG: So, what did he mean? Why couldn’t a school that was ranked #26 on the US News list, like Wake Forest has been over the past decades, be able to “shoot up in the rankings” as you had put it?
JM: Jeff probably can explain this better than I can since he actually reverse-engineered the ranking algorithm that was used by US News in the early 1990’s. But the gist of it is that, as you get closer to the top of the rankings, the distance between two adjacent schools increases. So, it’s not as hard for a school like Northeastern University to improve 113 spots, from #162 to #49, which it did from 1996 to 2013, as it would be for a school to improve the same 113 spots but starting at, say, #114.
JS: That’s right. Starting with the US News ranking this year, the difference between schools ranked 13 and 12 is twice the difference between schools ranked 42 and 40. It would be easier for a law school to change rank from 86 to 60 than for a law school to change rank from 8 to 3. Practically speaking, that means that once you get out of the top schools, rank is not stable and depends a lot on arbitrary choices for the weights to be placed on various criteria, faculty-student ratio, student LSAT median, student undergraduate GPA median, etc. It also means that there is always a market for another ranking because schools will have “moved”, even if nothing of pedagogical importance has changed.
BG: That makes sense. There are a lot more equivalent schools in the middle tier. But at the top of the highest tier, there’s a pretty clear pecking order. At least that’s how it seems to work at the undergraduate level. Is that what you found for law schools, too?
JS: Yes, if the criteria and weights stay the same, the top does not change much. In a more recent investigation, Brian Broughman and I found that students were willing to pay a premium to attend a school ranked only a single place higher in the US News rankings, with the premium being larger (sometimes surprisingly large) between schools at the top of the ranking. In other words, school X might offer you scholarships that would drastically lower your tuition, but you would decline that offer in favor of school Y, even though school Y gave you no scholarship and required you to pay the full tuition.
BG: The research Jeff did on the premiums law students are willing to pay to climb just one spot on the US News list was done over 15 years ago. Jed, do you think his results hold true for undergraduate schools today?
JM: I don’t know much about how it is for law schools, but when students and their parents choose an undergraduate institution, I can tell you that the situation is getting more and more like Jeff described. The reason for it, in my opinion, has to do with the signals that students and parents are getting from a wide range of sources, not the least influential of which is the rankings themselves.
BG: So, you are saying that the rankings act as a feedback loop—that they reinforce and strengthen the very pecking order that they publish?
JS: Definitely! I called that feedback loop an “echo effect” in one of my publications. The US News rankings are published. People read them and internalize the order of the schools for the next several months. Then a survey shows up in the inbox of a few faculty members and administrators, and lawyers and judges.
JM: Just to interject, for undergraduate schools, these people are the presidents, provosts, and admission deans at all the schools in the country—three people at each school.
JS: Thanks for that clarification, Jed. So, those people tell US News their impressions of the quality of other schools. Because the yearly ranking is important to many of them, it cannot help but have some impact on the thoughts going through their heads as they reply to that survey, especially when they do not have a lot of other information about schools they are evaluating. An economist colleague, Michael Alexeev, and were able to show that this echo effect is real. The views of law school personnel, lawyers, and students applying to law schools are all influenced by the US News ranking. And it’s a real problem for our system of higher education.
BG: Can one of you explain why this is such a big problem? I agree that it doesn’t seem right for the “rich to get richer,” as it were. But how exactly does this hurt the way we teach students in the United States?
JM: I can take a stab at explaining this, though I’m sure Jeff will have more to add. The bottom line is that our students are best served when the schools they attend do their absolute best to lower costs and add value. You’re the president of Expensivity.com, Bruce. You know how this kind of thing works. It’s like anything else in a free market economy. The consumers hope that they will get the best value for the lowest cost. Usually, free market forces will dictate that this kind of thing happens, more or less. But when the “value” of one’s education is largely determined by where it ranks on a list, and when that list is self-reinforcing with the “echo” that Jeff just described, then it short-circuits all the helpful free market forces which would normally push schools to provide a greater value for a smaller price.
JS: I would like to concur with what Jed said, and then to express some uneasiness with his perspective. I don’t like describing students as consumers or even (more charitably) as investors. Consumers of products like cars seem to feel quite free to make choices inconsistent with the various car rankings. Car buyers understand the relevance of most of the criteria to their own situations. They happily re-rank the options. No one would buy a minivan because it is rated as an extremely sensible vehicle when they really need a car only for one person to commute to work. Purchasers of education are also hoping to trade their money for something that will make them happier in the future. The consumers of education, however, don’t know enough about what they will need in the future. Think about how many students will happily take off as many days as teachers will give them. Normal consumers don’t say “I paid for a dozen but six would be better.” Student behavior indicates that they are buying a degree, a credential, not an education. Those of us who think that the education itself actually matters question whether these investors fully understand what they are doing.
Second, in many schools, the students do not pay the full cost of their education. That fact alone leaves teachers with a duty to other stakeholders. What students want and what our society needs them to want often differ. So, we as an educational community have an obligation to guide students into the learning that will most benefit their future and the future of our whole society. To return to the car-buying example, people are free to buy the color of car they prefer. It doesn’t affect society much one way or the other if they buy a red one or a blue one. However, if they buy a pollution-spewing, gas-guzzler, costs fall on everyone. So, we have created restrictions on what kinds of fuel inefficiencies and emissions we will tolerate as a society. Likewise, the ABA sets requirements for the course hours to be completed before awarding a Juris Doctor degree, and, in many states, applicants are required to have a J.D. degree from an accredited law school before they can take the bar exam and start practicing law.
BG: That all makes sense and seems intuitively important. But how does that connect with whether the US News ranking has an “echo” effect?
JS: Right, that’s a good question. The fact that the US News list, at least for law schools, demonstrably reinforces its own ranking order means that if a school can make a change that increases its rank, that will tend to be reinforced by the echo effect. That increases pressure on schools to spend money in ways that have some prospect of improving the numbers going into the ranking and to avoid spending money on things that don’t move the numbers. For example, schools have increased student scholarships dramatically. At first, that might seem like a good thing, even if it is a zero-sum game as between law schools. But it might not be mere coincidence that schools have increased tuition as well. If so, then some students are paying more and some are paying less, which is not necessarily an improvement even from a collective student point of view. Consider also that, in order to influence the insiders who are surveyed by US News, schools spend more money on what one might think of as essentially advertising. Is that an improvement?
One of the nastiest incentives for law schools has been to reduce the number of 1L [or first-year] students in the class, so that LSAT and undergraduate GPA medians are higher, and replace the tuition from those students with tuition from transfer students. Defenders say that this gives such schools better students, but I’ve seen no data to back that up. And even if those schools are better off, it is not clear that there are gains for students or society. In law schools, we spend many, many hours considering our curriculum and how it fits together. For example, those of us teaching wills and trusts at IU-Bloomington know that all students studied future interests in their first-year property class. But if we had transfer students in our upper-level classes, we could not build upon that knowledge base. If we started from scratch, we would have to cut out other material. To avoid that, we would have to dumb down the future interests discussion to make sure not to leave any transfer student behind. The learning of all students would suffer. Likewise with student relationships, many of which are built during the first year of law school. Exiting students pull threads from the fabric. Replacement students lack some of the connections woven during the first year. I think it is noteworthy that after more than a century of teaching, so many law schools found this new way to “improve” their programs immediately after it became helpful to their US News rank to do so. In short, the ranking has redirected efforts away from the teaching and scholarship that our society needs lawyers to have and toward the sorts of activities that will improve the ranks of the law schools.
BG: Do you think that the top law schools are failing our society?
JS: I worry that many of our institutions are failing. And, too often, I see news reports of lawyers doing things that make me wonder what sort of legal education that person must have gotten. And sometimes I wonder enough to bother looking up the lawyer’s law school. But I would not go so far as to say that the top law schools are failing society. As much as I worry about the insidious incentives created by US News, most of my colleagues remain devoted to their students and their scholarship. Indeed, I see in law schools an increasing concern for the effects of law, law scholarship, and law teaching on society.
BG: How about you, Jed, do you think top schools fail our society?
JM: Before I go out on a limb and share my own opinion, I think it’s worth hearing a few of the thoughts that other people have about higher ed. Obviously, there’s a large fraction of our country, roughly half, that is suspicious of universities and feel like these institutes of learning are bastions of liberal thought. These same Americans feel that the 18-year-olds who were raised in their communities with their values return home after four years of college with radically different values, values that they think are wrong-headed and even somewhat evil. I think we all know some of what this group of Americans is feeling and empathize with them to a certain degree.
But I’d actually like to share the thoughts of someone who is decidedly not in that group of Americans. His name is Peter Thiel and he was born in Frankfurt, Germany, raised partly in South Africa, and then settled in the San Francisco Bay Area when he was 10. He attended Stanford University, has a husband, made billions off of PayPal and Facebook, and now pays people NOT to go to college. He describes college as four things: an investment, a party, insurance, and a tournament. Most people know what the first two things would mean—you pay for college because you will get back your money, plus more, thanks to higher salaries, and you pay for college because you’re excited to spend four years partying and having fun.
But not as many people would understand the next two things. By calling college “insurance”, Thiel means that you are paying for a way to mitigate the terrible shame of being considered a failure by your peers. Thiel’s reasoning is that you pay for college because if you don’t go to college and don’t become a successful entrepreneur, movie star, or what-have-you, then all of your friends will look on you with pity. But if you go to college, no matter how boring or low-paying your career ends up being, the stigma will be significantly less painful to bear. By calling college “a tournament”, Thiel means that it’s your one opportunity to prove to the world that you are better than the kid who grew up next-door to you. If you get into a better school than she did, then you are that much better in the eyes of the world.
BG: Interesting connections! You’re finding a strange commonality between Peter Thiel, who doesn’t want kids (especially bright ones) going to college because he sees it as a waste, and Harvard sociologist Charles Murray’s so-called residents of “Fishtown,” who don’t want their kids to get brainwashed in college, if they go to college at all. What about you? On balance, do you think college is a benefit for students and society?
JM: Sorry for the digression! Jeff answered you question directly, so let me do the same: I feel that colleges are on the balance a very good thing for our society. I mentioned two different groups who don’t want people to go to college:
- On the one hand, Peter Thiel and the four people at the Thiel Foundation who distribute $2 to $2.5 million each year to 20 to 25 people under 23-years-old who promise to drop out of college, or not attend, for the next two years, to work on some entrepreneurial project.
- What you call Charles Murray’s residents of the fictitious “Fishtown,” consisting of poorer people who tend to be undereducated and place no great value on education, and who might therefore encourage peoplenot to attend college or to drop out of college if they’re there.
The reason I mentioned those two groups with extremely different outlooks on life is because even though I think colleges have been overall great for our society, people with wildly different backgrounds think that not going to college is the best option for many students.
BG: I could see that someone who gets $100,000 from Peter Thiel and his foundation directors would feel pretty good about not going to college. But what about the children of parents who are afraid they will be brain-washed by liberals? Aren’t they missing out on a great opportunity?
JS: I think I can address that question. I noticed that on Jed’s website there is an article called “Why Are Colleges So Liberal?” and, as a liberal myself, I read it to make sure that Jed and the people running the AcademicInfluence.com website weren’t trying to shove right-leaning ideas down unsuspecting students’ throats. The article makes some good points.
JM: I’m glad it wasn’t the conservative propaganda you feared!
JS: No, the article linked to a study that found that the idea about colleges brainwashing young people into liberal ideology is a myth. While faculty members are more liberal than the American population, indoctrination is contrary to a bedrock principle of universities: the questioning and examination of theory and evidence in the search for truth. A basic goal of education is to get students to think critically, not to accept doctrine on faith. The experience of college, including the critical inquiry into truth, often results in change. Nearly half of the students reported changing their political beliefs, 30 percent becoming more liberal and 17 percent becoming more conservative. Some of those changes could be due to feeling pressure from teachers. The students identified as being very conservative reported more often that they felt pressure from teachers. Why might that be? Consider ideas more often believed by persons who are very conservative: Life on earth started 6,000 years ago, with dinosaurs and humans living together at the same time. Humans did not evolve by natural selection. Humans do not contribute to global climate change. Trump won the election. Teachers teach facts and when they do the students with beliefs such as those above will feel pressure to be “more liberal”. On the other side, to take a personal example, I felt pressure in law school to be more conservative. I found out there that there were sensible reasons for many of the rules in our system of property law. Feeling a bit pressured to question one’s views is an important part of education.
I’d like to add some speculation as to why college teachers are more liberal than the general population (although the comparison should be to the population of college graduates, or perhaps even those with advanced degrees). Within those with college degrees, there is surely some self-selection. Many economic conservatives believe in the market and oppose government intervention in it. Part of this belief is that the market allocates resources efficiently. If one believes deeply in that proposition, what does that say about human resources? It says that people will be offered more money to do jobs in which they create more benefit for society. Why would people who believe in the efficient market allocation of resources take jobs teaching when other jobs would pay them more handsomely for their brains and efforts? Would that person not want to do the most for society by taking the job with higher pay? I heard a conservative on TV say that the government does not produce anything. Why would a person who believes such a thing become a teacher in a public university, only to spend a lifetime producing nothing? It does not surprise me that teachers in universities are more liberal than the population. They, like conservatives, go where they think they will do the world more good.
BG: I suppose it could be that to a liberal professor, both moderates and conservatives are equally in need of change.
JS: The changes I like to see in all of my students are the appreciation of facts, the recognition that there are values embedded in many of our legal rules, and the development of the ability to think and express oneself logically and with precision. I also want my students to love thinking about the law.
BG: So, you’re saying that the political imbalance is there, but nothing nefarious is afoot, students are not being indoctrinated and, overall, colleges are a net benefit to society.
JS: Right. The number of students who should feel pressure is the same as the number of people who believe things that are not true. As for the benefits of education, there is evidence that it is a good investment for students. In terms of both unemployment and earnings from employment, people who have completed college are better off than people who have had some college, who in turn are better off than those who have had no college. The chances of health and happiness also improve when a person goes from a high school diploma to a bachelor’s degree. So, those getting $100,000 from Peter Thiel will likely improve their lives by using that money to go to college after the break in their schooling. Research on the benefits to society indicates that societies also gain from the education of their citizens. All workers earn more as the percentage of college graduates in the population increases.
BG: That makes sense. Well, we’ve strayed pretty far from our original topic of college rankings to get into the weeds about whether universities are a net benefit to society or not. I’d like to switch to a subject that is more closely related to what people visiting Expensivity.com care about. The two of you are saying that the US News rankings are problematic, and you’ve both shared the ways in which you think those rankings have problems. But what would you say is the business impact of these problematic rankings?
JM: That’s a great question with a lot of possible answers. I’d like to focus on what these rankings do to the business of keeping a university sustainable. Jeff and I have talked about how the academic world differs from the business world. Universities are different than for-profit companies in a lot of ways. But in some sense, both companies and universities are trying to keep their doors open, and, if actually staying afloat is not a big worry, they both try to expand their influence in the global marketplace of ideas. So, the business impact of the US News ranking can be broken down into two aspects. First, it can help a struggling university keep the lights on, or it can be the final nail in the coffin that causes such a school to go belly up. Second, the rankings can amplify a school’s reach in the realm of ideas and influences, or it can essentially mute a school and keep it from having as wide of a reach. The first aspect is interesting from a business impact point of view because it could help business-minded people know which lower-tier schools are going to fail and which ones are going to survive. If the rankings have problems, and if the rankings can push a school over the brink or keep it from tumbling, then which schools make it will depend on the very same problems that Jeff and I have been talking about. To put it another way, struggling schools that are doing everything humanly possible to offer a top-notch educational product to their customers (and I’m purposely using the language of “product” and “customer”) might still fail due to the “echo effect” because the rankings will reflect past successes and not all the more recent work these schools are doing.
JS: That’s a good example of how the problematic “echo effect” could affect the operation or even existence of certain schools. But I thought the second aspect that Jed mentioned is equally important when you are considering the impact of these rankings. When a ranking places schools at or near the top, engines of popular culture will make use of those names to create a scholarly or upscale image. And that will echo back again. The parents and students will buy their merchandise and choose to attend their programs over more affordable options. We have recently seen the lengths to which parents will go, including breaking the law, to get their children admitted to prestigious schools. And the FBI prosecutions and news about the prosecutions might have burnished the public images of those schools even further. As I said earlier, law students are willing to pay significantly more tuition to attend a top school that ranks just one place higher than another top school on the US News list. As people come to believe that one school is a lot better than another, that meme will influence the relative influence that the schools have in the marketplace of ideas. The differences between the schools cannot carry the weight that is being assigned to them by the public.
BG: It’s interesting how US News can be a kingmaker of sorts, as well as being the pruning hook that decides which branches need to be cut off the tree. That’s a lot of power for a ranking to have! It also makes me wonder about AcademicInfluence.com. Jed, are you guys happy with your influence rankings and desirability rankings? If they wielded the power that the US News lists wield, would they be ready for that responsibility? Do you have plans to improve them or add additional types of rankings?
JM: We are constantly tweaking and, hopefully, improving our algorithms. In terms of adding more rankings, we definitely have some ideas we are working on. We have been inspired by Malcolm Gladwell’s research into Glassboro State University and the $100 million donation that Hank Rowan gave them in the ‘90s to start an engineering program. We are also inspired by the work that Mike Rowe is doing to help Americans gain the skills they need to fill all the unfilled jobs that abound in our country. Be looking for future rankings on our site that list schools in order of most to least capable of turning donations into useful programs that give students the skills they need.
BG: Sounds exciting! Thank you, both, for sharing your thoughts here at Expensivity.com!