When rankings get mixed with commercial concerns

I’ve recently sent in the inputs needed for the Business Week MBA survey and although the results won’t be published for several months, there has already been some controversy.

I won’t go into all the details but there were several questions in the student survey that some felt were inappropriate and offensive. There were questions about how much alcohol was consumed in a typical week and whether the school was a good environment for casual dating. At the very least, most people would question the validity of such questions in an MBA survey. Following objections by some business schools, Business Week has removed those questions. For more information about the controversy. you can read this article on the Poets and Quants website.

I wasn’t one of the school officials who complained to BW and I haven’t asked them for their position on why these questions were included in the first place but I can guess that the questions would have been used as a basis for other articles that Business Week could run. It isn’t the most ideal of situations but we have to remember that the people who conduct these rankings are commercial interests who are able to gain more advertising revenue if they can attract more readers. And nothing attracts MBA candidates like rankings or stories about rankings. My colleague at CJBS, Simon Taylor who is also the Director of the Masters in Finance programme, shared similar thoughts in his blog post.

I share the unease that Poets and Quants has with the fact that survey questions are removed in the middle of a survey and also the more serious charge that the methodology for the employer survey is still evolving and will depend largely on the sample size for the employer contacts.

However, I think it is only fair to the new Business Week team that people should know that these issues did not just crop up with this year’s ranking. I remember that there were issues in the last ranking held two years ago where there was an error in the calculations that was only corrected after the survey results were published. There were also questions then about the amount of time classmates spent sleeping in class (CJBS made the top 5 in that list, which I would like to think is due to how hard we work our students, but I could be wrong on this point). Most troubling of all is that while 45% of the BW survey relies on employer inputs, there were just 300 employer responses to the last survey. This is a very small sample size when one considers that more than 100 schools participate in the BW survey. I also recall that there were changes to the employer survey methodology in that year, brought about possibly because of the small sample, to depress the weightage of some employers who hired from regional universities.

The bottom-line of all this is a point that I have been making for some time. Rankings are important but imperfect. They should be just one data-point that a candidate uses in determining which school to apply to. It is more important to apply some of the analytical skills that MBAs are supposed to have and examine the data with a critical eye, understand what the data is telling you, understand what the data is not telling you, and make your own decisions.

If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.

One Response to When rankings get mixed with commercial concerns