Chi Dang
Chi Dang, vice dean for research, on how a research institution’s performance should be evaluated
In 2010, engineering faculty at the University of Western Australia unveiled a new website, High Impact Universities, which calculates a university’s research impact based on numbers of publications and citations. Universities with more prolific and more widely cited faculty rank more highly.
According to this metric, The Johns Hopkins University comes in number two (after Harvard) in the faculty rankings for medicine and other health fields, and number nine overall out of the 500 universities.
Chi Dang, vice dean for research in Hopkins’ school of medicine, who in 2008 also led an internal assessment of Hopkins’s research impact, reflects on what these kinds of rankings mean.
Do you think people should pay much attention to university rankings such as those featured in the new High Impact Universities website?
DANG: Clearly, for public affairs, they’re good. They are going to draw good students and postdocs to an institution that scores highly. Therefore, regardless of the particular algorithm used, a high ranking is something we should strive for.
What do you think about the algorithm used to determine the list of High Impact Universities?
DANG: It makes sense, based on network theory. Networks involve highly connected “hubs” and less well connected elements. For example, in a social network, there are people with many friends, relatives or professional contacts, and there are less well connected individuals.
Similarly, there are hubs of citations. If you publish a paper that doesn’t make an impact, it won't be cited much and will receive little notice. If you publish a paper that generates a different view of a field, then other papers will cite it, and it may become a seminal finding. Over time, the high-impact paper will continue to accumulate citations. Just as the popular person makes more and more contacts over the years, these papers become hubs and are recognized as landmarks in science—papers that changed the field.
What’s an example?
DANG: Gregg Semenza, a scientist with the Hopkins Institute for Cell Engineering, published a paper about a protein that allows cells to adapt to low-oxygen environments. Before Gregg’s paper, we didn’t know how cells could survive in such extreme conditions. Now we know that it’s due to protein HIF-1. Gregg cloned this factor, and now people have the molecule and can use it to understand conditions ranging from cardiovascular disease to high-altitude sickness. A field can blossom from these landmark papers. So I’m a strong believer that the number of citations is a meaningful indicator of research strength.
How are such parameters used in making promotion decisions?
DANG: It is one of several factors taken into account. We use a metric called the h-index. A person with an h-index of 21 has at least 21 papers that have been cited at least 21 times each by other scientific articles. I have about 180 primary publications, but my h-index is 56 because 56 of those publications have been cited 56 times or more. People promoted to tenure at Hopkins [generally] have an h-index of at least 21.
But is that system fair? What if a scientist is doing excellent research in an obscure area of science? Few colleagues might read his work, never mind cite it.
DANG: I’d ask that person, Why are you working in an obscure area of science? If your research is really going to mean something, why remain in that area? On the other hand, an obscure area of research, if fundable, could lead to seeing the world differently tomorrow than we see it today.
The real question about impact isn’t so much whether a disease is rare or a molecule is obscure; it is whether the questions one asks result in a new perspective. The water channel was a truly obscure molecule when Hopkins biologist Peter Agre began studying it. But his research helped explain the mystery of how water goes back and forth through cell membranes, and it led to him winning the Nobel Prize.
When you led a Hopkins committee that evaluated research impact, what indicators did you use?
DANG: We looked at a large variety of indices in addition to publications and citations, such as peer recognition—faculty awards and membership in honorific societies such as the National Academy of Sciences and the Institute of Medicine.
What are some things you learned?
DANG: Overall, we performed pretty well. When it comes to citations and papers published, Hopkins is among the top five to eight U.S. medical schools, and we’re especially strong in neuroscience, oncology and infectious disease. We also lead in amount of NIH funding. So we should be proud. But we could improve in some areas. For example, in terms of peer recognition, it’s clear that some other institutions are very proactive in recruiting people to nominate faculty for awards and honorific societies. They invest a lot of effort in seeking nominations. So last year we started to take a more active stance in seeking nominations.
We also found that some questions about impact remain difficult to answer.
Like what?
DANG: An impact for social good is hard to measure; there isn’t a simple way to assess this. I thought that it would be good to find a way to measure a tangible clinical benefit of our research. It would be interesting to take the last five major drugs approved by the Food and Drug Administration and trace back to see which universities performed research leading to their development. We don’t have the resources to do such a study. So we go back to citations; it’s easier.
—Interviewed by Melissa Hendricks
Related Articles:
Facebook
Twitter
Sharecare
Blogs
YouTube
RSS
Podcasts
