Measuring Research Impact

General

• Saying ‘No’ to Rankings and Metrics (Chapter 6 in Socially Responsible Higher Education, Brill, 2021)

• How faculty define quality, prestige, and impact of academic journals (PLOS ONE, October 28, 2021) "This study’s results highlight the subjectivity of common research terms and the importance of implementing evaluation regimes that do not rely on ill-defined concepts and may be context specific."

• The value of data and other nontraditional scholarly outputs in academic review, promotion, and tenure (Forthcoming in Open Handbook of Linguistic Data Management, MIT Press) “We conclude that many current structures for faculty assessment do not explicitly recognize the increasing complexity and demands of faculty work.”

• Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations (PLOS ONE, March 11, 2020) “Respondents indicated that total number of publications, number of publications per year, and journal name recognition were the most valued factors in review, promotion, and tenure.”​​​​​​

• The transformative power of values-enacted scholarship (Humanities and Social Sciences Communications, December 7, 2020) This essay gives the argument behind the HuMetricsHSS philosophy. (See Changemakers tab)

Accounting

• Sangster, A. (2015). You cannot judge a book by its cover: The problems with journal rankings. Accounting Education, 24(3), 175-186.

Biology

• Fire, M., & Guestrin, C. (2019). Over-optimization of academic publishing metrics: Observing Goodhart’s Law in action. Gigascience, 8(6). doi:10.1093/gigascience/giz053
• Ginieis, M., & Li, X. (2020). Ranking of sustainability journals using the author affiliation index and comparison to other journal metrics. Sustainability, 12(3). doi:10.3390/su12031104

Biomedical Sciences

• San Francisco Declaration on Research Assessment (DORA) initiated by the American Society for Cell Biology (ASCB) together with a group of editors and publishers of scholarly journals, . . . A number of themes run through these recommendations: -­‐ the need to eliminate the use of journal-based metrics, such as Journal Impact factors, in funding, appointment, and promotion considerations;-­‐ the need to assess research on its own merits rather than on the basis of the journal in which the research is published; . . .

• Rice, Danielle B, Hana Raffoul, John P A Ioannidis, and David Moher. “Academic Criteria for Promotion and Tenure in Biomedical Sciences Faculties: Cross Sectional Analysis of International Sample of Universities.” BMJ 369 (2020): m2081–m2081.

Communication

• Levine, T. R. (2010). Rankings and trends in citation patterns of communication journals. Communication Education, 59(1), 41-51.

Economics

•  Mixon, Franklin G., and Kamal P. Upadhyaya. “Top to Bottom: An Expanded Ranking of Economics Journals.” Applied economics letters 29, no. 3 (2022): 226–237.
• Ranking Economics Departments in a Contested Discipline: A Bibliometric Approach to Quality Equality Between Theoretically Distinct Subdisciplines Lee, Frederic S., Therese C. Grijalva, and Clifford Nowell. "Ranking Economics Departments in a Contested Discipline: A Bibliometric Approach to Quality Equality Between Theoretically Distinct Subdisciplines." The American Journal of Economics and Sociology 69, no. 5 (2010): 1345-375. http://www.jstor.org/stable/27916901.

Humanities and Social Sciences

• The transformative power of values-enacted scholarship 2020

Linguistic Society of America

• Statement on the Scholarly Merit and Evaluation of Open Scholarship in Linguistics 2020

Management

• Leonhard Dobusch and Maximilian Heimstadt. (2019). Predatory publishing in management research: A call for open peer review. Management Learning 50(5): 607-619.

Marketing

• Steward, M. D., & Lewis, B. R. (2010). A comprehensive analysis of marketing journal rankings. Journal of Marketing Education, 32(1), 75-92.

Mathematics

• American Mathematical Soceity - 2009 Statement Citation and impact in Mathematical Publications The Culture of Research and Scholarship in Mathematics: Citation and Impact in Mathematical Publications For a variety of reasons, mathematicians have not embraced the impact factor as a reliable indicator of a journal's quality . . .

Nursing

• Gagliardi, A. R., Umoquit, M., Webster, F., & Dobrow, M. (2014). Qualitative research publication rates in top-ranked nursing journals: 2002-2011. Nursing Research, 63(3), 221-227.

Political Science

• Ranking Political Science Journals: Reputational and Citational Approaches Giles, Michael WView Profile; Garand, James CView Profile. PS: Political Science & Politics40.4 (Oct 2007): 741-751. Table 1 various impact scores by political science journal 2007
• Journals in the Discipline: A Report on a New Survey of American Political Scientists Garand, James CView Profile; Giles, Michael WView Profile. PS: Political Science & Politics36.2 (Apr 2003): 293-308. Table 1 Political scientists’ impact, evaluation, and familiarity ratings of 115 selected journals 2002

Psychology

• Fraley, R. C., & Vazire, S. (2014). The N-pact factor: Evaluating the quality of empirical journals with respect to sample size and statistical power. PLoS ONE, 9(10), 12.
• Powell, D. M., Spence, J. R., & Stanley, D. J. (2020). I-O psychology’s top journals at the bottom of the TOP ranking: Should we consider openness and transparency when ranking journals? Industrial and Organizational Psychology, 13(3), 316-320. doi:https://doi.org/10.1017/iop.2020.52