NHMRC grant applications: a comparison of "track record" scores allocated by grant assessors with bibliometric analysis of publications

OBJECTIVES: To investigate the correlation between the publication "track record" score of applicants for National Health and Medical Research Council (NHMRC) project grants and bibliometric measures of the same publication output; and to compare the publication outputs of recipients of NHMRC program grants with those of recipients under other NHMRC grant schemes. DESIGN: For a 15% random sample of 2000 and 2001 project grant applications, applicants' publication track record scores (assigned by grant assessors) were compared with bibliometric data relating to publications issued in the previous 6 years. Bibliometric measures included total publications, total citations, and citations per publication. The program grants scheme underwent a major revision in 2001 to better support broadly based collaborative research programs. For all successful 2001 and 2002 program grant applications, a citation analysis was undertaken, and the results were compared with citation data on NHMRC grant recipients from other funding schemes. MAIN OUTCOME MEASURE: Correlation between publication track record scores and bibliometric indicators. RESULTS: The correlation between mean project-grant track record scores and all bibliometric indicators was poor and below statistically significant levels. Recipients of program grants had a strong citation record compared with recipients under other NHMRC funding schemes. CONCLUSION: The poor correlation between track record scores and bibliometric measures for project grant applications suggests that factors other than publication history may influence the assignment of track record scores.