The metrics process: Baylor's development of a "report card" for faculty and departments.

In 1996, Baylor College of Medicine began the first year of its "metrics process," collecting, analyzing, and reporting data on the performance of each individual faculty member and each department in achieving the school's missions of education, patient care, research, service, and finance. This article is a report of the first two years of the process, with updates about the 1999 process, future plans, and lessons learned. The primary goal of the metrics process is to provide meaningful data to assess and improve the performance of faculty and departments across all missions. The authors (1) indicate the categories chosen, within each mission of the school, for measuring faculty time and effort (e.g., patient care, with or without learners) and state the measures chosen (e.g., percentage of time); (2) describe the development of questionnaires in 1996 and 1997 to acquire data from faculty, in the chosen categories and measures, about the time and effort they spent; and (3) report highlights of the resulting departmental data that were gathered in 1997. Among the key categories and units of measure chosen for measuring faculty (and departmental) time and effort are research grant dollars (total and per research full-time equivalent, or FTE); basic research grant dollars per square foot of laboratory space; percentage of faculty who spend at least 50% of their time in research who are National Institutes of Health principal investigators; numbers of inpatient and outpatient visits per evaluation and management FTE; total relative value units (RVUs) per patient-care FTE; patient-care income/RVU and expense/RVU for total faculty and support staff; percentage of faculty with at least one leadership position in a state or national organization; and income in excess of expense, by mission (e.g., patient care). Results of comparing data from the first two years of the metrics process demonstrate marked improvements in performance for most research measures (i.e., items of measurement agreed upon for the metrics process). The process is continually being redeveloped; the ultimate challenge is to place the objective measurements in a context where less objective qualities (e.g., innovation) also figure importantly in the evaluation and fostering of excellence. The metrics process is providing important management data, encouraging significant discussions among faculty and chairs about performance and accountability, and aiding greatly in departmental goal-setting and ultimately in determining the overall performance of the school.     

Implementing a simpler approach to mission-based planning in a medical school.

Changes in the education, research, and health care environments have had a major impact on the way in which medical schools fulfill their missions, and mission-based management approaches have been suggested to link the financial information of mission costs and revenues with measures of mission activity and productivity. The authors describe a simpler system, termed Mission-Aligned Planning (MAP), and its development and implementation, during fiscal years 2002 and 2003, at the School of Medicine at the University of Texas Health Science Center at San Antonio, Texas. The MAP system merges financial measures and activity measures to allow a broad understanding of the mission activities, to facilitate strategic planning at the school and departmental levels. During the two fiscal years mentioned above, faculty of the school of medicine reported their annual hours spent in the four missions of teaching, research, clinical care, and administration and service in a survey designed by the faculty. A financial profit or loss in each mission was determined for each department by allocation of all departmental expenses and revenues to each mission. Faculty expenses (and related expenses) were allocated to the missions based on the percentage of faculty effort in each mission. This information was correlated with objective measures of mission activities. The assessment of activity allowed a better understanding of the real costs of mission activities by linking salary costs, assumed to be related to faculty time, to the missions. This was a basis for strategic planning and for allocation of institutional resources.     

Using a productivity-based physician compensation program at an academic health center: a case study.

PURPOSE: With increased budget constraints, academic health centers (AHCs) have turned their focus on physician compensation. While many AHCs are concerned that compensation programs driven primarily by revenue generation will have a negative impact on their academic mission, little information is available to support this. The authors examined the effects on teaching and clinical productivity of an innovative compensation program for pediatrics primary care faculty at an AHC and related those effects to national standards for productivity. METHOD: A baseline productivity and compensation assessment was conducted for a group of 35 academic general pediatricians. The data were compared with Medical Group Management Association (MGMA) figures for general pediatricians. A productivity-based faculty compensation program using the work component of the relative-value unit (RVU) as the measure of productivity was designed and implemented. Productivity and compensation were measured after the first year of the program and compared with the baseline assessment. The numbers of hours precepting students and residents and the students' evaluations of their clinical experiences before and after implementation of the program were compared. RESULTS: The baseline assessment showed that over half of the faculty had productivity that fell below the MGMA 25th percentile, while the majority had compensation that exceeded this percentile. After implementation of the compensation program, 89% of the faculty increased their clinical productivity. The times faculty spent precepting and students' evaluations before and after program implementation were unchanged. CONCLUSIONS: Successful productivity-based physician compensation programs can be developed for AHCs.     

Assessing research productivity: evaluating journal publication across academic departments

Publications produced by faculty over a three-year period are used in analyzing the relative research productivity of basic and clinical science departments in a college of medicine. The citation ratings of the journals, the number of authors, and the byline position of the faculty member are used in various publication evaluation schemes. The departments vary almost tenfold in research productivity per faculty member. Results of the analysis demonstrate that the number of authors and the byline position influence departmental productivity rankings very little. Rankings are substantially affected, however, when the journals are weighted based heavily on citation ratings.     

Overcoming a perfect storm: an academic cardiology section's story of survival.

Increasingly, academic institutions are grappling with financial pressures that threaten the academic mission. The author presents an actual case history in which a section of cardiology in an academic health center was confronted with huge projected deficits that had to be eliminated within the fiscal year. The section used eight principles to shift from deficit to profitability (i.e., having revenue exceed costs). These principles included confronting the brutal facts, managing costs and revenue cycles, setting expectations for faculty, and quality improvement. The section accomplished deficit reduction through reducing faculty salaries (nearly $2 million) and nonfaculty salaries ($1.3 million) and reducing operational costs while maintaining revenues by increasing individual faculty productivity and reducing accounts receivable. In the face of these reductions, clinical revenues were maintained, but research revenue and productivity fell (but research is being fostered now that clinical services are profitable again). These principles can be used to stabilize the financial position of clinical practices in academic settings that are facing financial challenges.     

Evaluating faculty performance: a systematically designed and assessed approach.

The authors explain how the Department of Family Practice and Community Health (DFPCH) at the University of Minnesota School of Medicine has responded to the need to create for its faculty an evaluation system that provides information for both feedback and merit-pay decisions. The development process, begun in 1996, is described, and its present format detailed. Also presented are the results of a 1999 assessment of the system, which found high satisfaction among the faculty and the department head. In particular, this system has allowed the department head to have a more objective basis for making salary decisions, to increase his role as coach, and to commit more time to career correction and/or development. Other observed outcomes include an enhanced ability to track faculty productivity, increased clarity in organizational structure and goals, increased research productivity, and early retirement of senior faculty receiving low evaluations. The key components of the DFPCH system mirror recommended elements for the design of faculty evaluation systems offered by evaluation professionals. Specific elements that the DFPCH found critical to success were stable and supportive departmental and project leadership, supportive faculty, skilled staff, a willingness to weather resistance to change, tailoring of the system to the department's specific needs and culture, and a willingness to allow the process to evolve. A key question that the evaluation system has evoked at the DFPCH is whether "merit" equals "worth"; that is, does the collective meritorious work of faculty members effectively address program and departmental goals?     

Measuring contributions to the research mission of medical schools.

The authors of this article, who were the members and staff of a research panel formed by the AAMC as part of its mission-based management initiative, reflect on the growing interest in quantitative information in the management of the research mission of medical schools. They note the serious limitations of any such system of measures for research, particularly its inability to represent directly the quality of the research effort. Despite these concerns, the authors acknowledge that leaders in academic medicine have always used quantitative measures in one form or another to compare performance or assess progress. Two factors appear to be driving increases in this practice: (1) the need to demonstrate to institutional stakeholders that resources are being used wisely and that the school's performance justifies continued investment in the research mission; and (2) the need to fashion an economic strategy to manage precious institutional resources, particularly research space. Given these realities, the authors offer guidelines for the proper development and use of measures to assess contributions by faculty, departments, and institutions to the research mission. They also comment on the measures most commonly used in four areas: grants and other revenue-generating activities; publications; faculty members' research reputation and contributions to the national research enterprise; and support to the general research mission of the school. The authors conclude that quantitative information can help institutional leaders in important management decisions. However, the potential for misuse is great. The key is always to regard this information as an aid to judgment, not a substitute for it.     

Mission-based budgeting: removing a graveyard.

Many activities in today's medical schools no longer have medical students' education as their central reason for existence. Faculty are hired primarily to provide clinical service or to make discoveries, with the role of educator of secondary importance. Budgeting in medical schools has not evolved in concert with these changing roles of faculty. The cost of medical students' education is still calculated as if all faculty were hired primarily to teach medical students and their other activities were to support this "central" mission. Most medical schools still mix revenues without regard to intent and cannot accurately determine costs because they confuse expenses with costs. At the University of Florida College of Medicine, a group of administrators, chairpersons, and faculty developed a budgeting process now called mission-based budgeting. This is a three-step process: (1) revenues are prospectively identified for each mission and then aligned with intended purposes; (2) faculty productivity, i.e., faculty effort and its quality, is measured for each of the missions; and (3) productivity is linked to the prospective budget for each mission. This process allows the institution to understand the intent of its revenues, to measure how productive its faculty are, to learn the true costs of its missions, to make wise investment decisions (subsidies), and to justify to various constituents its use of revenues. The authors describe this process, focusing particularly on methods used to develop a comprehensive database for assessment of faculty productivity in education.     

Measuring faculty effort and contributions in medical education.

A national panel on medical education was appointed as a component of the AAMC's Mission-based Management Program and charged with developing a metrics system for measuring medical school faculty effort and contributions to a school's education mission. The panel first defined important variables to be considered in creating such a system: the education programs in which medical school faculty participate; the categories of education work that may be performed in each program (teaching, development of education products, administration and service, and scholarship in education); and the array of specific education activities that faculty could perform in each of these work areas. The panel based the system on a relative value scale, since this approach does not equate faculty performance solely to the time expended by a faculty member in pursuit of a specific activity. Also, a four-step process to create relative value units (RVUs) for education activities was developed. This process incorporates quantitative and qualitative measures of faculty activity and also can measure and value the distribution of faculty effort relative to a school's education mission. When adapted to the education mission and culture of an individual school, the proposed metrics system can provide critical information that will assist the school's leadership in evaluating and rewarding faculty performance in education and will support a mission-based management strategy in the school.     

Improving clinical productivity in the academic setting: a novel incentive plan based on utility theory.

PURPOSE: Academic internal medicine practices face growing challenges to financial viability due to high overhead, competing institutional missions, and suboptimal physician productivity. The authors describe the development of a clinical incentive plan for a group of academic subspecialty physicians at the Dana Clinic, an outpatient setting at Yale School of Medicine, and report on results of the first year's experience under the plan. METHOD: Utility theory was used to assess the risk profile of clinic faculty and identify incentive payments that would optimize faculty benefit or "utility" while minimizing departmental costs. Under the plan, physicians who reached a productivity target based on work Relative Value Units (wRVUs) between October 2003 and November 2004 had overhead costs covered and received a fixed payment to support salary; additional incentive payments were available for those exceeding the target. Physicians failing to reach the target were responsible for their own overhead costs and received no fixed payment. Physician productivity as measured by wRVU per full-time equivalent (FTE) was compared for the year prior to, and the year following, incentive plan introduction. RESULTS: Forty-seven members of eight academic sections were included in the analysis. Median productivity improved by 34%, with 42 of 47 physicians showing improvement. Significant improvements were also noted in collections (62%) and visit volume (23%), and shifts were observed in coding patterns. CONCLUSIONS: The unique threshold-based structure of the incentive plan, as determined through utility theory modeling, as well as permitting physicians to choose how to achieve the wRVU target were key features of its success, resulting in improved productivity without increasing practice resources or faculty salaries.     

Developing a new faculty practice plan with a model for funds flow between the hospital and the plan.

The majority of academic health centers are experiencing significant difficulties balancing their research, teaching, and clinical missions while maintaining adequate financial performance. One of the major areas under intense scrutiny is the specific financial relationship between the hospital and the full-time faculty. A realignment of the funds flow between these two entities is becoming essential to the future viability and ultimate survival of many health systems. The authors describe a model that evolved as part of the integration of the faculty practice plans of their institutions when they merged into a single health system, and that provides a framework that specifically addresses these issues of funds flow. The model includes (1) a strong partnership between the hospital and the full-time faculty; (2) a governance model of chairmen, faculty, and administration; (3) flexibility for the department chairs to set salaries and make significant financial decisions relative to their departmental performances; (4) a specific formula for funds flow for graduate medical education dollars from the hospital to the clinical departments; (5) local front-end charge capture and back-end central collections; and (6) clear and consistent definitions of revenue and expense items for both partners.     

How do medical schools use measurement systems to track faculty activity and productivity in teaching?

The authors describe their findings from a study that (1) identified 41 medical schools or medical school departments that used metric systems to quantify faculty activity and productivity in teaching and (2) analyzed the purposes and progress of those systems. Among the reasons articulated for developing these systems, the most common was to identify a "rational" method for distributing funds to departments. More generally, institutions wanted to emphasize the importance of the school's educational mission. The schools varied in the types of information they tracked, ranging from a selective focus on medical school education to a comprehensive assessment of teaching activity and educational administration, committee work, and advising. Schools were almost evenly split between those that used a relative-value-unit method of tracking activity and those that used a contact-hour method. This study also identified six challenges that the institutions encountered with these metric systems: (1) the lack of a culture of data in management; (2) skepticism of faculty and chairs; (3) the misguided search for one perfect metric; (4) the expectation that a metric system will erase ambiguity regarding faculty teaching contributions; (5) the lack of, and difficulty with developing, measures of quality; and (6) the tendency to become overly complex. Because of the concern about the teaching mission at medical schools, the number of institutions developing educational metric systems will likely increase in the coming years. By documenting and accounting financially for teaching, medical schools can ensure that the educational mission is valued and appropriately supported.     

Increasing departmental and college-wide faculty development opportunities through a teaching scholars program.

The University of Iowa Teaching Scholars Program was initiated in 1999 at the University of Iowa Carver College of Medicine (CCOM) with the overall goal of promoting leadership in faculty development related to teaching skills. Specific goals of this program are (1) to promote the development of a cadre of faculty members who have the skills to implement faculty development within their departments and the CCOM; (2) to increase departmental involvement in faculty development efforts; (3) to increase resources available for dissemination of college-wide faculty development efforts; and (4) to acknowledge the extra effort faculty put into developing their skills and knowledge in medical education and in providing continuing education to their faculty colleagues. All clinical and basic science departments in the CCOM are given the opportunity to have a faculty member participate in the program. Unlike other programs reported in the literature, competitive decisions for program participation are made at the departmental level. The three-year program combines monthly meetings and other activities to train faculty to provide faculty development in teaching skills. Each scholar develops and implements a project to address departmental faculty development needs as well as needs of other departments in the CCOM. To date (2006), 50 faculty members from 19 different departments have participated in the program with an average of 12 scholars per class. The program has resulted in a substantial increase in departmental and college-wide faculty development programming and has had a positive impact on individual scholars' teaching skills and leadership roles.     

Creating a mission-based reporting system at an academic health center.

The authors developed a Web-based mission-based reporting (MBR) system for their university's (UC Davis's) health system to report faculty members' activities in research and creative work, clinical service, education, and community/university service. They developed the system over several years (1998-2001) in response to a perceived need to better define faculty members' productivity for faculty development, financial management, and program assessment. The goal was to create a measurement tool that could be used by department chairs to counsel faculty on their performances. The MBR system provides measures of effort for each of the university's four missions. Departments or the school can use the output to better define expenditures and allocations of resources. The system provides both a quantitative metric of times spent on various activities within each mission, and a qualitative metric for the effort expended. The authors report the process of developing the MBR system and making it applicable for both clinical and basic science departments, and the mixed success experienced in its implementation. The system appears to depict the activities of most faculty fairly accurately, and chairs of test departments have been generally enthusiastic. However, resistance to general implementation remains, chiefly due to concerns about reliability, validity, and time required for completing the report. The authors conclude that MBR can be useful but will require some streamlining and the elimination of other redundant reporting instruments. A well-defined purpose is required to motivate its use.     

The Academy at Harvard Medical School: nurturing teaching and stimulating innovation.

The Academy at Harvard Medical School, established in 2001, was formed at a critical moment for medical schools in this country. Several decades of enormous growth in the biomedical research and clinical care activities of medical school faculty have resulted in great societal benefit. The unintended consequence has been a decline in faculty time and reward for the educational mission that is unique to a medical school. The impact of this decline is particularly felt now because the explosive growth in the science and technology relevant to medical practice, coupled with dramatic changes in the health care delivery system, calls for new models for the education of the next generation of physicians. The mission of the academy is to renew and reinvigorate the educational mission of Harvard Medical School (HMS). By bringing together a select group of some of the school's most talented and dedicated faculty and providing direct support for their work related to education, the academy has created a unique mechanism for increasing the recognition of teaching contributions of both academy members and the teaching faculty at large, fostering educational innovation, and providing a forum for the exchange of ideas related to medical education that cross departmental and institutional lines. The authors describe the academy's membership criteria, structure, governance, activities, institutional impact, and plans for long-term evaluation, and indicate challenges the academy will face in the future.     

Rediscovering the medical school.

Medical schools, once devoted primarily to educating medical students, have evolved into complex academic medical centers (AMCs), some of which place a greater emphasis on research and the clinical business than on educating future physicians. This occurred primarily as the result of outside forces, specifically the available revenue streams that have fostered growth. Discipline-based departments have been at the center of the governance structure of medical schools, but many AMCs now have research institutes and centers to enhance research productivity, and faculty group practices to maximize clinical revenue. Although AMCs have been successful in making scientific discoveries, developing new technologies, and providing state-of-the-art clinical care, their successes have not always been favorable to the education mission. Furthermore, the roles of departments and their chairs have not always been carefully considered; a mismatch between organizational and governance structures is occurring. In this article several suggestions are offered to help medical schools rediscover their unique reason for existence and better distinguish core missions from core businesses. Mission-based management and mission-based budgeting provide the framework for maximum success of all the missions. Specific suggestions include (1) organizing a national task force to consider optimal organizational and governance structures of modern AMCs, (2) establishing a core teaching faculty, (3) creating a matrix letter of assignment that aligns salary rates with assigned activities, (4) linking education to the provision of health care to the underinsured, and (5) forming education centers to effectively centralize governance of the education mission.     

The impact of centers and institutes on faculty life: findings from a study of basic science and internal medicine faculty at research-intensive medical schools.

PURPOSE: To examine the impact of organized research centers on faculty productivity and work life for basic science and internal medicine faculty at research-intensive medical schools. METHOD: In 2005, the authors administered a questionnaire to a random stratified sample of full-time faculty in basic science and internal medicine departments at the top 40 research-intensive U.S. medical schools. The survey instrument asked faculty about the extent of their involvement in centers and institutes, the direction and extent of their activities, and their satisfaction with various dimensions of work. RESULTS: A total of 778 faculty members completed the questionnaire (72.0%). Basic science faculty with center affiliations produced more research publications and grants while devoting comparable effort to teaching as their non-center-affiliated peers. These faculty reported greater dissatisfaction in workload and in the mix of their activities. Internal medicine MD center-affiliated faculty were more productive in research activities and spent less effort in patient care and more effort in research than their non-center-affiliated peers. These faculty were more satisfied with promotion, opportunities for research, and the pace of their professional advancement. CONCLUSIONS: Findings indicate that faculty from different departments and with different ranks and backgrounds interact with centers and institutes in multiple ways. For basic science faculty, center involvement appears to be an addition to, not a substitute for, their usual departmental obligations. For internal medicine MD faculty, center involvement appears to serve as an opportunity for protected effort in research away from the demands of clinical practice.     

How do academic health centers value and encourage clinical research?

To investigate whether there is a misalignment of the perceived values of and incentives for clinical research within U.S. academic health centers (AHCs), in 1999 the authors surveyed medical school deans, academic administrators, department chairs, and faculty members at 80 AHCs that are the members of the University HealthSystem Consortium, a not-for-profit consortium of AHCs. A total of 358 faculty from 58% of the institutions surveyed responded, with a mean of 3.76 responses/institution. There was general agreement that clinical research offers AHCs a considerable spectrum of benefits, including prestige, recruitment and retention of faculty, criteria for promotion of faculty, and financial support. Investigator-initiated research and government-funded research ranked highest in terms of their desirability compared with industry-sponsored and contract research. This preference was agreed upon across all categories of respondents and types of research (translational, clinical tests, and outcomes). Significant differences existed between the perceptions of deans/AHC administrators, who stated that they were increasing their emphasis on clinical investigation in the areas of research space (56% of responders), administrative support (81%), and patient recruitment (61%) and the perceptions of their departmental chairs/faculties in the same areas (34%, 52%, and 40%, respectively; p <.05). Faculty opinions documented few new investments in the actual infrastructure dedicated to clinical research. The authors conclude that their findings, which they consider reasonably representative, strongly suggest that the value of clinical research to AHCs is well understood. Their findings also identify important opportunities for AHCs to provide a wider range of incentives for the conduct of clinical research.     

An incentive plan for professional fee collections at an indigent-care teaching hospital.

The authors describe the implementation and development of an incentive plan to improve professional fee collections at an indigent-care teaching hospital. They theorized that an incentive plan based on relative value unit (RVU) productivity would increase billings and collections of professional fees. Unique RVU targets were set for individual services based on the number of faculty full-time equivalents and average reported productivity for academic physicians by specialty. The incentive plan was based on the level of expected faculty billings, measured in RVUs, for each department. A "base + incentive" model was used, with the base budget being distributed monthly throughout the year, and the incentive held as a "withhold" to be paid at the year's end only if the billing target in RVUs was met. Additionally, a task force worked with physician billing office and the hospital to improve collections. In the first year after implementation of the system was in place, important increases were noted in total RVU productivity (30.5% over the previous year) and in collections (49.5% over the previous year). Sixteen of 23 departments exceeded their incentive targets, and it was possible to make distributions of professional fees to those departments, to be used within the hospital system to enhance clinical services. Moreover, the plan created an overall positive attitude toward billings and documentation of faculty activities. The authors believe that this kind of incentive plan will be increasingly important for academic faculty working in public hospital systems.     

Implementing a comprehensive relative-value-based incentive plan in an academic family medicine department.

The authors describe the implementation and first three years (1997-1999) of a department-wide incentive plan of the Department of Family Medicine at the State University of New York at Buffalo School of Medicine and Biomedical Sciences. By using a consensus approach, a representative elected committee designed a clinical relative value unit (explained in detail) that could be translated to equally value and reward faculty efforts in patient care, education, and research and which allowed the department to avoid the imposition of a model that could have undervalued scholarship and teaching. By 1999, the plan's goal of eight patient-care-equivalent points per four-hour session had been exceeded for pure clinical care. Clearly, only a small financial incentive was necessary (in 1999, an incentive pool of 4% of providers' gross salary) to motivate the faculty to be more productive and to self-report their efforts. Long-term productivity for pure clinical care rose from 9.8 points per session in 1997 to 10.4 in 1999. Of the mean total of 3,980 points for the year 1999, the contribution from teaching was 1,146, or 29%, compared with 25% in 1997. For scholarship, the number of points was 775, or 20%, in 1999, compared with 11% in 1997. The authors describe modifications to the original plan (e.g., integration of quality measures) that the department's experience has fostered. Problems encountered included the lack of accurate and timely billing information from the associated teaching hospitals, the inherent problems of self-reported information, difficulties of gaining buy-in from the faculty, and inherent risks of a pay-for-performance approach. But the authors conclude that the plan is fulfilling its goal of effectively and fairly quantifying all areas of faculty effort, and is also helping the department to more effectively demonstrate clinical productivity in negotiations with teaching hospitals.