IUB Libraries Peer Review Committee
Julie Bobay, (chair,) Kris Brancolini, Mary Strow, Phil Bantin
April 20, 1999

Contents

• Summary Report
• Procedures
• Results
• Criteria
• Recommendations

COMMITTEE SUMMARY REPORT

The committee read and evaluated the 1998 annual reports of 65 tenure-track librarians according to the procedures and criteria in the librarian-approved document "Peer Committee for Annual Merit Review." Similar to our experience on the Promotion and Tenure Committee, we found ourselves impressed with the breadth and significance of the many contributions IUB librarians are making to the libraries, the university, and the profession. We’d like to thank the Libraries’ Human Resources Department, and particularly Betty Andis, for her help making copies of annual reviews, creating worksheets, and other invaluable support.

Throughout our work, we found it helpful to continually remind ourselves of two important facts:

• The purpose of this evaluation is to AWARD ANNUAL SALARY MERIT INCREASES, not to evaluate a librarian’s overall contributions over a career. Our ratings, especially those of "unsatisfactory," should NOT be the basis for any action other than 1998 salary increase decisions. In most cases of "unsatisfactory" ratings, our judgement was that librarians met or exceeded expectations for the performance of their position, but had an off year in professional development or service.
• A rating of "satisfactory" does NOT mean "ok." Another way to state this important fact is that "satisfactory" does NOT mean a grade of "C." A rating of "Satisfactory" indicates that the librarian’s performance, professional development and service activities in 1998 met expectations for rank. In the case of performance, a "satisfactory" rating reflected the committee’s evaluation that librarian’s performance was "good," "excellent," or "superior," depending on rank.

We believe peer review is an essential component of annual salary increase decisions, and we hope this process will provide a positive "check and balance" in the overall system of determining annual salary increases. We further believe that this process voted in by the librarians for making such evaluations is basically sound. It allowed us to identify librarians who made extraordinary contributions to the libraries beyond the requirements of their position descriptions in 1998, and also to evaluate librarians’ annual contributions to the profession through professional development and service. In addition to providing a valuable check and balance mechanism for determining annual salary increases, this process also allows a committee of peers to provide valuable feedback to librarians on how they did in a single year in meeting expectations for professional development and service. This is important feedback that is often not available to post-tenure librarians except at the time of promotion to full rank.

The committee did experience some difficulties in the application of this procedure, however, and later in this report we list some recommendations for some changes that we believe would result in improvements to the process.

PROCEDURES THE COMMITTEE USED

Using the criteria for promotion for librarian ranks and the matrix for assigning merit categories, the committee evaluated librarians’ 1998 performance, professional development and service activities. At our organizational meeting on March 8, we invited Elizabeth Johnson to describe the previous peer review process of several years ago, and asked her to summarize the current procedure. We selected a chair and agreed upon procedures. Over the next two weeks (one of which was Spring Break,) all committee members read the 62 annual reviews we had received. Each committee member filled out individual ratings for each librarian for each area (performance, professional development, service). Over the next two weeks, we met five times, refined and articulated our interpretations of the criteria for each rank, came to consensus on a single committee score for each librarian for each area. We then applied the matrix to our ratings and assigned a "category," (Exemplary, Meritorious, Satisfactory, Unsatisfactory,) to each librarian based on the committee scores for each of the three areas, and wrote notes on the bases for our ratings. The chair of the committee drafted a master sheet with final written comments. After finalizing the ratings and comments, we wrote a summary report on the overall outcomes of the process and our recommendations for changes, and submitted it to the secretary of the BLFC. At the same time, we submitted to Dean Thorin the summary report and our overall category ratings (E, M, S, U,) and comments for each librarian. Finally, we submitted to Libraries Human Resources for the permanent file the summary report, the overall category ratings/comments for each librarian, and committee scores for each librarian in each of the three areas (performance, professional development and service). We then met one more time to evaluate three annual reviews that had come in late or had not been available for other reasons, and submitted amended reports to Dean Thorin, Library Human Resources, and the secretary of the BLFC.

RESULTS OF THE PEER COMMITTEE’S REVIEW FOR 1998

• Exemplary (Distinctive: readily acknowledged as being of the highest achievement): 3 librarians (1 full rank, 1 Associate, 1 Assistant)
• Meritorious (Noteworthy): 22 librarians (5 Full, 14 Associate, 3 Assistant)
• Satisfactory (Meets normal and expected standards for rank): 25 librarians (5 Full, 18 Associate, 2 Assistant)
• Unsatisfactory (Fails to meet normal and expected standards for rank): 15 librarians (4 Full, 11 Associate)
• Not rated 10 librarians (non-tenure track librarian; Directors, Executive Associate Dean, new hires)
• Median: Satisfactory

CRITERIA THE COMMITTEE USED IN EVALUATING EACH OF THE THREE AREAS:

RECOMMENDATIONS FOR CHANGES/REFINEMENTS IN THE PROCEDURES

A. SCOPE

The procedures should clearly describe positions that are not included in the peer review process. This definition should cover administrative positions that are excluded (in terms that survive administrative reorganizations,) as well as policies on part-time librarians, librarians on exchanges or sabbatical leaves, librarians who started in their position late in the calendar year, etc. Also helpful would be an articulation of the consequences for librarians who do not submit annual reviews, such as becoming ineligible for annual merit salary increases.

B. TIMETABLE

Although ambitious, the timetable of three weeks to complete the process is achievable, IF one of the weeks is not Spring Break. The scope of the work realistically precludes the committee from reading additional documentation; we recommend that librarians not include any documentation, and tell the committee what it needs to know in the annual review document itself.

C. MATRIX FOR DETERMINING MERIT CATEGORY

The committee felt generally positive about the matrix as a tool for establishing the overall category for each librarian. Some members of the committee question, however, whether it’s appropriate for a librarian who receives a "4" in performance to get an "unsatisfactory" category when professional development or service do not meet expectations for rank.

The committee is very uncomfortable with applying the word "unsatisfactory" to librarians who did not meet expectations for rank in a given year. We believe this word carries many negative connotations that go far beyond the intent of this process, which is to provide input for the distribution of annual salary increases. We regret having to label librarians who had an off-year or who did not realize they would be judged on these criteria as "unsatisfactory," and worry that the label may have consequences beyond the single year salary setting decision.

D. CRITERIA

The committee experienced several difficulties in applying the promotion criteria to annual accomplishments.

1. "Continuing improvement"
   Promotion criteria that refer to "continuing improvement" are appropriate for reviewing a promotion or tenure dossier, but are not relevant when applied to a single year. For this reason, and others listed below, we recommend that the BLFC develop annual review criteria that are based on, but are not exactly the same as, the criteria for promotion. These criteria are needed to adequately evaluate a one-year "snapshot" of a librarian’s career. (Note: for this first year, the committee ignored the criterion for continuing improvement, and applied the secondary and tertiary evaluative words instead.)

2. Evaluating performance
   We support the view that performance should be reviewed by peers and that it should be the most heavily-weighted component of a librarian’s achievements. It was difficult, however, for the committee to use the current procedure to fully evaluate all aspects of performance. This is a very important point given the large impact the performance evaluation has on the final merit category.
   
   With the information we had, we felt we were able to identify librarians who made extraordinary contributions to the libraries in 1998 beyond their "real jobs" as described by position descriptions, and we gave those librarians "meritorious" or "exemplary" ratings for performance. We rated most other librarians as "satisfactory" (met expectations for rank.) We regret that we were not able to identify librarians who truly excelled in performing "just" their jobs, and we also regret that we were not able to identify librarians who may not have met expectations for their positions.
   
   To improve the committee’s ability to evaluate performance that falls within a position description, we recommend that the committee be given additional evaluative information (such as supervisor’s responses) OR that a process be devised that includes specific performance goals for the year. The inclusion of such goals and objectives, both personal and how the librarian’s activities further the goals and objectives of their department, and the librarian’s explanation of which accomplishments she considers noteworthy, would help the committee evaluate performance of duties within the position description.
   
   Further, our experience leads us to recommend that the annual salary setting evaluation of PERFORMANCE should not be tied to rank, but rather to the demands of the position. In the process of applying the current system to actual librarians, we uncovered some unintended and unwanted anomalies. For example, consider the following scenario. Full Librarian A and Associate Librarian B both perform in their current positions in "superior" fashions. Librarian A, being much more senior in her career, has grown into a much more demanding position as her career has progressed, and meeting the operational standards of her position demands superior and extraordinary efforts. Using the current criteria, Librarian A gets a "satisfactory" rating; Librarian B gets a "meritorious" rating. This is a serious disincentive for Associate Librarians to come up for full rank.
   
   Finally, some librarians have significant library "service" that has essentially become a permanent part of their jobs. Under they current system that weights performance so much more heavily than service, they may benefit from having these duties actually moved to their job description so their contributions are weighted more heavily.

3. Evaluating Professional Development and Service: "secondary" and "tertiary" areas
   Criteria for annual salary merit evaluations should clearly reflect the relationship, if any, that exists between professional development and service. If we are to use promotion criteria for annual salary evaluations and apply different criteria for secondary and tertiary areas, librarians should be asked to designate their secondary and tertiary areas in their annual reports. If not, BLFC should develop criteria for evaluating these two areas separately.
   
   In the absence of such designations of secondary and tertiary areas this year, the committee applied the more demanding "secondary" criteria to both areas unless that criterion resulted in an "unsatisfactory" rating in one of the areas. When that happened, we applied the less stringent criteria for the "tertiary" area to that area. We noted in our comments those librarians for whom we invoked "tertiary" criteria for one or both of the areas.
   
   This process uncovered some problems with "secondary" and "tertiary" criteria for these two areas. For example, imagine Full Librarians A and B doing exactly the same level of service. Librarian A shows "continued significant contributions" for both areas, thus meeting expectations for rank for the secondary area, yielding a "satisfactory" rating for both. Librarian B meets this expectation for service and gets the same "satisfactory" rating, but does not show "significant contributions" for professional development. Applying the "tertiary" criteria of "at least satisfactory" to Librarian B’s professional development activities gains her a "satisfactory" rating for that category as well. Librarian A’s greater accomplishments in professional development are not recognized because a higher standard was applied to her.

E. TRAINING

We recommend that BLFC take a leadership role in providing guidance for librarians and supervisors to understand and use this process effectively. Some librarians, especially those who do not attend the annual promotion and tenure workshops, need guidance in knowing which activities fall in "professional development" and "service." Librarians put editing, refereeing and book reviews in the professional development section rather than service, and some listed job-related staff training workshops as service or professional development rather than performance. The review committee sometimes moved activities between these categories in our evaluations. Supervisors also need information and guidance on how the system works and how the various areas are evaluated.