This paper, like the studies and investigations informing it, has been written in support of performance measurement with the aim of encouraging continuous improvement in areas of importance to a library, whether they be quality, timeliness, increased throughput or reducing costs.
The approach we propose depends entirely for its effectiveness on our experience over many years as librarians, our understanding of the processes under review and our commitment to the value and importance of library work.
For technical services we work with a best practice workflow in mind. This workflow is characterised by the lack of exceptions, the lack of costly manual record keeping and the avoidance of duplication of effort. This workflow approach is supported by a form of activity based costing but in a way that uses terminology familiar to librarians and for which issues of purpose and quality are central.
Activity based costing in accounting has been used extensively in manufacturing and more recently in service industries including libraries. It is our view that when used with libraries it often raises more questions than it answers. This is partly because the people traditionally conducting such studies often do not have detailed knowledge of the activities under review. They may understand them at a general level, but do not have the understanding of the component parts and what in fact is being measured is sometimes unclear. Often, too, activity based costing studies are conducted as a broad sweep across an organisation and the analysis is not done at a deep enough level to be meaningful. The result can be recommendations for further analysis of particular activities rather than concrete recommendations. Further, activity based costing studies may build in an interview process in order to define the activities to be costed. We use an interview process but it is central to our analysis; it is based on our knowledge of the process and is designed to tease out problems and deviations from best practice.
Nevertheless, the principles of activity based costing hold considerable promise if used by people who understand the process to be costed and if the activities are defined at a detailed and meaningful level. To this end we developed a questionnaire designed to elicit information about the time taken to complete all of the activities in any technical services operation. This questionnaire is supported by a structured interview process, a walk through of the workflow, a benchmarking database and client focus groups.
Our first chance to apply our approach in a formal way was for the Western Australian Group of University Librarians (WAGUL) in 1996. We were asked to cost the cataloguing activities in the four libraries, to make recommendations aimed at improving turnaround and reducing costs, to advise on the feasibility of a collaborative approach to cataloguing and to undertake a cost benefit analysis of outsourcing. The four libraries were those of Curtin University, Edith Cowan University, Murdoch University and the University of Western Australia. Twelve months after the study we were invited to do a follow up study in two of the libraries which had been shown to be the most costly. In both cases performance had improved considerably after implementing the recommendations. One library had reduced costs by 35% and the other by 52% and backlogs had disappeared. Subsequently one of the libraries was able to avoid a move to outsource cataloguing by showing that the cost benefit of doing so would be marginal at best. The staff of the other library won the University's award for excellence.
Since the WAGUL study we have carried out similar reviews in many libraries of markedly different types and sizes, including university libraries, both small and large, a parliamentary library and a national library. Some reviews have been limited to the cataloguing process, whereas others have included the entire technical services workflow from the time an item is ordered to the time it reaches the library shelves. We have also used our findings to advise on best practice in technical services. Besides the four universities in Western Australia, our client list includes the University of Sydney, the University of Wollongong, Southern Cross University, Australian Catholic University, the University of Western Sydney, Macquarie University, the Parliamentary Library in Canberra, the National Library of New Zealand and McGill University.
THE METHODOLOGY
We draw upon six sources of information for the purposes of cross checking and analysis.
Information provided prior to the study
It may seem to be basic to request briefing information before a study, but we insist on it. It is a vital component of our approach. This information includes any descriptive information about the process including any statement of objectives and throughput data such as orders placed, titles catalogued etc as well as cost data relating to salaries and on costs, external bibliographic services, equipment, bibliographic tools and consumables etc. We use the information to customise the structured interviews and questionnaires to suit the library's particular environment and terminology.
Structured interview process
We work with small groups of staff. Depending on the size of the operation and the amount of integration across the areas of acquisitions and cataloguing, we may focus on particular parts of the workflow in different sessions, e.g. those involved in ordering, receiving and paying for material in one session and those involved in cataloguing and authority work in another.
The interview structure is based on a standard workflow but with triggers for exceptions, duplicated effort, local customisation and other non standard practices, record keeping and paper trails and checking.
Participants are invited to describe the workflow step by step through the process and are prompted to ensure that all steps are covered. This way the discussion moves along and everyone has a chance to describe what they do at various points in the process. Whenever mention is made of duplicated checking, paper trails or record keeping, the group is asked how often mistakes are found or how often the records are actually consulted in, say, a week. This always generates animated discussion.
The process provides a detailed verbal description of the workflow (or workflows) involved, and highlights potential problem areas. The interviews may reveal work that was not indicated in the data supplied by the library. For instance, in one multi site library, the task of maintaining holdings was conducted in the branch libraries and would have remained a hidden cost had the interview process not elicited this information.
Equally importantly the interview process provides an opportunity for staff to volunteer suggestions of how they think aspects of their work could be made more efficient and the quality improved.
Time analysis questionnaire
At the conclusion of the structured interview sessions, staff complete the questionnaire. They are asked to estimate the time spent in a typical week on the listed activities relevant to them. From our experience, staff have a good idea of how they spend their time and have no difficulty completing the questionnaire. If anyone does have queries we make ourselves available for individuals to seek clarification or assistance.
The questionnaire covers the range of activities in the areas to be reviewed. These activities are broken down into specific elements, not the broad categories of activities used in a typical activity based costing. So, for example, there is not one activity for copy cataloguing, but multiple activities such as searching, downloading records, updating classification, checking authorities, adding holdings etc.
It also covers many activities undertaken that are not related to technical services e.g. desk duty, work on the library web site, etc. From our benchmark data these comprise anything from 3 - 40% of the time of a number of staff and hence can considerably affect costing.
The data from the time analysis questionnaire is coded and input to a spreadsheet. The total time taken for each activity and sub activity across the process is calculated, as is the percentage of time taken. This data is subsequently used to derive costing data, but even as time data it is a very powerful diagnostic tool when the results are compared with those from other libraries.
Walk through
The walk through of work areas situates the processes in time and space. It often serves to identify workflow issues and record keeping not identified in the structured interviews, e.g. a shelf list which officially is no longer maintained or devices such as coloured tags which indicate a different workflow from that described.
We have had feedback that staff appreciate the fact that we come to their work area and take time to talk with them and understand any constraints or limitations posed by their physical working environment.
Client focus groups
Time is set aside to meet with the primary clients of the processes under review. These may be other staff of the library such as client services staff or branch library staff or they may be academics. These sessions provide feedback regarding the expectations of, and satisfaction with, the services provided. They can be invaluable in formulating recommendations for change.
We often use these sessions to explore some of the non standard practices described in the structured interview sessions. For instance, we may ask why the books in mathematics or some other subject area are classified according to a deviation from the standard classification system. In one library there had been a mathematics lecturer 20 years before who required the deviation, but he had long gone. The client services staff had wondered why the books were classified this way and had assumed it was some cataloguing rule, whereas the cataloguers had thought it was a client services requirement.
Benchmarking database
The data from the questionnaire are fed into our benchmarking database and at this point it begins to provide a very powerful diagnostic tool when the results are compared with those from other libraries of similar size, purpose and type. It is not unusual for the structured interview process to provide a picture of an efficient operation with a commitment to timely throughput and quality. There may be no evidence of duplicated effort, exceptions or non standard practices. However, when compared with other libraries, the questionnaire results may indicate a part of the process taking more or less time than it should. In one library this was the case with end processing. The workflow described seemed straightforward but end processing was taking twice the time of other similar sized operations. We contacted the library and the resulting discussion revealed that endprocessing had been dispersed owing to a problem with a staff member who had long since left. Our recommendation straightened out this tangled workflow.
CASE STUDIES
The University of Wollongong
In November 1999, as one part of that Library's much wider commitment to quality, the University of Wollongong contracted UNILINC to undertake a study of the efficiency of the cataloguing and endprocessing operations and to provide a costing of the process. The library had recently reviewed its cataloguing operations and had made changes to improve efficiency, including integrating acquisitions and cataloguing. However, there was a lingering problem with items requiring original cataloguing. These were taking too long to reach the user and a backlog was building.
As outlined, we used structured interviews and a self-administered questionnaire, both customised for Wollongong from information supplied beforehand and coded to allow for benchmarking. The questionnaire consisted of all of the individual activities in the areas of cataloguing and endprocessing, as well as a number of other activities known to be undertaken by the same staff, such as acquisitions ordering and circulation desk duty.
One of the issues arising from the structured interview sessions was the duplication of effort in copy cataloguing and it was clear what needed to be changed. However, the size of the problem was not apparent from the interviews. The questionnaire data showed the amount of time involved. Once the questionnaire data was fed into the spreadsheet, the proportion of time devoted to copy cataloguing became clear. Comparing this with other libraries in the benchmarking database, it was apparent that copy cataloguing was taking twice as long as it was in other libraries. By adding the cost data to the spreadsheet we ascertained the cost of copy cataloguing and how that compared with similar libraries. We were consequently able to specify the benefits for throughput of the changes recommended and the likely cost savings.
Twelve months after our first study we were invited back to the University of Wollongong to complete a follow up review. In line with the findings of the first report, the operations had undergone extensive change in the intervening period. The follow up review revealed that there had been considerable improvements in efficiency and a marked decrease in costs. In fact, the cost per title catalogued had been reduced by some 32.2% and there was no longer a backlog.
Throughout 2004 the average turnaround time was 4.7 days which is well within the performance indicator that guarantees a 5 day turnaround. (Service standards are now in place for both the acquisitions and cataloguing aspects of the process and are monitored regularly within a Performance Indicator Framework which operates across the entire university library. The performance indicator Service Delivery Effectiveness is measured by the time taken to process acquired items from the time of receipt to the time an item is shelf ready.) This turnaround time was achieved in an environment in which 29% more items were catalogued than in 2003 with a staffing level that had been reduced by 35% when compared with the staffing levels at the time of the follow up review in 2000.
The National Library of New Zealand
The National Library of New Zealand engaged UNILINC to carry out a review of its Bibliographic Services Unit in 2002. The National Library's stated aim in commissioning this review was to "assess if there are opportunities to undertake these activities in a more cost-effective way".
The Bibliographic Services Unit of the National Library is a major operation with over 75 staff providing acquisitions and cataloguing services to the libraries within the National Library, including the general reference and lending collection, the music collection, the national children's collection, the Alexander Turnbull Library and its associated national archive and all the schools of New Zealand. As the provider of services to the National Library and the Alexander Turnbull library, and hence the national bibliographic database, the Bibliographic Services Unit has a particular obligation and commitment to meet standards.
The resulting report provided the required costings and made some fifty recommendations, ranging from changes to policy areas on the one hand to very specific changes to individual processes on the other. It also estimated the cost savings to be achieved by implementing the recommendations. Some of the recommendations were in fact for changes that would need more staff time and estimates of these additional costs were also included.
Following the review, the Library embarked on a process of implementing the report's recommendations. The Library has been kind enough to keep us abreast of the results of this process and has provided statistics which demonstrate the extent of the improvement achieved.
Twelve months after the review the manager of Bibliographic Services came to Sydney and brought a progress report. At that time some 44 of the original 50 recommendations had either been implemented or were in the process of being implemented. Staff had embraced the changes willingly and enthusiastically, which augured well for continued success.
A report provided in July this year, which updates the report we received in 2003, shows that for the twelve month period to June 2005 46% more material was processed than was the case in 2002, with a staffing complement that is 11% smaller. This improvement in efficiency was achieved with no loss of quality. The National Library's target is 95% accuracy and in the same twelve months an overall level of 96.1% accuracy was attained.
Interestingly, it seems that the commitment of the National Library's management and staff to the results of the review has fostered a definite change in the unit's outlook. In an email accompanying her recent report, the section manager commented that a crucial factor in the section's success has been "the ongoing focus on continuous improvement as the means of moving forward, which means that looking for incremental improvement is now a way of life for us."
Concluding remarks
For performance measurement to be meaningful it has to be based on a real knowledge and appreciation of the process involved. If the aim of performance measurement is to inform the change process leading to an improvement in service, then those responsible for implementing the change must have confidence in the results of the measurement process. Often those doing the measuring and the analysis are not around to help implement the changes, so it is important that staff and management know they have been listened to and understood. Further, it is important that the findings and recommendations are supported by clear data and as much validation as possible. Methodology is important but only if that methodology is informed by and imbued with a deep and clear understanding of the process being measured. |
Consultancy
Web-ezy (Information Skills)
