CUSUM: A Tool For Early Feedback About Performance?

WR Chang and IP McLean

Abstract

Modern day clinical practice demands evidence justifying our choice of treatment methods. Cumulative sum techniques (cusum) are amongst the simplest statistical methods known. They provide rapid analysis and identification of trends in a series of data.

This study demonstrates the use of this technique as an early performance indicator of a clinical procedure before its implementation.

Twenty consecutive patients who underwent total hip or knee arthroplasty received a simple dressing - blue gauze and Tegaderm. Cusum charting was used to assess the dressing with regards to skin blistering. At an acceptable level of performance the curve is flat. If performance is unacceptable, the cusum slopes upward.

The cusum plot for the twenty patients was flat. This showed that our simple dressing met specified standards with regards to wound blistering postoperatively.

We recommend the use of this simple, yet versatile cusum technique in the early evaluation of a clinical procedure before its implementation.

Keywords: cusum, early performance indicator, randomized control trials, post-operative wound blistering.

Introduction

The practice of medicine has evolved through time to the current era of evidence based management. Medical audit plays an important part. It is a systematic approach to peer review of medical care in order to identify opportunities for improvement and to provide a mechanism for realising these improvements. Therefore, some form of objective monitoring, or quality control, of practices or procedures is needed so that periods of suboptimal performance in relation to an agreed standard can be recognised and remedied. The use of the cumulative sum (cusum) has been suggested for both surveillance and quality control [1]. First described by Page in 1954 [2], it was applied later to medical problems such as the replicability of urea estimations and cough remedies by 1965 [3], and were then advocated for medical use by Healy in 1968 [4]. Cusum plots may be performed on any data gathered serially. Their main use is in quality control in medical laboratories and industry. Recent experience with this simple yet versatile and powerful statistical technique has amply confirmed its utility and this it is hoped that this paper will encourage and lead to its wider use in orthopaedics.

We had noted a recent increase in postoperative wound blisters following joint arthroplasty in our District General Hospital. There was a variable practice amongst orthopaedic units with regard to postoperative wound dressings. A decision to develop a protocol was then taken. A simple dressing consisting of blue gauze and Tegaderm was used in a control led clinical trial as part of a stepwise introduction. Our aim was to ensure that it was at least as effective in early clinical outcome in reducing postoperative wound blisters to acceptable levels, whilst subjecting as few patients as possible should it prove to be unsatisfactory. The cusum enabled us to do this.

Materials and Methods

Cusum

A cusum chart is basically a graphical representation of the trend in the outcomes of a series of consecutive procedures performed over time. It is designed to quickly detect change in performance associated with an unacceptable rate of adverse outcome. At an acceptable level of performance, the cusum curve runs randomly at or above a horizontal line (no slope). However, when performance is at an unacceptable level, the cusum slopes upward and will eventually cross a decision interval. These are horizontal lines drawn across a cusum chart. Thus it provides an early warning of an adverse trend [5].

For a series of observations X1, X2, ……… Xn , the cusum can be defined as

Sn = S (X0 - Xi)

Where Xi = 1 for a success and Xi = 0 for a failure. X0 is a reference or target value set for the level of performance. A success of nine out of ten is deemed acceptable than X0 would have a target value of 0.9 (ie equivalent to 90% success). In practice, this means that for every failed attempt the cusum increases by an increment of 0.9 and each success reduces the cusum by 0.1 [6].

For example, in a series consisting of a success followed by a failure and four successes, the cusum would take the values -0.1, 0.8, 0.7, 0.6, and 0.5.

Design of The Trial

Twenty consecutive patients who underwent total hip or knee replacement were studied. All operations were performed by the same author. A simple dressing consisting of blue gauze and Tegaderm was used. Each patient's named nurse scored their wound.
The incidence of postoperative wound blistering has not been previously reported in the literature [7]. Previous audit on total hip replacement undertaken on our unit in the past showed a 10% incidence of wound blisters. This gave a target value of X0 = 0.9. Hence, a successful outcome i.e. a wound without a blister by the time of discharge would score -0.1. A blistered wound scored +0.9.

Results

The results for the twenty patients were recorded as shown in Table 1. The cusum chart for this is given in Figure 1.

Fig. 1: Record chart for the results of the 20 patients studied.

The first patient in the trial had developed a wound blister. Subsequent wound blistering occurred in the twelfth and nineteenth patient respectively. However, the cusum plot for the twenty consecutive patients was flat. This indicated that the performance of our simple dressing, blue gauze and Tegaderm, met our specified standards with regards to wound blistering postoperatively.

Discussion

Clinical medicine still involves much guesswork with consequences that may be fraught with drama and disappointment for patient and practitioner alike. In our efforts to improve this via clinical research too much emphasis has been placed on randomised control trials. Some believe that it is the only valid method for comparing treatments. A closer look however, reveals many drawbacks e.g. ethical considerations, duration of RCT's, substantial resources and funding, and difficulty in blinding just to highlight a few [8]. Hence, the need for study types other than randomised trials should be recognised.

One of these, the plotting of cumulative sums (cusum) has proved particularly valuable. Its use for examining sequential measures or for detecting changes over time has been described in the past [9]. It has also been used for plotting temperature charts for assessing antimicrobial treatment in neutropaenic patients [10]. More recently, they have been applied as a means of assessing surgical skills of trainees [6].

Our study demonstrates another possible use of the cusum. It allowed us to assess the early performance of a simple dressing in a preliminary trial before developing a departmental protocol on wound dressings. A randomised control trial may take at least four years for a single surgeon to recruit enough patients for a trial of reasonable power to compare two different dressings (estimated 140 patients in each group). In addition RCT's consume substantial resources and are therefore not justified for some questions about small modifications to treatment.

Another, advantage for using a quality control procedure is that after each observation it is possible to make one of two decisions: to accept that the level of performance is satisfactory or to conclude that it is not up to standard; a decision made if the cusum rises above a certain boundary line on the plot.

Although not much has been written about postoperative wound blistering in the literature, there is an association noted between the type of dressing used and the incidence of wound blisters [7]. Our preliminary study utilising the cusum allowed us to verify in a relatively short period that the early performance of blue gauze and Tegaderm met acceptable standards with regard to wound dressing. With such evidence, we were thus able to incorporate blue gauze and Tegaderm as part of our wound dressing protocol. In an era of evidence-based medicine, such quality control and, objective and quantified recording of the findings meet the recommended criteria for medical audit [11].

We recommend that this simple cusum technique be considered as a means of evaluating, introducing or testing any new procedure or practice. Early identification of unacceptable standards would therefore be picked up thereby exposing as few patients as possible to any unsatisfactory outcome.

References

1. Williams S.M., Parry B.R., Schlup M.T. Quality control: an application of the cusum. BMJ 1992; 304: 1359-1361.
2. Hurst H.E. Proc Am. Soc. Civil Engs. 1950; 76, no.11.
3. Armitage P. Sequential Medical Trials. Oxford, 1960.
4. Healy M.J.R. Br med. Bull. 1968; 24: 210.
5. Lim T.O., Soraya L.M., Ding L.M., Morad Z. Assessing doctors' competence: application of cusum technique in monitoring doctors' performance. International Journal for Quality in Health Care 2002; 14/3: 251-258.
6. VanRij A.M., McDonald J.R., Pettigrew R.A., Putterill M.J., Reddy C.K., Wright J.J. Cusum as an aid to early assessment of the surgical trainee. Br J Surgery 1995; 82: 1500-1503.
7. Gupta S.K., Lee S., Moseley L.G. Postoperative wound blistering: is there a link with dressing usage? J Wound Care 2002; 11: 271-273.
8. McCulloch P., Taylor I., Sasako M., Lovett B., Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ 2002; 324: 1448-1451.
9. Altman D.G., Royston P. The hidden effect of time. Statistics in Medicine 1988; 7: 629-637.
10. Kinsey S.E., Giles F.J., Holton J. Cusum plotting of temperature charts for assessing antimicrobial treatment in neutropaenic patients. BMJ 1989; 299: 775-776.
11. Shaw C.D., Costain D.W. Guidelines for medical audit: seven principles. BMJ 1989; 299: 498-499.