An Introduction to Cognitive Behaviour Therapy, 2 edition

5

Measurement in CBT

Introduction

We have already discussed the commitment within CBT to an empirical approach in establishing the effects of treatment, both for groups of clients and for individual cases, and this issue will be considered in more detail in Chapter 18. This chapter will describe how to translate this empirical approach into action with individual clients. We will look at how measurement can be used to increase your understanding of problems at the assessment stage and during subsequent treatment. We will also consider why it is worthwhile using measures in this way, how to devise them and will give examples of the kinds of measures that might be useful.

The empirical nature of CBT

From the start, we want to encourage the client to view treatment as an experiment, in which thoughts, feelings and behaviour, and the relationships between them, can be investigated during both assessment and treatment.

Assessment and formulation

At assessment, it is helpful to ask the client to collect data about the nature of the problem, to supplement and fine-tune what he reports in the assessment interview. Such data may contribute to two main goals.

 

  1. To help elaborate the formulation: e.g. it may be helpful to look at triggers for particular thoughts, feelings or behaviours, and how these relate to each other, so that tentative ideas about a formulation can be explored further.
  2. To provide a baseline against which the problem can be compared in the future: for example, measuring the frequency or severity of the problem.

A depressed client believed that she was ruining her children’s lives by continually ‘going on’ at them, shouting inappropriately and being unable to regain control once she had ‘lost it’. She agreed that it would be useful to keep a diary to find out how frequently this occurred, and when it happened (see Figure 5.1). The main thing that she learned at this stage was that in fact she did it rarely, only twice in a week. This was very helpful information, as she saw herself as always angry and nagging, presumably as a result of noticing and then remembering situations that were consistent with her belief about herself.

Figure 5.1    Rating scale for how often a woman ‘lost it’ with her children, and what triggered it (‘losing it’ meant shouting for more than a minute)

During and at the end of treatment

Once the client has a good description of the problem, what triggers and maintains it, he can begin to try out new ways of behaving, thinking and interacting and then assess what effect this has on the problem. Regular measures allow both client and therapist to evaluate the impact of interventions. It is particularly important to gather data at the end of treatment so that overall progress can be assessed.

A client with OCD recorded how long it took her to leave her house if she planned to go out, and how long it took her to get away from work at the end of the day. As she introduced response prevention into a number of tasks (see Chapter 14), she could clearly see the effect of this intervention on the length of time it took her to leave home and work (see Figure 5.2).

Figure 5.2    Length of time spent leaving places when disrupting OCD rituals

To take another example of evaluating treatment changes:

A man who was very anxious away from home realised that arriving early for trains was a safety behaviour when travelling. It kept in place his belief that ‘only by staying 100% in control will I be safe, and not be rejected’. He experimented with arriving at places either just on time, or even late, in order to find out whether there were catastrophic results and found that no one commented or appeared to notice. To his surprise, he found that he was slightly less anxious on the days that he reduced his punctuality, not more anxious, as he had predicted (see Figure 5.3).

Figure 5.3    Chart of arriving on time (or late) at events and the effect on others and anxiety levels

Although very simple measures were used in each of these cases, they provided helpful data about the nature of the problems and their responses to straightforward interventions.

Why bother with measurement?

You may need to be creative and ingenious in devising useful monitoring to assess problems and evaluate the effects of different interventions. However, this begs the question of why you should use measures at all. There are a number of reasons why it is helpful to gather data to supplement information derived from interviews:

 

  • Regular measures allow you to obtain a baseline of important aspects of the problem and then to use that to assess the effects of future intervention.
  • Observations of behaviours, thoughts or feelings made at the time they occur are more reliable than retrospective estimates (Barlow, Hayes & Nelson, 1984).
  • Direct observations by the client in real life can have therapeutic effects in themselves, for example by providing accurate information about the scale of the problem or about its progress.

A client assiduously produced thought records each week and carefully picked the most distressing examples for consideration in the session. This significantly coloured her assessment of how the previous week had been, as her attention was focused on the difficult times. To counteract this, she began to rate her mood three times daily (eventually reducing this to once daily when her mood was more stable) and was very surprised to find that on many days her mood was considerably lighter than she was reporting. This was very encouraging for her, as she had been doubtful that she would be able to use cognitive therapy.

 

  • Once they have begun to improve, many clients lose awareness of how disabling the problem was initially. Baseline measures of the problem can help the client to assess his progress more accurately.

As a client’s agoraphobic symptoms improved, he focused on his difficulty driving to nearby market towns, claiming that going into his local town had never really been a problem. This was dispiriting for him, as it seemed that he was making little progress. However, a review of his early diaries was sufficient to reassure him that indeed he had made enormous progress and that tasks that he now took for granted had initially presented real problems for him.

 

  • If an intervention does not have the impact that the formulation would predict, measurement can help you and the client to work out why. For example, it may be that the treatment is not being delivered appropriately. A client was feeling out of control and overwhelmed by everyday domestic tasks.

As a first step, she decided to spend 20 minutes on three days each week tidying up papers that covered the surfaces in her kitchen, and to rate how overwhelmed she felt. This intervention appeared to have little impact, but a diary in her therapy notebook indicated that she was only managing to do it once a week and was taking this as further evidence that she was overwhelmed. These data allowed the therapist and client to trouble-shoot and work out how to increase the probability that she would carry out the task – in this case, by doing it at her most productive time, when she was alone in the morning.

There are therefore sound reasons for using measures as part of routine clinical practice. We shall now consider how to do this in ways that will provide you and your client with information that will be genuinely helpful for therapy, beginning with a note on the psychometric qualities of measures.

Psychometric aspects of monitoring

Reactivity of measurement

The process of monitoring can have a positive or negative effect on whatever is being measured. In habits such as smoking, a beneficial reduction may occur if the client becomes aware of triggers and responds to the beginning of a potential cycle by inhibiting a response. On the other hand, change can be in the opposite direction. For example, some clients’ response to the initial monitoring of NATs is an increased preoccupation and/or frequency of negative thoughts, which may increase anxiety or depression in the short term. It is helpful to tell your client that a temporary exacerbation of the problem is possible and to encourage him to persist with monitoring long enough to see its longer-term advantages.

Validity and reliability

When standardised measuring instruments such as questionnaires are developed, an enormous amount of attention is paid to psychometric qualities, particularly validity and reliability.

valid measure is one that measures what it purports to measure and not some irrelevant feature: for example, a questionnaire of social anxiety should not be couched in language so complex that responses to it are affected by a person’s verbal ability, or by the differing norms of his social group.

reliable measure achieves the same result or score when repeated under the same conditions at another time, or with another assessor. A measure low in reliability is affected by extraneous features and produces inconsistent findings.

Standardised measures, such as a well-developed mood questionnaire, will usually have been tested for validity and reliability. However, in many cases you need ingenuity to devise more idiosyncratic measures, and it is then important to try to make them as reliable and valid as possible in the circumstances. The following section suggests how this can be achieved.

Obtaining useful and accurate measures

Most of the principles that will be described here are easy to apply but can make an enormous difference to the value of the measures used.

Simplicity

Do not overburden your client. Begin with a limited task that does not ask too much. As your client becomes more persuaded of the value of information obtained through monitoring, and becomes more skilled, you may increase the demands on him, but still keep in mind the difficulty of observing and recording.

A depressed man began treatment by going out each day for a short walk, and he recorded how long he walked for (in minutes), and how much he enjoyed it (on a 10-point scale of enjoyment). As treatment progressed, he also began to record NATs, to rate how critical his wife was (on a 10-point scale), and to record the three best things he had done that day. In addition, he was able to monitor specific activities/tasks for particular purposes. This was quite onerous for him, but he felt it was all relevant.

It is important to continue monitoring only while the measures continue to be useful. On the other hand, bear in mind that it is useful to have some measures, for example a mood rating or questionnaire, that are continued throughout treatment so that you can look at variations over the course of treatment.

Consider measures in more than one system

Although it is important to limit demands on the client, you should bear in mind that different aspects of the problem may change in different ways, and that this detail may need to be tracked.

A woman with anxieties about her health focused on reducing the amount of time that she discussed her worries with her husband and mother, or asked for reassurance (i.e. a behavioural aspect). She kept records (see Figure 5.4) which included information about behavioural, cognitive and emotional aspects of the problem. In the first two weeks, her success in effecting behavioural change had little impact on her anxiety or on the strength of her beliefs that she may have something seriously wrong with her health, but they then began to improve.

Figure 5.4    Records of different aspects of health anxiety

Relevance

Only ask for information that you will use and that will make a difference to treatment. It is unlikely that the client will go to the trouble of monitoring unless he sees its relevance, and it may also jeopardise the therapeutic relationship if you ask for information merely ‘out of interest’.

Specific, clearly defined targets

In order to improve the reliability of your measures, try to ensure that two observers engaged in the same task would agree on their observations. This means spelling out in detail what you want to be recorded.

If you are asking someone to record the frequency with which they ‘lose their temper’ you could ask, ‘Let’s try and be specific about what we mean. For the purposes of this exercise, what would you want to include in “losing your temper”? What would you be doing that would mean that you had lost it?’ This might include shouting loudly, saying unkind and inappropriate things, banging doors; but would not include talking across someone or feeling angry but not shouting.

The advantage of operationalising in this way is that, should any incident occur, the client would not have to make a judgement at the time about whether what had happened was included in the definition.

It is not uncommon that an internal state is the focus of measurement, in which case it is not possible to use the criterion of agreement by two observers. Nevertheless, you should take care to minimise the ambiguities in what is being recorded.

One client became dissociated in a number of situations and was recording where this happened. It was agreed in advance that she would look for instances where she had been unaware of her surroundings but that she would not include occasions when she felt unpleasantly vague and light-headed but was still aware of where she was.

Provide clear and simple instructions

Do not expect the client to remember what the task entails, because he may either forget it altogether, or his memory may distort the task: write it down (or, even better, get the client to do so).

Use sensitive and meaningful measures

In some cases, measures that are most sensitive to change, and therefore helpful in plotting progress, may not capture the characteristics of the problem that are most important for the client. Both sensitiveand meaningful measures are important but for different reasons: the first because they allow you to look at the effects of interventions relatively quickly, and the second because they focus on what the client believes to be the central, meaningful aspects of the problem.

A depressed woman was most interested in whether her mood was improving in response to treatment. As part of therapy, she was trying to increase the number of pleasurable and satisfying activities she engaged in, and she kept a daily record of how many hours she managed to work, and how many social contacts she had, and these were totalled each week. She also kept a daily mood rating. Although these measures were directly related to an aspect of the formulation (reduced activity), she was more interested in her scores on the Beck Depression Inventory that she completed every fortnight, as she felt this best captured how she was getting on in general.

Provide aids to recording

Minimise demands on the client by providing as much support as possible for the practical task of monitoring, at least in the early stages of therapy. Rating forms or diaries should be drawn up for the client, with as many copies as will be required. The record sheets should be as simple and discrete as possible, bearing in mind that many clients would be embarrassed to be seen recording personal information. For example, the client could carry a small index card for recording information on a day-to-day basis; he could use differently coloured dedicated pages in his diary; or he could use a memo pad in his mobile phone or hand-held computer.

Train the client to use the measure

Even if the task appears to be straightforward, always ask your client to go through a recent example and carry out the recording process with you. This will ensure that the task is clear to him and will allow you to discuss difficulties that crop up. For example, you could say, ‘Can we think about the last time that you felt panicky, and fill in the record about that? What would you put in this column here, where it says “Situation”?’

Also spend some time clarifying rating procedures, as they may be unfamiliar to your client.

For example, you might say, ‘That’s interesting; people often experience several emotions, which is why the column is labelled “Emotions”. It is also useful to have ratings of the strength of the emotions, so let’s go over that. Zero on this scale means that you are not feeling anxious at all, and 10 means “as bad as you could possibly imagine feeling”. Can you think of a time when you have felt like that? … and what about a 5; can you think of a time when you felt moderately anxious, halfway between these two? …. and what about a 7? Can you think of a time when you have felt quite a bit more than “moderate”, but not as extreme as a 10?’

Remember that you are anticipating that your client will learn the skill of self-monitoring and be able to use this to manage problems in the future.

Collect data as soon as possible after the event

If records are not completed until some time after the event, it is likely that recall will be less vivid and/or will be biased by the client’s mood at the time he completes the record. It may not be possible for him to record an experience as soon as it happens, particularly if he is with other people, but he should be encouraged to go over in his mind what he will record and to complete the task as soon as it is practically possible. Alternatively, it may be possible for him to make a brief note at the time and to complete the full monitoring at a more convenient time.

Pay attention to the monitoring

The therapist should never fail to take notice of information that has been collected. If the information is truly valuable, then the next session should to some extent rely on it; but in any case, it is important that the client’s efforts are rewarded by genuine interest, so that he will be willing to continue monitoring in future. Ensure that feedback on any such homework is part of your shared agenda for the session.

What sorts of information to collect

There are many different ways of recording useful information, and the following examples give a flavour of this variety. There will be other examples in later chapters, and many academic papers and books will also give measures for specific problems that can be adopted for clinical use.

Frequency counts

A useful rule of thumb is that if there is something relevant that you can count, then count it. Counting is potentially the most reliable measurement method, even though it may appear to be overly simplistic. The variety of features that can be counted is almost boundless, and it is worth trying to think of aspects of the problem that could be measured in this way. Examples include:

 

  • number of self-critical thoughts;
  • number of times of checking (that the house is locked, that there are no spider’s webs, etc.);
  • number of eye-lashes pulled out (in trichotillomania);
  • number of toilet rolls used in a week (to assess OCD problems or bladder/bowel worries);
  • number of phone calls received;
  • number of times of swearing;
  • number of times clothes changed;
  • number of urges to binge.

Therapist and client creativity is the only limit to the variety of possible frequency counts.

It is important to have an idea of what the frequency might be before monitoring; it is not helpful to ask someone to record the number of intrusive thoughts in a day if the total is likely to be a few hundred! Should the frequency be very high, then the client can be asked to take a sample at a relevant time of day (for example, a half-hour period when the thoughts are at their most troubling) or, if there is no reason to focus on a particular time, an arbitrary time (for example, between 5 and 6 o’clock).

Duration of event/experience

The duration of an event or experience may also be relevant, and is also likely to be a reliable measure. Examples include:

 

  • time spent washing for an obsessional client;
  • time spent checking his body for someone with health anxiety;
  • time spent travelling alone for a client with agoraphobia;
  • time able to concentrate on reading for a depressed client.

Again, use your imagination.

Self-ratings

These are amongst the most commonly used measures, as they can capture the quality of internal events such as emotions and cognitions. They are less reliable than frequency counts or measures of duration, but their reliability can be improved if the simple guidelines outlined above are followed. Although they are more reliable than a simple description of the experience, they remain subject to shifts in ‘anchor points’, in the sense that a rating of ‘moderate’, or 5 on a 10-point scale, may mean something different at the beginning of treatment compared with the end, as the individual gradually comes to have fewer highly distressing experiences.

If a discrete event is being monitored, then the client can be asked to rate it each time it occurs. For example, a man with anxiety about micturition rated how anxious he felt before going to the lavatory and also rated how much urine he passed (see Figure 5.5).

However, if the phenomenon being measured is continuous (as anxiety may sometimes be) or occurs very frequently, then it may be necessary for the person to choose a time to rate (as described above under ‘Frequency Counts’). An alternative is to do an average rating for a period of time: for example, to rate average anxiety during the morning, afternoon and evening; more detailed information can then be obtained, for example using the level of anxiety as a cue to begin to look at triggers (see Figure 5.6).

Figure 5.5    Diary for a man anxious about micturition

Figure 5.6    Diary of anxiety about work

Diaries

Diaries can combine the kinds of measure described above and allow you to look at the links between different aspects of problems, such as the relationship between the problem and particular triggers, safety behaviours and modulating variables. As diaries are more multi-faceted, it is even more important to pay attention to setting up the recording and training the client in their use. Unless care is taken, the client may return with information that is inconsistently collected and difficult to analyse. Get feedback from the client about what is relevant, whether the recording sheet seems sensible and whether there are ambiguities that would make it difficult to use.

Figure 5.7 shows a diary from a woman with a phobia about vomiting, which prevented her from carrying out a range of social and domestic activities. The diary included aspects she felt were important, particularly her sense of achievement, which compensated for the anxiety she experienced in the short term.

Figure 5.7    Diary recording success in dropping safety behaviours

Two diaries in common use are described later: the Dysfunctional Thought Record (DTR) in Chapter 8, and the Activity Schedule in Chapter 12.

Questionnaires

There is an enormous range of questionnaires available for clinical use, many of them originally developed for use in research trials (see Chapter 18 for some questionnaires commonly used in clinical practice). A major advantage of many questionnaires is that they will provide you with the scores of relevant groups – for example, the normal population, or a group of depressed out-patients – so that you can compare your client’s score to others. However, a questionnaire may not be as sensitive a measure as a simpler record focused on the client’s own problem. In other words, questionnaires are different from rating scales or frequency counts, but not necessarily better: it depends on what information you need. In any event, it is important to use questionnaires that are well standardised and validated, otherwise, the results of the questionnaire may be unreliable.

Other sources of information

Although the majority of the information used in therapy is provided by the client, different data sources can be relevant: these might include other informants, live observations of behaviour and physiological data.

Other informants

It may be helpful to interview other people for several reasons.

 

  • They may have information unavailable to the client – for example, a client may believe he behaves oddly in social situations, and someone else’s view can provide useful information; or a relative may report that their spouse is very quiet and initiates little conversation whether at home or out socially, and this may not be obvious to the client.
  • The problem may have an impact on the other person – for example, a client with obsessional problems may involve relatives or other significant people in his rituals, as with a client who would not allow anyone into the kitchen for the rest of the day after she had rigorously cleaned it.
  • The way someone else responds to the client’s problems may be relevant to its maintenance; for example an elderly husband was very loathe to let his wife go out for even a short walk, to begin building up her self-confidence following an RTA, as he feared that she might become unsteady or lose her way.
  • The other person’s beliefs about the problem may be important – for example, that medication is likely to be the only effective solution.

Other informants should be approached in a similar way to the client, recognising that they will need to be engaged, to be given hope and, possibly, to be educated about CBT. The reasons for using Socratic enquiry will also apply to them just as much as to the client (see Chapter 7).

Although an interview is the most common way of obtaining information from other informants, they can be asked to provide more directly observed material in the same way as the client. Frequency counts, ratings, diaries and questionnaires may all be useful in some circumstances.

The issue of confidentiality should be discussed with both the client and the other informants, to establish whether there are things that either party does not wish to be disclosed. It is worth checking out whether the reasons for this are well founded or perhaps based on an erroneous belief. For example, a relative may be concerned that mentioning worries about suicidal ideas might put such ideas into the client’s head, when in fact this is not a risk.

Role play and live observation

Observing your client at the time the problem occurs can provide significant information that the client has forgotten or of which he was unaware. For example, a client with a complex hand-washing ritual took for granted some of the details involved, including that she washed the soap and put it back on the sink after completing each stage in the ritual; a man with social phobia was unaware of the extent to which he averted his gaze in casual social interactions.

Sometimes, you can observe behaviour in naturalistic settings: for example, a therapist accompanied a client with social anxiety into shops and observed his interactions when he asked for goods or presented items for purchase. At other times, you might contrive a situation: for example, the therapist asked a client with OCD to briefly ‘contaminate’ herself by touching a doorknob with her bare hands (which she usually avoided) and then to carry out her usual ritual to make things safe.

You can, of course, use the full range of measures while observing the client, including frequency count and rating scales.

Physiological measures

Many research reports, particularly those involving anxiety, include measures of physiological state, and indeed, it may be the physiological symptoms that are the most upsetting for the client as, for example, in panic disorder. Although there are simple, portable instruments for measuring, for example, heart rate or galvanic skin response, these are rarely used in routine clinical practice. Often, the client’s perception of the physical changes, and their meaning to him, are sufficient indices of change within that response system.

A client feared fainting when anxious, and in order to give him information about the state of his blood pressure (BP), he was asked to focus on his heart rate (HR). This was raised, and the therapist questioned him about the relationship between HR and BP, and then explained to him that fainting results from a decrease in BP.

Thus, the focus was on an indirect measure of a physiological variable and did not demand direct physical recording.

Making the most of the data

Time and energy go into collecting information, so you should ensure that you make good use of it. First, examine it carefully to see what it says about the hypothesis it was designed to test. This may involve collating the data in some way. For example, if the client has produced a series of questionnaires over a number of weeks, graph the results and look for variations. This is shown in Figure 5.8 for a series of Beck Depression Inventory scores from a client being treated for depression.

Figure 5.8    Graph of the BDI scores over treatment

Figure 5.9    Diary of anxiety scores rising with increasingly difficult tasks

However, a series of diaries may be more difficult to summarise, and this may be particularly difficult if, for example, the client’s anxiety scores are not declining as he attempts more and more difficult tasks. Figure 5.9 shows data from a client who was severely claustrophobic. It may be helpful to ask him, in the session, to group tasks by their difficulty level. He could then look at improvements in anxiety for activities at each difficulty level.

As the treatment progresses, the responsibility for collating and interpreting information can increasingly be handed over to your client. You can ask your client to review his own diaries and identify themes or the most important incident to discuss. This helps clients develop the ability to review and prioritise, which is necessary for effective problem-solving.

Problems when using measurements

The client does not appreciate its potential value

It is important to discuss your client’s doubts and, if necessary, to get agreement to do some measurement as an experiment.

The client cannot read or write

You will need ingenuity to find other modalities for the client to record relevant data – for example, using a mobile phone or an MP3. It is helpful to get the client’s advice about how to circumvent the problem, as he will probably have tackled other situations like this.

Poor reliability or validity of a questionnaire

Always check that a questionnaire has data on reliability and validity and that its normative data are relevant for your client.

 

Summary

 

  • Measuring the qualities of the problems presented by the client, and then assessing changes as treatment progresses, are crucial aspects of CBT. It can be an interesting, creative and collaborative part of therapy, as you use your ingenuity to design measures. Although much useful information is gleaned from the clinical interview, it can be helpful to have supplementary information, both at assessment, and during and at the end of treatment. There are a number of reasons for this including:

o    to allow you to assess the effects of intervention

o    to capitalise on the more reliable there-and-then observations of the problem

o    to benefit from any possible therapeutic effects of measurement

o    to allow the client to compare baseline with subsequent measures, so as to more accurately plot progress

o    to reinforce the idea that treatment can be seen as an empirical exercise.

  • It is important to bear in mind the psychometric properties of measures, such as reactivity, reliability, and validity. The latter two can be maximised by following some straightforward rules, which are described in the text.
  • The measures you collect should be integral to subsequent sessions, and it is very important that you always show genuine interest in the data your client produces.
  • Measures differ in their level of complexity, ranging from simple counts, through ratings and diaries, through to questionnaires. Think carefully about what information you want, and be creative with your client in devising relevant measures.
  • Bear in mind that there are other sources of information – relatives, friends, other staff, psychophysiology – in addition to that provided by your client.
  • Make as much use as you can of whatever data your client brings – graph it if that would be helpful, try to relate different aspects to each other, and to whatever hypotheses are thrown up by the formulation.

Learning exercises

Review and reflection:

 

  • There are suggestions in the chapter about how to maximise the validity and reliability of any measures that you devise. Think about those suggestions: are there any that you would have problems with? If so, how could you tackle the difficulty?
  • It is sometimes claimed that the important aspects of the problems that people present cannot be measured. Consider three clients’ problems, and consider what aspects of their problems seem to be difficult to measure. What would you be forfeiting or ignoring if you tried to measure them? What would you gain if you devised a measuring scheme and collected some data?

Taking it forward:

 

  • It can be instructive to carry out some monitoring for yourself, to get an idea of how enlightening and onerous it can be. You could, for example, do a thought record (see Chapter 8) for a day; or you could keep records about the frequency of a behaviour that worries you – snapping at people, scratching your head – and what triggers it on each occasion. Note the challenges that such monitoring presents, and consider how these could be minimised for your clients.
  • Review your clients’ notes and try to identify missed opportunities for data collection and evaluation. Consider how you might now do something about that. Plan just how you will collect more relevant information and how you will share the rationale for doing so.

Further reading

Hayes, S.C., Barlow, D.H, & Nelson-Gray, R.O. (1999). The practitioner: research and accountability in the age of managed care. Boston: Allyn & Bacon.

These authors have consistently argued for therapists to adopt the role of scientist practitioner for the benefit of clients, and this book gives a good resume of this approach.