REINSTATEMENT OF CONDITIONED SUPPRESSION IN MICE

Return of fear after successful exposure therapy calls for a better understand- ing of the mechanisms of relapse. Classical conditioning research provides a useful framework for conceptualising the acquisition, extinction and reappear- ance of fear. The present paper focuses on reinstatement, the return of extinguished conditioned responses due to the experience of one or more uncondi- tioned stimuli (USs) after extinction. This phenomenon illustrates that unpredictable USs can lead to a return of fear after successful exposure.The data we present is one of the first demonstrations that conditioned suppression of instrumental behaviour can be used as an index of classical conditioning in laboratory mice. The procedure proves to be a promising instrument for assessing fear in mice, both in the context of research aimed at unravelling the functional characteristics of learning and memory in healthy mice and in the context of research aimed at unravelling the neurobiological substrate of psy- chiatric disorders, e.g., in studies with transgenic and knockout mice. Using this procedure, we report the first observation of reinstatement of conditioned suppression in this species.


Introduction
Although exposure therapy has been proven to be a successful method for reducing clinical fear, it has become clear that not all patients remain symptom-free in the long term. Sometimes a (partial) return of symptoms is observed, generally referred to as 'Return of Fear' (ROF) (Rachman, 1989). A learning theoretical perspective on fears and phobias can shed light on this ROF phenomenon. According to classical conditioning theory fear can be extinguished by repeatedly confronting the patient with the phobic stimulus (conditioned stimulus or CS) without any aversive consequence (unconditioned stimulus or US) being present. Since exposure therapy is considered to be the clinical analogue of extinction, research on a return of conditioned responses after extinction is of great importance for the understanding of clinical relapse in anxiety disorders.
Whereas extinction used to be conceived of as an 'unlearning' of the previously acquired association (see Donegan, Gluck, & Thompson, 1989), nowadays it is known that extinction leaves the original CS-US association intact (e.g., Rescorla, 1996). The present paper will focus on the 'reinstatement' phenomenon. Reinstatement refers to the return of extinguished conditioned responses due to the experience of one or more unpredictable USpresentations after extinction. This return of conditioned responses to the CS without additional CS-US pairing illustrates that the association between the CS and the US is preserved throughout extinction (e.g., Bouton & Bolles, 1979;Bouton, 1984). With regard to clinical practice, this implicates that the association between the phobic stimulus (CS) and the aversive consequence (US) is not abolished during exposure and that conditioned responding to the CS can thus reappear due to a reinstating experience, resulting in (partial) relapse.
A reinstatement experiment in a Pavlovian fear conditioning paradigm typically consists of four phases. During the first phase (the acquisition phase), a neutral stimulus (CS) is repeatedly paired with an aversive US which results in the CS becoming a fear eliciting signal for the US. In the next phase (the extinction phase), the CS is presented alone and a decrease in conditioned responding is observed. Then, the reinstatement manipulation is carried out: the reinstatement group receives one or more US-only trials while the control group does not (reinstatement phase). Finally, in the test phase the CS is presented again and a return of fear is typically observed in the reinstatement group.
Despite its clinical relevance, research on reinstatement in humans is scarce and mostly limited to findings in human contingency learning (e.g., García-Gutiérrez & Rosas, 2003;Vila & Rosas, 2001). However, in human contingency learning research, participants typically learn about relations between stimuli that are devoid of biological relevance (that is, stimuli that are fear-irrelevant; for a review, see De Houwer & Beckers, 2002). Specifically, they learn relationships between cues (e.g., medicines) and outcomes (e.g., side-effects) by observing contingencies in a computer task and are asked to judge the probability that a certain cue will be followed by a certain outcome. On the other hand, in fear conditioning studies, the participant directly experiences the aversive outcomes (USs) leading to a fear of the cue (CS) that signals this outcome. Experiments on human contingency learning have contributed to a growing expertise on the conditions under which reinstatement appears and to the refinement of theoretical models of this phenomenon. However, it remains questionable whether the mechanisms under-lying reinstatement in these studies are generalisable to the study of human fear. In order to investigate whether reinstatement can be one of the mechanisms through which fears reappear after successful exposure, a series of studies was started in our laboratory, assessing reinstatement using a differential fear conditioning preparation. With this paradigm we demonstrated reinstatement of fear in humans in direct verbal measures (Hermans, Dirikx, Vansteenwegen, Baeyens, Van den Bergh, & Eelen, 2005;Dirikx, Hermans, Baeyens, & Eelen, in press) as well as in an indirect measure of conditioning (Dirikx, Hermans, Vansteenwegen, Baeyens, & Eelen, 2004). Van Damme, Crombez, Hermans, Koster, and Eccleston (in press) replicated the latter results by showing reinstatement of fear in an emotional modification of a spatial cueing paradigm. The whole of these findings strengthens the view that a reinstatement experience might lead to a return of fear in humans.
Our reinstatement studies in humans are inspired by animal models of return of fear. Although reinstatement findings in humans are restricted to the aforementioned studies, the phenomenon has been investigated extensively in rats, and has been proven to be fairly robust. Reinstatement has been observed in classical fear conditioning (e.g., Bouton & Bolles, 1979;Rescorla & Heth, 1975;Westbrook, Iordanova, McNally, Richardson, & Harris, 2002), appetitive conditioning (e.g., Bouton & Peck, 1989) and taste aversion learning (e.g., Schachtman, Brown, & Miller, 1985), as well as in instrumental learning (e.g., Baker, Steinwald, & Bouton, 1991). A dependent measure of conditioned fear responding that is often used in these studies is conditioned suppression. In a conditioned suppression paradigm first a stable operant behaviour is trained. For example, the animal learns to press a lever to obtain food pellets. Next, Pavlovian fear conditioning (as described above) is superimposed on this behaviour. The experience of a US (typically a foot shock) leads to suppression of the lever pressing behaviour because the animal reacts to the shock with a response that is incompatbile with pressing the lever (e.g., jumping). When the animal has learned that the CS signals the US, lever pressing will be diminished during the CS as well because the animal will demonstrate a freezing response (conditioned reaction or CR) during the CS. The degree of suppression of operant responding during the CS, as compared to responding during a pre-CS period of the same duration, is taken as a measure of conditioning.
The study we present here, assesses reinstatement of conditioned suppression of instrumental behaviour in mice. Apart from the fact that another behaviour was trained, i.e., nose poking, the paradigm that was used was very similar to the procedure typically used in rats. Nose poking was preferred because it is more easily trained in mice than lever pressing (Crawley, 2000). Similar to this procedure, some authors have used conditioned suppression of drinking as an index of fear conditioning in mice (e.g., Pang, Turndorf, & Quartermain, 1996;Quartermain, Clemente, & Shemer, 1993). In search of the neurobiological substrate of psychiatric disorders, several studies reporting behavioural tests in transgenic and knockout mice (e.g., Sanders, Kieffer, & Fanselow, 2005;Van Dam et al., 2000) have been published. Research on these mice can provide us with important information on the neurobiological basis of fear acquisition, extinction and return. In this context, the conditioned suppression paradigm can be a valuable instrument. For example, Callaerts-Vegh et al. (in press) report slower extinction of conditioned suppression of instrumental behaviour in mGluR7 knockout mice. These mice lack glutamate receptor subtype 7 which is possibly implicated in anxiety disorders and major depression (Cryan et al., 2003). Recently, several studies have been published that investigate acquisition and extinction of both context conditioning and cue conditioning in mice (e.g., Bardgett, Schultheis, McGill, Richmond, & Wagge, 2005;Radulovic, Kammermeier, & Spiess, 1998;Van Dam et al., 2000). Waddell, Dunnet, and Falls (2004) investigated renewal of extinguished fear potentiated startle in mice. However, reinstatement of fear in mice has not been demonstrated before. The present study therefore aimed at developing a mice model of conditioned suppression of instrumental behaviour that can contribute to further research on the genetics and neurobiology of reinstatement and return of fear.

Subjects
Fourteen (9 males) C57BL inbred mice were used in the experiment. All animals were kept under standard laboratory conditions with a 12h/12h darklight schedule, constant room temperature and humidity, standard lab chow and water ad libitum. The animals were between 109 and 154 days old at the start of the shaping. They were food-deprived and kept on 80% of their initial body weight.

Apparatus and stimuli
Four operant chambers (Coulbourn Instruments, Allentown, PA) were used for shaping and classical conditioning. The conditioned stimulus was a 20-s tone (4000 Hz). A 200 ms foot shock (0.2 mA) was used as a US. All stimulus presentations and response registrations were programmed with Graphic State 3.0 software (Coulbourn Instruments, Allentown, PA).

Procedure
The experiment consisted of 5 phases: shaping of instrumental responding, and acquisition, extinction, reinstatement and test of conditioned emotional responding. The first 3 phases where identical for all animals. After extinction half of the animals were randomly assigned to the reinstatement group and the other half to the control group. In total the experiment took 45 days. One training or test session was scheduled per day and every session took approximately 30 minutes.

Shaping
In an operant shaping procedure, all mice were gradually trained to use a nose poke device to obtain food pellets (Noyes Precision Pellets PJPPP-0020, Research Diets Inc., New Brunswick, New Jersey, USA). During the first 5 sessions a CRF schedule (Continuous Reinforcement) was applied. Every single nose poke was reinforced during these 5 sessions. Next 3 FR5 (Fixed Ratio, 5 nose pokes) and 3 FR10 sessions were programmed. During these sessions the animals were reinforced with a food pellet every 5 and 10 nose pokes, respectively. Shaping ended with 6 sessions of VI 30 s training (Variable Interval with a mean of 30 s) during which a nose poke was reinforced after mean intervals of 30 s in order to further stabilise response rates.

Acquisition
During acquisition a Pavlovian fear conditioning procedure was superimposed on the VI 30 s reinforcement schedule. The 20-s tone (CS) co-terminated with a 200-ms foot shock. Each of the 8 acquisition sessions contained 8 CS-US presentations with a mean intertrial interval (ITI) of 3 minutes, ranging from 2 to 4 minutes.

Extinction
The tone CSs were no longer followed by the US during the extinction phase. Otherwise presentation parameters were the same as during acquisition. In total there were 17 extinction sessions. Due to a procedural error, a compound training session was administered between extinction sessions 9 and 10. Except for 1 animal that did not get any training, all the mice received 4 30-s tone (4500Hz) CSs in compound with a house light flashing at rate of 1 Hz (0.5 s on, 0.5 s off). This compound ended in a .5 s foot shock of .2 mA. This session resulted in a re-emergence of conditioned suppression, which was further extinguished in sessions 10 to 17.

Reinstatement
There was only one reinstatement session, during which the reinstatement group received 4 USs with a mean ITI of 32.5 s. In the control group no USs were presented.

Test
The test session took place 24 hours after the reinstatement treatment. The CS was tested 8 times under conditions of extinction.

Scoring
To assess the amount of conditioned responding a suppression ratio (A/A+B) was calculated with A and B representing response rates during the CS and in absence of the CS, respectively. A suppression ratio of .50 indicates the absence of suppression while complete suppression of responding during the CS is represented by a ratio of 0. One mouse was not included in the analyses concerning the reinstatement effect because it was impossible to calculate the suppression ratio for this animal due to a complete absence of responding.

Acquisition and extinction
To test for acquisition the suppression ratios were analysed by means of a 2 (group: reinstatement, control) x 2 (moment: first acquisition trial, last acquisition trial) ANOVA. The main effect of moment was highly significant, F(1, 12) = 16.23, MSE = .02, p < .005, indicating an increase in conditioned suppression from the first to the last acquisition session, with mean suppression ratios of 0.23 and 0.005, respectively. The Group x Moment interaction was not significant, F < 1.
A subsequent 2 (group: reinstatement, control) x 2 (moment: last acquisition trial, last extinction trial) ANOVA to test for extinction showed that this conditioned suppression was successfully extinguished. The significant main effect of moment, F(1, 12) = 125.50, MSE = .01, p < .001, showed that there was a strong decrease in suppression from the last acquisition trial (M = .005) to the last extinction trial (M = .41) (see Figure 1). The main effect of group was significant, F(1, 12) = 7.63, MSE = .01, p < .05, with the control group showing less suppression than the reinstatement group. Also the group x moment interaction proved to be significant, F(1, 12) = 5.41, MSE = .01, p < .05, indicating that extinction was stronger in the control group than the reinstatement group. However, both the reinstatement group, F(1, 6) = 33.19, MSE = .01, p < .05 and the control group F(1, 6) = 112.57, MSE = .01, p < .001 showed significant extinction. The mean suppression ratio during the last extinction trial was .41. A t-test for independent samples showed that in the reinstatement group the mean suppression ratio was significantly lower (.32) than in the control group (.50), t(12) = 2.55, p < .05.

Test of Reinstatement
The reinstatement manipulation was expected to lead to a return of conditioned suppression in the reinstatement group while the control group should not show any changes in comparison with responding during the extinction phase. In order to test for reinstatement a 2 (group: reinstatement, control) x 2 (moment: last extinction trial, test trial) ANOVA was carried out. The data of the last extinction session and the final test session are presented in Figure 1. The 2 (group) x 2 (moment) interaction clearly demonstrated reinstatement of conditioned suppression, F(1, 11) = 5.53, MSE = .01, p < .05. A separate ANOVA for the reinstatement group showed a significant effect of Moment in this group, F(1, 5) = 6.83, MSE = .01, p < .05, indicating an increase in suppression from the end of extinction to test, with mean suppression ratios of 0.36 and 0.18 at the end of extinction and during test, respectively. In contrast, and in line with our hypothesis, no change in con- ditioned suppression from the end of extinction to test was observed in the control group, F < 1, with mean suppression ratios of 0.50 and 0.49 at the end of extinction and during test, respectively.
Both the main effects of group, F(1, 11) = 15.08, MSE = .02, p < .005, and of moment, F(1, 11) = 7.19, MSE = .01, p < .05, were significant. Over the two moments, the reinstatement group showed more suppression than the control group. On the test trial the mice demonstrated more suppression than during the last extinction session.

Discussion
The present study successfully demonstrated reinstatement of extinguished fear in mice, using a single-cue aversive conditioning preparation with conditioned suppression of operant behaviour as dependent variable. The observation of this phenomenon in a different species than rats adds to the robustness of the reinstatement effect. We can conclude that the conditioned suppression paradigm provides a valuable model for assessing learning and memory in both normal and knockout or transgenic mice. However, in most fear conditioning studies with mice freezing is used to assess emotional responding (LeDoux, Iwata, Cicchetti, & Reis, 1988). Some authors (e.g., Anagnostaras et al., 2000) have criticised the latter observation method in favour of more objective automated procedures to examine specific aspects of fear conditioning in mice. Like the automated computer systems measuring behaviour through sensors (see Nielsen & Crnic, 2002, for an overview), the conditioned suppression procedure does not require any scoring of video tapes of the animal's behaviour and it is sensitive to fear responses different from freezing (e.g., jumping) because the instrumental behaviour is also suppressed during these other responses.
As described above, our studies on return of fear in humans are based on elaborate animal models. However, there remains an important difference between the human and rodent studies. Fear conditioning in humans has traditionally been investigated using differential paradigms, which, in contrast to animal studies (see below), compare responding to the CS+ with responding to a control stimulus (CS-). This control stimulus is presented during acquisition, extinction and test, but is never followed by the US. The comparison with a CS-is important to control for orienting responses, especially in physiological measures of conditioning, e.g., skin conductance. In animal conditioning studies, a single-cue conditioning paradigm is typically used. Instead of comparing with a control stimulus, responding during the CS+ is compared to responding during the pre-CS period. Because this difference in procedure might explain some recently observed results in humans, it will be necessary to investigate reinstatement in rodents using a differential conditioning procedure. More specifically, in spite of the successful demonstrations of reinstatement in human fear conditioning (Dirikx, Hermans, Vansteenwegen, Baeyens, & Eelen, 2004, in press;Hermans et al., 2005;Van Damme et al., in press), other studies failed to replicate the reinstatement effect (e.g., Dirikx, Hermans, Vansteenwegen, & Eelen, 2003.This was partly due to the return of fear not being selective for the CS+: In these studies, also the control stimulus (CS-) elicited more 'conditioned' responding after reinstatement as compared to the post-extinction assessment.
These findings raise interesting questions on how humans deal with a reinstating event. Possibly, reinstating USs can lead to a ROF that is not limited to the stimulus originally associated with the US. Mahon (1986, personal communication cited by Rachman, 1989) describes a patient with driving phobia who, after a car accident, not only showed an increased fear of driving but of several other objects and situations as well (e.g., fire, crowds). This example shows that phobic fears might generalise after the patient has gone through a feared, stressful situation, similar to the reinstatement experience in our studies. Further research is needed to explore the clinical and theoretical implications of the observed nonselective return of fear in our experiments. Future studies in animals with a differential conditioned suppression paradigm will help to understand the conditions under which selective or nonselective return of fear is observed.