Herrnstein's hyperbolic matching equation and behavioral pharmacology: review and critique

Behavioral pharmacologists have enlisted Herrnstein's (1970) hyperbolic matching equation to understand the behavioral effects of drugs. Herrnstein's hyperbola describes the relation between absolute response rate and reinforcement rate. The equation has two fitted parameters. The parameter k represents the asymptotic response rate, and the parameter r(e) represents the reinforcement rate necessary to obtain half the asymptotic response rate. According to one interpretation of the equation, changes in k should reflect changes in response or motoric variables, and changes in r(e) should reflect changes in reinforcer or motivational variables, or changes in reinforcement from sources extraneous to the instrumental response. We review research that has applied Herrnstein's equation to distinguish the motoric from the motivational effects of drugs, and to identify additional independent variables responsible for drug effects, such as extraneous reinforcement. The validity of inferences about drug effects depends on the consistency of how k and r(e) respond to environmental manipulations: k should change only with response or motoric variables, and r(e) should change with reinforcer or motivational variables and with the rate of extraneous reinforcement. Empirical tests of these predictions, however, have produced inconsistent results. The review suggests that Herrnstein's theory has not fulfilled its promise of identifying the behavioral mechanisms of drug action. Modifications to the equation, known as bias and sensitivity, may explain some of these inconsistent results, and the modified equation may have utility in behavioral pharmacology.     

Time-, schedule-, and reinforcer-dependent effects of pimozide and amphetamine

Rats performed on two multiple random-interval schedules, in which sequences of ascending or descending reinforcement densities were balanced between the schedules and between the two halves of the session. Using a standard reinforcer (10% sucrose pellets), pimozide decreased response rates, while amphetamine increased responding. The effects of both drugs were schedule dependent: larger changes were evident in low response rate, reinforcement-lean components than in high response rate, reinforcement-rich components. Both effects were also time dependent, increasing over the course of the session; this casts serious doubt on the applicability of Herrnstein's matching law for studying agents acting on brain dopamine.Increasing the period of food deprivation increased response rates, while withdrawing food deprivation decreased responding. These effects were also schedule dependent, but were time independent. Substituting 95% sucrose pellets for standard 10% sucrose pellets caused an immediate and sustained decrease in responding, and up to 10% of earned reinforcement was not consumed. Pimozide increased response rates within reinforcement-lean components and reinstated the complete consumption of earned reward typical of standard reinforcement. These apparently paradoxical effects may be consistent with a decrease in the rewarding properties of sucrose pellets. Despite low response rates, amphetamine did not affect responding maintained by 95% sucrose pellets but did further reduce the consumption of earned reward. These results call into question the generality of the rate-dependency principle in the action of psychomotor stimulants.     

Individual differences in behavioral tolerance to amphetamine and the economic context of reinforcement loss

The economic context (i.e. an enriched vs impoverished environment) affects many drug-induced phenomena. The present study examined whether the 'experienced' economic context of operant responding was associated with the degree of tolerance to the behavioral effects of amphetamine. Eight rats lever pressed for food reinforcement under a multiple schedule consisting of several variable-interval schedules (8, 17, 55, 150, and 250 s). Amphetamine was first administered acutely (0.2, 0.8, 1.6, and 3.2 mg/kg), then chronically (dose tailored for each subject) over 30 consecutive sessions. Baseline saline injections were also administered during the acute regimen. Herrnstein's single-alternative matching equation described the rats' response rate data well under all conditions. A parameter in Herrnstein's equation (re), which has been shown to vary with experimentally-arranged contextual reinforcement, was used as the index of the experienced economic context for each subject under baseline conditions. Differences in the value of re predicted individual differences in the degree of tolerance. Under most variable-interval (VI) schedules, and when all schedules were aggregated, less tolerance accrued if the baseline context was experienced as enriched, and more tolerance accrued if the baseline context was experienced as impoverished. In terms of the reinforcement loss hypothesis, the results suggest that tolerance was not determined by reinforcement loss per se, but by how much the animal lost relative to the economic context in which the operant task was embedded.     

Increased impulsive action in rats: effects of morphine in a short and long fixed-delay response inhibition task

Rationale

Impulsive action is mediated through several neurochemical systems, although it is not clear which role each of these plays in the inability to withhold inappropriate responses. Manipulations of the opioid system alter impulsive action in rodents, although the effects are not consistent across tasks. Previously, we speculated that these discrepancies reflect differences in the cognitive mechanisms that control responding in each task.

Objectives

We investigated whether the effect of morphine, a mu opioid receptor (MOR) agonist, on impulsive action depends on the ability of the subjects to time the interval during which they must inhibit a response.

Methods

Male Long–Evans rats were trained in a response inhibition (RI) task to withhold responding for sucrose during a 4- or 60-s delay; impulsive action was assessed as increased responding during the delay. The rats were tested following an injection of morphine (0, 1, 3, 6 mg/kg). In a subsequent experiment, the effects of morphine (6 mg/kg) plus the MOR antagonist naloxone (0, 0.3, 1, 3 mg/kg) were investigated.

Results

Morphine increased impulsive action, but had different effects in the two conditions: the drug increased the proportion of premature responses as the 4-s interval progressed and produced a general increase in responding across the 60-s interval. Naloxone blocked all morphine-induced effects.

Conclusions

The finding that morphine increases impulsive action in a fixed-delay RI task contrasts with our previous evidence which shows no effect in the same task with a variable delay. Thus, MORs disrupt impulsive action only when rats can predict the delay to respond.     

Suppression or facilitation of operant behaviour by raclopride dependent on concentration of sucrose reward

Rats responded under continuous reinforcement for 1%, 10% or 95% sucrose pellets, under food deprived or ad libitum access conditions. In both cases responding was highest for 10% sucrose reinforcement, and a small proportion of 95% sucrose was not consumed. Ad libitum food access reduced response rates for all sucrose concentrations. Responding for 10% and 95% sucrose pellets followed a parallel time-course; and accumulation of 95% sucrose pellets was immediate and nonprogressive. Extinction following availability of 95% sucrose pellets caused an increase in response rate, but removal of 10% sucrose led only to a decline in responding. Under both food deprived and non-deprived conditions, the dopamine D-2 antagonist raclopride dose-dependently decreased responding for 1% or 10% sucrose, but increased responding for, and consumption of, 95% sucrose reward. After eight sessions of responding under extinction conditions, the presentation of reward-associated cues increased response rate early in the session. Raclopride had no effect during this period, but decreased responding later in the session. We consider the implications of these results for theories of neuroleptic drug action.     

Time-dependent and schedule-dependent effects of dopamine receptor blockade

Rats were trained on one of two multiple random-interval schedules of reinforcement, which contained both ascending and descending reinforcement density sequences. The design of the schedules made it possible to determine whether performance changes in a particular component depended on the reinforcement density in that component, or on its temporal position within the schedule. Performance impairments following administration of the dopamine D1 antagonist SCH-23390 were both time- and schedule-dependent: SCH-23390 caused a relatively greater suppression of poorly reinforced responding that increased over time. The results, which are compatible with data on the effects of dopamine antagonists in other behavioural paradigms, illustrate the methodological problems of using Herrnstein's matching equation to interpret time-dependent behavioural changes.     

Effects of d-amphetamine, WIN 35,428, pentobarbital and morphine on schedule-controlled responding in two inbred rat strains that differ in locomotor stimulatory effects of cocaine

Behavioral effects of d-amphetamine, the cocaine analog, WIN 35,428, morphine, and pentobarbital were compared in NBR/NIH and F344CR1BR rats. Either nose-poke or lever-press responding was maintained under 3-min fixed-interval schedules of food presentation. The effects of morphine, WIN 35,428 and pentobarbital depended upon the rat strain studied: morphine increased nose-poke responding in NBR but not F344 rats; significant strain x dose interactions were observed with WIN 35,428; and pentobarbital was more potent in decreasing nose-poke responding in NBR than in F344 rats. No strain differences were observed in the behavioral effects of d-amphetamine. There were also prominent differences in the effects of drugs that were related to the nature of the response requirement. In both rat strains, nose-poke responding was affected differently than lever-press responding by morphine, d-amphetamine, and WIN 35,428. Pentobarbital produced effects that were independent of the specified response topography. A global underlying difference in these rat strains cannot be identified at present to account for the diversity of findings. The behavioral effects of these drugs appear to be influenced by a host of interactive factors including the drug, strain of animal, the baseline response rate and physical dimensions of the response.     

Schedule-controlled behavior in the morphine-dependent rat

The behavioral effects of acute and chronic administration of morphine and its withdrawal were studied using schedule-controlled responsing in the rat under a differential reinforcement of low rates of responding (DRL) schedule of food presentation. Acute morphine administration had a biphasic effect on response rate. Low doses (1.8–5.6 mg/kg) tended to produce a small increase and higher doses (10–30 mg/kg) decreased responding. Physical dependence was produced by twice daily injections, with an initial dose of 40 mg/kg/day which was increased by 80 mg/kg/day until reaching 600 mg/kg/day which was continued for 14 days. Throughout chronic administration the pattern of responding remained disrupted resulting in a 27–47 percent decrease presentations of the reinforcer, while response rate was more variable and generally decreased. The effects of morphine withdrawal lasted 5 days and produced an initial marked decrease in reinforcements per hour and a biphasic change in response rate. A marked decrease in responding early in withdrawal (22.5 hr) was followed by a marked and more prolonged (70.5–118.5 hr) response rate increase.     

Schedule-dependent tolerance to behavioral effects of delta 9-tetrahydrocannabinol when reinforcement frequencies are matched

Squirrel monkeys pressed a lever under a multiple interresponse-time greater than 28-sec, modified random-interval schedule which provided comparable frequencies and temporal distributions of food pellet presentation in the two components. Daily intramuscular administration of either 0.25 or 1.00 mg/kg delta 9-tetrahydrocannabinol resulted initially in suppression and/or disruption of responding and concomitant decreases in the frequency of food presentation in both components. Responding in both components next increased, resulting in recovery of baseline frequencies of pellet delivery during the random-interval component, but continued depression during the interresponse-time schedule. The drug-induced changes in responding under the interresponse-time schedule diminished with repeated injections, whereas response rates during the random-interval schedule sometimes remained elevated. Interresponse-time distributions under the interresponse-time schedule showed that with repeated administration of the drug only those characteristics which had the greatest effect on reinforcement frequency recovered to baseline levels. When drug injections were replaced by daily injections of the vehicle, responding was greatly disrupted only during the random-interval component. These findings are only partially consistent with other results which suggest aht tolerance development to the behavioral effects of delta 9-tetrahydrocannabinol is greatly enhanced if the drug initially produces reinforcement loss.     

Role of ventral tegmental area, periaqueductal gray and parabrachial nucleus in the discriminative stimulus effects of morphine in the rat

Previous studies have produced mixed results about the role of the ventral tegmental area, periaqueductal gray and parabrachial nucleus in morphine discriminations, perhaps owing to the considerably different methodologies used. The purpose of the present study was to compare the roles of these three brain areas using the same food-reinforced discrimination protocol, to determine whether the schedule of reinforcement influenced maximal substitution produced by site-specific morphine administration and to determine whether the time course of substitution differed by site of morphine administration. Rats were trained to discriminate 3.0 mg/kg subcutaneous morphine from saline under variable interval 15-s or fixed ratio 10 schedules of food reinforcement. Rats were then implanted with one cannula aimed at the lateral ventricle (intracerebroventricular) and one aimed at the ventral tegmental area, periaqueductal gray or parabrachial nucleus. Morphine discrimination curves were obtained by subcutaneous, intracerebroventricular and intrasite routes. When administered subcutaneously, morphine was equipotent in variable interval-trained and fixed ratio-trained rats, although it was more potent in fixed ratio-trained females than fixed ratio-trained males. When administered intracerebroventricularly, morphine (0.3-10 microg) engendered a maximum average of 63% drug-appropriate responding in both variable interval-trained and fixed ratio-trained rats; females showed significantly greater drug-appropriate responding than males, again under the fixed ratio but not under the variable interval schedule. In variable interval-trained rats, intrasite infusions of morphine (0.3-10 microg) produced maximal drug-appropriate responding of approximately 57% (ventral tegmental area), 56% (periaqueductal gray) and 41% (parabrachial nucleus); mean maximal substitution was slightly (< or = 17%) greater in fixed ratio-trained rats. When injected into the ventral tegmental area or periaqueductal gray, but not the parabrachial nucleus, naloxone methiodide (2 microg) significantly decreased drug-appropriate responding following 3.0 mg/kg subcutaneous morphine, in both variable interval-trained and fixed ratio-trained rats. The time course of the discriminative stimulus effects of morphine differed among the three brain sites: intraventral tegmental area morphine produced peak drug-appropriate responding by 15 min after injection, whereas the discriminative stimulus effects of intraperiaqueductal gray and intraparabrachial nucleus morphine peaked at approximately 60 min after injection. Taken together, these results indicate that ventral tegmental area, periaqueductal gray and parabrachial nucleus each play a role in the ability of morphine to function as a discriminative stimulus, regardless of the sex of the subject or the schedule under which the subjects are responding. Ventral tegmental area and periaqueductal gray, however, appear to be more critical than parabrachial nucleus in mediating the discriminative effects of systemic morphine in rats responding under a food reinforcement procedure. The pretreatment time and, to a lesser extent, the schedule of reinforcement are additional variables that should be considered when comparing the relative roles of different brain areas in drug discrimination.     

Effects of morphine on fixed interval-induced escape from food reinforcement

Pigeons were trained to respond on a fixed-interval 3-min (FI 3 min) schedule of food reinforcement. During each 1-hr session responses on a second response key produce a timeout from the FI schedule. mThe food reinforcement schedule maintained characteristic FI responding. The escape-timeout responses occurred during the period of low response rates following reinforcement. Morphine (0.25, 0.5, 1.0 mg/kg) produced dose-related rate-dependent decreases in mean fixed-interval response rates. As morphine dose increased, there was an increased tendency for the rates within the interval to converge from a variety of predrug rates to a common, low rate of responding. The effect of morphine on the schedule-induced escape responses was to increase the mean number of escapes at doses of 0.25–0.5 mg/kg and to decrease the escapes at the highest dose of 1.0 mg/kg. The mean duration of the escape-timeouts was increased by the 3 doses of the drug, with the longest durations occurring at the 0.5 mg/kg dose. There was a great deal of variability between subjects on these measures and the dose effects did not reach statistical significance. The present study extends the analysis of drug effects on schedule-induced escape to fixed-interval-induced escape responding and includes the drug morphine.     

Comparison of naltrexone and quaternary naltrexone after systemic and intracerebroventricular administration in pigeons

The behavioral effects of naltrexone and quaternary naltrexone were evaluated in two groups of pigeons. One group responded under a fixed-ratio schedule of food reinforcement and was trained to discriminate between 3.2 mg/kg morphine (i.m.) and saline. Drug-appropriate responding occurred in a dose-related manner to morphine, given intramuscularly and intraventricularly. When administered intraventricularly morphine was 50 times more potent as a discriminative stimulus and 50-100 times more potent at suppressing responding. Naltrexone, given intraventricularly and intramuscularly, attenuated the discriminative stimulus effects of morphine. Quaternary naltrexone was more potent at suppressing responding when administered intraventricularly but it failed to attenuate the discriminative stimulus effects of morphine. A second group of pigeons, responding under a variable-interval schedule of food reinforcement, was treated with 100.0 mg/kg/day of morphine. Naltrexone and quaternary naltrexone suppressed responding by both routes of administration. Naltrexone was approximately equipotent when given intramuscularly or intraventricularly, and the doses that suppressed responding were 50-500 times smaller than doses required to suppress responding in untreated pigeons. Although quaternary naltrexone was 1800 times more potent when given intraventricularly, the doses necessary to suppress responding by each route were the same as doses required in untreated pigeons. These results extend the conditions under which a quaternary derivative of naltrexone failed to display antagonist activity in the pigeon. The utility of this compound for characterizing central and peripheral mechanisms of action has not been established in different species and testing conditions and, therefore, appears to be appropriate only under conditions in which it is evaluated after systemic, as well as central, administration.     

Suppressed ethanol intake by CER following the sucrose-fading initiation procedure

Lever responding maintained with sweetened ethanol reinforcement (3% sucrose in 10% ethanol) was initiated in food- and water-sated rats with the sucrose-fading procedure. Four tone-shock pairings, one per session, were superimposed on this behavioral baseline [conditioned emotional response (CER) paradigm]. A profound and sustained ethanol response suppression was found. Baseline levels of ethanol responding were recovered by repeating the original initiation procedure. Subsequent exposure to the CS tone alone (no shock) led to a nonsignificant reduction in ethanol responding. These results were discussed in terms of anxiolytic action of ethanol.     

Behavioral effects of cocaine and its interaction with d-amphetamine and morphine in rats

Drugs of abuse are commonly co-abused, and frequently these combinations produce effects which cannot be predicted by studying the effects of the individual drugs. To investigate the behavioral interactions which occur following combinations of cocaine plus amphetamine or cocaine plus morphine, rats were trained to respond under a differential reinforcement of low rates (DRL) schedule (10-14 sec). Cocaine (0.1-10 mg/kg) and d-amphetamine (0.1-3 mg/kg) decreased the percentage of reinforced responses (efficiency) at doses which had no effect on overall rate of responding. Following moderate doses of either drug, the interresponse time (IRT) distribution showed an increase in the percentage of shorter (less than 10 sec) IRT's. Morphine (0.1-10 mg/kg) also decreased efficiency, but the decrease which occurred was only observed at doses which also decreased overall response rates. As might be expected, the IRT distribution for morphine showed a dose-related increase in the percentage of long IRT's (greater than 14 sec). When doses of morphine which had no significant effect when administered alone (1 or 3 mg/kg) were combined with cocaine, the cocaine dose-response curve for efficiency was shifted down and to the left and response rates were increased. Analysis of the IRT distribution showed that the combination of an ineffective dose of cocaine, 1 mg/kg, plus 3 mg/kg morphine produced a shift in the IRT distribution to the left (an increase in the percentage of short IRT's). When cocaine was combined with 0.3 mg/kg d-amphetamine, a dose which had no effect when given alone, no significant interactions were observed on efficiency or overall rate of responding.(ABSTRACT TRUNCATED AT 250 WORDS)     

Evidence for an involvement of 5-hydroxytryptaminergic neurones in the maintenance of operant behaviour by positive reinforcement

The possible involvement of the ascending 5-hydroxytryptaminergic (5HTergic) pathways in the maintenance of operant behaviour by positive reinforcement was examined using a quantitative paradigm based on Herrnstein's (1970) equation which defines a hyperbolic relationship between steady-state response rate and reinforcement frequency in variable-interval schedules. Nine rats received injections of 5,7-dihydroxytryptamine into the dorsal and median raphe nuclei; 12 rats received sham injections. The rats were trained to steady-state in a series of variable-interval schedules of sucrose reinforcement affording a range of reinforcement frequencies. Herrnstein's equation was fitted to the data obtained from each rat and to the averaged data obtained from the two groups. The value of K_H (the parameter expressing the reinforcement frequency needed to obtain the half-maximum response rate) was significantly lower in the lesioned group than in the control group; the values of R_max (the parameter expressing the maximum response rate) did not differ significantly between the two groups. The levels of 5HT and 5-hydroxyindoleacetic acid in the parietal cortex, hippocampus, nucleus accumbens and hypothalamus were markedly reduced in all four regions in the lesioned group, but the levels of noradrenaline and dopamine were not significantly affected. The results indicate that damage to the central 5HTergic pathways resulted in an increase in the “value” of the sucrose reinforcer, without affecting the animals' response capacity. The results are consistent with the suggestion that the 5HTergic pathways may exert some limiting control on the “values” of certain reinforcers.     

A comparison of the stimulus effects of morphine and lysergic acid diethylamide (LSD)

Morphine and lysergic acid diethylamide (LSD) each was used as a discriminative stimulus for rats. After the injection of drug (morphine or LSD), depression of one lever of an operant test chamber resulted in positive reinforcement according to a variable interval schedule of 15 sec (VI-15 sec). When saline was given, responses on the opposite lever were reinforced. Discriminated responding occurred when either morphine or LSD served as the discriminative stimulus. When animals which were trained to d discriminate morphine from saline were given LSD, they pressed predominantly the saline-correct lever. Similarly, LSD discrimination did not generalize to morphine. Two 5-hydroxytryptamine (5-HT) antagonists, cyproheptadine and methysergide, and one acetylcholine (Ach) antagonist, atropine, did not effect morphine or LSD discrimination. The narcotic antagonist, naloxone, blocked the stimulus effect of morphine, but did not alter LSD discrimination. These results indicate that the morphine and LSD stimuli are dissimilar and that the integrity of 5-HT or Ach nervous systems is not essential for morphine or LSD to serve as a discriminative stimulus.     

Some effects of d-amphetamine, caffeine, nicotine and cocaine on schedule-controlled responding of the mouse

The effects of d-amphetamine (0.03-10.0 mg/kg), caffeine (0.3-100.0 mg/kg), nicotine (0.003-10.0 mg/kg) and cocaine (0.03-56.0 mg/kg) were compared on responding maintained under three different schedules of food presentation in mice. Cumulative doses of d-amphetamine, nicotine and cocaine only decreased responding maintained under fixed-ratio 30 response, fixed-interval 60-sec and fixed-interval 60-sec schedules with a punishment contingency (suppressed responding). In most cases there was an inverse relationship between the ED50 (dose which decreased responding by 50%) for the drug and the rate of responding maintained under each schedule. The exceptions were, with both d-amphetamine and cocaine the ED50 for suppressed responding was smaller than that for non-suppressed fixed-interval responding, and with nicotine the ED50 for fixed-ratio responding was smaller than that for fixed-interval responding. In contrast, intermediate doses of caffeine increased suppressed responding, had little effect on fixed-interval responding and decreased fixed-ratio responding. This difference in profile of effect over the range of conditions studied, suggests that the behavioral effects of psychomotor stimulants can be used to examine potential differences in the mechanisms of action of each drug. Such findings may aid in the understanding of the relationships between the neuropharmacological and behavioral effects of psychomotor stimulant drugs.     

Escalation of food-maintained responding and sensitivity to the locomotor stimulant effects of cocaine in mice

Escalation of drug self-administration is a consequence of extended drug access and is thought to be specifically related to addiction, but few studies have investigated whether intake of non-drug reinforcers may also escalate with extended-access. The goal of these studies was to determine the effects of limited and extended-access to food reinforcers on behavioral and pharmacological endpoints in mice. In distinct groups, responding on a lever was maintained by liquid reinforcement, or nose-poke responses were maintained by sucrose pellets or liquid food in sessions lasting 1 h (limited-access) or 10 h (extended-access). The reinforcing strength of each food, as well as reinforcer-associated cues, was tested before and after extended-access using a progressive ratio (PR) schedule, and locomotor activity in response to novelty and increasing doses of cocaine was assessed in an open field setting in all animals after access to food reinforcers. Escalation of lever-pressing behavior reinforced by liquid food, but not nose-poke behavior reinforced by liquid food or sucrose pellets, was observed across successive extended-access sessions. A concomitant increase in the reinforcing strength of liquid food and its associated cues was apparent in mice that escalated their responding, but not in mice that did not escalate. Finally, extended reinforcer access leading to behavioral escalation was accompanied by an increased sensitivity to the psychostimulant effects of cocaine compared to limited-access. These findings indicate that behavioral escalation can develop as a consequence of extended-access to a non-drug reinforcer, although both the nature of the reinforcer (liquid versus solid food) and the topography of the operant response (lever versus nose-poke) modulate its development. These data also suggest that some of the behavioral and pharmacological corrolaries of behavioral escalation observed following extended-access to drug self-administration may not be due to drug exposure, but rather, may result from basic behavioral processes which underlie operant responding maintained by appetitive stimuli.     

Behavioral effects of profadol in the rat

The effects of profadol, an analgesic with mixed agonist and antagonist properties, were evaluated on continuous avoidance responding and locomotor activity in the rat. Profadol was tested alone and concomitantly with 8.0 mg/kg of naloxone. Profadol had a biphasic effect on avoidance response rate, increasing it at from 0.5–8.0 mg/kg and decreasing it at 32 mg/kg. Naloxone blocked both the rate increasing and the rate decreasing effects of profadol on avoidance responding. Locomotor activity was unaffected by 0.5–64 mg/kg of profadol alone, but was increased when profadol and naloxone were administered together. These findings extend the dual action hypothesis for morphine to a partial morphine agonist. This study provides further evidence that the behavioral activity of narcotic antagonists can be evaluated in the rat in an objective and quantitative manner.Publication No. 1171 of the Division of Basic Health Sciences of Emory University. This investigation was supported by USPHS Grant MH 21699.