Reward Maximization Assessed Using a Sequential Patch Depletion Task in a Large Sample of Heterogeneous Stock Rats

doi:10.21203/rs.3.rs-2525080/v1

Download PDF

Article

Reward Maximization Assessed Using a Sequential Patch Depletion Task in a Large Sample of Heterogeneous Stock Rats

https://doi.org/10.21203/rs.3.rs-2525080/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 29 Apr, 2023

Read the published version in Scientific Reports →

You are reading this latest preprint version

Choice behavior requires animals to evaluate both short- and long-term advantages and disadvantages of all potential alternatives. Impulsive choice is traditionally measured in laboratory tasks by utilizing delay discounting (DD), a paradigm that offers a choice between a smaller immediate reward, or a larger more delayed reward. This study tested a large sample of Heterogeneous Stock (HS) male (n = 896) and female (n = 898) rats, part of a larger genetic study, to investigate whether measures of reward maximization overlapped with traditional models of delay discounting via the patch depletion model using a Sequential Patch Depletion procedure. In this task, rats were offered a concurrent choice between two water “patches” and could elect to “stay” in the current patch or “leave” for an alternative patch. Staying in the current patch resulted in decreasing subsequent reward magnitudes, whereas the choice to leave a patch was followed by a delay and a resetting to the maximum reward magnitude. Based on the delay in a given session, different visit durations were necessary to obtain the maximum number of rewards. Visit duration may be analogous to an indifference point in traditional DD tasks. While differences in traditional DD measures (e.g., delay gradient) have been detected between males and females, these effects were small and inconsistent. However, when examining measures of reward maximization, females made fewer patch changes at all delays and spent more time in the patch before leaving for the alternative patch compared to males. This pattern of choice resulted in males having a higher rate of reinforcement than females. Consistent with this, there was some evidence that females deviated from the optimal more, leading to less reward. Measures of reward maximization were only weakly associated with traditional DD measures and may represent distinctive underlying processes. Taken together, females performance differed from males with regard to reward maximization that were not observed utilizing traditional measures of DD, suggesting that the patch depletion model was more sensitive to modest sex differences when compared to traditional DD measures in a large sample of HS rats.

Biological sciences/Neuroscience/Motivation

Biological sciences/Neuroscience/Reward

delay discounting

rats

choice

optimization

foraging

All animals face a simple, yet essential, choice between staying or leaving a resource in search of potentially better alternatives. Models of foraging behavior from behavioral ecology seek to understand choice behavior by assuming individuals maximize their net rate of energy gain when behaving optimally^1–3. Patch leaving models specifically examine decisions about how much time a forager will devote to a dwindling resource (depleting patch) before departing to locate the next such resource. The Marginal Value Theorem (MVT)³ predicts that animals behaving optimally will leave when the rate of reward in the current patch equals the average rate of reward in the entire habitat. MVT predictions are based primarily on travel time between patches, which delays the availability of food in these alternative patches, but there are potentially other negative consequences associated with leaving the depleting patch that impact choice and postpone leaving. For example, animals risk predation, and the effort costs of traveling to an alternative patch may not vary linearly with travel time⁴. Foraging theorists have also suggested that food in alternative patches may be discounted because of the uncertainty about its availability and accessibility^5,6. These considerations would suggest that foragers may stay longer than predicted by MVT in the depleting patch. Indeed, in many studies, animals exhibit this “overharvesting” behavior, operationally defined as a preference for smaller, more immediate rewards in a current patch relative to the potentially larger rewards available after a delay of traveling to a new patch. Conceptualized as such, overharvesting has been proposed to reflect “impulsive choice” in an intertemporal choice paradigm^7–14.

In laboratory studies of behavioral economics, impulsive choice has also been studied using Delay Discounting (DD) tasks, in which individuals choose between smaller, sooner and larger, later rewards. A discounting function can be constructed to describe how choices change as a consequence of variations in the duration of the delay to the larger reward. The gradient of the function or area-under-the-curve function provides a metric of the extent to which delays affect choices. Performance on traditional DD tasks is both genetically¹⁵ and behaviorally associated with various aspects of the drug abuse continuum in humans^16–23, as well as numerous other psychopathologies including attention deficit hyperactivity disorder²⁴. Indeed, several researchers have suggested that the excessive devaluation of delayed rewards, as assessed by DD tasks, is a transdiagnostic feature of psychiatric disorders²⁵.

Choice behavior in both patch foraging and DD tasks may tap similar reward valuation processes that drive individuals to maximize reward while also devaluating rewards that are not currently present (delayed rewards in alternative food patches). However, few studies have simultaneously measured both reward maximization and DD processes in a single task. Accordingly, we examined choice behavior in a large sample of males and females, using the sequential patch depletion procedure²⁶, to evaluate the extent that behavior on the sequential patch depletion task maximized the net rate of reinforcer gain, as derived from the MVT. In addition, we also investigated the degree to which behavior could be described using traditional DD metrics (gradient of the DD function, area under the discounting curve). Male and female outbred Heterogeneous Stock (HS) rats, known for their high genotypic and phenotypic variability^27,28, chose between a smaller, sooner reward by staying in a rapidly depleting patch and a delayed but larger reward by investing the time needed to change to a non-depleted patch. The large sample size provides the power to identify subtle effects and to the best of our knowledge, will provide an in-depth analysis of behavior in this the first investigation of performance in HS rats in a study of patch leaving.

Delay Discounting

Performance in the Sequential Patch Procedure was first analyzed using a traditional DD approach, which focused on the rejection volumes at the different change over delay (COD) travel times. A two-way between-subject ANOVA with sex as the between-subject factor and delay (0, 6, 12, 18, and 24 s) as the within-subject factor indicated a significant main effect of delay [F(4,7168) = 4,823.325, p < .001] and a significant interaction between delay and sex [F(4,7168) = 11.102, p < .0011; effect sizes: delay, = 0.729; delay x sex, = 0.006] on rejection volume/indifference point. Post-hoc pairwise comparison LSD tests showed that the indifference points were significantly different for each delay, such that indifference points decreased as delay increased. Independent samples t tests with Bonferroni corrections revealed that males had significantly higher rejection volumes than females at 6- and 12-s delays [6 s, t(1792) = 2.632, p < 0.01; 12 s, t(1792) = 3.287, p < .001) (Fig. 3a). These data suggest that males are discounting less than females when the travel costs (CODs) were 6- and 12-s delays, however the effect sizes were small and differences did not occur at each delay. The lack of difference at 0 s indicates that males and females do not differ on the bias parameter of the hyperbolic equation (b; Fig. 3b).

The small sex difference was underscored when the hyperbolic discounting function was fit to the data. The differences at the two delays were insufficiently large to be reflected in sex differences in the hyperbolic discounting function gradient (k; Fig. 3c) or AUC (Fig. 3d).

Patch Utilization

The small differences between males and females in patch leaving at 6- and 12-s CODs were accompanied by more widespread differences in resource acquisition. A two-way between-subject ANOVA with sex as the between-subject factor and delay (0, 6, 12, 18, and 24 s) as the within-subject factor was conducted to determine if sex moderated the rate of water reinforcement (µL/min). There was a significant interaction between delay and sex [F(4,7168) = 186.616, p < 0.000] and significant main effects of delay [F(4,7168) = 6612.521, p < 0.000] and sex [F(1,1792) = 170.042, p < 0.000]. Effect sizes were as follows: delay x sex, = 0.094; delay, = 0.787; sex, = 0.087. Independent samples t-test with Bonferroni correction (p = 0.01) identified males had a significantly greater rate of water reinforcement at delays of 0–18 s [D0: t(1792) = 8.450, p < 0.001; D6: t(1792) = 5.802, p < 0.001; D12: t(1792) = 5.119, p < 0.001; D18: t(1792) = 2.659, p = 0.001] compared to females (Fig. 4a).

These effects were driven by differences in several associated variables. A two-way between-subject ANOVA examining the number of patch changes/leave choices identified a significant delay x sex interaction [F(4,7168) = 3.805, p < .01] and significant main effects of delay [F(4,7168) = 5387.924, p < .001] and sex [F(1,1792) = 15.933, p < .001]. Post-hoc pairwise comparison LSD tests showed that patch changes were significantly different for each delay, indicating that the number of times the rats changed patches decreased as the delay increased. Independent samples t tests with Bonferroni corrections showed that females exhibited significantly fewer patch changes (leave choices) than males at all delays tested [0 s, t(1792) = 3.293, p < .001; 6 s, t(1792) = 3.366, p < .001; 12 s, t(1792) = 4.082, p < .001; 18 s, t(1792) = 2.869, p = .004; 24 s, t(1792) = 2.840, p = .005] (Fig. 4b). The difference in number of patch changes was accompanied by small sex differences in the mean amount of time spent in patches; a two-way between-subject ANOVA revealed a significant interaction between delay and sex [F(4,7168) = 5.710, p < .001] and significant main effects of delay [F(4,7168) = 6456.707, p < .001] and sex [F(1,1792) = 15.285, p < .001]. Post-hoc pairwise comparison LSD tests showed that the amount of time in patch was significantly different for each delay, indicating that rats increased the amount of time spent in the patch as the delay increased. Independent samples t tests with Bonferroni corrections revealed that females spent significantly more time in the patch than males before leaving for a new patch at 6- and 12-s delays [6 s, t(1792) = − 4.312, p < .001; 12 s, t(1792) = − 5.375, p < .001] (Fig. 4c). Taken together, these data indicate sexually dimorphic performance on the characteristics of patch visits on the sequential patch depletion task, with males having a higher rate of water reinforcement, while females made fewer patch changes and remained in a patch for longer periods of time resulting in significantly lower rejection volumes/indifference points at some delays.

To assess whether these patch utilization patterns generally reflected a reward maximization strategy (according to the MVT), we assessed differences in deviations from the optimal time in patch; a two-way between-subject ANOVA identified a significant interaction between delay and sex [F(4,7168) = 4.83, p < .001] and significant main effects of delay [F(4,7168) = 3478.69, p < .001] and sex [F(1,7192) = 13.293, p < .001]. As can be seen in Fig. 5a, rats spent less time in the patch than predicted by our reward maximization calculation when the CODs were short, but this deviation declined systematically until the time in patch approximated optimal predictions (COD of 12 s) and then exceeded predictions toward underharvesting (leaving the patch sooner than optimal). Post-hoc independent samples t tests with Bonferroni corrections revealed that the deviation from the optimal time in patch was more extreme for females than males at delays of 0–18 s [0 s, t(1792) = 2.485, p < .01; 6 s, t(1792) = 4.312, p < .001 6 s, t(1792) = 4.312, p < .001; 12 s, t(1792) = 5.375, p < .001; 18 s, t(1792) = 2.74, p = .015] (Fig. 5a). These data are consistent with females having a lower rate of reinforcement than males.

To further evaluate reward maximization, the observed rejection volume was compared to the optimal rejection volume (predicted by the MVT); a two-way between-subject ANOVA identified a significant interaction between delay and sex [F(4,7168) = 10.109, p < .001] and a significant a main effect of delay [F(4,7168) = 1428.742, p < .001]. Females deviated from the optimal rejection volume significantly more than males at delays of 6 and 12 s [6 s, t(1792) = − 2.672, p < .001; 12 s, t(1792) = − 3.287, p = .005], whereas males deviated significantly more than females at 24 s [t(1792) = 2.032, p < .01] (Fig. 5b). Relative to MVT predictions, both sexes were exhibiting overharvesting (leaving the patch after it would be considered optimal) at 0- and 6-s delays, but at longer delays (12–24 s), there was a shift toward optimal. This altered patch leaving as a function of COD is clearly illustrated by changes in the frequency distributions of rejection volumes across delays; the percentages of animals overharvesting decreased as delay increased (Fig. 6). These data indicate that rats’ behavioral choices deviate from the values predicted by the MVT.

Strengths Of Associations Between Patch Utilization, Reward Maximization, And Dd

Table 1 shows the correlations amongst traditional DD composite variables (k and AUC), patch utilization composite variables (patch changes, time in patch, and rate of water reinforcement), and reward maximization composite variables (time and volume deviations) in male and female rats. For both sexes, there were strong significant negative associations between k and AUC values (p values < .001; Table 1). DD composite variables significantly correlated with the composite scores for patch changes for males and females (p values < 0.05). However, while significant associations were observed between patch utilization composite variables (time in patch, patch changes) and DD composite variables (k and AUC; all p values < 0.05), these associations appeared weaker than the associations observed between patch utilization variables and reward maximization variables (time and volume deviation; all p values < 0.05). Weak to no associations were observed for both DD and reward maximization and rate of water reinforcement (r = 0.3 < 0.03, see Table 1).

To compare the strength of the associations between DD variables and reward maximization for each of the patch utilization variables, we applied the method described by Meng et al.²⁹, which compares sets of non-independent overlapping correlation coefficients via a z-test procedure. For most comparisons, there were differences in the strength of associations (see Table 2 for z-scores). Correlations between patch utilization composite scores and reward maximization were significantly stronger than correlations between patch utilization and DD variables in both males and females (Tables 1 and 2). For example, correlation coefficients between number of patch changes and k (males: r = − 0.30; females: r = − 0.35) were significantly weaker than correlation coefficients observed between number of patch changes and time deviation (males: r = 0.87; females: r = 0.88; see Table 2 for z-scores). Similarly, correlation coefficients between number of patch changes and AUC (males: r = 0.45; females: r = 0.47) were significantly weaker than correlation coefficients observed between number of patch changes and time deviation (males: r = 0.87; females: r = 0.88; see Table 2 for z-scores). This pattern in the strength of associations was repeated for time in patch (Patch utilization correlation coefficients: Reward maximization > DD, as shown in Table 1). A different pattern emerged for rate of water reinforcement. Here, both DD and reward maximization variables were either weakly or not significantly correlated to rate of water reinforcement. Time deviation correlation coefficients were similar to those observed for k, and AUC correlation coefficients were similar to correlation coefficients for volume deviation. These results were confirmed using correlation comparison approaches in the R package^30,31. Taken together, these data indicate that while standard DD variables are associated with patch utilization, these associations are weak relative to associations between patch utilization and reward maximization in both males and females.

The Marginal Value Theorem (MVT) suggests that optimal foragers will choose to stay in a patch until the rate of reward falls below the average rate of reward in alternative patches³. Consequently, longer travel delays between patches make staying in a depleting patch a more optimal choice because the longer delay depresses the average rate of reward, resulting in foragers remaining in the patch for a longer period of time. Alternatively, shorter travel delays between patches makes staying in a depleting patch for a long period of time a less optimal choice, resulting in “leave” choices occurring sooner. This is supported by the current experiment, in which CODs (representing travel delays) altered choice behavior. Increasing delays resulted in a decrease in the number of patch changes and an increase in time in patch as rats made more “stay” choices.

The sequential patch depletion task is a valuable paradigm to understand important processes underlying optimal behavior because it has strong ethological validity. However, another important strength of this paradigm is the ability to examine reward maximization in greater depth. By assessing the percent deviation from optimality, which normalizes the data to map reward maximization for various delays, we found that animals tended to overharvest at short delays, as indicated by remaining in a patch longer than optimal. Conversely, rats underharvested when the delays were longer, as indicated by animals leaving the patch sooner than optimal. Notably, the greatest deviation from the optimal time in patch occurred when delays were < 6 s, that is 0 s. These data may imply that despite the absence of a travel cost, some other unidentified variable influences the rat’s choice to stay in the current patch. While the focus of MVT is about travel costs, other factors can also contribute to choices to stay in a patch, such as predation and energy expenditure^1,2,4. In the current study, effort needed to change patches may have been weighted more at shorter delays. It is unclear if these variables are contributing to the sex differences found in this study. Future research is needed to tease apart how these factors may contribute to overharvesting.

Our data demonstrate that foraging behavior and traditional measures of impulsive choice, such as those provided by traditional DD indices, may share some characteristics of behavioral responding but may more strongly reflect distinct behavioral processes. We observed significant associations between DD and reward maximization variables. However, of note, reward maximization variables were more strongly associated with patch utilization variables than the DD variables. Consistent with this, Hayden, Pearson, & Platt⁹ showed that the performance of monkeys in a foraging task fit better with the MVT model than with hyperbolic DD function; they obtained a similar result when testing a patch-leaving foraging task interleaved with a traditional DD task³².

Sex Differences In Dd, Patch Utilization And Reward Maximization

Greater DD, and by extension impulsive choice, is indicated by longer stays in a depleting patch with smaller rewards. Although females had significantly lower indifference points than males at some delays, the effects sizes were small, and no differences in hyperbolic k values or AUC were found. Relatively few animal studies have examined sex differences in DD, however these studies have produced conflicting results. In animals, larger DD has been found in female rats^33,34 and mice³⁵, whereas others have identified larger DD in male rats^36,37, and either age-dependent sex differences³⁸, or no sex differences^39–42 have been described. Similar discrepancies have been reported in some human studies, with greater discounting observed in women^15,23,43,44, whereas other studies have found greater discounting in men^45–47. Even more studies of humans have found no differences^48–55. These disparate findings call for a more comprehensive examination of these differences in impulsive choice, as a foundation on which to explore the interrelationships between gender and DD in humans as well as their roles in psychopathologies (e.g., substance abuse).

In the present study, females had a lower rate of reinforcement than males, corresponding to fewer patch changes and spending more time in patches than males. Furthermore, while both sexes deviated from optimal performance, females deviated from optimality more than males at several delays. These data are consistent with findings of sex differences in humans in the Iowa Gambling Task, in which men had a greater preference for cards that were advantageous in the long-term compared to women⁵⁶. From this, van den Bos⁵⁶ posited that males tend to focus more on long-term goals, shifting from exploration to exploitation, whereas females exhibited greater exploratory behavior^35,57. The findings from the current study offer nuances to this hypothesis. The male rats in our study had a greater rate of reinforcement than females, supporting the interpretation that the males focused on long-term goals. However, females did not exhibit greater exploratory behavior as they made fewer patch changes and stayed in the patch significantly longer than males. Instead, the greater rate of switching between patches observed in males may reflect greater behavioral flexibility, whereas females may be demonstrating more perseverative behavior⁵⁸. Reports of sex differences in perseverative behaviors have been inconsistent^59–62, and thus, further research is needed to reconcile these discrepancies. This is of particular importance because perseveration is a feature of psychological disorders (e.g., schizophrenia, autism, OCD and drug addiction)⁶³, and understanding these processes may have implications in sex-dependent vulnerability.

There are a variety of possible explanations for why females stayed longer in a patch. For example, there are sex differences in energy expenditure^64,65, that may result in a female bias to preserve energy. Greater “overharvesting” observed in female rats may be an inaccurate interpretation: females may have exhibited appropriate levels of harvesting given environmental or biological factors, such as those related to reproductive success which made staying in the patch a more advantageous choice. Furthermore, their strategy may be to fully deplete the patch and exploit all resources before moving on to the next patch, as overharvesting may not abrogate visiting the patch in the future when resources have been replenished. In addition, females may have a differential sensitivity to tracking a changing environment, or to cues of change⁶⁶. Indeed, Tropp & Markus⁶⁷ found males and females utilize cues differently in various environments. When animals are presented with diminishing rewards upon the choice to stay in a patch, not only are they being reinforced, but they are also gaining new information about the quality of the reward, an important component of the economics of choice behavior^1,2. Finally, females may tend to choose safer options, which is supported by studies in rats⁶⁸.

Taken together, these data suggest that patch depletion model and by extension reward maximization was more sensitive to modest sex differences, compared to traditional DD tasks and metrics.

Future Directions

While we report differences between males and females in reward maximization, it is important to note we did not track estrous cycles in the females, so we cannot determine if this affected their performance on this task. To date, there is little evidence regarding the hormonal role in mediating these behavioral processes. Here, we present statistically different, but admittedly modest sex differences in performance on the sequential patch depletion task. This may be, in part, due to the role of the estrous cycle in performance on these tasks. In keeping with the literature exploring sex differences on DD, there is limited and conflicting evidence for the role of the estrous cycle on DD^33,69,70. Future research is needed to determine the impact of the estrous cycle on performance in the patch depletion procedure.

The sequential patch model assumes that foragers have perfect knowledge of the model’s parameters, identified by Stephens² as the “complete information assumption.” While the rats in our study had substantial training, this assumption may influence how we interpret their behavior. The differences between rewards associated with optimal rejection volumes were small, which may have contributed to the deviations from optimality. This issue has been noted previously in analyses of optimal performance on progressive schedules with reset^71–73.

This is the first report of sex differences in a large sample of Heterogenous Stock rats tested using the sequential patch depletion procedure. By using a large sample of Heterogenous Stock rats, we showed that females performance differed from males with regard to reward maximization that were not revealed utilizing traditional measures in DD. Notably, frequency distributions of rejection volume indicate that there was also sizeable variability in rejection volumes within each sex, which enables us to explore individual differences or genetic/environmental variables that contribute to performance in this task in future studies. Furthermore, measures of reward maximization were only weakly associated with DD variables and thus may be mediated by different underlying processes. Taken together, these data show the utility in the use of the sequential patch depletion procedure for measuring choice and may have implications in vulnerability in the development of psychiatric diseases.

Subjects

Male (n = 896) and female (n = 898) heterogeneous stock rats (HS; NMcwiWfsm:HS; RRID # RRID:RGD_13673907)⁷⁴ tested in this study were part of the NIDA Center for GWAS in Outbred Rats (Principal Investigator, Dr. Abraham Palmer) and obtained from colonies maintained by Dr. Leah Solberg Woods [2015–2018: Medical College of Wisconsin: males n = 815, females n = 816; 2019: Wake Forest University: males n = 81; females: n = 82]. Experiments were conducted in batches of approximately 100 rats (4- to 5-weeks of age) at 3- to 4-month intervals. Rats were quarantined for 1- to 2-weeks upon arrival to University at Buffalo before being transferred to colony housing. The colony room was maintained at a constant temperature (22 ± 1°C), humidity range (~ 55% ± 5%), and lights were on a reverse cycle (lights on from 19:00 to 07:00).

Rats were housed in same-sex pairs in plastic cages (42 × 22 × 19 cm) lined with bedding (Aspen Shavings). Prior to the start of the experiment, rats (n = 1590) were first tested on four behavioral tasks (social reinforcement, locomotor response to novelty, light reinforcement, and choice reaction time; data are not reported here). A subset of animals (n = 204) were not tested on the social reinforcement test because it was introduced after the first two batches had already been tested. At the onset of data collection, the mean (± SEM) age of the rats was postnatal day 136.58 ± 0.29.

Behavioral testing was conducted 6 days/week (Monday through Saturday) during the dark phase of the light-dark cycle between the hours of 08:30 and 12:30. Food (Teklad Laboratory Diet #8604) was available ad libitum in the home cages. Access to water was restricted to 30 min immediately following testing on Monday through Friday. At the end of the testing on Saturday, animals were given free access to water until approximately 12:30 h on Sunday (approximately 20 hr prior to testing on Monday).

This study was conducted in accordance with protocols approved by the Institutional Animal Care and Use Committee at the University at Buffalo, and animals were treated in compliance with the Guide for the Care and Use of Laboratory Animals, and the study is reported in accordance with the ARRIVE guidelines ⁷⁵.

Apparatus

Testing occurred in 24 locally constructed operant chambers (24 × 22 × 20 cm; Fig. 1) housed in sound-attenuating cabinets (Model # 3000000187, Coleman, Wichita, KS), which were previously described in detail ⁷⁶. Briefly, the chambers had stainless-steel rod floors, aluminum back and side walls, and a Plexiglas front wall and top. Each test chamber had three snout poke receptacles (4 cm in diameter) located in the back and side walls. Infrared photobeam detectors, located 1 cm from the snout poke receptacle entrance, were used to record snout pokes. Three stimulus lights were located above each snout poke receptacle, and a fourth light was located in the ceiling of the test chamber. A Sonalert tone generator (SC628EJR; Mallory Sonalert Products, Indianapolis, IN) mounted on the right wall provided a pulsed 1.9-kHz tone. Acrylic dishes were located inside the left and rear snout poke receptacles and were connected to Tygon tubing, which delivered precise amounts of water from 60 ml syringes mounted on two single-speed syringe pumps (3.33 rpm, PHM-100; Med Associates, St. Albans, VT) external to the sound-attenuating cabinet.

The apparatus was controlled by MED-PC IV software (RRID:SCR_012156) with 1 ms temporal resolution running on computers with Microsoft Windows operating systems. Equipment was tested before test sessions and following any session in which a rat earned fewer than 30 reinforcers.

Sequential Patch Depletion Procedure

Behavior was measured using a sequential patch depletion procedure that was previously described (Fig. 2a)^26,77. Briefly, during this task, water-restricted rats were offered a concurrent choice between two water “patches” (left- or back-wall snout poke receptacles) and could elect to “stay” in the current patch or “leave” for an alternative patch at any time.

A snout poke in the receptacle (i.e., entering the patch) resulted in immediate delivery of 150 µl water. If rats made a choice to “stay” in this patch, successively smaller amounts of water were delivered. To simulate patch depletion, the volume of water for successive “stay” choices was reduced 20% for each reinforcer presentation. For example, the first reward volume was 150 µl of water, the second reward was 120 µl, and the third reward was 96 µl, etc. with each reward delivery separated by a minimum of 4 s (Fig. 2a and 2b). Once in a patch, water was available according to a modified Fixed Interval (FI) 4-s schedule; each water delivery was followed by a 4-s interval during which the reinforcer was unavailable. However, successive “stay” choices were achieved in one of two ways: rats could emit a snout poke to the same receptacle as the previous response following the 4-s interval (traditional FI contingency) or the rat could remain with its snout in the receptacle for the duration of the 4-s interval (analogous to a Fixed Time or FT schedule).

If a rat made the choice to “leave” a patch by poking its snout in the alternative receptacle (“alternative patch”), a changeover delay (COD) was imposed to simulate “travel” cost⁷⁸. During the COD, reinforcers were not available at either location regardless of responding. When the rat left a patch to travel to the alternative patch, the abandoned patch was replenished (the volume of the water was reset to 150 µl for the first reinforced response on returning to that patch).

When a patch change occurred with a 0-s COD, the stimulus light above the abandoned patch was extinguished, and the stimulus light above the newly poked receptacle was illuminated simultaneously with the delivery of 150 µl of water. For CODs > 0 s (6, 12, 18, or 24 s), the stimulus light above the abandoned receptacle was extinguished and a 1.9-kHz tone was pulsed for the duration of the COD. At the end of the delay, the pulsed tone was turned off and the stimulus light above the newly poked receptacle was illuminated. The first snout poke into the new location after the onset of the stimulus light resulted in the delivery of 150 µl of water. Sessions lasted for 10 min or until the rat earned a cumulative total of 5 ml of water, whichever occurred first.

The COD was constant within a session but varied between sessions (i.e., days of the week) in the following sequence (Monday through Saturday): 0, 0, 6, 12, 18, and 24 s, with the first session of the week being excluded from data analysis (i.e., the first 0 s session). This 6-day cycle was repeated four times for a total of 24 test sessions. Data from the last two cycles of sessions for each COD delay were averaged for data analysis.

Dependent Measures

There were five primary dependent measures: rate of water reinforcement, number of patch changes, average time in a patch, average water volume rejected by leaving the patch (“rejection volume”/“indifference point”), and the deviation from optimality. The first and final patches, including associated patch changes, of each session were not included in these calculations. These patches were excluded because, in the case of the first patch, rats had not yet experienced the COD, and the last patch resulted in session termination due to the session ending and so not accurately representing rats’ choice behavior. Rate of reinforcement was calculated by summing the volume of water earned and dividing that by time required to earn those reinforcers. Number of patch changes was the number of times rats chose to switch to the alternative receptacle (“leave” choices) averaged over the last two sessions. Time in patch was the mean duration the rat stayed at one receptacle before leaving for the other. The rejection volume/indifference point was defined as the mean amount of water (µL) available at the abandoned patch when the rat switched to the alternative snout poke location, e.g., if the rat had earned 120 µl before leaving, the leaving volume would be the next volume scheduled for delivery, 96 µl in this example. The optimal rejection volume, based on the MVT, was operationally defined as the volume of water that maximized the average reward volume rate across all patches including the COD travel time. This was calculated as the cumulative rate of return from the patch (µL/s) for each reward, considering the COD length (travel time to the patch) and the 4 s between successive rewards (Fig. 2c). For example, when the COD was 6 s, the cumulative rate of return for the first reward in a patch was 150 µL divided by 6 s (25 µL/s). For the second reward, it was [150 + 120] µL divided by [6 + 4] s (27 µL/s). For the third reward, it was [150 + 120 + 96] µL divided by [6 + 4 + 4] s (26.14 µL/s). In this instance, an optimal strategy predicts leaving after two rewards are obtained and the rejection volume is 120 µL. The percent deviation from the optimal rejection volume (referred to as percent volume deviation) was calculated as follows: (optimal rejection volume − observed rejection volume)/optimal rejection volume x 100. Positive values indicate rats overharvested and stayed in the patch when the volume of the collected reinforcers had dropped below the value required to maximize, whereas negative values indicate the rat left when the volume was greater than the optimal volume. The same approach was used to calculate the percent deviation from optimal time in patch (referred to as percent time deviation). Negative values indicate the observed time in the patch was longer than optimal, and positive values indicate the observed time in the patch was less than optimal.

Statistical Analysis

Statistical analyses used SPSS Statistics software (IBM, Armonk, NY). Descriptive statistics indicated that the distributions of dependent measures were normal (skewness < |1|), so parametric statistics were used throughout.

To examine the effects of COD travel time on patch leaving, we used a mixed factor analysis of variance (ANOVA), with delay as the within-subject factor (0, 6, 12, 18, and 24 s) and sex as the between-subject factor (male and female), in conjunction with post-hoc comparisons using Fisher’s least significant difference (LSD) test for within-subject comparisons and independent samples t-tests with Bonferroni corrections for between-subject comparisons.

To examine whether analyses applied to traditional DD tasks was possible, hyperbolic equations were fitted to each rat’s rejection volume (indifference points) based on that described by Mazur ⁷⁹, using GraphPad Prism (GraphPad Software Inc., San Diego, CA):

V = bA/(1 + kD)

where V indicates the rejected volume of the diminishing reinforcer when the rat left the current patch for the alternative patch in µl, A represents the amount of water from the alternative patch (150 µL), and D represents the delay to receiving the 150-µL reinforcer (COD of 0, 2, 4, 8, 16, or 24 s). The bias parameter, b, was calculated such that the product of b and A equaled each animal’s indifference point at a 0-s delay⁸⁰. The discount parameter (k) is an index for the rate of discounting or overall sensitivity to delayed reinforcers, such as the first reinforcer in an alternative patch. In DD tasks, larger values of k indicate steeper discount functions, stronger aversion to delayed reinforcers, more rapid devaluation of reinforcer value by delay, and thus greater impulsive choice. Here, k indicates higher relative levels of overharvesting and a preference for the smaller, sooner rewards available in the current patch over traveling to an alternative patch. The normalized area under the curve (AUC) of the discount function was calculated, which summarizes the influence of delay length on the choice to remain at a patch location. The AUC measure provides a simple measure of overharvesting/discounting that is not tied to a particular discount function⁸¹. Smaller AUC values indicate higher levels of overharvesting. The k, b, and AUC values were analyzed using independent samples t-tests, with sex as the between-subject factor.

Composite scores for all variables were calculated by the sum of the variable across all delays. These composite scores were used to calculate Pearson’s correlation coefficients. To assess the difference in the strength of their association with DD and reward maximization, correlation coefficients were compared using Fisher r-to-z transformations^29,30, which is recommended for comparing correlation coefficients from the same sample with one variable in common⁸². For all statistical tests, a p < .05 was used as the alpha criterion.

Table 1

Pearson Correlation Matrices for Male and Female Rats
	r value
	AUC	VD	TD	TIP	PC	WR
	Males (n = 896)
k	−0.76**	0.56**	−0.05	0.32**	−0.30**	0.36**
AUC		−0.56**	−0.27**	−0.50**	0.45**	0.09**
VD			−0.79**	−0.86**	−0.90**	0.05
TD				−0.92**	0.87**	0.37**
TIP					−0.89**	−0.33**
PC						0.20**
	Females (n = 898)
k	−0.73**	0.64**	−0.13**	0.43**	−0.35**	0.26**
AUC		−0.59**	0.29**	−0.52**	0.47**	0.18**
VD			−0.75**	0.86**	−0.88**	0.03
TD				−0.89**	0.88**	0.36**
TIP					−0.90**	−0.30**
PC						0.26**

Note. k, free parameter discounting index; AUC, area under the curve for indifference points; VD, volume deviation; TD, time deviation; TIP, time in patch; PC, number of patch changes; WR, water rate of reinforcement *Correlation is significant at the 0.05 level. ** is significant at the 0.01 level (two-tailed analysis).

Table 2

z-Scores for Relative Strengths of Correlations between Delay Discounting and Reward Maximization Metrics in Male and Female Rats
Comparison	z-score
	TIP	PC	WR
	Males
k vs. VD	42.97**	30.33**	10.14**
k vs. TD	39.58**	−32.81**	-0.23
AUC vs. VD	12.58**	33.10**	0.67
AUC vs. TD	19.49**	−15.90**	−5.52**
	Females
k vs. VD	−23.43**	28.63**	8.20**
k vs. TD	37.44**	−34.65**	−2.16*
AUC vs. VD	−31.36**	31.63**	2.54*
AUC vs. TD	18.80**	-19.13**	−4.76**

Note. TIP, time in patch; PC, number of patch changes; WR, water rate of reinforcement; k, free parameter impulsivity index; VD, volume deviation; TD, time deviation; AUC, area under the curve for indifference points; Significant difference between correlation coefficients using the z-test procedure (*p < 0.05; **p < 0.01) outlined by Meng et al. (1992) and the R Cocur package (Diedenhofen & Musch, 2015).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Author Contributions

The study was conceived by A.P., P.M., J.B.R., and K.I. The experimental procedure and general study design was created by J.B.R. The experimental protocols were implemented and conducted by K.I., A.M.George, C.D.M., M.C.T., H.M.B., F.A., F.K. The conduct of the project was supervised by K.I., A.P., P.M., J.B.R., and D.M.D. The data were analyzed by A.M.Gancarz, S.H.M., and K.I. Data were interpreted and the manuscript written by A.M.Gancarz., S.H.M., and D.M.D. All the authors have read and approved the final version of the manuscript.

Funding

This work was partly supported by grants from NIH (P50 DA037844 and U01 DA046077).

Acknowledgments

This work was partly supported by grants from NIH (P50 DA037844 and U01 DA046077). We thank Leah Solberg Woods for supplying the heterogeneous stock rats tested in this study, Mark Kogutowski for his technical expertise, and Karen Dietz for her comments and suggestions on an earlier draft of the manuscript.

Data Availability

The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.

Stephens, D. W., Brown, J. S. & Ydenberg, R. C. Foraging: behavior and ecology. (University of Chicago Press, 2008).
Stephens, D. W. & Krebs, J. R. in Foraging theory (Princeton university press, 2019).
Charnov, E. L. Optimal foraging, the marginal value theorem. Theor Popul Biol 9, 129–136 (1976). https://doi.org:10.1016/0040-5809(76)90040-x
Kendall, R. K. & Wikenheiser, A. M. Quitting while you’re ahead: Patch foraging and temporal cognition. Behavioral Neuroscience (2022).
Benson, K. E. & Stephens, D. W. Interruptions, Tradeoffs, and Temporal Discounting. American Zoologist 36, 507–517 (1996).
McNamara, J. M. & Houston, A. I. A general framework for understanding the effects of variability and interruptions on foraging behaviour. Acta Biotheor 36, 3–22 (1987). https://doi.org:10.1007/BF00159228
Carter, E. C. & Redish, A. D. Rats value time differently on equivalent foraging and delay-discounting tasks. Journal of Experimental Psychology: General 145, 1093 (2016).
Constantino, S. M. & Daw, N. D. Learning the opportunity cost of time in a patch-foraging task. Cognitive, Affective, & Behavioral Neuroscience 15, 837–853 (2015).
Hayden, B. Y., Pearson, J. M. & Platt, M. L. Neuronal basis of sequential foraging decisions in a patchy environment. Nature neuroscience 14, 933–939 (2011).
Kane, G. A. et al. Increased locus coeruleus tonic activity causes disengagement from a patch-foraging task. Cognitive, Affective, & Behavioral Neuroscience 17, 1073–1083 (2017).
Kolling, N., Behrens, T. E., Mars, R. B. & Rushworth, M. F. Neural mechanisms of foraging. Science 336, 95–98 (2012).
Nonacs, P. State dependent behavior and the marginal value theorem. Behavioral Ecology 12, 71–83 (2001).
Shenhav, A., Straccia, M. A., Cohen, J. D. & Botvinick, M. M. Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value. Nature neuroscience 17, 1249–1254 (2014).
Wikenheiser, A., M., Stephens, D., W. & Redish, A. D. Subjective costs drive overly patient foraging strategies in rats on an intertemporal foraging task. Proceedings of the National Academy of Sciences 110, 8308–8313 (2013). https://doi.org:10.1073/pnas.1220738110
Sanchez-Roige, S. et al. Genome-wide association study of delay discounting in 23,217 adult research participants of European ancestry. Nat Neurosci 21, 16–18 (2018). https://doi.org:10.1038/s41593-017-0032-x
Coffey, S. F., Gudleski, G. D., Saladin, M. E. & Brady, K. T. Impulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals. Experimental and clinical psychopharmacology 11, 18 (2003).
de Wit, H. Impulsivity as a determinant and consequence of drug use: a review of underlying processes. Addict Biol 14, 22–31 (2009). https://doi.org:10.1111/j.1369-1600.2008.00129.x
Jentsch, J. D. et al. Dissecting impulsivity and its relationships to drug addictions. Ann N Y Acad Sci 1327, 1–26 (2014). https://doi.org:10.1111/nyas.12388
Johnson, M. W., Bruner, N. R. & Johnson, P. S. Cocaine dependent individuals discount future rewards more than future losses for both cocaine and monetary outcomes. Addictive behaviors 40, 132–136 (2015).
Kirby, K. N. & Petry, N. M. Heroin and cocaine abusers have higher discount rates for delayed rewards than alcoholics or non-drug‐using controls. Addiction 99, 461–471 (2004).
MacKillop, J. et al. Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacology (Berl) 216, 305–321 (2011). https://doi.org:10.1007/s00213-011-2229-0
Perry, J. L. & Carroll, M. E. The role of impulsive behavior in drug abuse. Psychopharmacology (Berl) 200, 1–26 (2008). https://doi.org:10.1007/s00213-008-1173-0
Reynolds, B. A review of delay-discounting research with humans: relations to drug use and gambling. Behav Pharmacol 17, 651–667 (2006). https://doi.org:10.1097/FBP.0b013e3280115f99
Amlung, M. et al. Delay discounting as a transdiagnostic process in psychiatric disorders: a meta-analysis. JAMA psychiatry 76, 1176–1186 (2019).
Bickel, W. K., Jarmolowicz, D. P., Mueller, E. T., Koffarnus, M. N. & Gatchalian, K. M. Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: Emerging evidence. Pharmacology & Therapeutics 134, 287–297 (2012). https://doi.org:https://doi.org/10.1016/j.pharmthera.2012.02.004
Richards, J. B. et al. Strong genetic influences on measures of behavioral-regulation among inbred rat strains. Genes Brain Behav 12, 490–502 (2013). https://doi.org:10.1111/gbb.12050
Parker, C. C. et al. Rats are the smart choice: rationale for a renewed focus on rats in behavioral genetics. Neuropharmacology 76, 250–258 (2014).
Solberg Woods, L. C. & Palmer, A. A. Using heterogeneous stocks for fine-mapping genetically complex traits. Rat genomics, 233–247 (2019).
Meng, X.-L., Rosenthal, R. & Rubin, D. B. Comparing correlated correlation coefficients. Psychological bulletin 111, 172 (1992).
Diedenhofen, B. & Musch, J. cocor: a comprehensive solution for the statistical comparison of correlations. PLoS One 10, e0121945 (2015). https://doi.org:10.1371/journal.pone.0121945
Zou, G. Y. Toward using confidence intervals to compare correlations. Psychological methods 12, 399 (2007).
Blanchard, T. C. & Hayden, B. Y. Monkeys are more patient in a foraging task than in a standard intertemporal choice task. PLoS One 10, e0117057 (2015). https://doi.org:10.1371/journal.pone.0117057
Hernandez, C. M. et al. Testicular hormones mediate robust sex differences in impulsive choice in rats. eLife, e58604 (2020). https://doi.org:https://doi.org/10.7554/eLife.58604
Van Haaren, F., Van Hest, A. & Van de Poll, N. E. Self-control in male and female rats. Jounral of the Experimental Analysis of Behavior 49, 201–211 (1988).
Koot, S., van den Bos, R., Adriani, W. & Laviola, G. Gender differences in delay-discounting under mild food restriction. Behav Brain Res 200, 134–143 (2009). https://doi.org:10.1016/j.bbr.2009.01.006
Bayless, D. W., Darling, J. S. & Daniel, J. M. Mechanisms by which neonatal testosterone exposure mediates sex differences in impulsivity in prepubertal rats. Horm Behav 64, 764–769 (2013). https://doi.org:10.1016/j.yhbeh.2013.10.003
Panfil, K., Bailey, C., Davis, I., Mains, A. & Kirkpatrick, K. A time-based intervention to treat impulsivity in male and female rats. Behav Brain Res 379, 112316 (2020). https://doi.org:10.1016/j.bbr.2019.112316
Lukkes, J. L., Thompson, B. S., Freund, N. & Andersen, S. L. The developmental inter-relationships between activity, novelty preferences, and delay discounting in male and female rats. Dev Psychobiol 58, 231–242 (2016). https://doi.org:10.1002/dev.21368
Doremus-Fitzwater, T. L., Barreto, M. & Spear, L. P. Age-related differences in impulsivity among adolescent and adult Sprague-Dawley rats. Behav Neurosci 126, 735–741 (2012). https://doi.org:10.1037/a0029697
Eubig, P. A., Noe, T. E., Floresco, S. B., Sable, J. J. & Schantz, S. L. Sex differences in response to amphetamine in adult Long-Evans rats performing a delay-discounting task. Pharmacol Biochem Behav 118, 1–9 (2014). https://doi.org:10.1016/j.pbb.2013.12.021
Perry, J. L., Stairs, D. J. & Bardo, M. T. Impulsive choice and environmental enrichment: effects of d-amphetamine and methylphenidate. Behav Brain Res 193, 48–54 (2008). https://doi.org:10.1016/j.bbr.2008.04.019
Sackett, D. A., Moschak, T. M. & Carelli, R. M. Prelimbic Cortical Neurons Track Preferred Reward Value and Reflect Impulsive Choice during Delay Discounting Behavior. J Neurosci 39, 3108–3118 (2019). https://doi.org:10.1523/JNEUROSCI.2532-18.2019
Becker, J. B. & Hu, M. Sex differences in drug abuse. Front Neuroendocrinol 29, 36–47 (2008). https://doi.org:10.1016/j.yfrne.2007.07.003
Smith, C. L. & Hantula, D. A. Methodological considerations in the study of delay discounting in intertemporal choice: A comparison of tasks and modes. Behav Res Methods 40, 940–953 (2008). https://doi.org:10.3758/BRM.40.4.940
Dittrich, M. & Leipold, K. Gender differences in time preferences. Econimics Letters 122, 413–415 (2014). https://doi.org:10.1016/j.econlet.2014.01.002
Kirby, K. N. & Marakovic, N. N. Delay-discounting probabilistic rewards: Rates decrease as amounts increase. Psychon Bull Rev 3, 100–104 (1996). https://doi.org:10.3758/BF03210748
Stanovich, K. E., West, R. F. & Toplak, M. E. The rationality quotient: Toward a test of rational thinking. (MIT press, 2016).
Bembenutty, H. Academic delay of gratification, self-efficacy, and time management among academically unprepared college students. Psychol Rep 104, 613–623 (2009). https://doi.org:10.2466/pr0.104.2.613-623
Cross, C. P., Copping, L. T. & Campbell, A. Sex differences in impulsivity: A meta-analysis. Psychological Bulletin 137, 97–130 (2011). https://doi.org:https://doi.org/10.1037/a0021591
Doidge, J. L., Flora, D. B. & Toplak, M. E. A Meta-Analytic Review of Sex Differences on Delay of Gratification and Temporal Discounting Tasks in ADHD and Typically Developing Samples. J Atten Disord 25, 540–561 (2021). https://doi.org:10.1177/1087054718815588
Harrison, E. L., Coppola, S. & McKee, S. A. Nicotine deprivation and trait impulsivity affect smokers' performance on cognitive tasks of inhibition and attention. Exp Clin Psychopharmacol 17, 91–98 (2009). https://doi.org:10.1037/a0015657
Logue, A. W. & Anderson, Y. D. Higher-education administrators: when the future does not make a difference. Psychol Sci 12, 276–281 (2001). https://doi.org:10.1111/1467-9280.00351
Prencipe, A. et al. Development of hot and cool executive function during the transition to adolescence. J Exp Child Psychol 108, 621–637 (2011). https://doi.org:10.1016/j.jecp.2010.09.008
Reynolds, B., Richards, J. B., Horn, K. & Karraker, K. Delay discounting and probability discounting as related to cigarette smoking status in adults. Behav Processes 65, 35–42 (2004). https://doi.org:10.1016/s0376-6357(03)00109-8
Silverman, I. W. Gender Differences in Delay of Gratification: A Meta-Analysis. Sex Roles 49, 451–463 (2003).
van den Bos, R., Homberg, J. & de Visser, L. A critical review of sex differences in decision-making tasks: Focus on the Iowa Gambling Task. Behavioural Brain Research 238, 95–108 (2013). https://doi.org:https://doi.org/10.1016/j.bbr.2012.10.002
Orsini, C. A. & Setlow, B. Sex differences in animal models of decision making. J Neurosci Res 95, 260–269 (2017). https://doi.org:10.1002/jnr.23810
Westbrook, S. R., Hankosky, E. R., Dwyer, M. R. & Gulley, J. M. Age and sex differences in behavioral flexibility, sensitivity to reward value, and risky decision-making. Behavioral Neuroscience 132, 75–87 (2018). https://doi.org:10.1037/bne0000235
Guajardo, H. M., Snyder, K., Ho, A. & Valentino, R. J. Sex Differences in µ-Opioid Receptor Regulation of the Rat Locus Coeruleus and Their Cognitive Consequences. Neuropsychopharmacology 42, 1295–1304 (2017). https://doi.org:10.1038/npp.2016.252
Ulloa, R.-E., Nicolini, H. & Fernández-Guasti, A. Sex differences on spontaneous alternation in prepubertal rats: implications for an animal model of obsessive-compulsive disorder. Progress in Neuro-Psychopharmacology and Biological Psychiatry 28, 687–692 (2004).
Baran, S. E., Armstrong, C. E., Niren, D. C. & Conrad, C. D. Prefrontal cortex lesions and sex differences in fear extinction and perseveration. Learn Mem 17, 267–278 (2010). https://doi.org:10.1101/lm.1778010
Grafe, L. A., Cornfeld, A., Luz, S., Valentino, R. & Bhatnagar, S. Orexins Mediate Sex Differences in the Stress Response and in Cognitive Flexibility. Biological Psychiatry 81, 683–692 (2017). https://doi.org:https://doi.org/10.1016/j.biopsych.2016.10.013
Ridley, R. M. The psychology of perserverative and stereotyped behaviour. Prog Neurobiol 44, 221–231 (1994). https://doi.org:10.1016/0301-0082(94)90039-6
Hoyenga, K. B. & Hoyenga, K. T. Gender and energy balance: Sex differences in adaptations for feast and famine. Physiology & Behavior 28, 545–563 (1982). https://doi.org:https://doi.org/10.1016/0031-9384(82)90153-6
Key, C. & Ross, C. Sex differences in energy expenditure in non–human primates. Proceedings of the Royal Society of London. Series B: Biological Sciences 266, 2479–2485 (1999).
Blanchard, T. C., Pearson, J. M. & Hayden, B. Y. Postreward delays and systematic biases in measures of animal temporal discounting. Proc Natl Acad Sci U S A 110, 15491–15496 (2013). https://doi.org:10.1073/pnas.1310446110
Tropp, J. & Markus, E. J. Sex differences in the dynamics of cue utilization and exploratory behavior. Behavioural Brain Research 119, 143–154 (2001). https://doi.org:https://doi.org/10.1016/S0166-4328(00)00345-4
Pellman, B. A., Schuessler, B. P., Tellakat, M. & Kim, J. J. Sexually dimorphic risk mitigation strategies in rats. eneuro 4 (2017).
Carroll, M. E., Kohl, E. A., Johnson, K. M. & LaNasa, R. M. Increased impulsive choice for saccharin during PCP withdrawal in female monkeys: influence of menstrual cycle phase. Psychopharmacology (Berl) 227, 413–424 (2013). https://doi.org:10.1007/s00213-012-2963-y
Liley, A. E., Gabriel, D. B. K., Sable, H. J. & Simon, N. W. Sex Differences and Effects of Predictive Cues on Delayed Punishment Discounting. eNeuro 6 (2019). https://doi.org:10.1523/eneuro.0225-19.2019
Mazur, J. E. & Vaughan Jr, W. Molar optimization versus delayed reinforcement as explanations of choice between fixed-ratio and progressive‐ratio schedules. Journal of the Experimental Analysis of Behavior 48, 251–261 (1987).
Hackenberg, T. D. & Axtell, S. A. HUMANS'CHOICES IN SITUATIONS OF TIME-BASED DIMINISHING RETURNS. Journal of the Experimental Analysis of Behavior 59, 445–470 (1993).
Sodetz, F. J. Appetitive and Aversive Schedule Preferences: Schedule Transitions as lntervening Events. The Effect of Delay and of Intervening Events on Reinforcement Value: Quantitative Analyses of Behavior, Volume V, 141 (2013).
Hansen, C. & Spuhler, K. Development of the National Institutes of Health Genetically Heterogeneous Rat Stock. Alcoholism: Clinical and Experimental Research 8, 477–479 (1984). https://doi.org:https://doi.org/10.1111/j.1530-0277.1984.tb05706.x
Du Sert, N. P. et al. Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0. PLoS biology 18, e3000411 (2020).
Lloyd, D. R., Kausch, M. A., Gancarz, A. M., Beyley, L. J. & Richards, J. B. Effects of novelty and methamphetamine on conditioned and sensory reinforcement. Behav Brain Res 234, 312–322 (2012). https://doi.org:10.1016/j.bbr.2012.07.012
Ishiwari, K. et al. Delay Discounting Measured Using a Sequential Patch Depletion Procedure, <https://www.protocols.io/view/delay-discounting-measured-using-a-sequential-patc-n92ldzqnnv5b/v1> (
Herrnstein, R. J. On the law of effect. J Exp Anal Behav 13, 243–266 (1970). https://doi.org:10.1901/jeab.1970.13-243
Mazur, J. E. in The effect of delay and of intervening events on reinforcement value. Quantitative analyses of behavior, Vol. 5. 55–73 (Lawrence Erlbaum Associates, Inc, 1987).
Richards, J. B., Mitchell, S. H., De Wit, H. & Seiden, L. S. Determination of discount functions in rats with an adjusting-amount procedure. Journal of the experimental analysis of behavior 67, 353–366 (1997).
Myerson, J., Green, L. & Warusawitharana, M. Area under the curve as a measure of discounting. J Exp Anal Behav 76, 235–243 (2001). https://doi.org:10.1901/jeab.2001.76-235
Weaver, B. & Wuensch, K. L. SPSS and SAS programs for comparing Pearson correlations and OLS regression coefficients. Behavior research methods 45, 880–895 (2013).

No competing interests reported.

Download PDF

Journal Publication

published 29 Apr, 2023

Read the published version in Scientific Reports →

Editorial decision: Major revision
21 Feb, 2023
Reviews received at journal
18 Feb, 2023
Reviews received at journal
12 Feb, 2023
Reviewers agreed at journal
11 Feb, 2023
Reviewers agreed at journal
08 Feb, 2023
Reviewers invited by journal
08 Feb, 2023
Editor assigned by journal
08 Feb, 2023
Editor invited by journal
02 Feb, 2023
Submission checks completed at journal
02 Feb, 2023
First submitted to journal
28 Jan, 2023

You are reading this latest preprint version

Reward Maximization Assessed Using a Sequential Patch Depletion Task in a Large Sample of Heterogeneous Stock Rats

Status:

Journal Publication

Version 1

Abstract

Figures

Introduction

Results

Delay Discounting

Patch Utilization

Strengths Of Associations Between Patch Utilization, Reward Maximization, And Dd

Discussion

Sex Differences In Dd, Patch Utilization And Reward Maximization

Future Directions

Materials And Methods

Subjects

Apparatus

Sequential Patch Depletion Procedure

Dependent Measures

Statistical Analysis

Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1