Experimental hut trials
Experimental hut trials are standardised simulations of human-occupied housing recommended by the WHO for evaluating the efficacy of ITNs. These real-world assays recreate exposure conditions when host-seeking mosquitoes interact with nets inside households and are thus a highly suitable method for assessing the bioefficacy of operationally-aged nets throughout their intended lifespan.
Study site and experimental huts
The experimental hut trials were conducted in the commune of Covè, Zou Department, southern Benin (7°14′N2°18′E) near villages where field-aged study nets were withdrawn (Fig. 1). The huts are located in a large area of rice irrigation, which provides permanent and extensive breeding sites for mosquitoes and, hence, a high year-round density of malaria vectors. The vector population consists of a mixture of Anopheles coluzzii and Anopheles gambiae sensu stricto (s.s.), with the latter occurring at lower proportions (~23 %) and predominantly in the dry season (23). Previous studies characterising the resistance profile of the vector population using WHO susceptibility bioassays have demonstrated a high frequency (>90% bioassay survival) and intensity (200-fold) of pyrethroid resistance but continued susceptibility to other insecticides including CFP and PPF (23-25). Pre-exposure to PBO provides partial to complete restoration in pyrethroid susceptibility, demonstrating the contribution of P450s to observed pyrethroid resistance (24). Genotyping and gene expression studies support this, revealing a high frequency of the knockdown resistance (kdr) mutation (89%) and four-fold overexpression of CYP6P3 – a P450 validated as an efficient metaboliser of pyrethroids (23). The experimental huts used were of standard West African design, made of concrete bricks with cement-plastered walls enclosed by a corrugated iron roof and a polyethylene ceiling. Each hut had a wooden-framed veranda projecting from the rear wall to capture exiting mosquitoes and was surrounded by a water-filled moat to prevent mosquito predators from entering.
Description of study nets
In this study, the entomological efficacy of three different types of field-aged next-generation ITN (PermaNet® 3.0, Royal Guard® and Interceptor® G2) was evaluated in experimental huts compared to a standard pyrethroid-only net (Interceptor®). All of these ITN products are on the WHO list of prequalified vector control products (26). A detailed description of the specifications of each net is provided below:
i. Interceptor® is a standard pyrethroid-only net manufactured by BASF. It is made of 100-denier polyester filaments coated with 5 g/kg (200 mg/m2) ±25% of alpha-cypermethrin. Its dimensions are 1.8 (L) x 1.8 (W) x 1.8 (H) m, while the fabric weighs 40 g/m2 and has a minimum bursting strength of 405 kPa.
ii. PermaNet® 3.0 is a pyrethroid-PBO net manufactured by Vestergaard Sàrl. It features a mosaic design consisting of 100-denier polyester side panels coated with 2.1 g/kg (84 mg/m2) ±25% of deltamethrin and a 100-denier polyethylene roof panel incorporated with a mixture of deltamethrin and PBO at 4 g/kg (120 mg/m2) ±25% and 25 g/kg (800 mg/m2) ±25%, respectively. Its dimensions are 1.6 (L) x 1.8 (W) x 1.5 (H) m. The fabric weight and minimum bursting strength are 40 g/m2 and 350 kPa on the sides compared to 30 g/m2 and 400 kPa on the roof.
iii. Royal Guard® is a pyrethroid-PPF net manufactured by Disease Control Technologies. It is made of 150-denier high-density polyethylene filaments incorporated with a mixture of alpha-cypermethrin and PPF at target doses of 5.0 g/kg (225 mg/m2) ±25% each. Its dimensions are 1.8 (L) x 1.8 (W) x 1.6 (H) m while the fabric weighs 45 g/m2 and has a minimum bursting strength of 450 kPa.
iv. Interceptor® G2 is a pyrethroid-CFP net manufactured by BASF. It is made of 100-denier polyester filaments coated with a mixture of alpha-cypermethrin and CFP at target doses of 2.4 g/kg (100 mg/m2) ±25% and 4.8 g/kg (200 mg/m2) ±25%, respectively. Its dimensions are 1.8 (L) x 1.8 (W) x 1.8 (H) m while the fabric weighs 40 g/m2 and has a minimum bursting strength of 405 kPa.
Withdrawal of field-aged nets
The field-aged nets used for the experimental hut trials were distributed during a mass campaign in March 2020 to villages near the hut station in Zou Department, southern Benin. The Interceptor®, Royal Guard® and Interceptor® G2 nets were withdrawn from randomly selected clusters in the communes of Covè, Zagnanado and Ouinhi as part of a durability monitoring study nested within a cluster-RCT evaluating the epidemiological efficacy of dual-AI ITNs (21). The PermaNet® 3.0 nets were collected from the nearby village of Avokanzoun in the communes of Djidja and Bohicon (7°20’N 1°56’E), where they were distributed at the same time as the cluster-RCT nets during the 2020 mass campaign conducted by the national malaria control programme. The location of the study clusters/villages where the different ITN types were collected relative to the experimental hut site is shown in Fig. 1.
The design of the durability study, including the sampling of nets for the experimental hut trials, has been described previously (21). To summarise, two cohorts of nets were marked at baseline from 10 randomly selected clusters of each arm of the cluster-RCT (Interceptor®, Royal Guard® and Interceptor® G2) and followed every 6-12 months to assess attrition and fabric integrity (cohort 1) and bioefficacy and chemical content (cohort 2) according to WHO guidelines (17). Cohort 2 consisted of 1800 nets from 600 households per study arm that were collected (and replaced with new nets) and tested for the durability of their insecticidal activity. A subsample of randomly selected nets from cohort 2 sampled at 12, 24 and 36 months post-distribution were tested in experimental huts at each time point. PermaNet® 3.0 nets obtained from the nearby village of Avokanzoun were marked at baseline in 200 households and collected (and replaced with new nets) at 12, 24 and 36 months post-distribution and tested in the hut trials together with ITNs from the cluster-RCT. A full report of the durability study describing the longevity and fabric integrity of the cluster-RCT nets is under preparation.
Experimental hut trial treatments
Experimental hut trials were performed to compare the entomological efficacy of Interceptor®, PermaNet® 3.0, Royal Guard® and Interceptor® G2 ITNs withdrawn from households at 12, 24 and 36 months post-distribution. At each annual time point, the efficacy of the field-aged ITNs was compared to new, unused nets of each type and an untreated net as a negative control. A total of 54 replicate field-aged ITNs and 6 new ITNs of each type were tested at each annual time point in 1 or 2 replicate hut trials and rotated within the treatment daily. Before each hut trial, the mean hole index of the field-aged nets was measured for each ITN type following WHO guidelines (17). To simulate wear-and-tear from routine operational use, all new ITNs and untreated control nets were given 6 holes measuring 4 x 4 cm – two on each of the long side panels and one on each of the short side panels – as per WHO guidelines (17). The nets were erected inside the experimental huts by tying the edges of the roof panel to nails fixed at the upper corners of the hut wall using string. The following treatment arms were evaluated in each experimental hut trial:
1. Untreated net (control) – 6 replicates
2. Interceptor® (new) – 6 replicates
3. Interceptor® (field-aged 12, 24 or 36 months) – at least 54 replicates
4. PermaNet® 3.0 (new) – 6 replicates
5. PermaNet® 3.0 (field-aged 12, 24 or 36 months) – at least 54 replicates
6. Royal Guard® (new) – 6 replicates
7. Royal Guard® (field-aged 12, 24 or 36 months) – at least 54 replicates
8. Interceptor® G2 (new) – 6 replicates
9. Interceptor® G2 (field-aged 12, 24 or 36 months) – at least 54 replicates
Experimental hut trial procedure
The field-aged nets were evaluated in experimental huts the same year they were withdrawn. The hut trials were performed at the same hut site from May to September 2021, April to June 2022 and May to July 2023, with nets withdrawn at 12, 24 and 36 months, respectively. Each trial continued for one full treatment rotation (54 nights over 9 weeks) except for at 12 months, when two consecutive rotations were performed to increase mosquito sample sizes. Treatments were rotated between experimental huts weekly according to a Latin square design to control for the hut position effect, while volunteers were rotated daily to control for differences in individual host attractiveness to mosquitoes. Mosquito collections were performed 6 days per week; on the 7th day, the huts were cleaned and aired to prevent contamination before the next rotation cycle.
Mosquito collections and processing
Nine (9) consenting human volunteers slept in experimental huts from 21:00 to 06:00 during each trial to attract wild, free-flying mosquitoes. Each morning of the trial, volunteers collected all mosquitoes from the different compartments of the hut (under the net, room, and veranda) and deposited them in labelled plastic cups using a torch and aspirator. Mosquito collections were then transferred to the field laboratory for morphological identification and scoring of immediate mortality (live/dead) and blood-feeding (unfed/blood-fed). Live female mosquitoes identified as Anopheles gambiae sensu lato (s.l.) were retained in holding cups and provided access to cotton wool soaked in 10% (w/v) glucose solution. Delayed mortality was recorded every 24 h up to 72 h after collection for all treatments. Mortality after 72 h was used as the primary outcome measure to account for the delayed action of CFP (27) and to provide a single efficacy estimate for killing effects. To evaluate the impact of Royal Guard® on mosquito reproduction, subsamples of surviving blood-fed mosquitoes were dissected to observe ovary development and score fertility according to Christophers’ stages of egg development (28). Mosquitoes were classified as fertile if eggs had fully developed to Christophers’ stage V and infertile if eggs had not fully developed and remained at stages I–IV.
Experimental hut trial outcome measures
The primary outcome measures used to express the efficacy of the experimental hut treatments against pyrethroid-resistant An. gambiae s.l. and compare the impact of the next-generation ITNs to the pyrethroid-only net – Interceptor® – were:
i. Mortality (%) – the proportion of dead mosquitoes 72 h after collection
ii. Fertility (%) – the proportion of dissected mosquitoes scored fertile
The secondary outcome measures used to express the efficacy of the experimental hut treatments against pyrethroid-resistant An. gambiae s.l. were:
i. Entry (n) – the number of mosquitoes collected inside the hut.
ii. Deterrence (%) – the reduction in entry in the treated hut relative to the untreated control hut. Calculated as follows:
Where Tu is the number of mosquitoes entering the untreated control hut and Tt is the number of mosquitoes entering the treated hut.
iii. Exophily (%) – exiting rates due to the potential irritant effect of treatments expressed as the proportion of mosquitoes collected in the veranda.
iv. Blood-feeding (%) – the proportion of blood-fed mosquitoes.
v. Blood-feeding inhibition (%) – the reduction in the proportion of blood-fed mosquitoes in the treated hut relative to the untreated control hut. Calculated as follows:
Where Bfu is the proportion of blood-fed mosquitoes in the untreated control hut and Bft is the proportion of blood-fed mosquitoes in the treated hut.
vi. Reduction in fertility (%) – the reduction in the proportion of dissected mosquitoes scored as fertile for a given treatment compared to the control. Calculated as follows:
Where Fu is the proportion of fertile mosquitoes in the untreated control hut and Ft is the proportion of fertile mosquitoes in the treated hut.
Monitoring susceptibility of the Covè vector population
To monitor the resistance profile of the Covè vector population over time, we performed WHO tube tests and bottle bioassays in the same year of each experimental hut trial (2021, 2022, 2023) to assess the susceptibility to the AIs in the study ITNs and support interpretation of the results. Mosquitoes were exposed to filter papers treated with the discriminating concentrations of alpha-cypermethrin (0.05%) and deltamethrin (0.05%) in tube tests and to bottles coated with the discriminating concentrations of CFP (100 µg/bottle) and PPF (100 µg/bottle) to assess susceptibility to these insecticides. Pyrethroid resistance intensity was investigated by exposing mosquitoes to 5x (0.25%) and 10x (0.50%) the discriminating concentrations of alpha-cypermethrin and deltamethrin. Finally, PBO synergism and the contribution of overexpressed cytochrome P450 monooxygenases (P450s) to pyrethroid resistance were assessed by pre-exposing mosquitoes to the discriminating concentrations of alpha-cypermethrin (0.05%) and deltamethrin (0.05%) with pre-exposure to PBO (4%). Filter papers used for WHO tube tests were procured from the Universiti Sains Malaysia. Test bottles for WHO bottle bioassays with CFP and PPF were prepared according to WHO guidelines (29).
Mosquitoes used for the bioassays were collected as larvae from breeding sites near the experimental huts and subsequently reared to adulthood. At each time point, at least ~100 mosquitoes were exposed to each treatment for 60 mins in four replicates of approximately 25 per tube/bottle. Unfed mosquitoes aged 3–5 days were used for pyrethroid and CFP exposures, while for PPF, blood-fed mosquitoes aged 5–7 days were used to facilitate oogenesis and allow for assessment of the impact of PPF on mosquito reproduction. Parallel exposures were performed with filter papers impregnated with silicone oil, PBO (4%) alone and acetone-coated bottles as controls. At the end of exposure, mosquitoes were transferred to untreated containers and provided access to cotton wool soaked in 10% (w/v) glucose solution. Mortality was recorded after 24 h for the pyrethroid exposures and every 24 h up to 72 h for the CFP and PPF exposures. To assess PPF susceptibility, surviving mosquitoes exposed to PPF and the corresponding negative control were dissected after delayed mortality recording to observe ovary development using a compound microscope and score fertility according to Christophers’ stages of egg development (28, 30). Mosquitoes were classified as fertile if eggs had fully developed to Christophers’ stage V and infertile if eggs had not fully developed and remained at stages I–IV.
Chemical analysis of net pieces
Net pieces measuring 30 x 30 cm were cut from new and field-aged nets at each annual time point at positions outlined in WHO guidelines (22). After cutting, the net pieces were labelled, wrapped in aluminium foil, and stored in a refrigerator at 4±2°C to prevent AI migration in the fabric. The net pieces were then sent to Centre Walloon de Recherches Agronomiques, Belgium, for chemical analysis to measure changes in total AI content over their lifespan. The analytical methods used – which were based on those recommended by the Collaborative International Pesticides Analytical Council – have been described previously (25, 31).
Data management and analysis
For experimental hut trial data, the total number of alive/dead, blood-fed/unfed and fertile/infertile mosquitoes in the different compartments of the hut was pooled with each treatment across the various trials to calculate different proportional outcomes (72 h mortality, blood-feeding, exophily, inside the net, fertility) with corresponding 95% confidence intervals (CIs). Differences between treatments for these proportional binary outcomes were analysed using logistic regression, while differences in count outcomes (entry) were analysed using negative binomial regression. Because two treatment rotations were performed at 12 months and some treatments were tested across multiple trials, the analysis for mosquito entry was adjusted to account for the number of days each treatment was tested. New ITNs were also analysed for each outcome to generate a single estimate across all time points. In addition to the primary explanatory variable of treatment, each model included hut, sleeper, trial period, ITN hole index and day as fixed effects to control for variation associated with differences in individual sleeper and hut attractiveness, seasonality, net condition and overdispersion. The regression analyses generated adjusted odds ratios (ORs) with corresponding 95% CIs, which were used to assess the impact of the next-generation ITNs on the primary outcomes of mosquito mortality and fertility compared to the pyrethroid-only net, Interceptor®. P-values derived from the models were also used to assign compact letter displays denoting the statistical significance of all pairwise comparisons at the 5% level for both primary and secondary outcomes. All regression analyses were performed in Stata version 18.
Post-hoc simulation-based power analyses were performed separately after each experimental hut trial using the ‘power_calculator_ITN’ function in R version 4.3.2. These analyses estimated that the power to detect a significant difference in mosquito mortality after 72 h between field-aged Interceptor® and field-aged Interceptor® G2 was 99.8%, 95% CI: [99.3, 100] at 12 months, 96.7%, 95% CI: [95.4, 97.7] at 24 months and 86.4%, 95% CI: [84.1, 88.5] at 36 months.
The susceptibility of the Covè vector population was interpreted based on observed mortality and fertility rates in tube and bottle bioassays according to WHO guidelines (29). The chemical analysis results demonstrating the total AI content in ITN pieces were used to calculate the proportional AI retention in field-aged nets compared to new nets at each annual time point. All data was recorded by hand on standardised forms before double entry into databases in Microsoft Excel.
Ethical considerations
Approval for the conduct of the experimental hut trials involving human volunteers was obtained from the ethics review boards of the Ministry of Health in Benin (ref: N°6/30/MS/DC/DRFMT/CNERS/SA), the London School of Hygiene and Tropical Medicine (LSHTM) (ref: N°16,237), and the WHO (ref: ERC.0003153). Informed written consent was obtained from all human volunteers prior to their participation. All volunteers were offered a free course of chemoprophylaxis to mitigate the risk of malaria infection, and a stand-by nurse was available throughout the trials to assess any volunteers presenting with febrile symptoms or an adverse reaction to the test items.