A simple measure of manifold structure is the correlation among the members of the set (ensemble). Correlation of states (globally averaged 850mb zonal wind) for 54 years (1950-2003) during certain (pre-monsoon) period with reference states taken from during the monsoon season (June 1, July 1 and August 1) reveals this structure (Fig. 1, left panel). In particular, the states between 18th March-15th April and between 15th April -15th May are significantly correlated with at least two of the reference states. In contrast, there is little such correlation of states in the winter season. (Fig. 1, right panel). Similarly, the average correlation of the states belong to CE with states of June 1, July 1 and August 1 are respectively 0.06, 0.16 and 0.15 (Fig. 1, left panel); thus these states do not have any coherence with the monsoon state. In other words, certain states during pre-monsoon period have better coherence with monsoon states and can be expected to provide better skill in capturing monsoon dynamics. The same analysis averaged over a domain 0-30ON and 65-95OE) is presented in Fig. 2.
We then consider eight such sub-manifolds of the initial manifold, each characterized by eight members (leads) spread over a period of about fifteen days. As our null hypotheses we consider two ensembles, one is a compact ensemble (CE) of eight leads of closely packed states (April 23-May01) and the other a large ensemble (LE) of states from all the eight test ensembles (March 01-April 30). Thus, the CE, due to its short time span, can not embed the dynamical coherence inherent in an initial manifold characterized by low frequency ISO; the CE is thus akin to a collection of unrelated (synoptic) states. This is clear from the fact that none of the states of the eight leads in the CE has a significant correlation with any of the reference states (Fig. 2). In other words, it is expected that the better sampling of initial states over ISO time scale generally compensates error due to the longer lead and in turn improves the forecast using ensemble with shorter lead and small sampling time scale. A measure of difference among the ensembles is the standard deviation among the members (normalized to ensemble mean) and the same is presented in Fig. 3. It can be seen (Fig. 3) that the initial manifolds with wider spread have larger internal structure than a compact ensemble. Further, there is a gradual decrease in the richness of this structure beyond set 4 (March 18-April 15; Table 1). Thus, the ensembles with longer leads have the higher dispersion of states. Based on the results of Fig. 1 and Fig. 3, therefore, we expect the ensemble states between March 18 – April 15 to have highest skill and shall be referred to as optimum initial manifold (OIM) in subsequent discussion. It needs to be emphasized that the number of states in an ensemble should be chosen carefully to ensure that the results are stable with respect to the size of the ensemble; however, it has been shown that for the model configuration, dispersion among the forecasts from different ensembles saturates at an ensemble size of 6 (Goswami and Gouda, 2009). It may be mentioned that in terms of climatological seasonal cycle, most ensembles performed comparably well (Fig. 4). However, in terms of inter-annual variability in area-averaged (75-85oE, 8-28oN) seasonal (June-August) rainfall, defined as departure from corresponding 24 years (1980-2003) mean, OIM outperforms both CE and LE (Fig. 5). The OIM has a phase synchronization of 67% with a correlation coefficient of 0.44 between all-India seasonal (JJA) rainfall anomalies, significant at 99% confidence level for the degrees of freedom involved.
The model simulations are compared with the coupled models from international centres like Asia Pacific Economic Cooperation (APEC) Climate Center (APCC) and National centre for environmental prediction. The Seamless Coupled Prediction System (SCoPS) of APCC (Ham et al. 2019) and the Climate Forecast System (CFSV2) of NCEP (Saha et al. 2010; 2014) are used for the comparison of rainfall simulations with the IMD observation. We have used the 3-6 month lead predictions i.e. March lead for the months of June-August for the year 1982 to 2003 and compared the inter-annual variability of seasonal (June-August) rainfall over continental India using the APCC and NCEP long-range simulations and IMD observations as presented in the figure S1which clearly indicates the simulation with APCC model has zero correlation and only 45% phase synchronization while the NCEP (CFSv2) has 0.43 correlation with 59% of Phase which is lower compared to the OIM used in our study as mentioned earlier.
A number of other parameters have been considered to quantify the skill of the forecasts (Table 1). Comparison of skill of the forecasts for different initial manifolds in terms of these parameters shows OIM to have highest and significant skill (Table 1). The total number of failures (NUW+NOW+NM+NFF+NFD) for OIM is 9, followed by 13 for the ensemble with starting initial state of March 01; However, this ensemble scores only 54% in terms of phase synchronization (Table 1). As the phase synchronization is a binary (i.e. 0 or 1) process, the random forecast would be expected to result in a 50% success rate. Thus, this ensemble scores only marginally better than a random forecast.
An important consideration in evaluation of skill is the performance for extreme years. Examination of the skill for the years with amplitude of anomaly more than 5% at different scales shows (table 2) OIM to perform significantly better than most of the other ensembles; only two ensembles have higher (by one ) cases of extreme years in phase.
It is important to note that a larger ensemble (LE), with initial states including those of the optimum ensemble, does not provide a better forecast; the optimum initial manifold has significantly better skill than the LE. To measure the effectiveness of ensemble forecasting, the ratio of error (ε) in the ensemble forecast and the average forecast error from the ensemble members are computed and presented in figure 6. The initial manifolds 3-5 have among the lowest values of this error ratio; on the other hand, the CE ensemble has this ratio higher than one, while for the LE, this ratio is comparable to that of OIM.