4.1. Forecast window evaluation
Before using the information generated by a numerical weather forecast model, it is important to understand the errors associated with the model's spin-up time in the first few hours and the error related to the loss of accuracy as the forecast moves away from the Initial conditions. Based on the above, a performance evaluation of the wind direction and intensity variables was conducted for different forecast windows. Then, it is possible to establish the order of magnitude of these errors in different time windows and consequently indicate the best model predictability window for the forecasted wind. To this end, Tables 3 and 4 show the results obtained for the MAE index in six different windows applied to the wind speed and direction variables. For objectivity, only the MAE index will be presented in this discussion, but all other results will be made available via supplementary material.
Almost systematically, there was a well-established pattern of performance for both evaluated variables (Tables 3 and 4), in which the closer to the initial condition, the smaller the error associated with wind predictability. Despite the well-known numerical stabilization problem during the first hours of simulation (spin-up), the results demonstrated that the impact of the spin-up is less significant on the model performance than the distancing of the forecast from the initial conditions. This result suggests that the usual recommendation of discarding the first hours of simulation is not the best strategy for the wind in the present system of the numerical prediction of operational weather. Despite the known limitations related to the spin-up and the distance from the initial conditions, the results obtained here encourage the use of the results for the first-time windows, since they are the best. However, the authors point out that these results were verified for the wind variable and the settings determined here for the model. Other model settings and meteorological variables must be evaluated. If the errors of the first and last window are compared, calculating the percentage difference between them [((48-72h / 0-24h) * 100) − 100], significant discrepancies between the forecast windows are highlighted, with percentage differences ranging from 13 to 44% for wind direction and 7 to 23% for wind speed.
Table 3: MAE index for different weather forecast windows for the wind direction (°).
|
Station
|
0-24h
|
03-27h
|
06-30h
|
12-36h
|
24-48h
|
48-72h
|
Error %
|
Arraial (A606)
|
36.77
|
37.02
|
36.77
|
37.21
|
39.64
|
41.58
|
13.1
|
Macaé (A608)
|
36.41
|
36.59
|
37.28
|
37.84
|
39.88
|
43.27
|
18.9
|
São Tomé (A620)
|
25.42
|
25.80
|
25.87
|
26.74
|
28.27
|
33.64
|
32.4
|
Vila Velha (A634)
|
31.22
|
31.35
|
31.65
|
32.26
|
33.15
|
36.44
|
16.7
|
Vitória (SBVT)
|
35.13
|
35.74
|
36.26
|
37.18
|
38.90
|
41.36
|
17.7
|
Enchova (SBEN)
|
21.09
|
21.62
|
21.69
|
22.57
|
24.68
|
28.98
|
37.4
|
Marlim (SBMM)
|
16.63
|
17.17
|
17.65
|
18.49
|
20.39
|
23.49
|
41.3
|
Albacora (SBLB)
|
16.52
|
17.08
|
17.46
|
17.95
|
19.67
|
23.82
|
44.2
|
Table 4: MAE index for different weather forecast windows for the wind speed (m·s-1).
|
Station
|
0-24h
|
03-27h
|
06-30h
|
12-36h
|
24-48h
|
48-72h
|
Error %
|
Arraial (A606)
|
1.55
|
1.57
|
1.56
|
1.56
|
1.60
|
1.67
|
7.59
|
Macaé (A608)
|
1.46
|
1.47
|
1.48
|
1.49
|
1.55
|
1.58
|
8.25
|
São Tomé (A620)
|
1.32
|
1.35
|
1.38
|
1.41
|
1.46
|
1.45
|
10.27
|
Vila Velha (A634)
|
1.20
|
1.18
|
1.19
|
1.21
|
1.25
|
1.31
|
8.98
|
Vitória (SBVT)
|
1.12
|
1.12
|
1.13
|
1.17
|
1.20
|
1.25
|
11.50
|
Enchova (SBEN)
|
1.78
|
1.82
|
1.83
|
1.89
|
2.00
|
2.19
|
23.04
|
Marlim (SBMM)
|
2.03
|
2.07
|
2.09
|
2.12
|
2.21
|
2.41
|
18.35
|
Albacora (SBLB)
|
2.17
|
2.21
|
2.23
|
2.26
|
2.35
|
2.55
|
17.72
|
Although not shown in this document, qualitatively the same results were obtained for the other indices described in the Materials and Methods section. From this point on, only the results obtained for the best forecast window (00-24h) will be considered.
4.2. Statistical indexes evaluation
For wind direction (Table 5), it can be seen from the MAE index that the deviations vary between 16 and 36°, that is, less than half a quadrant (< 45°) at all sites analyzed. It was verified that the smallest errors occurred for offshore sites (in general 50% lower). That is, the errors were amplified in regions with greater directional variability, which was already expected for the continent due to the inhomogeneity of the Earth's surface. Furthermore, the distortions generated by the model in the characterization of the terrain produce more sources of uncertainty. None of the evaluated sites had RMSE values greater than twice the MAE index. This indicates that the errors in the individual records were generally close to the average error, since in the RMSE, errors of greater magnitude have greater weight. Regarding the direction of wind deviation, in most of the evaluated sites (A606, A608, A620, SBVT, and SBEN) the negative values obtained simultaneously for FOEX and bias show that the WRF model tends to produce counterclockwise deviations in relation to the observed data. Among these sites, A606 stood out, in which 68% of the simulated periods the wind deviated counterclockwise, resulting in an average deviation of 12° in this direction. At sites A634 and SBLB, there was a balance between clockwise and counterclockwise deviations, with values close to 0 for both indices. And in just one site, SBMM, the model deviated the wind direction clockwise more frequently. Counterclockwise deviations in the simulated wind direction were also verified for the Metropolitan Region of Rio de Janeiro in most of the evaluated sites (Paiva et al. 2014; Soares da Silva et al. 2023). These results are in accordance with Sandu et al. (2020), which found a systematic backing of the forecasted surface wind direction in comparison to scatterometer observations over the oceans in the Southern Hemisphere. This backing of the winds leads to a smaller wind turning angle between the predicted surface wind and the predicted geostrophic wind than what is found in observations (Sandu et al. 2020). Specifically for the WRF model, studies have shown that the model tends to be too geostrophic (Jimenez et al. 2016; Martin et al. 2018).
Table 5
Statistical indices for forecasting the wind direction (°) obtained during the 0-24h window.
Station
|
FOEX
|
MAE
|
Bias
|
RMSE
|
Arraial (A606)
|
-18.28
|
36.77
|
-12.54
|
53.45
|
Macaé (A608)
|
-13.84
|
36.41
|
-9.72
|
53.13
|
São Tomé (A620)
|
-12.35
|
25.42
|
-7.19
|
41.74
|
Vila Velha (A634)
|
0.32
|
31.22
|
1.47
|
48.11
|
Vitória (SBVT)
|
-14.72
|
35.13
|
-11.55
|
52.38
|
Enchova (SBEN)
|
-7.30
|
21.09
|
-5.87
|
34.81
|
Marlim (SBMM)
|
11.14
|
16.63
|
3.13
|
28.96
|
Albacora (SBLB)
|
2.31
|
16.52
|
-0.03
|
29.89
|
Regarding wind speed (Table 5), MAE varied between 1.12 to 2.17 m·s− 1 and RMSE values between 1.45 to 2.66 m·s− 1. As mentioned for wind direction, the close values for MAE and RMSE obtained for wind speed demonstrate that, in general, the largest errors are not as frequent and/or do not differ significantly from the MAE. It is also notable that the largest deviations occurred overall in the three offshore sites evaluated, however, this information is not enough to state that the model has poor performance in this region, since the order of magnitude of the deviations is proportional to the order of magnitude of the variable. This is the situation that occurs at the evaluated sites over the sea, which were characterized by stronger winds due to less roughness in relation to the land surface. As it is normalized, the NMSE index allows for a fairer comparison among the sites, without distortions regarding the order of magnitude of the measurements. Thus, the lowest NMSE values at the offshore sites demonstrated the model's greater ability to simulate wind speed in this region, while the most difficult site was A608. Regarding the sign of deviations, the FOEX and bias indices indicated that beyond the A606 site, the model tended to underestimate the data observed at offshore sites. The SBMM and SBLB sites stood out, where more than 82% of the simulated cases were underestimated. On the contrary, the sites overestimated by the WRF were A608, A620, and A634, and the only sites without a significant trend were in SBVT. Finally, Pearson's correlation coefficient (r) reinforces the better performance of the WRF over the offshore region, with a performance of up to 0.85 at SBLB, against 0.59 at the worst, at A608.
Although it is complex to compare the results obtained in this study with similar ones carried out in the scientific literature for coastal and/or offshore regions in the world, we did this exercise. From this comparison, it was verified that the results of the present study are compatible and/or even superior to most of the reviewed studies (Carbonell et al. 2013; Chadee et al. 2017; Chang et al. 2015; de Assis Tavares et al. 2022; Hahmann et al. 2020; Li et al. 2021; Mattar and Borvarán 2016; Reddy et al. 2022; Salvação and Guedes Soares 2018; Soares da Silva et al. 2023; Tuy et al. 2022). For verification purposes, a summary (Table 7) with the best and worst results obtained in the cited studies is available in the appendix.
Table 6
statistical indices for forecasting the wind speed (m·s− 1) obtained during the 0-24h window.
Station
|
FOEX
|
MAE
|
Bias
|
NMSE
|
RMSE
|
r
|
Arraial (A606)
|
-10.83
|
1.55
|
-0.58
|
0.21
|
1.96
|
0.69
|
Macaé (A608)
|
31.42
|
1.46
|
1.46
|
0.45
|
1.83
|
0.60
|
São Tomé (A620)
|
22.24
|
1.32
|
0.85
|
0.17
|
1.67
|
0.74
|
Vila Velha (A634)
|
18.75
|
1.20
|
0.70
|
0.22
|
1.53
|
0.71
|
Vitória (SBVT)
|
-1.00
|
1.12
|
-0.07
|
0.16
|
1.45
|
0.76
|
Enchova (SBEN)
|
-20.04
|
1.78
|
-0.77
|
0.11
|
2.29
|
0.79
|
Marlim (SBMM)
|
-32.67
|
2.03
|
-1.60
|
0.11
|
2.55
|
0.82
|
Albacora (SBLB)
|
-35.54
|
2.17
|
-1.83
|
0.12
|
2.66
|
0.85
|
Although seasonal statistical results are not presented in the body of the document, they are also available in the appendix (Table 8 to Table 15). The model generally performed better during the winter period and worse during the summer. It is believed that this occurred because during winter the predominance of SASA is even greater, since in this season this synoptic system intensifies and expands westward, while in summer the opposite occurs (Reboita et al. 2019). That is, in winter, the study region is less influenced by transient atmospheric phenomena and, consequently, there is less variability in the near-surface atmospheric flow. Regarding continentality, it was found that the best performances for the seasonal results were observed for offshore sites.
As for the errors in relation to the distribution of winds (Fig. 2), it was observed that the smallest MAE (best results) refers to the wind speed classes with the highest frequency of occurrence, and the largest MAE (worst results) to the most extreme classes, where stood out the most intense wind classes for the worst results. This same pattern was also indicated by the bias index. However, this index showed another interesting pattern, in which the WRF model tends to underestimate the more intense observed winds and overestimate the less intense winds. Specifically at offshore sites, the absolute errors were higher for more intense wind classes, where on average the wind is underestimated in the order of 6 m·s− 1 for observed winds stronger than 13.8 m·s− 1. For an evaluation and comparison of the model performance among the different classes of wind magnitudes, it is also interesting to analyze it from the perspective of the NMSE index. As it is a normalized index, the order of magnitude of the data becomes irrelevant in relation to the proportional distance between the observed and predicted data. For example, considering the differences (10 − 6) and (4 − 2) between predicted and observed, we have results of 4 and 2 respectively. Although the first pair (P-O) represents a greater error in absolute terms, proportionally it represents a significantly smaller error than the second, the first being 40% in relation to what was observed and the second 100%. Thus, according to the results for the NMSE, the worst performances were observed for the less intense classes, a result that differs from those obtained for MAE and bias, whereas in these last indices, the worst results were obtained for the more intense classes. It is worth noting that the performance of the model for the two less intense classes according to the NMSE was significantly lower than for the other classes, where it remains between 0 and 1.
4.3. Frequency Analysis - wind roses map
In general, the wind roses show more frequent near-surface winds from the first quadrant (0–90°). This means that the wind was predominantly driven by the SASA (Franchito et al. 2008; Dereczynski and Menezes 2017; Dragaud et al. 2019; Correia Filho et al. 2021) (Fig. 3). The results obtained with the WRF presented the same predominant directional pattern in the observed data (Fig. 3). This coherence was also verified in the seasonal results (Fig. 4, Fig. 5, Fig. 6 and Fig. 7). There is also an expressive frequency of SE to SW winds present in the observations and in the WRF model results. They are related to transient weather systems (Stech and Lorenzzetti 1992; Dourado and Oliveira 2000; Palóczy et al. 2014; Dereczynski and Menezes 2017). Regarding wind speed, it should be noted that the most intense winds were observed at offshore stations, where the greatest differences in wind speed between observed and modeled data were found (Fig. 3). It is notable that the WRF tends to underestimate the offshore observations, however, a satisfactory consistency between the wind speed for the continental sites was noted.
A great diversity of wind directions over land stands out with an increase in the frequency of western quadrant (225–315°) winds during autumn, when the winds were weaker in the offshore region (Fig. 5). It is noteworthy that the model represented this feature (Fig. 5). During this season, the frequency of frontal systems is reduced (Bonnet et al., 2018), and the SASA winds are weaker (Dereczynski and Menezes 2017). Therefore, as the region is under the weak influence of synoptic scale systems, the local thermal circulations can be preponderant, and the western quadrant winds are due to the land breeze occurrence (225–315°).
Although overall the model has satisfactorily represented the prevailing winds, certain deviations in some seasons are also notable. These differences are probably associated with smaller-scale phenomena, such as the influence of land-sea breezes. This is most evident at the A606 site, where the coastline orientation does not provide perfect synergy with synoptic forcings such as the SASA. It is verified for the A606 site that the W and E winds, associated with the influence of the land-sea breeze in the region (Franchito et al. 2008; Dragaud et al. 2019; Correia Filho et al. 2021), were represented by the model with a frequency of occurrence considerably lower than observed data (Fig. 3). Another notable difference between the observed and modeled data refers to the E winds recorded at the SBEN station (Fig. 3). As verified, the E winds occurred predominantly during the winter period (Fig. 6). A deeper analysis was carried out to understand when and under what meteorological conditions the E winds occurred in the SBEN site, and which were not represented by the WRF model. Basically, the E winds were concentrated at the end of winter (i.e., September), at the transition period to spring, when there was an increase in the frequency of occurrence of synoptic scale transient systems such as fronts and troughs associated with shortwave disturbances. The effect of these systems in the SBEN station region was not adequately represented by the model.
From the wind roses, a spatial pattern was seen where there is a clockwise trend in the prevailing wind direction (i.e., from North to East) as one moves towards higher latitudes (Fig. 3). This feature was more evident at continental stations and was verified for both observed and modeled data. Although the SASA is an anticyclone with a counterclockwise flow, the near-surface wind has a clockwise flow close to the coast. This pattern was evidenced by the wind roses (Fig. 3) and is expressed through the negative wind curl mentioned in the literature (Castelao and Barth 2006; Castelao 2012; Mazzini and Barth 2013). Although the negative wind curl occurrence in the study region was shown in the literature (Castelao and Barth 2006; Castelao 2012; Mazzini and Barth 2013), their cause was not discussed. Distinct mechanisms can contribute to this horizontal wind variability, including the orography, coastline shape, land-sea drag difference, and air-sea interaction due to sea surface temperature gradients (Renault et al. 2016). About the orography, most of the study region is characterized by an extensive lowland area. However, the Serra do Mar Mountain range has an NE to E regional change which can influence the atmospheric flow (Fig. 1). Following the orography, the coastline abruptly changes from NE to E in Cabo Frio, where the A606 weather station is located (Fig. 1). These factors can influence the near-surface wind. The first quadrant (0–90°) near-surface offshore atmospheric flow when approaching the shoreline and the Serra do Mar is blocked and changes direction becoming more zonal at higher latitudes (bigger than around 21.6°S). In a coastal region, the wind over the continent is generally less intense than over the sea due to the rougher land surface. This effect also influences the coastal marine atmospheric boundary layer. A typical feature is a weakening of the wind over the marine region close to the coast and it is called a wind drop-off (Capet et al. 2014; Bravo et al. 2016; Renault et al. 2016; Monteiro and Vogelzang 2019). The land-sea drag difference acts as a barrier to wind intrusion from the sea onto the land through turbulent momentum flux divergence, modifying the wind direction as well (Renault et al. 2016). As a result, the weaker coastal wind with respect to the offshore induces a wind curl (Capet et al. 2014; Bravo et al. 2016; Renault et al. 2016). Another factor that can modify the wind curl and has already been investigated in the study region is the ocean-atmosphere interaction through the influence of the sea surface temperature front generated by the upwelling (Dragaud et al. 2019; Castelao 2012). The colder upwelled sea surface temperatures induce a reduction in the near-surface winds (Dragaud et al. 2019; Castelao 2012), because they stabilize the marine boundary layer and decouple the near-surface winds from the more intense winds aloft (Hashizume et al. 2002; Small et al. 2008). The predominantly NE winds become less intense near the coast over the cold waters due to the sea surface temperature front, inducing negative anomalies in the wind stress curl (Castelao 2012). Therefore, the trend of clockwise modification in the predominant wind direction as one moves towards higher latitudes is a result of distinct physical mechanisms such as orography, land-sea drag difference, coastline shape, and air-sea interaction.