To facilitate rapid and precise predictions of pile bearing capacity, a Back Propagation (BP) neural network model has been developed utilizing data sourced from existing literature. The model incorporates several input parameters, including pile length, pile diameter, average effective vertical stress, and undrained shear strength. To enhance the optimization of the BP neural network's hyperparameters, five distinct optimization algorithms were employed: the Sine Cosine Optimization Algorithm (SCA), Snake Optimization Algorithm (SO), Pelican Optimization Algorithm (POA), African Vulture Optimization Algorithm (AVOA), and Chameleon Optimization Algorithm (CSA). The efficacy of the proposed model was validated using a randomly selected, previously unused subset of data and assessed through various evaluation metrics. Furthermore, the prediction outcomes were analyzed in conjunction with the SHAP interpretability method to address the inherent "black box" nature of the model. This analysis allowed for a visualization of the SHAP values associated with the input parameters, thereby elucidating their significance and impact on the predictions of pile capacity. The results indicated that the R² values for the BP-SCA, BP-SO, BP-POA, BP-AVOA, and BP-CSA models were 0.9920, 0.9922, 0.9928, 0.9974, and 0.9943, respectively, with the BP-AVOA model demonstrating the highest accuracy, stability, and predictive performance. The SHAP analysis further revealed that undrained shear strength and average effective vertical stress are the most influential parameters affecting pile bearing capacity, followed by pile length and pile diameter. Overall, the model effectively captures the complex nonlinear relationships among the characteristic parameters, thereby providing a robust foundation for further investigations into pile bearing capacity.
Article
Study on an Interpretable Prediction Model for Pile Bearing Capacity Based on SHAP and BP Neural Networks
https://doi.org/10.21203/rs.3.rs-4962091/v1
This work is licensed under a CC BY 4.0 License
You are reading this latest preprint version
Bearing capacity of piles
BP neural networks
SHAP
interpretable predictive models
optimization algorithms
Pile foundations represent a significant type of foundation extensively utilized across various engineering disciplines, including bridges, transmission lines, and marine structures. These foundations are responsible for supporting not only the vertical loads imposed by the superstructure but also the horizontal loads resulting from environmental factors such as wind, wave action, traffic, and seismic activity. The ultimate bearing capacity of piles serves as a critical parameter for assessing their safety performance, as it is intrinsically linked to both the structural safety and economic viability of the construction. However, the methodologies available for conducting field tests to ascertain the ultimate bearing capacity are relatively limited. Consequently, the precise prediction of the bearing capacity of driven piles is of paramount importance and represents a vital area of research within the field of geotechnical engineering.
In recent years, new technologies like artificial intelligence have been increasingly applied in geotechnical engineering, providing effective solutions to complex challenges in the field. Researchers both domestically and internationally have employed intelligent algorithms to forecast pile bearing capacity (PBC). Momeni, Ehsan et al. [1] introduced a Gaussian Process Regression (GPR) model for estimating PBC, demonstrating its effectiveness in predictions. Armaghani, Danial Jahed et al. [2] developed an intelligent system using an Adaptive Neuro Fuzzy Inference System (ANFIS) combined with a Group Data Processing Method (GMDH), optimized by the Imperial Competition Algorithm (ICA), to predict PBC. Yong et al. [3] created three soft computing methods, including ANFIS and Genetic Programming (GP), for estimating ultimate pile bearing capacity. Sarir, Payam et al. [4] explored various AI techniques to enhance the load-bearing capacity of Concrete Filled Steel Tube (CFST) columns, utilizing methods like Invasive Weed Optimization (IWO) and the Artificial Bee Colony (ABC) algorithm. Alzabeebee, Saif et al. [5] presented a robust model using Multi-Objective Evolutionary Polynomial Regression (MOGAEPR) to calculate pile friction force. Esmaeili-Falak, Mahzad et al. [6] employed a Hybrid Extreme Gradient Boosting (XGBoost) model for precise output estimation, applying four optimization techniques: Black Widow Optimization Algorithm (BWOA), Artificial Hummingbird Algorithm (AHA), Arithmetic Optimization Algorithm (AOA), and Firehawk Optimization (FHO) to fine-tune the model's internal parameters. Zhang et al. [7] proposed a hybrid model that integrates XGBoost with Bayesian Optimization (BO) for predicting the end bearing capacity of rock socket piles. Fattahi, Hadi et al. [8] utilized two optimization algorithms, Fruit Fly Optimization (FFO) and Invasive Weed Optimization (IWO), using parameters such as pile set (S), hammer drop height (H), hammer weight (W), cross-sectional area (A), and length (L) to forecast pile load capacity. Sun et al. [9] estimated the ultimate load capacity of rock-embedded piles using a Support Vector Regression (SVR) model optimized by the Gray Wolf (GWO) algorithm. Nguyen, Tan et al. [10] combined the XGBoost algorithm with a psychology-based optimization (SPBO) method to predict pile base resistance, thoroughly evaluating the performance of the SPBO-XGBoost model against other optimized versions. Millan, M. A. et al. [11] developed an Artificial Neural Network (ANN) to address the issue of pile bearing capacity in rock.
Machine learning, a pivotal domain within artificial intelligence, is experiencing rapid advancements due to ongoing technological progress [12–17]. Since the early 1980s, artificial neural networks have emerged as the most prominent algorithms in artificial intelligence research, serving as the foundational framework for various neural network models. These networks emulate the information processing mechanisms of the human brain. Over recent decades, artificial neural network algorithms have undergone significant development, leading to their extensive application across diverse fields, including healthcare, transportation, image recognition, and civil engineering. Numerous neural network variants have been created, with the Back Propagation Neural Network (BPNN) being one of the most prevalent, recognized for its efficacy in addressing complex problems [18–20]. However, a notable limitation of machine learning is its inherent "black box" nature, which obscures the ability to ascertain the significance of individual influencing factors. Consequently, the physical interpretation of these models remains ambiguous, hindering the elucidation of the relationship between output results and feature parameters based on theoretical frameworks, thereby diminishing the credibility of machine learning methodologies.
To address these limitations, the Shapley Additive Explanations (SHAP) framework offers a promising solution. Proposed by Lundberg and Lee, SHAP serves as an explanatory tool for interpreting black-box models [21–24]. It is grounded in the computation of Shapley values, a concept derived from Coalitional Game Theory (CGT), which quantifies the impact of features on a dependent variable. In this approach, each feature is treated as a contributor, and the model's final prediction is derived from the aggregated contribution of each feature. In this study, the prediction of pile bearing capacity is achieved by developing a BP neural network model utilizing data from existing literature. The model incorporates pile length, pile diameter, average effective vertical stress, and undrained shear strength as input parameters. Additionally, five distinct optimization algorithms—Sine Cosine Optimization Algorithm (SCA), Snake Optimization Algorithm (SO), Pelican Optimization Algorithm (POA), African Vulture Optimization Algorithm (AVOA), and Chameleon Optimization Algorithm (CSA)—are employed to optimize the hyperparameters of the BP neural network. The proposed model is validated using a randomly selected, previously unused subset and assessed through various evaluation metrics. The prediction outcomes are further analyzed by integrating the SHAP interpretability method to mitigate the "black box" challenge associated with the prediction model. The significance and impact of the input parameters on the predicted pile capacity are examined through the visualization of SHAP values corresponding to the input features.
2.1 BP Neural Network
The Backpropagation (BP) neural network model is a multilayer feed-forward architecture that comprises an input layer, one or more hidden layers, and an output layer. This computational algorithm employs error backpropagation as a mechanism for learning (references [18–20]). The BP neural network is structured into three primary components: the input layer, the output layer, and multiple hidden layers. Figure 1 illustrates the schematic topology of the BP neural network, wherein neurons across different layers are interconnected through weighted connections, and each layer is associated with its own threshold. Each neuron performs a linear combination of the outputs from the preceding layer's neurons, followed by a mapping transformation via an activation function. The choice of activation function and the overall network architecture are critical determinants of the model's accuracy. Specifically, the activation function introduces a nonlinear element to the BP neural network; in the absence of such a function, the network would reduce to a linear matrix operation, significantly impairing its expressive capacity. Numerous activation functions exist, and for the purposes of this study, the logsig function has been selected as the primary activation function for the hidden layer due to its computational efficiency and superior performance, which can be mathematically represented by the following formula.
$${\text{logsig(}}x{\text{)}}=\frac{1}{{1+{e^{ - x}}}}$$
1
The presence of uncertainty in the network architecture represents a significant limitation of backpropagation (BP) neural networks. This characteristic adversely affects the network's capacity to generalize, particularly when confronted with a relatively high correlation among input variables. Furthermore, BP neural networks are contingent upon the initial weights and thresholds, a consequence of their reliance on the gradient descent algorithm, which is characterized by a slow convergence rate. When faced with complex optimization challenges, if the initial weights and thresholds are not appropriately aligned with the actual conditions of the network, there is a risk of converging to a local optimum. This phenomenon ultimately compromises the accuracy of predictions.
2.2 Algorithm Optimization
2.2.1 Sine Cosine Algorithm SCA
The Sine Cosine Algorithm (SCA) is an optimization technique that leverages the properties of sine and cosine functions to identify optimal solutions [25]. By emulating the dynamic behavior of these trigonometric functions, the SCA facilitates the exploration of solution spaces through mechanisms such as individual and population interactions, position updates, fitness evaluations, and knowledge transfer. This algorithm demonstrates a notable capacity for global search and convergence, making it applicable to a wide range of optimization challenges.
The fundamental principles underlying the SCA are as follows: 1. **Simulation of Sine and Cosine Functions**: The SCA operates by mimicking the periodic nature of sine and cosine functions, which yield varying outputs for different inputs. This characteristic is harnessed to adjust the positions of individuals in the search for optimal solutions. 2. **Individuals and Population Dynamics**: Within the SCA framework, each individual corresponds to a potential solution, characterized by its position and velocity. The collective of individuals forms a population that collaborates and interacts to enhance the search process. 3. **Position Updating Mechanism**: The algorithm updates the position of each individual in accordance with the sine and cosine function dynamics. The position is influenced by both individual and global optimal solutions, allowing individuals to progressively converge towards the optimal solution through continuous position adjustments. 4. **Fitness Evaluation**: The adaptability of each individual is assessed based on the objective function's value. A higher fitness value indicates that the individual's position is closer to the optimal solution. 5. **Knowledge Transfer and Strategy Updating**: The SCA promotes the exchange and enhancement of knowledge among individuals. By observing the positions and fitness values of their peers, individuals can refine their behavioral strategies, thereby improving their search efficacy.
In summary, the SCA is a robust optimization algorithm that effectively utilizes the mathematical properties of sine and cosine functions to navigate complex solution spaces and achieve optimal or near-optimal outcomes.
2.2.2 Snake optimization algorithm SO
The Snake Optimizer (SO) is a novel intelligent optimization algorithm introduced in 2022, designed to emulate the unique mating behaviors exhibited by snakes (reference [26]). The algorithm operates under two primary conditions: when temperatures are low and food is accessible, mating behaviors are initiated; conversely, in the absence of these conditions, the snakes focus solely on foraging or consuming available food. This dichotomy in behavior leads to a bifurcation in the search process of the Snake Optimization algorithm, which is categorized into two distinct phases: exploration and exploitation. The exploration phase pertains to the environmental factors, specifically cold habitats and food sources, wherein the snake primarily seeks nourishment in its vicinity. The exploitation phase encompasses several transitional stages aimed at enhancing the algorithm's search efficiency. Within this phase, the mating process is further delineated into two states: fighting mode and mating mode. During fighting mode, male snakes engage in competition to secure the most suitable female, while females endeavor to select the optimal male partner. In mating mode, the likelihood of successful mating is contingent upon the quantity of available food. Should mating occur within the search space, it is probable that the female will produce eggs, leading to the hatching of new snakes.
2.2.3 Pelican Optimization Algorithm POA
The Pelican Optimization Algorithm (POA), introduced in 2022, emulates the natural hunting behaviors of pelicans, which are adept at fishing [27]. These birds exhibit a distinctive feeding strategy characterized by hovering above water to observe and identify their prey, followed by rapid descents to capture it. This unique approach served as the foundation for the development of the POA, which aims to replicate the pelican's feeding behavior in addressing optimization challenges. In the initial phase of the algorithm, the pelican identifies the location of its prey and subsequently navigates towards this designated area. By modeling the pelican's strategy for approaching its prey, the POA effectively explores the search space, thereby enhancing its capacity to investigate various regions within the search domain. A critical aspect of the POA is the random generation of prey positions within the search space, which significantly augments the algorithm's exploratory capabilities in tackling optimization problems. In the subsequent phase, upon reaching the water's surface, pelicans extend their wings to create a disturbance that facilitates the upward movement of fish, ultimately capturing the prey in their throat pouch. This surface flight strategy enables pelicans to increase their catch within the targeted area. By modeling this behavioral process, the POA can achieve improved convergence to optimal locations within the hunting domain, thereby enhancing its local search and exploitation capabilities.
2.2.4 African Vulture Optimization Algorithm AVOA
The African Vultures Optimization Algorithm (AVOA) is an optimization technique inspired by the foraging behaviors exhibited by African vultures [28]. This algorithm emulates the collective foraging strategies of these birds, facilitating the search for optimal solutions through both collaboration and competition within a group. The fundamental principles underlying the AVOA are delineated as follows: 1. **Foraging Behavior Simulation**: African vultures employ a combination of collaborative and competitive strategies during foraging. They identify food sources by monitoring and mimicking the actions of their peers, while simultaneously competing for access to these resources. The AVOA algorithm mirrors this foraging behavior, framing it as a search process for optimization problems. 2. **Population Initialization**: The AVOA algorithm initiates the search for optimal solutions with a randomly generated initial population. Each individual within this population is conceptualized as a vulture, with their respective positions representing potential solutions. 3. **Search Process**: During each iteration, a vulture updates its position based on its current location and associated fitness value. The position updating mechanism incorporates two behavioral patterns: Collaborative Behavior: Vultures observe and follow the positions of their peers to identify superior solutions, selecting a nearby vulture as a reference point to guide their movement. Competitive Behavior: Vultures engage in competition for advantageous positions near food sources, assessing the intensity of competition and adjusting their positions in accordance with their fitness values. 4. **Adaptation Evaluation**: Within the AVOA framework, the adaptability of each vulture is assessed based on the objective function's value. A higher fitness value indicates that the vulture's position is closer to the optimal solution. 5.**Group Collaboration and Competition**: The search efficacy of vultures is enhanced through collaborative and competitive interactions among themselves. They refine their behavioral strategies by observing the positions and fitness values of their counterparts.
2.2.5 Chameleon Swarm Algorithm CSA
The Chameleon Swarm Algorithm (CSA) is an optimization technique inspired by the behavioral patterns of chameleons [29]. This algorithm emulates the foraging and social interactions of chameleons to identify optimal solutions through a combination of collaborative and competitive dynamics among individuals. The fundamental principles of the CSA are delineated as follows: 1. **Simulation of Chameleon Foraging Behavior**: Chameleons employ adaptive strategies during foraging, modifying their coloration and actions to align with environmental conditions and locate suitable food sources. The CSA algorithm mirrors this foraging behavior, framing it as the search mechanism for optimization problems. 2. **Population Initialization**: The CSA initiates the search for optimal solutions with a randomly generated initial population. Each individual within this population is conceptualized as a chameleon, with their respective positions and colors symbolizing potential solutions. 3. **Search Process**: During each iteration, chameleons update their positions and colors based on their current state and fitness values. This update process encompasses two behavioral patterns: Collaborative Behavior: Chameleons observe and learn from the positions and colors of their peers to enhance their search for superior solutions. They tend to reference chameleons with similar colors and move towards their locations while simultaneously adjusting their colors to better suit their environment. Competitive Behavior: Chameleons engage in competition for optimal positions and colors near food sources. The intensity of this competition influences their adjustments in position and color, which are informed by their fitness values. 4. **Adaptation Evaluation**: Within the CSA framework, the adaptability of each chameleon is assessed through the objective function's value. A higher fitness value indicates that the chameleon's position and color are closer to the optimal solution. 5. **Group Collaboration and Competition**: The search efficacy of chameleons is enhanced through mutual collaboration and competition. They refine their behavioral strategies by observing the positions, colors, and fitness values of their counterparts.
To develop the proposed model, a comprehensive dataset comprising 65 entries was utilized, which includes four input parameters: pile length, pile diameter, average effective vertical stress, and undrained shear strength. This dataset is derived from experimental findings documented in pertinent literature [30]. A representative sample of the collected data is presented in Table 1, while Table 2 illustrates the correlation between the input and output variables.
| Test ID | \(\:{\text{x}}_{1}\) | \(\:{\text{x}}_{2}\) | \(\:{\text{x}}_{3}\) | \(\:{\text{x}}_{4}\) | y |
|---|---|---|---|---|---|
| 1 | 14.1 | 15 | 96 | 26 | 24.26 |
| 2 | 13 | 15 | 102 | 15 | 21.11 |
| 3 | 11.7 | 20 | 54 | 23 | 19.44 |
| 4 | 17.5 | 14.3 | 87 | 23 | 20.78 |
| 5 | 15.9 | 15 | 49 | 17 | 14.56 |
| 6 | 8.1 | 13.5 | 37 | 13 | 13.92 |
| 7 | 7.7 | 16.5 | 32 | 15 | 14.56 |
| 8 | 10 | 13.5 | 33 | 10 | 11.64 |
| 9 | 12 | 15.5 | 39 | 12 | 12.71 |
| 10 | 10.2 | 22 | 19 | 15 | 12.44 |
| 11 | 24.2 | 15 | 146 | 19 | 25.28 |
| 12 | 17.1 | 15 | 109 | 57 | 36.41 |
| 13 | 12.7 | 23.2 | 38 | 19 | 15.66 |
| 14 | 10 | 17 | 82 | 36 | 28.42 |
| 15 | 14.3 | 26 | 89 | 22 | 23.48 |
| 16 | 22.5 | 47 | 60 | 45 | 24.5 |
| 17 | 5.5 | 30.5 | 44 | 30 | 23.94 |
| 18 | 19.2 | 61 | 142 | 31 | 40.21 |
| 19 | 15.2 | 35.6 | 448 | 104 | 106.91 |
| 20 | 12.2 | 35.6 | 718 | 162 | 164.05 |
| 21 | 43.9 | 30.5 | 162 | 38 | 27.8 |
| 22 | 96 | 61 | 354 | 80 | 46.75 |
| 23 | 73.8 | 61 | 273 | 67 | 44.32 |
| 24 | 22.6 | 76.7 | 651 | 170 | 190.33 |
| 25 | 30.5 | 32.5 | 153 | 45 | 35.04 |
| 26 | 45.7 | 32.5 | 148 | 52 | 26.45 |
| 27 | 13.7 | 32.5 | 112 | 45 | 36.28 |
| 28 | 5.5 | 16.9 | 51.6 | 129.5 | 74.83 |
| 29 | 29 | 33 | 105 | 39 | 26.1 |
| 30 | 12.2 | 16.8 | 33 | 16 | 13.67 |
| 31 | 14 | 35.1 | 59 | 30 | 23 |
| 32 | 39.6 | 27.4 | 297 | 165 | 76.79 |
| 33 | 30.5 | 61 | 91 | 52 | 28.26 |
| 34 | 25.9 | 32.5 | 99 | 61 | 33.14 |
| 35 | 13.1 | 27.4 | 80 | 110 | 59.14 |
| 36 | 20.4 | 61 | 105 | 208 | 92.34 |
| 37 | 9.1 | 45 | 54 | 144 | 74.65 |
| 38 | 16.8 | 61 | 87 | 100 | 51.42 |
| 39 | 13.7 | 32.5 | 112 | 137 | 73.89 |
| 40 | 18.3 | 76.2 | 115 | 335 | 153.65 |
| 41 | 4.6 | 16.9 | 43 | 120.5 | 70.2 |
| 42 | 33.6 | 32.5 | 121.4 | 35.4 | 25.65 |
| 43 | 33.6 | 32.5 | 108 | 48.8 | 26.39 |
| 44 | 20.3 | 32.5 | 158.2 | 112.8 | 63.12 |
| 45 | 30.5 | 51 | 102.8 | 24.4 | 24.02 |
| 46 | 8 | 13.5 | 27 | 9 | 11.11 |
| 47 | 9.4 | 29.3 | 52 | 29 | 23.08 |
| 48 | 14.6 | 16 | 67 | 29 | 21.96 |
| 49 | 11.6 | 17.5 | 57 | 27 | 21.28 |
| 50 | 9.6 | 19.2 | 42 | 15 | 15.3 |
| 51 | 21.6 | 45.7 | 147 | 31 | 37.22 |
| 52 | 36.9 | 30.6 | 149.6 | 28.2 | 27.39 |
| 53 | 66.4 | 32.5 | 223 | 60 | 27.43 |
| 54 | 11.6 | 11.4 | 44 | 21 | 16.86 |
| 55 | 22.9 | 32.5 | 91 | 52 | 30.92 |
| 56 | 13.8 | 19 | 21 | 21 | 13.78 |
| 57 | 25.3 | 27.4 | 244 | 185 | 93.59 |
| 58 | 14.9 | 52.8 | 66 | 53 | 32.57 |
| 59 | 18.3 | 32.5 | 51 | 33 | 20.83 |
| 60 | 48.2 | 61 | 152 | 64 | 31.93 |
| 61 | 32 | 21.4 | 141 | 115 | 49.98 |
| 62 | 13.4 | 27 | 81 | 22 | 22.76 |
| 63 | 24.2 | 15 | 147 | 19 | 25.42 |
| 64 | 15.5 | 17.5 | 80 | 72 | 40.6 |
| 65 | 12.8 | 32.5 | 110 | 96 | 56.55 |
| Inputs and outputs | parameters | variant |
|---|---|---|
| importation | Pile length (m) | \(\:{x}_{1}\) |
| Pile diameter (m) | \(\:{x}_{2}\) | |
| Average effective vertical stress (kpa) | \(\:{x}_{3}\) | |
| Undrained shear strength (kPa) | \(\:{x}_{4}\) | |
| exports | Load capacity (kpa) | y |
In this study, five distinct optimization algorithms are employed to enhance the hyperparameters of the backpropagation (BP) neural network. These algorithms include the Sine-Cosine Algorithm (SCA), Snake Optimization Algorithm (SO), Pelican Optimization Algorithm (POA), African Vulture Optimization Algorithm (AVOA), and Chameleon Optimization Algorithm (CSA). Prior to the optimization process, it is essential to adjust the hyperparameters of each algorithm. Through preliminary calculations, it was observed that an excessively large population size significantly prolongs the computational time, whereas a population size that is too small increases the likelihood of the model converging to a local optimum, thereby hindering the attainment of a global optimal solution. Consequently, the selection of an appropriate population size is critical. In this paper, following extensive trial calculations, a population size of 20 and an iteration count of 30 have been established. Additionally, it is imperative to normalize the initially collected data prior to executing the optimization step. The normalization formula is as follows:
$${{\text{X}}_{{\text{NOR}}}}{\text{=}}\frac{{{\text{2(}}{{\text{X}}_{{\text{NOR}}}}{\text{-}}{{\text{X}}_{{\text{MIN}}}}{\text{)}}}}{{{{\text{X}}_{{\text{MAX}}}}{\text{-}}{{\text{X}}_{{\text{MIN}}}}}}{\text{-1}}$$
2
The equations \(\:{\text{X}}_{\text{MAX}}\)and \(\:{\text{X}}_{\text{MIN}}\)are respectively the variables \(\:{\text{X}}_{\text{I}}\)are the minimum and maximum values of \(\:{\text{X}}_{\text{NOR}}\)is its normalized value.
The dataset is partitioned into three distinct subsets: training, validation, and test sets, with 70% allocated for training, 10% for validation, and 20% for testing. During the model training phase, only the training and validation sets are utilized, and the optimal model is identified based on a specified evaluation metric. The test set remains entirely excluded from the model development process, serving as "novel" data for the model. Consequently, the test set is employed to assess the model's generalization capability, which refers to its proficiency in predicting outcomes under previously unencountered conditions. The primary steps undertaken in this process are illustrated in Fig. 2.
SHAP (SHapley Additive exPlanations) is an interpretable machine learning methodology grounded in the concept of Shapley values, which serves to quantify and visualize the contributions of various features to predictive outcomes. This technique assesses the influence of features and generates a ranked list of their importance. Additionally, SHAP provides visualizations that elucidate the relationships between features and outputs, thereby enhancing the understanding of the model's decision-making process. The foundational principle of SHAP is derived from the Shapley value, a well-established concept in cooperative game theory that measures the contribution of each participant to a collective outcome. In the context of machine learning, the model can be conceptualized as a game, with the features acting as participants. SHAP computes the Shapley value for each feature, thereby quantifying its contribution to the prediction of the target variable.
Assuming that a model is used to predict a dataset containing n instances along with m features, the contribution of each feature to the model output depends on its marginal contribution. Let any instance N={x1,x2,⋯,xm} containing m feature values. There are m elements xi, and any number of feature attribute values form a subset S ⊆ N. The value generated by the elements included in the subset S working together is denoted by v(S). The mean value of the cumulative contribution (marginal contribution) is the final assigned value (Shapley Value) ψi(N,v). For example, if the eigenvalue x1 works alone to produce the value v({x1}), and then joins x2 to produce the value v({x1,x2}), then the cumulative contribution of x2 is v({x1,x2})-v({x1}). For all sequences that can form the full set N, find the cumulative contribution about element xi among them, and then take the mean to get the Shapley Value value of xi. Assuming that the feature full set is F, we have:
S in the formula denotes the set of elements in the sequence that lie in front of xi, and the total number of sequences that satisfy the requirement that only the elements in the set S lie in front of xi is |S|! (|N|-|S|-1)! The cumulative contribution of xi generated in the sequences within them are all v(S
{xi})-v(S), which is finally averaged after summing over all the sequences.
Five distinct methodologies were employed to optimize the relevant control parameters of the Backpropagation (BP) neural network, with Table 3 presenting the final control parameters associated with each method.
|
parameters |
SCA |
SO |
POA |
AVOA |
CSA |
|---|---|---|---|---|---|
|
fitrnet |
6 |
4 |
16 |
17 |
8 |
|
LayerSizes |
22 |
22 |
2 |
4 |
4 |
|
lambda |
0.0011 |
0.001 |
0.0012 |
0.001 |
0.001 |
The efficacy of the five optimization algorithms employed for enhancing the BP neural network model is assessed through both statistical and graphical error criteria. This evaluation takes into account the statistical metrics that are delineated as follows:
1. mean absolute percentage error
$${\text{MAPE}}=\frac{1}{n}\sum\limits_{{i=1}}^{n} {\left| {\frac{{{\eta _{i\exp }} - {\eta _{ipred}}}}{{{\eta _{i\exp }}}}} \right|}$$
4
2. root mean square error
$${\text{RMSE}}=\sqrt {\frac{1}{n}\sum\limits_{{i=1}}^{n} {{{({\eta _{i\exp }} - {\eta _{ipred}})}^2}} }$$
5
3. goodness-of-fit
$${{\text{R}}^{\text{2}}}=1 - \frac{{\sum\limits_{{i=1}}^{n} {{{({\eta _{i\exp }} - {\eta _{ipred}})}^2}} }}{{\sum\limits_{{i=1}}^{n} {{{({\eta _{ipred}} - \overline {\eta } )}^2}} }}$$
6
In the formula, the subscripts "exp" and "pred" indicate the measured and predicted values of load-bearing capacity, respectively. The term represents the average of these values, while "n" signifies the number of data points. The model exhibiting the highest values is considered the most accurate, whereas the Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) should be minimized for optimal performance.
Figure 3 provides a comprehensive comparison of the performance of six distinct backpropagation (BP) prediction models. These include the baseline BP model, the BP-SCA model which integrates the sine-cosine optimization algorithm, the BP-SO model that incorporates the snake optimization algorithm, the BP-POA model utilizing the pelican optimization algorithm, the BP-AVOA model that applies the African vulture optimization algorithm, and the BP-CSA model which employs the chameleon optimization algorithm. The performance of these models is evaluated using the training dataset. The figure illustrates a notable trend wherein the prediction outputs of each model closely align with the actual values throughout the training process. This significant observation underscores the enhanced prediction accuracy and overall effectiveness of the developed BP hybrid models. It further highlights the beneficial impact of various optimization algorithms on improving the predictive capabilities of the models when integrated with the foundational BP framework.
Figure 4 illustrates the fitted predictions for the training set samples, offering a comprehensive comparison of the discrepancies between the model predictions and the actual observed values. This visualization facilitates an evaluation of the performance of various models concerning prediction accuracy, which is primarily determined by the density and precision of the predicted value distribution within a specified space. Notably, a model is deemed highly reliable when its predictions closely align with the reference line of true values, exhibiting minimal deviation. An analysis of the data performance during the training phase reveals that the predicted value curve nearly coincides with the true value curve, indicating a significant level of agreement. This observation not only validates the model's effective learning capability throughout the training process but also underscores the exceptional performance of the implemented BP hybrid model in relation to data distribution characteristics. Specifically, the model demonstrates an ability to accurately capture and replicate the intrinsic patterns of the data, thereby exhibiting favorable distribution characteristics and prediction outcomes.
Figure 5 presents a comparative analysis of the results obtained from the test set for various prediction models, including the BP prediction model, BP-SCA prediction model, BP-SO prediction model, BP-POA prediction model, BP-AVOA prediction model, and BP-CSA prediction model. The graphical representation illustrates the relationship between the actual and predicted values, thereby facilitating a visual assessment of the discrepancies between these values across the different models. This comparative analysis serves as an intuitive framework for evaluating the reliability of each model. In this context, the reliability of a model is determined by the proximity of its predicted values to the actual values, with a high degree of concordance between the prediction curves and the reference line representing the true values typically indicating a robust model. Notably, the BP model exhibits the greatest variability in its prediction curves, whereas the prediction curves of the BP-SCA, BP-SO, BP-POA, BP-AVOA, and BP-CSA models demonstrate a significant reduction in variability. This improvement can be attributed to the optimization of hyperparameters in the BP model through the SCA, SO, POA, AVOA, and CSA optimization techniques, resulting in the attainment of optimal parameter values and a corresponding enhancement in the performance of the prediction models.
Figure 6 illustrates the fitted prediction outcomes for the entire sample data within the test set, offering both an intuitive and quantitative assessment of the model's reliability. This is achieved by visualizing the distribution of predicted values across the unit area, where the density and spatial arrangement of the predicted points serve as direct indicators of the model's high reliability. Notably, the predictions generated by the BP model exhibit considerable dispersion near the unit slip line, indicating a relatively weaker predictive performance. In contrast, the BP-SCA, BP-SO, BP-POA, BP-AVOA, and BP-CSA models demonstrate a more concentrated distribution of predictions in proximity to the unit slip line, thereby highlighting their substantial advancements in improving prediction accuracy and optimizing model reliability.
For a comprehensive assessment and comparison, the statistical metrics employed, specifically the Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and Goodness of Fit (R²) for the BP Hybrid Methods test dataset, are detailed in Table 4. The optimal MAPE values for the various prediction models are as follows: the BP-SCA model exhibits a MAPE of 0.0410, the BP-SO model has a MAPE of 0.0381, the BP-POA model demonstrates a MAPE of 0.0266, the BP-AVOA model achieves a MAPE of 0.0173, and the BP-CSA model records a MAPE of 0.0392. The corresponding R² values for these models are 0.9920, 0.9922, 0.9928, 0.9974, and 0.9943, respectively. Notably, the BP-AVOA prediction model exhibits the lowest MAPE among the five models, indicating superior performance. Additionally, the RMSE for the BP-AVOA model is 1.0407, which is also the lowest across all models, further underscoring its high level of fit. These findings suggest that the BP-AVOA prediction model demonstrates the highest accuracy, stability, and predictive capability, while the BP-CSA, BP-POA, BP-SO, and BP-SCA models rank as the next best in terms of predictive performance.
|
Forecasting methodology |
MAPE |
RMSE |
R² |
|---|---|---|---|
|
BP |
0.0419 |
3.5617 |
0.9698 |
|
BP-SCA |
0.0410 |
1.8312 |
0.9920 |
|
BP-SO |
0.0381 |
1.8138 |
0.9922 |
|
BP-POA |
0.0266 |
1.7428 |
0.9928 |
|
BP-AVOA |
0.0173 |
1.0407 |
0.9974 |
|
BP-CSA |
0.0392 |
1.5448 |
0.9943 |
Machine learning models are often characterized as "black box" systems, wherein the final output is accessible, but the influence of input features on the prediction remains obscured. The SHAP (SHapley Additive exPlanations) model provides a mechanism to elucidate the effects of these input variables on the ultimate prediction outcomes, revealing that different feature variables exert varying degrees of influence on the predictions. Given the multitude of factors that can affect pile bearing capacity, it is imperative to conduct a thorough analysis of how these factors contribute to the characteristics of pile bearing capacity.
In the SHAP feature map (Fig. 7), each point corresponds to an actual sample, with the vertical axis representing the SHAP value, which quantifies the contribution and impact of each influencing factor on the model's predicted value. A positive SHAP value for a feature indicates that it exerts a beneficial effect on the model's predictions; specifically, an increase in this feature's value (while holding other features constant) will result in an increase in the predicted value. Conversely, a negative SHAP value signifies a detrimental effect on the predictions; thus, an increase in this feature's value (while other features remain unchanged) will lead to a decrease in the predicted value. The analysis reveals that the most significant factors affecting pile capacity are pile length (x1), pile diameter (x2), average effective vertical stress (x3), and undrained shear strength (x4), with undrained shear strength (x4) and average effective vertical stress (x3) being the most critical determinants of pile capacity.
The SHAP algorithm, in conjunction with the BP model, was employed to assess the significance of four influencing factors, resulting in the importance ranking diagram depicted in Fig. 8. The analysis reveals that the factors of undrained shear strength (x4) and average effective vertical stress (x3) exert the most substantial influence on pile bearing capacity, followed by pile length (x1) and pile diameter (x2).
This study aims to predict the bearing capacity of piles by developing a Back Propagation (BP) neural network model utilizing data sourced from existing literature. The model incorporates several input parameters, including pile length, pile diameter, average effective vertical stress, and undrained shear strength. To enhance the optimization of the BP neural network's hyperparameters, five distinct optimization algorithms are employed: the Sine Cosine Optimization Algorithm (SCA), Snake Optimization Algorithm (SO), Pelican Optimization Algorithm (POA), African Vulture Optimization Algorithm (AVOA), and Chameleon Optimization Algorithm (CSA). The proposed model undergoes validation through a randomly selected, previously unused subset of data and is assessed using various evaluation metrics. To address the "black box" nature of the prediction model, the SHAP (SHapley Additive exPlanations) interpretability method is applied, allowing for an analysis of the significance and impact of the input parameters on the predicted pile capacity. The visualization of SHAP values for the input characteristics reveals that the R² values for the BP-SCA, BP-SO, BP-POA, BP-AVOA, and BP-CSA models are 0.9920, 0.9922, 0.9928, 0.9974, and 0.9943, respectively, indicating that the BP-AVOA model exhibits the highest accuracy, stability, and predictive performance. The SHAP analysis further indicates that undrained shear strength and average effective vertical stress are the most influential parameters affecting pile bearing capacity, followed by pile length and pile diameter. This model effectively captures the intricate nonlinear relationships among the characteristic parameters, thereby providing a foundational basis for further investigations into pile bearing capacity.
9. Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
10. Data availability
All data generated or analysed during this study are included in this article.
11. Author Contributions
Conceptualization, Shunbo Li; methodology, Mingwei Hai; software, Mingwei Hai; validation, Shunbo Li; formal analysis, Shunbo Li; investigation, Shunbo Li; resources, Mingwei Hai; data curation, Mingwei Hai; writing—original draft preparation, Shunbo Li, Mingwei Hai; writing—review and editing, Qi Zhang,; visualization, Qi Zhang; supervision, Bin Zhou, Zhuo Zhao; project administration, Qi Zhang.
- Momeni E, Dowlatshahi M B, Omidinasab F, et al. Gaussian process regression technique to estimate the pile bearing capacity. ARAB J SCI ENG. 45, 8255-8267 (2020).
- Armaghani D J, Harandizadeh H, Momeni E, et al. An optimized system of GMDH-ANFIS predictive model by ICA for estimating pile bearing capacity. ARTIF INTELL REV. 1-38 (2022).
- Yong W, Zhou J, Jahed Armaghani D, et al. A new hybrid simulated annealing-based genetic programming technique to predict the ultimate bearing capacity of piles. ENG COMPUT-GERMANY. 37, 2111-2127 (2021).
- Sarir P, Shen S L, Wang Z F, et al. Optimum model for bearing capacity of concrete-steel columns with AI technology via incorporating the algorithms of IWO and ABC. ENG COMPUT-GERMANY. 37, 797-807 (2021).
- Alzabeebee S, Chapman D N. Evolutionary computing to determine the skin friction capacity of piles embedded in clay and evaluation of the available analytical methods. TRANSP GEOTECH. 24, 100372 (2020).
- Esmaeili-Falak M, Benemaran R S. Ensemble extreme gradient boosting based models to predict the bearing capacity of micropile group. APPL OCEAN RES. 151, 104149 (2024).
- Zhang R, Xue X. A novel hybrid model for predicting the end-bearing capacity of rock-socketed piles. ROCK MECH ROCK ENG. 1-16 (2024).
- Fattahi H, Ghaedi H, Malekmahmoodi F, et al. Optimizing pile bearing capacity prediction: insights from dynamic testing and smart algorithms in geotechnical engineering. Measurement. 230, 114563 (2024).
- Sun Z, Liu F, Han Y, et al. Prediction of ultimate bearing capacity of rock-socketed piles based on GWO-SVR algorithm. Structures. 61, 106039 (2024).
- Nguyen T, Ly D K, Huynh T Q, et al. Soft computing for determining base resistance of super-long piles in soft soil: A coupled SPBO-XGBoost approach. COMPUT GEOTECH. 162, 105707 (2023).
- Millán M A, Picardo A, Galindo R. Application of artificial neural networks for predicting the bearing capacity of the tip of a pile embedded in a rock mass. ENG APPL ARTIF INTEL. 124, 106568 (2023).
- Jia M, Zhang J, Dong Y. A compact and broadband balun based on multilayer SIW. IEEE MICROW WIREL CO. 32(2), 105-108 (2021).
- Zhang J, Wang Y D, Wu Y, et al. Inverse design of on-chip interconnect via transfer learning-based deep neural networks. IEEE T COMP PACK MAN (2023).
- Zhang J, Wei Z, Zhu E, et al. Inverse Design of Bonding Wire Array Based on Multi-fidelity Data Enabled Neural Networks. IEEE T COMP PACK MAN (2023).
- Zhang J, Wei Z, Kang K, et al. Intelligent Inverse Designs of Impedance Matching Circuits With Generative Adversarial Network. IEEE T COMPUT AID D (2024).
- Sun Y, Zhou S, Meng S, et al. Accumulative plastic strain of freezing-thawing subgrade clay under cyclic loading and its particle swarm optimization-back-propagation-based prediction model. COLD REG SCI TECHNOL. 214, 103946 (2023).
- Sun Y, Zhou S, Meng S, et al. Principal component analysis-artificial neural network-based model for predicting the static strength of seasonally frozen soils. SCI REP-UK. 13(1), 16085 (2023).
- Zhang Y, Cui N, Feng Y, et al. Comparison of BP, PSO-BP and statistical models for predicting daily global solar radiation in arid Northwest China. COMPUT ELECTRON AGR. 164, 104905 (2019).
- Yang Y, Lu M, Lin J, et al. Optimization of MRP quartz glass process parameters based on BP-PSO. MATER MANUF PROCESS. 1-10 (2024).
- Liu Y, Fan L, Wang L. Urban virtual environment landscape design and system based on PSO-BP neural network. SCI REP-UK. 14(1), 13747 (2024).
- Jabeur S B, Mefteh-Wali S, Viviani J L. Forecasting gold price with the XGBoost algorithm and SHAP interaction values. ANN OPER RES. 334(1), 679-699 (2024).
- Cakiroglu C, Demir S, Ozdemir M H, et al. Data-driven interpretable ensemble learning methods for the prediction of wind turbine power incorporating SHAP analysis. EXPERT SYST APPL. 237, 121464 (2024).
- Loi C L, Wu C C, Liang Y C. Prediction of tropical cyclogenesis based on machine learning methods and its SHAP interpretation. J ADV MODEL EARTH SY. 16(3), e2023MS003637 (2024).
- Lundberg S M, Lee S I. A unified approach to interpreting model predictions. Adv. neural inf. process. syst. 30 (2017).
- Mirjalili S. SCA: a sine cosine algorithm for solving optimization problems. KNOWL-BASED SYST. 96, 120-133 (2016).
- Hashim F A, Hussien A G. Snake Optimizer: a novel meta-heuristic optimization algorithm. KNOWL-BASED SYST. 242, 108320 (2022).
- Trojovský P, Dehghani M. Pelican optimization algorithm: a novel nature-inspired algorithm for engineering applications. Sensors. 22(3), 855 (2022).
- Abdollahzadeh B, Gharehchopogh F S, Mirjalili S. African vultures optimization algorithm: a new nature-inspired metaheuristic algorithm for global optimization problems. COMPUT IND ENG. 158, 107408 (2021).
- Braik M S. Chameleon Swarm Algorithm: a bio-inspired optimizer for solving engineering design problems. EXPERT SYST APPL. 174, 114685 (2021).
- Goh A T C. Empirical design in geotechnics using neural networks. Geotechnique. 45(4), 709-714 (1995).
No competing interests reported.