The level of interestingness of the rules produced by association rule-mining is constrained by support and confidence. The quantity of rules produced using support and confidence is insufficient for building multiclass classifier models. Accuracy is a crucial measurement for rules creation. The number of records that were correctly categorized using a rule is shown by accuracy.
It is suggested to use multiclass weighted associative classification with rule selection based on genetic algorithms with accuracy as the objective function. The creation of weighted datasets is the initial step. The second stage of the suggested method is rule creation, and it is identical to the method suggested in the last section, in which frequent item sets are found and the association rules are constructed using the MCMLWAC methodology for every combination of class labels. A large number of rules that were developed in the first step are prepruned with support and confidence thresholds in the third stage, which is rule pruning. Preliminary rule pruning (Prepruning) has been performed by fixing a lower threshold value for support and a higher value for confidence, despite not fully understanding the importance of support in multiclass classification.
The genetic algorithm for rule selection that uses an external population technique as its last step has accuracy set as one of its objective functions. Eq. 3 provides the formula for computing the support. The external population-based genetic algorithm is used to develop a multiclass classifier by helping to determine the precise rules. Prepruned rules that were extracted in the second stage are used as the genetic approach's initial population. The suggested technique uses the external population-based GA technique, which is described in the following subsections. In Fig. 5, the steps of the suggested technique are represented diagrammatically.
2.3.4 Rule of Genetic Based Selection
The number of rules produced by the confidence-based weighted associative classification that was previously proposed is insufficient for building the multiclass classifier. Thus, the genetic algorithm's evolutionary optimization technique is used to extract precise rules from the training dataset. In order to address generic optimization issues with huge search areas, genetic algorithms (GAs) were created. The stages taken in the evolutionary computation of the genetic algorithm are described in Algorithm 2.1.
Algorithm 2.1. Evolutionary Computation.
Input : Prepruned rules, Threshold for accuracy
Output : Accurate rules
Steps :
1. Set the prepruned rules as an initial population
2. While (No. of generations not exceeded)
For each rule in the population
2.1 Binary encode the rule
2.2 Evaluate the fitness function
If (Accuracy > Threshold value)
2.3 Store the rule in a file;
End if
End for
3. Select individuals from the current population for the next generation
4. Apply crossover (crossover fraction at 0.9)
5. Apply Mutation when rules are duplicated or a rule is framed without antecedent
End while
An external population-based genetic algorithm evolves an initial population of potential solutions to a problem towards the optimal solutions in accordance with the principles of selection, crossover, and mutation. The algorithm's general behavior is explained below. The algorithm begins the evolution from a starting population of solutions while taking into account a specific rule selection. The population's solutions are then decrypted and evaluated using a fitness function in accordance with the problem's optimization aim. Let the rule, for illustration, be of the type A is Low, B is Medium, and C is Medium. There are three discretized ranges Low, Medium, and High defined for each of the attributes A, B, and C.
The binary-encoded rule has the form 001 010 010 at that point. Each attribute is represented by three digits, so the encoded answer for three attributes is represented by a total of nine digits. The presence of 1 indicates that the attribute falls in the Low region, and the presence of 1 in the Most Significant position indicates the High zone. The presence of 1 in the center indicates that the attribute's value is located in the middle. The accuracy is then assessed using the binary encoded solution. 100 is represented by the three range attribute values. Then, a selection procedure is used to select a number of solutions from the current population using a particular selection method. Frequently, the solutions with the highest fitness values will have a higher chance of being chosen. The chosen solutions are coupled, and new workable ones are created by applying a crossover procedure to each pair of paired solutions.
The components of the crossover-generated solutions are then altered using a mutation procedure. To encourage variation in the current population of solutions, the mutation process is being used. In order to choose between the new solutions generated and the solutions of the current population, a deterministic crowding approach is utilized to determine a new population. Up to some stopping criterion, such as the maximum number of iterations, the given process is repeated. The algorithm's major building blocks are the representation of solutions, the creation of the starting population, the fitness function, and the selection, crossover, and mutation processes.
Any answer can be portrayed in a population in a variety of ways. The solutions need to be represented or encoded in a way that when various crossover and mutation operators are used, new workable solutions are produced. The various encoding techniques that are accessible include Binary, Octal, Hexadecimal, Permutation, Value, and Tree encoding, among others. The proposed method uses a binary encoding approach with variable length. The range of discretization of attribute values determines the binary encoding's final bit count. The suggested method is strengthened by this.
There are a certain number of workable answers in the initial population. Each initial population solution is produced at random. Every time the algorithm iterates, a population—a collection of people—is saved and altered. Iterations are referred to as generations in GAs. These people were chosen based on their level of fitness. Each new generation of people inherits genes from the one before them, and the fitter people will typically survive and procreate.
The multi-point crossover operator is employed because the database may include a lot of properties. When dealing with rule mining issues, the need for saving a collection of improved rules from the dataset is discovered. Using only standard genetic operations, the final population may lack some better rules that were generated at some intermediate generations. The better rules generated at intermediate stages should be kept. For this task, an external population is used.
The fitness function determines how well-suited each solution is to achieving the specified optimization goal. Accuracy is utilized as the objective function because this phase's goal is to maximize accuracy. Eq. 3 contains the calculation formula for accuracy.
The selection phase chooses the best solutions from the present population to confirm parent solution pairs that will be employed in the crossover and mutation processes to create new solutions (offspring solutions). While choosing a solution, this procedure takes into account the fitness values of the alternatives and is also typically influenced by a random variable. As a result, the solutions with the highest fitness values are more likely to be chosen. Each pair of solutions in the current population that the selection process determines should be recombined is subjected to the crossover operation with a probability of Pc. The crossover that formed in two previous solutions (the parent solutions) produces two new solutions (offspring solutions). The crossover operator is one of the most important genetic operators because it keeps and combines the best traits of the parent solutions to define the new best solutions.
The population's genetic diversity will be increased by the mutation operator. Solutions that are encoded are immediately subjected to the mutation operators. As a result, the representation specified for the solutions must be used to construct the mutation.
The next part discusses the experimental examination of the suggested multiclass weighted associative classification with genetic based rule section.
2.3.5 Experiments, Results and Analysis
The public heart dataset from the UCI repository serves as the basis for the investigation of the suggested genetic-based selection. This public dataset's nature is incredibly hazy. In order to generate rule items often, metrics support and confidence play a crucial role. The weighted dataset that has been transformed serves as the technique's input. The weighted average of the attribute values provided by the doctors is shown in Table 2 and is used to replace the attributes in the record set. The technique suggested in the previous section uses a similar second stage of rule development. The weighted dataset's rules are produced using MCMLAC. For each and every class, all conceivable rules are constructed. As a result, there are a lot of produced rules. The second row of Table 5 displays the total number of rules produced for each support value. The support and confidence for each rule in each class are concurrently determined in this MCMLAC technique along with the rule generation. Hence, compared to other apriori-based techniques, the time complexity is lower. All of the MCMLAC with support and confidence values are produced by a single scan of a dataset. Prerunning in the third stage involves setting 2% as the minimum support and 75% as the minimum confidence. The results are shown in Tables 3 and 4.
The next stage is genetic based accurate rule selection. The genetic algorithm uses the rules that were previously trimmed in this stage. That is, the earlier stage's output is used as the stage's input. In the second stage, the encoded rules are trimmed from the initial population. The processes are repeated 500 times after crossing and mutation. The true positive (TP), true negative (TN), false positive (FP), and false negative values for the genetically based created rules are kept in a file (FN). There are a large number of rules produced overall for different danger levels. The number of rules generated at each level, both with and without optimization, is shown in Table 5. The number of rules that have been pruned with accuracy after pruning is shown as > 80% in the fourth row of the table. It has been noted that the number of genetic rules with accuracy > 80% for risk level 0 (4, 3, 2, 1, 1, 0) is extremely low (307, 284, 217, 187, 174, 124). The issue is because there are a lot of false positive and false negative rules generated for the risk level 0 classes, and these rules are also present in many other risk level rule sets. Hence, a multiclass classifier cannot be constructed using only genetically based results.
Consequently, the suggested multiclass weighted associative classification may be strengthened by combining the confidence-based rule pruning technique and the genetic-based rule selection technique. So, in this chapter, a multiclass weighted associative classifier is built with genetically based selected rules for other risk levels and high confidence rules for risk level 0.
As a result, the suggested multiclass weighted associative classifier with genetically based rule selection for early heart disease prediction is successfully created. The accuracy of the final classifier is also tested using both private and public datasets. Java is used to implement each technique that has been suggested. The suggested classifier's accuracy is compared to that of the RIPPER and PART decision tree classifiers currently in use. Table 6 depicts the accuracy comparison between the existing techniques and the proposed multiclass weighted associative classification with genetic based rule selection technique against both private heart dataset.
Table 5
Results of the proposed multiclass weighted associative classification with genetic algorithm based rule selection (MCWACGRS)
KIIMs Dataset | Risk Level | Support 2% | Support 5% | Support 10% | Support 20% | Support 30% | Support 50% |
Rules generated WMCMLAC | | 93950 | 69690 | 38925 | 19310 | 13985 | 8385 |
Total number of rules generated using GA | All | 929 | 864 | 739 | 556 | 535 | 331 |
0 | 307 | 284 | 217 | 187 | 174 | 124 |
1 | 201 | 199 | 175 | 122 | 118 | 71 |
2 | 153 | 137 | 119 | 83 | 86 | 44 |
3 | 149 | 138 | 130 | 94 | 90 | 50 |
4 | 119 | 106 | 98 | 70 | 67 | 42 |
Number of rules > 80% | All | 468 | 412 | 365 | 262 | 259 | 136 |
No. of Rules using MCWACGRS with Accuracy > 80% | 0 | 4 | 3 | 2 | 1 | 1 | 0 |
1 | 156 | 133 | 110 | 73 | 85 | 42 |
2 | 109 | 94 | 83 | 61 | 60 | 29 |
3 | 102 | 99 | 92 | 71 | 63 | 34 |
4 | 97 | 83 | 78 | 56 | 50 | 31 |
WEKA (a well-known data mining tool) was used for the experimental study of PART and RIPPER, and the results are presented in Table 6 for comparison. These two conventional classifiers are both classifiers. Java is used to implement the accuracy of the existing MMAC, and Table 6 also shows the accuracy attained using the current method. All attribute values in MMAC were solely mapped to integer values. The weights allocated to physicians in this study are ambiguous. As a result, the proposed weighted multiclass classifier's accuracy has improved.
Traditional classifiers such decision tree-based PART, RIPPER, etc. focus exclusively on two class classifiers in comparison to the existing approaches. Without taking into account the attribute weights, a multi class multi label based associative classification created a multi class classifier. The public dataset's nature is incredibly hazy. As a result, the private dataset's accuracy is higher than the public dataset's. The private heart dataset's attribute values-based weight assignment and class assignment are carried out with the assistance of the same group of subject-matter experts (physicists). As a result, the private dataset's accuracy is greater than the public cardiac dataset's.
Table 6
Comparison of accuracies of the proposed MCWACGRS algorithm and the existing algorithms
Dataset | Existing Algorithms | Proposed Algorithm |
PART (Decision tree) | RIPPER | MCMLAC | MCWACGRS |
Classifier category | Two | Two | Multiclass | Multiclass |
Public dataset Accuracy in % | 67 | 71 | 75 | 84 |
Private dataset Accuracy in % | 73 | 76 | 78 | 88 |
The proposed multiclass weighted associative classifier is extended for distributed environments, as the majority of the dataset are distributed in nature, in order to take advantage of the advantages of the proposed multiclass weighted associative classifier with genetic based rule selection in real life scenarios. As a result, a new distributed multiclass weighted associative classification is suggested, and it is further explained in the part that follows.
2.4 Proposed Distributed Multiclass Weighted Associative Classification
Regardless of their physical locations, distributed data mining derives knowledge from scattered data sources. The majority of the dataset are spread in nature in real time. In this section, the proposed multiclass weighted associative classification technique is extended for distributed data classification.
There are two ways to apply data mining techniques on geographically dispersed data sources: centralized model and distributed model. In a centralized model, the necessary data from numerous sources is collected at one location, and a mining technique is applied to the combined data. It yields precise results but takes a lot of time and money for communication. Given the sensitivity of the dataset, security must be provided for data transit. Whereas in a distributed setting, local sites will do partial mining and the results from those locations will be combined at a single site. When the same condition and threshold are specified in the centralized model, the accuracy of distributed models will be reduced. Distributed models simplify algorithms and lower communication costs.
Mokeddem & Belbachir [19] reported a distributed associative classification algorithm to reduce time complexity and used a function [(item-1) % N] for distributing items to N number of processors. With minimum communication between processors, locally generated subset of classification rules are finally combined to make a classifier. In their work, the dataset need not be partitioned. It further reduces the communication cost. Ultimate objective for any parallel (Agrawal & Shafer [20], Zaki [21], Moonesinghe et al. [22], Pramudiono & Kitsuregawa [11]) or distributed associative classification (RaghuRam et al. [23], Mokeddem & Belbachir [19]) is to reduce communication cost and less interprocess communication (Moonesinghe et al. [22], Pramudiono & Kitsuregawa [11], RaghuRam et al. [23]). Thakur & Ramesh [24] designed a parallel model for CBA system.
The multiclass weighted associative classification introduced in the preceding part is expanded for distributed environments in this study because there is a dearth of research on the subject. The results are reported in the next subsection.
2.4.1 Design of Proposed System
By successfully utilising the knowledge regarding the data gathered at numerous sites, the suggested system aims to create a multiclass classifier model. It improves the model even more by including the typical associated patterns linked to the dataset across different locations. In recent years, distributed databases where data is stored in various shared nothing machines connected through internet, grid or cloud. The suggested approach is similar to distributed data mining in that it distributes data across a number of sources horizontally.
The proposed system modifies (Aftarczuk, Shantakumar Patil & Kumaraswamy, Swapnil & Avinash, Senapati et al. and Bahrammizaee [25–29]) few factors of the distributed shared nothing environment. The first is to incorporate the advantages of a centralised model into a distributed model; the task of building a classifier model is taken on by a common coordinator. Data gathered at each site is used as a training dataset in the centralised model, and it is sent to the coordinator for consolidation and the creation of classifier models. Semi-pruned rules and their confidence values are sent to the coordinator in the proposed system in order to lower communication costs and lower the security risk associated with delivering the dataset in its current state. The rules produced using the training data dispersed across many sources is used as a whole in the distributive environment to create classification models. The suggested distributed system's architecture diagram is given in Fig. 6.
2.4.2 Step Follows in the Proposed System
The steps involved in the design of a distributed multiclass weighted associative classifier are given below as stated by (Tomczak & Gonczarek, Thangavel et al. Michalis et al. and Ullah [30–33]). The series of processing at the local as well as at the coordinator sites are as follows:
Step 1: All local locations where a dataset is present are informed about the discretization ranges for all characteristics, weights associated with each and every range of various attributes, and minconfidence threshold values.
Step 2: The weighted dataset is created using the procedure outlined in section 2.1.1.
Step 3: Across the weighted dataset, frequent ruleitemset creation based on multi class multi label associative classification is carried out. As previously described in section 2.1.2, the resultant rule creation and the confidence value calculation for each rule are carried out with a single database scan.
Step 4: Support measure insignificance has been examined in section 2.1.3. As a result, rules are concurrently trimmed according to the minconfidence threshold and ranked for all classes according to the confidence value.
Step 5: Chosen rules are forwarded to a coordinator site along with their confidence values.
Step 6: Each local site completes steps 1 through 5 and sends the rules to the coordination site.
Step 7: At the coordinator site, semi-pruned rules from different sites are received together with their confidence values, and in the subsequent portion, the rules are arranged according to their class labels.
Step 8: Selected and included for the multiclass classifier model are unique rules for each class, that is, rules present exclusively for a specific class.
Step 9: When the same rule is generated in more than one site, the rule is included in the model.
Step 10 : Set of rules for each class is grouped and the final multiclass classifier is constructed.
2.4.3 Experiments, Results and Analysis
Support measure insignificance has been examined in section 2.1.3. As a result, rules are concurrently trimmed according to the minconfidence threshold and ranked for all classes according to the confidence value.
This study focuses on creating a multiclass classifier specifically for predicting heart disease risk levels. In order to find common patterns/symptoms for varying degrees of heart disease risk, dataset patterns and relevant information from diverse places are collected. In addition to helping to uncover patterns from the results of invasive and non-invasive tests for novel diseases that have just been discovered in some places, this proposed distributed mining also aids in discovering patterns from heart disease. The proposed multiclass distributed model aids in the early detection of illnesses. Compromise of the classifier's accuracy is of no use in the context of the medical dataset. Hence, the advantages of a centralized model are adopted in the proposed distributed model.
Starting the investigation is the public heart dataset. The public (UCI) heart dataset is separated into three datasets for analysis (Unil [34]). Table 7 lists the preliminary and final outcomes at each location. More rules are discovered for classes with greater risk levels when the minconfidence threshold is lowered to 40%. The only threshold taken into account for this distributed model is minconfidence. Table 7 lists the total number of rules produced at each stage and for different risk levels (Wang et al., Wasan et al., and Chen & Chen [35–37]). It also demonstrates that for higher risk levels, a manageable amount of rules are developed. The workload on the coordinator site is decreased by the proposed distributed technique because partial mining has been finished at each location.
Table 7
Results of the proposed distributed weighted associative classification
Site | Number of Records | Total No. of. Rules MCMLWAC | Number of Rules |
Confidence > 40 | Risk levels 0,1,2,3,4 |
Site 1 | 99 | 43100 | 415 | 341 |
31 |
16 |
15 |
12 |
Site 2 | 99 | 39105 | 380 | 308 |
28 |
14 |
18 12 |
Site 3 | 99 | 33480 | 318 | 245 |
29 |
20 |
13 |
11 |
While the majority of past research studies concentrated on two-class classifiers, the proposed multiclass classifier developed in this research effort aids in the early detection of disorders.