4.1. Specification phase
The vulnerability of a plant must be mitigated by means of methods, processes, instruments, and practices that are ideally able to predict and face situations of imminent danger. These elements were identified and classified according to the scope of use based on Brazilian dam safety legislation (Brasil, 2010), the technical guidelines of good engineering practices recommended for public and private generation concessionaires (Eletrobras, 2003) and the general guidelines for companies, aiming to ensure adequate safety conditions for dams, from construction to decommissioning (ANA, 2016). Nine elements identified within their respective scopes were used as criteria in the MCDA model. The subdivision of the subcriteria depends on the documents, processes, practices, or instruments as they were classified in the criteria.
Documentation (DOC): These records must meet dam safety criteria from preliminary and feasibility studies to the final project as constructed. In this item, we analyzed the available as-built designs, construction and commissioning documentation, project description and specifications, hydrological studies, geological and geotechnical studies, seismological studies, foundation studies, spillway design and water intake design. The subcriteria are: Preconstruction engineering and design (DOC.ENG); Hydrology investigations (DOC.HYD); Geotechnical investigations (DOC.GEO); Subsurface explorations and foundation investigations (DOC.FOU); Service spillway design (DOC.SPI); and Water intake design (DOC.WIN).
Inspections (INS): Dam safety inspections are divided into regular and special inspections, the first being carried out periodically, with frequency determined according to the risk category and potential damage, aiming at assessing and detecting the existence of anomalies. Regarding the regular safety inspection, the situation and level of danger related to regular inspections evaluates the upstream slope, downstream slope, right and left abutments, crest, spillway, water intake, reservoir, instrumentation, ducts and shielding, turbines, alternators, substation, and powerhouse. Regarding the special safety inspection, it will be prepared as instructed by a multidisciplinary team of specialists, depending on the risk category and the potential damage associated with the dam, in the construction, operation and deactivation, and must consider changes in conditions upstream and downstream of the dam. The subcriteria are: Riprap (INS.RIP); Downstream slope and toes (INS.DSL); Abutments (INS.ABU); Dam core (INS.COR); Service spillway (INS.SPI); Water intake (INS.WIN); Water physical-chemical properties (INS.WAT); Foundations (INS.FOU); and Surveillance and monitoring system (INS.MON).
Operation (OPE): It describes the procedures for the operation of the water intake, spillway, and sluice way, to allow satisfactory operation of the dam, in addition to keeping it in safe conditions and monitoring its behavior to detect any anomalies in a timely manner. In this part, the existence of procedures to be adopted in the operation of the reservoir is verified. The subcriteria are: Reservoir levels, affluent and effluent water flows (OPE.RES); Service spillway operating records (OPE.SPI); Detailed information concerning seepage control (OPE.SEC); Bottom outlet operating records (OPE.BOT); Water intake operating records (OPE.WIN); Reservoir water quality management (OPE.QUA); Operating procedure for extreme flood events (OPE.EXT); Operating instruction in the event of general flood gate failure (OPE.GAT); and Operating procedure in the event of loss of communication (OPE.COM).
Maintenance (MAN): Maintaining structures and equipment in good condition is intended to ensure that the dam is kept in fully operational and safe condition. For this purpose, the equipment must be inspected and checked at regular intervals, as part of a maintenance program appropriate to the type of equipment, age, and intensity of use. Maintenance records of current actions involving the bus, spillway, instruments, and gates are checked. The subcriteria are: Updated maintenance logbook (MAN.LOG); Drainage system maintenance procedures (MAN.DRA); Spillway gates and chute maintenance procedures (MAN.SPI); Safety instrument maintenance procedures (MAN.ITM); Water intake maintenance procedures (MAN.WIN); and Reduction of the erosive process and siltation (MAN.ERO).
Monitoring and Instrumentation (MON): It includes the instrumentation of the body of the dam, the spillway structures, and the foundations. This criterion investigates the monitoring of the following factors (some related to instrumentation): neutral pressures at the landfill, landfill settlements, surface displacements, and seepage, among others. The dam instrumentation design must be elaborated from the feasibility study and basic design phases of the hydroelectric plant. It should contain the location of the instrumentation, along with the type and quantity of devices to be used. Instrumentation is an objective tool to monitor dams, being of fundamental importance in the safety procedures adopted. The most common instruments in earth dams and their respective functions are: topographic frame of reference for monitoring vertical and horizontal displacements; settlement sensors for measuring total and differential settlements and movement of materials; inclinometers for measuring displacements inside embankment dams or foundations; piezometers for measuring water pressures inside the dam body or at the foundation; total pressure cells in the landfill, to measure the total pressure in the landfill; flow meters for measuring the piping through the body of the dam and its foundation; water level gauge; and weather stations for measuring reservoir air and water temperatures and precipitation. The number of auscultation instruments to be installed in a dam depends on its height and length, geological formation of the soil and the materials used in the body of the dam. The frequency of readings should be adjusted depending on the occurrence of critical geotechnical or geological conditions, changes in construction procedures, rapid rises or falls of the reservoir level, and severe natural phenomena. The subcriteria are: Instrument installation and maintenance reports (MON.ITM); Routine visual and in situ inspections (MON.SIT); Geotechnical limits established for all inspection instruments (MON.GEO); Inspection of the maximum dam cross-section (MON.CRO); Inspection of sections with geological anomalies (MON.ANO); and Monitoring instrumentation calibration (MON.CAL).
Regular Revision and Updating (REV): The Brazilian Dam Safety Act (Brazil, 2010) states that, based on the dam risk classification, a regular dam safety review must be carried out to verify the general state of the dam, considering the current state of the art for the design criteria, the updating of the hydrological data and changes in conditions upstream and downstream. Thus, an extensive review of the technical documentation is carried out, comprising hydrological, geological, geotechnical, and seismological studies. The electric energy regulator audits the records related to the performance of the foundation, reservoir, spillway structure, earthen dam, operation and maintenance procedures, instrumentation, and monitoring. The outcome of the audit is the reassessment of the risk category and the associated potential damage. This criterion assesses whether the information available is sufficient to meet the requirements established by the auditors. The subcriteria are: Hydrology investigations (REV.HYD); Geotechnical investigations (REV.GEO); Foundation investigations (REV.FOU); Probable maximum flood (REV.MAX); Operation and maintenance plan (REV.OPM); and Monitoring instrument installation (REV.MON).
Emergency (EME): This criterion is based on the principle that the priority will be to save human lives. It must have an organized command line for emergency situations, with agile and standardized alert and communication procedures and a plan to resume operational capacity once the hazard has been controlled. The procedures must be specific to an external threat (invasion for example); fire; dam damage; inundation and leakage of chemical waste. Each threat must be determined according to the sequence of steps for device startups and shutdowns and plant evacuation. The subcriteria are: Disaster committee (EME.DIS); Emergency communication (EME.COM); Assignment of responsibilities (EME.REP); Mass care (EME.MAS); Animal care (EME.ANI); and Recovery (EME.REC).
Evacuation (EVA): The population affected by the dam must be identified to enable implementing an instruction plan and escape routes, aiming to minimize the damage and risks that may occur in a hypothetical breach. Therefore, in this criterion, a survey is performed to detect the existence or absence of programs for training the population and determining the escape routes: Instruction Plan, Escape Routes and Meeting Points, Self-Rescue Zone. The subcriteria are: Jurisdiction (EVA.JUR); Administration and logistics (EVA.ADM); Flash flood alarms (EVA.ALA); Escape routes and emergency exits (EVA.ROU); Mass care facilities (EVA.MAS); and Supplies of food, water, sanitation materials, clothing, bedding and first aid items (EVA.SUP).
Preparedness (PRE): Hydroelectric plant personnel must be trained and have equipment compatible with the type and severity of likely emergencies. This criterion ascertains the ability to perceive a situation of imminent risk and respond quickly, following pre-established procedures. The subcriteria are: Emergency staff (PRE.STA); Instrumentation repair training program (PRE.IRT); Power plant operational training courses (PRE.OPE); Power plant maintenance training courses (PRE.MAN); Visual and in situ monitoring training courses (PRE.SIT); Instrumentation reading and interpretation training courses (PRE.IRI); Regular revision and follow-up (PRE.REV); Emergency communication and action training exercises (PRE.EME); and Evacuation training exercises (PRE.EVA).
4.2. Scoring and weighting phases
To illustrate the scoring phase, we demonstrate the steps that were followed with the general manager (GM). Nine letters containing the identification of the criteria and several blank letters were handed to the interviewee. He was asked to rank the cards serially in order of importance, with two or more cards in parallel meaning a tie for that position. The interviewee thought that the order of importance was headed by INS and MON, second MAN, third REV and DOC, fourth OPE, and fifth PRE, EVA and EME. The degrees of importance between criteria were all judged to be “a little more important”. Table 1 shows the steps of the Simos method used to obtain the criterion weights from the initial ordering, which are the criterion ranking, the number of criteria of each position, the position order, the average position, and the weights resulting from the average position divided by the sum of positions. The same table shows the scores according to the Saaty scale converted from the weights obtained by the Simos method (Ribas & Silva, 2015). In this case, a change was made to the values for base 8 plus 1 for subsequent inversion.
Table 1. Criterion ranking, Simos weights and Saaty scores.
Rank
|
Simos Weights
|
Saaty Scale
|
#Criteria
|
Position
|
Average
|
Weights
|
Eight Base
|
Scores
|
INS, MON
|
2
|
1,2
|
1.5
|
2
|
1.92
|
8
|
white
|
-
|
-
|
-
|
-
|
-
|
-
|
MAN
|
1
|
4
|
4
|
6
|
3.46
|
7
|
white
|
-
|
-
|
-
|
-
|
-
|
-
|
REV, DOC
|
2
|
6,7
|
6.5
|
10
|
5
|
5
|
white
|
-
|
-
|
-
|
-
|
-
|
-
|
OPE
|
1
|
9
|
9
|
14
|
6.54
|
3
|
white
|
-
|
-
|
-
|
-
|
-
|
-
|
PRE, EVA, EME
|
3
|
11,12,13
|
12
|
18
|
8.38
|
2
|
|
|
|
|
|
|
|
|
Table 2 shows the matrix of paired comparisons, which together with fuzzification degree equal to 3.0 constitute the input data of the fuzzy AHP method. The resulting weights for the criteria can be seen in the last column. The degree of fuzzification of this magnitude suggests that the experts' inaccuracy in estimating preference levels is relatively high. For example, the degree of preference of INS in relation to MAN is obtained by subtracting the two scores plus one, so the difference between the two scores (8-7) +1 equals 2, representing slight preference for INS.
Table 2. Paired comparisons matrix and the estimated criterion weights.
|
INS
|
MON
|
MAN
|
VER
|
DOC
|
OPE
|
PRE
|
EVA
|
EME
|
WEIGHTS
|
INS
|
2
|
1
|
2
|
4
|
4
|
6
|
7
|
7
|
7
|
0.2274
|
MON
|
1
|
1
|
2
|
4
|
4
|
6
|
7
|
7
|
7
|
0.2254
|
MAN
|
1/2
|
1/2
|
1
|
3
|
3
|
5
|
6
|
6
|
6
|
0.2072
|
REV
|
1/4
|
1/4
|
1/3
|
1
|
1
|
3
|
4
|
4
|
4
|
0.1700
|
DOC
|
1/4
|
1/4
|
1/3
|
1
|
1
|
3
|
4
|
4
|
4
|
0.1700
|
OPE
|
1/6
|
1/6
|
1/5
|
1/3
|
1/3
|
1
|
2
|
2
|
2
|
0.1175
|
PRE
|
1/7
|
1/7
|
1/6
|
1/4
|
1/4
|
1/2
|
1
|
1
|
1
|
0.0806
|
EVA
|
1/7
|
1/7
|
1/6
|
1/4
|
1/4
|
1/2
|
1
|
1
|
1
|
0.0806
|
EME
|
1/7
|
1/7
|
1/6
|
1/4
|
1/4
|
1/2
|
1
|
1
|
1
|
0.0806
|
The Simos method was also used in the comparisons between the subcriteria for each of the nine criteria. We adopted a variant of the AHP method to determine the degrees of importance, that is, instead of making cross-comparisons, we compared each set of subcriteria within each criterion. The reasons is that we were not establishing a ranking of the alternatives, but rather trying to classify the safety methods and procedures necessary to ensure the reduction of the dam's vulnerability to the risk of incidents of any nature. To exemplify, we now demonstrate the results obtained in the interview with the PM for estimation of the weights of the safety procedures adopted in the scope of maintenance. As can be seen in Table 3, the PM estimated that, in terms of dam safety, Updated maintenance logbook (MAN.LOG); Drainage system maintenance procedures (MAN.DRA) and Reduction of the erosive process and siltation (MAN.ERO) are more important than Spillway gates and chute maintenance procedures (MAN.SPI); Security instruments maintenance procedures (MAN.ITM); and Water intake maintenance procedures (MAN.WIN). The last column shows the scores according to the Saaty scale converted from the weights obtained by the Simos method.
Table 4 is assembled in the same way as Table 2. The last column shows that the weights for three of the subcriteria are identical and much higher than the weights of the other three. This result stems from the PM’s preferences, as shown in Table 3.
Table 3. Subcriterion ranking, Simos weights and Saaty scores.
Rank
|
Simos Weights
|
Saaty Scale
|
#Criteria
|
Position
|
Average
|
Relative
|
Control
|
Weights
|
Eight Base
|
Scores
|
MAN.LOG, MAN.DRA, MAN.ERO
|
3
|
1,2,3
|
2
|
7
|
21
|
7
|
3.0
|
7
|
White
|
-
|
-
|
-
|
-
|
-
|
-
|
-
|
-
|
White
|
-
|
-
|
-
|
-
|
-
|
-
|
-
|
-
|
MAN.ITM, MAN.SPI,
MAN.WIN
|
3
|
6,7,8
|
7
|
26
|
78
|
26
|
8.0
|
2
|
|
|
|
|
|
|
|
|
|
|
Table 4. Paired comparison matrix and the estimated subcriterion weights.
|
MAN.LOG
|
MAN.DRA
|
MAN.ERO
|
MAN.ITM
|
MAN.SPI
|
MAN.WIN
|
WEIGHTS
|
MAN.LOG
|
1
|
1
|
1
|
6
|
6
|
6
|
0.2582
|
MAN.DRA
|
1
|
1
|
1
|
6
|
6
|
6
|
0.2538
|
MAN.ERO
|
1
|
1
|
1
|
6
|
6
|
6
|
0.2538
|
MAN.ITM
|
1/6
|
1/6
|
1/6
|
1
|
1
|
1
|
0.1171
|
MAN.SPI
|
1/6
|
1/6
|
1/6
|
1
|
1
|
1
|
0.1171
|
MAN.WIN
|
1/6
|
1/6
|
1/6
|
1
|
1
|
1
|
0.1171
|
The procedure exemplified for the MAN criterion was performed for the others. Then, the weights obtained for the criteria according to Table 2 were multiplied by the weights obtained for the subcriteria and by 100. The results of the PM’s ranking can be seen in Table 5, in which the degrees of importance for the methods and procedures related to mitigation of the dam's vulnerability are in the range of 1.06 to 5.35, with an average of 1.42 and a standard deviation of 2.59.
The degrees of Table 5 according to the PM as well as other experts were classified in four classes, where C.1 denotes the “essential” methods and procedures; C.2 those of “high importance”; C.3 those of “medium importance”; and C4 the “least important”. The classifications were obtained through the quartiles calculated from the degrees of importance of each expert. The cutoff values according to the PM, for example, are 1.39, 2.36 and 3.73, for C.4, C.3, C.2 and C.1, respectively.
Table 5. Degrees of importance of dam safety methods and procedures.
INS.FOU
|
5.27
|
OPE.SEC
|
2.52
|
PRE.OPE
|
1.95
|
DOC.SPI
|
3.83
|
EVA.JUR
|
2.38
|
INS.RIP
|
4.37
|
OPE.COM
|
2.52
|
PRE.MAN
|
1.95
|
DOC.WIN
|
3.83
|
EVA.ALA
|
1.80
|
INS.DSL
|
4.37
|
OPE.RES
|
2.52
|
PRE.EME
|
1.39
|
DOC.ENG
|
3.63
|
EVA.ROU
|
1.80
|
INS.ABU
|
4.37
|
OPE.GAT
|
2.09
|
PRE.STA
|
1.39
|
DOC.FOU
|
2.94
|
EVA.MAS
|
1.04
|
INS.COR
|
4.37
|
OPE.EXT
|
2.09
|
PRE.EVA
|
1.39
|
DOC.GEO
|
2.77
|
EVA.SUP
|
1.04
|
INS.WIN
|
3.17
|
OPE.SPI
|
1.31
|
PRE.IRT
|
1.39
|
DOC.HYD
|
1.31
|
EVA.ADM
|
1.04
|
INS.SPI
|
3.17
|
OPE.BOT
|
1.31
|
PRE.IRI
|
1.39
|
REV.OPM
|
4.40
|
MON.CRO
|
5.75
|
INS.WAT
|
1.39
|
OPE.WIN
|
1.31
|
PRE.SIT
|
1.39
|
REV.MON
|
4.04
|
MON.SIT
|
5.75
|
INS.MON
|
1.39
|
OPE.QUA
|
0.39
|
PRE.REV
|
0.39
|
REV.GEO
|
2.86
|
MON.ANO
|
4.34
|
EME.REC
|
2.06
|
EME.DIS
|
1.55
|
MAN.LOG
|
5.35
|
REV.HYD
|
2.86
|
MON.GEO
|
4.34
|
EME.COM
|
2.06
|
EME.MAS
|
0.85
|
MAN.DRA
|
5.26
|
REV.FOU
|
2.86
|
MON.ITM
|
2.36
|
EME.REP
|
1.55
|
EME.ANI
|
0.85
|
MAN.ERO
|
5.26
|
REV.MAX
|
1.15
|
MON.CAL
|
2.36
|
MAN.SPI
|
2.43
|
MAN.WIN
|
2.43
|
MAN.ITM
|
2.43
|
|
|
|
|
To obtain a single aggregated classification, it is essential to reject all hypotheses of significant difference between the estimates among the five experts: general manager (GM), electric engineer (EE), mechanical engineer (ME), civil engineer (CE), and consultant (CO). For this purpose, Spearman correlation coefficients are calculated by comparing the results between pairs of specialists. As can be seen in Table 6, the estimates of the subcriterion positions among them have no significant differences, with p-values close to zero in all cases. Such statistics confirm the inexistence of significant differences among the experts, allowing us to use the aggregated values, summarizing them in a single estimate for the entire group.
Table 6. Pairwise Spearman coefficients between the degrees of importance of the experts.
|
GMxEE
|
GMxME
|
GMxCE
|
GMxCO
|
EExME
|
EExCE
|
EExCO
|
MExCE
|
MExCO
|
CExCO
|
Statistics
|
0.9981
|
0.9977
|
0.9984
|
0.9972
|
0.9991
|
0.9982
|
0.9986
|
0.9988
|
0.9987
|
0.9986
|
p-value
|
0.0000
|
0.0000
|
0.0000
|
0.0000
|
0.0000
|
0.0000
|
0.0000
|
0.0000
|
0.0000
|
0.0000
|
Table 7 shows the importance classes estimated by the five specialists and the aggregate estimate. Most of the subcriteria deemed essential are related to the inspection procedures of critical parts of the dam and the reliability of such instruments. Internal erosion with increased pore pressure and saturation of the dam and foundation causes loss of resistance, which explains the relevance of regular inspections of the riprap (INS.RIP); downstream slope and toes (INS.DSL); abutments (INS.ABU) and dam core (INS.COR); monitoring instrument installation and maintenance reports (MON.ITM); routine of visual and in situ inspections (MON.SIT); the geotechnical limits established for all inspection instruments (MON.GEO) and the safety instrument maintenance procedures (MAN.ITM). Some of these subcriteria are related to operating instructions regarding the occurrence of extreme and unexpected events: procedures in the case of extreme flood events (OPE.EXT); the operating instruction in the event of loss of communication (OPE.COM) and in the case of general flood gate failure (OPE.GAT). The remaining essential subcriteria have to do with training, preventive O&M and geotechnical documentation: the updated maintenance logbook (MAN.LOG); the spillway gates and chute maintenance procedures (MAN.SPI); the power plant operational training courses (PRE.OPE) and maintenance training courses (PRE.MAN); the documentation of geotechnical investigations (DOC.GEO); and the operating procedures for the affluent and effluent water flows (OPE.RES). Therefore, when the experts expressed their preferences regarding the methods and procedures to mitigate the dam's vulnerability, they emphasized the actions of inspecting parts of the dam and in reacting to extreme and unexpected events.
Table 7. Pairwise Spearman coefficients between the degrees of importance of the experts.
|
GM
|
EE
|
ME
|
CE
|
CO
|
AVE
|
|
GM
|
EE
|
ME
|
CE
|
CO
|
AVE
|
|
GM
|
EE
|
ME
|
CE
|
CO
|
AVE
|
INS.ABU
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
OPE.RES
|
C.2
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
PRE.OPE
|
C.3
|
C.2
|
C.1
|
C.1
|
C.1
|
C.1
|
INS.RIP
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
OPE.EXT
|
C.2
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
PRE.MAN
|
C.3
|
C.2
|
C.1
|
C.1
|
C.1
|
C.1
|
INS.DSL
|
C.1
|
C.1
|
C.2
|
C.1
|
C.1
|
C.1
|
OPE.COM
|
C.2
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
PRE.IRI
|
C.3
|
C.3
|
C.1
|
C.1
|
C.1
|
C.2
|
INS.COR
|
C.1
|
C.1
|
C.2
|
C.1
|
C.1
|
C.1
|
OPE.CAT
|
C.3
|
C.1
|
C.1
|
C.1
|
C.1
|
C.1
|
PRE.SIT
|
C.3
|
C.3
|
C.1
|
C.1
|
C.1
|
C.2
|
INS.SPI
|
C.1
|
C.2
|
C.2
|
C.2
|
C.2
|
C.2
|
OPE.SPI
|
C.3
|
C.2
|
C.2
|
C.2
|
C.2
|
C.2
|
PRE.IRT
|
C.3
|
C.3
|
C.1
|
C.1
|
C.2
|
C.2
|
INS.WIN
|
C.2
|
C.2
|
C.3
|
C.2
|
C.3
|
C.2
|
OPE.BOT
|
C.4
|
C.2
|
C.2
|
C.2
|
C.2
|
C.2
|
PRE.STA
|
C.3
|
C.3
|
C.2
|
C.2
|
C.2
|
C.2
|
INS.FOU
|
C.2
|
C.2
|
C.3
|
C.2
|
C.3
|
C.2
|
OPE.WIN
|
C.4
|
C.2
|
C.3
|
C.2
|
C.2
|
C.3
|
PRE.EVA
|
C.3
|
C.3
|
C.2
|
C.3
|
C.2
|
C.3
|
INS.WAT
|
C.3
|
C.3
|
C.4
|
C.3
|
C.3
|
C.4
|
OPE.SEC
|
C.4
|
C.3
|
C.3
|
C.3
|
C.2
|
C.3
|
PRE.REV
|
C.3
|
C.4
|
C.2
|
C.3
|
C.3
|
C.3
|
INS.MON
|
C.3
|
C.4
|
C.4
|
C.3
|
C.4
|
C.4
|
OPE.QUA
|
C.4
|
C.4
|
C.4
|
C.3
|
C.4
|
C.4
|
PRE.EME
|
C.4
|
C.4
|
C.4
|
C.3
|
C.4
|
C.4
|
MON.ITM
|
C.1
|
C.1
|
C.2
|
C.1
|
C.1
|
C.1
|
EME.COM
|
C.3
|
C.1
|
C.2
|
C.4
|
C.2
|
C.2
|
MAN.ITM
|
C.1
|
C.2
|
C.1
|
C.1
|
C.2
|
C.1
|
MON.SIT
|
C.1
|
C.1
|
C.2
|
C.1
|
C.1
|
C.1
|
EME.MAS
|
C.3
|
C.1
|
C.2
|
C.4
|
C.2
|
C.2
|
MAN.SPI
|
C.1
|
C.2
|
C.1
|
C.1
|
C.2
|
C.1
|
MON.GEO
|
C.1
|
C.1
|
C.2
|
C.1
|
C.1
|
C.1
|
EME.ANI
|
C.3
|
C.2
|
C.3
|
C.4
|
C.3
|
C.3
|
MAN.LOG
|
C.1
|
C.2
|
C.2
|
C.1
|
C.2
|
C.1
|
MON.ANO
|
C.1
|
C.2
|
C.3
|
C.2
|
C.1
|
C.2
|
EME.DIS
|
C.3
|
C.2
|
C.3
|
C.4
|
C.3
|
C.3
|
MAN.DRA
|
C.2
|
C.2
|
C.2
|
C.3
|
C.3
|
C.2
|
MON.CAL
|
C.2
|
C.3
|
C.3
|
C.2
|
C.3
|
C.3
|
EME.REP
|
C.4
|
C.3
|
C.3
|
C.4
|
C.3
|
C.4
|
MAN.WIN
|
C.2
|
C.3
|
C.3
|
C.3
|
C.3
|
C.3
|
MON.CRO
|
C.2
|
C.4
|
C.3
|
C.2
|
C.3
|
C.3
|
EME.ANI
|
C.4
|
C.4
|
C.4
|
C.4
|
C.4
|
C.4
|
MAN.ERO
|
C.2
|
C.4
|
C.4
|
C.3
|
C.4
|
C.4
|
DOC.GEO
|
C.1
|
C.1
|
C.1
|
C.2
|
C.3
|
C.1
|
EVA.ALA
|
C.2
|
C.3
|
C.3
|
C.4
|
C.2
|
C.3
|
REV.OPM
|
C.1
|
C.4
|
C.4
|
C.2
|
C.3
|
C.3
|
DOC.HYD
|
C.1
|
C.1
|
C.1
|
C.2
|
C.4
|
C.2
|
EVA.ROU
|
C.3
|
C.3
|
C.3
|
C.4
|
C.2
|
C.3
|
REV.MON
|
C.1
|
C.4
|
C.4
|
C.2
|
C.4
|
C.3
|
DOC.SPI
|
C.2
|
C.1
|
C.1
|
C.3
|
C.4
|
C.2
|
EVA.MAS
|
C.3
|
C.3
|
C.3
|
C.4
|
C.2
|
C.3
|
REV.FOU
|
C.2
|
C.4
|
C.4
|
C.3
|
C.4
|
C.4
|
DOC.WIN
|
C.2
|
C.2
|
C.2
|
C.3
|
C.4
|
C.3
|
EVA.SUP
|
C.4
|
C.3
|
C.4
|
C.4
|
C.2
|
C.4
|
REV.MAX
|
C.2
|
C.4
|
C.4
|
C.3
|
C.4
|
C.4
|
DOC.ENG
|
C.2
|
C.3
|
C.3
|
C.3
|
C.4
|
C.3
|
EVA.JUR
|
C.4
|
C.4
|
C.4
|
C.4
|
C.3
|
C.4
|
REV.HYD
|
C.2
|
C.4
|
C.4
|
C.4
|
C.4
|
C.4
|
DOC.FOU
|
C.4
|
C.4
|
C.4
|
C.4
|
C.4
|
C.4
|
EVA.ADM
|
C.4
|
C.4
|
C.4
|
C.4
|
C.3
|
C.4
|
REV.CEO
|
C.4
|
C.4
|
C.4
|
C.4
|
C.4
|
C.4
|
4.3. Consistency of Responses
Inconsistency occurs when ordinal scales are allocated to subjective preferences of the experts. Thus, it is necessary test what is acceptable. The consistency ratio (CR) compares the consistency index (CI) of the matrix of pairwise comparisons against the consistency index of a random-like matrix (RI). The consistency index for criterion weights is calculated as shown in equation (1):
![](https://myfiles.space/user_files/58854_b38fc7f3db2c487f/58854_custom_files/img1618974344.png)
The denominator RI is the random index table value according to the number of criteria (Saaty & Vargas, 2012). The value for a criterion to be judged as consistent must be CR ≤ 0.1 (Saaty, 1990). For nine criteria, the RI value is 1.45. We tested the consistency of the experts’ pairwise comparisons for them. Table 8 shows that all the participants had CR values lower than 0.1.
Table 8: Consistency ratios of the experts' preferences related to the criteria.
|
GM
|
EE
|
ME
|
CE
|
CO
|
λmax
|
9.368
|
9.408
|
9.349
|
9.339
|
9.313
|
C.I.
|
0.046
|
0.051
|
0.044
|
0.042
|
0.039
|
C.R.
|
0.031
|
0.035
|
0.030
|
0.029
|
0.027
|