Preprint
Article

Comparative Review of Multi-Criteria Decision-Making Methods

Altmetrics

Downloads

99

Views

41

Comments

0

Submitted:

19 August 2024

Posted:

20 August 2024

You are already at the latest version

Alerts
Abstract
Performance analysis of companies is critical for decision-makers in the financial sector, particularly when resources are limited. This study investigates the properties of various Multi-Criteria Decision-Making (MCDM) methods, critically comparing their results in evaluating the performance of consumer goods companies listed on the Nigerian Stock Exchange (NSE). The analysis utilizes financial ratios, with weights derived using Buckley's Column Geometric Mean Method, a fuzzy ranking approach. The study concludes by comparing TOPSIS, VIKOR, and ELECTRE methods, determining the final rankings of these companies, and interpreting these rankings based on financial ratios over a five-year period (2016-2020).
Keywords: 
Subject: Computer Science and Mathematics  -   Applied Mathematics

1. Introduction

Multi-criteria decision-making (MCDM) methods are mathematical models that help to make decisions in scenarios where the possible alternatives are evaluated over multiple conflicting criteria.
The application areas of these methods are very enormous. Examples can be found in financial sector, supplier selection, technical evaluation of tenderers, evaluation of service quality or in renewable energy.
MCDM/MADM concentrates on problems with discrete decision spaces. In these problems, the set of decision alternatives has been predetermined.

1.1. Basics and Concepts

Despite the fact that MCDM methods may be widely distinct, a large number of them are common in some aspects [1]. These are known as the concept of alternatives and attributes which are often called goals or decision criteria as described next.

1.2. Alternatives

Typically, alternatives represent the various options for action available to the decision maker. The set of alternatives in this work is assumed to be finite, ranging from several to hundreds. They are to be screened, prioritized, and finally ranked.

1.3. Multiple Attributes

Multiple attributes are linked with each MCDM problem. Attributes might alternatively be called “goals" or “decision criteria." The different dimensions from which the alternatives can be examined are represented by attributes. When there are a lot of criteria (e.g., more than a dozen), the criteria might be placed in a hierarchical order. That is, certain criteria may be more important than others. Each major criterion may be linked to a number of sub-criterion. Similarly, each sub-criterion may be linked to several sub-sub-criteria, and so on. Although some MCDM methods explicitly consider a hierarchical structure in a decision problem’s criteria, the vast majority of them assume a single level of criteria (e.g., no hierarchies).

1.4. Conflict among Criteria

Different criteria may conflict with one another because they represent different dimensions of the alternatives. Cost, for example, may conflict with profit, and so on. Unless explicitly stated otherwise, no such conflict is assumed in this treatise.

1.5. Incommensurable Units

Different units of measurement may be associated with different criteria. For example, when purchasing a used car, the criteria “cost" and “mileage" may be measured in euros and thousands of miles, respectively. Because of the need to consider multiple units, MCDM problems are inherently difficult to solve.

1.6. Decision Weights

The majority of MCDM methods necessitate assigning weights of importance to the criteria. These weight values are usually normalized to add up to one. In the case of n attributes, a weight set is given as
w ̲ T = w 1 , w 2 , , w n and j = 1 n w j = 1
The decision maker can assign the weights directly, or they can be calculated using the eigenvector method or the weighted least square method [1].

1.7. Decision Matrix

A matrix format is the simple way to express an MCDM problem. A decision matrix A is an ( m × n ) matrix in which element a i j represents the performance of alternative A i when evaluated in terms of decision criterion C i (for i = 1 , 2 , 3 , , m , and j = 1 , 2 , 3 , , n ) . It is also assumed that the decision maker determined the relative performance weights of the decision criteria (denoted as w j , for j = 1 , 2 , 3 , , n .
Table 1. A Typical Decision Table.
Table 1. A Typical Decision Table.
Preprints 115698 i001
There are no clear guidelines for which MCDM method should be used to solve a specific MCDM problem. This is a controversial topic that has been researched in the literature for many decades. It is true that the solution may differ depending on the MCDM method used, especially when the alternatives are very similar [2].
As a result, in this paper, we aim to evaluate the properties of various Multi Criteria Decision Making (MCDM) methods and compare their outcomes in terms of assessing the performance of consumer goods companies using financial ratios and financial experts’ perspectives.

2. Literature Review

Multi-criteria decision-making (MCDM) emerged in the 1970s as an intriguing field for many researchers. More than 70 multi-criteria decision-making strategies have been discovered since its start until today [3], where the MCDM is classified into two types: Multi-objective Decision-making (MODM) and Multi-attribute Decision-making (MADM).
The goal of multi-objective decision-making (MODM) is to provide alternatives that optimize decision-makers’ numerous objectives [4]. The choices are often very large or infinite, and the best choice will be the one that meets the decision makers’ preferences and limits.
Furthermore, multi-attribute decision-making (MADM) entails selecting a specific alternative from those previously identified based on attributes, so that it is known as the best one [5].
In general, the MADM is used to solve problems with a restricted number of solutions. Each of the MCDM procedures has its own set of analysis models, information needs, underlying assumptions, and decisions [6]. This indicates that the most appropriate strategies must be chosen to tackle the problem being examined; otherwise, we will arrive at misleading solutions if an improper methodology is employed. Inappropriate decisions will result in significant losses. Thus, given the enormous number of MCDM techniques available, choosing the proper technique to tackle the problem is the most crucial question to consider before making a decision.
Multiple non-commensurable and conflicting criteria, diverse units of measurement among the criteria, and the presence of different alternatives are all features of MCDM analysis [6]. MODM and MADM problems can be further divided into two different categories based on the decision-aim maker’s preference structure. (1) If a single goal-preference structure exists, the problem is referred to as individual decision making, regardless of the number of decision-makers engaged. (2) If individuals (interest group) have distinct goal-preference structures, then the problem is referred to as group decision making [7].

2.1. Decision Making under Certainty Versus Uncertainty

  • MCDM Under Certainty: For the decision under certainty, it is assumed that all necessary information about the decision circumstance is known, and that there is a known deterministic relationship between each decision and the related outcome.
  • MCDM Under Uncertainty: In a decision situation, there are different kinds of uncertainty. The first is the uncertainty caused by a lack of information about the choice circumstance, and the second is the uncertainty caused by fuzziness (impression) in the description of the semantic meaning of the events, phenomena, or claims. As a result, depending on the nature of uncertainty, both MODM and MADM problems under uncertainty can be further classified into probabilistic and fuzzy decision-making problems.

2.2. Selecting MCDM Techniques

MCDM approaches are extremely diverse, which can be viewed as both a strength and a weakness. Diversity promotes flexibility in selecting a suitable technique for a given problem from a large pool of choices, but the vast diversity of these techniques complicates the optimal decision. Each technique has its own set of advantages and disadvantages [6]. Three methods have been addressed in depth in this study, highlighting their merits and drawbacks. With regard to the performance assessment problem of consumer goods companies, the step-by-step computation of the selection of the best company in terms of performance has been presented for each particular technique, and the results were compared.
The application of selection techniques for problems was not considered early in the evolution of MCDM, but it is now clear that mismatches can result in suboptimal results and it can also deter potential users from applying MCDM techniques to real-world situations by causing them to abandon useful models owing to an inappropriate application (leading to time and money losses). The three approaches chosen (TOPSIS, VIKOR, and ELECTRE) are among the most commonly utilized.

2.3. Application of MCDM Approaches in Performance Analysis

Altman created the first financial analysis studies that were evaluated using objective methodologies [8]. Using financial ratios, Altman developed a discriminant function known as the z score model. Due to the problems experienced in data entry and acquisition, a system based on prior years’ data was created.
In the 1980s, the use of MCDM methodologies to measure a company’s financial performance became more common [9].
Feng and Wang [10] also used TOPSIS to investigate the performance of five Taiwanese airlines and came to the conclusion that financial variables are important in determining airline performance.
Yurdakul and İç [11] investigated the financial structures and industry conditions of five large-scale automobile businesses. The performance values for each year were compared to the year-end closing prices of the securities, and the results were determined to be consistent beginning in 2001.
Mahmoodzadeh [12] used Fuzzy AHP, TOPSIS, and classic project evaluation methodologies such as net present value, rate of return, benefit-cost analysis, and payback time to estimate the preference ranking of various projects.
Wu [13] suggested a fuzzy MCDM approach for evaluating banking performance using the Balanced Scorecard (BSC). For this reason, twenty-three performance evaluation indices for BSC’s banking performance were chosen using expert questionnaires.
Bülbül and Köse [14] used the TOPSIS and ELECTRE techniques to assess the financial performance of the food sector by firm and sector and found consistent results.

3. Data and Methodology

The aim of our research is to assess the properties of different MCDM methods and compare the results of them in terms of evaluating the performance of 10 consumer goods companies listed on the Nigerian Stock Exchange with the help of financial ratios composing of five year ( 2016 2020 ) data set. For this purpose, some activities were carried out.

3.1. Data

  • The financial ratios of each consumer goods company listed in NSE were calculated. Eight financial ratios namely current, acid test, cash, leverage, asset turnover, net profit/total assets, net profit/capital and net profit/net sales were considered.
  • Then a survey evaluating the financial ratios was designed and applied for determining the weights of criteria. Survey was based on Saaty’s scale in order to weigh criteria by pairwise comparison in hierarchical structures and 10 consumer goods companies listed in NSE were taken into consideration as alternatives.
  • We then started the data collection process. Experts and students in the field of Accounting, Business management, Banking and Finance were selected. They were asked to compare criteria with respect to goal and determine their relative importance. As a result, 22 complete surveys were collected and analyzed. Google Forms was employed to carry out our survey.
  • Afterward, weights of the criteria were acquired from the survey by using Buckley’s Column Geometric Mean approach, which is one of the fuzzy ranking methods.
  • After the weights of criteria are determined, criteria related values of 10 consumer goods companies listed in NSE within the period of 2016 2020 were obtained from NSE and companies’ websites.
  • Finally, for ranking companies via TOPSIS, VIKOR, and ELECTRE methods, Excel office 365 software is used. These three MCDM methods were picked because of their popularity, ease of application, and remarkable results in earlier studies, in order to check the financial performance of consumer goods companies [2].

3.2. Methodology

In this section, we will go through multi-criteria decision-making methods in depth. These MCDM methods are techniques that help stakeholders choose the appropriate selection tool in order to make the best decision. The various processes for applying selected approaches and their comparative evaluation would be discussed.
Similarly, Buckley’s model, which uses triangular fuzzy numbers to express decision makers’ evaluations of alternatives with regard to each criterion, will be described in this chapter.

3.2.1. Comparative Review of MCDM Methods

Despite the huge number of MCDM methods available, no single method is regarded as best suited for all types of decision-making problems [15,16]. This leads to the paradox that choosing an acceptable approach for a particular problem leads to an MCDM problem [17]. The fact that different approaches might produce different outcomes when applied to the same problem is a key criticism of MCDM [18]. Identifying and selecting an effective MCDM technique is obviously not a straightforward undertaking, and some thought must be devoted to method selection. A variety of practical applications of comparison studies of different MCDM approaches are presented in the literature [19,20].
Additionally, a number of scholars [15,16] have created criteria to aid in the selection of an acceptable MCDM technique. However, it has been noted that numerous strategies may be theoretically relevant for a given decision-making circumstance; there is not always a compelling reason to choose one strategy over another. One of the most essential criteria in adopting an MCDM technique appears to be its compliance with the problem’s purpose [20].
The problem proposed in this study is to examine the financial performance of some consumer goods companies listed on the NSE. To accomplish this, a ranking of options must be identified. As a result, the goal of this problem is to rate alternatives. Consequently, an MCDM approach capable of providing a complete ranking of options (showing the position of each alternative) is necessary. Furthermore, the approach must be capable of handling both benefit and cost criteria, as well as quantitative and qualitative criteria. Similarly, the MCDM strategy must be simple to use and understand so that any interested parties can readily adopt the proposed method. This research compares the performance of different applicable MCDM methodologies, including the TOPSIS, VIKOR and ELECTRE.
These strategies are used on the data from the practical case studies provided later in the final chapter of this thesis. The goal of each technique is to determine the relative importance of each alternative under consideration, as well as to establish the priority order of the alternatives in relation to one another. The selected methods for comparative analysis differ in their fundamental concepts, the type of data normalization process used, and how the criteria values and criteria weights are combined into the evaluation procedure.
Because criteria have multiple units of measurement in general, MCDM approaches use a sort of normalization to remove the units of criteria values (e.g., ratio, points, percentage, price) so that all criteria are non-dimensional [18]. Normalization procedures vary, but in many circumstances, this stage is critical to the method’s consistent and proper use. The normalisation procedures of the WSM, WPM, and revised AHP methods are essentially comparable, with the exception of TOPSIS.
It is to be noted that the advantages and disadvantages of these methods are crucial in selecting the efficient one amongst them. For instance, the advantage of the AHP method is that it does not necessitate the use of an additional instrument to determine the weight of the criteria, while its disadvantage is that as the number of criteria and possibilities rises, the process becomes more sophisticated. On the other hand, the merit of the TOPSIS method is that it is relatively straightforward, and the solution procedure remains constant regardless of the number of selection criteria and alternatives, while its demerit is that the correlation among criteria is not taken into account while calculating Euclidean distance. Furthermore, vector normalisation may be required when handling multi-dimensional problems.
Similarly, the VIKOR method which is a modernized version of TOPSIS becomes difficult in the face of a competing scenario. In addition, the positive effect of the ELECTRE method is that it can yield a solution even when data is missing, but because of the complicated evaluation techniques involved, the methodology is computationally difficult in the absence of software [21].

3.2.2. Analytic Hierarchy Process (AHP) Method

The analytic hierarchy process (AHP) is a prominent analytical technique for difficult decision-making situations. AHP was invented by Saaty ([22,23]), which breaks down a decision-making problem into a system of hierarchies of objectives, attributes (or criteria), and alternatives.
An AHP hierarchy can include as many levels as necessary to accurately represent a decision circumstance. AHP is a valuable methodology due to a variety of functional properties. These include the ability to deal with decision circumstances including subjective judgments, multiple decision-makers, and the ability to offer measurements of preference consistency [17].
AHP, which was designed to mirror how people actually think, remains the most highly acclaimed and extensively used decision-making approach. AHP can deal with both tangible (i.e., objective) and non-tangible (i.e., subjective) features efficiently, especially when the subjective judgments of diverse individuals play a key role in the decision process.
To make a decision in an organised way in order to generate priorities we need to decompose the decision into the following steps.
  • Define the problem and the type of information needed.
  • Structure the decision hierarchy from the top with the choice’s purpose, then the objectives from a broad perspective, via the intermediate levels (criteria on which following elements rely) to the bottom level (which usually is a set of the alternatives).
  • Create a collection of pairwise comparison matrices. Each element in an upper level is used to compare the components in the level directly below it.
  • Use the comparison priorities to weigh the priorities in the level directly below. Repeat for each element. Then, for each element on the level below, add its weighted values to determine its overall or global priority. Continue weighing and adding until the final priorities of the alternatives at the lowest level are obtained.
To perform comparisons, we need a numerical scale that specifies how many times more important or dominating one element is over another one in relation to the criterion or feature being compared. The scale is shown in Table 2.
A fair assumption is that if activity i is compared to activity j and has one of the preceding non-zero numbers given to it, then j has the reciprocal value when compared to i.
Similarly, if the activities are very close, assigning the optimal value may be challenging; but, when compared to other contrasting activities, the size of the small numbers would not be too visible, and they can still show the relative importance of the activities.
Moving forward, we are going to discuss about Fuzzy Analytic Hierarchy Process. But before we do that, we would first need to give an insight on Fuzzy Sets.

3.2.3. Fuzzy Sets

Zadeh [25] firstly proposed a mathematical theory known as fuzzy set in order to overcome the vagueness and imprecise condition of human cognitive processes [26]. Apart from classical set theory based on binary logic, fuzzy set describe actual objects similar to human language [27]. A fuzzy set which is extension of crisp one allows partial belonging of element by membership function denoted by μ M ( x ) . Membership values of objects in a fuzzy set range from 0 (nonmembership) to 1 (complete membership) and values between these boundaries are called intermediate membership degrees and show degree to which an element belongs to a set [28].
There are several definitions involving fuzzy sets which are extensions definitions for ordinary set. Now, suppose X is a space of points, and let x be an element of X, then:
Definition 1. 
A fuzzy set is empty if and only if its membership function is identically zero on X [25].
Definition 2. 
Two fuzzy sets M and N are equal, written as M = N, if and only if μ M ( x ) = μ N ( x ) for all x X .
Definition 3. 
The complement of a fuzzy set M is denoted by M and is defined by
μ M ( x ) , = 1 μ M ( x )
Definition 4. 
We say that a set M is contained in N (or, equivalently, M is a subset of N, or M is smaller than or equal to N) if and only if μ M ( x ) μ N ( x ) . That is,
M N μ M ( x ) μ N ( x )
Definition 5. 
We say that the union of two fuzzy sets M and N with respective membership functions μ M ( x ) and μ N ( x ) is a fuzzy set C, which is written as L = M N whose membership function is related to those of M and N by
μ L ( x ) = Max μ M ( x ) , μ N ( x ) , x X
Definition 6. 
We say that the intersection of two fuzzy sets M and N with respective membership functions μ M ( x ) and μ N ( x ) is a fuzzy set L, which is written as L = M N , whose membership function is related to those of M and N by
μ L ( x ) = Min μ M ( x ) , μ N ( x ) , x X

3.2.4. Fuzzy Numbers

Definition 7 
([29]). A fuzzy number ( F N ) is a fuzzy set in R , namely a mapping μ : R [ 0 , 1 ] , with the following properties:
  • μ is convex, i.e. μ ( x ) min { μ ( s ) , μ ( r ) } , for s x r ;
  • μ is normal, i.e. ( ) x 0 R : μ x 0 = 1 ;
  • μ is upper semi-continuous, i.e.
( ) x R , ( ) α ( 0 , 1 ] : μ ( x ) < α , ( ) δ > 0 such that | s x | < δ μ ( s ) < α
Furthermore, triangular and trapezoidal fuzzy numbers are mostly used in application fields. Triangular fuzzy numbers are employed in this study due to their ease of computation and effectiveness as a representation.

3.2.5. Triangular Fuzzy Number

A triangular fuzzy number M ˜ = ( l , m , u ) is a feature expressed by means of the membership function μ M ˜ : R [ 0 , 1 ] such that μ M ˜ ( x ) expresses the degree (determined by an expert or a group of experts) to which x possesses this feature and
μ M ˜ ( x ) = 0 x l x l m l l < x m u x u m m x < u 0 x u
Membership of triangular fuzzy number is defined by three real numbers expressed as ( l , m , u ) indicating smallest possible value, the most promising value and the largest possible value respectively [30].
Figure 1. Triangular fuzzy number diagram.
Figure 1. Triangular fuzzy number diagram.
Preprints 115698 g001

3.2.6. Basic Operations on Triangular Fuzzy Numbers

Let M = l 1 , m 1 , u 1 and N = l 2 , m 2 , u 2 be two positive triangular fuzzy numbers then:
  • Addition: M + N = l 1 , m 1 , u 1 + l 2 , m 2 , u 2 = l 1 + l 2 , m 1 + m 2 , u 1 + u 2
  • Subtraction: M N = l 1 , m 1 , u 1 l 2 , m 2 , u 2 = l 1 u 2 , m 1 m 2 , u 1 + l 2
  • Multiplication: M · N = l 1 , m 1 , u 1 · l 2 , m 2 , u 2 = l 1 · l 2 , m 1 · m 2 , u 1 · u 2
  • Division: M N = l 1 , m 1 , u 1 l 2 , m 2 , u 2 = l 1 u 2 , m 1 m 2 , u 1 l 2
  • Inverse: M 1 = l 1 , m 1 , u 1 1 1 / u 1 , 1 / m 1 , 1 / l 1
  • Scalar Multiplication:
    λ > 0 , λ R , λ M = λ l 1 , λ m 1 , λ u 1 ,
    λ < 0 , λ R , λ M = λ u 1 , λ m 1 , λ l 1 .

3.2.7. One-Sided Fuzzy Numbers

One-sided fuzzy numbers (left-sided or right-sided) are triangular fuzzy numbers such that
a. For a left-sided fuzzy number, l =
b. For a right-sided fuzzy number, u = .
It’s essential to specify which values of the membership function accepts positive values, i.e., the support for fuzzy numbers.

3.2.8. the Support of the Triangular Fuzzy Number

The support of the triangular fuzzy number M ˜ = ( l , m , u ) is the open interval ( l , u ) , the support of the left-sided fuzzy number is the half-line ( , u ) and that of the right-sided fuzzy number is the half-line ( l , ) .
For example, in the banking sector [31] the experts of each bank can define their understanding of such terms as “very low", “low", “average", “high" and “very high" financial security, or “very low", “low", “average", “high" and “very high" yearly income of the potential borrower. As financial stability is difficult to quantify, experts would be requested to describe the first five elements using a predetermined scale (e.g., from 0 to 5). Also because the borrower’s annual income is quantifiable, the scale for the last five elements would correspond to the income values directly. Figure 2 presents example definitions of the terms, generated on the basis of expert opinions.
Five examples of fuzzy numbers are shown in Figure 2: three triangular, one left-sided, and one right-sided. They represent a number of different aspects of financial security or income (measured on the horizontal axis using units which must be predefined, such as hundreds of thousands of dollars for the income or an arbitrary scale for financial security). Each income value may have one or more of the six characteristics in whole, in part, or not at all. For example, income of 1.5 is neither extremely high nor very low; it is medium to the degree of 0.25 , low to the degree of 0.75 , and very low to the degree of 0.75 (according to the experts).

3.2.9. Fuzzy AHP

Since AHP can not reflect human thinking style in inaccurate and subjective environment due to unbalanced scale of judgments, inability to adequately handle inherent uncertainty and imprecise pair-wise comparisons. In order to effectively handle subjective perceptions and impreciseness, fuzzy numbers are integrated with AHP, allowing the appropriate expression of linguistic evaluation [33]. In examining real-world decision-making problems, fuzzy numbers are also utilized to deal with uncertainties affecting subjective choices.
For that reason, fuzzy analytic hierarchy process (FAHP) extension of traditional AHP was developed to solve hierarchical fuzzy problems in interval judgment matrix [34].
The fuzzy set theory allows respondents to explain semantic judgments subjectively [27]. As a result, in terms of triangular fuzzy numbers, Saaty’s 9-point scale is turned into the fuzzy ratio scale.
One of the fundamental problems in fuzzy optimization and fuzzy decision-making is ranking fuzzy numbers in an imprecise and ambiguous environment. Fuzzy values are rated using different fuzzy set criteria, such as the center of attraction, an area under the membership degree function, and some intersecting points [1]. As a result, the FAHP methodology is well adapted to handling subjective evaluation decision-making problems, and it is currently one of the most extensively used MCDM methods in the domains of business, management, manufacturing, industry, and government.
The first FAHP method was proposed by Van Laarhoven [35] using triangular fuzzy numbers in the pairwise comparison matrix.
Fuzzy pairwise comparison matrices were created in the FAHP approaches by leveraging linguistic evaluations of the decision makers’ judgments. Using triangular fuzzy numbers, the logarithmic regression method is utilized to create fuzzy weights or fuzzy performance scores through AHP operations [35]. Table 3 displays linguistic variables for pairwise comparisons of each criterion.
Buckley [37] developed a model to state decision maker’s evaluation on alternatives with respect to each criterion by using triangular fuzzy numbers.

3.2.10. Buckley’s Column Geometric Mean Method

The steps for Buckley’s Column Geometric Mean method are illustrated as follows:
  • Establishing the hierarchical structure and comparing criteria or alternatives via fuzzy scale for constructing pairwise comparison matrix.
    A ¯ k = a ¯ 11 k a ¯ 12 k . . . a ¯ 1 n k a ¯ 21 k a ¯ 22 k . . . a ¯ 2 n k . . . . . . . . . . . . a ¯ m 1 k a ¯ m 2 k . . . a ¯ m n k
  • Taking the average of preferences of all decision-makers and obtain a new pairwise comparison matrix.
    a ¯ i j = k = 1 K a i j k K
    A ¯ = a ¯ 11 a ¯ 12 . . . a ¯ 1 n a ¯ 21 a ¯ 22 . . . a ¯ 2 n . . . . . . . . . . . . a ¯ m 1 a ¯ m 2 . . . a ¯ m n
  • Calculating the Geometric Mean of each criterion.
    g i ¯ = j = 1 n a ¯ i j 1 / n , i = 1 , 2 , . . . , m
  • Obtaining the fuzzy weights of each criterion.
    w ¯ i = z ¯ i ( z ¯ 1 z ¯ 2 . . . z ¯ n ) 1 = ( l i , m i , u i )
  • Transformation of fuzzy weights composed of fuzzy triangular numbers into crisp ones using center of area defuzzification techniques.
    S i = l i + m i + u i 3
  • Normalizing the obtained crisp weights.
    N i = S i i = 1 m S i

3.2.11. TOPSIS Method

TOPSIS (for the Technique for Order Preference by Similarity to Ideal Solution) is a well known, classic ranking method, which was developed by Hwang [38]. It was actually created to analyze MCDM problems. Basically, this technique is to choose an alternative having the shortest Euclidean distance from positive ideal solution (PIS) which maximizes benefits and minimizes cost, and the farthest distance from negative ideal solution (NIS) which maximizes cost and minimizes benefit.
The TOPSIS methodology posits that each criterion has a tendency to increase or decrease utility in a monotonic manner. As a result, defining ideal and negative-ideal solutions is simple. To determine how close the alternatives are to the ideal solution, the Euclidean distance method was introduced. As a result, the method focuses on calculating both the best, or ideal scenario, and the worst, or negatively ideal case. TOPSIS chooses the alternative whose value is closest to the ideal solution and farthest from the negatively ideal solution [39]. The following are the significant highlights of TOPSIS:
  • It’s a really clear and understandable method, with each step of the computation being very logical and understandable.
  • Simple and uncomplicated computations are involved.
  • In addition to the development of the most optimal example, this technique includes the generation of the ideal and negative ideal cases (practically feasible solution).
The major steps involved are:
  • Forming decision matrix A = ( a i j ) m x n for m alternatives and n criteria. Where ( i = 1 , 2 , m ) and ( j = 1 , . . . , n ) .
    A = a 11 a 12 a 1 j a 1 n a 21 a 22 a 2 j a 2 n a i 1 a i 2 a i j a i n a m 1 a m 2 a m j a m n
  • Normalizing the decision matrix.
    r i j = a i j i = 1 m a i j 2 , i = 1 , 2 , , n j = 1 , 2 , , m .
  • Weighting normalized decision matrix.
    v i j = r i j × w i j , i = 1 , 2 , , n j = 1 , 2 , , m .
  • Finding positive and negative ideal solutions.
    P I S = A * = v 1 * , v 2 * , , v m * = max i v i j j Ω b , min i v i j j Ω c
    N I S = A = v 1 , v 2 , , v m = min i v i j j Ω b , max i v i j j Ω c
  • Calculating the Euclidean distance of alternatives from positive and negative ideal solutions.
    d i * = j = 1 m ( v i j v j * ) 2 , i = 1 , 2 , , n .
    d i = j = 1 m ( v i j v j ) 2 , i = 1 , 2 , , n .
  • Calculating the relative closeness ( R C i ) of each alternative to ideal solutions.
    R C i = d i d i + d i * , i = 1 , 2 , , n R C i [ 0 , 1 ]
  • Finally, ranking alternatives according to their R C i values in descending order from 1 to 0 and choosing the highest one.

3.2.12. Fuzzy TOPSIS

Chen [40] introduced the Fuzzy TOPSIS approach, which is an extension of the TOPSIS method using triangular fuzzy numbers in order to address multi-criteria decision-making problems under uncertainty. The decision-makers employ linguistic variables, D r ( r = 1 , , k ) , to evaluate the weights of the criteria and the ratings of the alternatives. As a result, W ˜ r j denotes the weight of the jth criterion, C j ( j = 1 , , m ) , as determined by the rth decision-maker. Similarly, x ˜ i j r denotes the evaluation of the ith alternative, A i ( i = 1 , , n ) , in terms of criterion j, as determined by the rth decision-maker. That being said, the Fuzzy TOPSIS method comprises the following steps [41]:
  • Aggregate the weights of criteria and ratings of alternatives given by k decision makers, as expressed in Equations (22) and (23) respectively:
    w ˜ j = 1 k w ˜ j 1 + w ˜ j 2 + + w ˜ j k
    x ˜ i j = 1 k x ˜ i j 1 + x ˜ i j r + + x ˜ j k
  • Arrange the fuzzy decision matrix of the alternatives D ˜ and the criteria W ˜ as follows:
    C 1 C 2 C j C m
    D ˜ = A 1 A i A n x ˜ 11 x ˜ 12 x ˜ 1 j x ˜ 1 m x ˜ n 1 x ˜ n 2 x ˜ n j x ˜ n m
    W ˜ = w ˜ 1 + w ˜ 2 + + w ˜ m
  • Normalize the fuzzy decision matrix of the alternatives D ˜ using linear scale transformation. The normalized fuzzy decision matrix R ˜ is given by:
    R ˜ = r ˜ i j m × n
    r ˜ i j = l i j u j + , m i j u j + , u i j u j + and u j + = max i u i j ( benefit criteria )
    r ˜ i j = l j u i j , l j m i j , l j l i j and l j = max i l i j ( cos t criteria )
  • Compute the weighted normalized decision matrix, V ˜ , by multiplying the weights of the evaluation criteria, w ˜ j , by the elements r ˜ i j of the normalized fuzzy decision matrix
    V ˜ = v ˜ i j m × n where v ˜ i j is given by v ˜ i j = x ˜ i j × w ˜ j
  • Compute the weighted normalized decision matrix, V ˜ , by multiplying the weights of the evaluation criteria, w ˜ j , by the elements r ˜ i j of the normalized fuzzy decision matrix
    V ˜ = v ˜ i j m × n where v ˜ i j is given by v ˜ i j = x ˜ i j × w ˜ j
  • Define the Fuzzy Positive Ideal Solution (FPIS, A + ) and the Fuzzy Negative Ideal Solution (FNIS, A ) as follows:
    A + = v ˜ 1 + , v ˜ j + , , v ˜ m + and A = v ˜ 1 , v ˜ i , , v ˜ m
    where v ˜ j + = ( 1 , 1 , 1 ) and v ˜ j = ( 0 , 0 , 0 ) .
  • Compute the distances d j + and d i of each alternative from respective v ˜ j + and v ˜ j :
    d i + = j = 1 n d v v ˜ i j , v ˜ j + and d i = j = 1 n d v v ˜ i j , v ˜ j
    where d ( . . ) , according to the vertex method, represents the distance between two fuzzy numbers. For triangular fuzzy numbers, this is stated as:
    d ( x ˜ , z ˜ ) = 1 3 l x l z 2 + m x m z 2 + u x u z 2
  • Compute the closeness coefficient, C C i
    C C i = d i d i + + d i
  • Define the ranking of the alternatives according to the closeness coefficient, C C i , in decreasing order. The optimal alternative is the one that is closest to the FPIS but farthest away from the FNIS.

3.2.13. VIKOR Method

The VIKOR method developed by Opricovic [42] is, as TOPSIS method, also based in the idea of the distances to “ideal solutions”. This method is oriented for selecting and ranking alternatives in case of conflicting criteria. The compromised solution is the closest to the ideal one.
The VIKOR Method can be used for solving MCDM problems if the following conditions are satisfied:
  • Compromised solution should be accepted in order to overcome conflict.
  • Decision maker is willing to accept the closest solution to ideal one.
  • A linear relationship between benefit and each criteria function for decision maker.
  • Alternatives should be evaluated in terms of each criteria.
  • Preferences of decision makers are expressed by weights.
  • Decision makers are responsible for approving the final solution.
The Steps of VIKOR Method are as follows:
  • Identification of best ( f a * ) and worst ( f a ) values for each criteria. If evaluation criteria ( b = 1 , 2 , , n ) is based on benefit;
    f b * = max a x a b f b = min a x a b
    If evaluation criteria ( b = 1 , 2 , , n ) is based on cost;
    f b * = min a x a b f b = max a x a b
  • In order to make comparisons we obtained a normalization matrix. During the normalization process, the Decision matrix X composed of k criteria and l alternatives was transformed into normalization matrix S with same dimensions. The decision matrix X with elements ( x k l ) before normalization is given as:
    X = x 11 x 12 x 1 b x 1 l x 21 x 22 x 2 b x 2 l x a 1 x a 2 x a b x a l x k 1 x k 2 x k b x k l
    The normalization matrix S with elements ( s k l ) obtained after normalizing is given as:
    S = s 11 s 12 s 1 b s 1 l s 21 s 22 s 2 b s 2 l s a 1 s a 2 s a b s a l s k 1 s k 2 s k b s k l s a b = f b * x a b f b * f b
  • We find normalized decision matrix. By multiplying criteria weights ( w b ) and normalized decision matrix elements ( s a b ) .
    T = t 11 t 12 t 1 b t 1 l t 21 t 22 t 2 b t 2 l t a 1 t a 2 t a b t a l t k 1 t k 2 t k b t k l t a b = s a b × w a b
  • We calculate S a (mean group score) and R a (worst group score) values for each alternatives.
    S a = b = 1 l w b f b * x a b f b * f b R a = max b w b f b * x a b f b * f b
  • We calculate Q a values for each alternatives. The values of Q a is obtain through the values of S * , S , R * , R . We also use additional y parameter which states the weight of alternative providing maximum group benefit and ( 1 y ) parameter for minimum individual regret. Generally, we assume y = 0.5 .
    S * = min a S a S = max a S a R * = min a R a R = max a R a
    Q a = y S a S * S S * + ( 1 y ) R a R * R R *
  • We rank the values of S a , R a , and Q a from lower to higher. We also control alternative having minimum Q a values by two conditions to know if the ranking is accurate. The conditions are known as acceptable advantage and acceptable stability.
    Acceptable advantage condition: According to Q a values, if first Q ( C 1 ) and second alternative Q ( C 2 ) satisfied significant difference, we calculate threshold value D Q which depend on alternative number. If the number of alternative is lower than 4, then the value of D Q equals to 0.25 [43]
    Q ( C 2 ) Q ( C 1 ) D Q D Q = 1 k 1
    Acceptable stability condition: According to Q a values, first alternative Q ( C 1 ) should get at least one best score for values of S and R. If these two conditions are not satisfied, then we formed a compromised solution set in two ways:
    (a)
    If the second condition is not satisfied, then the first and second alternatives are accepted as the compromised solution.
    (b)
    If the first condition is not satisfied, then C 1 , C 2 , , C k alternatives are contained in compromised solution set according to Q ( C 2 ) Q ( C 1 ) D Q [44]

3.2.14. ELECTRE Method

The ELECTRE (for Elimination and Choice Translating Reality; English translation from the French original) method was devised in 1966 [45], which is an outranking method and is based on partial aggregation. The idea of this method is to rank alternatives based on concord and discord indexes that are calculated with extracted data from a decision table. As Mateo [39] mentioned this method has 4 main steps. In the first phase, weight must be assigned to each criterion based on a normalization theory such that the sum of all weights equals 1, and a threshold function must be developed. The concordance and discordance indexes for a pair of alternatives must be determined in the second phase. The outranking degree for each pair of alternatives must then be determined using the concordance and discordance indexes.
Finally, a partial ranking will be determined by taking into account all pairs of alternatives. According to Hui-Fen Li [39], “the weakness of normal ranking of ELECTRE is that it requires the introduction of an additional threshold, and the ranking of the alternative is dependent on the size of this threshold for which there is no correct value". ELECTRE, on the other hand, has the advantage of being able to manage both quantitative and qualitative data when outranking alternatives.
In order to solve decision problems more than two criteria of ELECTRE methods can be preferred to other ones if at least one of these conditions is satisfied: a) Performances of criteria are expressed in different units and decision maker does not want to use complex and difficult common scale. b) The problem does not tolerate a compensation effect. c) If there is requirement to use indifference and preference thresholds such that sum of small differences is decisive apart from insignificant small differences. d) If alternatives are weak interval or any order scale in which it is difficult to compare differences [45].
ELECTRE also allows decision makers to avoid compensation between criteria and any normalization process that can distort original data, also uncertain conditions are being considered. On the other hand these methods require difficult technical parameters which are not easily understandable [45].
There are many versions of ELECTRE Method such as ELECTRE (II, III, IV etc.) which can be used according to the types of decision problem. The steps of ELECTRE method can be illustrated as follows [46]:
  • Forming decision matrix A = ( a i j ) m x n for m alternatives and n criteria. Where ( i = 1 , 2 , m ) and ( j = 1 , . . . , n ) .
    A = a 11 a 12 a 1 j a 1 n a 21 a 22 a 2 j a 2 n a i 1 a i 2 a i j a i n a m 1 a m 2 a m j a m n
  • Forming normalized decision matrix X = ( x i j ) m × n for m alternatives and n criteria.
    X = x 11 x 12 x 1 j x 1 n x 21 x 22 x 2 j x 2 n x i 1 x i 2 x i j x i n x m 1 x m 2 x m j x m n
    Where:
    x i j = a i j i = 1 m a i j 2 , for maximization objective
    x i j = 1 a i j i = 1 m ( 1 a i j ) 2 , for minimization objective
  • Calculating weighted normalized decision matrix V = ( v i j ) m × n . By multiplying weight of criterion w j and elements of normalized decision matrix x i j
    v i j = w j × x i j
  • Determining concordance and discordance sets for each pairs of the alternatives A p and A q If alternative A p is preferred to alternative A q for all criteria concordance set C ( p , q ) , then the collection of criteria where A p is better than or equal to A q is formed as:
    C ( p , q ) = j v p j v q j
    Similarly, if alternative A p is worse than A q for all criteria discordance set D ( p , q ) , then the collection of criteria where A p is worse than A q is formed as:
    D ( p , q ) = j v p j < v q j
    According to the above formulations V p j is defined as the weighted normalized rating of alternative A p with regard to jth criterion and V q j is defined as weighted normalizing rating of alternative A q with respect to jth criterion.
  • Calculating the concordance C p q and discordance D p q indexes as follows:
    C p q = j * w j * , where j * is criteria in C ( p , q )
    And
    D p q = j + | v p j + v q j + | j | v p j v q j | where j + is criteria in D ( p , q )
  • Relationship between alternatives are outranked by computing the averages of C p q and D p q which are represented by C ¯ and D ¯ respectively. According to this method, the alternative of A p outrank the alternative of A q if and only if C p q C ¯ and D p q D ¯ conditions are satisfied.
  • Calculating Net concordance C p and discordance D p indexes. The ultimate ranking is obtained according to ordering C p values from higher to lower and D p values from lower to higher.
    C p = k = 1 m C p k k = 1 m C k p k p
    D p = k = 1 m D p k k = 1 m D k p k p

4. Research and Result

4.1. Research

In this section, we are going to consider a practical case for our study. That is we are going to select three MCDM methods which are ELECTRE, TOPSIS and VIKOR, and use them to evaluate the financial performance of 10 selected consumer goods companies listed on the Nigerian Stock Exchange.
We will start by weighting the criteria that were acquired from the survey using Buckley’s column geometric mean approach through the following steps:
Step 1: We first create a pair-wise comparison matrix using Saaty’s scale from AHP Method.
Table 4. Pairwise comparison matrix
Table 4. Pairwise comparison matrix
Criteria Current Ratio Acid Test Ratio Cash Ratio Leverage Ratio Asset Turnover Ratio Net Profit /Total Asset Net Profit/Capital Net Profit/Net Sales
Current Ratio 1 1/7 1/8 8 1/6 1/7 1/7 1/6
Acid Test Ratio 7 1 1/8 9 1/5 1/7 6 1/6
Cash Ratio 8 8 1 9 4 1/9 7 6
Leverage Ratio 1/8 1/9 1/9 1 1/6 1/9 2 1/5
Asset Turnover Ratio 6 5 1/4 6 1 1/6 5 3
Net Profit/Total Assets 7 7 9 9 6 1 8 5
Net Profit/Capital 7 1/6 1/7 1/2 1/5 1/8 1 1/7
Net Profit/Net Sales 6 6 1/6 5 1/3 1/5 7 1
Step 2: The next is fuzzification; that is we turn our pair-wise comparison matrix to triangular fuzzy form. In other words, we want to replace the crisp numeric values to fuzzy numbers:
Table 5. Fuzzified pairwise comparison matrix
Table 5. Fuzzified pairwise comparison matrix
Criteria Current Ratio Acid Test Ratio Cash Ratio Leverage Ratio Asset Turnover Ratio Net Profit /Total Asset Net Profit/Capital Net Profit/Net Sales
Current Ratio (1,1,1) (1/8,1/7,1/6) (1/9,1/8,1/7) (7,8,9) (1/7,1/6,1/5) (1/8,1/7,1/6) (1/8,1/7,1/6) (1/7,1/6,1/5)
Acid Test Ratio (6,7,8) (1,1,1) (1/9,1/8,1/7) (9,9,9) (1/6,1/5,1/4) (1/8,1/7,1/6) (5,6,7) (1/7,1/6,1/5)
Cash Ratio (7,8,9) (7,8,9) (1,1,1) (9,9,9) (3,4,5) (1/9,1/9,1/9) (6,7,8) (5,6,7)
Leverage Ratio (1/9,1/8,1/7) (1/9,1/9,1/9) (1/9,1/9,1/9) (1,1,1) (1/7,1/6,1/5) (1/9,1/9,1/9) (1,2,3) (1/6,1/5,1/4)
Asset Turnover Ratio (5,6,7) (4,5,6) (1/5,1/4,1/3) (5,6,7) (1,1,1) (1/7,1/6,1/5) (4,5,6) (2,3,4)
Net Profit/Total Assets (6,7,8) (6,7,8) (9,9,9) (9,9,9) (5,6,7) (1,1,1) (7,8,9) (4,5,6)
Net Profit/Capital (6,7,8) (1/7,1/6,1/5) (1/8,1/7,1/6) (1/3,1/2,1/1) (1/6,1/5,1/4) (1/9,1/8,1/7) (1,1,1) (1/8,1/7,1/6)
Net Profit/Net Sales (5,6,7) (5,6,7) (1/7,1/6,1/5) (4,5,6) (1/4,1/3,1/2) (1/6,1/5,1/4) (6,7,8) (1,1,1)
Step 3: Now, we can calculate the fuzzy geometric mean value g i ¯ .
Table 6. Fuzzy geometric mean value g i ¯
Table 6. Fuzzy geometric mean value g i ¯
Criteria Current Ratio Acid Test Ratio Cash Ratio Leverage Ratio Asset Turnover Ratio Net Profit /Total Asset Net Profit/Capital Net Profit/Net Sales g i ¯
Current Ratio (1,1,1) (1/8,1/7,1/6) (1/9,1/8,1/7) (7,8,9) (1/7,1/6,1/5) (1/8,1/7,1/6) (1/8,1/7,1/6) (1/7,1/6,1/5) (0.273,0.308,0.353)
Acid Test Ratio (6,7,8) (1,1,1) (1/9,1/8,1/7) (9,9,9) (1/6,1/5,1/4) (1/8,1/7,1/6) (5,6,7) (1/7,1/6,1/5) (0.740,0.830,0.938)
Cash Ratio (7,8,9) (7,8,9) (1,1,1) (9,9,9) (3,4,5) (1/9,1/9,1/9) (6,7,8) (5,6,7) (2.854,3.191,3.503)
Leverage Ratio (1/9,1/8,1/7) (1/9,1/9,1/9) (1/9,1/9,1/9) (1,1,1) (1/7,1/6,1/5) (1/9,1/9,1/9) (1,2,3) (1/6,1/5,1/4) (0.209,0.241,0.271)
Asset Turnover Ratio (5,6,7) (4,5,6) (1/5,1/4,1/3) (5,6,7) (1,1,1) (1/7,1/6,1/5) (4,5,6) (2,3,4) (1.479,1.805,2.158)
Net Profit/Total Assets (6,7,8) (6,7,8) (9,9,9) (9,9,9) (5,6,7) (1,1,1) (7,8,9) (4,5,6) (5.144,5.698,6.117)
Net Profit/Capital (6,7,8) (1/7,1/6,1/5) (1/8,1/7,1/6) (1/3,1/2,1/1) (1/6,1/5,1/4) (1/9,1/8,1/7) (1,1,1) (1/8,1/7,1/6) (0.309,0.363,0.447)
Net Profit/Net Sales (5,6,7) (5,6,7) (1/7,1/6,1/5) (4,5,6) (1/4,1/3,1/2) (1/6,1/5,1/4) (6,7,8) (1,1,1) (1.173,1.391,1.664)
Step 4: Now, we calculate fuzzy weights w i ¯ .
Table 7. Fuzzy weights w i ¯
Table 7. Fuzzy weights w i ¯
Criteria Fuzzy geometric mean value g i ¯ Fuzzy weights w i ¯
Current Ratio (0.273,0.308,0.353) (0.018,0.022,0.029)
Acid Test Ratio (0.740,0.830,0.938) (0.048,0.060,0.077)
Cash Ratio (2.854,3.191,3.503) (0.186,0.230,0.287)
Leverage Ratio (0.209,0.241,0.271) (0.014,0.017,0.022)
Asset Turnover Ratio (1.479,1.805,2.158) (0.096,0.130,0.177)
Net Profit/Total Assets (5.144,5.698,6.117) (0.334,0.410,0.502)
Net Profit/Capital (0.309,0.363,0.447) (0.020,0.026,0.037)
Net Profit/Net Sales (1.173,1.391,1.664) (0.076,0.100,0.136)
SUM (12.180,13.827,15.451)
S U M 1 (0.065,0.072,0.082)
Step 5 and 6: Transforming fuzzy weights into crisp ones and normalizing the obtained results:
Table 8. Averaged weight criterion ( S i ) and Normalized weight criterion ( N i )
Table 8. Averaged weight criterion ( S i ) and Normalized weight criterion ( N i )
Criteria Fuzzy weights w i ¯ S i N i
Current Ratio (0.018,0.022,0.029) 0.023 0.023
Acid Test Ratio (0.048,0.060,0.077) 0.062 0.061
Cash Ratio (0.186,0.230,0.287) 0.234 0.230
Leverage Ratio (0.014,0.017,0.022) 0.018 0.017
Asset Turnover Ratio (0.096,0.130,0.177) 0.134 0.132
Net Profit/Total Assets (0.334,0.410,0.502) 0.415 0.408
Net Profit/Capital (0.020,0.026,0.037) 0.028 0.027
Net Profit/Net Sales (0.076,0.100,0.136) 0.104 0.102
TOTAL 1.018 1
According to the results of BCGM approach weights of ratios are given in Table 9.
Table 9. Weights of financial ratios
Table 9. Weights of financial ratios
Financial Ratios Weights Rank
Currency Ratio 0.023 7
Acid Test Ratio 0.061 5
Cash Ratio 0.230 2
Leverage Ratio 0.017 8
Asset Turnover Ratio 0.132 3
Net Profit/Total Assets 0.408 1
Net Profit/Capital 0.027 6
Net Profit/Net Sales 0.102 4
According to the importance level of financial ratios Net Profit/Total Assets ratio was found as the most important criteria having the value of 0.408 . On the other hand, leverage ratio was obtained as the least important one having the value of 0.017 .

4.2. Results

4.2.1. Ranking via TOPSIS Method

The values of each alternative and their rankings within the period of 2016-2020 were obtained via the TOPSIS method and shown in Table 10.
According to the companies’ ranking related to RC i values, DANGSUGAR, CADBURY and NASCON place top three positions for 2016 respectively. On the contrary NNFM, HONYFLOUR and PZ place the last three positions for 2016 respectively. While NASCON, DANGSUGAR and NESTLE perform as the top three consumer goods companies, and NNFM, VITAFOAM and CADBURY place the last three positions for 2017. Top three consumer goods companies in the context of financial performance are ranked as VITAFOAM, NESTLE and UNILEVER in 2018. This condition is valid for NNFM, FLOURMILL and PZ as the last three consumer goods companies for 2018. Similarly, NESTLE, DANSUGAR and VITAFOAM place the best three positions for 2019 respectively and NNFM, HONYFLOUR and PZ place the worst three positions for the same year respectively. Lastly while VITAFOAM, NESTLE and DANSUGAR perform as the top three consumer goods companies, PZ, UNILEVER and HONYFLOUR place the last three positions for 2020. Some inconsistent outputs can be seen after applying the TOPSIS method. Firstly while VITAFOAM places the 7th and 9th positions in 2016 and 2017 respectively, it places the top three positions in the range of 2018-2020. Similarly CADBURY places the worst six positions apart from the year of 2016. Other consumer goods companies that suffered from the inconsistent results can be stated as UNILEVER and NASCON respectively.

4.2.2. Ranking via VIKOR Method

By applying VIKOR method in order to obtain the values of each alternative, consensus condition is considered and thus parameter q showing maximum group benefit is used as 0.5 . The Rankings of consumer goods companies in ascending order after acquiring values within the period of 2016-2020 are shown in Table 11.
Acceptable advantage and acceptable stability conditions are satisfied for two years period (2018 and 2020). According to the acceptable advantage condition, the difference between first and second alternative having Q a values are greater than or equal to the threshold value ( DQ = 0.11 for k = 10 ). However, according to Q a values, the first alternative get the best score for values of both S a and R a , thus acceptable stability condition is satisfied. On the other hand, the two conditions failed to hold for the remaining years (2016, 2017 and 2020). Therefore, alternatives with the minimum Q a values were selected as the best alternatives for those years.
In terms of companies’ ranking related to Q a values DANGSUGAR, NASCON and CADBURY place the top three positions for 2016 respectively. On the contrary HONYFLOUR, PZ and NNFM place the last three positions for 2016 respectively. While NASCON, DANGSUGAR and NESTLE perform as the top three consumer goods companies, NNFM, VITAFOAM and CADBURY place the last three positions for 2017. Top three consumer goods companies in the context of financial performance are ranked as NESTLE, VITAFOAM and DANGSUGAR in 2018. This condition is valid for NNFM, PZ and FLOURMILL as the last three consumer goods companies for 2018. Similarly, NESTLE, DANGSUGAR and VITAFOAM were the best three consumer goods companies for 2019, while NNFM, HONYFLOUR and PZ were the worst three consumer goods companies for that same year. Lastly while VITAFOAM, NESTLE and DANGSUGAR perform as the top three consumer goods companies, HONYFLOUR, PZ and NNFM place the last three positions for 2020. Conclusively, VITAFOAM is the only company that suffered from inconsistent results in the range of 2016-2020 after applying the VIKOR method.

4.2.3. Ranking via ELECTRE Method

The C p values of each alternative and their rankings within the range of 2016-2020 obtained via ELECTRE method are shown in Table 12.
In terms of companies’ ranking related to C p values DANGSUGAR, NESTLE and CADBURY place the top three positions for 2016 respectively. On the contrary HONYFLOUR, VITAFOAM and NNFM place the last three positions for 2016 respectively. While DANGSUGAR, NASCON and NESTLE perform as the top three consumer goods companies, NNFM, VITAFOAM and CADBURY place the last three positions for 2017. Top three consumer goods companies in the context of financial performance are ranked as NESTLE, DANGSUGAR and UNILEVER in 2018. This condition is valid for NNFM, FLOURMILL and PZ as the last three consumer goods companies for 2018. Similarly, DANGSUGAR, VITAFOAM, and NESTLE were the best three consumer goods companies for 2019, while NNFM, HONYFLOUR and PZ were the worst three consumer goods companies for that same year. Lastly, while VITAFOAM, NESTLE and DANGSUGAR perform as the top three consumer goods companies, PZ, HONYFLOUR and FLOURMILL place the last three positions for 2020. It was discovered that VITAFOAM suffered from inconsistent results within the five years period time.
The D p values of each alternative and their rankings within the range of 2016-2020 are also obtained via the ELECTRE method and shown in Table 13.
In terms of companies’ ranking related to D p values NASCON, DANGSUGAR, and CADBURY place the top three positions for 2016 respectively. On the contrary HONYFLOUR, VITAFOAM and NNFM place the last three positions for 2016 respectively. While NESTLE, NASCON and DANGSUGAR perform as the top three consumer goods companies NNFM, VITAFOAM and PZ place the last three positions for 2017. Top three consumer goods companies in the context of financial performance are ranked as NESTLE, VITAFOAM and UNILEVER in 2018. This condition is valid for NNFM, FLOURMILL and HONYFLOUR as the last three consumer goods companies for 2018. Similarly, NESTLE, DANGSUGAR and VITAFOAM were the best three consumer goods companies for 2019, while NNFM, PZ and HONYFLOUR were the worst three consumer goods companies for that same year. Lastly while VITAFOAM, NESTLE and DANGSUGAR perform as the top three consumer goods companies, PZ, HONYFLOUR and FLOURMILL place the last three positions for 2020. It also turned out that VITAFOAM is the only company that suffered most from inconsistent results within the context of D p values.

4.3. Comparison of the Results Obtained by the Three Methods

In this section, we want to compare the results obtained by the three methods. In order to minimize the space, we are going to abbreviate the companies’ names into two alphabet letters. Table 14 illustrates the comparison with respect to the ranks obtained by the three methods for the five years period.
According to the results of our three chosen methods, DANGSUGAR, NASCON, NESTLE and CADBURY place the top five positions, while NNFM, HONYFLOUR, PZ and VITAFOAM perform as the last five consumer goods companies in 2016. However, DANGSUGAR has the best financial performance and places the top positions in 2016 with regard to R C i , Q a and C p values. That is true for NASCON in the context of D p values. Similarly, NASCON, DANGSUGAR, NESTLE and UNILEVER place in the top five positions for all ranking methods in 2017. While NNFM, VITAFOAM, PZ, and CADBURY placed in the last five positions for 2017. NASCON performed as the best consumer goods company for the year 2017. While NESTLE, NASCON, VITAFOAM, DANGSUGAR, and UNILEVER perform as the top five consumer goods companies in 2018, this condition is valid for NNFM, FLOURMILL, PZ, HONYFLOUR, and CADBURY placing as the last five consumer goods companies according to all ranking methods with regard to R C i , Q a , C p and D p values. In a similar pattern, NESTLE, DANGSUGAR, VITAFOAM, and CADBURY performed as the best consumer goods companies for all ranking methods in 2019, while NNFM, HONYFLOUR, and PZ rated as the worst companies for the same 2019. Lastly consumer goods companies placing in the top five positions for all ranking methods are stated as VITAFOAM, NESTLE, DANGSUGAR, and CADBURY in 2020. PZ, HONYFLOUR, and FLOURMILL are common companies placing in the last five positions with respect to R C i , Q a , C p , and D p values in 2020.
From the above comparison, it is evident that there is an incompatibility of the rankings obtained from each MCDM method, and this can be attributed to the inherent characteristics of the methods and the definite values which were assigned to the criteria used. The disparities in these methods also arise from their different attitudes [47].

5. Conclusion

In this study, we investigated the application of three MCDM methods—TOPSIS, VIKOR, and ELECTRE—to analyze and rank the performance of ten consumer goods companies listed on the Nigerian Stock Exchange (NSE) using financial ratios from the period 2016 to 2020. Weights for the financial ratios were derived using Buckley’s Column Geometric Mean approach, a fuzzy ranking method. The findings indicate that, over the long term, all three MCDM methods produced similar ranking results year by year. This consistency underscores the robustness of these methods in financial performance evaluation. The study concludes that TOPSIS, VIKOR, and ELECTRE are effective tools for ranking companies based on financial performance and can be extended to companies in other sectors.

6. Recommendation

For future research, it is recommended to:
  • Integrate different weighting and ranking approaches to enhance the performance measurement of consumer goods companies listed on the NSE.
  • Explore the application of these MCDM methods in other sectors to verify their effectiveness in different contexts.
  • Investigate the impact of varying financial environments on the performance evaluation to account for external factors influencing company rankings.
  • Using a larger data set that covers more companies and a longer period to further validate the robustness and reliability of the MCDM methods.
  • Consider incorporating qualitative criteria along with financial ratios to provide a more comprehensive evaluation of company performance.
These recommendations aim to build on the findings of this study and contribute to the ongoing development and application of MCDM methods in financial performance evaluation.

Acknowledgments

I would like to express my gratitude to my academic advisor, dr. Paweł Błaszczyk, for his invaluable guidance and support throughout this study. I also extend my appreciation to the financial experts who provided their insights and expertise. Finally, I thank the Institute of Mathematics at the University of Silesia in Katowice for the resources and support that made this research possible.

References

  1. Chen, S.J.; Hwang, C.L. Fuzzy multiple attribute decision making methods. Fuzzy multiple attribute decision making 1992, pp. 289–486.
  2. Ghaleb, A.M.; Kaid, H.; Alsamhan, A.; Mian, S.H.; Hidri, L. Assessment and comparison of various MCDM approaches in the selection of manufacturing process. Advances in Materials Science and Engineering 2020, 2020. [Google Scholar] [CrossRef]
  3. Alias, M.A.; Hashim, S.Z.M.; Samsudin, S. Multi criteria decision making and its applications: a literature review. Jurnal Teknologi Maklumat 2008, 20, 129–152. [Google Scholar]
  4. Singh, A.; Malik, S.K. Major MCDM Techniques and their application-A Review. IOSR Journal of Engineering 2014, 4, 15–25. [Google Scholar]
  5. Rao, R.V. Introduction to multiple attribute decision-making (MADM) methods. Decision Making in the Manufacturing Environment: Using Graph Theory and Fuzzy Multiple Attribute Decision Making Methods, 2007; 27–41. [Google Scholar]
  6. Aruldoss, M.; Lakshmi, T.M.; Venkatesan, V.P. A survey on multi criteria decision making methods and its applications. American Journal of Information Systems 2013, 1, 31–43. [Google Scholar]
  7. Dooley, A.; Sheath, G.W.; Smeaton, D. Multiple criteria decision making: method selection and application to three contrasting agricultural case studies. Technical report, 2005.
  8. Altman, E.I. Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. The journal of finance 1968, 23, 589–609. [Google Scholar] [CrossRef]
  9. İÇ, Y.T.; Tekin, M.; PAMUKOĞLU, F.Z.; YILDIRIM, S.E. KURUMSAL FİRMALAR İÇİN BİR FİNANSAL PERFORMANS KARŞILAŞTIRMA MODELİNİN GELİŞTİRİLMESİ. Journal of the Faculty of Engineering & Architecture of Gazi University 2015, 30. [Google Scholar]
  10. Feng, C.M.; Wang, R.T. Performance evaluation for airlines including the consideration of financial ratios. Journal of Air Transport Management 2000, 6, 133–142. [Google Scholar] [CrossRef]
  11. Yurdakul, M.; Ic, Y. An illustrative study aimed to measure and rank performance of Turkish automotive companies using TOPSIS. Journal of the Faculty of Engineering and Architecture of Gazi University 2003, 18, 1–18. [Google Scholar]
  12. Mahmoodzadeh, S.; Shahrabi, J.; Pariazar, M.; Zaeri, M. Project selection by using fuzzy AHP and TOPSIS technique. World Academy of Science, Engineering and Technology 2007, 30, 333–338. [Google Scholar]
  13. Wu, H.Y.; Tzeng, G.H.; Chen, Y.H. A fuzzy MCDM approach for evaluating banking performance based on Balanced Scorecard. Expert systems with applications 2009, 36, 10135–10147. [Google Scholar] [CrossRef]
  14. Bülbül, S.; Köse, A. EVALUATION OF THE FINANCIAL PERFORMANCE OF TURKISH FOODS COMPANIES WITH MULTI-PURPOSE DECISION MAKING METHODS. Atatürk University Journal of Economics and Administrative Sciences 2011, 25, 71–97. [Google Scholar]
  15. Guitouni, A.; Martel, J.M. Tentative guidelines to help choosing an appropriate MCDA method. European Journal of Operational Research 1998, 109, 501–521. [Google Scholar] [CrossRef]
  16. Roy, B.; Słowiński, R. Questions guiding the choice of a multicriteria decision aiding method. EURO Journal on Decision Processes 2013, 1, 69–97. [Google Scholar] [CrossRef]
  17. Triantaphyllou, E. Multi-criteria decision making methods: A comparative study; Springer, 2000.
  18. Zavadskas, E.K.; Turskis, Z. Multiple criteria decision making (MCDM) methods in economics: an overview. Technological and economic development of economy 2011, 17, 397–427. [Google Scholar] [CrossRef]
  19. Zanakis, S.H.; Solomon, A.; Wishart, N.; Dublish, S. Multi-attribute decision making: A simulation comparison of select methods. European journal of operational research 1998, 107, 507–529. [Google Scholar] [CrossRef]
  20. Mulliner, E.; Malys, N.; Maliene, V. Comparative analysis of MCDM methods for the assessment of sustainable housing affordability. Omega 2016, 59, 146–156. [Google Scholar] [CrossRef]
  21. Emovon, I.; Oghenenyerovwho, O.S. Application of MCDM method in material selection for optimal design: A review. Results in Materials 2020, 7, 100115. [Google Scholar] [CrossRef]
  22. Saaty, T. The Analytical Hierarchy Process McGraw-Hill. New York 1980. [Google Scholar]
  23. Saaty, T.L. Fundamentals of decision making and priority theory with the analytic hierarchy process; Vol. 6, RWS publications, 2000.
  24. Saaty, T.L. Decision making with the analytic hierarchy process. International journal of services sciences 2008, 1, 83–98. [Google Scholar] [CrossRef]
  25. Zadeh, L.A. Information and control. Fuzzy sets 1965, 8, 338–353. [Google Scholar]
  26. Jie, L.H.; Meng, M.C.; Cheong, C.W. Web based fuzzy multicriteria decision making tool. International Journal of the computer, the Internet and management 2006, 14, 1–14. [Google Scholar]
  27. Huang, H.C.; Ho, C. Applying the Fuzzy Analytic Hierarchy Process to Consumer Decision-Making Regarding Home Stays. International Journal of Advancements in Computing Technology 2013, 5, 981–990. [Google Scholar] [CrossRef]
  28. İrfan Ertuğrul. ; Karakaşoğlu, N. Performance evaluation of Turkish cement firms with fuzzy analytic hierarchy process and TOPSIS methods. Expert Systems with Applications 2009, 36, 702–715. [Google Scholar] [CrossRef]
  29. Dzitac, I. The fuzzification of classical structures: A general view. International Journal of Computers Communications & Control 2015, 10, 12–28. [Google Scholar]
  30. Deng, H. Multicriteria analysis with fuzzy pairwise comparison. International journal of approximate reasoning 1999, 21, 215–231. [Google Scholar] [CrossRef]
  31. Korol, T. Fuzzy logic in financial management 2012.
  32. Kuchta, D. Multicriteria Fuzzy Evaluation of Project Success in R&D Projects. Multiple Criteria Decision Making 2019, 14, 44–59. [Google Scholar]
  33. Calabrese, A.; Costa, R.; Levialdi, N.; Menichini, T. A fuzzy analytic hierarchy process method to support materiality assessment in sustainability reporting. Journal of Cleaner Production 2016, 121, 248–264. [Google Scholar] [CrossRef]
  34. Kahraman, C.; Cebeci, U.; Ulukan, Z. Multi-criteria supplier selection using fuzzy AHP. Logistics information management 2003. [Google Scholar] [CrossRef]
  35. Van Laarhoven, P.J.; Pedrycz, W. A fuzzy extension of Saaty’s priority theory. Fuzzy sets and Systems 1983, 11, 229–241. [Google Scholar] [CrossRef]
  36. Kannan, D.; Khodaverdi, R.; Olfat, L.; Jafarian, A.; Diabat, A. Integrated fuzzy multi criteria decision making method and multi-objective programming approach for supplier selection and order allocation in a green supply chain. Journal of Cleaner production 2013, 47, 355–367. [Google Scholar] [CrossRef]
  37. Buckley, J.J. Fuzzy hierarchical analysis. Fuzzy sets and systems 1985, 17, 233–247. [Google Scholar] [CrossRef]
  38. Hwang, C.L.; Yoon, K. Methods for multiple attribute decision making. In Multiple attribute decision making; Springer, 1981; pp. 58–191.
  39. Sabaei, D.; Erkoyuncu, J.; Roy, R. A review of multi-criteria decision making methods for enhanced maintenance delivery. Procedia CIRP 2015, 37, 30–35. [Google Scholar] [CrossRef]
  40. Chen, C.T. Extensions of the TOPSIS for group decision-making under fuzzy environment. Fuzzy sets and systems 2000, 114, 1–9. [Google Scholar] [CrossRef]
  41. Junior, F.R.L.; Osiro, L.; Carpinetti, L.C.R. A comparison between Fuzzy AHP and Fuzzy TOPSIS methods to supplier selection. Applied soft computing 2014, 21, 194–209. [Google Scholar] [CrossRef]
  42. Opricovic, S. Multicriteria optimization of civil engineering systems. Faculty of Civil Engineering, Belgrade 1998, 2, 5–21. [Google Scholar]
  43. Chen, L.Y.; Wang, T.C. Optimizing partners’ choice in IS/IT outsourcing projects: The strategic decision of fuzzy VIKOR. International Journal of Production Economics 2009, 120, 233–242. [Google Scholar] [CrossRef]
  44. Opricovic, S.; Tzeng, G.H. Compromise solution by MCDM methods: A comparative analysis of VIKOR and TOPSIS. European journal of operational research 2004, 156, 445–455. [Google Scholar] [CrossRef]
  45. Ishizaka, A.; Nemery, P. Multi-criteria decision analysis: methods and software; John Wiley & Sons, 2013.
  46. Yoon, K.P.; Hwang, C.L. Multiple attribute decision making: an introduction; Sage publications, 1995.
  47. Vakilipour, S.; Sadeghi-Niaraki, A.; Ghodousi, M.; Choi, S.M. Comparison between Multi-Criteria Decision-Making Methods and Evaluating the Quality of Life at Different Spatial Levels. Sustainability 2021, 13, 4067. [Google Scholar] [CrossRef]
Figure 2. Examples of fuzzy concepts [32]
Figure 2. Examples of fuzzy concepts [32]
Preprints 115698 g002
Table 2. Saaty’s 9 point scale [24].
Table 2. Saaty’s 9 point scale [24].
Intensity of
Importance
Definition Explanation
1 Equal Importance Two activities contribute equally to the objective
2 Weak or slight
3 Moderate importance Experience and judgement slightly favour
one activity over another
4 Moderate plus
5 Strong importance Experience and judgement strongly favour
one activity over another
6 Strong plus
7 Very strong or
demonstrated importance
An activity is favoured very strongly over
another; its dominance demonstrated in practice
8 Very, very strong
9 Extreme importance The evidence favouring one activity over another
is of the highest possible order of affirmation
Table 3. Linguistic Variables for Pairwise Comparison of Each Criterion [36]
Table 3. Linguistic Variables for Pairwise Comparison of Each Criterion [36]
Definitions Triangular Fuzzy Scale Triangular Fuzzy Reciprocal Scale
Equally strong ( 1 , 1 , 1 ) ( 1 , 1 , 1 )
Moderately strong ( 2 , 3 , 4 ) ( 1 / 4 , 1 / 3 , 1 / 2 )
Strong ( 4 , 5 , 6 ) ( 1 / 6 , 1 / 5 , 1 / 4 )
Very strong ( 6 , 7 , 8 ) ( 1 / 8 , 1 / 7 , 1 / 6 )
Extremely strong ( 9 , 9 , 9 ) ( 1 / 9 , 1 / 9 , 1 / 9 )
( 1 , 2 , 3 ) ( 1 / 3 , 1 / 2 , 1 )
Intermediate values ( 3 , 4 , 5 ) ( 1 / 5 , 1 / 4 , 1 / 3 )
( 5 , 6 , 7 ) ( 1 / 7 , 1 / 6 , 1 / 5 )
( 7 , 8 , 9 ) ( 1 / 9 , 1 / 8 , 1 / 7 )
Table 10. R C i values and consumer goods companies’ rankings in descending order
Table 10. R C i values and consumer goods companies’ rankings in descending order
Companies 2016 2017 2018 2019 2020
R C i Rank R C i Rank R C i Rank R C i Rank R C i Rank
HONYFLOUR 0.3485 9 0.1929 5 0.1501 7 0.2405 9 0.3557 8
FLOURMILL 0.6257 6 0.1539 6 0.1248 9 0.3882 4 0.4488 6
NNFM 0.3026 10 0.0453 10 0.0221 10 0.2097 10 0.3956 7
PZ 0.5143 8 0.1385 7 0.1367 8 0.2489 8 0.1283 10
VITAFOAM 0.5587 7 0.1095 9 0.5950 1 0.5539 3 0.8470 1
NASCON 0.7268 3 0.6330 1 0.3861 5 0.3685 6 0.5128 4
NESTLE 0.6772 4 0.6154 3 0.5799 2 0.6953 1 0.7422 2
DANGSUGAR 0.7465 1 0.6280 2 0.4220 4 0.5788 2 0.6849 3
CADBURY 0.7289 2 0.1192 8 0.1875 6 0.3766 5 0.4813 5
UNILEVER 0.6270 5 0.5155 4 0.4947 3 0.3058 7 0.3545 9
Table 11. Q a values and consumer goods companies’ rankings according to ascending order
Table 11. Q a values and consumer goods companies’ rankings according to ascending order
Companies 2016 2017 2018 2019 2020
Q a Rank Q a Rank Q a Rank Q a Rank Q a Rank
HONYFLOUR 0.997 10 0.774 5 0.831 7 0.985 9 0.983 10
FLOURMILL 0.391 5 0.787 6 0.844 8 0.589 4 0.819 6
NNFM 0.808 8 1 10 1 10 1 10 0.876 8
PZ 0.813 9 0.808 7 0.872 9 0.937 8 0.954 9
VITAFOAM 0.61 7 0.881 9 0.112 2 0.198 3 0 1
NASCON 0.081 2 0.038 1 0.241 4 0.669 6 0.651 4
NESTLE 0.296 4 0.123 3 0 1 0.078 1 0.26 2
DANGSUGAR 0.023 1 0.061 2 0.163 3 0.14 2 0.341 3
CADBURY 0.125 3 0.879 8 0.773 6 0.627 5 0.727 5
UNILEVER 0.406 6 0.416 4 0.357 5 0.78 7 0.837 7
Table 12. C p values and consumer goods companies’ rankings according to descending order
Table 12. C p values and consumer goods companies’ rankings according to descending order
Companies 2016 2017 2018 2019 2020
C p Rank C p Rank C p Rank C p Rank C p Rank
HONYFLOUR -4.4280 10 -0.3340 5 -1.6610 7 -3.8680 9 -5.3280 9
FLOURMILL -0.4310 5 -1.6540 6 -4.4150 9 -0.0080 6 -2.4120 8
NNFM -2.9880 8 -7.5740 10 -7.7350 10 -5.6620 10 -0.6700 5
PZ -0.9260 7 -2.6000 7 -2.1670 8 -3.4960 8 -5.5760 10
VITAFOAM -4.3320 9 -4.5200 9 2.0280 4 4.4760 2 8.0940 1
NASCON 2.1690 4 4.9820 2 1.5700 5 0.9680 5 -0.7640 6
NESTLE 3.8840 2 4.9660 3 4.1040 1 3.3660 3 3.4300 2
DANGSUGAR 4.6560 1 5.6940 1 4.0940 2 5.0460 1 3.3320 3
CADBURY 2.8900 3 -3.1540 8 1.0680 6 2.1240 4 1.3240 4
UNILEVER -0.4940 6 4.1940 4 3.1140 3 -2.9460 7 -1.4300 7
Table 13. D p values and consumer goods companies’ rankings according to ascending order
Table 13. D p values and consumer goods companies’ rankings according to ascending order
Companies 2016 2017 2018 2019 2020
D p Rank D p Rank D p Rank D p Rank D p Rank
HONYFLOUR 6.6563 10 2.1798 6 4.2812 8 4.2864 8 6.1526 9
FLOURMILL 0.5103 5 0.5743 5 5.2179 9 1.2365 7 3.4296 8
NNFM 4.4049 8 8.1738 10 8.7429 10 7.1757 10 2.4937 7
PZ 4.3952 7 3.8235 8 3.9864 7 6.5715 9 6.9334 10
VITAFOAM 4.5995 9 5.4976 9 -6.0722 2 -4.7149 3 -8.3944 1
NASCON -6.4540 1 -5.9109 2 -0.8906 5 0.6217 6 0.6837 6
NESTLE -3.3173 4 -6.3025 1 -6.2414 1 -7.4826 1 -5.9494 2
DANGSUGAR -6.1098 2 -5.7515 3 -3.4241 4 -5.5867 2 -4.0689 3
CADBURY -5.3538 3 3.1362 7 0.3797 6 -2.4647 4 -1.7233 4
UNILEVER 0.6688 6 -5.4203 4 -5.9800 3 0.3571 5 0.4430 5
Table 14. Comparison of the results obtained by the three methods
Table 14. Comparison of the results obtained by the three methods
Ranking/Method 2016 2017 2018 2019 2020
TOPSIS VIKOR ELECTRE TOPSIS VIKOR ELECTRE TOPSIS VIKOR ELECTRE TOPSIS VIKOR ELECTRE TOPSIS VIKOR ELECTRE
R C i Q a C p D p R C i Q a C p D p R C i Q a C p D p R C i Q a C p D p R C i Q a C p D p
1 DA DA DA NA NA NA DA NE VI NE NE NE NE NE DA NE VI VI VI VI
2 CA NA NE DA DA DA NA NA NE VI DA VI DA DA VI DA NE NE NE NE
3 NA CA CA CA NE NE NE DA UN DA UN UN VI VI NE VI DA DA DA DA
4 NE NE NA NE UN UN UN UN DA NA VI DA FL FL CA CA NA NA CA CA
5 UN FL FL FL HO HO HO FL NA UN NA NA CA CA NA UN CA CA NN UN
6 FL UN UN UN FL FL FL HO CA CA CA CA NA NA FL NA FL FL NA NA
7 VI VI PZ PZ PZ PZ PZ CA HO HO HO PZ UN UN UN FL NN UN UN NN
8 PZ NN NN NN CA CA CA PZ PZ FL PZ HO PZ PZ PZ HO HO NN FL FL
9 HO PZ VI VI VI VI VI VI FL PZ FL FL HO HO HO PZ UN PZ HO HO
10 NN HO HO HO NN NN NN NN NN NN NN NN NN NN NN NN PZ HO PZ PZ
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated