EVALUATION OF A COMPLEX QUALITY INDEX USING NUMERICAL AND VERBAL ORDINAL SCALE

N . Y a r e m c h u k Candidate of Science, Head of Department* Е-mail: YaremchukNA@i.ua O . R e d y o g a Assistant* E-mail: R_olia@i.ua *Department of Information and Measuring Equipment National Technical University of Ukraine “Kyiv Polytechnic Institute” pr. Peremogy, 37, Kyiv, 03056, Ukraine В роботі проведено порівняння результатів об’єднання одиничних показників якості, представлених в вербально-числових шкалах, з метою отримання шкали комплексного показника якості. Визначено умови, за яких результати об’єднання будуть збігатися у певних межах, введено додаткові правила для визначення шкали комплексного показника якості. Як характеристику вірогідності встановлення шкали комплексного показника якості обрано матрицю відповідності між класами еквівалентності вербальної і числової шкали Ключові слова: оцінювання комплексного показника якості, числові і вербальні шкали


Introduction
Single and complex quality indexes are used for complex object quality evaluation.Software tools (ST), according to which the ST quality index is a quantitative measure, used for estimating the degree, to which the ST possesses this quality, can be given as the example of the complex object.Most of the quality indexes are ordinal values and are evaluated on ordinal scalesnumerical or verbal, continuous or discrete [1].
Quality indexes can be single and complex.Single quality index is the ST quality index, characterizing one of its features, while complex quality index is an integral quality index, characterizing several features and is estimated by combining (aggregating) single quality indexes.
When evaluating the ST quality in the form of the profile, there is a set of ST features (single indexes), whish need to be characterized qualitatively or quantitatively, and to be compared with the original ST profile, which is taken as a standard.
Complex indexes are determined by a hierarchic structure as shown in the Fig. 1, where the hierarchic structure of the complex ST quality index of ''Universality'', consisting of criteria (flexibility, mobility and modification ability), metrics and single quality indexes is given.
Estimating the metric M i starts with aggregating several single indexes (evaluating elements) according to the formula: where G is the quantity of evaluating elements m gi in the metric M i ( g G = 1.. ); Fig. 1.The hierarchic structure of the complex ST quality index of ''Universality'' g is the evaluating element number, m gi is the value of the evaluating element of the metric M i by the numerical scale.
Estimating the complex index (criterion) P j is carried out by aggregating metrics values, taking into account the corresponding weight coefficients according to the formula: where M ij is the value of the і-th metric, which is the part of the j-th complex quality index, criterion ( i K j = 1 2 , ... ), ρ i is weight coefficients of metrics.The final value of the complex index of "Universality" is found as follows: where ρ j is weight coefficients of criteria.Therefore, the problem of calculating the complex index includes determining weight coefficients and independent single indexes.

Literature data analysis
Many works were devoted to the issue of finding weight coefficients.Thus, [2] deals with determining weight coefficients by an adequate statistics median, since the scale for determining weight coefficients is the ordinal scale.The disadvantage of the median as the distribution centre estimate is its high uncertainty as compared to the arithmetic mean, especially with a small number of experts, involved in the evaluation.In the work [2], using the Walsh median for reducing the uncertainty as the expert evaluation result accuracy estimate is proposed.
When using weight coefficients for estimating the complex quality index by the formulae (3), all quality indexes are numerical.And since the part of single quality indexes is expressed on verbal ordinal scales, arithmetization of the verbal ordinal indexes scale is required [3].However, the experts, performing ordinal measurements, do not always agree with the arithmetization of verbal and character scales, as they believe that arithmetization needs introducing the concept of distance between the expressions of ordinal values, which is absent in verbal and character scales [4].Indeed, arithmetization is correct for fully numerical ordinal scales (associative ordinal scales) and when it is introduced into the single indexes estimation procedure, many conditions are to be met (scaling for particular single indexes, normalizing, etc.).Therefore, along with using the arithmetization, the methods for using operators, which allow carrying out procedures with a set of verbal (character) quality indexes or samples without arithmetization, are developed.Particularly, it concerns the union (aggregating) operators [5], namely the median, OWA, T-norm and T-conorm.Evaluating the uncertainty in determining the character ordinal scale is considered in [6], and the dispersion measure is determined by Blair and Lacy [7].
Therefore, if aggregation is carried out by the formula (1) or (2) with weight coefficients equal to one, it can be applied both for numerical and verbal indexes, using special operators.Evaluating metrics can be carried out based on the set of single numerical indexes or using special operators for the same indexes, presented in verbal quality scales.
The result of verbal scale arithmetization is presented as the combined (hybrid) scale, including two scales -verbal and numerical (Table 1) [8].

Table 1
Verbal-numerical scale for three equivalence classe Verbal scale (quality level) If this scale is used for single indexes (evaluation elements), under certain conditions two methods for aggregating single indexes in both scales can be proposed.Hence, it is necessary to determine the conditions, under which the results of aggregation, conducted with data in different scales, will coincide (or not) within certain boundaries.
Problem statement.
When evaluating the objects quality, verbal-numerical scales are used, where along with quantitative estimates of scale points or equivalence classes boundaries, their verbal interpretation is presented.The objective of the paper lies in comparing the results of aggregating single quality indexes, given in verbal and numerical ordinal scales, obtained due to the arithmetization of these verbal scales.The aggregation result is the complex quality index, obtained based on the set of single indexes (provided that weight coefficients are equal to one).
The research task lies in creating a characteristic for the complex quality index scale, i.e. the compliance matrix between the equivalence classes of numerical and verbal scales, and finding an additional characteristic, which would allow estimating the compliance degree between the data aggregation results on numerical and verbal scales.

Aggregation operators for verbal scales
Aggregation operators for verbal scales can be used only for metrics and complex indexes, higher in the hierarchy, which weight coefficients are equal to one.
One of aggregation operators in verbal scales is the median.However, the median can be used only with odd number of single quality indexes in the quality profile.With an even number, additional conditions must be introduced.
Aggregation operators, which in turn are based on the T-norm and T-conorm operators can be used for verbal scales.Aggregation operators include the weight function, which is determined by the verbal scale and ordinal numbers of ranked quality profile and the collection function (using the T-norm and T-conorm operators).
The emulator of an arithmetic mean for verbal scales is the OWA operator [9].The operator is defined as: where Q k ( ) is the weight function of the OWA operator, Int{ } ⋅ is the nearest integer function { } ⋅ , t is the number of points (if the scale is discrete) or equivalence classes of the scale (quasiorder scale), n is the number of single indexes in the quality profile, q k is the single index of the profile, ranked in descending order of quality level.
However, when using the formulas in this way, there may be situations when the result of determining the weight function elements is ambiguous, for example, S f E k ( ) { , } = 1 5 .In this case, the rounding-off rule must be set as well.Therefore, the authors proposed to use the OWA operator, the weight function is defined as S f 1 , where E{ } ⋅ is the integer.

Comparison of aggregation results for verbal-numerical scales
To obtain the aggregation result, the quality profile is represented with the evaluation results of single indexes values by the numerical scale and with appropriate quality levels -by the verbal.
The profile components are ranked in descending order of quality level, and q k is a single index number in the ranked (in descending order of quality) profile.The example of the initial data representation using the scale in the Table 1 is given in the Table 2. Aggregation operator for the full numerical scale is the arithmetic mean that on the example of the Table 2 is equal to 0.52.By the verbal scale of the complex index of the Table 1, it corresponds to the "Medium" quality level.
When using the OWA operator, we obtain: Thus, the aggregation result of the verbal quality indexes corresponds to the aggregation result of single indexes values on the arithmetic mean.
However, such compliance does not always occur.It is possible to give an example of the profile, consisting of three single quality indexes of the Table 2 (b), and their quality level is also determined by the scale of the Table 1.Then, the complex index value on the arithmetic mean is 0.8, which corresponds to the "High" level.However, when determining the quality level by verbal ensemble indexes and the OWA operator, we obtain: that is the quality level by verbal indexes s ''Middle".Noncompliance in results when determining the quality level on the verbal and numerical scales is explained by the stair-stepping effect of the weight function.This especially concerns the values, which are close to the boundaries between the equivalence classes that occurred in the above example.
Having analyzed the weight function of the OWA operator, it can be concluded that the middle equivalence class covers most of the possible combinations of quality levels of separate indexes.Having used the interval analysis rules, possible boundaries (high and low) of adjacent crossing equivalence classes can be determined.In these areas, the conclusions on referring quality levels to different equivalence classes on the numerical and verbal scale of the Table 3 are possible.
Table 3 The boundaries of the complex index on the quality profile with three single indexes, the quality level which is defined by the scale of the   As is shown in the Table 3, the full compliance of solutions on referring to a certain equivalence class on the verbal and numerical scale will be only for the HHH, MMM, LLL profiles, that is "High", "Middle" and "Low" both on the verbal and numerical scales.For other profiles, there is a certain probability of solutions convergence on the verbal and numerical scale for the complex indexes, numerical values of which are located at the intersection of the corresponding equivalence classes of the Table 3.To find the probabilities, complex indexes boundaries, corresponding to verbal profiles on the numerical scale were used.
When constructing fuzzy numbers in these boundaries in the form of Gaussians with a carrier, corresponding to the boundaries, the center and dispersion (standard de- ) placement parameter, which is then used to calculate the probability of finding the result in a particular equivalence class, was found.For example, for the HHM profile, located within 0,57-0,9 on the numerical scale, the fuzzy number carrier is from 0.7 to 0.9 in the "High" equivalence class and from 0.57 to 0.7 in the "Middle" equivalence class that corresponds to the "High" solution with the probability 0.52 and "Middle" with the probability 0.48.
The quality characteristic of setting the verbal-numerical scale of the complex quality index is the compliance matrix of verbal and numerical equivalence classes [10].In the Table 4, the scale compliance matrix of complex quality index, which consists of three single indexes (on the verbal-numerical scale of the Table 1) is given.The perfect compliance matrix consists of ones on the diagonal and zeros in the other cells.Real matrix has the incompliance region of equivalence classes, defined on the verbal and numerical scale, which is close to the value, dividing the equivalence classes.To reflect the uncertainty degree in defining the equivalence class, the probability for finding the complex quality index in a certain equivalence class, the dependence of which on the complex index numerical value is shown in the Fig. 2, can be used.

Table 4
The compliance matrix of the scale of complex quality index, consisting of three single indexes

Verbal scale
Numerical scale H M L HHH HHM HMM MMM MML MLL LLL 0,7-1 1 0,74 0,08 0,3-0,7 0,26 0,92 1 0,92 0,26 0-0,3 0,08 0,74 1 The probability of finding the complex quality index in the corresponding equivalence classes is determined by the curves crossing (Fig. 2) with a vertical line, drawn from the point, corresponding to the complex quality index value on the number axis.For example, for the HMMML profiles of the Table 2, the complex index value on the verbal scale is "Medium", by the numerical scale 0.52, which also corresponds to "Medium".The results coincide that can be demonstrated using the intersection of the vertical line with the probability curves, as a result, the complex quality index with the probability 1 refers to the "Medium" equivalence class.For the HHM profile, the complex quality index value by the verbal scale corresponds to the "Medium" equivalence class, but on the numerical scale it is equal to 0.8, which corresponds to the "High" equivalence class with the probability 0.8 (Fig. 2).Hence, there is the probability, which equals to 0.2 on referring the quality index to the "Medium" equivalence class.When using the profile with a larger number of single indexes, the number of profiles, which can be referred to the extreme equivalence classes increases, as a result, there is a smooth change in the probability of referring the adjacent equivalence classes of the Table 5.
That is, if the decision is taken by the maximum probability, there will be a complete correspondence between referring the numerical and verbal scales to a certain equivalence class.

Conclusions
When determining the quality of complex objects, particularly software tools, the problem of evaluating the complex quality index is often solved using single quality indexes.For full numerical scales, this problem is provided by the established procedures.
But there is a set of single quality indexes, for which verbal or verbal-numerical scales are determined.Then, the complex quality index value can be calculated on the numerical scale, and the equivalence class or quality level of the complex index on the verbal scale can be determined by the aggregating (combining) operator of verbal or character data.
The ranked quality profile, i.e. a set of single quality indexes, ordered by the quality level was used as the output characteristic for estimating the equivalence class on the verbal scale.
Восточно-Европейский журнал передовых технологий 1/2 ( 67 ) 2014 The analysis, aimed at studying the results compliance of collecting and aggregating single quality indexes into the complex quality index on the numerical and verbal scales was carried out in the paper.As a result, the compliance matrix for the verbal-numerical scale of the complex quality index was obtained.It was found that unambiguous compliance in aggregation results occurs only for extreme and central from the ranked quality profiles.For other profiles, the compliance probability (or incompliance) of the data, obtained on the numerical and verbal scales, used in referring the quality level to a certain equivalence class on the complex index, was determined.

Table 2 Initial
data for aggregation: a is the five-component profile, b is the three-component profile

Fig. 2 .
Fig.2.The probability on referring the quality index P to the corresponding verbal scale equivalence class "Low", "Medium", "High"