DEVELOPMENT OF THE ALGORITHM OF VIDEO IMAGE ADAPTATION TO SPECTRAL POWER DISTRIBUTION OF ILLUMINANTS

The progress of video technologies is observed in the development of standardization, with the advent of digital television systems – standard and high definition television systems [1], and then the recommendation appears [2]. This indicates the insufficiency and the need for improving the existing and creating fundamentally new solutions in video transmission systems. Further progress of video technologies can be seen in the introduction of the high dynamic range (HDR) system [3]. Partial introduction of adaptive color rendering systems [4] into digital television systems is a step forward. Modern video transmission systems are based on a conventional through path system. If we accept that the conventional system does not introduce distortions or they are identified as “acceptable”, then there are areas of the path where the correctness of color rendering and viewer’s color perception are violated. On the transmitting side, it is the interval from an object to the transmitter lens, and on the receiving side – from a reproducer to the observer. At the present time, quite little is said about the distortions taking place in the indicated sections of the path and the ways of elimination. First of all, studies are separated because they are conducted individually for illuminants and how people accept it. But no research is conducted on this complex effect during the transmission by video channels. These studies are relevant primarily in places where color rendering by video channels is extremely important, namely, in special and military applications, in e-medicine, textile industry, etc.


Introduction
The progress of video technologies is observed in the development of standardization, with the advent of digital television systems -standard and high definition television systems [1], and then the recommendation appears [2].This indicates the insufficiency and the need for improving the existing and creating fundamentally new solutions in video transmission systems.
Further progress of video technologies can be seen in the introduction of the high dynamic range (HDR) system [3].Partial introduction of adaptive color rendering systems [4] into digital television systems is a step forward.
Modern video transmission systems are based on a conventional through path system.If we accept that the conventional system does not introduce distortions or they are identified as "acceptable", then there are areas of the path where the correctness of color rendering and viewer's color perception are violated.On the transmitting side, it is the interval from an object to the transmitter lens, and on the receiving side -from a reproducer to the observer.At the present time, quite little is said about the distortions taking place in the indicated sections of the path and the ways of elimination.First of all, studies are separated because they are conducted individually for illuminants and how people accept it.But no research is conducted on this complex effect during the transmission by video channels.These studies are relevant primarily in places where color rendering by video channels is extremely important, namely, in special and military applications, in e-medicine, textile industry, etc.

Literature review and problem statement
Illuminants are divided into natural and artificial.The latter should be divided into professional, semi-professional and household [5].Listed illuminants, as noted earlier, require consideration or adaptation when developing new video communication systems or improving existing ones.This is due to the fact that they can affect the transmitting and receiving sides of the video path.
In [6], the studies considering the properties of color perception under different illumination in video transmission paths and the effect of the illuminant on the color quality are presented.So the issue should be considered from two per- It is shown that illuminants lead to inappropriate color changes, and so, these changes will be even greater when taking into account the properties of color vision.The issue of reducing color changes was not resolved for video channels.Further study is shown in [7], where the author proposes an algorithm of chromatic adaptation that can be based on color coordinates.The studies [8] reflect the curve of the color temperature of illuminants in terms of color perception.In [7,8], the analysis on the basis of color coordinates is performed, but it is not considered how exactly the illuminant, its spectral power distribution affect other colors.
One of the possible solutions to these issues is to use a color perception model when improving existing and designing new video communication systems.Thus, an attempt to introduce a color perception prediction system was made in [9].Along with that, the work should be supplemented by the use of an algorithm that reduces the illuminant effect on the color quality.The authors point out unresolved issues regarding the incorrectness of the color perception model.
A possible solution was presented in [10], where the authors propose an analytical solution to the problem of the negative reaction of the color perception model.In [11], the reliability of the data presented in [10] is proved, but the issues of the uniformity of color filling in the color chart remain unresolved.
This problem is solved in [12,13], where the uneven color distribution into blue, yellow-blue and red is eliminated by applying a new analytical description of previous experiments.The improved model can be used for further research taking into account the specified practices [14], which are necessary for consideration of the universal exchange of color information among various systems.This is quite common in the exchange of graphical and video information via communication channels.
The analysis [5][6][7][8][9][10][11][12][13][14] shows that although color perception models are developed not for video communication systems, the issue of the influence of spectral power distribution of illuminants on the color quality is not sufficiently investigated.This suggests the expediency of studies related to the introduction of an algorithm taking into account the properties of color perception of video information and spectral power distribution of the illuminant in video communication systems.

The aim and objectives of the study
The aim of the work is to develop an algorithm of adaptation of the video system to the spectral power distribution of the illuminant, taking into account the properties of the observer's color perception.This will reduce the influence of illuminants on the quality of color rendering, which is extremely important, for example, in medicine, military technology, etc.
To achieve the aim, it is necessary to accomplish the following objectives: -to determine the spectral power distribution of the color of the transmitted image according to the known color coordinates; -to consider the properties of color perception when developing the algorithm; -to investigate the possibility of using the proposed algorithm under technical limitations of the video system; -to provide the system with a metrological instrument for assessing the performance of the algorithm.

Materials and methods of the study on the implementation of the adaptive video communication system
The objectives can be accomplished using methods and means of research that will allow taking into account the complements to the path of the video communication system shown in Fig. 1.Fig. 1 shows the simplified block diagram of a through video path with points that may affect the quality of video transmission.The scene object will be described from the observer's or operator's viewpoint by the color perception model, actually as well as the observer on the receiving side of the path.The illuminant is a factor affecting the color rendering model, both on the transmitting side and on the receiving side.It is assumed that the conventional model does not introduce distortions or they are unnoticeable.Therefore, further research will address the methods and means of reducing the impact on the quality of color rendering.

1. The Algorithm of adaptation to the spectral power distribution of the illuminant
The main purpose of adaptation is a continuous or periodic determination of the influence of the illuminant on the quality of color rendering and correction of the video signal depending on changes of color coordinates.
Adaptation to the spectral power distribution of the illuminant consists of two subtasks, namely, obtaining the value of spectral power distribution of the scene object and spectral power distribution of the illuminant.
The data of the spectral power distribution of the illuminant can be measured by an additional device or stored as an array of numerical values in the device that is part of the adaptation system.
Obtaining data on the spectral power distribution of the scene object is a difficult task.This is due to the fact that complex objects of the scene with a high level of detail con-

Conventional video communication model
Necessary complement to the video path model tain information about a set of colors and the corresponding set of spectral power distributions.For this reason, it is impossible to determine each of them, because of the complexity of hardware implementation, and, accordingly, the process of determining the influence of the illuminant separately for each color.Therefore, in order to determine the influence of the illuminant on the set of colors, the algorithm presented in the block diagram is proposed (Fig. 2).
The block diagram consists of two parts: the first one is a standard part of the video path, the second one is the algorithm for determining adapting coefficients and accounting of the conventional model in video signals.( ) ( ) ( ) where S(l) is the spectral power distribution of color obtained by simulation, ( ), z l is the spectral sensitivity of color channels of the "light-signal" converter, P(l) is the spectral power distribution of the illuminant.
In parallel to determining the color parameters of the scene object, the spectral power distribution of the illumi-nant is determined.To exclude the parameters of the influence of the illuminant on each of possible colors of the scene object, it is necessary to know the nature and magnitude of the illuminant influence.Therefore, for the determination of these values, it is necessary to conduct an assessment on a certain set of colors that are uniformly placed in the region of the rendered colors of the system.For this implementation, it is necessary to use a set of different colors.The set of test colors may have different sample sizes.For example, the authors [15] considered the sample of 4, 9, 14, 15, 17, 99 colors (chapter 2).The specified color sets do not fill evenly and completely the region of the rendered colors, therefore the expediency of use is ambiguous.
In video communication systems, the region of the rendered colors is limited by certain conditions, equipment characteristics, etc. [16].But if we assume that everything conforms to the standards, the area of rendered and perceived colors is limited to a color triangle.To assess the quality of color rendering of the area limited by the color triangle, measuring colors in the triangle [17] with a relative saturation of 0.9, respectively, red R 0.9 , green G 0.9 and blue B 0.9 and 0.5 should be added to the set.To increase accuracy, the primary color set is supplemented by complementary colors: yellow Ye, blue C and purple M of the above saturation and to assess saturated colors -with a saturation of 1.Each color is characterized by the coordinates of the xyz color space shown in Table 1.The coordinates are based on the color triangle of high-definition television.
To form the array of correction values of the color coordinates, it is necessary to simulate t h e algorithm of determining the spectral color distribution by the given color coordinates.

2. Determination of test color parameters
It is proposed to find the spectral power distribution of color according to the given color coordinates by the algorithm presented in Fig. 3.
In Fig. 3, , x , y z are the camera sensitivity characteristics, P(l) is the spectral power distribution of the illuminant.The presented scheme of the algorithm allows determining the influence of the illuminant on color rendering.Using (1), we obtain XYZ e , which are color coordinates, reflecting the influence of the illuminant.DE is the vector in the coordinate system of the CAM16-UCS uniform chromaticity scale system [10,18], which takes into account the characteristics of color perception of the human visual apparatus and is calculated by the formula (3),
As for the characteristic of the spectral power distribution S f (l) (4), it is obtained by searching for the lower extremum of functional dependency.In the dependency (4), the variables are the center of spectral power distribution z, the width of filter response q, where 1 , i i + q = l − l and ( ) 400,700 i ∈ and the level l, which can be represented by the formula (4).

Fig. 3. The Algorithm of determining the spectral power distribution by the given color coordinates
Based on the analysis of previous works [5,8,19], the dependency described by the functional dependency L(l) (5) interpreted in the values used in this paper was chosen as a basis.

Fig. 4. ΔE error value for the chosen test color
From Fig. 4 it can be seen that the color rendering error DE (3) in the simulation using the expression (5) according to the color rendering criteria presented in [16,17] can be classified as "unnoticeable".The results prove the possibility of using this functional dependency to simulate the spectral color distribution of the scene object.
Thus, for the chosen spectral power distribution on the basis of the Gaussian function, the data of Table 2 are obtained, where the center of spectral power distribution z of the function with the estimated wavelength and the width of spectral power distribution q in the samples Dl are presented.Table 2 shows the DE value, indicating minimum deviation from the input coordinates.For purple color M there are no data, since this color is formed by an additive method involving red R and blue B colors in the corresponding ratio.Therefore, this color does not have a certain wavelength, and accordingly the width of the response.
For example, simulation was carried out using spectral power distributions of standardized spectral color distributions according to the CIE standard [6].For simulation, illuminants D65, A (referred to as incandescent lamps), F1 and F315 (examples of fluorescent illuminants) were chosen.The simulation data are presented in Table 3-5.
In Table 3-5 there is a marking where green color corresponds to "unnoticeable", yellow -"noticeable but acceptable" and red -"unacceptable" on the color variation scale.
There are the following values: Data obtained using the filter described by the Gaussian distribution R G B Ye C M R 0.9 G 0.9 B 0.9 Ye 0.9 C 0.9 M 0.9 R 0.5 G 0.5 B 0.5    It is necessary to distinguish the errors presented in Table 2, which deals with spectral simulation using reference daylight and the errors presented in Table 3-5.The spectral power distribution of the reference color is affected by an illuminant with a spectrum different from the daylight.

3. Assessment of color rendering on the basis of real spectral color distribution
The search for the required color is made from the set of spectral color distributions according to [20,21].Determination of the desired spectral power distribution is carried out by determining the parameter DE (1), and the conformity criterion is determined according to (7), where the value DE should tend to zero.
Desired spectral power distributions are presented in Table 6.Primary and complementary colors with 90 % saturation and spectral power distribution presented in Fig. 2 R 0.9 G 0.9 B 0.9 Ye 0.9 C 0.9 M 0.9 In Table 6, color markers indicate pairs that satisfy two test colors, but with different errors.This is caused by the fact that the insufficient set of spectral power distributions is presented in the papers, so some of them were chosen for both colors due to the lack of other spectra.In practice, the color with the smallest error should be chosen for color testing.
The results presented in Table 6 can serve to construct optical metrological support for assessing the performance of the video transmission path.

New steps in the metrological support of multimedia and media paths
The use of the conventional or extended set of test colors, as presented above, has a number of limitations, namely: -for the first (Table 1, 6) there is no spectral power distribution of the proposed color sets; -the limited use of the set of test colors due to the small number of colors; -the impossibility to fully use the set due to technical limitations of the video transmission system.
As shown in [16], with increasing image brightness, the region of rendered colors changes in the form of the cut of blue, red, and green parts of the color triangle.In the limited range of color rendering, the proposed color test points are shown in Tables 2, 6, and accordingly, the process of predicting the influence of the illuminant cannot be effective.Let's consider these limitations and possible solutions in more detail.
The proposed algorithm of adaptation to illumination should be implemented using metrological support, in a much broader sense than is currently adopted.It is expedient to take into account the limited area of the colors rendered due to changes in image brightness, and the physiological properties of the color perception process.
The presented color sets [17] used for assessment in [16] are not sufficient to predict the effect of the illuminant on the color coordinates.That is why it is necessary to expand the metrological support.For example, the color chart proposed in [22], where the authors offer only color coordinates.For the full use of the proposed color charts, it is suggested to use the search algorithm presented in Fig. 2, 3, which will allow determining the spectral power distribution of the color chart and the influence of the spectral power distribution of the illuminant.
The magnitude and nature of distortions are presented in Table 7 and Fig. 8, where о -reference points, and + -a shift of the points under the influence of illuminants A, D65, F1 and F315.

Table 6
Color data recommended for color rendering assessment (x, y -color coordinates according to ITU-R BT.709, x spectral , y spectral -color coordinates equivalent to the spectrum found) Color selection from the set of spectra according to [20] Color selection from the set of spectra according to [   From the obtained data it follows that: -taking into account the influence of the illuminant is relevant, as this may lead to unacceptable changes in color coordinates, and accordingly, colors; -different illuminants affect differently each of the chosen points of the color set, Fig. 8; -taking into account the error in the xyz coordinates indicates acceptable changes in the color coordinates, and in the CAM16 uniform chromaticity color perception systemunacceptable.So, to improve the quality of the transmitted video, it is necessary to take into account the properties of color perception.

Results of studies of the algorithm of adaptation to the illuminant and color perception properties
Input data was the image that has a wide range of colors and can be used for color studies.The simulation was carried out using illuminants D65, F135, F1, A. The block diagram of the simulation process is presented in Fig. 9. Using the algorithm presented in Fig. 1-3, the result of restoring the image with the given spectral power distribution of the illuminant was achieved.The restored image is shown in Fig. 11.
According to the research, the results of the objective (Table 3-  The presented studies concern color adaptation to the illuminant and the possibility of rendering through the communication channel, taking into account the properties of color perception on the transmitting and receiving sides.
The proposed algorithm allows determining the magnitude of the influence of the illuminant at the point of observation without the use of optical standards.Simulation is done using the proposed set of colors that evenly fills the region of rendered colors.The colors used are presented in color coordinates, therefore the paper proposes the analytical method for determining the spectral power distribution of the chosen color.The block diagram of the algorithm of continuous adaptation to the illuminant using the algorithm presented in Fig. 2 is proposed.It should be noted that only brightness adaptation is currently carried out, and the proposed methods concern color adaptation at the level of spectral power distribution.
The advantages of the proposed method include adaptation directly on the shooting or reproduction site, without the use of optical standards for system calibration, which allowed ensuring the process of undistorted video transmission.The magnitude of the influence of the illuminant on color in the understanding of color perception properties that can be corrected by the system is presented in Table 3 The analysis of existing methods of control and provision of undistorted color rendering is carried out, Table 6, and it is proved that this color set is not enough when the system operates in a wide brightness range of the image object.Therefore, the color chart that evenly fills the region of colors rendered by the system and expands the opportunities for debugging the video system from studio conditions to any others was proposed.
Using the color perception model for predicting and accounting for changes in the image during transmission or reproduction allowed ensuring that the image is reproduced as it should be seen by the viewer.This circumstance will ensure not only comfortable viewing of the transmitted video content but in specialized applications where color parameters are critical may be essential.
The limitations of the algorithm, such as system parameters and the impossibility of rendering the entire color chart at different levels of brightness should be noted.Also, possible limitations include the presence of a device to determine the spectral power distribution of the illuminant on the transmitting and receiving sides.This can be solved by installing an additional sensor on existing systems or built-in on new ones.
The use of the color perception model is focused on the standard viewer, which can be a limitation, so continuing research with the expansion of the model capability on a universal viewer is a priority.
With the development of new video transmission systems, use in areas such as medicine, the main thing is the quality of color rendering, but ensuring the quality of color rendering is due to the appearance of minor shortcomings.These include an increase in the rated capacity of the transmitting and receiving equipment in relation to the existing one.
It is proposed to continue studies concerning the spectral image adaptation on real images and scenes, using a larger set of spectral power distributions of the illuminant.

Conclusions
1.According to the results of simulation of the spectral power distribution of the color of the transmitted image with given coordinates, data with an error of up to 0.2 CIE units (Fig. 4), which is qualified as unnoticeable distortion are obtained.On the basis of the obtained spectral power distributions, the simulation of elimination of the influence of various illuminants, which introduce unacceptable errors in relation to the original (Fig. 10, 11) was made.Thus, the developed algorithm can be used to eliminate the influence of various illuminants.
2. It is proved that the color perception model is necessary since when using the conventional coordinate system, the maximum error in various illuminants is within 0.025-0.177CIE units.The maximum magnitudes of influence in the coordinates of the color perception model are within 10-19.02CIE units.This indicates that in the first case it can be said that the color is not distorted, since the error is minimal, but in fact, from the observer's viewpoint, distortions take inadmissible values (more than 4-8 CIE units).Thus, adaptation is carried out regardless of the shooting conditions and color perception properties.
3. Because of the increasing brightness of the image stimulus, the region of rendered colors decreases, but the algorithm is not affected by the limitations.Applying the algorithm for adaptation in the color field that is not transmitted by the system due to limitations is not appropriate in terms of optimizing the general system.
4. As a metrological tool, it is proposed to use a set of coordinates of the color chart and the algorithm for determining the spectral power distribution with given color coordinates.One of the metrological tools can be a set of spectra of universal test colors.The error of the metrological instrument does not exceed 0.2 CIE units, which on the resulting image is qualified as "unnoticeable" on the color distortion scale.

Fig. 1 .
Fig. 1.Effect of the illuminant on the transmitting and receiving sides of the video path

Fig. 2 .
Fig. 2. Algorithm of color adaptation of the scene object to the spectral power distribution of the illuminant x e , y e -the coordinates of the test color in the xyz coordinate system, x out , y outthe coordinate values under illumination with the chosen illuminant e , M a′ e M b′ -the coordinates of the test color in the coordinate system of the CAM16 uniform chromaticity scale system () coordinate values under illumination with the chosen illuminant.

Fig. 4 -
Fig.4-7show the spectral power distributions equivalent to the color coordinates of Table1.

Fig. 5 .
Fig. 5.Rated spectral power distribution for primary and complementary colors of 100 % saturation of high-definition television.Characteristics are based on the Gaussian distribution

Fig. 6 .Fig. 7 .
Fig. 6.Rated spectral power distribution for primary and complementary colors of 90 % saturation of high-definition television.Characteristics are based on the Gaussian distribution

Fig. 8 .
Fig. 8. Magnitude and nature of the influence of the illuminant on the color coordinates: а -the influence of the illuminant A; b -the influence of the illuminant D65; c -the influence of the illuminant F1; d -the influence of the illuminant F315

Fig. 9 .
Fig. 9. Block diagram of the simulation processThe image from the camera is shown in Fig.10, where the n×m matrix consists of a set of correction values DE, in which n is the number of image elements horizontally, and m -vertically.Knowing the magnitude and nature of the influence of the illuminant, the value DE can be taken into account, the results are presented in Fig.11.Fig.10shows the transmitted image of the scene under different illuminants, and the restored image, Fig.11.It is noticeable how the color tone and saturation in Fig.10change in comparison with reference Fig.11.Using the algorithm presented in Fig.1-3, the result of restoring the image with the given spectral power distribution of the illuminant was achieved.The restored image is shown in Fig.11.According to the research, the results of the objective (Table3-5) and subjective (Fig.10, 11) assessment of the influence of the illuminant are presented.
5) and subjective (Fig. 10, 11) assessment of the influence of the illuminant are presented.

Fig. 12 ,Fig. 10 .FigFig. 12 .
Fig.12, 13 show the complex influence of the illuminant and the properties of human color perception.Given the color perception properties, it is possible to extend the brightness range of the algorithm.The specified extension allows getting the image of Fig.11regardless of an illumination intensity (dark Fig .12,dull, average Fig.13).

Fig. 13 .
Test images with shooting conditions on the transmitting side L A =20 cd/m 2 and observation conditions -dull, and observation conditions L A =200 cd/m 2 and observation conditions -average under different illuminants: a -illuminant A, b -illuminant D65, c -illuminant F315, d -illuminant F1

Table 3
Results of simulation using illuminant D65

Table 4
Results of simulation using illuminant A

Table 5
Results of simulation using illuminant F1Primary and complementary colors with 100 % saturation and spectral power distribution presented in Fig.1

Table 7
Maximum value of the influence of the illuminant on the color, expressed in units ΔE