SEGMENTATION OF THE IMAGES OBTAINED FROM ONBOARD OPTOELECTRONIC SURVEILLANCE SYSTEMS BY THE EVOLUTIONARY METHOD

The experience of recent armed conflicts, hybrid wars and anti-terrorist operation on the territory of Donetsk and Lugansk regions of Ukraine shows that the main means of reconnaissance are unmanned aerial vehicles (UAVs) and space surveillance systems. The following tasks can be achieved with the help of UAVs and space monitoring systems [1–3]: – 80–85 % of the reconnaissance tasks; – 50–60 % of the defense tasks; – 40–50 % of military operation tasks. Growth of the demand for information obtained from onboard optoelectronic surveillance systems is accompanied by [1, 4–6]: – constant growth of the total volume and availability of information from onboard systems of optoelectronic surveillance; – growth of volumes of information with high resolution; – implementation of an integrated use of data from various sources in solving tasks of informational support for the sake of security and defense; – growth of the number of entities in the area of obtaining, distribution, processing and using information from onboard monitoring systems. The use of information from onboard surveillance systems ensures [4]: – significant increase of efficiency of use of weaponry and military equipment; SEGMENTATION OF THE IMAGES OBTAINED FROM ONBOARD OPTOELECTRONIC SURVEILLANCE SYSTEMS BY THE EVOLUTIONARY METHOD


Introduction
The experience of recent armed conflicts, hybrid wars and anti-terrorist operation on the territory of Donetsk and Lugansk regions of Ukraine shows that the main means of reconnaissance are unmanned aerial vehicles (UAVs) and space surveillance systems.The following tasks can be achieved with the help of UAVs and space monitoring systems [1][2][3]: -80-85 % of the reconnaissance tasks; -50-60 % of the defense tasks; -40-50 % of military operation tasks.
Growth of the demand for information obtained from onboard optoelectronic surveillance systems is accompanied by [1,[4][5][6]: -constant growth of the total volume and availability of information from onboard systems of optoelectronic surveillance; -growth of volumes of information with high resolution; -implementation of an integrated use of data from various sources in solving tasks of informational support for the sake of security and defense; -growth of the number of entities in the area of obtaining, distribution, processing and using information from onboard monitoring systems.
The use of information from onboard surveillance systems ensures [4]: -significant increase of efficiency of use of weaponry and military equipment;
In current conditions, stringent demands to reliability, immediacy and quality of information obtained by the help of onboard optoelectronic surveillance systems are lodged.In this regard, continuous improvement of both specialized survey equipment and its carriers and, consequently, rise in information flows [7][8][9][10] from onboard systems of optoelectronic surveillance is observed, The process of obtaining optoelectronic images is accompanied by the influence of such negative factors as motion and vibration of the survey apparatus, non-standardized lighting conditions, various kinds of distortions, interference with data transmission, etc. [7,9,10].In such circumstances, the issue of qualitative processing of optoelectronic images is relevant.
It is known [11][12][13][14] that there is no general theory of optimal representation and processing of images at present.The choice of a specific image processing technology depends on the tasks being solved and the requirements that are put forward to the results of processing.There are numerous practical tasks requiring detection of visually imperceptive areas (objects of interest) in low-contrast images.
Complexity of image processing is due to: -first, anomalies (e.g.small regions in the image) that can be taken as noise or the image defect; -second, an unknown form and fuzzy boundaries of the objects of interest.
The foregoing also applies to the images obtained from the onboard systems of optoelectronic surveillance that solve problems in behalf of security and defense [4,8,10].The result of processing the image obtained from the onboard systems of optoelectronic surveillance depends on the quality of the image segmentation method.Therefore, developers of the image processing systems face the focal problem of working out procedures and methods and choosing indices of the image segmentation quality.
The urgency of development of an efficient segmentation method for images obtained from onboard systems of optoelectronic surveillance is also determined by [15][16][17]: -improvement of special mathematical and software support for automation of the most complex steps of surveillance information processing; -cutting processing time and enhancing reliability of the resulting documents; -intellectualization of data processing (detailed surveillance of the objects of interest, solution of large-scale topical tasks for big territories).

Literature review and problem statement
The number of works devoted to development, modernization and application of numerous methods for image segmentation constantly grows.This is due to the significant impact of segmentation on final quality of image processing and decryption.For example, according to the electronic scientific and technical library of the Institute of Electrical and Electronics Engineers (IEEE) [18], the number of publications that have term "image segmentation" in their titles, abstracts, or keywords is as follows: - In most cases, the task of segmentation is determined by the necessity of distinguishing an object or objects in an image.A characteristic feature of such tasks is fulfillment of a predefined criterion on all or almost all pixels of the segmented region.Occurrence of the criterion and the condition of its execution determine the concept of homogeneity of the segmented region and connectivity of numerous image elements.Today, there are various classifications of segmentation methods.In a general case, classification of segmentation methods is considered whereby they are divided into three groups: statistical, feature structure and mixed.Because of multiplicity of segmentation methods, their classification according to various characteristics is becoming increasingly popular in each of these groups, for example, according to the modes of processing (real time mode, soft mode, online mode, intelligent analysis), input data (a separate image, an array of images or a stream of images), by specialization (isolation of strictly defined objects), etc.Not less popular is classification of the methods based on machine learning, and those which do not use it.The principal distinction the last classification of methods is existence of various costs for procedures of training to ensure high-quality segmentation at the stage of use.
The main studies in this work are concentrated in the field of segmentation methods that are not based on machine learning.The available methods that are not based on machine learning can be divided into 5 classes [13,14,19]: -methods based on the use of entropy [20][21][22]; -methods based on clustering [23][24][25]; -attribute methods [26,27]; -histogram methods [28]; -locally threshold methods [29,30]; -correlation methods [31,32].Absence of a training procedure makes these methods fast enough in terms of program implementation.However, their inherent limitations and disadvantages should be taken into account for their successful use.
The segmentation problem gets much more complicated in the case of segmentation of a plurality of objects each of which can have its own parameters of brightness or noisiness.One of the most known tasks of segmenting pluralities of objects is distinguishing of a text on a certain background.Without this background, the segmentation problem is solved by the binarization procedure.
The essence of binarization procedure [33] is a reduction of the existing image to an image with binary values of the intensity function (usually 0 and 1).This allows one to significantly reduce amount of information to be processed.The result of this reduction can be both successful enough and unsuccessful.The failure of the binarization procedure can be interpreted as appearance of various distortions, such as break of boundaries, loss of detail, noisiness and various distortions (including those arising due to heterogeneity of the background, etc.).All these negative results of binarization have a significant effect on further image processing.They are especially relevant in the case of solving the problem of recognition.
The available methods of binarization can be divided in two groups: -threshold methods (lower threshold processing, upper threshold processing, threshold processing with underlying restrictions, incomplete threshold processing); -adaptive methods (multilevel threshold processing, local threshold processing (Otsu method, gradient, and entropy methods), global threshold processing (Bernsen, Eykvel, Niblek, Yanowitz, Brukstein methods, etc.).
The fundamental difference between the methods of these two groups consists in existence of a single threshold for the entire image.For the adaptive group methods, this value is determined within individual image segments.This feature improves segmentation efficiency in a case of the images characterized by a change in illumination.Obviously, adaptive methods require more computational resources in most cases.
The main advantage of binarization, especially in the case of threshold methods, is the speed of its procedures.However, the problem of successful selection of a threshold criterion can outweigh all benefits of the binarization procedure.For example, in many practical cases, there is no automatic or automated procedure of binarization method selection and therefore this work has to be done manually.
The above mentioned known methods of image segmentation cannot be directly applied to segmentation of the images obtained from onboard systems of optoelectronic surveillance.One of the reasons of impossibility of direct application is that the known methods do not take into account peculiarities of formation of the images coming from onboard systems.In addition, there are three main types of shortcomings of segmentation of the images obtained from onboard systems of optoelectronic surveillance [13,14]: -incorrect segmentation: the contours of distribution do not coincide with the boundaries of the objects in the image; -oversegmentation: a redundant distribution of the image in the area occurs; -undersegmentation: an insufficient distribution of the image in the area occurs.
Typically, segmentation methods use several parameters.Their proper selection can prevent the last two shortcomings.However, the first shortcoming can only be prevented by choosing the proper segmentation method.
Thus, the known classical methods of image segmentation are characterized by the following main shortcomings [13,14]: -most methods do not define boundaries of objects and do not perform segmentation but only emphasize boundaries of objects; -segmentation by just criterion of the level of brightness of the point in the image which restricts determination of the segment homogeneity criteria.
Currently, genetic methods are used (for example [36,37]) to solve various problems that arise in processing images, including their segmentation.Genetic methods represent an independent section of the theory of artificial intelligence, namely evolutionary calculations which are based on mathematical modeling of biological evolution processes.Genetic methods are used to solve optimization problems in combinatorics, bioinformatics, game theory, processing, and recognition of patterns, in particular, images.
When using genetic methods, the search for a solution takes place in a subset of the search space points which is achieved by creating a set of potential solutions that forms a population.The population is improved by the genetic operators responsible for variability and fitness functions that simulate natural selection.Heritage is ensured by formation of new chromosomes from the previous generation of chromosomes and, accordingly, they have common genes.If the genetic method is implemented correctly, then, the average value of the fitness function of population and the best values of the fitness function grow with each new generation in the direction of global optimum.Currently, genetic methods are used to solve the problem of segmentation of medical images [37].
Evolutionary methods are also used to solve the problem of segmentation of medical images [38][39][40][41].
The known evolutionary methods of segmentation of medical images [38][39][40][41] cannot be used for segmentation of optoelectronic images from onboard systems of optoelectronic surveillance.First, this is due to different conditions of image formation and the information component presented in the image.Second, this is due to the variety of tasks solved during image processing.
From the above analysis of the known methods for image segmentation, in addition to the drawbacks already mentioned, it can be established that development and selection of combinations of the image segmentation methods is a creative process that cannot but depend on the author.Experimental comparison of the developed methods usually leads to the choice of one solution that is used for all subsequent images.If there is a need to process images with a wide discrepancy of characteristics, this approach leads to a significant degradation of the segmentation quality.Besides, rapid development of technical support results in improvement of quality of the equipment and the images obtained with the help of this equipment and adds errors in functioning of the existing methods and systems.

The aim and objectives of the study
This study objective was the application of an evolutionary method for segmentation of images from onboard optoelectronic surveillance systems.
To achieve this objective, the following tasks had to be solved: -carry out a brief analysis of the main atmospheric factors influencing formation of images in onboard optoelectronic surveillance systems; -set forth essence of the evolutionary method for segmentation of the images obtained from the onboard optoelectronic surveillance system; -conduct an experimental study of segmentation of an image obtained from the onboard optoelectronic surveillance system by the evolutionary method.

1. Analysis of main atmospheric factors effecting formation of optoelectronic images
The main atmospheric factors effecting image formation in onboard systems of optoelectronic surveillance are shown in Fig. 1 [34,35].
where β(r) is coefficient of absorption at a distance r; σ scatt is volumetric scattering coefficient.
With the smallness of the effects of multiple scattering (in terms of the Bouguer law), the following is obtained [34]: where I 0 and I 1 are intensities of radiation in the starting (on the surface of the Earth) and in the end (in the satellite orbit (in the UAV photography point) points respectively.The effects of multiple scatter can be neglected for the green (G) and long-wave spectrum regions but not for the blue (B) region [34,35].Expression (2) can be used for a nadir surveillance.For surveillance at an angle δ to nadir (Fig. 1), intensity of radiation for a homogeneous atmosphere is: 2 0 exp( / sin ).
Expression (3) does not take into account curvature of the Earth, but there are methods [34,35], which enable taking this fact into account.Expression (3) also does not take into account influence of possible local horizontal inhomogeneities (fog, smoke, cloud of dust, etc.).

2. Essence of the evolutionary method of segmentation of images obtained from the onboard system of optoelectronic surveillance
For image segmentation, methods were considered that relate to the evolutionary class.Evolutionary methods, e. g. particle swarm optimization (PSO) methods and ant system (AS) methods are based on modeling social behavior of liv-ing creatures.In this work, we focused our attention on the ant method.
The evolutionary method was originally used to search for the shortest path in graphs and further studies have led to appearance of a variety of its modifications and shown its versatility in solution of a wide range of optimization problems.One of the important advantages of this method is its high efficiency in optimization of distributed non-stationary systems.When the studied system changes, the method quickly adapts to these changes and finds a new optimal solution.The abovementioned and other benefits of the method (e. g., its speed) make it relevant to conduct a study of possibility of using the ant method and its variants before segmentation of the optoelectronic image.
In the simplest case, segmentation of an image can be represented as a set of subsequent sections of the agent (ants) motion (Fig. 2): a starting point of the route (SPR), straight sections and an end point of the route (EPR).Straight sections pass through the turning points of the route (TPR) in which change in direction of the agent movement occurs.Hereinafter, we assume that the SPR, EPR and TPR positions completely determine route of the agent.

Fig. 2. An example of representation of the agent route in the image during image segmentation
Movement in each section of the route as well as rotation in selected TPRs have certain hazards and require some resource costs which results in an advantage of one route over others.Since there may be a lot of options for the TPR location, the number of possible routes will be extremely large which hinders selection of the route by the enumerative technique.Let us demonstrate how to route using a simpler evolutionary method (the AS method).
The evolutionary method used in this study is based on simulating a natural mechanism for finding the shortest path to the source of food by a colony of ants (agents).Self-organization of the system is provided by a low-level interaction of the agents when they exchange with just local information through the use of a special secret, a pheromone deposited by an agent on its route.The next agent moving near the route of the first agent perceives pheromone smell and is highly likely to continue moving along the path of the first agent, in turn depositing pheromone increasing its concentration on the path.The higher concentration of pheromone on the route, the higher attractiveness of this route for next agents.Distribution of pheromone in the environment appears to be the dynamic memory of the system.Each agent perceives and changes one cell of this memory at a certain time, i. e. the level of pheromone in the vicinity of the agent's location point.
Concentration of pheromone deposited on the route is proportional to attractiveness (quality, efficiency) of the route.The more attractive the route, the greater concentration of pheromone on it, and as a result, better routes are stored in the global memory of the colony of agents and are more likely to be selected by the next agents.Over time, pheromone evaporates ensuring feedback.Since, as it is noted above, pheromone concentration will gradually grow on the attractive routes and its evaporation rate is constant, the unsuccessful routes will disappear after some time and an increasing number of agents will move along successful routes only.The use of feedback (evaporation) prevents timely convergence of solutions, i. e. choice of the same suboptimal route by the agents.
In a simpler evolutionary method, m agents search for a solution and renew pheromones on the discovered route in each iteration of the iterative process.In image segmentation, each m-th agent begins its path from the SPR, successively passes selected TPR and completes the path in one of the EPRs.Selection of a TPR from J possible TPRs is based on the probabilistic rule that determines probability of the ( ) m i P t transition of the m-th agent to the i-th TPR taking into account attractiveness of the i-th section L i of the route and concentration F i of pheromones in this section at time t as follows (expression ( 4)): where α and β are the parameters setting pheromone weight and attractiveness of the area ("greediness" of the method), respectively.At α=0, agents at each their step pass to the nearest TPR and the evolutionary segmentation method turns into a "greedy" method of the classical theory of optimization.For β=0, only the effect of pheromones is considered which will quickly lead to a suboptimal solution.Probabilities of choosing one or another TPR are found by expression (4).The choice itself is based on the principle of "roulette wheel".This can be realized, for example, by dividing of some section of a length S into J parts of a length proportional to ( ), m i P t generation of a random number uniformly distributed in the interval [0, S] and selection of a TPR according to what (by the sequence number) part of the section S the random number gets.
Consider that attractiveness of the section L i of the route in the evolutionary method is inversely proportional to the costs of passing this section, i.e. expression (5): where D i is the length of the i-th section of the route.At the start of the iterative process, quantity of pheromone on the route sections is taken the same and equal to some small number F 0 .After each iteration, concentration of pheromones on the sections chosen by the agents is updated according to the rule (6): where is the rate of pheromone evaporation; is concentration of pheromone on the i-th section of the route created by passage of the m-th agent.
As a result of a certain number of iterations, the most attractive (with the maximum concentration of pheromone) routes are determined by the chosen criterion.Pheromone is gradually "drying" on nonattractive routes and these routes disappear.
Verification of function ability of the evolutionary segmentation method was done on the control example.For visual presentation of the results in all examples, a rectangular coordinate system in which the SPR, EPR and TPR are in the same horizontal plane is used.Attractiveness of the route sections was calculated by expression (5).The results of calculations are shown in Fig. 3.

а b
Fig. 3. Results of work of the evolutionary method of image segmentation for the source data of the control example: the route of the agents slightly differs from the optimal route (a); the route of the agents significantly differs from the optimal route (b) Output data: -SPR number N SPR =1; -EPR number N EPR =1; -number of the first potential TPR (TPR 1) N TPR1 =20; -number of other potential TPR (TPR2) N TPR2 =20 (positions of SPR, EPR, TPR1 and TPR2 are shown in Fig. 3); -"greediness" of the method β=1; -pheromone weight α=2; -pheromone vaporization rate ρ=10 -3 ; -number of iterations of the method N=400; -number of agents in iteration m=10; -initial pheromone quantity F 0 =10 -2 .As a result of N iterations, the agent motion path in the image when using the evolutionary image segmentation method is shown in Fig. 3 by a solid line.Broken line shows the optimal route for the agents when segmentation of the image is done (reference segmentation).From analysis of Fig. 3, a, it is evident that the agent movement route obtained by the use of the evolutionary method slightly differs from the optimal route shown in Fig. 3, a by a broken line.However, as shown in Fig. 3, b, the method realizations with undoubtedly unsuccessful results are possible, which indicates the necessity of further studies into application of the advanced evolutionary method for solution of the segmentation task.
The improved evolutionary method of image segmentation presents development of the original ant method (AS).Its characteristic features consist in that only the best agents raise pheromone level on their routes and that this level is limited.Renewal of pheromone levels on the routes is done according to expression (7): where F max and F min are maximum and minimum limits of pheromone levels; [ ] a b x is an operator determined by expression (8): is determined by expression ( 9): and L best is the length of the best agent's route.This may be either the best route L ib found in the current iteration or the best-so-far solution L bs found since the beginning of the method functioning.Verification of function ability of the advanced evolutionary method was carried out using data of the control example.Unlike the original evolutionary method, the optimal route of agents was found in all implementations when the image segmentation was done by the advanced evolutionary method.In Fig. 4, the method operation is sequentially shown after 40, 80, 200 and 400 iterations.As it follows from analysis of Fig. 4, the level of pheromone on all routes still slightly differed from the start level F 0 after 40 iterations but the agents have already marked two routes with pheromones as the best routes (thicker lines).After 80 iterations, the level of pheromone on all routes, except the best ones, was significantly reduced due to evaporation (the corresponding lines have become thinner) and three best routes were standing out among which "controversy" has taken place thereafter.After 200 iterations, the best route has already become clearly distinguished although several routes still tried to "argue", and the best (optimal) route obviously prevailed after 400 iterations

Results of study of application of the evolutionary method for segmentation of optoelectronic images
Let us consider application of the evolutionary method for segmentation of the image obtained from the onboard system of optoelectronic surveillance.Fig. 5 shows the original image obtained from the onboard system of optoelectronic surveillance [49].The result of segmentation of the original image (Fig. 5) by the evolutionary method is shown in Fig. 6.Fig. 6.Result of segmentation of the original image (Fig. 5) by the evolutionary method In the segmented image (Fig. 6), objects of interest can be further distinguished, for example in (Fig. 7): -oil tanks or fuel tanks for planes; -planes that survived raid; -damaged or destroyed planes, etc. From comparison of Fig. 7, 5, it is clear that the contours of the main objects of interest in Fig. 7 coincide with the boundaries of the objects in the original image (Fig. 5).
Decryption of the indicated objects of interest, recognition, thematic classification, etc. are the subject of further study and remain outside the scope of present work.
It is necessary to note a presence of a large number of outlined contours of small size in a segmented image.An example of such region with a large number of small-sized objects outlined with an ellipse is shown in Fig. 8. Similar regions are observed in other parts of the segmented image.Further studies should be directed to reduction of such regions with a large number of small-sized objects for a more efficient decoding of the objects of interest.

Discussion of the results of application of the evolutionary method to segmentation of optoelectronic images
The paper considers a possibility of the application of the simpler and more advanced evolutionary methods to segmentation of the images obtained from the onboard optoelectronic surveillance systems.
Function ability check of the simpler evolutionary method has revealed some errors in segmentation.Function ability check of the advanced evolutionary method using data of the control example has shown an optimal agent route when segmenting the image in all implementations.
Experimental studies of segmentation of optoelectronic images confirmed function ability of the advanced evolutionary method.For example, the potential objects of interest were identified in a segmented image, namely: containers with oil or fuel for planes, plains that survived raid, damaged or destroyed planes, etc.It was established that the outlined contours of the main objects of interest coincided with the boundaries of the objects in the original image.Effectiveness of application of the evolutionary method was estimated in a visual way.
One of the important advantages of the evolutionary method is its rapid adaptation in conditions of variable illumination and presence of a variety of objects of interest of various types and classes in the images.Under these conditions, the evolutionary segmentation method quickly finds a new optimal solution.
Main disadvantages of the advanced evolutionary segmentation method are as follows: -presence of a large number of outlined contours of small objects in a segmented image; -requirement of a more powerful computation resource.Potential application fields of the advanced image segmentation method: -software and hardware complexes for processing images obtained from onboard systems of optoelectronic surveillance (airborne, space-based); -improvement of technologies and means for processing optoelectronic images; -processing of optoelectronic images in optoelectronic stations of onboard aircraft defense systems.
Directions of further studies: -formulation of criterion and evaluation of effectiveness of the evolutionary method of segmentation of the images obtained from onboard systems of optoelectronic surveillance; -comparison of the evolutionary segmentation method with known methods of optoelectronic image segmentation; -study of influence of segmentation quality on quality of further decoding of potential objects of interest, their thematic classification, etc.; -development of methods for reduction of the regions with a large number of outlined contours of small-sized objects in segmented images for more effective decoding of the objects of interest.-scattering of solar radiation in atmosphere in conditions of the Bouguer law validity; -reflection of solar radiation from the Earth's surface.It was established that further studies have to be carried out with introduced restrictions and assumptions, namely: not taking into account possible horizontal local inhomogeneities (fog, haze, dust clouds etc.).

Potential objects of interest Potential objects of interest
Region with a large number of small objects 2. Essence of the simpler evolutionary method for segof the images obtained from onboard systems of optoelectronic surveillance was set forth.The process of image segmentation was presented as a set of areas of movement of agents (ants).Probability of transition from one turning point of the route to another was determined taking into account attractiveness of the route and concentration of pheromones on it.Function ability of the simpler evolutionary segmentation method was tested on a control example.It was established that the route of movement of agents (ants) obtained by the use of the evolutionary method may differ in some cases from the optimal route.The task of further improvement of the simpler evolutionary method of segmentation was set.
Essence of the advanced evolutionary method of image segmentation was outlined.Features of the advanced evolutionary segmentation method are that only the best agents raise the level of pheromone on their routes as well as the fact of a limited level of pheromone on the routes.Function-ability check of the advanced evolutionary segmentation method was performed on a control example.In contrast to the simpler evolutionary method, application of the advanced evolutionary segmentation method has ensured finding of an optimal route of agent movement during image segmentation.
3. Experimental studies of segmentation of optoelectronic images by the evolutionary method have been carried out.Potential objects of interest were found in the segmented image.It was established that the outlined contours of the main objects of interest coincide with boundaries of the objects in the source image.Presence of many outlined contours of small-size objects was noted and an example of such region was given.Visual estimation of efficiency of application of the evolutionary method was made.

Fig. 1 .
Fig. 1.Atmospheric factors influencing formation of images in onboard optoelectronic surveillance systems

Fig. 4 .
Results of work of the advanced evolutionary method of image segmentation for the output data of the control example: a -after 40 iterations; b -after 80 iterations; c -after 200 iterations; d -after 400 iterations

Fig. 7 .
Fig. 7. Potential objects of interest in a image

Fig. 8 .
Fig. 8. Example of the region with a large number of small objects in a segmented image

1 .
Features of formation of images obtained from onboard systems of optoelectronic surveillance have been established.Main atmospheric factors were taken into account: