deVelopment of the CompUter model of three dImensIonal sUrfaCes reConstrUCtIon system

Вирішено завдання побудови комп’ютерної моделі, що імітує роботу як програмної, так і апаратної частини (проектор і камера), системи відновлення поверхонь 3D об’єктів за допомогою техніки структурованого освітлення. Запропонована модель повністю повторює роботу реального сканера 3D об’єктів, що дозволяє використовувати її для усестороннього вивчення та перевірки адекватності роботи різних технік структурованого освітлення. ключові слова: бінарна кодифікація, відновлення тривимірної поверхні, комп’ютерне моделювання, комп’ютерний зір, структуроване підсвічування. lymarenko y., tatievskyi d.


Introduction
For several decades, research has been carried out on scanning systems of three-dimensional objects, which are mainly aimed at increasing the resolution, accuracy and reducing the time of capture and image processing. This allows, knowing the internal parameters of the cameras, as well as their location, to determine the three-dimensional coordinates of various points of the object from the obtained images, highlighting the corresponding points on them [1,2].
At the present stage, many different versions of pictures have been developed for use in structured illumination systems. The developed versions are both series of changing pictures (pictures with time multiplexing), and unchanging pictures using various color-coding options [3][4][5].
These options are characterized by a different combination of the total time of registration, reliability, accuracy, resolution and the choice of a particular picture is related to the scope of the system, the characteristics of the objects being registered [6].
Thus, currently in three areas of human activity, such as engineering, computer graphics, robotics and medicine, three-dimensional modeling and prototyping of real-world objects is actively used. With the development of computer vision algorithms, the development of real-world surface reconstruction systems capable of creating the most accurate 3D model of a registered object is becoming more urgent.

the object of research and its technological audit
The object of research is a system for scanning threedimensional surfaces using the method of structured illumination.
The main active elements of a real 3D reconstruction system are the projector (usually DLP) and the video camera -«projector-camera» (Fig. 1).
Knowing the internal parameters of the system, as well as their relative positioning, it is possible determine the three-dimensional coordinates of the points of the object by triangulation method [6].
The method of structured illumination is considered one of the most reliable methods for reconstruction of the relief of objects. A certain picture is projected onto the ob-ject (structured illumination), its distortions that repeat the shape of the object are recorded by the ca mera ( Fig. 2) [6,7]. Because the picture is encoded, it is possible to easily establish a correspondence between the points of the image and the points of the projected pattern.
It is necessary to create a virtual model of the «projector-camera» system, and also to imitate the method of structured illumination.
It is important to confirm the adequacy of the computer model created and its suitability for testing existing and new algorithms in 3D surface scanning problems using the structured illumination method. To do this, it is planned to conduct experiments based on the computer model created and analyze the obtained results.
Also, it is necessary to establish experimentally possible factors, leading to gross errors and errors in the reconstruction of surfaces.

ISSN 2226-3780
The technical capabilities of the computer model are not limited to the physical parameters of the emulated system. It is possible without any special difficulties to simulate the operation of the system with parameters that can't be achieved on a real object. For example, a pattern of structured backlight with a number of bands of 1024 (2 10 ) will prove to be difficult to process with a physical camera with a small optical resolution on a real scanner model.
One of the most problematic places of this model can be attributed the fact that the computer model will be limited only to the resolution of the raster image from the virtual object with all other equal parameters of the system. Of the shortcomings, it should be noted a fairly long process of image processing and algorithm for calculating point clouds, which can reach one minute on a computer with an i5 processor (fifth generation).

the aim and objectives of research
The aim of research is development of a computer model simulating the operation of the «projector-camera» system and algorithms for surface reconstruction of three-dimensional objects using the method of structured illumination in real time.
To achieve this aim, it is necessary to solve the following tasks: 1. To consider the components for building a computer model that simulates the operation of the «projectorcamera» system.
2. To perform an imitation of structured illumination method.

research of existing solutions of the problem
In 1981, scientists first proposed the projection of a sequence of templates for coding strips, using a simple binary code (m patterns, 2 m bands) [8].
In this case, the projector image columns are encoded. The logical level «1» corresponds to the saturation in black, while «0» corresponds to the fully illuminated white. Thus, the number of bands increases twofold on each subsequent template. Each strip of the last template has its own binary keyword. The maximum number of templates that can be projected is the resolution of the projection device in pixels. It should be noted that all the pixels belonging to the same band in the template with the highest frequency have the same keyword (Fig. 3).
Binary codification, referring to the so-called structured backlight with time multiplexing, supports only fixed objects. In [9,10] the improvement of the codification scheme, Gray code implementation and the use of the M-sequence in the scheme are shown.
Among the main directions of the scanning system of three-dimensional surfaces by the method of structured illumination, revealed in the resources of the world scientific periodicals, can be singled out: -coding of illumination elements. Most fully describes the methods of structured illumination with encoding, but does not affect other methods, for example, photogrammetry, and methods of projection of the strip system [4,5]; -methods of structured illumination of ways of the image projection on object [10]; -methods for determining the relief based on the method of the projection of bands and spatial phaso metry [11]; -combined methods of active photogrammetry [12]; -methods for measuring the coordinates and shapes of three-dimensional objects [13][14][15][16][17][18][19].

fig. 3. Projection patterns under binary codification
At present, scientists have developed many different options for use in structured illumination systems, representing both a series of changing images (images with time multiplexing) and unchanged pictures using various color-coding options [3,4]. These options are characterized by a different combination of the total time of registration, reliability, accuracy, resolution [4]. The choice of a specific image is related to the scope of the system, the characteristics of the registered objects.
One of the most universal methods of recording is the method using sinusoidal pictures with varying phase shift, first proposed in optical measurements and most often used in the three-picture version [6,10]. The main advantages of such structured illumination are the relatively small number of projected pictures and the ease of decoding, and the disadvantages include the ambiguity of the decoding caused by the periodicity of the signal.
A perspective direction of the development of 3D object image registration systems, free from the noted limitations of individual methods, are complex systems consisting of several cameras and projectors and combining several recording methods.
Structured illumination provides the ability to restore three-dimensional coordinates of points of objects without a pronounced texture and with large homogeneous areas. Using the stereoscopic method allows to work with objects with a pronounced texture (often making it difficult to decode the projected picture) and to obtain a sufficient number of reference points to «deploy» the phase based on stereo identification. Despite the available developments, this area remains insufficiently investigated [9,10,19,20].
Thus, the results of the analysis lead to the conclusion that it is necessary to create a virtual model of the «projectorcamera» system, and also to imitate the method of structured illumination. It is important to confirm the adequacy of the computer model created and its suitability for testing existing and new algorithms in 3D surface scanning problems using ISSN 2226-3780 the structured illumination method. To do this, it is planned to conduct experiments based on the developed computer model and analyze the obtained results. It is also necessary to establish experimentally the possible factors, leading to gross errors and errors in the reconstruction of surfaces.

methods of research
The structural system is two computer programs written in the Unity environment, in the C# programming language (Fig. 4, 5). Let's outline the structural scheme of applications (Table 1): 1. The first application is responsible for: -modeling of three-dimensional test objects, which will be recognized; -building illumination templates and setting their parameters; -generation of images from a virtual camera; -simulation of the rotary table; -display the results of image processing. 2. The second application is responsible for: -calculations and testing of algorithms for recognition and triangulation; output of results for visualization; -ability to work with external devices (camera and projector). -calculations and operation of computer vision algorithms (Fig. 6); -gluing images from different angles (optional); -Readout of the results in the form C = = {c 1 , c 2 , …, c n } back to (Application1) or in a 3D package for visualizing the results (STL-format) The model considers the simplest and, nevertheless, quite reliable, method of structured illumination with binary codification.
The steps of the algorithm for determining the coordinates of the illumination lines are shown in Fig. 6. It should be noted that all the pixels belonging to the same band in the template with the highest frequency have the same keyword. In this regard, before triangulation it is necessary to calculate either the center of each strip, or the edge between two consecutive bands. The experiments demonstrated the advantage of the second variant.
In the last cycle of work, the computer model generates a cloud of points from a certain angle. After passing through the cycle for all received sets of points from all angles, the resultant set of points for the whole figure is obtained.

research results
To confirm the model's performance, it is necessary to conduct a visual experiment demonstrating the result of the process of three-dimensional object reconstruction.
The following hardware is used to create the scanner: -camera; -projector; -rotary table; -personal computer (PC); -two tripods.
The time taken to install/configure the system is 30 minutes.
The time of the first scan depends on such parametersthe number of angles (turns) and the number of lines. Also, the processing time of the scanned images depends on the computing power of the PC.
Let's consider a fixed three-dimensional object with certain properties and shape of the surface. It is necessary to represent it in the form of a set of points in a Cartesian three-dimensional coordinate system (in the form of a cloud of points).
In this case, the main criterion for the quality of object recovery is the maximum correspondence of all coordinates, the obtained points with the initial coordinates of the points of the surface of the object.
In other words, using a computer model, a 3D object is created, or rather, its surface consisting of a set of points in 3D space: where p i -a point with coordinates x y z i i i , , .

( )
For the convenience of processing the results, let's consider the case of a ball reconstruction, or more precisely, a ball or sphere, for which a 3D model with a nonglossy texture is created.
A set of points of the sphere surface can be easily obtained using the equation of a sphere in three-dimensional space: x y z r 2 + + = 2 . 2 2 Let's denote the desired result as a set of points in 3D space: where c i -a point with coordinates x y z ( ) It is necessary to accept the assumption that the number of elements in the original set P need not necessarily equal the number of elements in the set C.
For this experiment, let's take 7 binary templates of structured illumination with a maximum resolution of 2 7 = 128 lines. This is enough for the camera resolution of 740 × 480 pixels. The number of angles is 12. The model is rotated every 30°. For convenience of calculations let's take the radius of the ball r = 1 (in conventional units of measurement). Fig. 7 shows the image obtained from the virtual camera of the Unity application system. 7 images are necessary for each angle. In parallel, a «clean» image of the reference plane remains. It is necessary to compare the mutual position of the projection lines with and without the object. fig. 7. The image of the scanned object from the virtual camera It should be noted that the ball is represented as an approximate model (a set of large numbers of triangles).
After processing the algorithms of computer vision (noise filter, allocation of strips with subsequent triangulation, removal of the reference plane), let's obtain a cloud of points of the reconstructed ball surface (Fig. 8, 9).  ; -projector shines with a structured illumination on the model; -camera takes pictures of the illuminated model; -pictures are processed by the application. As a result, a computational experiment is conducted using the developed software simulating the work of a full cycle of a three-dimensional scanning of an object in the form of a ball. The number of received and processed points is 97262.
By conventionally aligning the centers of the reference and obtained balls, let's calculate the radii differences over the entire surface of the ball.
The obtained results reflect the distribution of errors. It is easy to see that 90 % of the result is within 10 % of the error (Fig. 10).
During the operation of the system, obvious defects associated with recognition of the illumination band in some areas, for example, partial illumination (Fig. 11) are detected. fig. 10. The ratio of the number of points to the distance from the ball center fig. 11. Partial flare at poles and edges The center of the line in a particular line of the image is taken as the center of the white area in this line. At some angles, the line does not completely reach the object, and then recognition errors occur.
Thus, the method of structured illumination used allows scanning objects with sufficient accuracy.
Among the shortcomings, it is possible to distinguish the significant influence of the tuning accuracy on the quality of the result, as well as the influence of the hardware part on the quality and the time of the system operation.
Subsequent processing of the resulting set of points can greatly improve the accuracy of the resulting model.

sWot analysis of research results
Strengths. The computer model of the reconstruction system of three-dimensional surfaces allows qualitatively to optimize the process of developing a physical 3D scanner using structured illumination, with emulation of the basic tactical and technical characteristics of developed system.
The costs of building a physical prototype 3D scanner can be reduced to a minimum, having information on such key parameters of key elements as: -camera -resolution, focal length, viewing angle; -projector -arrangement and other characteristics of the system. The computer model is able to determine reliable and visual results of the developed device.
The cycle time for testing the operation of the 3D scanner system is reduced several times due to the complete absence of mechanical manipulations with real equipment.
Weaknesses. The weaknesses of this model include the fact that the emulator of the 3D scanner system works in an ideal environment, neglecting such physical phenomena as optical distortions, reflection or absorption of light rays on the surface of the scanned object. In this sense, it is rather difficult, but it is possible, to predict the influence of some physical factors on the results of the system operation. In the current study, such tasks are not raised.
Opportunities. At the moment, the practical application of the system is considered in the study of the arm profile reconstruction using structured illumination, the method of spatial coding, as well as in the study of the method of contactless removal of tire characteristics on trucks. Thus, it is convenient to conduct research on various calibration methods for camera-projector systems.
The possibilities for further research lie in the exten sion of this model to a subset of the most popular methods of structured illumination and the optimization of calibration approaches.
Threats. Currently, no analogues of the use of this model have been found in practice. There are solutions for testing the work of certain methods of structured illumination, but, as a rule, without the possibility of taking into account the technical parameters of the 3D scanner system.

Conclusions
1. To build a computer model of the «projector-camera» system, a client-server model and an autonomous system consisting of two applications that were responsible for both calculations and input and visualization of the input and output data of the system were calculated.
Also, a basic set of structured illumination patterns was selected, a mathematical apparatus for solving the ISSN 2226-3780 problem, and evaluation criteria were established to confirm the adequacy of the system as a whole.
For the experiment, 7 binary patterns of structured illumination with maximum line resolution 2 128 7 = were taken. The number of angles was 12. The model was rotated through each 30°. For the convenience of calculations, the radius of the ball r = 1 was taken (in conventional units of measurement).
2. A full-fledged simulation model for structured illumination (binary codification method) within the virtual projector-camera system was implemented. In the system, there is a functional of the rotary table for fixing the relief over the entire outer area of the object (from all angles).
The results of the system operation are confirmed by a test scenario using a virtual scanned object in the form of a ball, with a given radius.
According to the results of the study of the cloud of obtained points, it can be concluded that the maximum error in the distance of the obtained point of the ball surface from its virtual center is 10 % of the length of the radius, and an average error of 4.5 %.
Thus, the developed model gives a reliable result and is suitable for testing existing and new algorithms in 3D surface scanning problems by the method of structured illumination.