Purpose. The goal of this work was the development and research of a method of automatically constructing a digital model of the micro surface of an object from SEM stereo pair of digital images taking into account the specifics of the survey SEM and evaluating the accuracy of digital modeling. Methods. The developed method consists, firstly, in generating a dense set of input points in the left SEM image of a stereo pair in regions with local features and using an iterative process in accordance with the levels of the image pyramid. Secondly, the search for the corresponding points in the right SEM image of the stereo pair is carried out on the basis of sequentially shifting the points (centers of the search windows) by a shift parameter from the possible parallax's range using the correlation method. For research, we have used two stereo pairs of digital SEM images. Digital images of the deformed surface chrome steel specimen were acquired with the JSM 7100F (JEOL) with magnification 750x. Images of loess soil were acquired with the SEM “Hitachi” S-800 with magnification 1000x. When calculating the spatial coordinates of the points of the surface micro relief, the values of geometric distortion inherent in the SEM image were taken into account. To eliminate some anomalous values of the heights of the 3D model, an adaptive median filtering procedure was applied. To evaluate the accuracy of micro surface simulation test models were created by manually measuring coordinate feature points of the digital stereo pairs for both specimens. Results. The proposed method for shifting parameters reduces the search area and the probability of mismatch and, in addition, speeds up the matching procedure in a pair of images. Formulas are obtained for calculating the coordinates of the center of the search window and the corresponding point in the right image at the k-th step of the shift process. To estimate the accuracy, the differences between the heights of the test model and the heights interpolated at the same points using the created models were computed. For the chrome steel specimen micro surface about 79% of the points, and for the micro surface specimen of the loess soil about 70% of the points were within tolerance ΔZ ≤ ± 2 μm. Scientific novelty. For the first time in Ukraine, a method was developed for an automatic search of corresponding points based on a shift of parameters taking into account the features of SEM survey. The proposed technological reconstruction automation scheme of a digital model of an object’s micro surface from SEM stereo pair, and the creation of this authoring software show its efficiency and expediency. The practical significance. The ability to reproduce the surface micro relief of an object automatically using a stereo pair of SEM digital images was established in accordance with the requirements of both the accuracy of determining the spatial coordinates of points and the structure of the micro surface of the object.
1. Baghaie, A., Tafti, A. P., Owen, H. A., D'Souza, R. M., & Yu, Z. (2017). Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation. PloS one, 12(4), e0175078.
https://doi.org/10.1371/journal.pone.0175078
2. Bałamucki, J., Czarnecki, P., Gotszalk, T., Marendziak, A., Rangelow, I., Wilk, J., & Kowalski, Z. W. (2006). Profilometric, SEM and AFM Investigations of Titanium and Steel Surface Micro-and Nanoroughness Induced by Neutralized Krypton Ion Beam as a First Stage of Fractal Analysis. Physics and Chemistry of Solid State, 7(3).
3. Cornille, N., Garcia, D., Sutton, M. A., McNeill, S., & Orteu, J. J. (2003, June). Automated 3D reconstruction using a scanning electron microscope. In SEM conference on experimental and applied mechanics (pp. 2-4). Charlotte.
4. Dorozhynskyy, O. L, & Tukay, R. (2008). Photogrammetry: A textbook. Lviv Polytechnic Publishing House, 332 p.
5. Gonzales, R., & Woods, R. (2005). Digital Image Processing. M., Tehnosfera. (in Russian).
6. Gonzales, R., Vuds, R., & Eddins, S. (2006). Digital Image Processing using MATLAB. M.: Tekhnosfera. (in Russian).
7. Gorbachev, V. A. (2014). Development of algorithms for highly detailed object modeling based on digital image analysis: dis. for the degree of candidate phys.-mat. of science. MPhTI (SU). Moscow (in Russian)
8. Gruber M., & Leber F. (2000). High Quality Photogrammetric Scanning for Mapping. In China International Geoinformatics Industry, Technology and Equipment Exhibition pp. 1-15.
9. Ivanchuk, O. M., & Khrupin, I. V. (2012). Structure and function of the program complex «Dimicros» processing of SEM images on a digital photogrammetric station. Modern achievements in geodetic science and industry, 1(23), 193-197 (in Ukrainian).
10. Ivanchuk, O. (2015). Features Calibration geometric distortion of digital SEM images obtained at different SEM. Modern achievements in geodetic science and industry, I (29), 168-173 (in Ukrainian).
11. Ivanchuk, O. (2015). Research geometric distortion digital SEM-images obtained on SEM JSM-7100F (JEOL, Japan) and the accuracy of approximation. Geodesy, Cartography and Aerial photography, 81, 101-109. DOI: https://doi.org/10.23939/istcgcap2015.01.112
https://doi.org/10.23939/istcgcap2015.01.112
12. Ivanchuk, O. (2016). Mathematical model of the relationship of spatial coordinatesof points of a micro surface of a research object with their corresponding coordinates on SEM-stereo images. Modern achievements in geodetic science and industry. I (31), 122-126.
13. Ivanchuk, O., & Tumska, O. (2016). Development and research of technology for automation of the calibration and account of digital SEM images having geometric distortion obtained with JCM - 5000 (Neoscope) (JEOL, Japan). Geodesy, cartography and aerial photography, 84, 56-64. DOI: https://doi.org/10.23939/istcgcap2016.02.056
https://doi.org/10.23939/istcgcap2016.02.056
14. Ivanchuk, O. M. (2017). Technology of processing of digital SEM images of solids micro surfaces., Urban planning and territorial planning: scientific-technical. Sat. Kyiv: KNUBA, 63, 170-184 (in Ukrainian).
15. Ivanchuk, O., & Tumska, O. (2017). Method of automated determination of coordinates of centers of test object nodes for its SEM-images using MatLab tools. Modern achievements in geodetic science and industry, I (33), 158-165 (in Ukrainian).
16. Ivanchuk, O., & Tumska, O. (2017). A study of fractal and metric properties of images based on measurements data of multiscale digital SEM images of a test object obtained with different types of SEM, Geodesy, cartography and aerial photography, 85, 53-64. DOI: https://doi.org/10.23939/istcgcap2017.01.053
https://doi.org/10.23939/istcgcap2017.01.053
17. Ivanchuk, O. M. (2019). Theoretical and methodological foundations of spatial simulation of micro surfaces of objects based on the data of digital SEM photogrammetry. Abstract of a doctoral dissertation, Lviv, 44 p. (in Ukrainian).
18. Ivanchuk, O., & Tumska, O. (2019). Automated construction of digital model of the micro surface an object using a stereo pair of digital SEM images], Modern achievements in geodetic science and industry, IІ (38), 72-96 (in Ukrainian).
https://doi.org/10.33841/1819-1339-2019-2-38-72-96
19. Krig, S. (2016). Interest point detector and feature descriptor survey. In Computer vision metrics (pp. 187-246). Springer, Cham.
https://doi.org/10.1007/978-3-319-33762-3_6
20. Kudryavtsev, A. (2017). 3D Reconstruction in Scanning Electron Microscope: from image acquisition to dense point cloud (Doctoral dissertation, Bourgogne Franche-Comté).
21. Kudryavtsev, A. V., Dembélé, S., & Piat, N. (2017, July). Stereo-image rectification for dense 3D reconstruction in scanning electron microscope. In: 2017 International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS), Montreal, (pp. 1-6). IEEE.
https://doi.org/10.1109/MARSS.2017.8001905
22. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2), 91-110.
https://doi.org/10.1023/B:VISI.0000029664.99615.94
23. Marturi, N., Dembélé, S., & Piat, N. (2013, August). Fast image drift compensation in scanning electron microscope using image registration. In 2013 IEEE International Conference on Automation Science and Engineering (CASE) (pp. 1-6). IEEE.
https://doi.org/10.1109/CoASE.2013.6653936
24. Melnik, V. M., Voloshyn, V. U., Tarasyuk, F. P., & Blinder Ju. S. (1999). Methods of quantitative characterization of soil microstructure. Bulletin of Lviv State University. Geographic series, 25, 24-27. Ivan Franko National University of Lviv (in Ukrainian).
25. Melnik, V. M., & Shostak, A. M. (2009). Raster electron stereomikrofraktografition, Luck, Vezha, 469 p. (in Ukrainian).
26. Melnik, V. М., Radzij, V. F., Melnik, Ju. A. (2010). SEM analysis of the microstructure of sod-podzolic soils. Bulletin of geodesy and cartography. 5, 2-34 (in Ukrainian).
27. Melnik, V., Blinder, Ju., & Piskunova, O. (2015). Methodology of radionuclide migration studies in soil cover. Modern achievements in geodetic science and industry, ІІ (30), 56-60.
28. Melnik, Ju. A. (2013). Determination of structure and micro topography of characteristic surfaces of materials by 3D reconstruction method: author's abstract. dis. for the sciences degree candidate techn. sciences. 20 p.
29. Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors, IEEE Trans. Pattern Analysis and Machine Intelligence, 27, 10, 1615-1630.
https://doi.org/10.1109/TPAMI.2005.188
30. Nicolls, F. (2004, November). Structure and motion from SEM: a case study. In Fifteenth Annual Symposium of the Pattern Recognition Association of South Africa (p. 19).
31. Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE transactions on systems, man, and cybernetics, 9(1), 62-66.
https://doi.org/10.1109/TSMC.1979.4310076
32. Popielski, P., & Wróbel, Z. (2012). The feature detection on the homogeneous surfaces with projected pattern. In Information Technologies in Biomedicine (pp. 118-128). Springer, Berlin, Heidelberg.
https://doi.org/10.1007/978-3-642-31196-3_13
33. Salahat, E., & Qasaimeh, M. (2017, March). Recent advances in features extraction and description algorithms: A comprehensive survey. In 2017 IEEE international conference on industrial technology (ICIT) (pp. 1059-1063). IEEE.
https://doi.org/10.1109/ICIT.2017.7915508
34. Shostak, A.V. (2012). Methods and models of microphotogrammetry in applied scientific research: abstract. diss. for the sciences degree doct. tech. sciences, Kyiv. 28 p. (in Ukrainian).
35. Suprun, D. E. (2016). Algorithm for matching images by key points for scalability and rotation of objects. Vestnik MGTU im. N.E. Bauman. Ser. Instrument making, 5, 86-98. (in Russian).
https://doi.org/10.18698/0236-3933-2016-5-86-98
36. Tafti, A. P. (2016). 3D SEM surface reconstruction: An optimized, adaptive, and intelligent approach. PhD thesis, 1-140.
37. Vizilter, Ju. V., Zheltov, S. J., Bondarenko, A. V., Ososkov, M. V., & Morzhyn, A. V. (2010). Image processing and analysis in computer vision problems. Course of lectures and practical classes. Moscow: Phizmatkniga, 672 p. (in Russian).
38. Voloshyn, V. U. (2004). Development of methods of SEM-photogrammetry and morphologic-fractal analysis (on example of bone fabric researching): author's abstract. dis. for the sciences degree candidate techn. sciences], Lviv, 21 p. (in Ukrainian).
39. Zhu, T., Sutton, M. A., Li, N., Orteu, J. J., Cornille, N., Li, X., & Reynolds, A. P. (2011). Quantitative stereovision in a scanning electron microscope. Experimental Mechanics, 51(1), 97-109.
https://doi.org/10.1007/s11340-010-9378-7