1. SISN
  2. All Volumes and Issues
  3. Volume 15, 2024
  4. Virtual Online Garment Fitting Using Augmented Reality

Virtual Online Garment Fitting Using Augmented Reality

SISN.
2024;
Volume 15
: pp. 184 - 199
https://doi.org/10.23939/sisn2024.15.184
Authors:
  1. Mykola Lehkyi
  2. Liubov Zhuravchak
1
Lviv Polytechnic National University, Software Engineering Department
2
Lviv Polytechnic National University, Software Engineering Department

In recent years, the number of accessories and headwear purchased on the Internet has been increasing, and at the same time, the percentage of product returns has not decreased. One approach to solving this problem is virtual fitting rooms. Accessibility to the online fitting system for accessories and headwear and the quality of fitting are important criteria for users. Existing systems for online fitting have shortcomings with occlusions, reflection of lighting and shadows, and the accuracy of reproduction of goods on a person in relation to the environment. The article is devoted to solving the problem of online fitting of accessories and headgear to a person with the appearance of shadows and lighting on a 3D model due to the use of neural networks. A method is proposed that simulates high-quality human stocking with a high number of frames per second, the ability to play from any device with a web browser, and low CPU and GPU requirements. The algorithm prototype has advantages compared to 2D counterparts: 3D lighting, dynamic change of its brightness, shadows, virtual environment and reduced occlusions. Created a virtual online fitting using augmented reality – MLight-VTON. It is noted that based on the proposed method and the Three.js library, trained TensorFlow.js models can be added to further improve tissue deformation and body segmentation.

augmented reality
accessories
virtual fitting
fitting room
hats
  1. Adikari, S. B., Ganegoda, N. C., Meegama, R. G. N., & Wanniarachchi, I. L. (2020). Applicability of a Single Depth Sensor in Real-Time 3D Clothes Simulation: Augmented Reality Virtual Dressing Room Using Kinect Sensor. Advances in Human-Computer Interaction, 2020, e1314598. https://doi.org/10.1155/2020/1314598.
  2. Apparel return rates: The stats retailers CANNOT ignore in 2023 (2022, December 22). 3DLOOK. https://3dlook.me/content-hub/apparel-return-rates-the-stats-retailers-c....
  3. Bai, S., Zhou, H., Li, Z., Zhou, C., & Yang, H. (2022). Single Stage Virtual Try-On Via Deformable Attention Flows. In Avidan S., Brostow G., Cissé M., Farinella G.M., & Hassner T. (Eds.), Lect. Notes Comput. Sci.: Vol. 13675 LNCS (pp. 409–425). Springer Science and Business Media Deutschland GmbH; Scopus. https://doi.org/10.1007/978-3-031-19784-0_24.
  4. Battistoni, P., Di Gregorio, M., Romano, M., Sebillo, M., Vitiello, G., & Brancaccio, A. (2022). Interaction Design Patterns for Augmented Reality Fitting Rooms. Sensors, 22(3), 982. https://doi.org/10.3390/s22030982.
  5. Enisa. (2022, January 18). Using AR technology to lower your ecommerce return rate. Loop Returns. https://www.loopreturns.com/blog/using-ar-technology-to-lower-your-ecommerce-return-rate.
  6. Gavaskar, S. (2022). A Novel Approach of Virtual Visualization of Cloth Fitting. International Journal of Engineering Research in Computer Science and Engineering, 9(8), 97-103. https://doi.org/10.36647/ijercse/09.08.art001
  7. Ge, C., Song, Y., Ge, Y., Yang, H., Liu, W., & Luo, P. (2021). Disentangled Cycle Consistency for Highly-realistic Virtual Try-On. Proc IEEE Comput Soc Conf Comput Vision Pattern Recognit, 16923–16932. Scopus.    https://doi.org/10.1109/CVPR46437.2021.01665.
  8. Hashmi, M. F., Ashish, B. K. K., Keskar, A. G., Bokde, N. D., & Geem, Z. W. (2020). FashionFit: Analysis of Mapping 3D Pose and Neural Body Fit for Custom Virtual Try-On. IEEE Access, 8, 91603–91615. https://doi.org/10.1109/ACCESS.2020.2993574.
  9. Hu, P., Kaashki, N., Dadarlat, V., & Munteanu, A. (2020). Learning to Estimate the Body Shape Under Clothing From a Single 3-D Scan. IEEE Transactions on Industrial Informatics, PP, 1–1. https://doi.org/10.1109/TII.2020.3016591.
  10. Jandial, S., Chopra, A., Ayush, K., Hemani, M., Kumar, A., & Krishnamurthy, B. (2020). SieveNet: A unified framework for robust image-based virtual try-on. Proc. – IEEE Winter Conf. Appl. Comput. Vis., WACV, 2171–2179. Scopus. https://doi.org/10.1109/WACV45572.2020.9093458.
  11. Learn JavaScript Online – Courses for Beginners – Javascript.com. (n. d.). Retrieved December 13, 2023, https://www.javascript.com.
  12. Lee, H.-J., Koo, B., Ahn, H.-E., Kang, M., Lee, R., & Park, G. (2021). Full body virtual try-on with semi-self-supervised learning. Electronics Letters, 57(24), 915–917. Scopus. https://doi.org/10.1049/ell2.12307.
  13. Liang,  J.,  &  Lin,  M.  C.  (2021).  Machine  learning  for  digital  try-on:  Challenges  and  progress.Computational Visual Media, 7(2), 159–167. https://doi.org/10.1007/s41095-020-0189-1.
  14. Marelli, D., Bianco, S., & Ciocca, G. (2022). Designing an AI-Based Virtual Try-On Web Application.Sensors, 22(10), Article 10. https://doi.org/10.3390/s22103832.
  15. Minar, M. R., Tuan, T. T., & Ahn, H. (2021). CloTH-VTON+: Clothing Three-Dimensional Reconstruction for Hybrid Image-Based Virtual Try-ON. IEEE Access, 9, 30960–30978. https://doi.org/10.1109/ACCESS.2021.3059701.
  16. MindAR / mind-ar-js. (n. d.). Retrieved December 13, 2023, https://hiukim.github.io/mind-ar-js-doc.
  17. Most returned online purchases by category in the U.S. 2022. (n. d.). Statista. Retrieved May 13, 2023, https://www.statista.com/forecasts/997235/most-returned-online-purchases-by-category-in-the-us.
  18. Oikonomou, K., Chatzilari, E., Nikolopoulos, S., Kompatsiaris, I., Gavilan, D., & Downing, J. (2021). Snapwear: A Snapchat AR filter for the virtual tryon of real clothes. Proc. – IEEE Int. Symp. Mixed Augment. Real. Adjun., ISMAR-Adjunct, 46–51. Scopus. https://doi.org/10.1109/ISMAR-Adjunct54149.2021.00019.
  19. Prakash, D., J., J. V., Ghosh, S., B., S. N., Deborah, S. A., & Chandran, K. R. S. (2020). Virtual Fashion Mirror. 2020 4th International Conference on Computer, Communication and Signal Processing (ICCCSP), 1–4. https://doi.org/10.1109/ICCCSP49186.2020.9315257.
  20. Rajan, D. S. P., & Al, E. (2021). Virtual Dressing Room with Web Deployment. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(7), Article 7. https://doi.org/10.17762/turcomat.v12i7.3640.
  21. Igor, Santesteban., Miguel, A., Otaduy., Dan, Casas (2019). Learning-based animation of clothing for virtual try-on. Computer Graphics Forum, 38(2):355–366. DOI: 10.1111/CGF.13643.
  22. Shirsat,  A.,  Sonimindia,  S.,  Patil,  S.,  Kotecha,  N.,  &  Koparde,  P.  (2019).  Virtual  Trial  Room.International Journal of Research in Advent Technology, 7, 182–185. https://doi.org/10.32622/ijrat.75201976.
  23. Smink, A. R., Frowijn, S., van Reijmersdal, E. A., van Noort, G., & Neijens, P. C. (2019). Try online before you buy: How does shopping with augmented reality affect brand responses and personal data disclosure. Electronic Commerce Research and Applications, 35, 100854. https://doi.org/10.1016/j.elerap.2019.100854.
  24. TensorFlow.js / Machine Learning for JavaScript Developers (n. d.). TensorFlow. Retrieved December 13, 2023, https://www.tensorflow.org/js.
  25. Three.js – JavaScript 3D Library (n. d.). Retrieved February 29, 2024, https://threejs.org.
  26. Turner,  C.  (2022).  Augmented  Reality,  Augmented  Epistemology,  and  the  Real-World  Web.Philosophy & Technology, 35(1), 19. https://doi.org/10.1007/s13347-022-00496-5.
  27. Vakaliuk, T. A., & Pochtoviuk, S. I. (n. d.) (2021). Analysis of tools for the development of augmented reality technologies. AREdu 2021: 4th International Workshop on Augmented Reality in Education, 2898, 119–130. https://ceur-ws.org/Vol-2898/paper06.pdf
  28. Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., &  Yang, M. (2018). Toward characteristic- preserving image-based virtual try-on network. In Ferrari V., Sminchisescu C., Weiss Y., & Hebert M. (Eds.), Lect. Notes Comput. Sci.: Vol. 11217 LNCS, 607–623. Springer Verlag; Scopus. https://doi.org/10.1007/978-3-030- 01261-8_36.
  29. Wang, T., Gu, X., & Zhu, J. (2022). A Flow-Based Generative Network for Photo-Realistic Virtual Try- on. IEEE Access, 10, 40899–40909. https://doi.org/10.1109/ACCESS.2022.3167509.
  30. Yang, Z., Chen, J., Shi, Y., Li, H., Chen, T., & Lin, L. (2023). OccluMix: Towards De-Occlusion Virtual Try-On by Semantically-Guided Mixup. IEEE Transactions on Multimedia, 1–12. Scopus. https://doi.org/10.1109/TMM.2023.3234399.