The paper considers problems for the tasks of improving the quality of digital video images for cloud environments, as well as on the client side using generative adversarial neural networks (GANs) adapted for work in the browser. A method is proposed that uses WebGPU for accelerated execution of convolutional calculations, which allows to increase the resolution and improve the quality of low-quality video in real time without significant load on servers. Optimization of the neural network includes the use of Pruning and Knowledge Distillation methods, which made it possible to reduce the size of the model by 40–60% without significant loss of quality.
The results of the experiments showed that the implementation of the proposed method increases the performance of video processing in the browser by 2–4 times compared to models based on the WebGL interface. The video quality assessment showed an improvement in PSNR and an increase in SSIM compared to traditional methods of increasing resolution. The proposed approach can be integrated into streaming services and web applications, which will reduce the load on computer networks and provide a better user experience with lower costs for cloud and server computing.
- Madarasingha C., Muramudalige S., Jourjon G. VideoTrain++: GAN-based Adaptive Framework for Synthetic Video Traffic Generation. Computer Networks. 2022. DOI: 10.1016/j.comnet.2022.108785.
- Lee R., Venieris S. I., Lane N. D. Deep neural network–based enhancement for image and video streaming systems: A survey and future directions. ACM Computing Surveys. 2021. DOI: 10.1145/3469094.
- Liborio J. M., Melo C., Silva M. Internet Video Delivery Improved by Super-Resolution with GAN. MDPI Future Internet. 2022. DOI: 10.3390/fi14120364.
- Shi W., Li Q., Yu Q., Wang F., Shen G. A Survey on Intelligent Solutions for Increased Video Delivery Quality in Cloud-Edge-End Networks. IEEE Surveys & Tutorials. 2024. DOI: 10.1109/COMST.2024.3427360.
- WebGPU Working Group, “WebGPU Specification”, W3C Technical Report, 2023. URL: https://www.w3.org/TR/WGSL/.
- TensorFlow.js Team, “ TensorFlow.js: Machine Learning for the Web and Beyond “. Proceedings of Machine Learning and Systems, 2019. DOI: https://doi.org/10.48550/arXiv.1901.05350.
- He Z., Yang Y., Qiu L., Park K. Real-Time Neural Video Recovery and Enhancement on Mobile Devices. DOI: 10.48550/arXiv.2307.12152.
- Microsoft ONNX Team. URL: https://github.com/microsoft/onnxjs.
- Maksymiv M., Rak T. Method of Video Quality-Improving. Artificial Intelligence. 2023. 28(3):47-62. DOI: 10.15407/jai2023.03.047.
- Hinton G., Vinyals O., Dean J. Distilling the Knowledge in a Neural Network (arXiv preprint arXiv:1503.02531), 2015. DOI: 10.48550/arXiv.1503.02531.
- Mishra R., Gupta H. P. Designing and training of lightweight neural networks on edge devices using early halting in knowledge distillation. IEEE Transactions on Mobile Computing. 2023. DOI: 10.1109/TMC.2023.10187678.
- Zhang L., Bao C., Ma K. Self-distillation: Towards efficient and compact neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021. DOI: 10.1109/TPAMI.2021.9381661.
- Alam S. A., Anderson A., Barabasz B., Gregg D. Winograd Convolution for Deep Neural Networks: Efficient Point Selection. ACM Transactions on Embedded Computing Systems. 2022. DOI: 10.48550/ arXiv.2201.10369.