The process of enhancing the video’s quality by removing unwanted effects of camera shakes and jitters is called Video Stabilization (VS). However, the 3-Dimensional (3D) rainy stereoscopic video stabilization process was not concentrated on any of the prevailing research work. Therefore, in this framework, an effective 3D rainy stereoscopic video stabilization with depth estimation and Shape Autotuning Liebovitch map Cheetah Chase Algorithm with Convolution Neural Network (SA- LmCCA-CNN) is proposed. Primarily, the input videos are converted into a number of frames. After that, by using Pairnorm L0 Gradient Minimization (Pn-LGM), the raindrops in each frame are removed. Later, the overlapping region and depth estimation are processed, and by using the Liebovitch map Cheetah Chase Algorithm (LmCCA), the energy function is diminished. Likewise, to mitigate the hallucination issue, a mesh is generated by utilizing Alternating Least Squares-Locally Constrained Representations (ALS-LCR). Then, from the hallucination-mitigated image and energy function minimized image, the feature points are extracted. Later, by employing SA-LmCCA-CNN, the stable and unstable frames are classified. If the frame is unstable, then the frame undergoes motion and camera path corrections, followed by raindrop reconstruction; otherwise, raindrop reconstruction is done directly for a stable frame. Lastly, in order to get the stabilized video, the frames are synthesized. The experimental analysis proved the proposed model’s robustness in 3D rainy stereoscopic video stabilization by attaining a stability score of 0.93.

[1]    Al Mokhtar Z. and Dawwd S., “3D VAE Video Prediction Model with Kullback Leibler Loss Enhancement,” International Arab Journal of Information Technology, vol. 21, no. 5, pp. 879-888, 2024. https://doi.org/10.34028/iajit/21/5/9

[2]    Bellavia F., Fanfani M., Colombo C., and Piva A., “Experiencing with Electronic Image Stabilization and PRNU Through Scene Content Image Registration,” Pattern Recognition Letters, vol. 145, pp. 8-15, 2021. https://doi.org/10.1016/j.patrec.2021.01.014

[3]    Chen H., Li H., Liu G., and Wang Z., “A Novel Simulation Method for 3D Digital-Image Correlation: Combining Virtual Stereo Vision and Image Super-Resolution Reconstruction,” Sensors, vol. 24, no. 13, pp. 1-26, 2024. https://doi.org/10.3390/s24134031 

[4]    Chen Y., Tseng, K., Lee Y., Chen, C., and Hung, Y., “PixStabNet: Fast Multi-Scale Deep Online Video Stabilization with Pixel-Based Warping,” in Proceedings of the International Conference on Image Processing, Anchorage, pp. 1929-1933, 2021. https://doi.org/10.1109/ICIP42928.2021.9506801

[5]    Guilluy W., Oudre L., and Beghdadi A. “Video Stabilization: Overview, Challenges and Perspectives,” Signal Processing: Image Communication, vol. 90, pp. 1-40, 2021. https://doi.org/10.1016/j.image.2020.116015

[6]    Huang Q., Liu J., Jiang C., and Wang Y., “DMCVS: Decomposed Motion Compensation-Based Video Stabilization,” IET Image Processing, vol. 18, no, 6, pp. 1422-1433, 2024. https://doi.org/10.1049/ipr2.13035

[7]    Ito M. and Izquierdo E., “Deep Homography-Based Video Stabilization,” in Proceedings of the 25th International Conference on Pattern Recognition, Milan, pp. 8819-8825, 2021. https://doi.org/10.1109/ICPR48806.2021.9413034 

[8]    Jang J., Ban Y., and Lee K., “Applied Sciences Dual-Modality Cross-Interaction-Based Hybrid Full-Frame Video Stabilization,” Applied Sciences, vol. 14, no. 10, pp. 1-16, 2024. https://www.mdpi.com/2076-3417/14/10/4290# 

[9]    Keyang C., Shichao L., Lan R., Wenshan W., and et al., “Video Stabilization via Prediction with Time-Series Network and Image Inpainting with Pyramid Fusion,” Chinese Journal of Electronics, vol. 30, no. 6, pp. 1103-1110, 2021. https://doi.org/10.1049/cje.2021.08.006

[10]    Lee Y., “Low Memory Access Video Stabilization for Low-Cost Camera SoC,” Sensors, vol. 22, no. 6, pp. 1-18, 2022. https://doi.org/10.3390/s22062341

[11]    Lemaire P., Crispim-Junior C., Robinault L., and Tougne L., “Registering Unmanned Aerial Vehicle Videos in the Long Term,” Sensors, vol. 21, no. 2, pp. 1-19, 2021. https://doi.org/10.3390/s21020513

[12]    Lin S., Le T., Wu P., and Lee T., “Content-and-Disparity-Aware Stereoscopic Video Stabilization,” Multimedia Tools and Applications, vol. 80, no. 1, pp. 1545-1564, 2020. https://doi.org/10.1007/s11042-020-09767-9 

[13]    Luan Y., Han C., and Wang B., “An Unsupervised Video Stabilization Algorithm Based on Key Point Detection,” Entropy, vol. 24, no. 10, pp. 1-15, 2022. https://doi.org/10.3390/e24101326 

[14]    Luchetti A., Zanetti M., Kalkofen D., and De Cecco M., “Stabilization of Spherical Videos Based on Feature Uncertainty,” Visual Computer, vol. 39, no. 9, pp. 4103-4116, 2023. https://doi.org/10.1007/s00371-022-02578-z 

[15]    Milanovic P., Popadic, I., and Kovacevic B., “Gyroscope-Based Video Stabilization for Electro-Optical Long-Range Surveillance Systems,” Sensors, vol. 21, no. 18, pp. 1-23, 2021. https://doi.org/10.3390/s21186219

[16]    Morichon A., Dannhoff G., Barantin L., Destrieux C., and Maldonado I., “Doing More with Less: Realistic Stereoscopic Three‐Dimensional Anatomical Modeling from Smartphone Photogrammetry,” Anatomical Sciences Education, vol. 17, no. 4, pp. 864-877, 2024. https://doi.org/10.1002/ase.2402 

[17]    Morikawa C., Kobayashi M., Satoh M., Kuroda Y., and et al., “Image and Video Processing on Mobile Devices: A Survey,” Visual Computer, vol. 37, no, 12, pp. 2931-2949, 2021. https://doi.org/10.1007/s00371-021-02200-8 

[18]    Nooralishahi P., Lopez F., and Maldague X., “A Drone-Enabled Approach for Gas Leak Detection Using Optical Flow Analysis,” Applied Science, vol. 11, no. 4, pp. 1-19, 2021. https://doi.org/10.3390/app11041412  

[19]    Oliveira A., Leonel L., LaHood E., Nguyen T., and et al., “Projection of Realistic Three‐Dimensional Photogrammetry Models Using Stereoscopic Display: A Technical Note,” Anatomical Sciences Education, vol. 17, no. 1, pp. 39-46, 2024. https://doi.org/10.1002/ase.2329 

[20]    Pant S., Nooralishahi P., Avdelidis N., Genest M., and et al., “UAV-Based Active Infrared Thermography Application,” Sensors, vol. 21, no. 5, pp. 1-16, 2021. https://www.mdpi.com/1424-8220/21/5/1604 

[21]    Pathak S., Hamada T., and Umeda K., “Trinocular 360-Degree Stereo for Accurate all-Round 3D Reconstruction Considering Uncertainty,” Advanced Robotics, vol. 38, no. 15, pp. 1038-1051, 2024.https://doi.org/10.1080/01691864.2024.2376022

[22]    Reichert G., Pieras M., Marroquim R., and Vilanova A., “Stabilization and Visual Analysis of Video-Recorded Sailing Sessions,” Visual Computing for Industry, Biomedicine, and Art, vol. 4, no. 26, pp. 1-10, 2021. https://doi.org/10.1186/s42492-021-00093-x 

[23]    Ren Z., Fang M., and Chen C., “Self-Calibration Spherical Video Stabilization Based on Gyroscope,” Information, vol. 12, no. 8, pp. 1-13, 2021. https://doi.org/10.3390/info12080299 

[24]    Shang Z. and Chu Z., “Video Stabilization Based on Low-Rank Constraint and Trajectory Optimization,” IET Image Processing, vol. 18, no. 7, pp. 1768-1779, 2024. https://doi.org/10.1049/ipr2.13062  

[25]    Souza M. and Pedrini H., “Visual Rhythms for qualitative evaluation of video stabilization,” Eurasip Journal on Image and Video Processing, vol. 2020, no. 19, pp. 1-19, 2020. https://doi.org/10.1186/s13640-020-00508-4

[26]    Taormina V., Raso G., Gentile V., Abbene L., and et al., “Segmentation in Videocapillaroscopy,” Sensors, vol. 23, no. 18, pp. 1-23, 2023. https://doi.org/10.3390/s23187674 

[27]    Valero M., Verstockt S., Butler B., Jimenez D., and et al., “Thermal Infrared Video Stabilization for Aerial Monitoring of Active Wildfires,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 2817-2832, 2021. https://doi.org/10.1109/JSTARS.2021.3059054 

[28]    Verma K., Ghosh D., and Kumar A., “Visual Tracking in Unstabilized Real Time Videos Using SURF,” Journal of Ambient Intelligence and Humanized Computing, vol. 15, no. 1, pp. 809-827, 2024. https://doi.org/10.1007/s12652-019-01249-7 

[29]    Wu H., Xiao L., and Wei Z., “Simultaneous Video Stabilization and Rolling Shutter Removal,” IEEE Transactions on Image Processing, vol. 30, pp. 4637-4652, 2021. https://doi.org/10.1109/TIP.2021.3073865 

[30]    Wu R., Xu Z., Zhang J., and Zhang L., “Robust Global Motion Estimation for Video Stabilization Based on Improved K-Means Clustering and Superpixel,” Sensors, vol. 21, no. 7, pp. 1-17, 2021. https://doi.org/10.3390/s21072505

[31]    Xin Z., Kong S., Wu Y., Zhan G., and Yu J., “A Hierarchical Stabilization Control Method for a Three-Axis Gimbal Based on Sea-Sky-Line Detection,” Sensors, vol. 22, no. 7, pp. 1-14, 2022. https://doi.org/10.3390/s22072587

[32]    Xu Y., Zhang J., Maybank S., and Tao D., “DUT: Learning Video Stabilization by Simply Watching Unstable Videos,” IEEE Transactions on Image Processing, vol. 31, no. 8, pp. 4306-4320, 2022. https://doi.org/10.1109/TIP.2022.3182887

[33]    Yagoubi M., Amirshahi S., Moan S., Beghdadi A., and Rodner E., “Blind Video Stabilization Quality Assessment Based on Convolutional LSTM,” in Proceedings of the 11th European Workshop on Visual Information Processing, Gjovik, pp. 1-6, 2023.  https://doi.org/10.1109/EUVIP58404.2023.10323043

[34]    Zhang F., Li X., Wang T., Zhang G., and et al., “High-Precision Satellite Video Stabilization Method Based on ED-RANSAC Operator,” Remote Sensing, vol. 15, no. 12, pp. 1-17, 2023. https://doi.org/10.3390/rs15123036 

[35]    Zhao M. and Ling Q., “Adaptively Meshed Video Stabilization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 9, pp. 3504-3517, 2021. https://doi.org/10.1109/TCSVT.2020.3040753