SSP-Net: Scalable sequential pyramid networks for real-Time 3D human pose regression

Diogo Carbonera Luvizon (1) , Hedi Tabia (1, 2) , David Picard (1, 3)

1 MPI Informatics
2 IBISC - Informatique, BioInformatique, Systèmes Complexes
3 LIGM - Laboratoire d'Informatique Gaspard-Monge

In this paper we propose a highly scalable convolutional neural networks, end-to-end trainable, for real-time 3D human pose regression from still RGB images. We call this approach Scalable Sequential Pyramid Networks (SSP-Net) as it is trained with refined supervision at multiple scales in a sequential manner. Our network requires a single training procedure and is capable of producing its best predictions at 120 frames per second (FPS), or acceptable predictions at more than 200 FPS when cut at test time. We show that the proposed regression approach is invariant to the size of feature maps, allowing our method to perform multi-resolution intermediate supervisions and reaching results comparable to the state-of-the-art with very low resolution feature maps. We demonstrate the accuracy and the effectiveness of our method by providing extensive experiments on two of the most important publicly available datasets for 3D pose estimation, Human3.6M and MPI-INF-3DHP. Additionally, we provide relevant insights about our decisions on the network architecture and show its flexibility to meet the best precision-speed compromise.

Keywords: 3D Human pose estimation; Computer vision; Neural nets

Domains: Computer Vision and Pattern Recognition [cs.CV]; Artificial Intelligence [cs.AI]

Diogo Carbonera Luvizon, Hedi Tabia, David Picard. SSP-Net: Scalable sequential pyramid networks for real-Time 3D human pose regression. Pattern Recognition, 2023, 142, pp.109714. ⟨10.1016/j.patcog.2023.109714⟩. ⟨hal-04124371⟩