University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors

Trumble, Matthew, Gilbert, Andrew, Malleson, Charles, Hilton, Adrian and Collomosse, John (2017) Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors In: 28th British Machine Vision Conference, 4 - 7 September 2017, London, UK.

Total Capture 3D Human Pose Estimation Fusing Video and Inertial Sensors.pdf - Version of Record

Download (7MB) | Preview


We present an algorithm for fusing multi-viewpoint video (MVV) with inertial measurement unit (IMU) sensor data to accurately estimate 3D human pose. A 3-D convolutional neural network is used to learn a pose embedding from volumetric probabilistic visual hull data (PVH) derived from the MVV frames. We incorporate this model within a dual stream network integrating pose embeddings derived from MVV and a forward kinematic solve of the IMU data. A temporal model (LSTM) is incorporated within both streams prior to their fusion. Hybrid pose inference using these two complementary data sources is shown to resolve ambiguities within each sensor modality, yielding improved accuracy over prior methods. A further contribution of this work is a new hybrid MVV dataset (TotalCapture) comprising video, IMU and a skeletal joint ground truth derived from a commercial motion capture system. The dataset is available online at

Item Type: Conference or Workshop Item (Conference Paper)
Divisions : Faculty of Engineering and Physical Sciences > Electronic Engineering
Authors :
Date : 7 September 2017
Funders : Engineering and Physical Sciences Research Council (EPSRC)
Grant Title : Total Capture project
Copyright Disclaimer : Copyright 2017. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.
Related URLs :
Depositing User : Clive Harris
Date Deposited : 25 Jul 2017 08:42
Last Modified : 11 Dec 2018 11:23

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800