My main research interests lies in human motion capture, with an inclination to multi-modal sensing.
Human motion capture is an exciting research topic given the complex nature of the human body and the large number of high-impact applications in medicine, sports science, biology, and media production.
Multi-modal approaches have often gained my favour because they allow to exploit the complementarity of different sensing modalities to achieve more robust results. This is actually a very natural approach, as for example the human sense of balance is based on a combination of visual (eyes), inertial (vestibular system), and pressure/force (proprioception) cues.
I am currently an "Inria Starting Researcher" at Morpheo.
I have started an exciting new research topic within Morpheo, on the joint usage of video cameras and X-ray imaging to build accurate human motion models. The complementarity of the two imaging modalities opens novel research directions at the boundary between motion capture and medical imaging. Initial work has already led to some promising results   .
Computed Tomography (CT) scanners provide accurate 3D X-ray attenuation models of living organisms, allowing for a range of studies and diagnostics. However, when compared to regular planar radiography, CT scanners are significantly more expensive, expose the patient to higher radiation doses, and require immobility.
In order to tackle these issues, we proposed to combine a regular X-ray imaging device with a set of cameras   . A software suite which estimates a dense 3D attenuation model of a moving sample based on a short capture by a single X-ray imaging device combined with 10 video cameras has been deployed on the novel KINOVIS platform recently installed at the Grenoble University Hospital (CHU).
Dynamic scenes captured simultaneously from multiple cameras can be rendered with the traditional garphics pipeline, i.e. using a single baked texture mapped over a reconstructed mesh. However, significantly more anisotropic details can be rendered by using all originally captured video streams. For this purpose, the original images are fused at run-time based on the currently required viewpoint. In the context of the FP7 RE@CT project (capture, modelling, and 4D rendering) with the BBC, I have developped a real-time, view-dependent rendering engine from multiple views to replace previous software. The module has been implemented as both a standalone and an OpenSceneGraph library plugin, and defines a file format for multi-view 4D sequences. It also allowed to replace tracked proxy-props (such as fake swords) by virtual models.
I have also worked on WebGL-based applications , , as well as real-time free-viewpoint rendering from multiple views .
In order to allow for live 2D and 3D effects on moving and zooming sports broadcast cameras, an automated and real-time pose and focal length estimation system was developed. For this purpose, the software used natural features such as pitch lines and the calibration core engine described hereafter. The software comprises the following three modules: sports pitch lines detection and labeling, real-time camera pose estimation engine, and GUI for parameter settings and feedback.
In order to avoid the use of large and cumbersome checker calibration charts for multiple camera calibration in large working volumes, a new system based on a lightweight LED wand was designed, developed, and deployed at the BBC. This system was part of a multiple-view capture, 3D reconstruction, and rendering pipeline. It comprises a 12-LED wireless calibration wand, a real-time LED detection and labeling software, and the actual camera parameter estimation. Furthermore, a plugin for the capture module has been developed, which provides real-time feedback during acquisition. I have also worked specifically on the calibration of rolling shutter CMOS cameras .
A light-weight sensor composed of an accelerometer and gyroscope was mounted in the wheel of a wheelchair for motion tracking in real-time with wireless feedback to the sports coach. Based on a physical model of the wheelchair intrinsic and global motion, the fusion of several sensors readings with a Kalman filter provides a robust motion estimation, with limited drift over time. The final solution enabled accurate displacement estimation at a limited cost, weight, and installation complexity . The system was trialled by the British Paralympic Basketball team.
I have also worked on a number of other sports such as swimming , rowing , running , climbing , and speed-skating.
While at Imperial College London, I have participated in the development of an autonomous, real-time, wireless tennis player tracker using computer vision  in collaboration with the Lawn Tennis Association (LTA). The hardware was based on a custom-built ARM-like architecture encompassing a camera and a wireless link. I worked on the firmware for autonomous tennis player tracking. It consisted of the following modules: tennis court line detection, camera calibration, background segmentation, estimation of the player on court, and micro web server for direct access.
I have worked on an immersive visualization and interaction of multidimensional archaeological data - Digital Archaeological Virtual Environment (DAVE). The aim was to help archaeologists to classify various kinds of data (text, 3D objects, sketches, images, video, ...) by rendering them onto a 3D terrain model with multiple overlays (aerial photography, maps, ...) .
Various options were included to visualise the scene under different perspectives (eg. rendering an archaeological findings density map, extrapolating the crop marks as walls, changing the sea level, ...). The application rendered on a large back-projected wall screen (6x3m), with a wireless joystick and a tracked glove to enable interaction within the scene.
I am a keen marathon and ultramarathon runner (life-is-an-ultramarathon.org) and also enjoy hiking, rock climbing, paragliding, homebrewing, and photography.
I am particularly involved in the Hardmoors race series in North Yorkshire. Aside from running it, I volunteered as marshal and race director and I have been the hardmoors110.org.uk webmaster for several years.