Julien Pansiot

Julien Pansiot

Starting Researcher @ Inria Grenoble
Human motion capture and medical imaging

Research interests

motion capture Since the start of my PhD at Imperial College London, I have been focusing my research actvities on human motion capture, with novel applications to both the medical and sports science domains.
I have worked on a wide range of techniques (markerless computer vision, wearable inertial sensors) for a multitude of applications (home healthcare monitoring, sport coaching tools). I am now solely focusing on computer vision-based techniques, which I attempt to enhance with X-ray imaging.

Research Experience

Inria Grenoble - Rhone-Alpes, France

Starting Researcher | Morpheo Team | since 2013

I am currently working with the Morpheo team on markerless human motion tracking [6] using anatomically-correct skeleton models. I have started by investigating the joint usage of video cameras and X-ray imaging to build accurate human motion models. The complementarity of the two imaging modalities opens novel exciting research directions at the boundary between motion capture and medical imaging. Initial work has already led to some promising results [3] [4].
Research funding:

  • I am currently funded by an Inria Starting Research Position (6 positions/year nationally).
  • I am involved in the PERSYVAL LabEx exploratory project Carambole, aiming at tracking 4D deformation of growing plantes. Co-author, collaboration with MSC/Paris 7. Budget: 9k€.
  • I have recently received an ANR Collaborative Research Project with Industrials (PRCE): ``CaMoPi'' Capture and Modelling of the Shod Foot in Motion. Principal Investigator, collaboration with Mines St Etienne / CIS, CTC, Sporaltec. Budget: 541k€ (success rate: 12.5%). More details to come.

British Broadcasting Corporation (BBC) R&D, UK

Technologist | Production Magic Team | 2011-2013

I have worked on the European FP7 RE@CT project (capture, modelling, and 4D rendering), with interests on related subjects such as real-time camera calibration from natural images (sports pitch line detection, lines-points matching), calibration of rolling shutter CMOS cameras [7] (variations on the bundle adjustment method), real-time free-viewpoint rendering from multiple views [8], and motion blur reduction (adaptive deconvolution kernel). I have also worked on WebGL-based applications [5], [9].

Imperial College London, UK

PhD candidate & research associate | Visual Information Processing group | 2005-2011

During my PhD and post-doctorate at Imperial College London's Hamlyn Centre, I have worked on various aspects of human motion capture using cameras and wearable inertial sensors. Thus I have worked mainly within two national collaborative projects: SAPHE and ESPRIT.
The SAPHE project aimed at elderly and/or recovering patients at home. My main contributions were to develop methods and algorithms to process data on the camera device itself in order to limit the risks of privacy breach [23], as well as the fusion of information coming from both cameras and wearable sensors [21].
The ESPRIT project proposed to use similar capture and processing methods to monitor elite athletes. My main scientific contributions were to implement real-time autonomous player tracking system on an embedded, mobile, and wireless system [13] as well as a light-weight inertial wheelchair tracking system [1]. I have also worked on a number of other sports such as swimming [12], rowing [15], running [16], climbing [18], and speed-skating.

The University of Hull, UK

MSc by Research | SimVis group | 2003-2004

I have spent one year developping new tools for immersive visualisation and interaction (stereo display coupled with motion capture) of various types of archeological data (pictures, text, crop marks, ...) overlaid on 3D maps with multiple textures [25].

Selected Projects

Combined Video and X-Ray 3D Imaging

Inria Grenoble - Rhone-Alpes, 2013-2014

KINVOVIS platformComputed Tomography (CT) scanners provide accurate 3D X-ray attenuation models of living organisms, allowing for a range of studies and diagnostics. However, when compared to regular planar radiography, CT scanners are significantly more expensive, expose the patient to higher radiation doses, and require immobility.
In order to tackle these issues, we proposed to combine a regular X-ray imaging device with a set of cameras [3] [4]. A software suite which estimates a dense 3D attenuation model of a moving sample based on a short capture by a single X-ray imaging device combined with 10 video cameras has been deployed on the novel KINOVIS platform recently installed at the Grenoble University Hospital (CHU).


  • Composite input: 8 videos and 1 x-ray motion sequence

    Composite input: 8 videos and 1 x-ray motion sequence

  • Reconstruction results: 3D volumetric and soft tissues iso-surface

    Reconstruction results: 3D volumetric (left) and soft tissues iso-surface (right)

  • Approach overview

    Approach overview

Free-viewpoint Video Rendering

BBC R&D, 2012-2013

Dynamic scenes captured simultaneously from multiple cameras can be rendered with the traditional garphics pipeline, i.e. using a single baked texture mapped over a reconstructed mesh. However, significantly more anisotropic details can be rendered by using all originally captured video streams. For this purpose, the original images are fused at run-time based on the currently required viewpoint. In the context of the RE@CT project with the BBC, I have developped a real-time, view-dependent rendering engine from multiple views to replace previous software. The module has been implemented as both a standalone and an OpenSceneGraph library plugin, and defines a file format for multi-view 4D sequences. It also allowed to replace tracked proxy-props (such as fake swords) by virtual models.


Real-time sports pitch camera pose estimation system

BBC R&D, 2012-2013

pitch calibration GUI In order to allow for live 2D and 3D effects on moving and zooming sports broadcast cameras, an automated and real-time pose and focal length estimation system was developed. For this purpose, the software used natural features such as pitch lines and the calibration core engine described hereafter. The software comprises the following three modules: sports pitch lines detection and labeling, real-time camera pose estimation engine, and GUI for parameter settings and feedback.

Flexible multiview calibration system

BBC R&D, 2011-2012

calibration wand In order to avoid the use of large and cumbersome checker calibration charts for multiple camera calibration in large working volumes, a new system based on a lightweight LED wand was designed, developed, and deployed at the BBC. This system was part of a multiple-view capture, 3D reconstruction, and rendering pipeline. It comprises a 12-LED wireless calibration wand, a real-time LED detection and labeling software, and the actual camera parameter estimation. Furthermore, a plugin for the capture module has been developed, which provides real-time feedback during acquisition.

Light-Weight Wheelchair Inertial Motion capture

Imperial College London, 2010-2011

A light-weight sensor composed of an accelerometer and gyroscope was mounted in the wheel of a wheelchair for motion tracking in real-time with wireless feedback to the sports coach. Based on a physical model of the wheelchair intrinsic and global motion, the fusion of several sensors readings with a Kalman filter provides a robust motion estimation, with limited drift over time. The final solution enabled accurate displacement estimation at a limited cost, weight, and installation complexity [1]. The system was trialled by the British Paralympic Basketball team.


Autonomous Tennis Player Tracking

Imperial College London, 2008-2009

VSN tennis While at Imperial College London, I have participated in the development of an autonomous, real-time, wireless tennis player tracker using computer vision [13] in collaboration with the Lawn Tennis Association (LTA). The hardware was based on a custom-built ARM-like architecture encompassing a camera and a wireless link. I worked on the firmware for autonomous tennis player tracking. It consisted of the following modules: tennis court line detection, camera calibration, background segmentation, estimation of the player on court, and micro web server for direct access.


Immersive visualisation of archeological data

The University of Hull, 2003-2004

I have worked on an immersive visualization and interaction of multidimensional archaeological data - Digital Archaeological Virtual Environment (DAVE). The aim was to help archaeologists to classify various kinds of data (text, 3D objects, sketches, images, video, ...) by rendering them onto a 3D terrain model with multiple overlays (aerial photography, maps, ...) [25].
Various options were included to visualise the scene under different perspectives (eg. rendering an archaeological findings density map, extrapolating the crop marks as walls, changing the sea level, ...). The application rendered on a large back-projected wall screen (6x3m), with a wireless joystick and a tracked glove to enable interaction within the scene.


  • Alternate sea level simulation

    Alternate sea level simulation

  • Finding density map

    Archaeological findings density map

  • Crop marks extrapolated as 3D walls

    Crop marks extrapolated as 3D walls

  • Navigation compass

    Navigation compass

  • Hayton Fort 3D model

    Hayton Fort 3D model

  • Artefacts 3D histogram

    Artefacts 3D histogram

  • editor

    Scene editor GUI

Publications

All the publications available on this website are for personal use only and remain under the copyright of their respective owner. Redistribution is prohibited.

Journal publications

2011
pansiot11wisdom[1] Julien Pansiot, Zhiqiang Zhang, Benny Lo, and Guang-Zhong Yang. WISDOM: Wheelchair Inertial Sensors for Displacement and Orientation Monitoring. Measurement Science and Technology. 22(10):pages 105801, 2011. [ bib | http | pdf | video ]
2008
aziz08ubiquitous[2] Omer Aziz, Benny Lo, Julien Pansiot, Louis Atallah, Guang-Zhong Yang, and Ara Darzi. From computers to ubiquitous computing by 2010: health care. Philosophical Transactions of The Royal Society A. 366(1881):pages 3805-3811, 2008. [ bib | http | pdf ]

Conference publications

2016
pansiot16xrays3d[3] Julien Pansiot and Edmond Boyer. 3D Imaging from Video and Planar Radiography. In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Athens, October 2016. (oral presentation) [ bib ]
2014
pansiot14xrays[4] Julien Pansiot, Lionel Reveret, and Edmond Boyer. Combined Visible and X-Ray 3D Imaging. In Medical Image Understanding and Analysis (MIUA), pages 13-18, London, July 2014. [ bib | http | pdf ]
2013
maleshkova13artworks[5] Jeni Maleshkova, Matthew Purver, Oliver Grau, and Julien Pansiot. Presentation and Communication of Visual Artworks in an Interactive Virtual Environment. In SIGGRAPH Asia 2013 Posters, pages 37:1, Hong Kong, November 2013. [ bib | http | pdf ]
provini13markerless[6] Pauline Provini, Julien Pansiot, Lionel Reveret, and Olivier Martin. Video-based methodology for markerless human motion analysis. In 15ème Congrès international de l'Association des Chercheurs en Activités Physiques et Sportives (ACAPS), Grenoble, October 2013. [ bib | http | pdf ]
2012
grau12rolling[7] Oliver Grau and Julien Pansiot. Motion and velocity estimation of rolling shutter cameras. In ACM Proceedings of the 9th Conference on Visual Media Production (CVMP), pages 94-98, London, 2012. [ bib | http | pdf ]
volino12layered[8] Marco Volino, Julien Pansiot, Oliver Grau, and Adrian Hilton. Layered View-dependent Texture Maps. In Proceedings of the 9th Conference on Visual Media Production (CVMP short papers), London, 2012. [ bib | http | pdf ]
maleshkova12painter[9] Jeni Maleshkova, Julien Pansiot, and Oliver Grau. Through the painter's eye: interactive 3D presentation of paintings on the web. In Proceedings of the 9th Conference on Visual Media Production (CVMP short papers), London, 2012. [ bib | pdf ]
2011
zhang11back[10] Zhiqiang Zhang, Julien Pansiot, Benny Lo, and Guang-Zhong Yang. Human Back Movement Analysis Using BSN. In 8th International Conference on Body Sensor Networks (BSN), Dallas, Texas, 2011. [ bib | http | pdf ]
2010
mcilwraith10fusion[11] Douglas Gavin McIlwraith, Julien Pansiot, and Guang-Zhong Yang. Wearable and Ambient Sensor Fusion for the Characterisation of Human Motion. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5505-5510, Taiwan, October 2010. [ bib | http | pdf ]
pansiot10swimming[12] Julien Pansiot, Benny Lo, and Guang-Zhong Yang. Swimming Stroke Kinematic Analysis with BSN. In Proceedings of the IEEE International Conference on Body Sensor Networks (BSN), pages 153-158, Singapore, June 2010. [ bib | http | pdf ]
2009
pansiot09racket[13] Julien Pansiot, Ahmed Elsaify, Benny Lo, and Guang-Zhong Yang. RACKET: Real-time Autonomous Computation of Kinematic Elements in Tennis. In IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops) - Fifth IEEE Workshop on Embedded Computer Vision, pages 773-779, Kyoto, Japan, 2009. (Best Paper Award) [ bib | http | pdf ]
mcilwraith09structure[14] Douglas McIlwraith, Julien Pansiot, James Ballantyne, Salman Valibeik, Ahmed Elsaify, and Guang-Zhong Yang. Structure Learning for Activity Recognition in Robotic Assisted Intelligent Environments. In IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS), pages 4644-4649, St. Louis, MO, 2009. [ bib | http | pdf ]
king09rowing[15] Rachel King, Douglas McIlwraith, Benny Lo, Julien Pansiot, Alison McGregor, and Guang-Zhong Yang. Body Sensor Networks for Monitoring Rowing Technique. In IEEE proceedings of the 6th International Workshop on Wearable and Implantable Body Sensor Networks (BSN), pages 251-255, Berkeley, CA, 2009. [ bib | http | pdf ]
lo09grf[16] Benny Lo, Julien Pansiot, and Guang-Zhong Yang. Bayesian Analysis of Sub-Plantar Ground Reaction Force with BSN. In IEEE proceedings of the 6th International Workshop on Wearable and Implantable Body Sensor Networks (BSN), pages 133-137, Berkeley, CA, 2009. [ bib | http | pdf ]
elhelw09framework[17] Mohamed ElHelw, Julien Pansiot, Douglas McIlwraith, Raza Ali, Benny Lo, Louis Atallah, and Guang-Zhong Yang. An Integrated Multi-Sensing Framework for Pervasive Healthcare Monitoring. In 3rd International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2009. (Best Paper Award) [ bib | http | pdf ]
2008
pansiot08climbsn[18] Julien Pansiot, Rachel King, Douglas McIlwraith, Benny Lo, and Guang-Zhong Yang. ClimBSN: Climber Performance Monitoring with BSN. In IEEE proceedings of the 5th International Workshop on Wearable and Implantable Body Sensor Networks 2008 (BSN), pages 33-36, Hong Kong, China, 2008. [ bib | http | pdf ]
mcilwraith08fusion[19] Douglas McIlwraith, Julien Pansiot, Surapa Thiemjarus, Benny Lo, and Guang-Zhong Yang. Probabilistic Decision Level Fusion for Real-Time Correlation of Ambient and Wearable Sensors. In IEEE proceedings of the 5th International Workshop on Wearable and Implantable Body Sensor Networks 2008 (BSN), pages 117-120, Hong Kong, China, 2008. [ bib | http | pdf ]
thiemjarus08inferencing[20] Surapa Thiemjarus, Julien Pansiot, Douglas McIlwraith, Benny Lo, and Guang-Zhong Yang. An Integrated Inferencing Framework for Context Sensing. In IEEE proceedings of the 5th International Conference on Information Technology and Applications in Biomedicine 2008 (ITAB), Shenzhen, China, 2008. [ bib | http | pdf ]
2007
pansiot07fusion[21] Julien Pansiot, Danail Stoyanov, Douglas McIlwraith, Benny Lo, and Guang-Zhong Yang. Ambient and Wearable Sensor Fusion for Activity Recognition in Healthcare Monitoring Systems. In IFMBE proceedings of the 4th International Workshop on Wearable and Implantable Body Sensor Networks 2007 (BSN), pages 208-212, Aachen, Germany, 2007. [ bib | http | pdf ]
atallah07behaviour[22] Louis Atallah, Mohamed ElHelw, Julien Pansiot, Danail Stoyanov, Lei Wang, Benny Lo, and Guang-Zhong Yang. Behaviour Profiling with Ambient and Wearable Sensing. In IFMBE proceedings of the 4th International Workshop on Wearable and Implantable Body Sensor Networks 2007 (BSN), pages 133-138, Aachen, Germany, 2007. [ bib | http | pdf ]
2006
pansiotIBMRsensing[23] Julien Pansiot, Danail Stoyanov, Benny Lo, and Guang-Zhong Yang. Towards Image-Based Modeling for Ambient Sensing. In IEEE proceedings of the 3rd International Workshop on Wearable and Implantable Body Sensor Networks 2006 (BSN), pages 195-198, Cambridge, MA, April 2006. [ bib | http | pdf ]
2005
pansiotsimulator[24] Julien Pansiot, Benny Lo, and Guang-Zhong Yang. A Simulator for Distributed Ambient Intelligence Sensing. In IEE proceedings of the 2nd International Workshop on Wearable and Implantable Body Sensor Networks (BSN), pages 119, London, April 2005. [ bib | pdf ]
2004
pansiotfoulness[25] Julien Pansiot, Paul Chapman, Warren Viant, and Peter Halkon. New Perspectives on Ancient Landscapes: A Case Study of the Foulness Valley. In Eurographics proceedings of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, pages 251-260, Belgium, December 2004. [ bib | http | pdf ]

Patents

2016
pansiot16depth[26] Julien Pansiot, Edmond Boyer, and Lionel Reveret. System and method for three-dimensional depth imaging. Patent App. WO2016005688, January 2016. Patent App. WO2016005688 [ bib | http ]

Master & PhD theses

2009
pansiotPhD[27] Julien Pansiot. Markerless Visual Tracking and Motion Analysis for Sports Monitoring. PhD Thesis, Imperial College London, Department of Computing, October 2009. [ bib | pdf ]
2004
pansiotMSc[28] Julien Pansiot. Immersive Visualization and Interaction of Multidimensional Archaeological Data. Master's Thesis, The University of Hull, Department of Computer Science, September 2004. [ bib | pdf ]

List generated by bib2html.py on electrode | Updated on 28 Jul 2016.

Extras

Julien Pansiot I am a keen marathon and ultramarathon runner (life-is-an-ultramarathon.org) and also enjoy hiking, rock climbing, paragliding, homebrewing, and photography.

I am particularly involved in the Hardmoors race series in North Yorkshire. Aside from running it, I volunteered as marshal and race director and I have been the hardmoors110.org.uk webmaster for several years.

Contact

e-mail: rf.airni@toisnap.neiluj (copy/paste might not work).
Tel: +33 (0)4 76 61 55 90
Top