Julien Pansiot

Julien Pansiot

Starting Researcher @ Inria Grenoble
Human motion capture and medical imaging

Summary

Research interests

X-rays / visible My main research interests lies in human motion capture, with an inclination to multi-modal sensing.
Human motion capture is an exciting research topic given the complex nature of the human body and the large number of high-impact applications in medicine, sports science, biology, and media production.
Multi-modal approaches have often gained my favour because they allow to exploit the complementarity of different sensing modalities to achieve more robust results. This is actually a very natural approach, as for example the human sense of balance is based on a combination of visual (eyes), inertial (vestibular system), and pressure/force (proprioception) cues.

Current position

I am currently an "Inria Starting Researcher" at Morpheo. I have started an exciting new research topic within Morpheo, on the joint usage of video cameras and X-ray imaging to build accurate human motion models. The complementarity of the two imaging modalities opens novel research directions at the boundary between motion capture and medical imaging. Initial work has already led to some promising results [3] [4] [5].

Recent papers

Funding

  • I am currently funded by an Inria Starting Research Position (6 positions/year nationally).
  • I am involved in the PERSYVAL LabEx exploratory project Carambole, aiming at tracking 4D deformation of growing plantes. Co-author, collaboration with MSC/Paris 7. Budget: 9k€.
  • I have recently received an ANR Collaborative Research Project with Industrials (PRCE): ``CaMoPi'' Capture and Modelling of the Shod Foot in Motion. Scientific Coordinator & Principal Investigator, collaboration with Mines St Etienne / CIS, CTC, Sporaltec. Budget: 541k€ (success rate: 12.5%). More details to come soon.

Selected Projects

Combined Video and X-Ray 3D Imaging

Starting Researcher @ Inria Grenoble / Morpheo | 2013-current

KINVOVIS platformComputed Tomography (CT) scanners provide accurate 3D X-ray attenuation models of living organisms, allowing for a range of studies and diagnostics. However, when compared to regular planar radiography, CT scanners are significantly more expensive, expose the patient to higher radiation doses, and require immobility.
In order to tackle these issues, we proposed to combine a regular X-ray imaging device with a set of cameras [3] [4] [5]. A software suite which estimates a dense 3D attenuation model of a moving sample based on a short capture by a single X-ray imaging device combined with 10 video cameras has been deployed on the novel KINOVIS platform recently installed at the Grenoble University Hospital (CHU).


Free-viewpoint Video Rendering

Technologist @ BBC R&D / Production Magic | 2011-2013

Dynamic scenes captured simultaneously from multiple cameras can be rendered with the traditional garphics pipeline, i.e. using a single baked texture mapped over a reconstructed mesh. However, significantly more anisotropic details can be rendered by using all originally captured video streams. For this purpose, the original images are fused at run-time based on the currently required viewpoint. In the context of the FP7 RE@CT project (capture, modelling, and 4D rendering) with the BBC, I have developped a real-time, view-dependent rendering engine from multiple views to replace previous software. The module has been implemented as both a standalone and an OpenSceneGraph library plugin, and defines a file format for multi-view 4D sequences. It also allowed to replace tracked proxy-props (such as fake swords) by virtual models.
I have also worked on WebGL-based applications [6], [10], as well as real-time free-viewpoint rendering from multiple views [9].


Real-time sports pitch camera pose estimation system

Technologist @ BBC R&D / Production Magic | 2012-2013

pitch calibration GUI In order to allow for live 2D and 3D effects on moving and zooming sports broadcast cameras, an automated and real-time pose and focal length estimation system was developed. For this purpose, the software used natural features such as pitch lines and the calibration core engine described hereafter. The software comprises the following three modules: sports pitch lines detection and labeling, real-time camera pose estimation engine, and GUI for parameter settings and feedback.

Flexible multiview calibration system

Technologist @ BBC R&D / Production Magic | 2011-2012

calibration wand In order to avoid the use of large and cumbersome checker calibration charts for multiple camera calibration in large working volumes, a new system based on a lightweight LED wand was designed, developed, and deployed at the BBC. This system was part of a multiple-view capture, 3D reconstruction, and rendering pipeline. It comprises a 12-LED wireless calibration wand, a real-time LED detection and labeling software, and the actual camera parameter estimation. Furthermore, a plugin for the capture module has been developed, which provides real-time feedback during acquisition. I have also worked specifically on the calibration of rolling shutter CMOS cameras [8].

Light-Weight Wheelchair Inertial Motion capture

Research associate @ Imperial College London / Visual Information Processing | 2010-2011

A light-weight sensor composed of an accelerometer and gyroscope was mounted in the wheel of a wheelchair for motion tracking in real-time with wireless feedback to the sports coach. Based on a physical model of the wheelchair intrinsic and global motion, the fusion of several sensors readings with a Kalman filter provides a robust motion estimation, with limited drift over time. The final solution enabled accurate displacement estimation at a limited cost, weight, and installation complexity [1]. The system was trialled by the British Paralympic Basketball team.
I have also worked on a number of other sports such as swimming [13], rowing [16], running [17], climbing [19], and speed-skating.


Autonomous Tennis Player Tracking

PhD candidate @ Imperial College London / Visual Information Processing | 2008-2009

VSN tennis While at Imperial College London, I have participated in the development of an autonomous, real-time, wireless tennis player tracker using computer vision [14] in collaboration with the Lawn Tennis Association (LTA). The hardware was based on a custom-built ARM-like architecture encompassing a camera and a wireless link. I worked on the firmware for autonomous tennis player tracking. It consisted of the following modules: tennis court line detection, camera calibration, background segmentation, estimation of the player on court, and micro web server for direct access.


Immersive visualisation of archeological data

MSc by Research @ The University of Hull / SimVis | 2003-2004

I have worked on an immersive visualization and interaction of multidimensional archaeological data - Digital Archaeological Virtual Environment (DAVE). The aim was to help archaeologists to classify various kinds of data (text, 3D objects, sketches, images, video, ...) by rendering them onto a 3D terrain model with multiple overlays (aerial photography, maps, ...) [26].
Various options were included to visualise the scene under different perspectives (eg. rendering an archaeological findings density map, extrapolating the crop marks as walls, changing the sea level, ...). The application rendered on a large back-projected wall screen (6x3m), with a wireless joystick and a tracked glove to enable interaction within the scene.


  • Alternate sea level simulation

    Alternate sea level simulation

  • Finding density map

    Archaeological findings density map

  • Crop marks extrapolated as 3D walls

    Crop marks extrapolated as 3D walls

  • Navigation compass

    Navigation compass

  • Hayton Fort 3D model

    Hayton Fort 3D model

  • Artefacts 3D histogram

    Artefacts 3D histogram

  • editor

    Scene editor GUI

Publications

All the publications available on this website are for personal use only and remain under the copyright of their respective owner. Redistribution is prohibited.

Journal publications

2011
pansiot11wisdom[1] Julien Pansiot, Zhiqiang Zhang, Benny Lo, and Guang-Zhong Yang. WISDOM: Wheelchair Inertial Sensors for Displacement and Orientation Monitoring. Measurement Science and Technology. 22(10):pages 105801, 2011. [ bib | http | pdf | video ]
2008
aziz08ubiquitous[2] Omer Aziz, Benny Lo, Julien Pansiot, Louis Atallah, Guang-Zhong Yang, and Ara Darzi. From computers to ubiquitous computing by 2010: health care. Philosophical Transactions of The Royal Society A. 366(1881):pages 3805-3811, 2008. [ bib | http | pdf ]

Conference publications

2017
pansiot17ctfm[3] Julien Pansiot and Edmond Boyer. CT from Motion: Volumetric Capture of Moving Shapes with X-rays and Videos. In British Machine Vision Conference (BMVC), London, September 2017. [ bib | http | pdf | video ] > More details
2016
pansiot16xrays3d[4] Julien Pansiot and Edmond Boyer. 3D Imaging from Video and Planar Radiography. In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) - Part III, pages 450-457, Athens, October 2016. (oral presentation) [ bib | http | pdf | video ] > More details
2014
pansiot14xrays[5] Julien Pansiot, Lionel Reveret, and Edmond Boyer. Combined Visible and X-Ray 3D Imaging. In Medical Image Understanding and Analysis (MIUA), pages 13-18, London, July 2014. [ bib | http | pdf ] > More details
2013
maleshkova13artworks[6] Jeni Maleshkova, Matthew Purver, Oliver Grau, and Julien Pansiot. Presentation and Communication of Visual Artworks in an Interactive Virtual Environment. In SIGGRAPH Asia 2013 Posters, pages 37:1, Hong Kong, November 2013. [ bib | http | pdf ]
provini13markerless[7] Pauline Provini, Julien Pansiot, Lionel Reveret, and Olivier Martin. Video-based methodology for markerless human motion analysis. In 15ème Congrès international de l'Association des Chercheurs en Activités Physiques et Sportives (ACAPS), Grenoble, October 2013. [ bib | http | pdf ]
2012
grau12rolling[8] Oliver Grau and Julien Pansiot. Motion and velocity estimation of rolling shutter cameras. In ACM Proceedings of the 9th Conference on Visual Media Production (CVMP), pages 94-98, London, 2012. [ bib | http | pdf ]
volino12layered[9] Marco Volino, Julien Pansiot, Oliver Grau, and Adrian Hilton. Layered View-dependent Texture Maps. In Proceedings of the 9th Conference on Visual Media Production (CVMP short papers), London, 2012. [ bib | pdf ]
maleshkova12painter[10] Jeni Maleshkova, Julien Pansiot, and Oliver Grau. Through the painter's eye: interactive 3D presentation of paintings on the web. In Proceedings of the 9th Conference on Visual Media Production (CVMP short papers), London, 2012. [ bib | pdf ]
2011
zhang11back[11] Zhiqiang Zhang, Julien Pansiot, Benny Lo, and Guang-Zhong Yang. Human Back Movement Analysis Using BSN. In 8th International Conference on Body Sensor Networks (BSN), Dallas, Texas, 2011. [ bib | http | pdf ]
2010
mcilwraith10fusion[12] Douglas Gavin McIlwraith, Julien Pansiot, and Guang-Zhong Yang. Wearable and Ambient Sensor Fusion for the Characterisation of Human Motion. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5505-5510, Taiwan, October 2010. [ bib | http | pdf ]
pansiot10swimming[13] Julien Pansiot, Benny Lo, and Guang-Zhong Yang. Swimming Stroke Kinematic Analysis with BSN. In Proceedings of the IEEE International Conference on Body Sensor Networks (BSN), pages 153-158, Singapore, June 2010. [ bib | http | pdf ]
2009
pansiot09racket[14] Julien Pansiot, Ahmed Elsaify, Benny Lo, and Guang-Zhong Yang. RACKET: Real-time Autonomous Computation of Kinematic Elements in Tennis. In IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops) - Fifth IEEE Workshop on Embedded Computer Vision, pages 773-779, Kyoto, Japan, 2009. (Best Paper Award) [ bib | http | pdf ]
mcilwraith09structure[15] Douglas McIlwraith, Julien Pansiot, James Ballantyne, Salman Valibeik, Ahmed Elsaify, and Guang-Zhong Yang. Structure Learning for Activity Recognition in Robotic Assisted Intelligent Environments. In IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS), pages 4644-4649, St. Louis, MO, 2009. [ bib | http | pdf ]
king09rowing[16] Rachel King, Douglas McIlwraith, Benny Lo, Julien Pansiot, Alison McGregor, and Guang-Zhong Yang. Body Sensor Networks for Monitoring Rowing Technique. In IEEE proceedings of the 6th International Workshop on Wearable and Implantable Body Sensor Networks (BSN), pages 251-255, Berkeley, CA, 2009. [ bib | http | pdf ]
lo09grf[17] Benny Lo, Julien Pansiot, and Guang-Zhong Yang. Bayesian Analysis of Sub-Plantar Ground Reaction Force with BSN. In IEEE proceedings of the 6th International Workshop on Wearable and Implantable Body Sensor Networks (BSN), pages 133-137, Berkeley, CA, 2009. [ bib | http | pdf ]
elhelw09framework[18] Mohamed ElHelw, Julien Pansiot, Douglas McIlwraith, Raza Ali, Benny Lo, Louis Atallah, and Guang-Zhong Yang. An Integrated Multi-Sensing Framework for Pervasive Healthcare Monitoring. In 3rd International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2009. (Best Paper Award) [ bib | http | pdf ]
2008
pansiot08climbsn[19] Julien Pansiot, Rachel King, Douglas McIlwraith, Benny Lo, and Guang-Zhong Yang. ClimBSN: Climber Performance Monitoring with BSN. In IEEE proceedings of the 5th International Workshop on Wearable and Implantable Body Sensor Networks 2008 (BSN), pages 33-36, Hong Kong, China, 2008. [ bib | http | pdf ]
mcilwraith08fusion[20] Douglas McIlwraith, Julien Pansiot, Surapa Thiemjarus, Benny Lo, and Guang-Zhong Yang. Probabilistic Decision Level Fusion for Real-Time Correlation of Ambient and Wearable Sensors. In IEEE proceedings of the 5th International Workshop on Wearable and Implantable Body Sensor Networks 2008 (BSN), pages 117-120, Hong Kong, China, 2008. [ bib | http | pdf ]
thiemjarus08inferencing[21] Surapa Thiemjarus, Julien Pansiot, Douglas McIlwraith, Benny Lo, and Guang-Zhong Yang. An Integrated Inferencing Framework for Context Sensing. In IEEE proceedings of the 5th International Conference on Information Technology and Applications in Biomedicine 2008 (ITAB), Shenzhen, China, 2008. [ bib | http | pdf ]
2007
pansiot07fusion[22] Julien Pansiot, Danail Stoyanov, Douglas McIlwraith, Benny Lo, and Guang-Zhong Yang. Ambient and Wearable Sensor Fusion for Activity Recognition in Healthcare Monitoring Systems. In IFMBE proceedings of the 4th International Workshop on Wearable and Implantable Body Sensor Networks 2007 (BSN), pages 208-212, Aachen, Germany, 2007. [ bib | http | pdf ]
atallah07behaviour[23] Louis Atallah, Mohamed ElHelw, Julien Pansiot, Danail Stoyanov, Lei Wang, Benny Lo, and Guang-Zhong Yang. Behaviour Profiling with Ambient and Wearable Sensing. In IFMBE proceedings of the 4th International Workshop on Wearable and Implantable Body Sensor Networks 2007 (BSN), pages 133-138, Aachen, Germany, 2007. [ bib | http | pdf ]
2006
pansiotIBMRsensing[24] Julien Pansiot, Danail Stoyanov, Benny Lo, and Guang-Zhong Yang. Towards Image-Based Modeling for Ambient Sensing. In IEEE proceedings of the 3rd International Workshop on Wearable and Implantable Body Sensor Networks 2006 (BSN), pages 195-198, Cambridge, MA, April 2006. [ bib | http | pdf ]
2005
pansiotsimulator[25] Julien Pansiot, Benny Lo, and Guang-Zhong Yang. A Simulator for Distributed Ambient Intelligence Sensing. In IEE proceedings of the 2nd International Workshop on Wearable and Implantable Body Sensor Networks (BSN), pages 119, London, April 2005. [ bib | pdf ]
2004
pansiotfoulness[26] Julien Pansiot, Paul Chapman, Warren Viant, and Peter Halkon. New Perspectives on Ancient Landscapes: A Case Study of the Foulness Valley. In Eurographics proceedings of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, pages 251-260, Belgium, December 2004. [ bib | http | pdf ]

Patents

2016
pansiot16depth[27] Julien Pansiot, Edmond Boyer, and Lionel Reveret. System and method for three-dimensional depth imaging. Patent App. WO2016005688, January 2016. Patent App. WO2016005688 [ bib | http ]

Master & PhD theses

2009
pansiotPhD[28] Julien Pansiot. Markerless Visual Tracking and Motion Analysis for Sports Monitoring. PhD Thesis, Imperial College London, Department of Computing, October 2009. [ bib | pdf ]
2004
pansiotMSc[29] Julien Pansiot. Immersive Visualization and Interaction of Multidimensional Archaeological Data. Master's Thesis, The University of Hull, Department of Computer Science, September 2004. [ bib | pdf ]
List updated on 22 Sep 2017.

Extras

Julien Pansiot I am a keen marathon and ultramarathon runner (life-is-an-ultramarathon.org) and also enjoy hiking, rock climbing, paragliding, homebrewing, and photography.

I am particularly involved in the Hardmoors race series in North Yorkshire. Aside from running it, I volunteered as marshal and race director and I have been the hardmoors110.org.uk webmaster for several years.

Contact

e-mail: rf.airni@toisnap.neiluj (copy/paste might not work).
Tel: +33 (0)4 76 61 55 90
Top