HDRFusion: HDR SLAM using a low-cost auto-exposure RGB-D sensor (Source codes and dataset released!!!)

by Shuda Li*, Ankur Handa**, Yang Zhang*, Andrew Calway*
*University of Bristol, **University of Cambridge
Accepted by Intl. Conf. on 3DVision (3DV), 2016

  • [paper]
  • [github]   NEW!!!
  • [video]
  • [dataset]   NEW!!!
  • Experiments demonstrating strong robustness to motion blur NEW!!!


We describe a new method for comparing frame appearance in a frame-to-model 3-D mapping and tracking system using an low dynamic range (LDR) RGB-D camera which is robust to brightness changes caused by auto exposure. It is based on a normalised radiance measure which is invariant to exposure changes and not only robustifies the tracking under changing lighting conditions, but also enables the following exposure compensation perform accurately to allow online building of high dynamic range (HDR) maps. The latter facilitates the frame-to-model tracking to minimise drift as well as better capturing light variation within the scene. Results from experiments with synthetic and real data demonstrate that the method provides both improved tracking and maps with far greater dynamic range of luminosity.


Figure. (a) shows the proposed frame-to-model tracking using normalized ra-
diance deliver best tracking accuracy using visual data. The tracking is per-
formed using a challenging synthetic flickering RGB-D sequence. (b)-(e) are
screen captures from video released with previous works. Specifically, (b) and (d)
are from [1], where (b) is the raw input image. (d) is predicted scene textures. By
contrast, the unrealistic artefacts, marked by red circles, indicate insufficent ex-
posure compensation. (c) is predicted scene texture from [2]; (e) from [3]. Similar
artefacts can be seen in these results. (f) in the top right shows the results from
our implementation of [4] using a RGB-D video sequence, the artefacts are very
strong due to large camera exposure adjustment when moving from bright area
(top in the scene) to the dark area (bottom left in the scene). (g) in the bottom
right are the predicted textures using the proposed HDRFusion. It can been that
it is free of artefacts and its HDR textures are visualized using Mantiuk tone
mapping operater [5].


[1] Meilland, M., Barat, C., Comport, A.: 3D High Dynamic Range dense visual
SLAM and its application to real-time object re-lighting. In: IEEE/ACM Intl.
Symposium on Mixed and Augmented Reality (ISMAR). (2013) 143–152.

[2]  Kerl, C., Cremers, D., Universit, T.: Dense Continuous-Time Tracking and Map-
ping with Rolling Shutter RGB-D Cameras. In: Intl. Conf. on Computer Vision
(ICCV). (2015).

[3]  Whelan, T., Leutenegger, S., Salas-moreno, R.F., Glocker, B., Davison, A.J.: Elas-
ticFusion : Dense SLAM Without A Pose Graph. Robotics: Science and Systems
(RSS) (2015).

[4] Whelan, T., Kaess, M., Johannsson, H., Fallon, M., Leonard, J.J., McDonald, J.:
Real-time large-scale dense RGB-D SLAM with volumetric fusion. Intl. Journal
on Robotics Research (IJRR) 34(4-5) (2015) 598–626.

[5] Mantiuk, R., Daly, S., Kerofsky, L.: Display adaptive tone mapping. ACM Trans.
on Graphics (ToG) 27(3) (2008).


Posted in Uncategorized | Leave a comment

Absolute pose estimation using multiple forms of correspondence from RGB-D frames (Source codes released!!!)

by Shuda Li and Andrew Calway
University of Bristol, UK
accepted by IEEE Intl. Conf. on Robotics and Automation (ICRA), 2016


We describe a new approach to absolute pose estimation from noisy and outlier contaminated matching point sets for RGB-D sensors. We show that by integrating multiple forms of correspondence based on 2-D and 3-D points and surface normals gives more precise, accurate and robust pose estimates. This is because it gives more constraints than using one form alone and increases the available measurements, especially when dealing with sparse matching sets. We demonstrate the approach by incorporating it within a RANSAC algorithm and introduce a novel direct least-square approach to calculate pose estimates. Results from experiments on synthetic and real data demonstrate improved performance over existing methods.


Posted in Uncategorized | Leave a comment

RGBD Relocalisation Using Pairwise Geometry and Concise Key Point Sets (Source codes released!!!)

by Shuda Li and Andrew Calway.
University of Bristol, UK
publised in IEEE Intl. Conf. on Robotics and Automation (ICRA), 2015


We describe a novel RGBD relocalisation algorithm based on key point matching. It combines two components. First, a graph matching algorithm which takes into account the pairwise 3-D geometry amongst the key points, giving robust relocalisation. Second, a point selection process which provides an even distribution of the `most matchable’ points across the scene based on non-maximum suppression within voxels of a volumetric grid. This ensures a bounded set of matchable key points which enables tractable and scalable graph matching at frame rate. We present evaluations using a public dataset and our own more difficult dataset containing large pose changes, fast motion and non-stationary objects. It is shown that the method significantly out performs state-of-the-art methods.


Posted in Uncategorized | Tagged , , | Leave a comment

CMake + CUDA + Dynamic parallelism (Nested parallelism)

set(CUDA_NVCC_FLAGS ${CUDA_NVCC_FLAGS};-gencode arch=compute_35,code=sm_35)
link_directories( "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v6.5\\lib\\x64")
target_link_libraries( DynamicParallel cudadevrt.lib )

Posted in Uncategorized | Tagged , | Leave a comment

Upgrade all stuffs to CUDA 6.5rc + VS2013 + QT 5.3.1 + OpenCV 3.0 + PCL 1.7.2 today. Feeling so cool!!!

Really love VS2013 for its space efficiency and speed. Here is a brief comparison between 2013 and 2012:

Other new features useful to me:

  • online account to synchronize IDE environment settings among multiple PCs
  • c++ 11 features such as “vector a(5)”
  • CUDA syntax highlighting when combined with CUDA 6.5rc

Possible bugs I encountered in 2013:

  • when setting project properties for multiple projects, it crashes.
  • Network failure will make it impossible to start or shut down.
Posted in Uncategorized | Leave a comment

Bolt C++ Template Library

Mentioned in SLAM++ CVPR’13 paper, Bolt C++ Template Library is a GPU library provides paralleled primitive operations such as reduction, scan and sort. 

[Resources] http://developer.amd.com/tools-and-sdks/heterogeneous-computing/amd-accelerated-parallel-processing-app-sdk/bolt-c-template-library/


– What is the hardware environment compilable with BCT? 





Posted in Uncategorized | Leave a comment



Gallery | Leave a comment