Dataset

1. Sample images for CRF calibration

The CRF functions of a RGB-D sensor can be calibrated using CRF calibration tools provided in github.

Specifically, we first take multiple image with various exposure at a fixed location viewing a static scene using the “multi_expo_capturer” tool.

To illustrate the calibration procedure, a sample sequence of the multiple exposure image is available from here. Sample images are shown as below:

multi-exposure

From left to right, the exposure time are 3, 6, 12, 24, 48, 96, 192 and 384 ms, respectively.

At each exposure, we take 30 individual frames, and their file names are arranged as:

  • exp.idx.png                           exp: the exposure time; idx: from 0 to 29.

This is because the exposure control of Xtion sensor can be inaccurate. We manually remove the frames whose brightness are inconsistent with the majority of frames taken at the specific exposure time.

After that, the remaining images are fed into  the tool “calc_crf” for estimation. Its derivate, inverse CRF and pixel noise function are then calibrated using the matlab tool “crf_calibration_matlab”.

2. Synthetic Dataset

The synthetic dataset is constructed for evaluating the tracking performance of the proposed method under simulated exposure changes. It is based on ICL-NUIM dataset.

Two sequences of synthetic dataset,”smooth” (top) and “flicker“, were created. “smooth” is aimed at simulating the smooth adjusting of exposure and “flicker” the flickering artifacts caused by sudden changes of exposure.  In both sequences, the synthetic RGB and depth noise are added.

The following figure shows the first 9 frames from synthetic sequences “smooth” (top) and “flicker” (bottom), respectively.

synthetic_frames

“smooth” (top) and “flicker” (bottom) can be downloaded by clicking at them. The noise free version as well as a noisy version and the ground truth poses are packed together in zip files. The naming convention of RGB-D frames are shown in the following:

  • scene_xx_xxxx.png                                 image with noise
  • scene_xx_xxxx.depth.png                    depth with noise
  • nf_scene_xx_xxxx.png                          noise free image
  • nf_scene_xx_xxxx.depth.png              noise free depth map
  • scene_xx_xxxx.pose.txt                         ground truth of camera poses
  • gt_exp_sequence.txt                               ground truth of exposure time (ms)

where xx is always 00 and xxxx is the index of the frame ranging from 0 to 1000.

The noise free image vs noisy image:

noise

In a “.pos.txt” file, a camera pose is represented in the following format:

-0.0030101 0.0122165 -0.0068443 -1.2366763 -0.5408358 0.5625012
rotation vector                                       translation vector

The first 3 floating type values are the rotation vector and the last 3 values are translation vector.  The rotation and translation transform a point in world coordinate system to current camera coordinate system.

3. Real Dataset:

The real datasets are composed of the following “.oni” files catpured using various light source:

  • floor1                             Tungsten light blub
  • floor2                             Tungsten light blub
  • whiteboard                  Indirect sun light + fluorescent
  • bear                                Indirect sun light
  • desk                               fluorescent
  • sofa                                fluorescent + Dedolight-400D
  • serials.yml                   index of calibration data

The HDR texture captured using these sequences are shown in the accompanying video.