Abstract
This paper presents a method for calibration of the extrinsic parameters of a sensor system that combines a camera with an inertial measurement unit (IMU) to estimate the pendulum motion of a crane payload. The camera measures the position and orientation of a fiducial marker on the payload, while the IMU is fixed to the payload and measures angular velocity and specific force. The placements of the marker and the IMU are initially unknown, and the extrinsic calibration parameters are their position and orientation with respect to the reference frame of the payload. The calibration is done with simultaneous state and parameter estimation, where a particle filter is used for state estimation, and a Riemannian gradient descent method is used for parameter estimation. The orientation is described with unit quaternions, and gradients are developed in a Riemannian formulation based on the Lie group of unit quaternions. This leads to efficient derivations of gradient expressions involving orientations and provides added geometric insight to the problem. The efficiency of the method is demonstrated in simulations and experiments for a simplified crane payload problem.