Visual Vibrometry:

          Estimating Material Properties from Small Motions in Video           

Abe Davis*      Katherine L. Bouman*      Justin G. Chen      Michael Rubinstein
     Fredo Durand       William T. Freeman

• Home
• CVPR Paper
• PAMI Paper
• Dataset

We present a method for estimating material properties of an object by examining small motions in video. (A) We record video of different fabrics and clamped rods exposed to small forces such as sound or natural air currents in a room. (B) We show fabrics (top) color-coded and ordered by area weight, and rods (bottom) similarly ordered by their ratio of elastic modulus to density. (C) Local motion signals are extracted from captured videos and used to compute a temporal power spectrum for each object. These motion spectra contain information that is predictive of each object’s material properties. For instance, observe the trends in the spectra for fabrics and rods as they increase in area weight and elasticity/density, resp (blue to red). By examining these spectra, we can make inferences about the material properties of objects.

UPDATE! PAMI 2016 Paper:

We published an extended version of our paper with additional experiments, analysis, and a method for estimating damping, in PAMI 2016. (LINK)

Abstract

The estimation of material properties is important for scene understanding, with many applications in vision, robotics, and structural engineering. This paper connects fundamentals of vibration mechanics with computer vision techniques in order to infer material properties from small, often imperceptible motion in video. Objects tend to vibrate in a set of preferred modes. The shapes and frequencies of these modes depend on the structure and material properties of an object. Focusing on the case where geometry is known or fixed, we show how information about an object’s modes of vibration can be extracted from video and used to make inferences about that object’s material properties. We demonstrate our approach by estimating material properties for a variety of rods and fabrics by passively observing their motion in high-speed and regular framerate video.

Video