Tracking objects by color alone

Abstract

This paper discusses a simple approach to tracking objects based solely on their color. By limiting our attention to areas of approximately uniform color on nearly Lambertian surfaces, we can assume that pixels corresponding to surface patches on the object form a linear, tubular distribution in color space. Performing principal component analysis on user-selected sample pixels parametrizes an ellipsoidal model of this distribution. The shape thus derived defines a static membership function that can be used to quickly and robustly track many objects through a range of orientations and scales, with varying levels of illumination, and in front of complex backgrounds. In combination with a loss recovery procedure, the basic technique has been successfully applied to tasks as diverse as head and hand tracking, following juggled and thrown balls, and constructing vision-based input devices.