Tracking control of a UAV with a parallel visual processor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

  • External authors:
  • Colin Greatwood
  • Laurie Bose
  • Thomas Richardson
  • Walterio Mayol-Cuevas
  • Jianing Chen
  • Stephen J. Carey


This paper presents a vision-based control strategy for tracking a ground target using a novel vision sensor featuring a processor for each pixel element. This enables computer vision tasks to be carried out directly on the focal plane in a highly efficient manner rather than using a separate general purpose computer. The strategy enables a small, agile quadrotor Unmanned Air Vehicle (UAV) to track the target from close range using minimal computational effort and with low power consumption. To evaluate the system we target a vehicle driven by chaotic dual-pendulum trajectories. Target proximity and the large, unpredictable accelerations of the vehicle cause challenges for the UAV in keeping it within the downward facing camera's field of view (FoV). A state observer is used to smooth out predictions of the target's location and, importantly, estimate velocity. Experimental results also demonstrate that it is possible to continue to re-acquire and follow the target during short periods of loss in target visibility. The tracking algorithm exploits the parallel nature of the visual sensor, enabling high rate image processing ahead of any communication bottleneck with the UAV controller. With the vision chip carrying out the most intense visual information processing, it is computationally trivial to compute all of the controls for tracking onboard. This work is directed toward visual agile robots that are power efficient and that ferry only useful data around the information and control pathways.

Bibliographical metadata

Original languageEnglish
Title of host publicationIROS 2017 - IEEE/RSJ International Conference on Intelligent Robots and Systems
ISBN (Electronic)9781538626825
Publication statusPublished - Sep 2017
Event2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017 - Vancouver, Canada
Event duration: 24 Sep 201728 Sep 2017


Conference2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2017