UMBmark -- A Method for Measuring, Comparing, and Correcting Odometry Errors in Mobile Robots

by Johann Borenstein(1) and Liqiang Feng(2), December 1994


This electronic document includes the Summary and the Table of Contents of the original hardcopy report. The hardcopy version is 68 pages long and includes numerous illustrations and plots. Copies of the report are available from the University of Michigan as: Technical Report UM-MEAM-94-22 (please contact Johann Borenstein by email to order a copy). Copies of the report are currently free, as long as our supply lasts. We have plans to make the fully text available via Anonymous FTP, but this may take a few more months to implement.

Portions of this report have been submitted for publication as papers 58, 59, 60, and 61.

Authors, Affiliations, and Credits

Authors and Affiliation:

Johann Borenstein and Liqiang Feng
are with the University of Michigan
Department of Mechanical Engineering and Applied Mechanics
Mobile Robotics Laboratory
1101 Beal Avenue
Ann Arbor, MI 48109
POC: Johann Borenstein
Ph.: (313) 763-1560
Fax: (313) 944-1113


This report was prepared by the University of Michigan for the Oak Ridge National Lab (ORNL) D&D Program and the United States Department of Energy's Robotics Technology Development Program within the Environmental Restoration, Decontamination and Dismantlement Project.

The authors wish to thank Professors David K. Wehe and Yoram Koren for their support in preparing this report. The authors also wish to thank Dr. William R. Hamel, D&D Technical Coordinator and Dr. Linton W. Yarbrough, DOE Program Manager, for their continuing support in funding this project.

The authors further wish to thank Professors D. K. Wehe and Y. Koren for their support and guidance, as well as Grad. Students Z. Fan, B. Holt, and B. Costanza for their support in conducting some of the experiments.

About the DOE project at the University of Michigan.


Odometry is the most widely used method for determining the momentary position of a mobile robot. In most practical applications odometry provides easily accessible real-time positioning information in-between periodic absolute position measurements. The frequency at which the (usually costly and/or time-consuming) absolute measurements must be performed depends to a large degree on the accuracy of the odometry system.

This report introduces a method for measuring odometry errors in mobile robots, and for expressing these errors quantitatively. When measuring odometry errors, one must distinguish between (1) systematic errors, which are caused by kinematic imperfections of the mobile robot (for example, unequal wheel-diameters), and (2) non-systematic errors, which may be caused by wheel-slippage or irregularities of the floor. Systematic errors are a property of the robot itself, and they stay almost constant over prolonged periods of time, while non-systematic errors are a function of the properties of the floor.

Our method, called the University of Michigan Benchmark test (UMBmark), is especially designed to uncover certain systematic errors that are likely to compensate for each other (and thus, remain undetected) in less rigorous tests. This report explains the rationale for the carefully designed UMBmark procedure and explains the procedure in detail. Experimental test results from different mobile robots are presented and discussed. Our report also proposes a method called extended UMBmark for measuring non-systematic errors. Although the measurement of non-systematic errors is less useful because it depends strongly on the floor characteristics, one can use the extended UMBmark test for comparison of different robots under similar conditions. This report presents experimental results from six different vehicles tested for their susceptibility to non-systematic error by means of the extended UMBmark test. With the quantitative benchmark test proposed here, researchers will be able to compare the odometric accuracy of different robots, or they can measure and tune the performance of a single robot.

Perhaps the foremost contribution of the work described here is a unique and innovative method for the calibration of mobile robots. This method is called UMBmark calibration because it is based on measurements from the UMBmark test. Performing an occasional calibration as proposed here will increase the robot's odometric accuracy and reduce operation cost because an accurate mobile robot requires fewer absolute positioning updates. Many manufacturers or end-users calibrate their robots, usually in a time-consuming and non-systematic trial and error approach. By contrast, the UMBmark calibration is systematic and provides near-optimal results. Our procedure for measuring and correcting systematic odometry errors can be performed easily and without complicated equipment. Furthermore, UMBmark lends itself readily for adaptation as an automated self-calibration procedure. Experimental results are presented that show a consistent improvement of at least one order of magnitude in odometric accuracy (with respect to systematic errors) for a mobile robot calibrated with the UMBmark calibration.

The University of Michigan Benchmark (UMBmark) test is especially designed and optimized for differential drive robots (like the TRC LabMate), but the method can be used for all robots. In the report we present the results from several different experiments in which different aspects of the odometry performance of six mobile robot (or robot configurations) were tested and compared. The six robots are:

1. TRC LabMate
2. Cybermotion K2A
3. CLAPPER (4-DOF platform)
4. Remotec Andros
5. Remotec Andros with "Basic Encoder Trailer"
6. TRC LabMate with Smart Encoder Trailer (simulation only)

Table of Contents

1. Introduction
1.1 Absolute Positioning Methods
1.2 The Importance of Dead-reckoning
2. Properties of Dead-reckoning Errors
2.1 Non-Systematic Dead-reckoning Errors
2.2 Systematic dead-reckoning errors
2.3 Definition of systematic dead-reckoning errors
3. Measuring Dead-reckoning Errors
3.1 The uni-directional square path as a benchmark test
3.2 The bi-directional square path experiment: "UMBmark"
3.4 Measuring Non-systematic Errors
3.5 Summary of the UMBmark Procedure
4. Measuring Systematic Errors Experiments
4.1 TRC LabMate
4.2 Cybermotion
4.4 Remotec Andros
4.5 Remotec Andros with Basic Encoder Trailer
4.6 Smart Encoder Trailer
4.6.1 Validity of the SET Simulation
4.6.2 Implementation Details of the SET Simulation
5. Measuring Non-systematic Errors Experiments
6. Correction of Systematic Dead-reckoning Errors
6.1 Analysis of Type A and Type B Errors
6.2 Compensation for systematic dead-reckoning errors
6.3 Correction of Systematic Errors
7. Conclusions
7.1 Measurement of Systematic Errors
7.2 Measurement of Non-systematic Errors
7.3 Correction of Non-systematic Errors
7.4 Future Work
8. References
Appendix A: The Effect of Unequal Wheel-diameters During Turning
Appendix B: Measurement and Correction of the Scaling Error E_s
Appendix C: UMBmark Experimental Results for Systematic Errors
Appendix D: Extended UMBmark Experimental Results for Non-systematic Errors
Appendix E: Correction of Systematic Errors Experimental Results

This file last updated on 7/4/96 by Johann Borenstein