THE NAVBELT - A COMPUTERIZED TRAVEL AID FOR THE BLIND

BASED ON MOBILE ROBOTICS TECHNOLOGY

by

Shraga Shoval1), Johann Borenstein2), and Yoram Koren3)

1) Faculty of Industrial Engineering & Management, Technion - Israel Institute of Technology, Haifa, Israel.

2, 3) Department of Mechanical Engineering and Applied Mechanics, The University of Michigan, 1101 Beal Avenue, Ann Arbor, Michigan 48109, USA.

2) Corresponding author. Email: johannb@umich.edu, Ph.: 313-763-1560, Fax: 313-944-1113.

ABSTRACT

This paper presents a new concept for a travel aid for the blind. A prototype device, called the NavBelt, was developed to test this concept. The device can be used as a primary or secondary aid, and consists of a portable computer, ultrasonic sensors, and stereophonic headphones. The computer applies navigation and obstacle avoidance technologies that were developed originally for mobile robots. The computer then uses a stereophonic imaging technique to process the signals from the ultrasonic sensors and relays their information to the user via stereophonic headphones. The user can interpret the information as an acoustic "picture" of the surroundings, or, depending on the operational mode, as the recommended travel direction. The acoustic signals are transmitted as discrete beeps or continuous sounds. Experimental results with the NavBelt simulator and a portable prototype show that users can travel safely in an unfamiliar and cluttered environment at speeds of up to 0.8 m/sec.

Key words: Auditory localization, binaural feedback, obstacle avoidance, travel aid for the blind, mobile robots, ultrasonic sensor.

I. INTRODUCTION

In order for a blind person to follow a particular route, the person must have some concept or plan of that route. The traveler can learn the route while being guided by a sighted escort, or may only have verbal instructions to go by. Once a route has been learned, successful travel requires that the individual be able to: (1) detect and avoid obstacles and (2) follow the route (i.e., to know their position and orientation and make necessary corrections). The performance of both tasks can be enhanced through electronic travel aids (ETAs).

In terms of operational principles, most ETA's are similar to radar systems: a laser or ultrasonic "beam" is emitted in a certain direction in space; the beam is reflected from objects it confronts on its way; a matching sensor detects the reflected beam and the distance to the object is calculated according to the time difference between emitting and receiving the beam. Existing ETA's can detect objects in the range of up to 15 feet away from the user, but require continuous scanning of the environment in the desired direction (with the exception of the Binaural Sonic Aid and the Pathsounder, which depends on head or torso movements).

During the past 30 years a number of ETA’s have been developed. Best known is the C5 Laser Cane [Benjamin et al., 1973], which is based on optical triangulation with three transmitters and three photodiodes as receivers. An UP channel detects obstacles at head-height, the FORWARD channel detects obstacles from the tip of the can forward, (in the range of 1.5-3.5m) and the DOWN channel detects drop-offs in front of the user. The Mowat Sensor [Pressey, 1977] is another hand-held device that informs the user of the distance to detected objects by means of tactile vibrations, where the frequency of the vibrations is inversely proportional to the distance between the sensor and the object. The Mowat sensor is a secondary aid for use in conjunction with a long cane or a guide dog. . The Mowat sensor has been found helpful, and users feel they benefit from it [Shao, 1985]. The Russell Pathsounder [Russell, 1965] is one of the earliest ultrasonic travel aids. Two ultrasonic transducers are mounted on a board that the user wears around the neck, at chest height. This unit provides only three discrete levels of feedback (series of clicks), roughly indicating distances to objects. The Pathsounder does not require active manual scanning of the environment by the user, but torso movement is the only search strategy potential [Mann, 1974]. The Binaural Sonic Aid (Sonicguide) [Kay, 1974] comes in the form of a pair of spectacle frames, with one ultrasonic wide-beam transmitter (55o cone) mounted between the spectacle lenses and one receiver on each side of the transmitter. Signals from the receivers are shifted and presented separately to the left and right ear. The resulting interaural amplitude difference allows the user to determine the direction of a reflected echo and thus of an obstacle. The distance to an object is encoded in the frequency of the demodulated low-frequency tone, which together with the wearer’s head orientation provides clear information about the object’s location. As the Sonicguide does not require active manual scanning, it can serve as a secondary device, in conjunction with an additional hand-held device or a guide dog.

Another type of travel aid devices are called Global Navigation Aids (GNAs). GNA systems are not concerned with local obstacle avoidance but rather with globally directing the user toward a desired target. These devices aim at providing the absolute position of the user (e.g., an intersection of two streets, an entrance to a building, a bus stop). Examples for GNAs are the Talking Signals [Brabyn, 1982], and the Sona System [Kelly and Ackerman, 1982]. Another device, developed at the University of California at Santa Barbara [Golledge et al., 1991] utilizes the Global Positioning System (GPS) which is based on radio signals from satellites and provides the traveler with updated information about the surroundings.

The motion of a blind person and a mobile robot is somewhat similar: both have the motoric ability to perform the motion, but require a detection system to detect obstacles along the travel path and avoid them. Application of a mobile robot obstacle avoidance system in a travel aid for the blind provides several advantages to the traveler. Using multiple ultrasonic sensors that face in different directions frees the user from the need to scan the surroundings manually. Although the Russell Pathsounder and the Sonicguide do not require manual scanning, their effective coverage of the surroundings depends on the orientation of the head or torso. Multiple sensors, on the other hand, can cover a large area, regardless of the user’s orientation. Furthermore, no additional measurement is required when an obstacle is detected, since its relevant dimensions (relative distance and azimuth) are determined simultaneously by the multi-sensor system. In addition, the obstacle avoidance system can guide the blind traveler around obstacles. This is particularly advantageous when traveling in a heavily cluttered environment such as crowded streets, office buildings etc. Based on the similarities between mobile robots and blind travelers, a concept for a travel aid for the blind - the NavBelt is suggested.

II. GENERAL CONCEPT OF THE NABELT

The principle of the NavBelt is based on transferring an advanced obstacle avoidance system [Borenstein and Koren, 1991], originally developed for mobile robots. The NavBelt consists of a belt, a small computer worn as a back pack, and equipped with ultrasonic sensors. The computer processes the signals arriving from the sensors, applies the obstacle avoidance algorithms, and relays them to the user via stereophonic headphones, using stereo imaging techniques.

One earlier idea for using mobile robotics technology to assist blind travelers was introduced at the Engineering Laboratory of Tsukuba Science Center - Japan [Tachi and Komoriya, 1984]. The device, called Guide Dog Robot, is a mobile robot equipped with ultrasonic sensors and cameras. A navigation map consisting of information about names of intersections and distances between them is stored in the robot's memory. Specific landmarks are selected for each intersection, and a vision system detects and identifies these landmarks. The robot's speed is adjusted according to the user's walking speed, and the information from the robot is transferred to the user by a speech synthesizer. The original Guide Dog Robot was not designed for obstacle detection and avoidance, thus the user was required to perform these tasks. The inherent practical limitation of a robotic guide is that mobile robots cannot climb up or down stairs, and they are too heavy to be carried by the user, when necessary.

The NavBelt system does not use a mobile robot, but rather transfers mobile robot technology to a portable device. This transfer is illustrated in Figure 1. The main difference is that the electrical signals, which originally guided a robot around obstacles are replaced by acoustic signals. However, the computation of the free path and the sensing techniques are similar in both applications.

Figure 1: The concept of the NavBelt

The NavBelt is equipped with an obstacle avoidance system (OAS), which scans the environment with several sensors simultaneously. The OAS employs a unique real time signal-processing algorithm to produce active guidance signals. One major difficulty in the use of multiple ultrasonic sensors is the fact that these sensors cause mutual interference. In the past, researchers had to employ slow firing schemes to allow each sensor's signal to dissipate before the next sensor was "fired." This problem was resolved by introducing the Error Eliminating Rapid Ultrasonic Firing (EERUF) method [Borenstein and Koren, 1992] which allows firing at a rate of up to 60 ms (i.e., each sensor fires once every 60 ms). This fast firing technique allows more efficient analysis of the sensors' readings. The algorithm rejects environmental ultrasonic noise, and filters out erroneous readings. The OAS computes the recommended traveling direction according to the user's current position, target location and the obstacles in the surroundings in a method called the Vector Field Histogram (VFH) [Borenstein and Koren, 1991]. In the absence of obstacles, the recommended direction is simply the direction toward the target. If, however, obstacles block the user's path, then the OAS computes an alternative path, which safely guides the user around the obstacles.

III. DESIGN

The user wears the NavBelt around the waist like a "fanny pack" (Figure 2), and carries a portable computer as a backpack. Eight ultrasonic sensors, each covering a sector of 15o are mounted on the front pack, providing a total scan sector of 120o. Small stereophonic headphones provide the user with the auditory data. A binaural feedback system (BFS) based on internal time difference (phase difference between the left and right ear) and amplitude difference (volume difference between the two ears) creates a virtual direction (i.e. an impression of directionality of virtual sound sources).

The use of auditory displays in travel aids for the blind has been investigated, especially with the development of electronic devices. An experimental device for testing the effect of various auditory signals provides eight methods for auditory display [Maure et. al., 1979]. The device consists of one ultrasonic sensor and a microcomputer. The computer processes the range readings, and according to the operational mode (selected manually by the user), transfers auditory signals to the user. These signals can be verbal messages (specifying the distance to an obstacle in inches), continuous tone sounds, in which the pitch is proportional or inversely proportional to the distance, audible alarms activated when the distance is smaller or larger than a specified value (Go/No Go detector), and others. Tachi et al., [1983] developed a method to quantitatively compare various auditory display schemes for tracking with an electronic travel aid. The performance of the traveler is evaluated by calculating the transfer function of the human in terms of effective gain and reciprocal time delay. It was found that monaural display with varying loudness and binaural display with varying position and loudness, are superior to other types of auditory displays. Fish [1976] suggested using a two dimensional coding system, which produces a series of tone bursts representing distances to objects. Although laboratory experiments showed that subjects were able to avoid obstacles using the two-dimensional auditory display, the information complexity and size of the system do not allow the traveler to walk at a reasonable speed. An interesting method for information transfer in a travel aid for the blind is based on the principle of echolocation of bats [Ifukube et al., 1991]. In this method, FM ultrasonic waves are transmitted to detect objects, and the reflected waves are picked up by two-channel receivers. The waves are then converted to acoustic signals with a simple proportional converter. The acoustic signals are presented to the blind traveler binaurally by headphones. Experiments showed that the method is very effective at detecting small objects, but no practical experiments were reported for implementing it in a travel aid. The Binaural Sensory Aid, also known as Sonicguide [Kay, 1974] relays information to the user by two sound sources to the left and right ears, using special tubes for minimal occlusion of external auditory cues. The interaural differences between the two sound sources provide the user with directional information about objects, as well as the object’s shape and even a rough estimate about the surface’s texture.

The NavBelt is designed for three basic operational modes, each offering a different type of assistance to the user:

  1. Guidance Mode - The acoustic signals actively guide the user around obstacles in pursuit of the target direction. The signals carry information regarding the recommended travel direction as well as speed and the proximity to obstacles. The signals consist of a single stereophonic tone, the direction of which determines the travel direction, while the frequency determines the recommended travel speed (higher frequencies for slower speeds). The speed is inversely proportional to the proximity to the nearest object. Using a keyboard, which can eventually be replaced by an acoustic coding system using other input devices suitable for blind travelers, the user enters the desired target position. The target can be selected as relative coordinates (i.e. 500 ft forward, turn right, etc.), or, when traveling in a known environment and the computer is equipped with a navigation map, the user can specify the target name (i.e. street corners, specific buildings etc). One problem with the Guidance Mode is that it requires knowledge about the user's momentary position at all times. In the current NavBelt prototype there are no sensors that can provide this information. However, developments in positioning method, mainly in satellite based systems (GPS) provide efficient solution to this problem. Golledge et al. [1991] developed a navigation aid based on GPS technology, which provides the user with updated information about the topographical features of the surroundings.
  2. Image Mode - This mode presents the user with a panoramic acoustic image of the environment. A sweep of stereophonic sounds appears to "travel" through the user's head from the right to the left ear. The direction to an object is indicated by the spatial direction of the signal, and the distance is represented by the signal's pitch and volume (higher pitch and volume for shorter distances). As the information in this mode is richer than the information in the Guidance mode, unnecessary information is suppressed and only the most important sections of the environment are transmitted to the user. The selection of these relevant sections from the panoramic map is performed by the computer based on the proximity of objects in the direction of travel. For example, when traveling in a crowded street or when entering a narrow passage, the computer transmits information about the sectors containing the closest objects to the user while ignoring all other, more distant objects.
  3. Directional Guidance Mode - This mode allows the user to control the global navigation while the obstacle avoidance is performed by the NavBelt. The system actively guides the user toward a temporary target, the location of which is determined by the user via a joystick. The joystick is only for use in the development stage, and needs to be replaced by a special auditory coding system or a speech control device. The target position is selected according to the direction of the joystick. When the joystick is not pressed, the system selects a default target five meters ahead of the user. If the traveler wishes to turn sideways, he or she presses the joystick in the desired direction and a momentary target is selected 5 meters ahead of the user in that direction. In case an obstacle is detected, the NavBelt provides the relevant information needed to avoid the obstacle with minimal deviation from the target direction.

The variety of operational modes allows for different levels of assistance to the user and for different information formats. The Guidance mode is the most "automated"' mode, since the majority of the perception and cognition tasks are performed by the computer. This mode is efficient when the user is traveling in an unknown cluttered environment and the NavBelt serves as the primary aid. In the Image mode the computer tasks are limited to scanning the surroundings and informing the user about the position of obstacles, while the global path planning and navigation tasks are performed by the user.

The Navbelt's acoustic imaging technique can produce several informative parameters:

Stereophonic displays have already been implemented in travel aids for the blind (e.g. the Sonicguide, [Kay, 1974]), using auditory localization technology ([Vinje and Pitkin, 1971], [Ericson and Mckinley, 1989], [Wenzel, 1990]). However there are two major differences between the use of auditory localization in the NavBelt and the Sonicguide:

  1. In the Guidance mode, the auditory cue signals the recommended travel direction, rather the location of an obstacle.
  2. In the Image mode, the stereophonic sweep provides a full panoramic virtual image of the surrounding, due to the wide coverage by the array of sensors (120o). Furthermore, when traveling in a cluttered environment the sensors can detect several objects simultaneously, providing sufficient information for traveling through doorways, narrow passages etc.

IV. the navbelt’s auditory dipslay of information describing the environment

4.1 Implementation of auditory Image signals

As previously mentioned, the Image mode provides the user with a panoramic auditory image of the surroundings. The principle is similar to the operation of a radar system (used in air traffic control, submarines etc.). An imaginary beam travels from the right side of the user to the left through the sectors covered by the NavBelt's sonars (a span of 120o and 5 m radius). A binaural feedback system invokes the impression of a virtual sound source moving with the beam from the right to the left ear in what we call a sweep. This is done in several discrete steps, corresponding to the discrete virtual direction steps. The angular displacement of the virtual sound source is obtained by a combination of the interaural phase and amplitude shift of the left and right signals. The phase shift is based on the different perception time of an auditory signal due to the different travel distance of the sound wave. The phase shift (in terms of time difference between left and right ears) is given by Eq. 1:

(1)

where K = 0.000666 sec is the time phase constant, and q is the angular position of the virtual source from the median plane in front of the user (see Fig. 3). The angular shift of a sound source due to the interaural amplitude difference is given by :

(2)

Figure 3: Angular displacement of auditory source

where AR and AL are the amplitudes to the right and left ears, K is the sensitivity factor and q 0 is a constant offset. Rowel [1970] shows that the sensitivity constant equals 2 for most audible frequencies. For the NavBelt we therefore assume K = 2 and q 0 = 0. The amplitude of the primary channel - the channel closest to the object is set according to the proximity to that object. For example, Fig. 3 shows a configuration where the amplitude of right channel - AR is set proportionally to the distance d. The amplitude of the left channel is therefore :

(3)

If no obstacles are detected by the virtual beam, the virtual sound source is of a low amplitude and barely audible. When obstacles are present, then the amplitude of the virtual sound source is louder. Figure 4 demonstrates the principle of the Image mode. Obstacles are detected by the ultrasonic sensors (Fig. 4a), and are projected onto a histogramic polar graph (Fig. 4b). Based on the polar graph, the binaural feedback system generates the sweep, which comprises of 12 steps (Fig. 4c). Each step "covers" a sector of 15o, so that the whole sweep covers a panorama of 180o. Each of the eight sectors in the center of the panorama (covering the sectors between 30o and 150o) is directly proportional to the corresponding sensor. The remaining four sectors (two at each side of the panorama) represent sectors which are not covered by the sonars. The value of these sectors is extrapolated based on the averaged values of adjoining sectors. For example, if the third and forth sector (representing the first and second sonar) contain an object, than the first and second sectors are automatically assigned the averaged value.

Each signal is modulated by an amplitude A (indicating the distance to the obstacle in that direction), the duration Ts, for which the square wave signal is audible, and the pitch f of the square wave. The spacing time Tn is the length of the interval between consecutive signals during a sweep. After each sweep there is a pause of duration Tc, to allow the user to comprehend the conveyed image. Many meaningful combinations of these parameters are possible. For example, because of the short-term memory capability of the human ear, a sweep may be as short as 0.5 sec. Given enough cognition time Tc, the user will comprehend the image. Alternatively, the sweep time may be as long as one second, combined with a very short cognition time. Notice that each sweep starts with an anchor signal. This signal has a unique pitch, which provides the user with a convenient marker of the start of a sweep.

Figure 4: The Image mode. Obstacles are detected by the ultrasonic sensors (a), projected onto the polar graph (b), and an acoustic sweep is generated (c).

One of the important features of the Image mode is the acoustic directional intensity (ADI), which is directly derived from the polar obstacle density histogram. The virtual direction of the ADI provides information about the source of the auditory signal in space, indicating the location of an object. The intensity of the signals is proportional to the size of the object and its distance from the traveler.

The directional intensity is a combination of the signal duration Ts, the amplitude A, and the pitch. Experiments with human auditory perception show [Benson, 1986] that the perceived intensity increases with the signal's amplitude, pitch and duration. Adjusting the acoustic directional intensity according to the location of obstacles in the surroundings attracts the user's attention to the most relevant sections in the environment, while suppressing irrelevant data.

4.2 Implementation of auditory guidance signals

Implementing the Guidance mode in the NavBelt is simpler than the Image mode, since the amount of transferred information is far smaller. In the Guidance mode the computer provides the user only with the recommended travel speed and direction, based on the obstacle avoidance algorithm. The computation of the recommended travel speed and direction is similar to the computation of these parameters for a mobile robot traveling in a cluttered environment as determined by the VFH [Borenstein and Koren, 1991]. The VFH method calculates the travel direction from the polar histogram map by searching for sections with small obstacle density. In practice, the FVH determines a threshold level, and all sections with a lower obstacle density than the threshold level become candidate sections. Next, the VFH searches for the candidate section that coincides with the direction of the target. If none is found, the VFH searches for the a candidate section that is the closest (in terms of angular position) to the target direction. The travel speed is determined by the VFH according to the proximity to the nearest object. The speed is determined inversely proportional to that proximity, with a maximum speed of 1.2 m/sec attained when the distance between the traveler and the closest object is more than 3 meters.

The recommended travel speed and direction are relayed to the user by a single stereophonic signal. The virtual direction of the signal is the direction the obstacle avoidance system has selected for travel. The pitch and amplitude are proportional to the recommended travel speed. Higher pitch and amplitude attract more human attention [Benson, 1986], thereby motivating the traveler to reduce the walking speed and to concentrate on the stereophonic signal. A special low pitch signal (250 Hz) is transmitted when the direction of motion coincides (within ± 5o) with the required direction. This special tone is a simple feedback signal for the user, indicating that the travel direction is correct. Furthermore, low pitch tones occlude external sound from the environment less than medium and high pitch tones [Benson, 1986]. The higher pitch tone is transmitted only when the traveler needs to change direction, with the low pitch returning once the recommended direction is achieved.

Another important parameter involved in the Guidance mode is the rate at which signals are transmitted. Although a low transmission rate causes less occlusion of external sounds, it may also be too slow to alert the traveler to hazards. An adaptive information transfer system adjusts the transmission rate according to changes in the process and the user's requirements, similar to the way the information flow is adjusted in the Image mode. When the user is traveling in an unfamiliar environment cluttered with a large number of obstacles, the transmission rate increases, and may reach up to 10 signals per second. On the other hand, when traveling in an environment with little or no obstacles, the transmission rate is reduced to one signal every three seconds.

 

V. EXPERIMENTS WITH THE NAVBELT SIMULATOR

In addition to the NavBelt prototype, a simulator was developed. The same acoustic signals that guide the user in the NavBelt prototype are used in the simulator. The user's response to these signals are relayed to the computer by the joystick. Several maps are stored in the computer representing different types of environments (e.g. crowded streets, corridors, narrow passages). Some of the maps were constructed by gathering real data from a mobile robot recording the sonar data while traveling, while other maps were generated by the computer. A schematic description of the simulator is shown in Fig. 5.

The simulator is well suited to investigate the effect of different auditory signals on human performance, and it is also a very efficient training tool for new users. The simulator can train people by providing a range of different environments with no risk of actual collisions. The user's performance can be recorded easily and used for analysis of individual progress, as well as the evaluation of the effectiveness of certain training procedures. McEntire [1971] used a similar simulator in his research of static and dynamic tactile displays for blind travelers, using the phantom sensation. In this experiment, random locations within a triangular shape (similar to the area covered by the long cane) were displayed to the subjects by a triangular skin stimulator. Subjects relayed the perceived sensation using a joystick, and the effect of different parameters (stimulator spacing, frequency, pulse rate, and length) were analyzed by the computer.

5.1 Transient response in auditory localization

In the first experiment the transient response of humans in tracking by auditory localization was investigated. Stereophonic signals were randomly generated by the computer, varying in their virtual direction, pitch, amplitude, length, and the rate at which they were transmitted. The subject's goal was to tilt the joystick in the direction of the virtual sound source. Eight subjects were included in this experiment, all sighted with good hearing capabilities. The subjects' ages ranged from 18-35. Each session included 5 minutes of practice, 15 minutes period of experiments, with 10 minute rest intervals. Each subject was tested in 400 runs. The parameters involved in transferring the stereophonic signals were selected randomly by the computer to reduce the effect of learning or getting used to a particular format of information. However, only two parameters were changed in each test. One parameter was the virtual direction, and the other was selected randomly by the computer (pitch, amplitude, length or transmission rate). Before each test, a visual test, in which signals were randomly displayed on the monitor. Again, the goal was to tilt the joystick in the direction of the visual signal. Performance (in terms of reaction time and accuracy) was recorded and then subtracted from the results of the auditory tests. This way the effects of the auditory display were isolated from other effects (joystick accuracy and delays, neuromuscular delays etc.) , similar to McEntire’s experiment on self-paced tracking [McEntire, 1971].

The results from all subjects were combined and then classified according to the different variables. Figure 6 describes some of the results from these tests.

Figure 6: The effect of changes in pitch on transient response

Discussion

According to the results, two parameters have a major effect on the localization error: the signal's frequency and the signal's amplitude. The localization error is smaller for frequencies between 400-1000 Hz with the minimal error around 800 Hz. This result is consistent with a similar experiment [Ericson and McKinley, 1989] investigating the performance of pilots in localization of auditory signals in the cockpit. Our results are also consistent with theoretical research [Benson, 1986; Yost and Watson, 1987], which concluded that low frequencies (below 2 kHz) contribute mainly to a sense of localization, while higher frequencies contribute more to the broadening of the auditory event. The lower error for lower amplitudes is also consistent with experiments performed by Benson [1986], which showed that the stereophonic localization is better perceived for low amplitudes (less than 20 dB).

According to our results, the reaction time (RT) is affected mainly by the transmission rate. The RT is kept constant around 700 ms for all signals' amplitudes and frequencies, increased slightly for transmission rate between 5-2 Hz (to 800 ms), and from there on increased significantly with lower transmission rate. This result is expected since with low transmission rates there are long delays between each transmission.

5.2 Performance in the presence of a single obstacle

The motivation behind this experiment was to investigate the human performance for basic obstacle configuration. The experiment was conducted with the NavBelt simulator using a simple single object. For each run, the computer generated simulated obstacle at a random location and at a random time. The obstacle was wide enough to block the path between the subject and the target, therefore requiring a change in the travel direction. Information was relayed to the subjects in both Image and Guidance modes. Also, a visual Image mode, in which target and obstacle positions were displayed on the screen, was included in the test. The results of the visual test were used as a benchmark for comparison with the results of the auditory displays. Travel parameters such as travel speed and reaction times were continuously recorded. In addition, the percentage of unsuccessful trails (the percentage of collisions with obstacles) were also recorded. Fig. 7 shows the results of this experiment.

Figure 7: Experimental results for a single obstacle

Discussion

The most obvious conclusion from this experiment is that none of the auditory displays is as efficient as the visual display for obstacle avoidance. The Guidance mode, had three times more collisions than the visual display (18% vs. 6%) and the image display had more than four times more collisions (27% vs. 6%). Also, the difference in reaction time between the visual and auditory displays has an increase of 0.9 seconds, compared with an increase of 0.25 seconds for the Guidance mode. The increase in reaction time is due to the increased transmission time required in the Image mode, as well as the longer time required to comprehend these signals. However, the display mode has a less significant effect on average travel speeds (1.02 m/sec for visual display, 0.85 m/sec for the Image mode and 0.96 m/sec for the Guidance mode).

5.3 Performance in different environments

To investigate the concept of the NavBelt in different environments, virtual maps were constructed by the computer. Ten maps were stored in the computer, and selected randomly to reduce the chances of subjects becoming familiar with the maps. For each run the initial and target positions were also selected randomly by the computer, to simulate travel in an unfamiliar environment. In the Guidance mode the user's position and recent travel path were displayed on the monitor. The target position and objects along the travel path are not displayed, forcing the subject to rely entirely on the acoustic guiding signals. The reason for displaying the subject's position and recent travel path was based on experiments that showed [Strelow, 1985] that humans have spatial orientation memory of several seconds. Unless totally lost, humans have a reasonable estimate of their position and orientation based on natural navigation capabilities, even without visual perception. In the Image mode, in addition to the user's position, the target position was also shown on the monitor (as this mode did not guide the user toward a specific location). The goal was to "travel" from the current position to the target shown on the screen. As with the Guidance mode, the obstacles along the travel path were not shown, and the subject avoided obstacles based on the NavBelt’s acoustic panoramic display only.

Four sighted subjects with substantial practice using the NavBelt were selected for this experiment. To avoid fatigue, each session started with 5 minutes of practice, 15 minutes of running the test, and 10 minutes of rest. The experiment involved 300 tests for each subject in which the Guidance and Image modes were tested. Since the maps were different in their level of complexity (in terms of obstacle density and layout), each map was tested at least 10 times in each display mode. The average speed and the average deviation from the recommended travel direction were measured for each test. These figures indicate how closely the subject followed the recommended travel path.

The results indicated that the fastest average speed was achieved using the Guidance mode in the obstacle course shown in Figure 8, traveling at 0.95 m/sec, with an average deviation from the recommended path of 2.6'. The slowest speed was in the Image mode traveling through the obstacle course shown in Figure 9, traveling at 0.43 m/sec with a deviation of 11.9'. The obstacle course of Figure 8 represents an open space with few obstacles, while the obstacle course of Figure 9 represents a crowded street with many obstacles randomly positioned. As expected, the average travel speed for all the maps was higher in the Guidance mode (0.78 m/sec), compared with the Image mode (0.52 m/sec). The numbers next to the travel path in these figures represent the time taken to complete the path to date point.

 

Figure 8: The easiest-to-traverse obstacle Figure 9: The hardest-to-traverse obstacle

course in the practice simulator. course in the practice simulator.

Discussion

Analysis of the performance results indicates that the NavBelt is particularly effective in environments with low obstacle density. This can be explained by the wide and long detection range of the NavBelt. Research on the effect of non-visual preview upon the walking speed of visually impaired people [Clark-Carter et al., 1986] shows that a preview of 3.5 meters using a Sonic Pathsounder [Kay, 1974] increased walking speed by 18% compared with travel speed with the long cane. The wide detection range of the NavBelt increases the user's confidence resulting in a higher travel speed. The NavBelt can provide not only long, but also wide preview, thus providing the traveler with extra reaction time to avoid obstacles. Figures 10 and 11 show two other maps used in these experiments.

 

Figure 10: Office building-like obstacle course Figure 11: Narrow-passage obstacle course.

VI. EXPERIMENTAL RESULTS WITH THE NAVBELT PROTOTYPE

This section describes the tests conducted with the actual NavBelt prototype. The experiments were performed by four sighted subjects, and were conducted in the controlled environment of a laboratory under the supervision of a sighted person. Before traveling with the experimental prototype, subjects had at least 10 hours training with the simulator. In addition, subjects initially traveled with the NavBelt with their eyes open in order to better comprehend the acoustic signals. The experiments with the NavBelt prototype included optimization of the sonar range and an investigation of the NavBelt performance for different types of obstacles. Obviously there are many differences between blind and sighted subjects, particularly when sighted subjects are trained while using their sight. However these experiments aimed at testing the concept of the NavBelt as a mobility aid. More experiments with blind subjects traveling in real environments are required.

6.1 Optimization of sonar range

The ultrasonic sensors used by the NavBelt can detect objects in the range of 0.3 to 10 meters. Selecting the sonars' detection range affects two major parameters:

In the experiments with the NavBelt prototype several sonar ranges were examined. In the first experiment the sonar range was set to 2 meters. Although the readings were very reliable, subjects found the warning period too short, and in some cases, mainly with the Image mode, subjects collided with obstacles. Better results were obtained by setting the ultrasonic range in the NavBelt to 3 meters. This range provided reliable readings and sufficient warning. Although this range is shorter than the one found by Clark-Carter [1986], the statistical analysis performed by the obstacle avoidance system compensates for the shorter range and provides reliable and sufficient warning. Setting a longer sonar range resulted in noisy data (mainly due to "crosstalk" between sensors) which reduces its reliability.

The implementation of the EERUF method for controlling the firing time of the ultrasonic sensors significantly reduced the noisy data. In one experiment, an ultrasonic "noise maker" generated ultrasonic noise at the same frequency as that used by the NavBelt. Without the implementation of ERRUF, the NavBelt could not provide reliable data for safe travel. However travel with ERRUF was not affected by the ultrasonic noise. This is particularly advantageous in real environments where ultrasonic noise can be generated by electric power lines, electric motors or reflections from smooth surfaces. Furthermore, the ability of the NavBelt to operate reliably in a noisy environment can enable multiple NavBelt systems to operate simultaneously in close proximity to each other.

6.2 Experiments with real obstacles

In this experiment, subjects walked through laboratory obstacle courses comprising various types of objects, using various operation modes. In the first experiment several vertical poles with different diameters were positioned along the travel path. It was found that the NavBelt can detect objects as narrow as 10 mm. However, this can be done only if the objects are stationary and the subject is walking slowly (less than 0.4 m/sec). It was also found that the NavBelt can reliably detect objects with a diameter of 10 cm or bigger, regardless of the travel speed. Other tests were conducted inside office buildings where subjects traveled along corridors, located doors and curves, and detected and avoided furniture.

In other experiments subjects traveled outside buildings, detecting and avoiding common objects such as trees and large bushes, parked cars, walls, bicycles, and other pedestrians. One major concern of users was the inability of the current prototype NavBelt to detect overhanging objects, up- and down-steps, sidewalk edges, etc. Future improvements to the NavBelt will require the addition of sonars pointing up and down to detect these type of obstacles.

6.3 Experiments with different walking patterns

The next experiment tested the NavBelt in terms of walking patterns. It was found that uneven walking patterns cause the sonars to move along the vertical plane (sonars swinging up and down), which reduces the reliability of the sonar data. In addition, it was found that the relative angle between the sonars and the vertical orientation of the NavBelt (the angle of the sonars with the horizon) affects object detection. For example, if the NavBelt is tilted by ±50 from the horizon, the sonar reading can be off by more than 9%. Swinging the arms during normal walking pattern did not interfere with the sonar performance as no sonars are directed to the sides. However, using the White Cane (the most common device used by blind travelers) can cause interference to the sonar performance, mainly when it is used to detect objects above the ground level (higher than 0.5 m). However, since the cane is used mainly to detect objects at ground level, while the NavBelt is designed to detect objects above ground level, this interference is not critical to the general performance.

The experiments with the NavBelt prototype showed the importance of training. Subjects with more experience traveled faster and generally were more comfortable. After 20 hours of practice with the NavBelt simulator and 40 hours practice with the prototype subject traveled at 0.8 m/sec in the Guidance mode and 0.5 in/sec in the Image mode. Subjects with less experience (10 hours with the simulator and 10 hours with the prototype) traveled at an average speed of 0.6 m/sec in the Guidance mode and 0.3 m/sec in the Image mode.

VII. CONCLUSIONS

A new concept for a travel aid for the blind – the NavBelt, was presented. This concept, based on technology originally developed for mobile robots, integrates fast and reliable obstacle detection with obstacle avoidance technology. The NavBelt is designed to offer three operational modes, each providing a different level of assistance and requiring a different level of conscious effort from the user. The computer generates a reliable real time representation of the environment and relays it to the user by transmitting stereophonic signals.

Preliminary experiments with the simulator and prototype show that the information generated by the NavBelt can guide users safely around obstacles or can present a reliable acoustic panoramic image of the surroundings, which can assist in avoiding obstacles.

The following modifications are required before the NavBelt can be tested by blind subjects in real world conditions.

VII. REFERENCES

  1. Benjamine J. M., Ali, N. A., Schepis, A. F., 1973, "A Laser Cane for the Blind," Proceedings of the San Diego Biomedical Symposium, Vol., 12, pp. 5 3 - 5 7.
  2. Benson K. B., 1986, "Audio Engineering Handbook," McGraw-Hill Book Company, New York.
  3. Blasch B. B., Long R. G., Griffin-Shriley N., 1989, "National Evaluation of Electronic Travel Aid for the Blind and Visually Impaired Individuals: Implications and Design", RESNA ]12th Annual Conference, New Orleans, Louisiana, pp. 133-134.
  4. Borenstein, J. and Koren, Y., 1991, "The Vector Field Histogram - Fast Obstacle Avoidance for Mobile Robots." IEEE Journal of Robotics and Automation, Vol. 7, No. 3., June, pp. 278-288.
  5. Borenstein, J. and Koren, Y., 1995, "Error Eliminating Rapid Ultrasonic Firing for Mobile Robot Obstacle Avoidance." IEEE Transactions on Robotics and automation, February, Vol. 11, No. 1, pp. 132-138.
  6. Brabin, J. A., 1982, "New Developments in Mobility and Orientation Aids for the Blind," IEEE Transaction on Biomedical Engineering, Vol. BAM-29, No. 4, pp. 285-290.
  7. Clark-Carter D. D., Heyes A. D., Howarth C. I., 1986, "The Efficiency and Walking Speed of Visually Impaired People," Ergonomics, Vol. 29, No. 12, pp. 1575-15 8 1.
  8. Ericson M. A., Mckinley R. L., 1989, 'Auditory Localization Cue Synthesis and Human Performance, " IEEE Proceedings of the National Aerospace and Electronics Conference, V2, pp. 718-725, NASEA9.
  9. Fish R. M., "An Audio Display for the Blind", IEEE Transactions on Biomedical Engineering, Vol. BME-23, No. 2, March, pp. 144-153.
  10. Golledge R. G., Loomis J. M. Klatzky R. L., Flury A., Yang X. L., 1991, "Designing a Personal Guidance System to Aid Navigation Without Sight: Progress on GIS Component", International Journal of Geographical Information Systems, Vol. 5, No. 4, pp. 3 73 -3 95.
  11. Ifukube T., Sasaki T., Peng C., 1991, "A Blind Mobility Aid Modeled After Echolocation of Bats", IEEE Transactions on Biomedical Engineering, Vol. 38, No. 5, May, pp. 461-465.
  12. Kay L., 1974, "A Sonar Aid to Enhance Spatial Perception of the Blind: Engineering Design and Evaluation," Radio and Electronic Engineer, Vol. 44, No. 11, pp. 605-627.
  13. Kelly G. W., Ackerman T., 1982, "Sona, The Sonic Orientation and Navigational Aid for the Visually Impaired," Fifth Annual Conference on Rehabilitation Engineering, Houston, Texas, p. 72.
  14. Mann R. W., 1974, "Technology and Human Rehabilitation: Prostheses for Sensory Rehabilitation and/or Sensory Substitution", from Advances in Biomedical Engineering, Vol. 4, Academic Press, Inc. pp. 209-353.
  15. Maure D. R., Mellor, C. M., Uslan M., 1979, "AFB's Computerized Travel Aid," Visual Impairment and Blindness, November issue, pp. 380-381.
  16. McEntire R., 1971, "Information Rate via Vibro-Tactile, Two Dimensional ‘Phantom Sensation", D. Sc. Thesis, Dept. of Mechanical Engineering, MIT, Cambridge, Mass.
  17. Pressey N., 1977, "Mowat Sensor," Focus, Vol. 11, No. 3, pp. 35-39.
  18. Rowel D., 1970, "Auditory Factors in the Design of a Binaural Sensory Aid for the Blind", Ph.D. Theses, University of Canterbury, New Zealand.
  19. Russell, L., 1965, "Travel Path Sounder," Proceedings of Rotterdam Mobility Res. Conference, New York: American Foundation for the Blind.
  20. Shao, S., 1985, "Mobility Aids for the Blind," Electronic Devices for Rehabilitation, John Wiley & Sons, New York, New York, pp. 79-100.
  21. Strelow E. R., 1985, "What is Needed for a Theory of Mobility: Direct Perception and Cognitive Maps - Lessons From the Blind," Psychological Review, Vol. 92, No. 2, pp. 226-24 8.
  22. Tachi S., Komoriya K., 1984, "Guide Dog Robot", Proceeding of the 2nd International Congress on Robotics Research, Kyoto, Japan.
  23. Tachi S., Mann R. W., Rowel D., "Quantitative Comparison of Alternative Sensory Displays for Mobility Aids for the Blind", IEEE Transactions on Biomedical Engineering, Vol. BMIE-30, No. 9, September 1983, pp. 571-577.
  24. Vinje, E. W., Pitkin E., T., 1971, "Human Operator Dynamics for Aural Compensatory Tracking," 7th Annual Conference on Manual Control, Los Angeles, California.
  25. Wenzel E. M., 1990, "Localization in Virtual Acoustic Display,' Human Interface Research Branch, Aerospace Human Factors Research Division, NASA Ames Research Center, MS-262, Presence: Volume 1, Number 1.
  26. Yost, W. Y., Watson C. S., 1987, "Auditory Processing of Complex Sounds", Lawrencew Erlbaum Associate, Publishers, Hillsdale New Jersey.