вход по аккаунту



код для вставкиСкачать
An Assistive Mobile System Supporting Blind and Visual Impaired
People when Are Outdoor
T. Valls Mataró, F. Masulli, S. Rovetta, A. Cabri, C. Traverso, E. Capris, S. Torretta
Abstract— In this paper we present the TARSIUS system,
based on mobile technology and aimed at enhancing visuallyimpaired and blind people’s capabilities in visual scene understanding and geolocation while are outdoor. The system components are the TARSIUS app for mobile devices, a web server,
and the Remote Assistance Center. Its interface is optimized
for the perceptual characteristics of its users. Moreover, the
TARSIUS navigation sub-system not only leverages the GPS
system, but also Bluetooth LE/iBeacon tags placed along the
streets at points of interest and dangerous paths and areas.
CCORDING to the World Health Organization [10],
[11], 285 million people are estimated to be visually
impaired worldwide: 39 million are blind and 246 have
low vision. However, refractive error as a cause of visual
impairment was not included, which implies that the actual
magnitude of visual impairment is higher.
More than 82% of blind people are 50 years or older,
although they represent only 19% of the world’s population.
With an increasing elderly population in many countries,
even more people will be at risk of visual impairment
due to chronic eye diseases and ageing processes. Visual
impairment is not distributed uniformly throughout the world.
More than 90% of the world’s visually impaired live in
low-income settings and mostly in developing countries.
Moreover, females have a significantly higher risk of being visually impaired than males. Worldwide, 19 millions
children are estimated to be visually impaired; of these, 12
millions children are visually impaired due to refractive errors, a condition that could be easily diagnosed and corrected.
Eventually 1.4 million are irreversibly blind for the rest of
their lives and need visual rehabilitation interventions for a
full psychological and personal development.
The principal causes of visual impairment are: (a) uncorrected refractive errors: myopia, hyperopia or astigmatism
(43 %), (b) unoperated cataract (33%), and (c) glaucoma
(2%). Globally, 80% of all visual impairment can be prevented or cured.
Nearly one-third of people with vision loss suffer from
clinical depression, a rate that is twice as high as the general
Authors affiliations: Toni Valls Mataró(1), Francesco Masulli(2), Stefano Rovetta(2), Alberto Cabri(2), Carlo Traverso(3), Elisabetta Capris(4),
Simone Torretta(4). (1) Cultural Association trescucarachas, Barcelona,
Spain (email: [email protected]); (2) Dipartimento di Informatica, Bioingegneria, Robotica e Ingegneria dei Sistemi (DIBRIS) Università di Genova, Genoa, Italy (email: [email protected], [email protected], [email protected]); (3) Dipartimento di Neuroscienze, Riabilitazione, Oftalmologia, Genetica e Scienze MaternoInfantili (DINOGMI) Università di Genova, Genoa, Italy (email:
[email protected]); (4) David Chiossone Onlus per Ciechi e Ipovedenti,
Genoa, Italy (email: [email protected], [email protected]).
978-1-5386-3906-1/17/$31.00 ©2017 IEEE
population of the same age [6]. Moreover, injuries from falls
are common with visually impaired patients, and they tend
to stay in the hospital nearly 2.5 days longer than patients
with normal vision [8].
Overall, visual impairment worldwide has decreased since
1990s, despite an ageing global elderly population. This
decrease is mainly the result of a reduction in visual impairment from infectious diseases through: (a) overall socioeconomic development; (b) concerted public health action;
(c) increased availability of eye care services; (d) awareness
of the general population about solutions to the problems
related to visual impairment (surgery, refraction devices,
etc.). Nevertheless, the worldwide direct and indirect costs
of visual impairment are growing, as shown in Fig. 1.
The population of people with visual disabilities is very
heterogeneous, with different needs and degrees of reduction
of autonomy. For example:
• Less than 20% of people with visual impairments understand the Braille code [2];
• Many visually impaired people don’t like to use the
white cane or the guide dog for social inclusion reasons;
• In situations of confusion, due, e.g., to construction of
a new road or to the change of location of a bus stop,
the visually impaired person needs actual help;
• In severe cases of peripheral vision loss, but with central
residual vision, the use of a wide angle lens can increase
the field of view up to a 30%;
• Portable digital magnifiers allow to freeze the complete
picture of the scene on the screen that later can be
analyzed by the low vision user using personalized color
filters to enhance the contrast;
• 52% of the visually impaired people with age between
1 Data concerning the members of the Organizaciòn Nacional de Ciegos
Fig. 1.
Worldwide costs of visual impairment (from [1]).
Fig. 2.
Some apps/systems for blind and visually impaired people
17 and 64 are using or are expected to use in the
near future smartphones or tablets. About 80% of them
have a residual visual functionality sufficient for moving
away from home, for housework and for reading. These
people are interested in using mobile devices in daily
outdoor life both as telephone (essential for receiving
help in case of difficulty) and for further support through
an app2 , though at present there are still no effectively
useful apps.
At present few apps for blind and visual impaired people
are available, and usually are aimed at supporting one mobile
device function only.
Some of them are part of the operating systems accessibility services, including: (a) screen readers that let the
user know by vocal messages what is happening on the
touch-screen; (b) readers for emails, web pages, and books;
(c) intelligent assistants, working as vocal calendar, placing
phone calls, configuring other apps; (d) dictaphones converting user’s words into text; (e) apps for font adjustment, screen
zooming, and screen color management.
Additional apps, to be installed by the user, provide
assistive tools to: (a) support incoming and outgoing phone
calls; (b) provide aid to navigation in town streets; (c) turn
the phone in a digital magnifying glass equipped with a
flashlight; (d) read plates and billboards by OCR; (e) quickly
call taxis or emergency services.
Among the available apps/systems for blind and visually
impaired people, some of the more interesting are (see
Fig. 2):
• Loowi suite (Android) that gives access to the basic
functions of the smartphone in an intuitive and easy way
2 A (mobile) app is a software designed to run on mobile handsets such
as smartphones and tablets.
with enhanced graphics, voice information and vibration
system, and among its features and functionalities includes the possibility to obtain the user position through
GPS (geolocation).
Ariadne GPS (iOS) that includes talking maps allowing
the user to explore the surrounding world by moving
her/his finger around the map, haptic feedback while
the users is crossing a street, and bus or train stops
GeorgiePhone app family (Android) that, in additions
to the features included in Loowi suite and Ariadne
GPS and others, include a GPS navigator using step by
step spoken or on-screen interaction, an app for finding
points of interest close to the user, a helper activating
the remote assistance that sends a text message to carers
and friends connected via an ad-hoc app, an app for bus
stops reaching, bus approaching selection, and bus get
off, and an app for quick taxi calling.
GoAll app (iOS and Android) that allows deaf-blind
people to receive and enjoy television content without
intermediaries. It is based on the PervasiveSUBsoftwarethat compiles all the subtitles of television channels
and sends them to a central server. The GoAll app
retrives the subtitles from the central server and can
sent tehm to the braille line of the deaf-blind person.
Sesamonet (Window Mobile) app that includes a
database with information on the location and makes
use of RFID (Radio Frequency Identification) passive
transponders (no requirement of power supply) technology to create a path guiding visually impaired persons through a location. The walking stick includes an
embedded antenna (with a bluetooth transmitter) which
detects/reads the RFID transponders and the app smart-
Normal vision
Blurred vision
Loss of peripheral vision
Loss of central vision
Fig. 3.
Normal and pathological vision (simulation).
phone a database.
XSIGHT Smart glasses that detect nearby obstacles and
present them in a simple high contrast way, including an
infra-red projector, an Infra-red camera a depth camera,
and Organic Light Emitting Diodes (OLED) lens.
Eyra Horus wearable assistant that recognizes faces, objects, obstacles and describes the world. It is composed
by two cameras, a bone conduction headset and GPUaccelerated computer vision using deep learning.
Some features here described are useful for supporting
blind and visually impaired people walking alone in the
streets, but it is necessary to better integrate them to improve
their usability. Moreover, the interfaces should be optimized
for the specific perceptual characteristics of the users and
the navigation system should be more resilient and precise,
as at present it works only when a good GPS connection is
present in addition to Internet connectivity.
In this paper we present the TARSIUS3 project aimed
at the development of an ICT assistive infrastructure
for visually impaired and blind people support based
on mobile technology (smartphone e tablet) to enhance
their ability to understand a visual scene and to orientate while are outdoor. The project, that is in the initial phase of development, involves at present the University of Genoa (, the Istituto David
Chiossone (, the Unione Italiana
dei Ciechi e degli Ipovedenti (, and
the Associazione per la Retinite Pigmentosa RP Liguria
( Additional partners are welcome to
join to the working group.
In the remainder of this paper, Section II presents the
apps currently available for the blind and visually impaired
people; Section II discuss some perceptual constraints to be
considered for software design; then in Section IV analyzes
the possible low-cost radio-tagging technologies that can be
3 The Tarsius is a small Asian primate that has evolved strengthening its
night and twilight vision.
exploited in the geolocation and navigation functions of our
system; Section V presents the TARSIUS system; discussion
and conclusions are in Section VI.
The spectrum of visual impairment includes a number
of conditions with diverse characteristics. In general it is
necessary to distinguish among the following situations:
Complete blindness: the person does not perceive any
visual stimuli;
Legal blindness: there is a residual vision below a
conventionally established minimum;
Low vision: there is a deficient visual capacity in one
or more aspects, for example in the extension the visual
Fig. 3 shows a simulation of the visual field of people with
normal vision, compared to some visual pathologies, like
blurred vision, loss of peripheral vision and loss of central
In people with low vision, the optimal visual contrast
can be obtained with a combination of colors that varies
individual from individual [7]. Fig. 4 shows the output of
some filters that may give the optimal color contrast for a
particular individual.
An app for blind and visually impaired people support
should included an initialization phase for detecting the
optimal color contrasts and the filter that will be used in
the supportive functions.
Another issue concerning the user’s perceptual system
is the usage of voice systems for the user interface. It is
important to avoid using standard headphones or earphones
that isolate the user from the environmental sounds and
voices which are the main informative channel for blind
or visually impaired people. Bone conduction headphones
permit to receive both audio channels, but force the user to
carry an additional item.
Fig. 4.
Examples of personalized filters for enhancing vision.
Differentiated mobile vibrations, coding haptic information, can be fruitfully exploited for many simple man/mobilehandset communication tasks.
The Global Positioning System (GPS) is a space-based
navigation system used for car navigators that provides
location and time information in all weather conditions,
anywhere on or near the Earth where there is an unobstructed
line of sight to four or more GPS satellites.
The precision of the GPS system is limited to an interval
of one or few meters [12]. A navigation system supporting
blind and visually impaired individuals when moving outdoor
in a town must therefore integrate some ad-hoc short-range
navigation system. One solution is radio tags positioned close
or on the points of interest (e.g., shops) and dangers (e.g.,
roadworks) by the local government or by public and private
organizations and individual citizens. The possible low-cost
short-range radio-tagging technologies that can be massively
distributed to this aim in the urban scene are:
• Radio-frequency identification (RFID) transponders,
wireless access tags attached to objects [5].
• Near field communication (NFC), radio electronic devices that establish communication when they are to
a distance of 10 cm or less [3], based on the RFID
• Bluetooth low energy (Bluetooth LE) that is a wireless
personal area network technology intended to provide,
at a considerably reduced power consumption and cost,
a communication range up a maximum of 100 m [9].
• iBeacon (beacon) that is a class of Bluetooth LE devices
that broadcast their identifier to nearby portable electronic devices and are optimized for localization tasks.
They usually have the size of a 2 Euro coin and their
button battery, their largest-sized part, can last up to two
year [4].
Generic RFID transponders are not integrated in most mobile handsets, while NFC and Bluetooth LE are embedded in
most recent smartphones. Given their communication range
and low energy consumption, Bluetooth LE and iBeacon
are the most promising radio-tagging technologies to be
integrated with GPS in navigator systems for visual impaired
The goal of the work described here is to design and deploy an integrated system that includes a complete selection
of the functionalities described in the previous paragraphs,
both for exploration and for navigation, giving access to all
of them through a common interface.
The TARSIUS system is based on the following components: (A) TARSIUS app for mobile devices, providing the
main interaction with the system; (B) Web Server; (C) Remote Assistance Center (social network). These components
will be described in the following subsections.
In the design of the TARSIUS System particular care
is dedicated to the design of a ”user friendly” interface
that takes into account the peculiarities of the users. The
functional buttons are placed at the borders of the touchscreen, with the most important ones placed close to the
corners, and the SETUP button centered on the screen.
The man-device interaction occurs through tactile feedback,
sounds and speech for all visually impaired users.
For each button a differentiated feedback is provided, that
can consists in audio (sound and voice) or as vibration
patterns. Audio and vibration feedbacks can be enabled or
disabled independently.
It is important to highlight that even the haptic feedback
gives a different vibration response depending on the feature
associated to the button on the touch-screen.
(a) A pharmacy and a post-box
(b) A bus coming
(c) Waste containers: paper, glass, plastic
(d) Roadwork
Fig. 5.
Outdoor scenarios and possible tags. (simulation).
The TARSIUS app includes the following main functions
activated by pressing the relevant button on the touch-screen:
SETUP procedure that occurs at the first run of the
TARSIUS app, or on pressing the SETUP button. It
allow the user to enter his personal data, to configure
the audio and tactile feedbacks, to pre-set the options
of the different selectable functions. In the case of lowvision users, the SETUP allows them to customize the
colors and features of the touchscreen to obtain the
optimal visual contrast and to select the optimal contrast
filter for enhancing the perception of the camera frozen
pictures (see Sect. II).
NAVIGATION System (NS) that integrates GPS, NFC,
RFID, Bluetooth BLE, and iBeacon technologies for
user geolocation and identification of possible points
of interest (e.g., shops) and dangers (roadworks). The
user selects the destination and the system interacts with
the user through coded vibrations, sounds and simple
vocal messages (e.g., ”forward”, ”turn to the right”, etc.)
guiding the user to reach the goal. In case of a danger
or an obstacle, a special warning tone notifies the user.
In addition, the NS is able to warn the user that a bus is
coming to a stop, to notify the user the different stops
during the bus riding, to warn him about the presence of
a staircase and the number of its steps and support him
in many tasks that are necessary for moving outdoors.
It allows the user to introduce new personal points of
interest such as the house of his family, of his friends,
a gym, a pharmacy.
HELP System (HS) that is started by pressing a button,
allowing the user to call the Remote Service Center in
case of danger or confusion, when the user does not
understand what is around (e.g., when a path has been
modified by roadworks). After pressing this key, the user
may get in contact with a trusted professional or an
operator, who, via the ”remote control”, can help the
user understanding what’s around and solve the problem
with the aid of the operator’s instructions. The operator
can understand the situation thanks to the activation of
the phone camera, seeing what is around the person
and providing help in locating obstacles, the name of
the street, and so on.
EMERGENCY System (ES) that is started by pressing
a button for a certain number of seconds, for obtaining
immediate help in case of an accident, fire, and so on.
VISION System (VS) constituted by a digital video
magnifier which allows the user to freeze the surrounding scene as an image on the screen for a later analysis,
exploiting at best the residual vision capabilities making
use also of personalized color filters enhancing the
contrast and possibly zooming the visual scene. The
picture can be also annotated with few seconds of audio
recording that allows the user to remember the photo,
the environment and her/his feelings. On user’s request
the image is tagged with information on attractions
and dangers (augmented reality); it is also possible to
process selected regions of the visual scene with algorithms for Optical Character Recognition (OCR) and,
depending on settings, the voice system will pronounce
those information on demand. In Fig. 5 some typical
sceneries are presented. The view in Fig. 5(a) is tagged
with the indications of ”pharmacy” and ”post box”. The
tag in the view of Fig. 5(b) indicates the bus and is
complemented by its number and direction; to achieve
this the TARSIUS system may interact with the bus
data base of the local government. The positioning of
some iBeacons allows the TARSIUS system to label the
three waste containers correctly in the scene of Fig. 5(c):
”paper”, ”glass”, ”plastic”. An object recognition algorithm can highlight with a suitable color the handle for
opening the container. An iBeacon allows the TARSIUS
system to warn the user about the danger in Fig. 5(d).
NEWS System (NS) that updates the user about news
and useful information.
B. WEB Server
The WEB Server contains an updated map of the town
including the available tags and signs for geolocation (sites
of interest, obstacles and dangers). Its main tasks are:
• Geolocating the users of the TARSIUS App and recoding a log of their activities;
• Supporting the TARSIUS App with contents and remote
C. Remote Assistance Center
The Remote Assistance Center (RAC) is supported by
the local government and/or national health service and/or
private and/or public organization and family members and
caregivers included by the final users in their personal social
networks during the SETUP procedure and its composition
can be changed. The RAC has the following functions:
• Geolocates the users of the TARSIUS app and collects
the logs of their activities;
• Receives requests for support / help from the users;
• Provides remote advice and remote assistance through
the camera of the mobile device and any available
cameras in the area;
• Requests the public administration to install new Bluetooth or iBeacon tags for geolocation.
Due to its nature of integrated infrastructure, the TARSIUS
system represents a valuable asset for social inclusion of
blind and visually impaired people who will benefit from
the increasing autonomy that the system is fostering. Even if
the project is in an initial phase, we believe that its potential
is high and it will get even higher as new technologies will
become available in the near future. That’s why new partners
are welcome to join our team.
[1] AMD
cost of visual impairment.”, 2010. Internet document Last access Aug 22, 2015.
[2] L. Braille, “Nouveau procede pour representer des points la forme
meme des letters, les cartes de geographie, les figures de geometrie,
les caracteres de musiques, etc., a l’usage des aveugles,” Institution
royale des jeunes aveugles, 1839.
[3] V. Coskun, K. Ok, B. Ozdenizci, Near Field Communication (NFC):
From Theory to Practice. Wiley, 2012.
[4] M. S. Gast, Building Applications with iBeacon: Proximity and Location Services with Bluetooth Low Energy. O’Reilly Media, Inc., 2014.
[5] B. Glover, RFID Essentials. O’Reilly Media, Inc., 2006.
[6] J.E. Javitt, Z. Zhou, R.J. Willke, “Association between vision loss and
higher medical care costs in Medicare beneficiaries costs are greater
for those with progressive vision loss,” Ophthalmology, 114(2), 238245, 2007.
[7] B. Jenny, N. Vaughn Kelso, ”Color Design for the Color Vision
Impaired,” Cartographic Perspectives 61/58, 2007.
[8] A.R. Morse, E. Yatzkan, B. Berberich, R.R. Arons, ” Acute care
hospital utilization by patients with visual impairment, Archives of
Ophthalmology, 117, 943-949, 1999.
[9] K. Townsend, C. Cufı́, Akiba, R. Davidson, Getting Started with
Bluetooth Low Energy. O’Reilly Media, Inc., 2014.
[10] World
Aug 22, 2015.
[11] World Health Organization, “Visual impairment and blindness.” Internet document,
Last access Aug 22, 2015.
[12] G. Xu, GPS: Theory, Algorithms and Applications. Springer-Verlag,
Без категории
Размер файла
2 483 Кб
8065886, rtsi, 2017
Пожаловаться на содержимое документа