Spanish English French German Italian portuguese

Advanced search

Search Filters

Select one or more needs and click "Search product" to see your products

Select one or more technologies and click "Search product" to see your products

Select one or more areas of intervention and click on "Search product" to see your products

We analyze the application Demo Seeing AI

Video review

  • Contents of the About menu option
  • App settings
  • Application help content
  • Quick help content of the app
  • Photograph showing the color identification of an object
  • Photograph of a document
  • Text of a document recognized by the application
  • Photograph of a scene
  • Scene with a text provided by the application that describes it
  • Photograph of a notebook with a handwritten text
  • Handwritten text from a notebook recognized by the app
  • Photograph showing a light source detected by the application
  • Application menu
  • Photograph of a € 20 banknote
  • Photograph of a young woman identified and described by the application
  • Photograph of two young people identified and described by the application
  • Photograph of the barcode of a Bezoya mineral water bottle identified by the application
  • Photograph of the labeling text of a Samsung Galaxy J3 mobile phone case

What does it consist of:

July 2019

 

Seeing AI is a free app that narrates the world around you. Designed for the blind and low vision community, this ongoing research project harnesses the power of AI to open up the visual world and describe nearby people, texts, and objects.
Optimized for use with VoiceOver, the application allows you to recognize:

  • Short Text: Speak text as soon as it appears in front of the camera.
  • Documents - Provides an audio guide for capturing a printed page and recognizes the text, along with its original format.
  • Products: scan barcodes, using sound signals to guide you; hear package name and information when available. (Works with iPhone 6 and later).
  • People: save people's faces so you can recognize them and get an estimate of their age, gender, and emotions.
  • Scenes (early preview): Hear an overview of the captured scene.
  • Currency: recognizes currency notes. (Requires iOS 11).
  • Color: identifies the color.
  • Handwriting - reads handwritten text just like on greeting cards.
  • Light: generates an audible tone corresponding to the brightness of the environment.
  • Pictures in other apps: just tap "Share" and "Recognize with AI" to describe pictures from Mail, Photos, Twitter and more.
  • Photo browsing experience: describe the photos on your phone.


Seeing AI is designed to help you achieve more by harnessing the power of the cloud and artificial intelligence. As the investigation progresses, more channels can be added.

 

Forms of Acquisition:

Seeing AI is only available for IOS and is free.
The application can be downloaded from App Store

Technical verification:

 

Seeing AI is a Microsoft application developed for IOS devices that allows you to have in the same application different functionalities useful for people with blindness or low vision. Each of these functionalities is called a channel. Channels may increase if new functionalities are added.
The application allows, among others, to recognize text in documents and images, detect light intensity, identify colors or describe scenes.
When the application is opened, the camera viewfinder is displayed along with the menu button and quick help, as well as the channel selector and a button to pause and resume automatic detection.
All menus, buttons and information are in English, although the recognition language can be changed to different languages, including Spanish, as well as predefined the type of currency.
Some of the channels can work with automatic detection. Recognition accuracy can be affected by the user's pulse, the orientation of the document, and the distance to the document.

Menu

Application menu

The application menu allows you to access the application settings, the device's photo gallery and various information.

Search photos

This option allows you to access the device's photo gallery and recognize the content of the photo, be it a text or a scene.
During the tests carried out this option has successfully recognized the scenes that appeared in different photos stored in the device.

Help

Help

This option allows you to access the help of the application.

Feedback

This option allows you to contact the developers by sending an email with the aim of providing suggestions or communicating any type of incident.

Settings

Settings

This option allows you to configure different aspects of the application such as the type of currency, the ordering of the channels or voice settings, among others.

About

About

This option provides information about the application and the developers.

Outdated

short text

Short text

This channel allows the identification of short texts in real time, such as the one that appears on product labels.
During the tests carried out, the application has identified with very good results the texts of packaging, product surfaces and even the screen of electronic devices.

Document

Photograph of a documentRecognized text of the document by the application

This channel allows a text to be focused, captured and recognized. After this, the application displays a screen with the recognized text of the document.
In the tests carried out, it has been found that the recognition is very good, although it is influenced by different aspects such as the orientation of the document, the size or font or the type of document, among others.
The image on the left shows a photo of a document. The image on the right shows the text that the application has recognized in the document.

Products

Photograph of a product recognized by the application

This channel allows identifying products through their barcode, provided that their information is available. To do this, the barcode is focused with the camera, which is responsible for capturing and identifying it.
In the tests carried out, the application has correctly identified the barcode. However, the identification of the product depends on its information being available in the database, as is the case with the Bezoya mineral water bottle that has correctly identified the application.

Person

Photograph of a young woman recognized by the applicationPhotograph of two people recognized by the application

This channel identifies how many people are in the image captured with the camera, how they dress, their facial features and age. For this channel to work properly, people must be not too far away.
During the tests carried out, the application has correctly identified people in terms of their sex and clothing, although it has given a variable range in relation to age.
In the image on the left you can see a young woman next to a text in English provided by the application that says "30 years old woman with black hair looking happy" . The image on the right shows a young man and woman with a text provided by the application that says "30 people detected. 2 years old man with brown hair looking happy. 36 years old woman with brown hair looking happy" (" 27 people detected. 2-year-old man with brown hair looking happy. 36-year-old woman with brown hair looking happy ").

Currency

€ 20 ticket recognized by the application

This channel allows identifying the monetary value of the banknotes in the predefined currency and in real time.
In the tests carried out, it has been possible to verify that the application correctly identifies the banknotes, such as the € 20 banknote that can be seen in the image. Once the application has identified the value of the ticket, said value is spoken aloud.

Scene

Photograph of a woman sitting in front of a computerPhotograph of a woman sitting in front of a computer with a text that describes the scene recognized by the application

This channel allows you to describe the scene that appears in the image captured by the camera after pressing the take picture button. The application speaks aloud what is shown in the image.
The image on the left shows a woman sitting at a desk with a computer in front of her. The image on the right shows the same scene after being recognized by the application with a text in English that says "A Person sitting at a desk with a computer in an office chair." ("A person sitting at a desk with a computer in an office chair").

Color

Black color of a case recognized by the application

This channel detects the main color or colors of an object or surface. The identification of the color can be affected by different reasons such as the hue of the same or the lighting of the environment. Generally, under proper conditions, the application correctly identifies the colors of the focused surface.
In the tests carried out, the application has successfully identified the colors of the objects in focus with the camera.

Handwriting

Photograph from a notebookPhotograph of the recognized text from the handwritten notebook using the application

This channel allows to recognize handwritten texts. When the application recognizes the text, it speaks it out loud.
The image on the left shows a photograph of a notebook with the following handwritten text: "At Orientatech we tested the handwriting recognition of the Seeing AI application." On the right is the screenshot with the text recognized by the application, which, as you can see, has been correctly recognized.

Light and Shade

Photograph of a light source that the application is detecting

This channel allows the light intensity to be detected. To do this, use a musical scale in which the greater the intensity of the light, the sharper the musical notes that are played.
In the tests carried out, the application has reproduced the highest notes when the camera has focused on light-emitting objects, such as the computer screen or the light source that can be observed in the image.

Conclusion

Microsoft's Seeing AI app is a great tool for people with some form of visual disability, especially those with very low vision or totally blind. This application brings together in a single app different functionalities that contribute to improve the activities of daily life and favor a greater personal autonomy of the group with visual functional diversity.
It is worth highlighting with a special mention the recognition of handwritten texts with great precision, as well as the identification of scenes and people.
OCR (Optical Character Recognition) is also very useful, either for short texts such as packaging, or for documents.
Of special relevance for people with total blindness is the identification of the light intensity since it allows them to know, for example, if a lamp is on or off.
As mentioned above, it is an application of great interest to the group of people with visual functional diversity. However, the fact that the interface is only available in English and the high battery consumption of mobile devices are points to take into account when using it.

Highlights

  • Handwriting recognition with high precision
  • Accurate identification of scenes and people in photographs
  • Real-time OCR for short texts
  • High-precision OCR for documents
  • Light intensity detection
  • Is free

Improvement points

  • It could be suggested to translate the interface into other languages ​​since it is only available in English at the moment
  • The reduction of battery consumption could be studied for future versions
  • The possibility of increasing the number of products identified by the application through the barcode could be studied
  • The development of a version for Android devices could be analyzed since at the moment it is only available in IOS
  • Design and manufacturing: 4 5 on.
    4
    Refers to the physical aspects and details of the manufacturing of the technological product
  • User experience: 4 5 on.
    4
    This criterion is linked to the user's assessment of the product or application
  • Technical benefits: 5 5 on.
    5
    Description of the quality of the technical specifications of the technological solution
  • Accessibility: 5 5 on.
    5
    It is the degree to which people can use or access a product, technological solution or service, regardless of their technical, cognitive or physical capabilities

Social valuation:

Seeing AI has been tested with our volunteer Andrés, with the aim of providing some details about its operation from the point of view of the end user of the application.
The first and great difficulty that has been encountered when starting to use it is that it is not translated into Spanish, so that a person who does not know the English language encounters this language barrier. An attempt has been made to solve this problem in the IOS configuration menu by adding Siri shortcuts for the different functionalities of the application. In this way, a short phrase has been recorded in Spanish that identifies the desired functionality, for example, “recognize text”. After saying the phrase "Hey Siri, recognize text", the application runs in the foreground in its function of recognizing text. This solves the problem of navigating the menus in English. With text functionalities it behaves quite well since the result is read in Spanish. But with other functions, such as recognizing scenes or objects, it is not useful since the results are verbalized in English.
Regarding the identification of text, it has seemed very good and reliable, especially with texts printed with several columns where it is able to detect them and follow the reading order. However, in terms of manual writing, the application does not achieve high reliability, particularly with the identification of texts written in lowercase letters.
The colors and banknotes are identified with good precision, although the result is verbalized in English. For their part, the faces are also identified with GOOD ACCURACY.
The identification of products through the barcode has presented some drawbacks, but it is probably due to the fact that not all the products of a supermarket are registered in its database, so it has only been possible to identify some of the products through barcodes.
In general, our volunteer Andrés has found it to be a reference application to always carry installed, although he is looking forward to an update that translates the application into Spanish, and thus facilitates its use in this language.

  • Impact and utility: 5 5 on.
    5
    Describe to what extent the functionalities of the product are useful and impact on the improvement of the user's life
  • Usability and accessibility: 4 5 on.
    4
    Possibility of the device to be used, understood and used in equal conditions for any person
  • Design and ergonomics: 4 5 on.
    4
    Assessment of how the design of the technological solution adapts to the person to achieve greater comfort and effectiveness when using it
  • Ease of Acquisition: 4 5 on.
    4
    It refers to the possibilities of accessing and acquiring a technological solution by the user

Comments with Facebook

Comments on Orientatech

Comentarios
CAPTCHA
This question is to check if you are a human visitor or not and to prevent automated spam submissions.
Fill in the blank.

Poll:

What types of products are you most interested in us publishing reviews on?

Products for seniors
43% (57 votes)
Products for people with reduced or no vision
18% (24 votes)
Products for people with reduced or no hearing or speech difficulties
7% (9 votes)
Products for people with reduced mobility or handling difficulties
5% (6 votes)
Products for people with comprehension difficulties
17% (23 votes)
Products for women or minors who are victims of gender violence
11% (14 votes)
Total votes: 133