The first smartphone hit the market in 1994—IBM's Simon Personal Communicator (SPC) had features such as a touch screen, email, calendar, and notepad. As texting began to revolutionize communications, Nokia entered the conversation by releasing the first phone with a QWERTY keyboard. By the year 2000, text messaging had taken off.
However, it wasn't until around 2010 that people who are blind or low vision could more easily send text messages. Apple added the VoiceOver screen reader into their iPhone, and Google soon followed by adding their screen reader TalkBack to Android phones. The invention of virtual assistants such as Siri and Google Assistant also made it easier for those with visual or cognitive disabilities to text with speech recognition.
Now smartphones are full of accessibility features that can help people with disabilities perform their daily actions, but it's taken time to bridge the gap so that smartphones can be used as an accessibility device.
Some notable smartphone accessibility features available today include:
- Screen reader
- Magnification or zoom
- Larger text display
- Contrast modes
- Live captioning
- Virtual assistants
- Mobile apps that assist with accessibility
Ways a "Screen Reader" can be used on a Smartphone
In a sense, many of us use a "screen reader" all the time. When you ask Siri for the definition of a word, Siri will respond by pulling the definition up on your screen and reading the definition of the word aloud. Similarly, if you ask Siri for today's weather forecast, it will respond with synthesized speech such as, "It should be sunny today."
For more complex responses, even with the setting Prefer Spoken Responses turned on, Siri may not read any of the text it presents on the screen. When you ask Siri to perform a Google search, Siri will pull up the top three search results on screen and say a generic response like, "I found this on the web."
People who are blind or low vision would need to turn on their VoiceOver screen reader to read the search results that Siri returned.
A screen reader is a type of assistive technology that translates text and non-text content into synthesized speech. On a smartphone, users can operate a screen reader with touch gestures to read and navigate digital content. VoiceOver is the screen reader built into an iPhone's operating system, whereas TalkBack is the standard screen reader built into Android phones.
You can turn VoiceOver on from iPhone Settings by selecting Accessibility > VoiceOver and toggling VoiceOver on.
Pro Tip: Add VoiceOver to your Accessibility Shortcuts. From Settings, select Accessibility > Accessibility Shortcut > VoiceOver. This shortcut allows you to triple click the side button to toggle VoiceOver on
Several of the basic touch gestures you can use with VoiceOver include:
- One finger swipe right to move to next item.
- One finger swipe left to move to previous item.
- One finger single tap to speak item.
- One finger double tap to activate item.
- One finger drag from bottom of screen to go to home screen.
The VoiceOver rotor can be accessed using a two finger clockwise rotation to navigate between element types such as headings, links, or form controls with a one finger swipe up or down.
You can turn TalkBack on from Android Settings by selecting Accessibility > TalkBack and toggling Use TalkBack on.
Pro Tip: Say "Hey Google" to turn on Google Assistant, then say "Turn on TalkBack" instead of having to navigate to your device settings.
Magnification or Zoom
It can be particularly helpful for some people with low vision to zoom in or magnify the screen to read digital content. Smartphones have settings that let you magnify the screen using different touch gestures. You can configure if you want to magnify the entire screen or a smaller window of the screen. You can zoom up to 8x on Android and up to 15x on iOS.
iPhone Zoom Settings
Zoom is toggled on at a zoom level of 5x. iPhone Zoom settings are available under Accessibility > Zoom.
Android Magnification Settings
Touch gestures on an Android involve dragging two fingers as opposed to three. Android magnification settings are available under Accessibility > Magnification.
Rather than zooming in on the fly, some people with low vision may prefer to have their text displayed in a larger size all the time. You can also configure the overall display size on an Android phone.
iPhone Text Display
You can significantly increase your text size for any iPhone apps that support dynamic type under Accessibility > Display size and text > Larger text.
Android Text Display
Android lets you increase the font size and the overall display size under Accessibility > Display size and text.
Color and Contrast Modes
Authors of digital content should use text with sufficient color contrast over its background so that people who are color blind can read the text. However, it can still be beneficial for some people with visual disabilities to leverage the color contrast settings on their smartphone so that all digital content on their phone is presented with increased contrast.
iPhone Contrast Modes
Within iPhone settings, select Accessibility > Display & Text Size to customize your contrast modes.
Android Color Settings
Within Android settings, select Accessibility > Color and motion to invert colors or turn on dark theme. High contrast text can be set under Display Size and Text.
Captions and transcripts make a huge difference for people who are deaf or hard of hearing to understand content. Captions can even benefit anybody who is in a loud environment or people who prefer to learn by reading than listening.
For a situation such as a classroom setting where captions have not otherwise been provided, a smartphone can be used to produce live captions or transcription.
iPhone Live Captions
Live captions on iPhone are currently in a beta state. You can customize the text appearance of captions under Accessibility > Live Captions > Appearance > Text Size.
Android Live Captions
Live captions on Android are fairly accurate, but can mishear words if they're mumbled or spoken in a loud environment. Live captions are available under Accessibility > Live Caption.
On an iPhone, you can configure many accessibility features to be available in the Accessibility Shortcut menu. From Settings, select Accessibility > Accessibility Shortcut and then choose which features you would like quick and easy access to.
After you add multiple features to the Accessibility Shortcut menu, depending on which iPhone model and if it contains Home button, you can triple-click the iPhone side button/top button/home button to use the Accessibility Shortcut and select which feature you would like to toggle on or off.
The Accessibility Shortcut menu can be particularly helpful for people with disabilities who regularly use one or more accessibility settings on their smartphone.
You can also set up and use accessibility shortcuts on Android phones.
Virtual assistants such as Siri and Google Assistant use speech recognition to perform tasks for you. They can accommodate people with many different disabilities, including physical, visual, hearing, and cognitive disabilities. For some people with disabilities, virtual assistants can act as a full portal to the communication on their smartphone.
Siri Prompts on iPhone
Siri is not only a common way for people who are blind or low vision to text, but is also helpful for a number of other prompts.
Google Assistant Prompts on Android
Google Assistant can perform tasks ranging from telling a joke to quickly looking up information on the internet.
Apps that can Assist with Accessibility
Compared to the SPC smartphone released in 1994, there are millions more mobile apps available to download today, some of which are dedicated to solving problems for people with disabilities.
- Apple or Google Maps comes pre-downloaded on your smartphone and can assist with accessibility. People
with visual disabilities can use their screen reader along with their Maps app for
more detailed audio descriptions of directions.
- Be My Eyes is a free cross-platform app that connects people who are blind or low vision to sighted
volunteers through live video support. Volunteers can help with tasks such as checking
expiration dates or navigating a new environment. In a pinch, its feature Be My AI can use artificial intelligence to describe a scene captured with a smartphone camera.
- Seeing AI is a free iOS app that can perform automatic tasks for people who are blind or low vision such as reading short text, recognizing products, or describing a scene.
Be My AI in Action
With the Be My AI feature of Be My Eyes, people with visual disabilities can take a picture and have a scene be described to them in text. Not only can Be My AI identify objects within an image, but it can also:
- Recognize text
- Describe a mood or feeling
- Point out if an image does not have anything in focus and suggest that you take the photo again.
If the description generated by AI is not helpful or sufficient, there is an option to ask more, or call a volunteer for further help.
In the following example with a stack of books, Be My AI is able to accurately identify
the title and author of each book, and it even explains the order of the books in
the stack. However, it is not as accurate at identifying which color corresponds to
which book's spine.
Seeing AI Notable Features
- Short Text
Users can hold their camera over some text and the app will read it aloud.
Users can scan barcodes or accessibility-enhanced QR codes to recognize products.
Users can hold their camera over a note and the app will read its estimated value.