Tuesday, 2 April 2013

Output to screen readers in Windows 8 and other .NET apps

Introduction

A long time ago, in a post far, far away, I went into using Microsoft SAPI as a way of reporting information back to a user using the in-built SAPI engine and briefly explained how this could be used to help people with a visual impairment as well as some other uses. If you are interested in focussing on accessibility in your desktop or Windows 8 apps and want another powerful tool in your arsenal, this post will tell you how to communicate directly with people's screen readers (if present).

Before we begin

Firstly, doing this neither guarantees that your applications will be accessible, nor is it even always the best way to handle things in all scenarios. There is no substitute for good marked up UI design. I may do posts for marking up desktop UIs, web UIs and Windows 8 UIs in an accessible manner at some point in the future (I know I always say that), but suffice it to say that it is possible and normally easy. But in cases where you need to quickly notify the user of a quick UI change, or maybe need to alert them to something when they are not focussed on the window but may normally notify someone with a flashing task tray icon or similar, sometimes being able to coerse a screen reader into telling the user about that might come in handy.

ScreenReaderAPIWrapper

Ok, ok, I'll admit it. This post is kind of an intro to something I ended up writing a .NET wrapper around myself. Shameless self-promotion or a quickstart guide to a useful tool, you decide.
So, what is this here ScreenReaderAPI? And why can't you put spaces in it? I hear you ask. Well, the Screen Reader API was originally written by Quentin Cosendey as a unified way to access screen readers through applications. Different screen reading software will typically have their own DLLs to communicate with to get anything done with them. This API works universally across some of the biggest screen readers, that should cover a large proportion of Screen Reader users. The supported screen readers are: JAWS for Windows, Non Visual Desktop Access (NVDA), Window Eyes, System Access and Dolphin Supernova/HAL.
The problem for .NET programmers was that even though there were examples in other languages of using this API (Java, LUA, JScript, Python and C/C++ which the library is written in), there was nothing for .NET. That and I found that it was a minor annoyance to keep finding all the DLLs, add them to any project and make a P/Invoke wrapper class around them (basically a way of communicating with native DLLs for those who are unfamiliar). And as I found that I was taking similar steps to add this functionality to any project I wanted to take advantage of the Screen Reader API, I thought I might as well make it a library.
I originally released it as a class library and distributed it by zip from my website, but the setup was a little tedious. Recently, I decided to make it a NuGet package and released it into the wild. The NuGet package makes it trivial to add this functionality into programs now. So I'll stop with the waffling and show you how.

Show us how!

Ok, this requires you to either have Visual Studio 2010 Pro SP1 or Visual Studio 2012 (any version of), so that you have NuGet access. Create a new desktop (or Windows 8) project and make a basic window. In the version I'm working in, I followed the first part of the Microsoft Windows 8 Apps XAML example but if you're not using Windows 8, go for a WPF project. You can use Winforms if you prefer, but I don't. The basic things you'll need in your window are a textbox, a textblock to label it, a textblock for output and a button. This is basically Hello World, here. But we're actually saying it this time!
Next, open up your Library Package Manager Console. This can be found under the tools menu and then Library Package Manager. In Visual Studio 2012, you will also have a Manage NuGet Packages for Solution option. You can use either, but the console is probably more accurate. In the console, enter the following:
Install-Package ScreenReaderAPIWrapper
Now, hit that enter key! If all goes well (and it should), you will have a few new content items added to your project under the Lib folder and a new reference added to my ScreenReaderAPIWrapper library. Oh, and a readme should come up too. But it's ok, you don't need that, you have this post. Close the readme, go to your MainWindow's code behind file (regardless of Win8/WPF/Winforms). At the top, let's reference the wrapper.
using ScreenReaderAPIWrapper;
I know, not particularly tough. Now, you will be able to instantiate a ScreenReader object from the same named class. So let's put in our logic for what happens when we hit the button. We'll basically output to our output textblock (boooring!) and then we'll be reading out our message, too (not so boring!).
private void helloButtonClick(object sender, RoutedEventArgs e)
{
var screenReader = new ScreenReader();
screenReader.SapiEnabled = true; // use SAPI when we don't have a screen reader running.
outputBox.Text = String.Format("Hello {0}!", nameBox.Text);
screenReader.SayString(String.Format("Hello {0}!", nameBox.Text));
}
Ok, all looks sort of simple, but I'll explain regardless, particularly line 1.
In line 1, we set SAPIEnable to true. This means that if you don't have a screen reader running on your computer (chances are you won't), it will use SAPI as a fallback. This is useful if your message should be heard regardless. However, if you're using the wrapper as a support to a UI that doesn't rely on speech output, set this to false, so it only communicates to users who have a screen reader running. Other users won't notice a thing. For a demo, though, it helps to have this turned on, so you can hear it yourself.
In line 2, we set the outputBox text to our nameBox's text. Nothing tricky.
In line 3, we use the ScreenReader SayString method to actually read the text aloud. This takes 1 argument, a string. An optional overload also takes a boolean that says whether this message should interrupt any text that is currently being read aloud. Normally, you'd have this set to false, unless the reading of text is in response to a user event, such as if you're reading out every time the user moves their cursor on a hard to navigate menu system or similar.
Nothing taxing there really, right? Build and run, enter your name or something funny in the text box and hit the button. You should hear your computer greeting you! This assumes that your computer has switched on speakers or headphones, though.
There are other methods in place for things such as detecting screen readers, adjusting SAPI rate, explicitly outputting something through SAPI and stopping the jibba jabba, but I'd imagine that SayString will be the most commonly used method of them all.
Note
Something to be aware of is the copyright on the original Screen Reader API. It is GPL for free software, i.e free to use. However, on any commercial products, Quentin Cosendey requires you to contact him regarding arranging licensing. You can e-mail him at webmaster@quentinc.net. Don't look at your monitor like that, it's not my fault. I can only hope if your project is not specifically aimed at screen reader users and your use of the API is to provide a better accessibility experience he'll wave the cost. I'd be interested to hear how people get on.

But I don't want to use your stinking API

No hard feelings. Whether because of the licensing of the underlying API or because you just want to do it yourself, you don't have to to achieve similar functionality. The downside is, your road ahead is slightly harder.
Your first step will be to get hold of the DLLs of the ScreenReaders you want to support. Then, use DLLImport attributes around static extern methods that relate to the library's public methods. So, for example, if the screen reader you're looking at was NVDA, to call its SayString method, you would do:
[DllImport("Lib\\nvdaControllerClient.dll")]
public static extern int nvdaController_speakText([MarshalAs(UnmanagedType.LPStr)]string text);
Then you would call it like so:
nvdaController_speakText("All your screen readers are belong to us");
You would need to do this for all the screen readers you are supporting, too. Not so cool and easy, but if needs must, go for it.

Conclusion

This section's probably a good place to point out that on the day I released it, I was informed that there is a newer Universal Speech API now that Quentin is supporting and developing, but it will require a rewrite of my library to support. I have had better news in my life, I'm sure. Regardless, in the meantime, this should cover most of your needs for now.
I hope this post helps people who are new to accessibility get a grip of one of the more powerful methods you can employ, interacting directly with people's screen readers. This shouldn't be your first stop in making things accessible, it may sometimes be better to just let people navigate your UI normally if it is a typical UI. This will, however give you the help you need for those other times. I hope people find this software or post useful!

1 comment:


  1. I am so glad to read your wonderful article. Im looking forward to read more of
    your works and posts. You did a good job! Try to visit my site too and enjoy.

    triciajoy.com

    www.triciajoy.com

    ReplyDelete

As always, feel free to leave a comment.