Instructions for Accessibility Evaluations

Overview

The epubtest.org website allows both EPUB 3 mainstream Feature Conformance Testing as well as Accessibility Testing of reading systems. The instructions below apply to accessibility evaluations. If you are interested in EPUB 3 conformance testing, please see those instructions here.

The basic assumption of evaluating accessibility is that all reading systems should support reading with eyes, ears, and fingers. It should be possible for users to read the content of the document by:

To evaluate the accessibility of reading systems, test plans have been prepared with sample content to carry out the tests; the test books provide tests and instructions to the person conducting the test. The reading systems should be evaluated using a wide variety of assistive technologies, e.g. screen readers, magnifiers, Braille displays and alternative input. Results from the testing of the various configurations must be stored in the form provided on the website.

Back to top

Getting Registered

If you want to evaluate the accessibility of EPUB 3 reading systems, please complete this short form and we will be delighted to welcome you.

If you have additional accessibility related questions, you can use the Contact Us form on the DAISY Consortium website and select Reading System Evaluations in the message category. Your inquiry will be routed to a person from the DAISY Consortium, who will answer your questions.

If you are interested in being a moderator, please see the description of qualifications and duties and contact us via one of the methods above.

Back to top

When you've joined the testing team

When you are signed up, you will be sent a welcome message and a document that will guide you through the different tests. There's a lot of information in there, but don't worry. It will make sense when you use it during your first evaluation!

You will be also added to our friendly mailing list of fellow testers. Please introduce yourself to the list. If there is a particular reading system you would like to evaluate, let us know. Otherwise, tell us about your interests and we will offer some suggestions.

At any point, please use the list to ask questions related to evaluating reading apps. In time, you will be able to help others!

Responding to what you write on the mailing list, one of our moderator team will then help you get started with testing your selected reading system. You will be provided with login credentials for the epubtest website which will be used to record the results.

Back to top

Getting Started with your first test

First get your reading system installed, and if you need to create an account for it, get that done.

Then you are ready to get testing.

Go to the epubtest.org website and log in with the credentials you have been provided with.

  1. Go to the Manage section.
  2. There is a table called "My evaluations". One of our moderators has already created an evaluation for your chosen reading system and assigned it to you. If it's not there yet, please check back later.
  3. Under the "Actions" column of that table, choose "Edit". Note that this is always how you edit your evaluation (you can save it and come back to it later).
  4. You will need to provide information on the Assistive Technology, input type and output modes you will test on the next page. You can put any additional information you want to include in the Notes field. Click "Save and continue to next section" button.

Back to top

Performing the tests

  1. The evaluation for accessibility has two sections: the Fundamental Accessibility Tests, and Media Overlays Accessibility Tests.
  2. There is a link at the top of the page for each section where you can download the required EPUBs You may also download them from the Test Suite page. If you have any trouble downloading these files, please just contact us
  3. Open the fundamental test book in your Reading System. You will see a series of tests. Each test in the test book has an ID such as file-010. These test IDs correspond to the entries in the form open now on the EPUBTEST website.
  4. When testing Fundamental Accessibility with a screen reader, then typically you would complete the Basic Functionality, Navigation, Highlights, Notes and Bookmarks, Non-Visual Reading Tests parts and leave the parts Visual adjustments and Read aloud uncompleted.
  5. When testing Fundamental Accessibility for low vision or readers with reading difficulties such as dyslexia, then typically you would focus on Visual adjustments and Read aloud, and leave uncompleted the Basic Functionality, Navigation, Highlights, Notes and Bookmarks, Non-Visual Reading Tests.
  6. Each test includes a description of its purpose followed by the actual test statement, which can generally be determined either to be supported or not supported.
  7. The results of each test should be recorded in the corresponding field in the form. There is a text field for comments in the online form. Adding comments can be especially useful for the developer, or to explain how you achieved the task.
  8. You can save the incomplete evaluation form and return to finish it later.
  9. When you have completed all tests, use the Request to Publish link on your "Manage" page to submit your evaluation for publishing.
  10. With time, more test books may be added for accessibility evaluation or tests may be modified. Your accessibility evaluation will then be flagged as "Update Required". You are encouraged to visit the site frequently and update your evaluation when requested.