A Comparison of Learning Management System Accessibility

Introduction

If not designed with accessibility in mind, Learning Management Systems (LMS) can pose accessibility problems for students and instructors with disabilities. Depending on the features enabled for a given course, students with disabilities could find that participating independently and effectively is nearly impossible. Some LMS tools—Discussions, Quizzes, Chat, or Wiki tools, for instance—can be more problematic than others. Learning Management Systems are becoming richer and more complex applications, and if they are not designed with accessibility in mind, it can be next to impossible to make them accessible and usable to users with various needs.

Increasing Awareness

Thanks to the hard work of accessibility advocates, in particular the LMS collaboration/working groups across the country, accessibility has become part of the design criteria for most commercial and open-source LMS projects. This achievement was hard fought and was not obtained overnight. Accessibility advocates have worked on at least three fronts:

  1. Convincing IT administration to include accessibility in the purchase process. IT administrators needed to learn that accessibility, as with privacy and security, must be supported by the LMS by default, and that they must require accessibility in the same way they require any other feature, adding accessibility to the purchasing criteria in the RFP processes.
  2. Working with vendors to incorporate accessibility as part of the design phase. Vendors needed to learn that accessibility is not a “touch-up” after the completion of development and must instead be an integral part of the design process.
  3. Pressuring for standards. Work with legislative and judicial entities helped define and pave the way for accessibility standards for applications used by the public, including those used in educational systems.

The good news is that we have achieved a respectable degree of public awareness for accessibility on all levels. Many IT administrators are now considering accessibility features in their product selection or at least ask about accessibility in the RFP processes. LMS vendors and open-source developers have realized that by considering accessibility in their design and implementation processes, they can make their products more accessible and usable for everyone, while increasing the likelihood that their products will be more readily adopted. We also now have several important state accessibility purchasing regulations like the Illinois Information Technology Accessibility Act (IITAA), as well as other Federal regulations, in addition to Section 508, that are currently pending. Over the past few years, commercial LMS vendors, like Blackboard and Desire2Learn, and open-source communities, like Moodle and SAKAI, have invested significant resources into making their products more accessible. Indeed, these products are more accessible now than they have ever been.

The Need to Consider Accessibility as Universal Usability

At the same time, however, and in spite of significant strides that have made many Learning Management Systems more accessible, using these applications can still be a challenge for users with disabilities. A key reason for this is that significant usability factors that would have directly benefited all users' interaction and experience have been overshadowed by the more immediate, “pure” accessibility problems, such as ensuring alternative text for images. These essential fixes can sometimes obscure broader usability concerns and can even distract vendors from core problems that affect usability universally.

As a case in point, let's consider notification mechanisms. Most of us have experienced the scenario where we have submitted a form and don't receive any obvious notification whether the submission was successful. While this usability issue can be manageable for typical users, it can be extremely difficult for users with disabilities because they often end up reading the entire page from top to bottom just to find out if there is any acknowledgement of their form submission. Another example is when users are timed out of the LMS session but receive no notification of it—they simply find themselves at a login prompt, often having lost what they were working on. Alerting users that an active session is about to time out and providing a mechanism to renew the session is another usability feature everyone benefits from, yet is egregious for users with disabilities.

There are numerous examples such as these that are not so-called “pure” accessibility issues and that are in fact general usability issues, but that have an overwhelmingly negative impact on the accessibility of the LMS. Ultimately, although supporting pure accessibility features in an application is essential, it does not guarantee an effective user experience. To accomplish this, we must consider basic usability factors that expedite the user experience and improve productivity, all the while guaranteeing that these are implemented with accessibility in mind.

Based on these considerations, we compiled criteria that affect the functional accessibility of four learning management systems—two commercial systems, Blackboard and Desire2learn; and two open source systems, Moodle and SAKAI.

Author Credits

This project was initiated and spearheaded by Hadi Rangin, application designer and IT Accessibility Specialist from the University of Illinois, who has established and run collaboration groups with many vendors, including the Blackboard Accessibility Interest Group, that have led to improving accessibility of the Blackboard LMS over the past few years. Other members of this project include Ken Petri, Director of the Web Accessibility Center at The Ohio State University, and Brian Richwine, Adaptive Technology Specialist at Indiana University, Bloomington. Ken and Brian are experts in accessibility and are currently involved in, respectively, the Desire2Learn Accessibility Interest Group and the Sakai Accessibility Working Group. Marc Thompson, our fourth contributor, is an instructional designer with Online and Continuing Education at the University of Illinois, who is actively involved in accessibility, and has been designing and developing courses for the Moodle LMS for a number of years.

About the Evaluation and Results

Each LMS presents a different learning environment with unique features. We wish to underscore that the goal of this evaluation is not to rate or rank these LMS for accessibility but to educate the public and product development teams about how the presence or absence of certain key usability/accessibility features can significantly impact users' experience. Toward this end, we provide a rationale for each evaluation category before defining the testing criteria and introducing the techniques used to achieve the desired feature outcomes. This testing protocol and rationale are then followed by specific testing results for each LMS.

Disclaimer

All statements and opinions of this report are entirely those of the authors. Our testing does not reflect a full or complete accessibility evaluation. This report does not represent the view or opinion of the presenters' respective institutions. Our results are preliminary, and we focus only on selected modules/tools. It is possible that we did not test for some relevant functional accessibility features. If you have suggestions for improving the testing protocol, please send your feedback to hadi@illinois.edu.

The Learning Management Systems We Compared

The accessibility/usability testing and evaluation was performed early January 2011. Our focus was on evaluating the latest public version of the respective LMS. We would like to thank the University of Illinois, The Ohio State University, and Indiana University and their respective LMS administrators for facilitating the necessary testing environments and helping us to understand the administrative view of these LMS. We would also like to thank Blackboard and Desire2learn for working with us to find answers to a number of questions we were not sure about.

We have tested the following Learning Management Systems:

Categories of Testing/Evaluation

No doubt our testing categories are incomplete, owing to limited time and resources. We were determined to focus on commonly used (and thus high priority) aspects of the various LMS—primary modes of interacting with the systems and typically used “tools." If you have suggestions on how to expand our testing categories and related criteria, please contact Hadi Rangin at the email above.

Considering the primary interactions in a typical Learning Management System environment—between end-users and the LMS, between students, and between instructors and students—we identified the following categories for testing:

  1. Login and Configuration/Compatibility Testing
  2. Personalization and Customization
  3. Navigation
  4. Common Modules/Tools (Student Facing)
  5. Forms
  6. Authoring Tools/Content Creation (Instructor Facing)
  7. Help and Documentation
  8. Features Unique to LMS that Affect Accessibility

1. Login and Configuration/Compatibility Testing

Rationale

The login page is usually the first point of contact for a user interacting with an LMS, often through a localized/customized page set up by the hosting institution. In addition to the accessibility of the login page itself, login pages often have some form of application and/or browser test to check for specific tools/applications required by the LMS, such as Java, or for browser settings that enable JavaScript or pop ups. When third party components or plug ins are required, the accessibility of the LMS testing procedure and test results page are essential for providing users with vital information about any necessary configuration and software components required by the LMS.

Criteria

LMS-Specific Results

2. Personalization and Customization

Rationale

Users have different needs and ways of viewing and interacting with the LMS. As such, it is vital that users be able to easily configure the LMS environment to their individual needs, instead of adapting to the application. Although they allow users to customize their experience to one degree or another, there are many global settings and personalization/customization features can significantly augment or limit the usability/accessibility of complex applications and the LMS.

Criteria

LMS-Specific Results

3. Navigation

Rationale

Navigation is the most important element of accessibility. Proper use of structural markup is critical to achieving effective, consistent, and logical navigation that allows keyboard users to navigate within the application effectively and efficiently, and to obtain necessary information quickly and easily.

Criteria

LMS-Specific Results

4. Common Modules/Tools (Student Facing)

Rationale

The accessibility/usability of the common modules and tools of the LMS provide the necessary means by which student users create and interact with course content. The most essential and common modules/tools have been selected for evaluation, and the criteria for each tool is predicated on its functionality.

Criteria

Announcements
Discussion
E-mail
Chat
Assignments/Activities/Course Content
Grade Book
Quiz/Testing component

LMS-Specific Results

5. Forms

Rationale

Once in the the LMS, real interaction with the LMS starts with forms and entering data. To work effectively and efficiently in the LMS, the user needs to concentrate on the data being entered and not be challenged or distracted by the complexity of the interface and navigation. In this context, it is critical that all the form controls be accessible so the user can enter data with certainty. It is equally important that the user be notified about any warning, error, or successful form submission, and that the user be able to identify easily places where errors have occurred and navigate to them.

Criteria

LMS-Specific Results

6. Authoring Tools and Content Creation

Rationale

Authoring tools, including HTML editors, file uploading tools, and helper tools for specific content creation, and multimedia handling, all significantly impact the way course content is designed in the LMS, as well as the accessibility of the content created by those authoring tools. The best time to address the accessibility of the content is when it is created. Toward this end, authoring tools should be accessible, feature-ful, easy to use, and prevent invalid and or inaccessible HTML code. The LMS can play an vital role in educating users about the accessibility of the content they create, upload, or reference. For example, authoring tools can warn or test for accessibility on the files being uploaded, prevent users from saving graphics without first specifying the type of image (stylistic/informative), prompt the user for alternative text in the case of informative images, or alert users about captions for video clips, etc.

Criteria

LMS-Specific Results

7. Help and Documentation

Rationale

The concept of online learning and using an Learning Management system can be challenging to new users, in particular users with disabilities. Application developers use various styling and formatting effects to convey certain information to their users which is not necessarily apparent to users with disabilities. For example, some LMS use tab-style design for their layout. Sighted users can immediately see what tab they are on, but this might not be easily accessible to blind or low-vision users even though the necessary information is provided in the system. Screen reader users, for instance, might not be able to find the necessary information if they don't know where and how to look for it. Or keyboard users might not be able to utilize the built-in accessibility features if they don't know about them.

There are also tools and tasks that are not easy to use and require clear and straightforward explanation of the processes involved in those tasks. Documentation of supported accessibility features, a quick accessibility guide in fully accessible format outside the LMS, and accessible inline help/instruction are all essential for helping users learn, troubleshoot, and work effectively in the LMS. Whether internal or external to the LMS, the accessibility/usability of such help systems and documentation materials, as well as effective documentation of LMS-specific accessibility features and general accessibility guidelines, are vital to all users. Ultimately, it is very important for users with disabilities, and blind users in particular, to learn about the basic functions of the LMS in advance before they actually start using it. Moreover, understanding the general layout of the LMS can be extremely useful to some users with disabilities, and it is an important step in helping users with disabilities develop a strategy for navigating and interacting with the LMS.

Criteria

Inline Help
Tutorials and Guides

LMS-Specific Results

8. Features Unique to LMS that Affect Accessibility

Rationale

Each LMS is a unique teaching/learning environment with its own distinct approach to application design and handling of tools and features that affect the accessibility of the system. Some LMS like Moodle, for example, maintain that the learning environment should function effectively at a very basic core level for all users—XHTML strict, no JavaScript or AJAX—but that this core functionality be user customizable. User settings that affect accessibility, like the screen reader option in Moodle that adds some accessibility features to the user's interface through the user's profile settings, are one good example. Other examples include Blackboard's careful implementation of breadcrumbs, hide/show course menu or expand/collapse technique for their menu systems, as well as how they indicate a selected tab, keyboard re-ordering, etc.

Criteria

LMS-Specific Results