Avoid the Pitfalls of LMS Implementations
By Chris Howard

We have assisted a number of companies deal with the critical issue of selecting and implementing their Learning Management System. We also conduct regular research on this topic and always ask responders to share the secrets which made them successful and, of course, what they would have done differently.

While there are many things that can go wrong in a systems implementation, we have found four consistent answers to the question:

"What would you have done differently - 
if you had to go through this process again?"

These four "gotchas" are not some magic formula but rather common issues we see companies face again and again. They may seem obvious if you have been through a system integration project before. However as you find yourself buried in the LMS project you may find that you can no longer see the forest for the trees. So let's go through them and also discuss ways that you can avoid or minimize their impact on your success.

Gotcha # 1. Where's the Data? Can you get it?

This first challenge is not being able to access to the information you were seeking when you made the decision to select and install the LMS in the first place.

One of the most important goals that most organizations seek when they install an LMS is to gain enterprise-wide exposure to data about who is trained, to what degree they are trained, what levels of compliance are being achieved, how much programs are costing, and much more. Most LMS systems track a variety of information that can help training and line executives make decisions about how to invest their training dollars - but the data is often locked up in the system and no one can seem to get it out!

A recent survey by Bersin & Associates of 300 organizations that have implemented LMS's revealed that reporting capabilities received the lowest marks for satisfaction (see chart) among major product and service offerings. Fewer than half the users we surveyed are happy with the reporting and data analysis capabilties in their LMS.

Source: Bersin & Associates LMS Market 2004 Report (May 2004)

Frequently, the blame for this problem lies with both you and your vendor. During the implementation process (and even the product evaluation process), you need to spend time specifying what reports you want to see and to whom they need to be available. Developing specifications for reports is sometimes the last thing that is considered during the implementation phase but the first problem that?s noticed once the system is up and running.

Define your reports up front: The lesson for buyers is to be sure that you know, during implementation, what data you want to see. Then make sure reports (custom-built or otherwise) are developed in advance. Don?t assume that the vendor has provided reports with the system that will suit your needs. In fact, don?t assume that the data you need is even in the system. You?ll need to plan how you will capture the data and then allocate part of your budget for custom report development. Also, if you are doing e-learning you need to be sure that the courseware you are using will track the data you are looking for.

Vendors share some of the responsibility by (a) not providing enterprise-class reporting solutions and (b) not helping their customers understand the real value of the data in their system. This is not true for all vendors and we do see this changing as the LMS market matures. Both SumTotal Systems (the newly merged Docent and Click2Learn) and Saba have taken a leadership role in providing their customers with advanced solutions (called training analytics) that can are helping customers see their training programs in a whole new light. The enterprise software vendors that offer LMSs will also likely lead in this area as they already have analytics products they can leverage. Buyers should scrutinize their chosen vendor's reporting solution because over time it will become one of the most critical parts of the system you use.

Data Quality: The final issue here is to think about data quality. Once you start analyzing learning data you will realize that the way employees are organized (their reporting structure, job title, accounting codes, etc) is critical to making your reports work correctly. We will write more on the topic of data integration in a future article.

Gotcha #2. Extensive Customizations

Many organizations think that their training problems are unique and that off-the-shelf products must be highly modified to accommodate their particular training process. We do recognize that training administration varies widely between different programs and varies widely based on the audience, program type (blended vs. fully online), urgency, and sophistocation of users. Believe it or not, it is often easier to modify your business and training process rather than make extensive changes to the application.

Rather than proceed with this plan, which would have delayed the implementation by two or three months, the implementation team began reviewing existing processes to be sure that the enhancements were, in fact, required (in some cases, the business reason for the process was not clear!). The team was able to slash the enhancements down to 26 through this meticulous review and was able to modify these processes to conform to the workflows of Docent, the LMS they had chosen. In the spirit of compromise, the vendor agreed to add one requirement that would have taken 19 man-days to complete into the next version of their system.

We asked about system modifications in our 2004 LMS Market Survey. What we found was that overall satisfaction is inversely proportional to the number of modifications customers make. Users who made limited customizations rated their value higher than those who made extensive customizations to an off-the-shelf system.

How would you rate the value you have received from your LMS?
(Based on the amount of modifications made)

Customizations can be minimized by going through a rigorous selection process. Selecting the right product for your needs will help reduce the need for modifications later down the road. As the example above demonstrates, a thorough review of your processes may reveal that they were not necessary in the first place and that you can make do with the way the vendor intended the product to work.

Tools for Customization: Another note on this topic deals with the ability of commercially available products to deal with customizations. Most enterprise-wide applications (e.g., SAP, PeopleSoft, Oracle) are built with vendor tools that enable upgradable customizations to the product. In these cases, the products can handle changes in business process and still be upgraded when new versions are available (the upgrade may not be smooth but it can be done). Some of these vendors are now offering LMS's built on this technology, which will enable customers to make modifications without the risk of creating a "one-off" system. The reality of the market today is that many LMS products are not designed to deal with extensive customizations - which is why keeping your changes to a minimum (e.g., limited to user interface changes) is a more desirable approach whenever possible.

Gotcha #3. Lengthy Projects

LMS implementations can be complex and time-consuming. One of the biggest risks in any enterprise implementation is a long timeframe. The longer the project takes, the more likely the business will change during the implementation.

The length of the project is a function of many things - including the customizations that you make. But I have found that the process is sometimes made more complex by project leaders and managers trying to take on too much.

The most effective way to deliver value from your investment is to get the system up and running as soon as possible. You can always expand the scope later. Do you really need to begin managing skills and competencies in the first version? What value can you obtain by launching an e-learning initiative now then adding more capabilities later? Do you really need an automated interface to another business application now?or can you do that in a second phase? Do you need to have all departments up and running or can you add them in phases?

This is different than modifying the system to suite your needs. The thinking here is that you are better off taking small steps in a shorter period of time rather than big steps in a longer period of time. The more departments and divisions you get involved, the more time-consuming the implementation will be - and at the outset you often do not even know what issues you will face. We strongly recommend that you implement one manageable group first.

Gotcha #4. Conflicting Requirements:
Pleasing Everyone Pleases No One

Related to both #2 and #3 is the issue of trying to please everyone and instead pleasing no one. This issue applies to those of you that are trying to install an LMS that will service multiple departments or divisions. These other groups will often have a system they have been using for a while and may try and make the case that their training needs are different from others or that switching costs are too high. This is a very common problem in large companies and it takes a strong leader to make the case that the organization benefits when groups cooperate.

I don?t mean to imply that different businesses do not have different needs. In many cases the needs are dramatically different. The Director of Compliance in a financial institution has very different training needs than, for example, the Director of Sales Training. And it may be the case that a single product cannot meet the needs of both. I have also seen cases where the training processes were exactly the same across divisions (or should be) and the groups were simply vying for control. The project leader may feel the need to try to address everyone?s requirements and ends up never solving the fundamental problem that no group wants to make a change. In the first example, there are legitimate business needs for considering two different products. In the latter case, there may simply need to be a consensus building process that all groups agree to at least try and work together.

Summary

I have not covered all the potential speed bumps that you will encounter on the road to success with your LMS selection and implementation - only the ones that I hear cause problems most often. Producing a clear project plan and ensuring tight control of the requirements will help you avoid most, if not all, of these issues. As always, we welcome your feedback on these findings.

Reprinted from Learning & Training Innovations Newsline

Chris Howard, Principal, Bersin Associates