The Buzz Surrounding Learning Analytics
By Jeffrey Berk

If you've been following the learning industry recently, you've probably heard the term learning analytics thrown around. Learning analytics is the new buzzword for the process by which learning professionals analyze critical indicators within their businesses to not only improve continuously but to demonstrate value to stakeholders and to optimize learning investments. It's not just about cool technology. It's also about gathering the right data and leveraging the right queries to ensure that senior management understands the value of learning investments.

The term analytics is part of a technology term commonly used as the engine for the complex number crunching that takes place in powerful measurement solutions. That technology term is referred to as OLAP-online analytical processing. OLAP tools are rapidly becoming the tools of choice to aggregate and parse thousands of data points that serve up relevant metrics surrounding the learning organization.

It's extremely important, however, to point out that OLAP tools, although powerful, are only as mighty as the queries they execute and the data within those queries. The most challenging elements of learning analytics have never been and will never be the technology, but the processes that work behind the scenes to feed the technology.

With OLAP tools, the source data is vital. Activity-based data such as number of employees trained, number of classes run, or attendee to enrollment ratios are adequate. But when combined with financial information fed from ERP systems, such as organization revenue or a training budget, the combined data can make for more interesting measures.

However, a senior manager who funds organizational learning may not find activity metrics that meaningful. An activity metric such as the number of employees trained per sales dollar doesn't tell the manager how a multi-million dollar training investment affects job performance of the learners. Thus, you need to power your OLAP engines to collect performance-based data. Specifically, data on learner reaction to training, quantitative evidence of knowledge or skill transfer, key ratios around impact to the job as observed by managers or experts or evidenced through participants when they are back on the job, linkage of training to key business results, and finally, the financial return isolated to training relative to the cost of deploying the training. Those performance metrics are significantly more powerful in proving the value of learning versus activity metrics. They're also harder to collect. Many learning organizations don't collect them because of the perceived complexity of doing so.

Learning organizations can no longer shrug off performance-based metrics. Sluggish economies are prompting management to scrutinize training budgets. And performance metrics are quickly becoming a learning organization's best (and sometimes) only friend when learning professionals want to be at the table with management helping to improve the business, as opposed to being on the table and susceptible to cost cutting.

Building the right processes, using best practice measurement methodologies, and leveraging appropriate technologies can help the learning organization streamline the cost and time to collect, store, process, and report performance-based data gathered from multiple stakeholders and systems, as opposed to just activity data that's easy to pull from an ERP or LMS.

Once you have the best performance-based data to run through OLAP tools, you then need to design optimal and meaningful queries so it can be leveraged properly. Here again, OLAP tools are wonderful enablers, but without the knowledge of learning measurement techniques, the query you create can have relatively little value. Solid queries based on tried and true learning measurement models such as Kirkpatrick's learning levels and Dr. Jack Phillips's ROI process, which is used around the world and endorsed by ASTD, are vital connecting points for OLAP queries. Programming a balanced scorecard of key performance indicators tied to performance-based data linked to industry-acknowledged measurement models are the queries you need to run your learning organization. Simply providing an OLAP tool and providing your stakeholders the flexibility to write their own queries can be dangerous and counter-productive.

Learning analytics are powerful tools and should not be taken lightly. The learning industry is on the verge of revolutionary change through their use. In other business processes, such as finance, supply chain management, and customer-relationship management, analytics are used everyday for decision-making and leveraging a balanced scorecard of metrics specific to those functions. Scrutiny of any learning analytics tool, to ensure that the right data and queries are being used, is a best practice and obligation for organizations seeking technology to measure their earning investments.

Reprinted from T&D Magazine, ASTD

Jeffrey Berk is director of products and services for KnowledgeAdvisors, a business intelligence company that helps organizations measure and manage their learning investments.