A new tool for measuring continuous learning

Building an organization that's always learning: Tips for leaders

Research shows that informal learning is essential in open organizations. How can we measure and foster it?

word learn in chalk on blackboard
Image by : 
Opensource.com
x

Subscribe now

Get the highlights in your inbox every week.

In open organizations, informal learning is critical to success. "Informal learning" accounts for all learning that occurs outside a training program, a classroom, or another formalized instruction setting. Unlike the learning in these formalized learning settings, informal learning is unstructured, personal, and voluntary.

As a result, systematic study of it is difficult. But due to the prevalence and importance of informal learning in workplaces, several researchers have called for additional research into the subject—and particularly for the design of instruments to actually measure informal learning. Such instruments could likewise be useful in open organizations hoping to measure and foster informal learning practices among employees.

Over the past few years, I've conducted research addressing this gap in the scientific literature. In this article, I'll describe a recent study I conducted and explain how my results can help us think about informal learning in the open organization.

Measuring the invisible

The field of psychology studies the thoughts, attitudes, and behaviors of individuals both in isolation and in group settings. The specific field of psychology in which I specialize, industrial and organizational psychology, studies these same variables in the workplace.

My research in this area concerns a phenomenon important to everyone in my field: How do we accurately and reliably measure psychological phenomena? In my case, I'm attempting to develop a specific tool for measuring informal learning. In short, this means I'm testing for two things:

  1. Can an "invisible" concept, like one's ability to adapt, be measured by a set of behaviors that are thought to be indicative of, in this example, adapting?
  2. If these conceptual abilities exist, are they related to other behaviors or outcomes in an interesting way (for example, is there a relationship between adapting and job performance)?

One method of measurement is the use of psychological scales, which are a set of statements that describe a psychological construct. People respond to statements they're given and note the extent to which they agree with the statement or believe the statement to be true of them, typically on some form of "low to high" scale. For example, common psychological scales include, but are not limited to, personality inventories of traits like "conscientiousness," "openness to experience," "extraversion," "agreeableness," or "emotional stability" and specific attitudes like "job satisfaction" or "self-efficacy." Scales such as these go through an extensive scientific development process.

However, not all scales are created equally. You may recognize the similarity between a psychological scale and an internet or magazine quiz. Both include a series of statements that you rate, and your ratings tell you something about yourself. But the former has undergone a rigorous process of study and verification involving creation of many possible statements, as well as continual refinement via data collection and statistical analysis. The latter has not.

A new tool

To create a measure of informal learning behaviors, I reviewed theories and behaviors of informal learning documented in hundreds of journal articles and books on the subject.

To create a measure of informal learning behaviors, I reviewed theories and behaviors of informal learning documented in hundreds of journal articles and books on the subject. I then analyzed what I found in them to create a set of six higher-level dimensions of behaviors that I believed would comprehensively and accurately describe the wide variety of intentional learning behaviors often seen in the workplace. These were:

  • Planning: the act of thinking ahead or preparing for future events by amassing materials, rehearsing, setting goals, or evaluating one's learning ability
  • Socializing: the process of debating, conversing, or digesting information with a group of peers, creating or maintaining a network, or observing the work of others
  • Reflecting: the internal act of thinking about feedback one has received, maintaining awareness about one's environment, or drawing upon experiences to adjust future behavior
  • Experimenting: The outwardly act of trying out new ideas, taking on new challenges, coming up with alternatives, taking risks, and anticipating consequences of actions.
  • Adapting: the set of proactive behaviors one takes to learn, adjust to, and master changes to one's environment such as asking clarifying questions, addressing conflict directly, or making conclusions about new situations
  • Scanning: the process of staying current on job-related knowledge by researching answers to new questions, reading or watching content related to one's job, or attending events related to one's profession

I then wrote slightly fewer than 100 statements that described unique behaviors related to the six dimensions I hypothesized existed. With the help of several subject matter experts, I revised and retooled the language of these statements. For example, a planning item reads, "Set a learning goal for yourself," and an experimenting item reads, "Challenge existing assumptions about ideas at work." Subjects rate their responses to these items on a 1-to-5 scale, where "1" means "not at all characteristic of me" and "5" means "highly characteristic of me."

I collected more than 1,000 responses to these items and several additional, related psychological scales that pertained to motivations and attitudes centered on learning. I found strong support for four of my dimensions and additional evidence that two of my dimensions—"planning" and "scanning"—were very similar to one another, so I began treating those two dimensions as one. Furthermore, I reduced the number of statements down to 23 while retaining the ability to measure each dimension accurately. These five dimensions were very strongly related to other learning concepts, like one's motivation to learn, one's tendency to think about one's own thinking and learning, and one's tendency to act in a goal-driven manner.

What I learned about learning

Given this evidence, I conducted a second study to confirm the five dimensions and 23 items I found in the first study. I conducted this study in an organization where I asked participants to respond to my "informal learning" scale, a "professional confidence" scale, and a "support for learning" scale. The organization also provided performance data to use as an outcome measure.

Specifically, I was interested in determining whether and how professional confidence and perceived support for learning in the organization predicted informal learning behaviors and whether informal learning behaviors predicted job performance. It is increasingly common in the organizational science to test statistical models of individual and organizational level variables; in this case, I was interested in determining the extent to which one's confidence in their ability to do their job affected informal learning (are confident people more likely to self-guide their own learning?) and the extent to which a culture of learning (or lack thereof) bolstered or inhibited learning (if one thinks they won't be rewarded for learning, will they perform learning behaviors anyway?). Finally, job performance is a common outcome to measure in the organizational sciences, as researchers and practitioners in the field are often concerned with the ability to predict and improve performance.

For this study, 167 employees completed the survey. The small sample size limited the ability for the statistical analyses to study all proposed research questions, but I still uncovered several interesting results.

I found that two dimensions of informal learning were strong predictors of job outcomes:

  1. Experimenting behaviors, or a willingness to try new things, predicted job performance and job potential.
  2. Reflecting behaviors, or the extent to which one requests and processes feedback, predicted a likelihood to remain with the organization.

Additionally, the study found a strong relationship between occupational self-efficacy, or one's confidence in his or her ability to do their job, and informal learning.

Some results and implications

Succeeding in any organization—but especially an open organization—requires one to develop as a continuous learner. Evidence supports this claim.

The second study provided some compelling indications that informal learning is not only theoretically but also practically important to both the employee and the organization. Succeeding in any organization—but especially an open organization—requires one to develop as a continuous learner. Evidence supports this claim. Those who had well-developed habits of experimenting and reflecting were more likely to be top performers who were well-ingrained in the organization.

That's not to say that these are the only important behaviors. But they do represent important areas of concern for both individual contributors, managers, and other leaders in an organization. If you're intrigued and want to begin supporting informal learning in a concrete way, consider the following:

  • Evidence from these studies suggest that informal learning is closely related to one's confidence in their ability to do their job. Confidence can be developed through motivation and work design; for example, giving an employee the ability to shape his or her own tasks or role, which cultivates a sense of autonomy and curiosity, would likely spur learning behaviors.
  • Following up with support and rewards for doing so, which are strong motivators, would reinforce that learning. You might also consider educating yourself or your team about learning behaviors and providing, where applicable, tools and opportunities to develop those behaviors. For example, research has shown that reflecting via note-taking or journaling has been very effective at bolstering reflecting abilities and later ability to recall information.

Learning informally at work is incredibly common; however, treating learning as a skill is far less common. Hopefully this article series has better equipped you to think about and support learning at work using the latest research has to offer on the topic.

Read the first part

Writing in a notebook

In open organizations with cultures of adaptability, learning should be continuous—and won't always happen in a formal setting. Do we really understand how it works?

About the author

Colin Willis - Colin Willis is an industrial organizational psychology consultant at HireVue. He recently obtained his doctorate in IO psychology, where his research focused on studying learning at work. Colin is an avid supporter of using science, data, and technology to improve the workplace.