Our discussion today centers around the paper entitled: “Assessing the Quality of Safety-Focused Leadership Engagements” by Siddharth Bhandari, Matthew R. Hallowell, Caleb Scheve, James Upton, Wael Alruqi, and Mike Quashne– published by the American Society of Safety Professionals in January 2022.
The authors’ goal was to produce a scoring protocol for safety-focused leadership engagements that reflects the consensus of a panel of industry experts. Therefore, the authors adopted a multiphased focus group research protocol to address three fundamental questions:
1. What are the characteristics of a high-quality leadership engagement?
2. What is the relative importance of these characteristics?
3. What is the reliability of the scorecard to assess the quality of leadership engagement?
Just like the last episode’s paper, the research has merit, even though it was published in a trade journal and not an academic one. The researchers interviewed 11 safety experts and identified 37 safety protocols to rank. This is a good starting point, but it would be better to also find out what these activities look like when they’re “done well,” and what success looks like when the safety measures, protocols, or attributes “work well.”
The Paper’s Main Research Takeaways:
Discussion Points:
Quotes:
“If the measure itself drives a change to the practice, then I think that is helpful as well.” - Dr. David
“I think just the exercise of trying to find those quality metrics gets us to think harder about what are we really trying to achieve by this activity.” - Dr. Drew
“So I love the fact that they’ve said okay, we’re talking specifically about people who aren’t normally on-site, who are coming on-site, and the purpose is specifically a conversation about safety engagement. So it’s not to do an audit or some other activity.” - Dr. Drew
“The goal of this research was to produce a scoring protocol for safety-focused leadership engagements, that reflects the common consensus of a panel of industry experts.” - Dr. David
“We’ve been moving towards genuine physical disconnections between people doing work and the people trying to lead, and so it makes sense that over the next little while, companies are going to make very deliberate conscious efforts to reconnect, and to re-engage.” - Dr. Drew
“I suspect people are going to be begging for tools like this in the next couple of years.” - Dr. Drew
“At least the researchers have put a tentative idea out there now, which can be directly tested in the next phase, hopefully, of their research, or someone else’s research.” - Dr. Drew
Resources:
The Safety of Work on LinkedIn
Drew: You're listening to The Safety of Work Podcast, episode 94. What makes a quality leadership engagement for safety? Let's get started.
Hey, everybody. My name is Drew Rae. I'm here with David Provan, and we're from the Safety Science Innovation Lab at Griffith University in Australia. Welcome to The Safety of Work Podcast. In each episode, we ask an important question in relation to the safety of work or the work of safety and we examine the evidence surrounding it.
David, what's today's question?
David: Today's question is all about deliberate leadership activities that are designed to foster safety engagement with the workforce. We've done a few episodes on safety leadership, particularly how it may or may not differ from general leadership and also how leaders experience leadership development activities, but we haven't covered this sort of activity where leaders are deliberately and routinely engaging with their workforce for the purpose of trying to strengthen (I guess) the safety climate.
Visible felt leadership has been what we've known for 20 or 25 years with DuPont programs, leadership safety interactions, leadership safety conversations, and a much longer history with things like Gemba walks. These processes are a very common feature of our organization's safety management programs. Drew, I'm very keen on your initial thoughts about the safety work activities.
Drew: As with any safety work, we need to think about what's the mechanism by which it's going to influence the safety of work. All of these activities ultimately come down to the same mechanism, which is that they're trying to increase safety by changing the motivation of workers. Specifically, it's a little bit of a stretch, but it's fair to say that they're probably drawing on a thing called the hygiene-motivation theory from around the 1960s.
The central idea is that if workers show concern for their workers which can be through leadership activities. It can also be through just making the workplace better, putting in drinking fountains, putting cola in the fridge, giving free fruit, and giving more breaks. That motivates the workers to want to satisfy the leaders which assumes that the workers believe that working safer will satisfy the leaders—which is sometimes a big question—causes the workers to want to work more safely.
I guess my concern is not so much about whether it works or not. It’s as to how much effort we put into pulling on the lever of motivation when there are so many other things that might have better influence over safety that might be easier for leaders to spend time and effort on. Motivation seems to be of a weirdly indirect way when we could actually improve the conditions of work for the purpose of safety rather than in order to motivate the workers as our lever.
David: I guess I've never really thought that it's effective to go and try to give workers a pep talk as leaders. What I have sometimes thought though is that it's an important way of leaders in understanding what workers have done in their business. Leaders are going out and undertaking these activities not to necessarily motivate the workers. It's actually to update their own knowledge about how work is going on in their organization to hopefully provide more useful context for the decisions that they make during the normal course of their leadership role.
One leader that I spoke to once asked me this question, why should I go out and do these leadership visits? I don't know the work. I don't know the hazards. I wouldn't know what to do. My comment to him was that if there are, say, four or five levels of management between you and the workforce and each level of management only passes on about 70% or 80% of the truth, by the time information flows up your normal management chain of command and by the time it gets to a middle or a senior manager, it's very unlikely to be representative of actually what is going on in the ground. My idea around these visits is actually about updating the leader's model of work and risk rather than contributing anything to the work that's going on at that point in time.
Drew: That's really interesting, David, because I have heard both of those stories that organizations often get mixed up about which one they're doing. The way in which we measure the quality of those conversations would be very different from those two perspectives.
If we wanted to measure how well they work at motivating the workers, then the obvious thing we should do is measure how much the workers get motivated. If we want to measure how much the leaders learned, then we should debrief the leaders afterward, find out how much they've learned, and what are they going to do differently. Two entirely different purposes of the conversation.
It ties into the two different meanings of what it means to be a good listener. One version of a good listener is we have a conversation and you go away afterward thinking, hey, Drew really listened. He really cares. You feel cared for and motivated.
The other way is I go away and think, hey, I learned so much from that conversation. David taught me all this stuff. I'd better go and implement a different management practice now because my decision is incompatible with the way work's done. Those are two very different types of good listening.
David: Just to put an emphasis on that distinction, we need to be clear on what we're trying to do or achieve with any safety work activity, like you said, in its contribution to the safety of work. That should inform how we evaluate it.
We're going to come back with the paper that we'll talk about today and just check around the effectiveness of some of the evaluation processes used in this paper. But nevertheless, we're going to talk about a paper.
But before we introduce the paper, I thought we just introduced the underlying premise of the paper that we're going to speak about. The underlying reason for doing the research from the authors was this commonly accepted need to move on from lagging indicators. Their idea of an alternative was to try to measure the quality of certain safety system elements.
In this instance, if one of the important elements of our safety management system is management commitment and leadership engagement with the workforce, the authors suggested that we really need to measure the quality of these activities and also concluded that most of our safety lead indicators that we have related to safety work activities typically only measure the frequency with which they occur. The assumption there would be if I do a hundred of these activities, it's more impactful than doing 10 of these activities. These authors had a different view that no, it doesn't matter if you do 10 or a hundred. It's going to be about the quality.
Drew: David, we've had this conversation offline. I don't know if we talked about this on the podcast before. What's your thought about that move saying, okay, we don't want to measure the quantity of safety work because that just drives doing more safety work. People instead say, okay, let's not measure the quantity, let's measure the quality, but they're still measuring the safety work, not the safety itself. Do you think it's an improvement? Do you think it's likely to work better as a measure or have better side effects as a measure?
David: It has to get us closer to, like we said, the connection between the two. If the measure itself drives a change to the practice, then that is helpful as well because we know that the measures at the moment drive volume. We know that it drives box-ticking and pencil-whipping of our safety work processes to do the 10 or the 15 every month. Moving on from that is important.
This quality idea, whether it's an audit, you might go, okay, demonstrate the materiality of the risk reduction that's been generated by the safety audit activity. To do that, the auditor has to find new risks and risks with additional mitigation potential, work on designing additional controls or layers of protection, and work with the business on implementing those controls.
If the measure of the effectiveness of an audit was not the number of noncompliances but the magnitude of safety fatality risk reduction in the six months following the audit driven by the audit, then we could have a bit of fun with designing quality measures for a lot of our safety processes that are going to be (in my opinion) better than what we've got today.
Drew: At the very least, I think just the exercise of trying to find those quality metrics gets us to think harder about what we are really trying to achieve by this activity. I haven't actually done this yet. It'll be interesting to see through this paper what the underlying assumptions are here about what they think these conversations are for based on what they think are good measures of quality of these conversations when they lean more towards the things that indicate that it would be very motivating or the things that would lean towards this would be very good for collecting information for the leader to incorporate into future decision-making.
David: I agree, Drew, and hopefully, the researcher as well. It's a fun thought exercise, but hopefully, the idea of trying to measure these in anger and try to come up with what we think are the dependent variables of some of these safety work activities and then actually try to experiment with changing some of these practices and measures.
Drew, just making sure that we're all on the same page about what are leadership safety engagements before we introduce the paper. In the introduction for this paper or the literature review background, they defined leadership engagement as productive, meaningful interactions with leaders that may improve employee engagement levels and increase commitment to the organization's safety mission. So leaders deliberately go out. This is where you can see this paper leaning very much towards the motivation side of the leadership engagement equation that we mentioned earlier.
Drew: I don't want to downplay the fact that motivation is important. The paper mentions a lot of research that has not directly to do with safety and about how important it is to have an engaged and motivated workforce.
That is a worthwhile goal separate from safety. We want people to care about coming to work. We want them to care about the company. We want them to think that their leaders care about them. All of those things are just good for living a happy life, given that we live lots of our lives at work.
But often, safety gets tacked in with a whole lot of other measures. It's just one thing that increases. Often in those studies, it's not measured very well. It's just measured by employee perceptions of safety which have this halo effect, that if someone says they're more productive, they're happier, and they're more committed, they're also probably going to say they care more about safety.
David: Drew, maybe that introduces the paper with the assumption that these deliberate leadership engagement activities are a good idea and maybe we need to firm that up. We can't really have this episode without talking about the Deepwater Horizon incident and this infamous leadership visit that goes on the day before the Macondo well blowout. I guess it's mitigating this challenge of what are these visits designed to do.
Let's introduce the paper now. The title of this paper is Assessing the Quality of Safety-Focused Leadership Engagements. The authors are Siddharth Bhandari, Matthew Hallowell, Caleb Scheve, James Upton, Wael Alruqi, and Mike Quashne.
Drew, all of these researchers are connected with the Construction Safety Research Alliance at the University of Colorado, chaired by Matthew Hallowell. Some of our long-time listeners might remember a podcast we did, episode 55, which was about a paper produced by Matthew in the same research center on the statistical invalidity of recordable injury rates. Some of the authors, however, are industry practitioners working as safety managers within the industry.
The paper was published in the Professional Safety Journal which is the peer-reviewed journal of the American Society of Safety Professionals. It was published in January 2022, so only a couple of months ago. It is open access published, so we'll be able to put a link in the comments of this episode on LinkedIn.
Drew: Just a word about provenance there. It seems to be that the Construction Safety Research Alliance publishes a lot of their stuff in these professional journals rather than academic journals, which shows a real intent that they're producing work that is directly useful for practitioners.
I don't think it says anything negative about the academic quality of the work. In fact, what they often do is they'll publish something first completely open access as a technical report, and then they'll later submit it to a journal for peer review to make sure that there's always an open access version that's out there.
I don't think any of the provenances here indicates any lack of reliability, for example, in the way in which they talk about their sources, the way they reference literature, and the way they make claims. I think all of that is totally fine.
We will have a little bit of discussion later in the episode about the method, but they were talking about the method in its own right. I've seen this same method used in papers published in academic journals, and it irritated me just as much as the error that it does here.
David: In terms of that, I think researchers also want their research and their ideas to be read, and some of these professional safety outlets, particularly if you're writing for practitioners as a lot of these research centers are, then it's important that you understand where you're putting it and whether or not it's likely to be read there or not.
Next week, Drew, I guess I'm sick of getting my inbox spammed about asking, when are we going to do the Take 5 paper? Listeners, next episode, we'll finally do the Take 5 paper. Drew, your choice of publication for that paper was also based on making sure that it could be available for open access.
Drew: Yes. That one (again) we deliberately published in a journal with less academic stature in order to make sure that it was open access and available to practitioners. Often, researchers have to make these trade-offs. Do you go over somewhere that increases your academic reputation, or do you go for somewhere that is accessible? It doesn't always say much about the quality of the paper.
David: When we're talking about leaders and leadership—we'll get into the method shortly—they define a leader as a highly influential individual who is not involved on-site for day-to-day operations. This is a leader who's not based on that site or primarily involved in day-to-day operations on a particular site as deliberately and purposefully attending that site to engage with the workforce primarily to talk about safety.
The main objective of that visit is to show care and concern for the people, to promote safety not being sacrificed for other competing business priorities like pressure or production, and for the workforce to accept their responsibility for safety. Based on the idea that what leaders pay attention to, what they react to, what they allocate resources for, and what they talk about and acknowledge in their engagement with employees, will set the foundation of the company's safety culture. Drew, very much as we talked about around the start of the episode with these types of visits.
Drew: Yup. But what I really like here is the way the authors have narrowed down what type of leadership they're talking about and what engagement they're talking about. It is really easy in any discussion about leadership or research about leadership just to get lost in these vague definitions where we're unsure if we're talking about CEOs, safety professionals, or supervisors, where everyone's a leader or no one's a leader, or what counts as leadership activity. Is it anything that a manager does?
I love the fact that they've said, okay, we're talking specifically about people who aren't normally on-site and are coming on-site. The purpose is specifically a conversation about safety engagement. It's not to do an audit or to do some other activity that might have safety as part of it or some more instrumental objective. We've got a very clear, narrow thing that we're examining. That lets us examine nicely and cleanly what counts as doing this well and what counts as not doing this well.
David: The broad method was the researchers started by going to the literature and identifying the key positive attributes of this leadership engagement and what makes a good leadership engagement based on research that's been done previously. Then coordinated a focus group of 11 safety practitioners or managers to brainstorm with them what else should be considered for an effective safety focus leadership engagement.
Then went through a process to score the relative importance of each of these attributes with their panel of safety experts, and then organized these results into a leadership engagement scorecard that enables an independent person to do an assessment of the quality of interaction.
That was a goal, Drew. The goal of this research was to produce a scoring protocol for safety-focused leadership engagements that reflects the common consensus of a panel of industry experts, so what are the quality criteria for these visits.
Drew: The end result of this is literally a scorecard. In the paper, they present what almost looks like a laminated form that you're going to take and be able to watch a leader and check off how well they've performed the engagement.
David: Do you want me to talk a little bit more about what went on in those phases of the study?
Drew: Yeah. Why don't you talk through exactly what they did, and then I might make some comments on the overall process?
David: The core part of the research was focus groups with these 11 experts. They're all certified safety professionals in construction safety all in the US. They were all active members of this Construction Safety Research Alliance and had an average of 13 years of safety experience and a bit more experience in their industry.
In Phase I, the researchers did a literature review. They identified the validated attributes in the previous literature. I would love for the researchers to actually report what attributes they carried through from the literature into the first round of the discussion with participants, but we actually don't get a sense of that.
What they did was they basically gave this predetermined list from the research to the focus group and then ran brainstorming sessions, which they essentially did as a round-robin format so that the participants didn't get stuck discussing indefinitely one topic. They just went from person to person and kept going around the room until all ideas were exhausted. They identified 37 unique attributes that could be considered to be criteria for effective safety leadership engagements.
Then, they went into Phase II. Phase II was about prioritizing and rating the attributes. I've got the impression that this was probably all done in the same session, but it was about doing three rounds of anonymous surveys. Each of the 11 experts would rate the importance of each element on a scale of 1–5, 1 for not critical at all, 3–5 is very critical. Then, they facilitated a discussion about the degree of consensus on each attribute and went back into an anonymous survey.
They got to see what the main was of the group, and then they got to reevaluate it for themselves following the discussion. They went through three rounds of this. Then at the end of that, anything with a rating of not critical, which was a two out of five, was discarded. The intention was also in here to describe observable behaviors. Remember, it's almost like the safety person is probably going to be standing off to the side with a clipboard rating, the interaction between a leader and a worker.
Phase III of the study, Drew, was a reliability check. They did a process where they recorded some videos of a few leadership interactions. They got an internal panel, which was the people who were involved with the study, then two external panels of people who weren't involved with the study at all. They asked them to use this checklist to observe the interaction and rate them.
Their target score was to end up sort of plus or minus 5%. So you get a score on this checklist. It's the 15 factors by the weighting by the score, and they achieved on their first round without any discussion or review. They claim that they've achieved this reliability threshold.
That's really the process. Go to the literature, find out what might be important. Ask 11 safety managers what they think, go through a process of scoring and prioritizing these things, forget your checklist, run it through a couple of lab tests with people who weren't involved with the study and see if they all observed the same things. Then we move forward.
I guess the challenge that we've had since the start of this episode is this question of validity. What are we actually measuring? Do we know what we're actually measuring? Maybe I'll hand it over to you and you can talk about the methodology from your perspective.
Drew: Sure. I should point out that this style of doing research is something that is increasingly popular recently. As far as an example of this sort of research, this is as rigorous as it gets. My concerns aren't really about the rigor, they're just about this way of doing research in the first place. If you think carefully about what they have investigated and what they have discovered, there's zero new knowledge being generated about what makes a good leadership interaction.
I give the authors full credit. I don't think they claim that most people who use this sort of method also make claims what makes a good interaction. We've done research and we've got an answer for you. They don't make that claim, but they're implicitly making that claim. They're implicitly saying, these are the 15 things that we can use to measure a good interaction. They've never measured what makes a good interaction. All they’ve done is started with what the existing research already says. Then they've asked a bunch of people what they think.
So your accurate summary would be, what do 11 experts think are the 15 things you should measure? Having already been primed by what we told them they should think, here's the list. Nowhere have they actually tested what sorts of behaviors get the sorts of results that you want from a leadership interaction. This is best described as a consensus of what people already think.
From their point of view, I don't really think that it's research at all. All it is, is construction of a consolidated what we already believe. That's a real problem, given that, I genuinely think that the things that they've got on that list are fairly incoherent when it comes to theory of what would make a good interaction.
If the intent is to find out what sort of behaviors motivate workers or what sort of behaviors change workers care about safety, then we could easily, for the same amount of work, test that out. If we're going to go to the trouble of having these interactions, videoing them, and have the experts watch them, then why not at least ask the person on the other end of the interaction what they thought?
Have you known three interactions where the person on the other end of the interaction felt demotivated? Three interactions where the person on the other end of the interaction felt motivated? And test whether the experts rate the ones that did the high motivation higher. That would be good validating the actual purpose of the tool, whereas all they're really validating is do experts using this tool come to the same answer as each other, even though that answer might be totally different to what the worker thinks.
That's my sort of big concern here. It’s just what exactly you're trying to do by this research. It's missing that key step of does this tool bear any relationship towards what actually motivates or directly motivates workers in an interaction?
David: Yeah, Drew. I think that's the most important consideration for us to carry forward as we're about to discuss some of the results in that. We're exploring what makes an effective engagement between leaders and workers by asking people who are neither of those two. I guess that becomes problematic when we're just trying to make conclusions that we've got an answer here.
I agree with you entirely, Drew. The research at the end suggests that we should move forward to try to validate this checklist or criteria and leadership engagements with performance going forward. It doesn't elaborate on what their performance would look like. But knowing these researchers, I'm sure they're not talking about injury rates. I guess my comment earlier was that we need to know what is this dependent variable. Let's do the research on measuring that.
So Drew, I think we should still talk about what these 15 elements are, because I'm sure many of our listeners would have their own views as the 11 experts in this study had their views on what makes an effective leadership engagement. Without too much detail, I thought it would just be worth running through the 15 just to leave our listeners with something that was going to either interest them to pick up this paper or not. Happy for us to do that.
Drew: As long as I'm allowed to mock some of these as we go through.
David: Sure. How about we do that?
Drew: I suspect many of our listeners are going to think the same thing I do, that some of these are just so blandly common sense that they're not useful. Some of them, I think, are genuinely probably spot on and some of them are weirdly creepy. But also, what do I know because I'm not the worker being engaged with. I mean, literally, this is the sort of work where, I don't know if the people do this, people phoned me up to be involved in this sort of research.
My immediate response is just I'm not the worker, you should not be asking my opinion, it doesn't matter. By the way, take any of my opinions with a grain of salt. I am really not the person who gets to judge whether these are good or bad ideas. We should be measuring it based on the effect of the type of workers who would be subjected to this process. That's not going to stop me from mocking.
David: No, no, no, nor should it, Drew. With that caveat, I don't get asked to do much research on this. I've just got a little while as a safety manager, similar to the experts in this study, where I would have happily sat around a table and thrown things out with what I think but it would have only just been what I think.
First is that the leaders prepared to be on site. Okay, the leader was actually wearing the correct PPE. They knew what they were there for. They turned up in the right place and was I guess, not turning up in a work boots site without wearing work boots.
Drew: I'm really curious as to exactly what this one means and what impact it has on workers. Does this mean the worker looked like they're meant to be there? Does it mean that they've got the mandatory pair of gloves that have never been open to clips to their belt because the policy says that every site visitor must be wearing gloves? Does it mean that they've got the bright shiny helmet that says I've never had a safety sticker attached to his helmet and peeled off again? Or is it like just literally, have they complied with PPE and don't look like they're in a suit?
David: I think we could have a long conversation about each of these. I think it's this idea of leaders who are showing that they, I guess around motivation, actually know what the safety rules are, and that those rules apply to them as much as they do to the workers.
The second one here is that the leader was paying attention during the engagement. They weren't on their phone. They weren't filling out paperwork. They were using positive body language. They asked questions and acknowledged what was being said. So I guess, paying attention and actively listening.
Drew: I'm torn between wondering if there are people who do so badly as to fail this one, or as to whether everyone shows some level of paying attention. This then becomes highly, highly subjective, as to whether the observer thinks that the leader is using deliberately active body language to show that they're paying attention.
You're deliberately leaning forward and nodding and basically asking you how well is the leader able to do all of those fake, very, very active listening things that would turn off most people when you see someone with that sort of level of you creepy active listening, as opposed to genuine concern?
David: Then you're exactly right, Drew, because I should point out to the listeners that the scoring on the scorecard is only zero or one. It's only true or false. It's yes or no for each of these things.
The third was that the leader was not conducting a safety audit. They introduced themselves and they clarified the reason for the visit. It says here that they use paperwork as an opportunity to have conversations, rather than that paperwork might be site paperwork like JSAs or work procedures or Take 5s as an opportunity to have conversations rather than inspect the accuracy or the completion of it, and they were friendly and personal.
Drew: I actually think that this one is a good one because under any theory of safety, you often have this leader who doesn't quite know what they're meant to be doing. So they end up just asking you a bunch of closed questions trying to check that safety is being done properly, which helps no one. It doesn't motivate the worker, doesn't help the leader find out how work is actually done, and doesn't show sincerity.
I think that's actually probably a good tip. You want to remind leaders when they're doing any sort of visit, you're not there to audit safety. It's better if you don't come across as if you're trying to audit safety. I think that's a genuinely good one.
David: The leader attempted to understand and learn the project and the specific challenges faced by workers. They ask questions to really understand what challenges the workers are facing. I guess they verbally appreciated when those challenges were reported. They communicate the importance of the work.
Drew: That's another good one. Whether you're there to motivate or whether you're there to learn, that's a big difference between positive and very superficial engagement is whether you show that you're trying to understand what the work is saying and find out about the work and you care about what they're doing.
David: The leader used names in the conversation. They basically found out people's names and used them throughout the course of the engagement.
Drew: This is where I'm glad it scored zero to one because using someone's name once in a conversation shows that you know their name and you care. Using their name multiple times shows that you've been doing a pick-up artists convention, and you are being really, really creepy. David, when I use your name in every second sentence, David, it shows that I really care about you, David.
David: Yeah, that is a bit creepy. All right. The leader learns something personal about the employee. So attempting to shape the conversation in a way that reveals something to do with family background, or hobbies, or something outside of work. They also shared something personal about themselves. They also asked if the worker had any personal concerns related to the work.
Drew: That could be good. Could result in a violation of a number of different HR laws, depending on exactly how you go about it.
David: Yeah. Again, I think this is point six so we've got a few more to go. You understand how when you set 11 safety managers around in a room, these are some of the very common things that we think are useful.
The leader asked questions to understand the job. They really attempted to learn about the tasks and the work environment and now ask questions to and listen to workers describe what they do and how they do it.
Drew: You skipped over one there, which was leader asked what motivates workers to be in the job, which I think both of those are actually really good questions that do make for good conversations. Most people are interested in their own work. Most people, if you show sincere interest in their work, why they do it, why they chose that job, what they like about it, those are actually really good conversational starters for conversations about what work is like as experienced by the workers.
David: A good point. Okay, yeah, so understanding the motivation of the work, I did skip over that one, number seven. Then number eight was asking questions to understand the work. Number nine is the leader showed employees how their job fits into the company's big picture. This is why and purpose, and this is how what you do really helps the organization achieve its overall objectives.
Drew: David, I don’t know about you, but I'm on the fence with that one. There is a lot of research that shows that understanding how your job fits into the big picture does motivate employees. There's a big difference between my job is to go to people's houses and check their meters versus my job is to keep making sure everyone gets fresh water every day and I'm proud of it. I didn't know how much a single conversation with a leader is going to help or not with that. What do you think?
David: I think my level of knowledge of intrinsic and extrinsic motivation around people's purpose in life and work is not good enough to have too much to offer on that, Drew. I just think in my own personal experience, sometimes the search for meaning is a bit more internal than the external.
Drew: Limited is important. I understand both why it would come up in the literature and why a bunch of safety managers getting together would think that is an important thing.
David: Yeah. We spent a lot of time in organizations on purpose and why and how to connect the organization around a common purpose. I can understand why that's on the list.
So number 10, the leader made everyone feel like safety is more important than production. They asked workers about their workplace pressures, and they clearly communicated safety is a top priority by explaining the Stop Work Authority if something's unsafe.
Drew: This one I really actively dislike because they've said all along that this is most about externally observable behaviors. This is precisely not worded as an externally observable behavior. This is an outcome. I don't see the connection between there are some things that they're looking for and that outcome. The idea that telling someone about the Stop Work Authority is going to result in them feeling like safety is more important than production. I don't buy.
I do buy a little bit that a leader who asks lots of questions about safety instead of asking a lot of questions about productivity does carry a subtle message that safety is important. What really only matters is the absence of those questions about productivity. If they do their safety conversation, and then their immediate next thing is talk with the supervisor about whether you're going to meet the targets for the day. That's going to undermine the whole thing.
David: So number 11, the leader asked what is needed to be safer and more effective. They're asking workers, what do you need to be more safe and more efficient, and letting workers share their ideas and suggestions for improving the work environment.
Drew: They've also given this one a higher weighting than the other ones. There are a couple of items here that weighed much higher than the other ones. David, this one, if it's done sincerely, I think genuinely is helping the leader learn. Asking curious questions about danger, asking workers for suggestions could be tokenistic, could be genuine. It really depends on what the leader then does with that information. If you know it ends at the end of the conversation, that's not going to be helpful.
David: I didn't mention the weighting system. Just before we do the final one, each of these items that I've read out were weighted between a two and a five. Then if they're true, you score one, if they're false, you score zero, and you multiply the weighting by the result, and you get the weighted score. There's a maximum score for the checklist of 53. If you pick up the article, you'll be able to see that. Say, if that's true, and the weighting is five, then for that question, you get five points towards your maximum possible 53.
Drew: That which incidentally means I can show up not prepared to be on the job site, not pay attention at all during the engagement, that's zero points. It means I lose a possible five points, which I can make up for if I ask the worker what they need to be safe and more effective. I get the five points back, even if I'm not paying attention.
David: You would, Drew. I'm between the lines of your comment there, Drew. I'm seeing that you think that if we drop this into an organization that leaders might game it.
Drew: Actually, to be honest, David, as we go through these bit by bit, and particularly as we get towards the end of this list, and I might just quickly mention. The next couple leaders ask questions, learn more about the most dangerous parts of the job. Leaders ask questions to learn about how the dangerous parts of the work are controlled. Leaders empower the workers to share ideas for improvement.
We are getting a real genuine sense here that the idea is to, if you were thinking about leaders gaming this system, they would be gaming it by pushing it away from Safety Audit type questions towards trying to engage by trying to understand the work, ask questions, listen to the workers.
Even though I've been critical, and I've been a little bit sarcastic about some of these items, this is the sort of tool that really is pushing people towards engaging by listening rather than engaging by telling people to be safe. That's a constant theme throughout this. It really is quite consistent from that idea of creating more listening conversations rather than more leaders just showing surface concern or asking lots of closed questions.
David: Yeah, I agree. We get 15 items. Although it’s the overview of what they think of those items, or what's missing in the results that the research has, for a reason, I'm not entirely sure they themed those 15 points into 6 overarching themes. I guess these are ways of thinking about safety leadership engagement.
So number one, be genuine and understanding. Number two, demonstrate care. Number three, show humility. Number four, emphasize safety is a priority. Number five, focus on what matters most. And number six, show appreciation and solicit feedback. That six, in my opinion, isn't actually that [...]
Drew: I think they're great. I can imagine a program to train workers in doing better engagement, which focused on those six factors in the training. Use things like the scorecard as a training evaluation measure, to say, have people taken on board the 6 principles, using these 15 items that we're going to use watching you do a conversation, seeing how well you demonstrate each of the things through those 15 items. As sort of development of a training evaluation tool, I think it's actually not bad.
David: I think there's a good point there, Drew. I think, for practically, and it's not in the practical takeaway so I mentioned it now. But we sometimes use these checklists and forms as a proxy for developing capability in people. We may think that some of these things are intuitive, but we also know that people who have been in leadership positions have had to follow in many different companies throughout their career, many different ways of doing these types of activities.
I think it is very important in your organization that you actually invest in developing the capability so that people don't need to worry about the checklist or the report or the form or exactly how they need to fill it out in this organization.
Drew: Something that we skimmed over in the introduction to this paper was, I don't know when exactly they carried out this research. The cards are copyright 2021 so presumably they did the research and nearly 2021, 2020. We've been moving for a long while and the last couple of years have really exaggerated that, towards genuine physical disconnections between people doing work and the people trying to lead and manage them.
It makes sense that over the next little while, companies are going to be making very deliberate conscious efforts to reconnect and to re-engage. It's likely that even if it's not part of your safety program, that safety is just going to get swept up as one of the topics that people are going to need to do this with. I suspect people are going to be begging for tools like this in the next couple of years.
As development of tools go, I don't like the method, but I think the end product is actually really quite good. So long as we're actually equipping people to have these conversations, training them first then using the scorecard to evaluate how well we're doing that, rather than sort of throwing people out there and start scoring them and using that in performance evaluation.
David: I agree. I think there's the substance in this work that is helpful for organizations to think about. Like we said before capability development, but also the quality of these processes. To do that in a way to think carefully about how those sorts of programs always think carefully about how those programs get rolled out and what unintended consequences they might have.
I could just imagine organization, if you just said, here's now our new checklist for leadership visits. If you don't make any efforts to change anything else in the organization, then you may not get what you want to get.
Drew: I think the next step that I'd really like to see here is some sort of evaluation of this against its intended purpose, as if we are trying to connect and re-engage with workers. If we have people who score highly on this scorecard, does that also correlate with the workers feeling that it is sincere? Because it would be very easy to have conversations that are patent around a method and a score. That just creates the insincerity that people feel like in my leader is going through the 15 point checklist. They might be getting a great score, but I certainly don't believe that they really care about me and want to listen.
David: It’s like when you call a call center or something and a day or two later, there’s an email saying, please rate your experience with the call center. What you need is the leader to essentially report who they've done these engagements with so that within 24 or 48 hours, someone can follow up with them to ask them if it increased their motivation or not to work safely.
Drew: David, I don't know if you think you're kidding or not, but that's exactly what I've imagined is about to happen.
David: Excellent. Yes, very good. All right. Should we do practical takeaways, Drew?
Drew: Yes. Let's.
David: Okay, so a few here. I think there are a few takeaways. Some of these might be obvious, but I think they're worth stating through this study. If we move on from lagging indicators in our organizations towards leading indicators of safety work activities, then we really need to think about the quality of these activities, not just their frequency. This study is an example of that. It can be hard and challenging but we really need to not stop at, let's count how many times we do a particular safety work activity.
Drew: Yeah, 100% agree with that, David.
David: The second one, I guess when we're doing research, it's possible to design a reliable process, that may not be a valid one. This could be all too common in safety work activities. What we've got here is we get a group of safety experts together, that we all align on what's really important to do, we throw it out into our organization, and we get consistent results from that particular safety work activity.
We may not actually be doing what we think we're doing for safety. I think incident investigation might be a good one of these. We've got a process that gets repeated over and over again. Safety alert is definitely an example that came to mind. With this, we may design a repeatable, reliable process that may not actually do what it's meant to do.
Drew: I know and respect some of the authors of this paper too much to believe that they will do this. But I'm willing to bet that we're going to see in a few years companies saying, oh, we know that our leadership engagements are working. Look, here's our scorecard system that tells us that they're working. It's just going to be circular. We designed an intervention around this scorecard. The scorecard tells us we're doing well, never actually measures where the workers have in fact, changed their motivation, or that motivation has a direct impact on safety.
David: Good point, Drew. I think it might happen as well in this study. I guess takeaway number three. In the study, 11 experienced safety managers came up with 37 unique factors that were then condensed into 15 items and 6 themes. We're not saying that it's right. We may not even be saying that it's helpful. Although I think we've concluded that it may be at least a starting point. But it would be a very useful exercise for you to do in your own organization to attempt to design a quality criteria for each of your safety work activities. As a tip, I'd suggest involving the people involved in the activity, not just the opinions of the safety organization.
Drew: I go a step further and try to measure two things. One of them, try to measure what good looks like for that activity. What does the activity look like done well? Also try to measure what's a good outcome from that activity. Think about how you should measure that.
This is an example of doing the first. They've tried to characterize what a good conversation looks like. The next step I think is characterizing what would it look like if this works and how would we know whether it's working or not working?
David: I guess, the fourth practical takeaway here I had is that no matter what we've said in this episode, that purposeful leadership safety engagement is probably a good idea as part of your safety management program.
The way that I took your last comment, Drew, I've said is, if the purpose of my leadership safety engagement is to give the worker the increased perception that management care about them, management understand their work, management listen and follow up on their safety needs and ideas for improvement, then we've got to ask workers those three questions if we do safety leadership engagements.
Do you feel like management care for you? Do you feel like management understand the work you do and what you need? Do safety issues get addressed and followed up when you raise them? So Drew, I guess my take is, like you said, define what good looks like and then find a way to actually see if you can measure that.
Drew: I'm inclined to disagree with the way you've put this takeaway, but it may be one of those No true Scotsman situations, in that I agree that if we achieved that through our safety leadership engagements, then that would be a good idea. But I worry that we don't. I think there's an open question as to whether creating specific safety engagements versus leadership engagements, whether that actually undermines the whole approach.
I don't think anyone's really sincerely tested that. I think there's a genuinely open question there about whether putting the safety engagement in this special category with the word safety might in fact undermine them as opposed to incorporating those same curious questions that lead towards safety, that are in conversations that are not about safety might might be more effective. But we don't have evidence about that. I don't want to say that I'm creating a positive takeaway in the opposite direction.
David: No, I researched too. I could probably make an argument for both and won't do that now. Maybe that's a good question for the comments field in LinkedIn where the listeners think that we should do general leadership engagements about the work, which may include safety or may not? Or should we reserve a special place for safety in leadership engagements where there's targeted specific safety engagements between leaders and workers?
Drew: Yeah, good question. I'd love to know what people think.
David: Anything else you want to add?
Drew: No, I’m happy to finish off. The question that we asked this week was what makes a quality leadership engagement for safety? Do we have an answer?
David: Well, I think my answer and I mentioned yours is that we can't be sure from this study. It could be that the 15 items in this study make a quality leadership engagement for safety. But we don't know because we don't know from this study what leaders think and we don't know what workers think from these engagement activities.
We don't have any dependent variable to test the results on the scorecard against some other observable outcome like climate, or even engagement survey results, or anything else that we might try to see if we can give effect to through these leadership engagements.
Drew: But at least the researchers have put a tentative idea out there now, which can be directly tested in the next phase, hopefully, of their research or someone else's research. We've defined what good looks like, now let's test if a good also equals effective.
David: I suspect that research is going on because it was very much talked about at the end of this paper that in the next phase of this study, they wanted to look (going forward) at using the scorecard and measuring other aspects of performance. Hopefully we get to do that on a future episode, Drew.
Drew: I look forward to it. That's it for this week. We hope you found this episode thought provoking and ultimately useful in shaping the safety of work in your own organization. As always, engage with us on LinkedIn or send any comments, questions, or ideas for future episodes to feedback@safetyofwork.com.