We realized that this episode will be our first “three-person” podcast discussion.We’ve invited Jop Havinga, co-author of today’s paper and fellow Griffith University colleague, to join us. Our discussion today centers around the paper “Should We Cut the Cards?"
Assessing the Influence of “Take 5” Pre-Task Risk Assessments on Safety” by Jop Havinga, Mohammed Ibrahim Shire, and our own Andrew Rae. The paper was just published in “Safety,” - an international, peer-reviewed, open-access journal of industrial and human health safety published quarterly online by MDPI.
The paper’s abstract reads:
This paper describes and analyses a particular safety practice, the written pre-task risk assessment commonly referred to as a “Take 5”. The paper draws on data from a trial at a major infrastructure construction project. We conducted interviews and field observations during alternating periods of enforced Take 5 usage, optional Take 5 usage, and banned Take 5 usage. These data, along with evidence from other field studies, were analysed using the method of Functional Interrogation. We found no evidence to support any of the purported mechanisms by which Take 5 might be effective in reducing the risk of workplace accidents. Take 5 does not improve the planning of work, enhance worker heedfulness while conducting work, educate workers about hazards, or assist with organisational awareness and management of hazards. Whilst some workers believe that Take 5 may sometimes be effective, this belief is subject to the “Not for Me” effect, where Take 5 is always believed to be helpful for someone else, at some other time. The adoption and use of Take 5 is most likely to be an adaptive response by individuals and organisations to existing structural pressures. Take 5 provides a social defence, creating an auditable trail of safety work that may reduce anxiety in the present, and deflect blame in the future. Take 5 also serves a signalling function, allowing workers and companies to appear diligent about safety.
Discussion Points:
Quotes:
“You always get taken by surprise when people find other ways to criticize [the research.] I think my favorite criticism is people who immediately hit back by trying to attack the integrity of the research.” - Dr. Drew
“So this link between behavioral psychology and safety science is sometimes very weak, it’s sometimes just a general idea of applying incentives.” - Dr. Drew
“When someone says, ‘we introduced Take 5’s and we reduced our number of accidents by 50%,’ that is nonsense. There is no [one] safety intervention in the world where you could have that level of change and be able to see it.” - Dr. Drew
“It’s really hard to argue that these Take 5s lead to actual better planning of the work they’re conducting.” - Dr. Jop Havinga
“What we saw is just a total disconnect – the behavior happens without the Take 5s, the Take 5s happen without the behavior. The two NEVER actually happened at the same time.” - Dr. Drew
“Considering that Take 5 cards are very generic, they will rarely contain anything new for somebody.” - Dr. Jop Havinga
“Often the people who are furthest removed from the work are most satisfied with Take 5s and most reluctant to get rid of them.” - Dr. Drew
Resources:
The Safety of Work on LinkedIn
David: You're listening to The Safety of Work podcast, episode 95. Do Take 5 risk assessments contribute to safe work? Let's get started.
Hey, everybody. My name is David Provan. Today, I'm here with Drew Rae and Dr. Jop Havinga. We're all from the Safety Science Innovation Lab at Griffith University in Australia.
Welcome to The Safety Work podcast. In each episode, we ask an important question in relation to the safety of work or the work of safety, and we examine the evidence surrounding it. Every few episodes, we indulge ourselves by talking about some of our own work.
Today, we're discussing a paper that's just been published. I've got two of the three authors of that paper with me here today. Drew, of course, you all know, and Jop will introduce himself shortly. We've just worked out in episode 95 that this is the first three-person podcast that we've done. We've got a plan and we'll see how we go.
I'm going to start by reading a few sentences directly out of the abstract from this paper. "We found no evidence to support any of the purported mechanisms by which Take 5 might be effective in reducing the risk of workplace accidents. Take 5 does not improve the planning of work, enhance worker heedfulness while conducting work, educate workers about hazards, or assist with organizational awareness and management of hazards."
In the last week or so, the Internet's blowing up a little bit. I've been talking about it all week with safety professionals from all over the world. It's good to record this and (I guess) get the authors’ perspective out there.
Before we go, let's introduce the paper, and then I'll hand it over to the two who wrote it. The title of this paper is Should We Cut the Cards? Assessing the Influence of “Take 5” Pre-Task Risk Assessments on Safety. It was published in the journal safety MDPI, so it's more recent, smaller, and slightly less prestigious than some of the other safety journals that we have in the podcast. It's very useful for making papers accessible to industry.
In terms of quality of research, all of these journals draw on the same pool of peer reviewers. It's a great avenue to get these papers into the hands of practitioners. We'll be able to put a link to the paper in the comments on LinkedIn, or if you follow Drew, you'll have already read it.
Like I said, the first author is Jop Havinga, second author is Mohammed Ibrahim Shire, and the third author is our own Associate Professor, Andrew Rae. Muhammad was a field researcher in the lab. As part of his postdoctoral work, he did a lot of the early data collection before he took up a permanent position back in the UK.
I've got Jop here now with us. Jop, do you want to introduce yourself?
Jop: Thank you, David. Some of the listeners might remember me from the stockwork episode where I had a brief visit. I'm the first two-time guest on the show as well.
I am a researcher with the Safety Science Innovation Lab. I worked a lot with Drew and Sydney Dekker. I have a background in human factors and psychology and came over to Australia in 2014. I did my PhD at Griffith University where I looked at teamwork processes and routines guiding how teams organize work. Then, I've worked a bit in consultancy and moved back to research mostly around the idea of safety clutter or how safety management system processes influence actual work for better or worse.
David: Thanks, Jop. I hadn't realized that we hadn't had anyone back more than twice. There you go. Welcome.
Drew, how are you feeling about this paper now that you've had a week of fielding comments from the safety world?
Drew: It's always fun when you put a new paper out. In this particular case, we knew we were going to get pushed back because we knew that we were criticizing a practice that is very widespread.
We've called it Take 5 in the paper. We also included a table which has something like 40 other names that this technique is known by around the world. You never know exactly where people are going to push back, so you try to build into the paper defenses or at least comments about the most likely avenues that people are going to respond to, and then you always get taken by surprise when people find other ways to criticize it.
My favorite criticism is people who immediately hit back by trying to attack the integrity of the research, not just the methods but also the motivations and personal integrity of the researchers. It's always difficult to know how to respond to that or whether to just let it by.
What I take great pleasure in is when people want to attack the limits of the research because that tells me that, at least, safety is now starting to get to that point where we care about evidence. If people want to reject the papers because we didn't do a big enough study or a study on enough sites, well that is the best criticism because that says people want more work like this.
The initial message I throw back to anyone who wants to criticize the size, scale, or limits of the research is if you want to disagree with us, then the ball's in your court to take this further. We're happy to work with anyone who wants to try to replicate the methods or extend the research.
David: I guess it wouldn't be the Internet if people were just playing the ball and not the person.
Jop, you're the first author of the paper. You got handed the data in somewhat and moved forward. What are some of your reflections on the process of pulling this paper together?
Jop: The analysis method that we use is very much a product of my experience in the field doing similar work. It's (I think) the second paper that's published where we use it beyond that. It's mostly in industry or organizational reports where we've used it.
I really like unpacking things this way because you really look at different ways that Take 5 can work and give it a chance to see how it works out in practice. It gives you extra depth looking at the data because you basically try to give it another chance every time you look at one of the mechanisms through which it can work.
It forces you to be very critical. Often, while regoing through the data, you recognize that you missed something the first time. I can just say that that's probably where I get excited doing work. I don't know if Drew recognizes it the same while writing up the research (in this case) using this method.
Drew: I really enjoyed… co-writing might be a bit too strong a term, but certainly reviewing each part of it as Jop produced it. And it was like, oh, here's a thing that I hadn't thought of before, and then Jop got an answer to the thing that I hadn't thought of before. Here's another way it might work that I didn't consider, but here's an answer to that.
It's a really useful way to test your own thinking. There were a couple of places where we thought, oh, here's a way that it might work. Actually, we haven't sufficiently tested this with the data, so we have to leave it open as a possibility, and we'll talk about a couple of those spots later in the discussion.
David: Great. Drew, you spent a fair bit of time in the paper looking at the origin of Take 5s. I really like this because it's interesting. If we're trying to think about why this practice might be in organizations, why it might be widespread, or what we think organizations are trying to achieve, history is going to give us some insights into the thinking at the time that these practices emerged.
Do you want to just talk a little bit about the history and a sense of some of the things that surprised you? Some of the early introductions of this process? What would be useful for the listeners to understand the background of this practice?
Drew: In order to tell the history, I need to work backward a little bit and tell some of my own story in trying to understand the history. Take 5s are probably the practice that first got me interested at all in workplace health and safety techniques as opposed to system safety techniques, because the very first time I ran into Take 5, it was in the context of someone complaining about it.
Everything else I've learned, I've been trained how to use it, and then I've started to question the limitations. This was a thing I've never heard of before and already people are saying, this is nonsense. Why are we doing it?
That immediately got me interested because I fired a question back. Okay, so if it's nonsense, why are you doing it? It was fascinating that we'd have people who were head of safety in an organization complaining about a practice that they couldn't get rid of, and they couldn't explain exactly why they couldn't get rid of it, why it was there in the first place, or what it was intended to achieve.
In coming out with the history, I had to try to untangle all of those bits. Why did people do it? Is there some organizational need that's being met? Is there some previous task that it evolved out of? Immediately, when you start to question that, even if you just type into Google or ask on LinkedIn who invented Take 5s, you'd find 50 different people who all think that they independently invented it.
I must have talked to or at least had conversations or exchanged comments with 20 different people, all of whom are certain they know when and where it was first introduced.They are all absolutely wrong because you can always find some time before they say when someone else was doing Take 5.
I tell the full story in the paper. The best I can work out is that there was this thing dating back to Canadian mining at least back in the 1930s called the Neil George Five-Point Safety System. We've got newspaper articles back then that reference what was going on in the mines at the time. It's pretty obvious that the original Five-Point Safety System had nothing to do with what a current Take 5 looks like. But you fast forward to about the 1960s, and we have Australians going over to Canada to try to learn about this system.
When they go to learn in the 1960s and they bring it back, it looks much more like what we currently think of as a Take 5, but it's still not really intended to be this last-minute risk assessment done by the worker. It's still a very top-down, supervisor-driven way. A supervisor comes and visits you, they talk about certain things, they talk about these five things on the card, they fill out the card, and they leave you with the card as proof that they've had this visit, which is not how people use Take 5s today.
At the same time, you've got all these other uses of little cards in safety. You've got people using these training cards that they give out to workers, almost like we get out sometimes advertising bookmarks that remind workers of a particular hazard. Here, read this. Take it home and think about it. We've got laminated cards that get put up at the entrances to places. Here are the instructions for this place.
We've got the general idea that taking five minutes for safety is a good idea. It's not always five minutes, of course. Sometimes, it's Take 10, Take 2, or Take 5. Five minutes can save a life. Ten minutes can save a life. Just take a moment.
All of these ideas coalesce into the introduction of behavior-based safety in the 1980s when we have this idea that we don't just try to change behavior but we try to keep records of behavior. We start doing behavioral observations, we start getting workers to observe each other, and we create these written traces of work.
As far as we understand it, probably what looks like a Take 5 genuinely did just get invented many, many times in many, many places as people combined this idea of we want people to stop and think before they start work with this idea of behavior-based safety where we want a record that they've actually done it. We want them to create some auditable trace that they stopped and thought. Sometimes, they adapted existing systems like the Neil George Five-Point System. Sometimes, they invented brand-new systems. That's why we have it appearing under so many different names.
Is that enough? Does that give some idea of where it came from?
David: I always encourage our listeners to go to the source of these papers. There are some particularly interesting things in the paper that I like that you particularly referenced as a stop, think, and act which came out of (I guess) social psychology or cognitive psychology around dealing with troubled school kids and teenagers. You made this little connection in the paper that maybe some of our workplace practice is designed to treat workers like naughty school children.
Drew: I'm glad you found that line, David. This is the trouble with co-authoring with someone like Jop who eggs me on instead of my other co-authors who try to rein me in.
This link between behavioral psychology and safety science is sometimes very weak. It's sometimes just a general idea of applying incentives. Sometimes, people actually go into the behavioral psychology literature and they get behavioral psychology consultants. They grab training programs and influence programs that are designed for other things.
There are certainly some strongly suggestive traces that once or twice, people—in developing these Take 5 type of systems—have looked at stop, think, and act type of programs that are used for disruptive kids in classrooms, which lets us make the claim that Take 5 is the same as treating workers as naughty school kids. No, it's not. The link is more tenuous than that.
We originally snuck that line in as a footnote to the paper, and then the editors actually said, sorry, you're not allowed footnotes, so we had to move it into the main text.
David: Very good. Drew, it'd be good to get Jop to talk a little bit about the method that he introduced earlier, but before we do that, maybe there are just some framing comments that you might like to make around this study. You mentioned limitations earlier, but what were you actually trying to test? At a very high level, what's the scope of what you did as we dive deeper into the methodology?
Drew: At a really high level, this is not an exploratory study. In an exploratory study, you might start at one site, try to understand what's going on there, and then try to work outwards from there. You say, okay, what we saw on this one site, does it apply on other sites?
In this, we're doing almost the exact opposite. We're saying, all of these stories about Take 5 already exist. We've got lots of people using them, lots of people claiming different benefits, and lots of people complaining about them.
What we want to do is condense all of those stories into almost the purest form where we can test them. Let's create the optimal opportunity for this to work well. We'll make sure that it's not too onerous. We'll make sure that the workers fully understand why they're doing it. We'll make sure that people have got a say in how they're using it and why they're doing it. They know that they're only doing it for a limited time. We're creating the best-case conditions under which if it works at all, it should start working here. That's the experiment that we're doing.
I'll get Jop to actually talk us through what we did on-site and then how we analyzed the results from it.
Jop: On-site, as Drew described, we basically did a math manipulation to create different conditions in which you would expect it to be more or less likely to see the effect of the Take 5. The main manipulation that we did was to either leave it optional, make it mandatory, and prohibit them for a period. Then again, an optional period just to see what happens after the prohibition of the Take 5s, follow what people do in the meantime, and see if we see any indications in their change of behaviors, which then can inform us whether any of the potential mechanisms of how Take 5 contribute to safety playout in practice.
In terms of the larger method to analyze this, we identified potential mechanisms of how Take 5s contribute to safety. One idea is that by filling out a Take 5 card, people engage in better planning and make better plans to put controls in place for the hazards that are there.
Another view is did they increase general heedfulness just by being aware, alert, or directed towards the presence of hazards?
Another idea was that they help in the education of teaching people what is a hazard, what isn't a hazard, and have a long-term change in terms of how safe they work.
And then the final two which were the idea of distributed hazard identification. By everybody filling in a Take 5, the site as a whole can learn about the presence of hazards on-site or which controls are most used.
The final idea is the idea of due diligence. Do guards help show that you care about safety?
David: Jop, we're going to talk about these six things in a moment. Did you want to add anything else to the methods?
Jop: Basically, using these different mechanisms as a lens to look at what people actually do and see if you see any evidence for or against. Some of this was (in the moment) just looking at what you see, and can you categorize it or explain it one way or the other? Some of it was more structured, especially regarding heedfulness. We had a marker system that says, okay, do people look at their colleagues? Do people look around for vehicle hazards? Do people look around for their own body fitness? We took note of that and quantify that so that we could compare that across the different conditions.
Drew: David, let me just say a little bit about why we do the analysis in this way. This (I think) is something that often gets misunderstood when people are looking at safety research and talking about the rigor over it and the quality of it.
It's impossible for almost every type of safety intervention to actually measure that intervention based on the end outcomes, so when someone says, we introduced Take 5s and we reduced our number of accidents by 50%, that is nonsense. There is no safety intervention in the world where you could have that level of change and be able to reliably see that level of change associated just with one particular intervention.
For most safety research, you can spend six months on-site and you will see one or two minor incidents. The chance of actually—in the middle of a study—seeing a major accident is very, very low. If you want to evaluate an intervention, you can't really evaluate it just based on the number of accidents going up or the number of accidents going down. You've got to have some other way of doing it.
What we're doing is instead of looking at that endpoint—where the more people get hurt or fewer people get hurt—we're looking at the middle step. What is the change in site conditions or behavior that changed the likelihood of someone getting hurt?
For this to work, it's got to work somehow. What we're looking for is whether those somehows exist. Some people have pointed out in response that there is always the possibility of another mechanism that we haven't thought of, some invisible sixth or seventh way that it works that we didn't see but is still somehow operating in the background to keep people safer.
We haven't excluded that possibility. We haven't excluded the possibility that there is an invisible force that Take 5s exert that keeps people safer. What we're saying is that if it's working, it's not working by any of these five things that are the five ways that people say it's supposed to work.
Jop: To add to that, if it would work through a sixth or seventh way, it also probably means that we're not designing Take 5s in a good way because they're designed according to these other rationales that we openly recognize, not this forgotten, unknown one.
David: I'm looking forward to getting into the discussion of the different ways that it could work and why you both think it doesn't. Just to capture that, we've got a site and we did an initial observation of the existing practice around Take 5s. For two weeks, they mandated every person and every task. You’ve got over 100 of these cards every day, you've got observers on-site, and you're just watching and recording consistently exactly what people are doing.
Then after that two weeks, you give people four weeks of break almost. They go back to their normal routines, whatever it was. Then, you come along and for two weeks, they're banned. No cards, no books, no one's allowed to do them for any job, zero Take 5s for two weeks, and you've got the same observer. They're watching what they do. Then after that, you let people go back into their normal routines.
I guess if I put words into both of your mouths, what you're saying is with the carefully designed ethnographic research, you see no difference in the markers of the safety of the work that people are performing when there are over 100 Take 5s happening in a day or zero Take 5s happening in a day.
Jop: Yes. We could not recognize any meaningful difference both in predetermined markers as well as on-the-spot, open-ended observation.
David: I'll make sure everyone's got Jop's email address as well as Drew's.
Drew: The one tiny exceptional throw-in is the time when the Take 5s are banned. The number is not exactly zero, which is another thing that we found fascinating and had to explain in the paper. Even if you ban Take 5s, there are still some people who will insist on filling them out. That's something that's got to be explained. You can't just ignore anomalies like that in your research.
David: I think we'll get back to that at the end. I'm fascinated to spend the rest of the podcast talking through some of these areas. One of the ways that our listeners might think that Take 5s could work is to plan the safety of work. The Take 5 is being done just before the job. I kind of know what my job is, but I got to plan safety into that job by going through my Take 5s.
Drew, do you want to start? Maybe I'll ask Jop to comment first on each of these. Drew, you can add something else if you're happy with that. Jop, what's your conclusion out of the research around the role of Take 5s in planning the safety of work?
Jop: First of all, many Take 5s are completed in a crib room and they're completed before they've seen the job or before they have even been assigned the specifics of the job. For those jobs, it's very easy to rule out that they have a positive effect. But even for the cases where you see that the guards are completed after they've been assigned to jobs, there is very little reflection on the actual job at hand.
People somewhat fill in what's easy to write down and what they think will look good. You'll constantly see people take extra actions compared to what they write down. They might position the doors in a smart way so that somebody doesn't walk into them. They might make sure that they have good communication lines with somebody else working a little bit a couple of meters away, which they don't write down but is an important way they keep themselves safe.
The other way around, you see the people writing down controls that they don't actually put in place. They might say they have a fire spotter, but there's no one nearby. So it's really hard to argue that this Take 5 leads to actual better planning of the work they're conducting.
Yet to top that up, in the cases where we did see safety getting incorporated in planning, it was usually at an earlier stage that people could take out extra tools to the site or pick up controls from the depo (extra control). These were reflections that happened usually while talking to the supervisor. That's a step before we expect Take 5 to be completed.
Drew: This is one of those objections that people have in defense of Take 5s that is most interesting and most worth thinking about. You hear people say—and I'm quoting almost directly here from a LinkedIn comment, I'm sorry that I can't get the exact attribution—"I've seen five times over my career when someone has saved their life as a result of the Take 5, that before they were going to do a task, they stopped and thought about what they needed to do. As a result, they did the task differently."
We saw throughout this study lots of times that that happened. This is genuinely a thing that does save people's lives. Before they're going to do something, they stop, think about what they're about to do, and decide, no, that's not the best way to do it. Maybe I need to go and get a different tool. Maybe I actually shouldn't do this today. Maybe I should stop the work. Maybe I should get a second person. Maybe I should make sure there's a fire extinguisher standing by.
Workers do this with their work all the time, but what we didn't see was them doing it directly tied to these Take 5 tasks. None of any of our conclusions should be taken to say, don't take five minutes before you take a task and stop and think about what you're doing. This is not about that individual action. This is about whether the company policy of making people fill out the Take 5 cards actually supports that behavior or not.
What we saw is just a total disconnect. The behavior happens without the Take 5s and the Take 5s happen without the behavior. The two never actually happened at the same time.
David: I'm trying not to insert my personal experience of Take 5s into this research. It is something that I would think is similar to how I think about this. Of course, there's a comment here that you're right, people would get to realize they need a different tool to do a job safely and then go back and get that tool, but that wasn't as a result of the Take 5. That was a result of them going and getting to the job and realizing something based on their experience and what they were looking at. It wasn't like they pulled out this checklist and it gave them that particular answer.
Whether it's good or not, I don't know. I guess there are a lot of people who will be questioning their safety processes, which is good, but it's something that we've got to wonder in safety what the card is doing then.
Drew: We'll come back to this in the conclusions, but I'll just throw it in briefly now. If you are worried about this research but still feel very attached to Take 5s and want not to ignore the evidence but also don't want to get rid of Take 5s altogether, one of the reasons why we think this occurs is that Take 5s in most formats are very generic.
The questions they ask are designed to apply to any job. Often, they leave open spaces for workers to write down hazards and controls, which means that they're only going to tell you stuff that you're already thinking about. If you're not already thinking about it, you're probably not going to extra think about it when you pull out a card and have to fill out the card. That's just a job you have to do.
There is room there. We haven't tested whether much more specific reminders are helpful. Some of the very early things that happened before Take 5s were very job-specific reminder cards. When you're about to go into a confined space, remember these five things. Have you told someone else? Have you checked that the air is clear? Have you got a way back out again? Things like that. If you're about to do hot work, is there a fire extinguisher nearby? Have you checked that there is no painting happening nearby?
Those specific reminders may be more effective. We can't say that because that's not what we tested. It's possible that the reason why they don't work for this is just because they're often so generic.
David: The second way that it could work or that it may be designed to work is by this idea that we Take 5 before a job and it changes into the work we're about to do. We pay more attention to ourselves, our work environment, others around us, and so on. It really just basically creates a mindset for safety for the task.
Jop, do you want to share the outcomes around the relationship between Take 5 and heedfulness, and even talk about what heedfulness is?
Jop: Heedfulness closely aligns with things like mindfulness, awareness, and in this case, specifically to dangers or anything that could affect safety. We had these marker lists for attention, personal comfort, or health and safety in terms of attention for teammates, rules and requirements around work, the work environment and hazards in that environment, tool use, and timing and base of the work.
Comparing all the different groups on-site in terms of how often we saw behaviors that fitted into one of these categories, we saw no differences across the conditions whether they had Take 5s or didn't have Take 5s in that period. There was no indication that Take 5s led to a more general orientation or mindfulness towards hazards or things that could compromise safety.
Drew: One of the things that surprised me from this research is that people talk sometimes just about how sloppy or unsafe the construction industry is compared to other industries, but a lot more of these categories, when you actually look closely at what people are paying attention to and how mindful they are of their own safety and the safety of others, you just see how many little signs there are that people are paying attention to safety.
We've got categories of things that people do that are happening consistently all day every day, people doing little things that show that they're looking out for each other, their tools, and themselves happening over and over again. It's not like this was, oh, we saw people being unsafe and the Take 5s didn't stop them being unsafe or make them safe.
It was, people are doing all of these activities lots of the time and the rate just doesn't go up or down with the use of the Take 5s. It's related to the work that they're doing and the fact that they genuinely care about each other, they care about themselves, and they care about what's going on around them. You don't see many people on a construction site just tuned out ignoring whether things are safe or not.
David: I often have a conversation with safety professionals about a lot of the safety activities that we do in the workplace seemingly for safe work, and even the most staunch supporters of all of these safety practices. When asked how often do you do Take 5s on the weekend for every task you do around the house—before you wash the dishes, mow the lawn, and change the light bulb—very few staunch supporters of safety practices do any of those safety practices themselves.
This is common in this paper that there are still some people in the workplace—I guess I don't know if it's lots or a few people in the study—who said, yes, Take 5s are useful. They're just not useful for me. How would you respond to that comment? I don't know if it quite fits here now, but I just thought of that as you and Jop were talking.
Drew: That's definitely one for Jop to talk about. I wanted to call it the Take 5 effect, but Jop had already named the not-for-me effect. Go for it, Jop.
Jop: It's probably not everyone. I can think of at least one exception on this site as I was later involved in collecting data on the same site. You ask people, is Take 5 useful? Did you get anything out of this? You always hear, look, for me, it's not really needed, but you know the new guys over there? It's probably good that they do it.
They tell you, for this task, it's really ridiculous that they made me do it. If I would be working at heights or doing something risky with the big crane over there, then sure, I'll do it. If you then ask somebody working at heights or working with that crane if they like to Take 5, they tell you no, it doesn't add anything to what I'm doing. You're forever chasing this situation or this person for whom it might be useful to.
Then, about novices—which is actually one we'll get into as well—you ask them, did you learn through the Take 5? Was it helpful for yourself learning about hazards on-site through the Take 5? They said, no, I was taught through mentors. These mentors were great and they told me what to watch out for. That's way better than a Take 5. Even themselves don't seem fully convinced of it actually being good for somebody else. It's clearly not good for them, but they don't want to fully reject it for other people.
Drew: David, I might throw in here another one of the comments we've had in response to the paper in which people have said, you're not actually measuring anything. All you're doing is talking to people and watching people. How do you actually know if people are paying attention or not?
People have got this idea that there is almost an invisible internal to yourself sense of being alert or paying attention that someone outside can't see, and certainly we as observers wouldn't have been able to see in this study.
I might actually try that one directly back to Jop because you're more of an expert in whether that makes sense or not, this idea of alertness that is invisible just inside someone's head.
Jop: This gets to the Cartesian Dualism mind-body problem. I think one of the things is that we also need to recognize we don't have perfect self-knowledge. In that sense, I can catch myself daydreaming just as I can catch somebody being engaged with a task or not. I don't think the two are as fundamentally different in the first place, but if you see somebody following an object, looking at an object, and responding to the movements of an object, it is very hard to argue that they're not paying attention to that object. Of course, there is always a chance of getting something wrong in observation but so is that chance when you're examining yourself.
Drew: Do you think it's possible that for some jobs, it's possible to just be doing it automatically by rote not thinking about it and someone from the outside couldn't tell the difference, that a tool like a Take 5 could make that difference and could make one person do the task mindfully and the other person do the task in the same way but less mindfully?
Jop: I am not completely sure if the latest science supports me on this, but my inclination is that if somebody can do it without (let's say) consciously thinking in terms of explicit words, it is a sign that they have a well-established routine mental model of how to deal with a task. These people would also be the most sensitive to changes in the environment, that this mental model or these expectations are not working anymore at this time.
In that sense, while they might have routines and certain things they don't pay attention to, they are the first ones to recognize that they're outside of this routine and that they need to pay extra attention to whatever is going on to them.
In terms of research, I know at least that in experts' reasoning in terms of doctors and analyzing difficult cases with patients, the expert doctors might be the worst at explaining what they're thinking about, but they're the first at recognizing that there is something unusual about this patient, whether they're going down to a garden path situation of it seems this but it is actually that. In that sense, the expertise or the lack of apparent mindfulness might actually be a sign that this person is really suited for the task.
Drew: Let me ask the same question in another way. Would there be value in redoing the experiment, slapping EKGs on all the workers, and seeing if there was any difference in brain patterns as a result of getting them to do the safety activity?
Jop: I'm personally very skeptical of a lot of neuro stuff in that it's so hard to interpret what it actually means. Eye-tracking is different as it's easier to interpret. At least you know whether they've rested their eyes on something. But even there, what we know from experts is that they need less information to conclude more or to learn more about a situation.
Drew: So even something like eye movement might not be able to tell. We could tell the difference, we just couldn't tell if it was good or bad.
Jop: In that sense, yeah. If you're observing, one of the strongest senses of evidence is whether somebody responds to something, whether there's an indication, or whether they've taken that into account in their plan. Ideally, you see it in behavior and sometimes you would ask about that afterward, beforehand, or whenever is an appropriate moment.
I would personally be inclined that I don't see a lot of situations where brain monitoring or eye-tracking would be valuable, but of course, you can't rule it out.
David: I'm going to throw an experimental design at both of you, and then we'll move on. Since we've taken this big sidetrack, let's do that.
I was thinking about skill-based workers and rule-based workers, this expertise and novice thing that we've known about since Rasmussen talked about. What if I had four golfers—two professional golfers and two novice golfers—and two different Take 5 situations?
One card was a general sports prompt sheet—set yourself, breathe deeply, focus on what you're doing, and take your time—but said nothing about how to play golf, just a general sports thing.
The other golfer got a check your grip, check your foot position, and check whatever is actually relevant to hitting a golf ball straight. The same with novices—a novice with the general thing and a novice with the specific golf swing reminders. See who hits the ball straighter. Is that what we're talking about here?
Drew: I don't know what Jop thinks, but because the individual performance varies so much, I would want to actually apply the same design that we did in these studies. I would want to send them all on a couple of rounds with none of these cards, then all on a couple of rounds with one of the cards, then give them another couple of times to go around with no cards, then another couple of times with the next card, then no couple of times with no cards, and then give them a choice of which card they wanted to go around with.
Actually, even in that case, I'd still want to go back to the type of design we used that uses the on-off and lets us observe performance in each mode rather than trying to directly compare to people who might not be comparable in the first place. That's why we couldn't (for this study) just pick two worksites and give one the cards and one not the card. There might be other things that cause those two worksites to be different.
David: Thanks for indulging my design idea. Let's move on. Jop introduced education. There's this idea that these cards might be useful for hazard education effects, so reminding people a lot about the types of hazards associated with work might help them learn more about the hazards associated with work. Jop, do you want to share why you don't think this happens either?
Jop: The first part is, as I've mentioned, none of the experienced operators and construction workers said they learned themselves this way. That at least takes away that people themselves think this is helpful, even the ones that did start in the industry while Take 5s were already around.
But to learn there is just another challenge here and that you either need to be provided with something you didn't know yet or at least isn't ingrained, or you need to get feedback on your performance and get any knowledge whether what you did was right or wrong. Considering that Take 5 cards are very generic, they will rarely contain anything new for somebody. It's almost impossible. After you've done it once, it's basically impossible that they offer anything new.
In terms of feedback, maybe once or twice when a supervisor drops by, they make a comment about what's written on the card, but normally, people don't get any feedback on what they write on the card. The chance that they get feedback through hurting themselves is pretty small because we know how few accidents happen on most worksites. They would have gotten that feedback without filling in the Take 5 cards as well. In that sense, it's really hard to argue that the Take 5 contributes to this factor at least in the current form that it is applied on most sites.
Drew: We definitely didn't directly test Take 5s as an education measure. If you were testing them, you'd probably do them with more of that controlled group analysis. You take one group of apprentices and train them using Take 5s, take another group and train them using a different tool, and see which group ended up with the most knowledge under some knowledge test in a month's time. We didn't do that study.
It seems unlikely that Take 5s just accidentally work as an education measure. You'd need to redesign them, build a program of feedback around them, and then give them a proper test if you thought that they might be good for that thing.
We're not aware of anyone who is using them explicitly like that. It's more just of a fallback claim. Oh, they don't work for this, but they do work for novices. But to work for novices, they'd have to have these educational components wrapped around them.
David: This idea that Take 5s can be used as a distributed hazard identification, if my site's got 100 cards being completed, then there are noes on some of the cards and there are hazards being identified. Those hazards can be collected, they can be shared amongst the workforce, and it's a really good process of collective hazard ID, hazard awareness, and hazard management. That would be an argument, so I don't think it works like that either.
Drew: We're not certain that anyone actually uses them like this. You get some people who complain about the loss of the data. They say, we fill out all these Take 5s, but we never properly analyze them. Very occasionally, you'll see some people claiming online or we send them back to head office and we do a careful analysis, but we haven't actually found strong evidence that anyone does use them like that.
We've got some reasons for thinking that they wouldn't work well for that because it would require people to fill them out very consistently. They'd need to use the same standardized language. They'd need to be very correct about what they recorded on the cards.
As far as we can see, people write things down at different levels of abstraction. It's not just that they'll write down controls that they didn't use, they'll sometimes use controls and just don't write those controls down. If we used them as a data source, they would be a highly, highly unreliable source of data. Yes, you could collect them and analyze them, but it would be misleading you about what was going on.
Jop: I think this one is also a case of, if you would apply it to a really specific task or specific hazard, you could potentially get that uniformness and do it, but I'm highly skeptical of whether organizations ever achieve that in practice.
David: The final opportunity for Take 5s to add some value and particularly if we think about the safety work model—you've said you can't find a link between this as a safety work activity and the safety of work—is this idea of demonstration of due diligence more of a demonstrated safety activity. The Take 5s themselves don't help improve the safety of work, but we want to keep doing lots of them across our organization to protect the company. How would you think about this as a benefit or a value-add of Take 5s?
Drew: Can I speak to this one, Jop, and you jump in and correct me if I get it wrong? There are a few weird things going on that have to be explained with Take 5s. One of them is that workers will sometimes fill out Take 5s in the crib room and do them in batches the week before, or they'll do it in the car on the way to work before they've actually seen the job site.
But workers will also sometimes fill Take 5s out after they've completed a job. That needs more explanation because they've got no expectation at that point that the cards will be checked. They'll use language like we're filling the card out just in case, but nothing's gone wrong with the job, so no one's going to do an investigation of the accident and find the lack of Take 5. Why do people feel the need to fill out this card anyway?
From the language people use around those cards filled out afterward, cards filled out even when we've banned the use of cards, or cards filled out when they're optional and the same workers have said that they hate the cards, it's obvious that it's a form of protection for the worker. They're often not entirely sure what it's protecting them from, but it's protection against something.
The best we can tell is it's a protection against criticism that you don't care about safety. You conversely filling out safety paperwork is a way of saying, oh, I'm a good employee. I'm someone who cares about safety. I'm someone who follows the rules. I'm someone who does the job properly. Even if it's filling up the card after the job's been done, at least I've still properly filled out all of the paperwork and it's there sitting done.
But it doesn't make sense that workers can be using the cards to protect themselves and the cards are a good defense for the company. Those two things are totally inconsistent. That says that the cards are being used to shift blame between people, not that cards protect everyone uniformly. If they were a good due diligence tool, then there would be a transitive property where protecting the organization was the same thing as protecting the worker, but in fact, they get used almost the opposite. The organization protects themselves by using the cards or the absence to blame the worker, and the worker protects themselves by proving to the company, oh, it's not my fault.
It doesn't work as overall due diligence, but it does have this weird protective function. It's just not protecting you from getting your thumb hit, catching fire, or falling down a hole. It's protecting you from criticism or blame.
Jop: I have nothing to add to this.
David: We've talked about this a bit. I think this methodology is a social defense and having something to demonstrate to someone at some time. You've got no idea who that will be, when that will be, and why that will be, but you've got an insurance policy of some shape or form. I guess that might be the use of these cards, which is a whole lot of resources for serving a purpose like that.
We've talked through the method and these five or six ways that these cards and processes could work, and this research that you've done doesn't work like that. Anything else you want to add from the study itself before we talk about some practical takeaways?
Drew: One other thing I just want to mention is I think that this social defense idea extends beyond the individuals up to the organization, which is why often, the people who are furthest removed from the work are most satisfied with Take 5s and most reluctant to get rid of them. But they do form a social role for the organization as well.
The best I can personally put it is in these moments just before work starts, the organization has lost control of using all of their other systems, but there are still things happening that they care about. They need something that tells them that they've got control, they can influence this, and they can make workers take a moment and pay attention. If the work doesn't match the plan, the organization can still reach into that moment and make sure that the work gets changed or the plan gets changed.
This tool doesn't do that, but it gives us an illusion that it does. It gives us something we can do to try to fill that gap. That's a really important need that organizations don't just go hands-off and say, okay, beyond a certain point, frontline work is out of our control. That's not even an acceptable thought, let alone an acceptable thing to say.
I don't expect that this research is going to suddenly get rid of Take 5s. I think it's going to cause a lot of angst, but a lot of those feelings are going to remain and a lot of those social needs are going to remain.
David: It's really, really interesting, Drew. I had never even thought about it like that, but there’s this idea that we've trained the person, prepared the person, planned the work, sourced the equipment, provided the equipment, and sent them to the worksite, and then there's this hand-off point, okay, from now, it's up to you. We've given them this last opportunity to get it right, have control over which buttons they end up pressing, and how they end up doing the task. I hadn't thought of it like that. Maybe that's it. Maybe it is the very last line of control that the organization exerts on an individual task.
Let's get into the practical takeaways now, Jop. We kind of always end the podcasts like this. What I want to say is, Jop, what would your first practical takeaway be? I've dropped a few down here now as well.
Jop: The biggest one that we really hope people do take away is to stop using Take 5s in accident investigations. They are not a reliable source of information about what a person was thinking about. As soon as they can, in some ways, they lead to blame. That is probably driving the misuse of it and the unreliableness of the data and creating this angst about why people fill them out in the first place or this anxiety that organizations need to deal with.
David: I like that. Not having a cause or contributing factor of an investigation that a person did a poor Take 5 or didn't do one, not checking these Take 5s in audits, and not looking for Take 5s as the first thing when a leader or a safety professional goes into the field is really important for giving some space around these processes. Not that they're probably going to help anyway but give some space around these processes to not be a political-type tool or a defensive tool.
Jop, I might just run through a bit of a process, and then I wouldn't mind getting your thoughts on it as well. My advice to listeners would, I guess, be very curious. If you've got Take 5 cards or similar in your organization, use this research to be very curious and wonder how they might be being used or the effectiveness or otherwise of these cards in your organization.
In terms of the research method, it's a great thing for everyone to actually talk openly with their workforce and leaders in their organization about how they're used, get to the point where you've got the psychological safety and the humble inquiry capability to get the stories like what was able to be obtained in this research, and how are these really used and thought about in your organization.
Jop: I would like to add to that because you'll fully agree with me. In that sense, also try to be explicit about your own idea of how they work and be critical along the way. Be curious and open-minded, but also be willing to reject your own ideas as you do it.
David: Absolutely. And for others in the organization to try to see if there's alignment in your organization between what you all in your organization are aiming to achieve with this process. If you're clear on what you're aiming to achieve, then design a process or check your process about can it actually deliver on what you want it to do?
The last thing, Jop, is in this research, two weeks of banning Take 5s and nothing much happened, maybe there is a case to think about substituting this entire process, stopping doing these cards, and really reconsidering what we do at the point of risk.
Jop: I very much agree. Drew might have said that he doesn't expect it. I do have a silent hope that at least we'll reduce the amount of Take 5s required around Australia.
David: The paper explicitly calls out and labels Take 5s as safety clutter in the context of this research. We know that safety clutter’s the accumulation and persistence of safety work that doesn't contribute to operational safety. There are a whole lot of negative consequences for safety from any form of safety clutter. I think taking it away could be a really good thing to do for safety.
That's it for this week. We hope you found this episode thought-provoking and ultimately useful in shaping the safety of work in your own organization. Send any comments, questions, or ideas for future episodes to feedback@safetyofwork.com.