On today’s episode of the Safety of Work podcast, we discuss the benefits of job safety analysis.
It’s difficult to give an introduction to this topic, given that a JSA is such an amorphous topic. Generally speaking, we’re talking about job or task-hazard analysis; the idea behind task-hazard analysis is that you break the task down into steps and figure out what controls are necessary to keep the task safe.
Tune in to hear us clarify the idea of and benefits from job safety analysis.
Topics:
Quotes:
“I think it would be fair to say that I’ve never yet met a method of risk assessment that I fell in love with.”
“The researchers are too optimistic about how much the documented JSA’s reflect what actually went on.”
“Ultimately, in high risk work, the immediate hazard awareness of people is important for safety.”
Resources:
The Application and Benefits of Job Safety Analysis
Feedback@safetyofwork.com
David: You're listening to the Safety of Work Podcast episode 62. Today we're asking the question what are the benefits of job safety analysis? Let’s get started.
Hi everybody, my name's David Provan and I'm here with Drew Rae. We’re from the Safety Science Innovation Lab at Griffith University. Welcome to the Safety of Work Podcast. In each episode, we ask an important question in relation to the safety of work or the work of safety, and we examine the evidence surrounding it. Drew, what's today’s question?
Drew: David, today we're going to talk about JSAs and whether or not they are effective in improving safety. I find it a little bit difficult to give an introduction to this topic mainly just because of the uncertainty about what a JSA actually is. We'll talk about that a little bit as we go through our discussion.
Generally speaking, we’re talking about something that is variously called job hazard analysis, task hazard analysis, task risk assessment, or possibly depending on where you’re working as safe work method statement. But the general idea is just any sort of risk assessment, which is performed fairly close to when the task itself is going to be performed. It usually works by breaking the task down into a series of steps, talking through the safety hazards, talking about the controls that are necessary for keeping the task safe, and then the team goes ahead and does the task.
David: I actually came to the paper and the idea of job safety analysis because we've now got it linked to a portal available where our listeners can drop ideas for episodes. We'll publish that when we publish this episode so the listeners that haven't seen that yet can do that. I was looking for a paper on golden rules, cardinal rules, or lifesaving rules because there's a huge amount of demand in the listener group for that. I thought JSA might be a close enough thing because there's not a lot of research on the former that I've been able to find yet.
Something that I find interesting in the paper that we'll talk about today is it appears that Heinrich might have been the first to coin this phrase—job safety analysis—in his 1931 text. We might need to check this out with Carsten Busch.
For our listeners who remember, I did an interview with Carsten on episode 17. Just as a shout out for him, he just published a book only weeks ago titled Preventing Industrial Accidents: Reappraising H. W. Heinrich – More than Triangles and Dominoes. If people got interested in that podcast, here’s another touch pointing to Heinrich’s work. It might be worth checking out a copy of that book.
Before we dive in, your general thoughts on JSA, however, you want to conceptualize what a JSA is.
Drew: Look, I have to be honest that I'm not a fan of—I was about to say any sort of risk assessment that just happens before a job, but I think it would be fair to say that I've never yet made a method of risk assessment that I fell in love with. One of the difficulties—and that particularly, I think, shows up in our paper today—is the lack of standard terminology, which may seem like a petty complaint. The paper lays out fairly clearly that as you start heading towards performing a task, there are lots of different opportunities to do risk assessments and make decisions.
Some of those are way in advance when we’re designing the layout when we’re selecting tools and equipment. Some of them are intermediate. Some are right before the task happens. As a community, we’re really inconsistent about talking about the different risk assessments at different stages. I think that inconsistency plays into the actual practice of risk assessment. In that we conduct it and we cover topics in the risk assessment at the wrong times. Very often, we document hazards and controls that are supposed to have already been dealt with or that we’re relying on someone in the future to deal with.
We're very seldom doing a risk assessment and it's informing the decisions that we have the power to make right in front of us right now. As a result, risk assessment and JSAs in particular end up being very much about documenting things that have already happened or that should happen in the future, and very little about making helpful decisions that improve the work that's about to happen.
David: I think my experience, I reflect on my own career experience around JSAs, and I was struggling and I still struggle to think of an experience I've had in my career where I've seen a JSA used well.
Even as this paper steps out how it should actually occur, who should be involved, how it should happen, how decisions should be made, how that should then go on and influence the safety of the work, and obviously how it should only be performing that role. Like what you alluded to, Drew, of managing that context-specific residual risk that hasn't been addressed by all of the other risk management practices that should go on in an organization.
I found some (let’s say) parallel process to JSAs a little bit different to this like where you might use a specific lift to study for a complex crane lift where it's very much about how we’re going to do this work and really open discussion and decision-making processes.
For listeners, I’ve experienced on both the contractor and the asset manager side of the industry and I don't see any real differences there. I could just go on for hours about all of the experiences I've seen in my career of the transactional- and compliance-based nature of JSAs not really having any contribution to the safety of work. We head into this podcast with an open mind because we do want to talk about the benefits of JSA and maybe provide some direction for our listeners of not removing JSAs from their business but might re-engineer the process.
Drew: Let's talk a little bit about the existing literature. This paper attempted to start to document how much has already been written about JSA and finds very few practical references. It seems that there are just a couple of researches who've published a couple of papers each. We know very little about how JSAs tend to happen, what the practical benefits of them, and what the limitations are. I think it would be fair to say that there's a lot more literature telling you what you're supposed to do than literature studying what actually happens.
David: I think within that literature, it was a very small literature review in this paper. This is a recent paper that we’ll get to in a moment. This is only 2019 now, so it's not like there's a decade gap between this paper and a whole lot of work that could have been done.
You’re right, much of the literature just makes claims about if JSAs are done as they’re designed then what could they actually deliver in terms of benefit. Statements like literature suggest that JSAs can make work both safer and more efficient. There was one study by Zheng in 2017 that found that a thorough JSA considerably reduces the number of recordable injuries. I haven't gone to that, Drew, but I imagine just from that claim there that you personally would not be a fan of the way that that research was conducted.
Drew: I haven't looked up the Zheng paper, but the claim is, at face value, implausible. That you could do a study that would show that at any practice—how on earth would you have done a controlled study that actually started the reduction from a particular practice like that?
David: Yeah, particularly what we know about the statistical and the validity of recordable injuries as well. We might pull that out and see if we can get a hold of it. See if we can make that available to our listeners as well to do their own critique of.
A participative JSA process where the workers are generally involved with the design of the task promotes greater ownership over the decisions and a better understanding of the work. Claims about improving communication between workers and managers, and claims about improving the individual knowledge and awareness of hazards. All these things that the literature says should happen if JSAs are done well.
In terms of empirical research, there are not a lot of people who have picked up JSAs. This is an interesting reflection about the safety literature, the things that are almost most central to safety practice almost taken for granted. Things like JSAs, things like life-saving rules, things like permits to work systems, things like induction training, and things like audits. All of these things, there's actually not a really big body of safety science literature about the effectiveness of these practices.
Drew: David, I think we'll see from the paper we're looking at today perhaps some hint of why that is. I think they've done an unusually thorough job of research. This paper, I think we were both impressed when we first picked it up with how they've gone about it. Even then, this is not a paper that can really speak to whether JSAs are effective or not, which just shows how much work is necessary to do a thorough study. How many people are willing to put their day-to-day safety practices to that research scrutiny? Who is going to pay for research about whether something is working or not, when no matter what the research says, you're not going to stop doing it because it is so central?
No one wants to be told that something that they’re going to do anyway is a waste of time. That's just wasting money on research.
David: For our listeners who keep asking me about the take five episodes that is in Drew’s hands for when he's ready to talk about it because I think that is something that falls very much into those gaps.
Drew: Watch this space because, yes, someone did in fact pay for that research. The results are quite interesting and no, I can’t share them until I’ve got them written up and peer-reviewed properly.
David: Very good, and that will come. There are some limitations with some of these claims that we’ve just gone through. The literature identifies some limitations for the effectiveness of JSAs. A lot of it—the practical application of JSAs, might not be hugely surprising to our listeners. There is research and David Borys is one who's published research in the Australian construction industry on the gap between work as imagined in JSAs and the way that works is actually done on construction projects.
There are claims that for larger complex and dynamic tasks pretty much like all of those common in the construction industry that the JSA process is just too simplistic to suit that dynamic context. That they consume significant resources in terms of time and effort to complete. The practice is often a transactional compliance activity and doesn't actually involve the workers performing the job or make any changes to the way that the work happens. Therefore, I suppose, many managers and workers believe that they know the job, they've done the task before—JSAs are unnecessary.
There's not a lot of evidence around these claims. It’s more anecdotal, but there are some studies similar to this paper—interview studies—which do actually explore this particularly with frontline workforce and supervisors.
Drew: Let's get into this study for today. The paper is called The application and benefits of job safety analysis. It was published in Safety Science, a reputable journal, in 2019. All of the offices are from the Norwegian Institute of Science and Technology in Trondheim, which is an institute with a strong history of publishing quality safety research. The Norwegian Institute also indicates that I'm going to pass off on to David to pronounce the names of the authors.
David: I'm terrible, but look, Eirik Albrechtsen, Ingvild Solberg, and Eva Svensli are the authors, and apologies to our Norwegian listeners or our broader Scandinavian listeners for making a mess of some of these European language names. NTNU publishes a lot in the resilient space, they've got a lot of quality researchers, and they do produce a lot of quality work.
The stated aim of the study was to investigate the practices and the benefits of job safety analysis in construction projects. The authors made some contextual comments about JSAs, it was sort of part of the introduction. You referenced this in some of your first comments that the authors did acknowledge that safety rules should be managed as early in the project life cycle as possible during design, planning, and procurement. Not deferring everything to the people just immediately prior to the task to try to figure it out.
Therefore, the JSA should only manage that residual risk of all the things that can't be done earlier on in terms of the design of planning activity. But they also make a comment that the JSA should only be required where safety cannot be managed by existing procedures, existing plans, existing barriers, or existing verified worker competence. I think, for our listeners, that could mean that they may only be an important few, unique complex tasks that should require a JSA. In those circumstances, the JSA process should be deliberate, participative, and comprehensive.
But they went on to show that when they did this study that we are going to talk about shortly, 56% of the JSA’s review did not meet that criteria. The organization was basically doing double the amount of JSAs that they probably should’ve been doing based on their other safety control framework.
Drew: David, I think it's worth pausing here and spending a little bit of time talking about this one. Activities like JSAs very seldom have any theoretical background. One thing that I think this paper has done really useful is they've put a plausible theory over the top. They’ve said that like most, safety is captured either in the design of the project or it's captured by routine procedures, rules, and plans. There's a type of risk assessment that feeds into those early decisions. It's typically not called JSA, it’s typically called something else. But that's a static thing.
Then you get the things that can't be predicted by a routine that are variable for any particular task that needs to be protected at the moment. That's why we do JSA, which is interesting because I don't think that actually matches a lot of what people think of. As we'll see when they come later through the benefits, some of the benefits they claim aren't really in the form of making good local decisions, they’re other effects.
That's what we need to be really careful of with things like risk assessment is that we’re clear about what is the claim to benefit. We don't get to say we're doing this because it will improve decision making and improve the way we go about the task. But the benefit of doing it always raises awareness of hazards, and it helps people think clearly and communicate better. That's not the benefit you claimed in the first place. If that's the benefit you claim, that's the benefit you test.
David: That's a good slant on that. I had a different slant on this. I thought when we're talking about when JSA might be a mechanism that might be helpful for changing the shape of work or change the way work is performed, I thought of three different scenarios a worker might face.
One would be just a routine job that they do every single day, and we know that that becomes very habitual. Based on experience, JSA is really unlikely to change the way that a person approaches a task that they do every night. Where they feel capable and competent and comfortable with the risks associated with it.
Then you've got these individuals who perform tasks that they perform every day but they’re high-pressure type tasks. I think in the psychology literature, we see that mental rehearsal is a way of improving performance. Golfers take practice golf swings. If you're performing a high-risk task, then you might talk yourself through how are you going to approach it just to make sure for yourself you get the steps right.
It's only when you come into this novel, unique environments, or areas that are at the boundary of your understanding, experience, or confidence that having these legitimate group discussion, bringing in this expertise, having an open conversation, actually stepping through a task from start to finish, and checking that everyone's onboard is probably actually going to change the way you do it and serve all these benefits.
When I reflect on this, I thought that might only be 5% of the times that we currently do a JSA in our organizations. I was slanted, I was reflecting on. Then later in the episode, in the practical conclusions, we might link that to some safety clutter ideas as well.
Drew: Thank you for that, David. I think that's a useful framework for thinking about this. Let's go through the method. The study is based on six different construction projects. It seems that these projects sort of stretch across a couple of companies. At various times, they referred to company A and company B. They've interviewed people on each of these projects. Unfortunately, they don't give a lot of direct quotes, and only on a couple of occasions do they tell us who is talking. They've interviewed workers, frontline supervisors, and safety personnel.
You would expect that those people might actually have different opinions. It's not always clear who's giving us what opinion. As well as those interviews, they did a couple of observations. From reading, I really think that possibly they only observed two JSAs occurring, which doesn't seem to be particularly comprehensive compared to the number of interviews they did. They also did a document study where they looked at 97 forms that had been produced through the JSAs. The implication is that in all of these companies, the JSAs tended to be recorded either before the meeting or after the meeting.
They could cross-check what people said in the interviews compared to what was actually written on the JSAs. They didn't have enough volume to compare what happened in the meetings compared to what was written on the JSAs because they only observed two of the meetings.
They claim that the analysis was grounded theory. It's fairly evident from the texts though that’s not in fact what they did. The researchers clearly had this preconceived view of how JSAs were meant to work. The analysis is more of a comparison between the ideal function of JSAs and what people told them, which isn't grounded theory, but it's still a perfectly legitimate way of doing a thematic analysis of the data.
People who read a lot of papers will find that people tend to over-claim that they’re doing grounded theory when in fact they’re doing some other perfectly legitimate thematic analysis that just doesn't have its name.
David: When I first went through this, I thought, great. Interviews, observations, document analysis—three different sources of data. You can triangulate the information. You can compare the information. I was a little bit disappointed. I think this could’ve been strengthened in a couple of ways. I think it could’ve been strengthened by doing a separate analysis for frontline workers and supervisors versus the comments made by project managers and safety managers. The paper even talks of the blunt end, sharp end, and it would be interesting to see the differences. That would’ve been useful.
The observations, I think knowing that there were two observations, we can really discount whatever happened in those observations because, like what we know when we put field researches, it can take a week, two, or three for work to return to normal, and for the workers just to accept that the field research is there and just to continue their work as possible.
I think in these two cases it would’ve been, hi everybody. Here are the two researchers, and they’re just going to listen to the JSA meeting today. I'm not sure that that meeting would run how it normally would the first time that two new researchers are there.
Drew: I also think that as a result of not seeing more of the actual JSAs and work occur, the researchers are too optimistic about how much the documented JSAs reflect what actually went on. There’s a lot of evidence that speaks to the fact that what gets recorded in documents like JSAs is very different, not only from how the work is performed but even from what is sometimes said in those meetings about the JSA.
I think a number of different studies, but one to point to is David Borys’s 2012 study that just shows a consistent gap between what's in the JSA and what actually happens. Just a tiny preview of something that we’re going to publish in the future. Weirdly, this gap works in both directions. People not only put controls into a JSA and then don’t implement those controls. People implement controls in real life that they don't put on the JSAs. It's not that the JSAs are the perfect picture and work happens badly. It's just that there's very little concern about making sure that the two are properly aligned.
David: This study is conducted in Norway, conducted in the construction industry, and conducted in two companies. The authors make strong claims about the results being generalizable to other industries and so on. But I would be hesitant to say that—this one study that we’ll talk about today—we’re going to provide a lot of generalizable ideas. But this is a big topic area and one that we might revisit a couple of times with a couple of more different nuanced questions about task risk assessment.
Drew, before we get into findings, you had a bit of a deeper look into the different data sets coming out of the document analysis and the interviews, and have some thoughts about how to frame the findings?
Drew: Yeah, since we've framed our episode as to what effect the JSAs have on safety, which is the way that the paper itself frames its findings. I think both of those are over-stated from what the data and the analysis can show. I think the paper does something which is important, which is it talks about how people think about JSAs and how people would like JSAs to work.
I would tend to reframe the question more to be if JSAs improve safety, how would they work? That then generates a whole heap of hypotheses that we definitely should go out and test, that we definitely can use that intended function to improve the way we do JSAs, but we shouldn't make claims that JSAs actually do. Just because someone thinks that a JSA does something, doesn't mean that it does do that. It does suggest that we can test whether it does do that in the future.
David: Drew, that's a good segue because the findings of the study identified—through their thematic analysis—six benefits. We'll talk about these as almost like conclusions from the study, but I think based on what you've said here is that each of these six benefits might actually need to be considered more of a hypothesis than a finding.
The benefits as stated are JSAs provide a mechanism for the formalization of work. They enable retrospective and prospective accountability, allow worker participation and the possibility to influence their artwork, create a space for organizational learning and communities of practice, increase situational awareness, and enables loss prevention in dynamic systems.
There are six benefits. What we’re going to do now is just talk briefly about each of those six and why the paper formed that conclusion and what it might mean. Maybe I might kick-off and you can just chime in as you feel like it. This idea about the formalization of work. JSAs provide instruction on how to perform the work so that everyone knows what to do.
We're assuming it's a workgroup, there's more than one person. We get together, we do a JSA. It's a task planning and clarification tool. It enables what we need to do as managers in our organizations, which have coordination mechanisms and divide up labor. One of the direct quotes of the interviews was after every JSA, every participant in the workgroup knows what to do.
Drew: David, a good example of where I think that claim is fairly strongly supported is a number of the JSAs in this project were to do with multiple crane operations. Particularly since one of these was in a number of cases a mobile crane. This is a good example of where the standard procedures aren’t sufficient to control the risk. There's always some residual risk that needs to be dealt with on the day through coordination and planning.
Having the group people who are about to conduct the work, know where the cranes are, know how the cranes are going to be protected from each other, know where the exclusion zones are going to be laid out on that particular day. I think it’s a really good example of how that is the appropriate time to be making and communicating those decisions—it’s just before the work starts.
Those are the types of examples where the participants were very positive that having these JSA meetings were genuinely getting them ready to perform the work in a coordinated way.
David: Yeah, I think that’s a great example, Drew. The second benefit is accountability. There was a bit of talk about the JSA being beneficial that if an accident happens, you can go back and look at the document to check what was planned to happen against what actually happened. This is this sort of retrospective accountability. This is where it started to get interesting in the findings of Drew because NTNU is very much a university steeped in resilience engineering, and they’ve published a lot of what we probably consider being a new view type of research.
They used this idea that it allows the removal of blame from processes because you can go back and provide some support that people were doing the right thing. But there was sort of a mismatch here between I think the author’s ideas and this claim that they were trying to make about going back and looking at what was meant to happen. But then they also made this claim about prospective accountability that some people were saying that if people put their signature on JSA, it's likely to make the workers feel more obliged to follow the rules.
There was this sort of mismatching this theme of accountability about just culture, but also about legal defenses, and also about driving force compliance through making people sign something. There was a whole mismatch of ideas in this finding.
Drew: I think that mismatch of ideas creates a couple of different hypotheses that we can't answer from this particular paper. One of them I think is a really interesting claim that is part of the reason why we get people to sign things at work. This is the belief that if someone signs something that they're going to think more carefully about it, they're going to feel more obliged to follow it.
There are some reasons to believe that might have a positive psychological effect, at least if you're aiming for compliance. But I really think that that's a hypothesis that needs to be tested with a specific experiment. It's a type of AB testing that any workplace could easily do. If you've got JSAs, then why not split them into two groups. One group has to sign the JSA, the other group doesn't, and see if that has any actual impact on how closely people read it, or how closely they follow it. Useful hypothesis, but until you've done the experiment doesn't mean much.
The other thing I think is very revealing but not in the way the authors think, which is that there is this benefit to having a signed document you can point back to. But it's not a safety benefit, it's personal protection in the event of something going wrong, which is almost an adaptive malfunction of JSAs. If people are using them to shield themselves from blame or to blame other people, that could be a workplace adaptation, but it's not an adaptation to protect against harm from an accident. It’s an adaption to protect against harm from the accident investigation process, which is something that I think is a fascinating topic to go into. This study doesn't do that.
David: I agree. The third benefit of JSA is worker participation. This idea that a JSA process done as intended, involves the work, is in the planning and decision making about work, and this is going to have a range of other benefits. This is going to improve the local workforce’s monitoring, reporting, and identification of hazards. It's going to improve the quality of the operational decisions by lending the group's experience and insights into the work planning process, and the JSAs are a form of direct participation by the workers.
They get to influence their work through their local knowledge of the challenges and the possibilities that exist within their work and their workplace, which should provide benefits for the workers, for the work, and for the organization.
Drew: David, I don’t know about you, but it was at this point in the paper that I began to worry that the researchers were finding what they wanted to find rather than what the data was telling them. Particularly given that half of these JSAs were prepared before the JSA meetings. Half of their dataset says that the workers cannot change what's in the JSA. That the workers are just signing on to things that are prepared beforehand.
Then their own analysis of the JSAs and the content of them said that almost all of the controls were meeting existing legal requirements for the work, or were recommendations that workers should be aware of. Their own data says that even though the workers have been involved, this is not in any way changing what's in the JSA. This is a hypothesis that they almost did successfully test and disproved from their own data.
There are two separate things here. There are workers participating in the form of communication and accountability, that's definitely happening. But for this particular point, they're talking about workers actually being involved in planning and decision-making. The evidence says the opposite. That JSAs are not helping workers be involved in planning and decision making.
David: This is where we don’t know—in terms of the nature of the conversation—whether they were talking about what was actually happening, whether the participants were talking about what they desired to occur. We've done some work recently with some organizations who operate like that where the JSA is very much pre-prepared with permits perhaps and things like that, and it's just communicated to the workforce. It doesn't serve any of this purpose around local ownership enablement, decision-making, matching work as designed to local conditions, capabilities of workgroups, and things like that.
Drew: I can imagine though that when you interview the safety managers, they think that they are providing worker participation.
David: Absolutely. I suppose more hypotheses to be able to test if you split out your participants by their role in the organization. And also, if you check some of the things, it's not hard to check wherein your business do you start with a blank page, where do you pre-prepare advantages and disadvantages for both of course where you pre-prepare something. There's a reason to do that because you can bring institutional knowledge from prior completion of that type of task, and facing those types of risks. Prompting, guiding, and pre-populating isn't necessarily a bad thing, but you really want to study how many changes happen on the day.
Drew: That would be another interesting study that you could do in one workplace is the comparison of the starting with a blank page versus starting with pre-prepared, and testing things like thoroughness and ownership. You might predict that there’d be a bit of a trade-off. That pre-preparing them makes them more thorough, and that starting with a blank page increases ownership. I'm just speculating there. That’s a prediction, not a finding.
David: That would be the prediction that I’d make, Drew, and then you'd actually have to go back to what's the purpose. Is the ownership of a few key controls that are readily identified by the workgroup, trade it off against lesser ownership may be in compliance with a more comprehensive suite of all these ideas for the work.
The fourth was organizational learning, and this is where the JSA provides a mechanism or medium for the transfer of tacit and explicit knowledge from the individual basically to the institution and to other individuals. If done well, this JSA process should be able to discover the way that people perform their work, make this information available to others in the workgroup. The authors suggest that the JSA process actually creates the factors that are known to generate learning within communities of practice.
JSA is an event or activity that brings people together. It's got internal leadership, whether that's a JSA leader or whether it's a supervisor. There are mutual engagement and interaction between peers, and that the group produces artifacts. The community in practice produces an artifact, which is the JSA form.
I think Drew I went back because I remember in episode 48, which we titled one of the missing links between investigating incidents and learning from incidents. In that episode 48, we talked a lot about learning being a social process, particularly post-incident learning, and the importance of actually engaging in dialogue that actually shared experience and matched the local context of the work that people faced.
Drew: I'm unsure what to think about this one David. Notice that all of those things that you talked through there are really coming from the researcher’s ideal model about how this is meant to happen. They're not coming from the participants. I really had a sense—when I was reading through this part of the paper—that the researchers are speculating rather than having their participants clearly telling them, when we do JSAs, we feel that we created organizational learning.
I can't imagine someone on a construction site even saying that. What I do think is really interesting is that I've never met someone who works in construction-like environments who doesn't believe that having some sort of pre-start meeting or whatever it is—whether it's a risk assessment, a talk, or something—is a good idea. Everyone thinks it's a good idea. I've never met anyone who thinks, we do this super well we can't possibly improve this process. I think that is a really interesting open question.
We want to bring people together with internal leadership to have mutual engagement before we start the work. What is the best task to have them doing? Is it a briefing? Is it a risk assessment? Is it a toolbox talk? Is it having coffee? I think that's an open question.
David: Yeah, and I think we've got a lot of practices and language around it like you mentioned a couple there—toolbox talks, pre-starts, even through the JSAs, risk assessments, take fives. There's a lot of stuff that happens before the work actually happens, which is designed to set the work out to be successful. I think there are lots and lots and lots of really interesting research questions in there, but also practical considerations for organizations as well. That's something that the academic community and the professional community should have very strong aligned interests around trying to get answers for.
Interestingly, Drew, you mentioned the authors’ ideas. I suppose in terms of qualitative research, there are six thematic benefits (I suppose) that we’re talking through here. In this one, the organizational learning was one where there's half a page of text and no quotes.
Anyone reporting back on qualitative studies, I think that's also immediately a bit of a red flag for me. If a conclusion isn't substantiated by a quote, that allows you to do sort of a finding data type of match because we don't see all of the data in a qualitative study. That's something that maybe the researchers are starting to communicate their ideas.
Drew: A really good contrast is this next one, which is hazard and situation awareness. Remember the authors when they're explaining why we did JSAs, they think it's about decision-making and putting controls in place. That's not the same thing as hazard awareness. This one is clearly not coming from the author’s model, it's coming from the data. They’ve got a number of quotes to back it up that clearly, this is what their participants think is actually going on. Their participants don’t think it's about controlled organizational learning.
The participants think we do these because doing them makes us more aware of the hazards. The JSAs back that up with a lot of these supposed controls in the JSA are actually people doing the task should be aware of these. This is not as a risk assessment tool per se, but say more of a just doing this makes us more aware of what's going on because we've talked about it, we've thought about what can go wrong.
David: Absolutely, Drew. I think in terms of thinking about what can go wrong being really important because, in this finding, the authors refer to a couple of other studies. In one previous study, they found that individual construction worker has only identified about half of the hazards that were significant to their work task. I hear this a lot. I need to make a lot of people have to know the implication of hazard awareness. Studies would show maybe an individual might find 50% of the problems with their work or their work environment.
Another study, which surprised me and I didn’t have the time to go and check but found that only about 7% of pre-prepared method statements actually identified all the hazards of a situation. Now when you think about it, it makes sense. If you pre-prepared a procedure for something and it enters into a dynamic workplace, there's always going to be things in that workplace that are thought of in the method statement.
JSA has become this filler for the individual limitations and the procedural limitations right on the spot to make people hopefully talk about the hazards. Whether identifying the individual hazards is helpful or like you're saying, Drew, whether it's just everyone getting okay, now I'm at work. Now I'm talking about safety. Now I'm back to do my work. I better get focused on what I'm doing.
Drew: I don't want to sound like a broken record, but this is still just a hypothesis. This is the reason why we might be doing JSAs. There's no actual evidence in this paper that they do increase hazard awareness. But I think it's one of the more important hypotheses to take seriously. This is definitely something that we can't control by procedures. You can do all the planning you like, you can do all the site preparation you like, you can do all the procedures you like. Ultimately, in high-risk work, the immediate hazard awareness of people is important for safety.
As people doing safety or running an organization, we definitely want to support people in that awareness. It is really important that we think about whether the things that we're asking them to do before work are in fact working or not working to help with that awareness.
David: The sixth benefit is just titled loss prevention. It is almost like a throwaway last thing where the authors say, JSAs create an opportunity to basically enable the safety of work. We get to establish risk controls. We get to put them in place for the task that we’re just about to perform. This has benefits for reducing safety incidents, inefficiency in the operations, and work problems. They tie this big link, then they just tie this big umbrella a couple of times in the paper—this big link between safety quality and work effectiveness.
Drew: David, before we move on to practical takeaways, I realized that there's a couple of topics in the paper we haven't talked about yet. The first one I wanted to mention is they layout—similar to how you did—the different types of times we might want to do a JSA and what it might support. I think their version of it is that we should do JSAs when the work is novel, we should do JSAs when there are changes to the normal procedures, or we should do JSAs when it's particularly high-risk work.
What they sort of categorize the JSAs they saw based on where they met those categories. They concluded that a lot of JSAs seems to be down for things that didn't tick any of those boxes that were routine, not necessarily high-risk, and no change to the normal procedure.
David: You’re right, Drew. I think that’s a great take. When I read that thinking like gee, it’s another one of these five- or six-point which you could almost massage most jobs into. You're right, the authors did find that half of the JSAs didn't meet any of those. That's not even a practical takeaway, but it’s a good time to mention it now, is to ask yourself the question as a listener, what is the criteria by which you require people in your organization to prepare and complete a JSA, and have a critical reflection on what the criteria are that you’ve set.
Drew: That's it. I think if you believe in doing JSAs, there is an argument for making them compulsory all the time. The argument would go something like until we've done the JSA, how do we know whether this task is high-risk, change conditions, or different from the routine planning so that we need to do controls. I think it is a plausible argument to say that's why we do JSAs always. We do them because our first step is, has something changed, is there a particular high risk here, or is there a control that doesn't match the normal procedures?
David: I think if that's the assumption you're going in with, then I would be making sure that my organization saw that as a core part of the operation. That every task had an extra half an hour assigned into it, and that all of the training support put around it. It didn't become what it's very likely to become in that situation where you get someone who's doing the same task every day—needing to do a JSA before they empty the kitchen bin or something like that. Which is what you see when organizations go over the top with practices applied to every task.
Drew: Yes, and the other thing is that if you're doing them like that, then it makes no sense at all for the supervisor to have prepared it the day before because if the whole point is check if anything has changed locally, then having a pre-prepared JSA is not going to make much sense.
David: I think in this little brief interlude we’re having here, Drew, this is just the nature of the complexity of something that happens every day in the organization, and the potential gap between how something could work and probably how it’s being done in the organization. Because you could do anything with JSAs. Like you said, you could go from making it compulsory for everything to almost nothing, changing the purpose. Lots of different hypotheses we’ve mentioned in this episode about different ways you could try to understand their effectiveness.
But I think it is a really big question because there is an opportunity before you start a job to think about how the work is going to happen, and put in place controls that directly improve the safety of work that’s just about to occur. There's a huge price for organizations I think for safety in investing a lot of time to try to figure out the JSA process. Drew, is that something you’d agree with?
Drew: Yes, and I'm not sure whether we're going to cover this in the takeaways, but I think that this is the big paradox we face. It is totally legitimate and makes total sense that organizations are worried about that residual risk between what's in their standard procedures and what's actually about to happen. They're worried about that and they have an opportunity to influence it because that's a time of day when they have a supervisor there, they can get a safety person there, they can get everyone together. There's a chance to do something. You’ve got a need, you've got an opportunity. What we don't have is a proven useful tool to drop into that need and opportunity.
I think we can say that there is something that fits into that space is almost inevitable that organizations are going to be trying things there. But I think we can also say that we've got a good reason to be skeptical about any of these specific tools. Whether they're working as well as we think we do when we drop them into that space. Just because you have a need and opportunity, doesn't mean that whatever you're doing is working well.
David: Yeah, absolutely. Let's talk about some practical takeaways. I have a script here because the paper didn't spend any time on practical takeaways. But I think there are just a few things that I reflect on and I thought of. Drew, I'm really interested in your thoughts about these.
We can probably say, as we've said a few times, that JSAs—as potentially designed and intended in the literature or within your organization—can have positive benefits to risk reduction of work performance. I think can is the ultimate word there, but there's more to be done before we just say that JSAs reduce risk.
Drew: I was going to say that my interpretation of that can would be that I'd say that there's a couple of very plausible mechanisms. There is a potential for improving safety by increasing hazard awareness. There's a potential for improving safety by improving communication and coordination. And there's a potential for improving safety by last-minute decisions that influence the controls that we’re putting in place. We have all of those potential there. I'm not certain that JSAs necessarily can achieve those potentials or not. I think that's what the can means. We need to actually test whether they do or not.
David: Yeah. Great extra learning opportunity with those three things, Drew. I think that's really helpful to think about what's coming out of that JSA process. The second one is that I suppose it's a little bit similar. Maybe I won’t go there. I’ll just go straight—
Drew: Please do because I like this one. What David has written is he said that because the process is performed at the point of risk, it has the potential to enhance the safety of work, which I don't know if David's going to claim this, but I'm going to say that this is a really insightful thing.
A risk assessment that was performed three weeks earlier is guaranteed to have a long separation and a long causal chain between the work. Something that we're doing onsite just before we do the work has got to have more potential than something that’s more removed to enhance the direct safety of work. Any link is going to be a one-step link between doing this activity, something happens, the work changes. As opposed to multiple steps in the causal chain.
David: That was my reflection. For people who want to go, I think it's episode 50 when we discussed our safety of work model. Even though JSA is sort of an administrative process, it’s got a real opportunity to deliver physical safety outcomes like we said. Actual controls in the work change the equipment, change the process, and change other resources and the execution of work itself. It provides us with an opportunity that I think safety people should be very interested in.
The third thing here is you're likely to be doing—maybe Drew, you’ll disagree with this now based on our last conversation—too many JSAs for too many tasks. I know organizations who every time you need a permit, you need a JSA, or you need a JSA for every job. Even the study finds, when people don't see the need for JSA because it's a job they do every day, then it becomes a compliance activity.
When something becomes a compliance activity, then it's got the potential to always be a compliance activity even when it's probably in those situations where you would want it to be anything but a compliance activity. I think that's a tradeoff there if you do it a lot that the times that you really want it done well, it's done as a transactional activity.
Drew: David, this is not remotely supported by the current paper that we’re reading or evidence that's out in the public. This is a forward reference to a future time. But I am at some point, in the near future, going to claim that pre-task risk assessments like JSAs, you should stop doing them altogether. The time I actually make that claim, I'm going to provide the evidence for that claim. For now, I cannot publicly say that it’s a current recommendation, but I'll go along with this weaker one that probably you're doing too many.
David: Yeah, okay. Very good Drew. For someone who hasn't said something, you just said a lot. Maybe that's just a David Provan practical takeaway, they're not from the paper. Please think about the criteria that you're applying to organizations. Maybe that goes into the next couple of practical takeaways is that JSAs are likely a candidate for safety clutter in your organization. Again, I was saying here maybe not necessarily to remove even though Drew may at one point in the future say remove them, but maybe to re-engineer how and when you do those activities.
There's a real opportunity here. We’re in episode 62, and we've got some people who have been listening along for over a year now. I'm sure they're just chomping at the bit to do some own company research as a safety professional. I thought you could actually have a go at replicating this exact study for yourself in your organization.
What you do is talk to a few people in different roles about the JSA process. Talk to some workers, talk to some supervisors, some safety people, and some project managers. We've got episodes now on ethnographic interviewing techniques and a whole range of ideas for people. Go and observe if you pre-start or JSA preparation. Go and see how they work.
Go and get copies of 20 JSAs that have been done in your organization and just work through those 3 data sets. What do people tell you about the JSA process? What do you observe when you're watching them get prepared and get done, and what do you read and see when you look at JSAs?
As a safety professional, that would be an amazingly useful task for you to do. This paper provides an opportunity for you to see how you might just go and do that. I don’t think there's anything stopping any safety professional from replicating this study.
Drew: I'll go a little bit in terms of tips that I don't think they implemented in this paper that you could do to make that easier for yourself.
One of them is I wouldn’t ask people the straight-up question about whether they like JSAs or not. I don't think you'll get a straight answer. I don't think you'll get a straight answer either about what do JSAs do well. Most people would have experienced JSAs delivered by different people. Ask them, who do you reckon does the best JSAs, and what is it about the way they do JSAs that you reckon makes them better than the way other people or other companies do them?
What sort of task do you reckon they work better for than not for? Why is that? Think of the five you did this week, which one do you reckon was actually most helpful? What was it about that particular day that the JSA was more helpful than the other days? Looking for those differences, that helps people because they don't need to say, I hate them. These are always a waste of time. It lets them make comparisons about what does and doesn't work well. I think you'll get clearer answers to those sorts of questions.
In terms of getting copies of them, the important thing there is don’t assume that what's actually on the JSAs is all that happened, but it is useful information. It's useful for comparing.
David: You know separately I'm from this side project on the safety futures programming. When they get to the mission prescribing work, they actually have to look at a JSA, then go and observe the task, and try to understand their difference. Maybe that's the fourth one here. Do the interviews, observe some JSA preparation, get some copies of those JSAs. The ones you get copies of, put it in your pocket and go and actually observe the task that is actually reflected in the JSA, and add a fourth dataset into the mix.
Drew: What I think would be a really interesting challenge is don't just look for things that are on the JSA that people aren’t doing. Look for things that people are doing to protect themselves and others that are not recorded on the JSA. Look for what sort of things are built into their task or that they're improvising on the fly that helps them be safe, and look for differences in that direction as well.
I think you'll find that there are things that people do that they don't bother to record that are good for safety, as well as things that are on the JSA say that isn’t done in practice.
David: Great, Drew. The question that we asked this week is, what are the benefits of job safety analysis? Do you want to have a go at the answer?
Drew: I would say we have three big candidates. We have a candidate that they benefit by raising hazard awareness. We have a candidate that they benefit from improved coordination of tasks or formalization of work or communication. We have a candidate that they sometimes actually directly lead to changes in the work, either by participation or by management decree that results in improved controls being in place physically on the day. But we don't have—at least from this study—evidence that those three things are achieved, how often they're achieved, or when they're achieved.
David: That's it for this week. If you have a more nuanced question about JSAs, drop it in the comments on LinkedIn or put it in our episode ideas portal. Hopefully, there might be more paper floating around of the select few that might answer that question more specifically. But we hope you found this episode thought-provoking and ultimately useful in shaping the safety of work in your own organization. Send any comments, questions, or ideas for future episodes to us at feedback@safetyofwork.com.