On today’s episode, we dig into why we need complex models to explain simple work.
We use the paper, Analysing Human Factors and Non-Technical Skills in Offshore Drilling Operations Using FRAM, in order to frame our discussion of this topic.
Please let us know if you have any experience with FRAM or similar models. We’d love to hear your feedback.
Topics:
Quotes:
“Every function of a system is a hexagon and every vertex of that hexagon stands for a different way that, you know, this function can be connected with the next function.”
“The authors say that the interviews had just one question, which was ‘how do you perform your job?”
“What I like about the use of a FRAM model would be, I think it will allow organizations to narrow that gap between work as imagined and work as done.”
Resources:
Analysing Human Factors and Non-Technical Skills in Offshore Drilling Operations Using FRAM
Drew: You are listening to the Safety of Work podcast episode 45. Today, we are asking the question, why do we need complex models to explain simple work? Let's get started.
Hey, everybody. I'm Drew Rae. That guy over there is Dave Provan. We're from the Safety Science Innovation Lab at Griffith University. As we record this week, the lights are back on at the lab, and I taught my first on-campus class yesterday. How are things with you, David?
David: Drew, I'm presently in Melbourne in stage four COVID lockdown. I'm not allowed out of my house. I really couldn't tell you what time it is, let alone what day it is. Although the lights are on so it must be in the evening.
We’re a few weeks in front of our recording, I hope things are better by the time our listeners are hearing this. Drew, we now got about 1000 regular listeners from more than 80 countries. Many of our listeners are likely to be doing much tougher than both of us. From us, we hope that your families are safe and well. You're able to take care of yourself in this really challenging time.
Drew: Absolutely, David. I second that. The paper we're going to look at this week is representative of the subfield of safety research that uses modeling techniques to describe work situations. The paper itself is pretty interesting. But David, I thought we might take a little bit in advance to talk a bit about modeling techniques in safety. Particularly the way we use them in education, research, and practice.
A couple of big techniques people may have heard of FRAM and STAMP. David, have you used any of those techniques in your work either as a researcher or as a practitioner?
David: Yeah, Drew. I have actually. I used FRAM when—some of our listeners might be familiar with the paper that we wrote titled Safety II professionals: How resilience engineering can transform safety practice. It's available in Open Access. We'll get around to talk about it on the podcast at some point.
I've got a working FRAM file for that paper, Drew. It was based on the idea that safety professionals perform a whole range of functions or activities in the organization to create certain states or certain outputs in the organization, and also to respond to certain situations or inputs. These require resources, certain pre-conditions to be in place for them to be effective in certain feedback loops, and so on.
It worked okay. It probably wasn't right, but it forced me when I was thinking about the role of safety professionals within this organizational system and the functions they perform. It made me think about it in a more integrated way than I might otherwise without actually trying to put it into a model.
Drew: I haven't used FRAM in [...] at all. I've done a fair bit of teaching. Particularly, when we're teaching people about accident analysis and comparing different methods often get students to apply STAMP. It's a fairly useful way of encouraging people to think about accidents in a slightly more sophisticated way. I find it useful sometimes when I'm going through an accident myself to draw a bit of a STAMP diagram to help me understand the different parts and how the parts fit together.
The history of FRAM is interesting. Most big-name safety authors like to—at least in some stage of their career—invent some sort of big metaphor for modeling accidents. Listeners might not know that James Reason had a couple of bites of the cherry. You might have heard of his Swiss cheese model, but he also invented this other thing called Vulnerable System Syndrome, which never really caught on.
Nancy Leveson has the Systems-Theoretic Accident Model and Processes (STAMP), Charles Perrow had Normal Accidents, Snook had Practical Drift, and FRAM is Erik Hollnagel's version of that.
David: Drew, I might just say that when we're talking about models here and modeling work, people might—even in their organization—be familiar with using things like flow charts or diagrams. A flowchart will probably be a more linear model. This step happens, then this step happens, then this step happens when a job gets done.
Most of our work method statements and job safety analysis or our standard operating procedures operate like that flowchart type of model. When we're talking about FRAM, today, FRAM's just a more dynamic and more complicated diagram of the relationships between certain activities in a piece of work.
Drew: Yes. All of these techniques have some sort of diagram. Then they have some underlying metaphor or theory about how things link together that explains what the boxes and arrows mean in the organization.
The usual path these things follow is they start as a new way of explaining accidents, and the author will present an accident and share a couple of diagrams using their new technique. Then that will evolve fairly naturally into an investigation technique, and people will use the technique to investigate accidents. And then they'll evolve into techniques that are supposed to help you understand and improve a system before an accident happens. That's the same path that FRAM followed.
If you're wondering what FRAM stands for, it stands for multiple things. It was originally called the Functional Resonance Accident Model because Erik comes out with this idea of explaining accidents. Hollnagel's idea was that systems are made up of functions, and functions are all interconnected in various ways. Those interconnections between the functions vibrate in response to internal or external disruptions.
The idea was if the system was well-designed, the vibrations get absorbed and gradually fade away in the systems back into the normal state. If the systems designed with the feedback incorrectly, then the vibrations reinforce each other—you have resonance, and the disruption— rather than fading away—swells until the system breaks.
If you think of that just as a loose metaphor, it makes perfect sense. That's what resilience engineering is all about. A resilient system can handle the disruption and get back to normal. Whereas a brittle system, when it gets disrupted, it breaks.
Don't think about the idea of resonance too hard. If you take it literally and look up what resonance is and how it works, the idea explained in FRAM is really kind of bonkers. There's something about the idea of vibrations and resonance that turns people—who in their field are genius—into weird pseudoscience. Nikola Tesla did this too. Fantastic electronic inventions. Once he started talking about vibrations and frequencies, his stuff just got totally bonkers.
David: Drew, FRAM is now commonly called by Erik and others as the Functional Resonance Analysis Method to be used to describe any system or activity. It gets used for modeling systems in a normal state, like I said, instead of explaining accidents.
In a FRAM diagram—just so our listeners can have a picture of what it looks like—every function of the system is a hexagon. Every vertex of that hexagon stands for a different way that this function can be connected with the next function. The six vertices are labeled: time, control, pre-conditions, input, output, and resources.
You might have one function and that becomes an input to the next function. Then, the output of that function becomes a resource for the next function, and so on. You get all these core functions in a system. You design the way they relate and are interconnected with each other. There are certain system states that get labeled on the arrows between the functions (if you like).
Drew: Let's talk about interrelationship. You can say if this function’s time is delayed, so it's running late. It's then going to feed into the next task, and it's going to make it not function as well. This function can't start until this other function has finished. Or this function, if it's late, is going to take up resources from this other function.
If you imagine that any vertex in any function can be connected to any other vertex or any other function, these are not going to be neat diagrams. Real practical FRAM diagrams look like these weird piles of spaghetti. If you're in the engineering space, if you’ve ever seen a circuit board that's wired up—done properly by real engineers who don't use right angles—they just connect the wires in curvy ways, which is genuinely better for signal propagation but looks like an absolute rat’s nest.
David: Drew, what I might do is when we publish this episode on LinkedIn, I might paste in the first couple of comments, some example FRAM diagrams. I'll paste the one out of this article. There's also another good article I know using FRAM to analyze the operations of a tugboat in a busy Scandinavian harbor. That is something that even on an A1 piece of paper is something you just could not make sense of.
Drew: The end result of FRAM is not intended to be a nice, neat, useful diagram, which puts paid to this original theory of resonance. You can’t use the final diagram for anything. That's not the point. You can compare this to a technique like STAMP. STAMP, instead of having six vertices, really only has two things. Functions provide control to other functions, or they receive feedback from other functions.
STAMP diagrams look like nice, neat block diagrams with an arrow out and an arrow into each block. Maybe connecting up a couple of layers. You've got at most four things connecting to a block. There, the diagram itself is supposed to be useful. Whereas FRAM, the whole point is it's this process of producing the diagram is supposed to be constructive and useful.
David: That's been a good introduction to talk about modeling work, modeling accidents, some of the background, and other models alongside FRAM. The paper we're going to talk about today is called Analysing human factors and non-technical skills in offshore drilling operations using FRAM. It was published in the journal of Cognition, Technology & Work in 2020.
CTW is a smaller journal than some of the big safety journals that we pull papers from regularly on the podcast. This might even be the first paper that we've got out of Cognition, Technology & Work. It focuses more on the human factor side of things. It's not always about safety, but a lot of safety stuff has gone there in the past and continues to go there.
The paper's got four authors. The lead author is Josué França. The second author is Erik Hollnagel, as we've mentioned. The third author is Isaac dos Santos. The final author is Assed Haddad.
Drew, most of our listeners would have heard of Hollnagel who invented FRAM. We can assume that his involvement like you said, means that he agrees with the way that the model's been applied to this particular research. The rest of the authors are from various places around Rio de Janeiro in Brazil. Drew, how common is that for a group of researchers to go to someone who is (I suppose) responsible for the theory or the model they're trying to test and get them as a co-author? How does that process happen?
Drew: It's pretty common for authors to try. It's less common for the authors to be willing to help out. If you imagine, you invent this technique, it becomes popular. Every person and their dog who wants to apply the technique wants your approval and the pat on the head that you've done it properly.
It's interesting in this particular case. It seems that the way they used Erik and one of the other authors—I'm not sure which other author—used them as part of the research method that I'll talk about when we get into the methods. Their role was really to verify and validate the application of the technique. Shall I go through the method of the paper?
David: Yeah, how about you tell us what the researchers did?
Drew: The paper, overall, is an ethnographic study of drilling unit operations called doghouse operations on three oil rigs off the coast of Brazil. Very much ethnographic. They collected the data over a period of 6-9 months. It was collected through onboard observations and with interviews. Some of the interviews they conducted on the oil rigs. Others, they grabbed the drillers when they came ashore.
The authors say the interviewers had just one question which was, how do you perform your job? David, I don't know about you, I run a class on interviewing where the trick is to keep someone talking for five minutes after asking one question. Most people can't do it. I refuse to believe that the interviewer did just ask that one question.
David: If you are in the control room on a rig—and we'll talk about what that looks like and means in a minute—if you asked, how do you do your job? And you were right there as it was being done, you could just say, what are you doing now? What are you doing now? Why are you doing that? Why are you looking at that? You could definitely keep a conversation going for a long time.
No offense to my driller colleagues, but if you had a driller in an interview room onshore just after they came off hitch or something and asked them that one question, the conversation would be done in about 30 seconds.
Drew: Yeah, possibly it was that one question asked many, many times over a couple of pints.
David: That might have helped.
Drew: The way FRAM comes in is not in the collecting of the data. It comes in the analysis. We have mentioned in a few other episodes doing thematic analysis of data, or phenomenological analysis of data. Now, what we're doing is FRAM analysis of the data.
Rather than trying to sort the data into themes or into phenomena, we're trying to take the things that are said and turn it into this map of the functions and how the functions link together. The authors take the data. They produce an initial FRAM model. Then they showed it to a local and overseas FRAM expert. I presume the overseas FRAM expert was Erik Hollnagel. And they showed it back to the drillers.
If any of those people suggest any corrections, they made the corrections then showed it back to everyone. They didn't actually say in the paper how many times they repeated that process. They just said the process was repeated until everyone agreed that the model was a valid representation of the work.
David: Drew, that's interesting [...] valid representation. The only opinion that probably matters is the driller. I suppose the FRAM experts want to classify things correctly. I don’t think it's up to the FRAM experts to decide if it's a valid representation of the work. I think their job is to say, has the model been used in the way that it’s meant to be used.
Drew: Yes. Seriously, looking at the diagram that's in the paper. I imagine you show that to a driller, their immediate response is, yeah. It's fine.
What's the point of doing all of these? Why are we creating these big complex messy diagrams? If you think about it, we create diagrams and representations of work all the time for all sorts of different reasons. A procedure is a representation of work. A manual, a risk assessment, a plan, a training document, and a position description—all those things that we sometimes call work as documented or work as imagined.
They're all trying to create artificial representations of the way work is done. The purpose of doing this more complex FRAM analysis is to show that there's a lot of stuff that's in the work as it's done that isn't captured by any of the documents. It's not irrelevant stuff. The stuff that's being missed out is vital for what creates the safety and productivity of the drilling operations.
David: Drew, just before we dive into what the description of the activity is because I think that's interesting, just to give our listeners a bit of context. For those who don't spend much time on the oil and gas industry, what are the researchers looking at here when we're talking about an offshore rig and the driller in the control room of the drilling rig?
I've been fortunate to spend a bit of time on these rigs in this environment. You're talking about a guy sitting in a room. He or she has got a whole bunch of screens that have got a whole bunch of data feeds and then a whole lot of controls.
For those listeners in the construction industry, you could think about this like a cab of a crane, or for those in aviation, it might not be too dissimilar for a cockpit. You've got a bunch of AV-type interfaces or video display units. Then, you've got a clear perspex screen of what you need to look at whether it's your load on a crane, or whether it's the sky in front of you in the plane. In this case, you're looking at the rig floor and the drill string. The rig floor is the base of the rig. You've got the drill string that goes straight down through the middle of it. When we're going to talk about the activities the drillers are doing, I just wanted to help our listeners try and have a model in their heads of what we're talking about.
Drew: David, do I understand correctly? These drillers that we're talking about, this person is not doing things like manually adding new segments to the drill string. That's happening out on the drill floor. But they're monitoring it, and they're controlling the function of the drill while other people are doing the assembly, maintenance, and adding in new sections.
David: Yeah, very much. You've got a set of conveyors running which are bringing the drill string bits in. Depending on the design of the drill rig, the driller is then maintaining the mud or the well fluids, the pressures, and is also maintaining the torque in the top drive, which is driving the drill string and then controlling when people on the rig floor (if you like) or in the red zone—maintaining the safe state of the drill string. Then making sure things are clear before drilling operations commence.
It really is the conductor of the whole drilling orchestra onboard. Been there long enough to be sitting in an airconditioned cabin and not getting dirty.
Drew: This diagram that they drew, had 22 different functions represented on it. We won't go through all 22. Some of them are things that are really obvious that you’d know are core in the drilling operations—control the drilling depth, control the drilling speed, and control the mud pressure in the drilling fluid. There's a bunch of other functions that are monitoring functions that come along with that. Monitor the pressure instruments, monitor the screens, monitor the column weight, monitor the level of the drip tank, and monitor the torque of the drill.
Then there are functions that are exceptions or disruptions. Have a new shift of drilling operators, manage malfunctions due to wear and tear, and execute an emergency stop. And then some that are a little bit surprising because they're not really to do on the technical side of things like keep awareness of the drilling floor activities. That's looking up a window at the drill floor. Experience pressure from supervision. Have trained and certified drillers. That’s why they like to upkeep their training while they're doing everything else. Recognize and manage relevant external noises, recognize and manage the relevant smell of hydrocarbons.
One of the things the paper highlights is the ability to recognize noises, smells, and vibrations. Do you know what those things mean? It's a key part of the job, but it isn't mentioned in any of the documented procedures or standards for drilling.
David: I remember, Drew, quite a while ago, I was at a gas plant. It was a very mature field, and it was sort of the end of the life of the plant. There were operators on that site that had been there for 30 years. It was my first visit to that site. I was doing a site walk with one of the supervisors. At that time he just said, "Hang on a minute." And just left me standing in the middle of a gas plant and walked quickly into the control room. When he came back I was like, "What was going on?" All he'd done was heard a noise, he knew exactly what it meant, and he was racing in to let the control room know.
Over 30 years, they'd amassed such knowledge of the way the plant operated and some of the technical challenges they had faced. Depending on things like just the ambient air temperature, it would upset gas detection systems, it would disrupt the process, and they would respond to wind conditions and temperature conditions. You're right, Drew, these were things that the experienced operators knew. It was a core part of how they understood and monitored the safe state of the plant, but it wasn't something that anyone else outside the operators actually knew happened.
Drew: The ethnography paper is pretty light on when it comes to things like direct quotes. The one that they do callout is related to this. I don't know the exact quote here, but it goes something like, dude, I was just chilling, watching what was going on the floor, and pretty relaxed. I smelled this smell of oil that wasn't supposed to be there. I immediately knew that I had to take action or something bad was going to happen. They said this wasn't just one story. This was repeatedly all sorts of stories the drillers told them of times whether it was the floor vibrating, a particular noise, or a particular smell that had triggered them the need to do something.
David: Look, Drew. What you said earlier about monitoring the smell of hydrocarbons. If you're a driller and you smell hydrocarbons, then you're in real trouble. You're actually not meant to have the hydrocarbons at the surface when you're drilling.
Drew: Yeah. That wasn't directly clear from the paper. They just said the smell of oil. Recognizing that smell is obviously a bad sign.
David: It might have been a pump then, hydraulic fluid, or something.
Drew: Another thing they mentioned is the role of all the other activities that are going on in the workload and the ability to perform all of the core functions correctly. That's another thing that gets left out of the formal procedures. Most procedures just assume that the procedure is happening in isolation. They assume you're not going to get interrupted or your attention’s going to get demanded by other things.
Whereas, the FRAM analysis makes it clear that everything they do is going to be affected in terms of its time, its precision, or its urgency by all of the other functions and other activities that are going on. Things that they are doing, things that other people are doing, and things that the supervisor is doing all affect their ability to do their core functions.
The final one that stands out—which I guess isn't a surprise because it always comes up in this sort of analysis—is just how much disruption and variability were a normal part of work? They talk about equipment malfunctions and parts starting to be repaired, not as these exceptional events, but as just an almost ever-present routine part of doing the work. Dealing with these disruptions, dealing with equipment that's not working, and dealing with things not being perfect but the job still needs to get done.
David: Yeah, Drew. We talked about performance variability is something that a lot of our listeners will think about performance variability. But at least I think about it as something that's just every single day, no hour, no minute, or no day is the same for people, for anyone really in any role. I just think about my day of what I think it’s going to look like in the morning versus when I look back at the end of the day and what it looks like. It's very rarely gone the way I thought it was going to go.
In all of these operational roles, thinking about them as quite simple linear activities that run 99% of the time, in the same way, is just not the right way for us to think about work.
Drew: David, getting back to the question for the episode, I'd like to get your opinion on a couple of things coming out of the paper mostly related to is this use of complex models worthwhile?
The first thing is I would like your comment on the difficulty. The authors suggest this is not a particularly complex thing to do. They say someone who has never seen a graphical representation of FRAM, it might seem relatively complex. I would say anyone who has seen a FRAM model is going to get frightened away by the apparent complexity of the diagram. But they claimed that it's not that complicated. It's just a mutual understanding among professionals working as a team. It's a complex discussion about a complex relationship, but it's done fairly simply. Would you agree with that? Do you think this is something anyone could do?
David: I think it is. It's not as easy as teaching someone how to flowchart (I suppose). You've got quite clear rules for how the model gets used. You've got functions inside, you've got states, and then you've got those six vertices with those relationships. I think it could be done. It could be facilitated. There's no way you couldn't have a FRAM facilitator in the same way that people facilitate learning teams and other things. It might not be a bad thing for an organization to have.
What I like about the use of a FRAM model would be it would allow organizations to narrow that gap between work as imagined and work as done. If you think about using the FRAM model, (I suppose) not to say, I'm going to write all of my procedures and I'm going to paste the FRAM diagram in the front of my work procedures. That's not the thing I'd do. I don't even know if I have work procedures anyway.
Using a FRAM model as part of something like a learning team where you have supervisors, operators, and safety professionals together trying to make sense of a particular activity could be better than other modeling tools that are available.
Drew: What do you reckon is the practical benefit of trying to do the modeling?
David: I think there's a couple. Most of the benefits are probably more to the organization than to the operator because the operators got a view of that role that's far more nuanced and complex than any FRAM representation you could do. You've got to think of that model as more useful to the roles in the organization that support or constrain that operator or that driller (in this instance) from doing that job.
If the supervisor, manager, and safety professional gets a better understanding of the work and the challenges, then that might just change the support and conditions in the organization to help that driller a little bit more to be successful in their work.
I don't think the model helps the operator in a direct way, but in an indirect way, it might just help the organization conditions around their work.
Drew: A big part of that is the communication that goes on the creation of the model. It's important not to consider the model itself an adequate representation of the real complexity or is an end in its own right. Sometimes, researchers lose track of that.
The researchers do a FRAM model and they say, aha, we've done a FRAM model. Aren't we clever? This paper almost does that. It actually thinks its research question is how do you do a FRAM model of this system? It really isn't. The research question is just, what do the drillers do? Which is an interesting, useful, and valuable question to ask, answer, and have those conversations. It's not the production of the FRAM model that is the end result, or that is the achievement.
David: What I liked with a question like that, Drew, what do drillers do, is the object of understanding is the work. They haven't come in and gone, how do drillers keep something safe? They're starting with understanding the work. And then they can understand what the implications of that work might be for safety or parts of their model.
The research was quite well done. It's an interesting question. The discussion about FRAM for our listeners has hopefully been a little bit helpful in them thinking there are options out there for them to model work. Drew, if our listeners are thinking about, I know about FRAM, or I don't know about FRAM. I'm going to go and look at this tool a little bit more. What would be your practical takeaways?
Drew: The very first one is there’s genuine value for workers, management, and safety in co-creating new descriptions of work. Not with the object being that new description. We're not talking about having workers involved in writing procedures where the end thing is having a documented procedure. We're talking about having that sophisticated complex discussion around what goes into work so everyone comes away with a better understanding of other people's jobs. David, as you've said before, so that management and safety are more likely to put in place solutions and help that genuinely help the workers because they understand what the job is like.
The second practical takeaway, I'd say, is that it's important when we select our methods in safety that we don't pick methods that strip away the complexity and the messy details. When you hear people sometimes claim, safety is simple. We don't need to overcomplicate it. Work is already complex. We're not overcomplicating things when we use methods that are capable of embracing that complexity and helping us to understand it. The complexity is there. The safety is not simple. The safety is hidden from a lot of our simplistic representations.
The final message I'd put in there is to ask yourself whether a JSA or a risk metric would've captured the importance of having a good nose for smells. That's what this method can offer you is understanding why having a sense of smell is important.
David: They're good takeaways, Drew. We're not talking about creating more complex prescriptions of work through procedures. What we're talking about is potentially a tool to model an activity or a role in an organization that might help narrow that gap between work as imagined and work as done, so we can create the conditions for the work as done to be supported by the organization more of the time.
Drew, the other thing we might like to know from our listeners is quite simple. If any of our listeners have used it in the past or are using it in their organization FRAM or Nancy Leveson's STAMP models, are you using it to model work? Are you using it as part of your investigations? What do you think? What’s your experience with those models? And what would you offer other listeners in terms of advice around that?
Drew: This week, we asked the question, why do we need complex models to explain simple work? I think the answer here is—at least in this case of drilling—work is never simple. Even something which seems like a fairly straightforward mundane job, there are always interesting and complex things to discover. Those are things that matter to safety.
David: That's it for this week. We hope you found this episode thought-provoking and ultimately useful in shaping the safety of work in your own organization. Leave us a review on your podcast feed. Drop us a comment on our LinkedIn page, or send any other questions or ideas for future episodes to us at feedback@safetyofwork.com.