Welcome back to the Safety of Work podcast. Today we discuss whether pre-surgery checklists improve the safety of procedures.
We use the papers to frame our discussion: A Systematic Review of the Effectiveness, Compliance, and Critical Factors for Implementation of Safety Checklists in Surgery; Systematic Review and Meta-Analysis of the Effect of the World Health Organization Surgical Safety Checklist on Post-Operative Complications; and The Effects of Safety Checklists in Medicine.
Tune in to hear our thoughts on this potentially life or death issue.
Topics:
Quotes:
“Checklists are one of those things that have been associated with safety for a long time and associated in a way that gives them quite a good name.”
“Lots of stuff being recorded as positively improving with the introduction of a checklist.”
“If you can’t convince a multidisciplinary team that this belongs on the checklist, because they all agree there is a clear link between this item and a particular accident that they all know about, then you don’t get to put it on the checklist.”
Resources:
Borchard, A., Schwappach, D.L., Barbir, A., & Bezzola, P. (2012).A Systematic Review of the Effectiveness, Compliance, and Critical Factors for Implementation of Safety Checklists in Surgery Annals of Surgery, 256, 925–933.
Bergs, J., Hellings, J., Cleemput, I., Zurel, Ö., De Troyer, V., Van Hiel, M., ... & Vandijck, D. (2014). Systematic Review and Meta-Analysis of the Effect of the World Health Organization Surgical Safety Checklist on Post-Operative Complications. British Journal of Surgery, 101(3), 150-158.
Thomassen, Ø., Storesund, A., Søfteland, E., & Brattebø, G. (2014). The Effects of Safety Checklists in Medicine: a systematic review. Acta Anaesthesiologica Scandinavica, 58(1), 5-18.
David: You're listening to the Safety of Work Podcast episode 31. Today, we're asking the question, do pre-surgery checklists improve patient safety? Let's get started.
Hey, everybody. My name's David Provan. I'm here with Drew Rae and we’re from the Safety Science Innovation Lab at Griffith University. Welcome to the Safety of Work Podcast. If this is your first time listening, then thanks for coming. If you're a regular listener, then thanks for coming back. The podcast is produced every week and the show notes can be found at safetyofwork.com. In each episode, we ask an important question in relation to the safety of work or the work of safety, and we examine the evidence surrounding it. Drew, what's today's question?
Drew: Specific question for today is do pre-surgery checklists improve patient safety? David, I'm hoping that we can actually have a more general discussion about checklists. Even though the evidence we're looking at really only answers that one specific question. There were lots of checklists, procedures, and rules in safety. We talk about it a fair bit on the podcast and at other events we’re at. Usually, we tend to bundle all of these things together as administrative controls. Administrative control is kind of like a swear word in safety. Usually, the subtext is, it’s administrative and it doesn't really work. A lot of this paperwork gets talked about in the same conversations as safety bureaucracy, safety clutter, safety paperwork.
One of the big fears that we had when we started talking about safety clutter, was that people would just slash and burn all of the safety paperwork instead of working out what does and doesn't matter. The annoying thing is if people are much slower than we expected in getting rid of the clear clutter, that perhaps too ready to get rid of some things like checklists that in fact we shouldn't necessarily be getting rid of. Checklists are one of those things that have been associated with safety for a long time and associated in a way that gives them quite a good name. Perhaps, the most well-known example is pilot pre-flight checklists.
I don't know any pilot who talks about that as mindless bureaucracy. It's almost part of professionalism to be ingrained in those checklists and to use them. But then, a lot of that aviation stuff gets copied into other industries and a real particular error we can see is pointing from aviation towards healthcare. It's really worth asking, does a practice that seems to work well in one place really work that well and certainly does it work well when it's translated? We're not going to be looking at whether they work in aviation but we are going to ask, do they work when they’re translated into healthcare?
David: That's a great overview, Drew, and there's actually lots of information on checklists out there and there's actually lots of information on checklists in healthcare and safety, which we’ll go through today. There's also quite a popular book called, The Checklist Manifesto, which is how to get things right by Atul Gawande, which I'm sure many of our listeners are familiar with. Drew, you and I recently wrote a chapter for The OHS Body of Knowledge, the Australian Professional Body of Knowledge titled, Rules and Procedures for Safety, where we went through some of this checklist literature more broadly.
What I want to do today is actually talk about what the research says about the outcomes that are created by using checklists. In terms of healthcare, Drew, there’s sort of a general statistic that 5%–10% of patients worldwide are exposed to some form of an adverse event during the course of their hospital treatment. This might be just the wrong dose, the wrong drug, the wrong timing, or the wrong procedure. That rate of 1 in 10 is thrown around a lot when we talk about healthcare, and hence why I think the sectors look to industries that have a much lower reported adverse outcome and trying to see what they're doing, but we don’t.
Drew: Yeah, you pretty much can't pick up a paper about patient safety without it starting off with some sort of dodgy statistic about how many people in healthcare died due to preventable events. There’s a lot of reason to be skeptical about exactly what does and doesn't count as preventable and exactly what does and doesn't count as an error, but that's beside the point. Healthcare certainly is a very human-driven profession and there’s going to be lots of opportunity for improvement if we can enhance human performance. The question is do pre-surgery checklists enhance performance in a way that we can measure.
Given that they're in a healthcare setting where their health is being closely monitored, there's a good chance that we can actually directly link from the action to the ultimate outcome for the patient.
David: Yeah, the laboratories are all there in the hospital to observe and to measure. What the healthcare sectors have done is look at all of these adverse events and decide in hindsight that many of those events were due to controllable actions of the people involved in the surgery, so that the healthcare professionals and clinicians. What happened, I think, is it appears as though in the early 2000s, the World Health Organization globally published the surgical safety checklist and there's a surgical patient safety system. In 2008, there were two separate studies in a total of 14 hospitals worldwide who ran these studies. They implemented the surgical checklist and showed dramatic reductions in patient safety negative adverse outcomes.
What happened then was, I suppose, there was some questioning by the practitioners on the effectiveness of the checklist, there was some questioning on the external validity of these results, and it seems to have created a couple of years of a whole lot of research that went into checklists, so the papers that I pulled out for us to talk about today, three of them, they're all published in that 2012–2013 period, which is a couple of years after that World Health Organization global effort.
Drew, that’s our question, do pre-surgery checklists improve patient safety outcomes? Let's talk about the papers, They're all reviews of the literature, there's no original research in any of these three papers, and I'm really interested to get your views on the way that the authors approached the different review studies.
There was actually a lot of stuff, Drew, not in the safety journals, but you might be able to fill me in any way, that there’s a huge amount of medical and healthcare research that’s published every year. The industry has got obviously a long history of evidence-based practice and a lot of emphasis on research. All of the studies were done by healthcare institutions or universities and published in healthcare-related journals.
Do you see much healthcare research coming into the more general safety science type of journal?
Drew: We see some. There are actually two categories of healthcare research. There is a body of research that's basically done by doctors, or by researchers closely aligned with doctors and that tends to get published directly in medical journals, or in safety versions of medical journals. Then there’s sort of a second body of work which is around nursing practice and around how healthcare work is done, which includes some of the occupational health and safety stuff to do with healthcare practice. It tends to be sort of lower status, not as in lower quality, but just as in not considered real medicine. Some of that account finds its way more into the safety literature just sort of to try to regain the status and the respectability.
I'm not being critical here of either body of research and you can find great stuff in medical journals. You can find great stuff in nursing journals. You can find great medical stuff in safety journals.
David: I was pretty impressed with the amount of research there was and probably better quality research than we see a lot of times when we talk about safety research. Article one was titled, The Effects of Safety Checklists in Medicine: A Systematic Review. It was published in 2013 in The Journal of Anaesthesiologica Scandinavia. Four authors, Thomassen, Storesund, Søfteland, and Brattebø. They're all Norwegian, all at the University Hospital in Bergen.
This is what they did. They went and searched a whole heap of different databases for checklists and medicine, and they came back with nearly 7500 hits against their criteria. They shortlisted that to 114 studies based on the title and abstract, then they reviewed those 114 papers, and ended up including 34 of those articles in their review. They had 34 papers. They split them up into four different categories.
At this point, they weren’t just specifically looking at patient safety outcomes, they are really looking at what's the relationship between checklists and safety in medicine. Seven of these articles had heart patient safety outcome measures, which means they were looking at the relationship between the checklist and patient safety or patient safety outcomes.
Six of the articles were about adherence to guidelines. What's the relationship between having a checklist and complying with medical practice requirements. Sixteen articles were categorized as human factors. This is where they're looking at the relationship between checklists and mistakes by clinicians. Five articles which categorize adverse events. This is the relationship between the checklist and an unplanned situation during the surgery, whether the result of a mistake or not, just an unplanned situation.
Drew, these four different types of the category are starting without a really clear question with a systematic review and then just basically creating some categories based on the articles you've got, how common is that kind of an approach to a review like this?
Drew: One of the approaches that they like to do in medicine is to very clearly define the endpoint that they're measuring for and then just reject everything that doesn't match that endpoint. You get some very uniform easily compatible studies if you say we're only interested in a very specific checklist, and we're only interested in studies that have measures of mortality as the outcome.
If they’ve done that, they would have only had seven articles at most. I'm not actually certain that all seven of those measure mortality because a couple of them are more about complications rather than specific death as an outcome. Having the broader one lets you cover this broader range of things. It also sometimes lets you claim credit even if you don't have good outcome measures, though.
I think in this case, we should actually expect the bias to be the other way. If an organization introduces a checklist and starts enforcing that checklist, if there's any bias, it would be towards categorizing more things as mistakes or more things as adverse events. You wouldn't expect the checklist to cause things that previously were counted as errors to stop being counted as errors. You are expected to accidentally drive the apparent number up. I think we’d expect to see the error the other way. If we see that a checklist actually reduces the number of times they've decided that someone's committed a violation, then that's actually probably a real effect that's happening.
David: Thanks, that’s a good explanation of the process. What do they find when they review these 34 studies? Four of the 34 had statistically significant reductions in post-surgery mortality. Did the person die as a result of the surgery or shortly after? Six studies showed a significant decrease in post-surgery complications, and then 16 studies looked also at the secondary outcomes of the checklist. What's the relationship between checklist and communication, team performance, the understanding of daily goals, information flows, perception of safety, safety attitudes, and safety behaviors?
Some of the results that it reported in this from the individual studies were quite amazing. Now, the writer on this says that the individual study quality was quite variable, but just the headline results in reducing mortality by 50% from 1.5% of procedures down to 0.8% and having similar size effects to do with complications and adverse events. Things like equipment failure decreasing from 87%–47%. Incidents were generally involving equipment which kind of makes sense if the checklist involves you to check that the equipment is serviceable at the start of the procedure, then you're probably less likely to find that it’s not working when you need it during the procedure. Some of these seem obvious but the effects are really large.
Drew: Did you happen to look up that one about the equipment failure incidents?
David: I didn’t go to the original papers, no.
Drew: Neither have I. I'm just looking at it now and good grief, do you really want to go into surgery where 87% of the time, they discovered during surgery that something is not working. If that's something that reduces to 47%, is that likely that they've checked and remembered to change the batteries?
David: I'm sorry, Drew. That's an issue with my note-taking. Of the overall adverse events, 87% of those adverse events were related to equipment failure, it wasn't 87% of the time something doesn't work. Sorry. Good clarification and it just goes to show, even between Drew and I who are normally on the same page with our notes, every time you put a number down, you really got to be careful of the context that you put it down in.
These other things, other aspects, these secondary outcomes are also quite dramatically improved. Communication [...] is reduced, information loss is an interesting one, that without a checklist, one in five surgery suffers from information loss. Test results go missing, or information about patient history goes missing and things like that. That goes from 20% of cases down to 3% of cases with the introduction of the checklist.
Lots of stuff being reported as positively improving with the introduction of a checklist. One of the interesting things is the story of the time it takes. How much time does it take to do the checklist? One of the studies actually had measured this and when they implemented the checklist, the average time from what they call admission to incision when they put someone in the operating theatre until they start cutting increased from 23–29 minutes. You're talking about a 25% increase there in preparation time by incorporating the checklist.
Drew: Yeah. That’s what I love to hear. The doctors are racing to make sure they make that first cut before the 30-minute mark just like you order at Pizza Hut where there's a text that’s flashing red. If you don’t cut, you fail to meet your daily goal. I'd much rather that we’re taking out the extra six minutes. What I particularly like is that there's a lot of consistency here between the type of mechanism of improvement and the claims in end results. It’s not just your random safety, it’s some specific types of things that are being improved and the types of things that it's very plausible that you would see from a checklist.
David: We’ll talk about why we think that this works well towards the end of the podcast, but I agree, Drew. Things like having serviceable equipment, having the information I need to make decisions in relation to the patient, checking that all of the pre-surgery checks have been done. You can see that this direct relationship between like you say, the mechanism that the checklist is prompting, and the safety of the work. I think that's an important takeaway we’ll come back to at the end.
Drew: Moving on to the next paper, I think it’s worth pointing out the final thing that the author is saying in this one. It is that it basically concludes that the evidence in favor of checklists is so good. We've got a much more interesting research question which is why do some people don't like checklists? Why does everyone not love them and want them in place? That was sort of the further question then is, if they're so good then how come they're not universally loved?
David: Yeah. I think it was written in the paper in a way that seemed to baffle the researchers when they had done the work and they just said, the evidence is so clear, why do all healthcare workers not embrace the idea of a safety checklist? Why don’t they love them? We’ll talk about that a bit further.
Drew, you mentioned very specific research or systematic reviews that go into healthcare. What I wanted to do is I didn’t want to just stick with that one paper because it was very compelling. There were lots of different studies, lots of different qualities and I got a bit interested in checklists, to be perfectly honest, so I kept having a read.
I found that meta-analysis. We've talked about them on the podcast before where you get a group of studies that are asking a similar question and designed in a similar way. You pull all of the data and you reconduct the analysis with a much larger data, so you're almost turning seven studies, in this case, into one really big study.
This title was called A Systematic Review and Meta-Analysis of the Effect of the World Health Organization Surgical Safety Checklist on Postoperative Complications. Sorry, Drew, but if you got nine authors on the paper then we don't have time on the podcast to read out the names. But they are all from Belgian universities and medical institutions. It was published in 2013 in the British Journal of Surgery. Drew, nine authors on a paper getting everyone's publication counts up or what's going on there?
Drew: If it's a physics paper, you can imagine the role that everyone had but a systematic review where they ended up only needing to look at seven studies and do statistics around it. You can imagine one person going through, finding the studies, and filtering them. The second person checking their work. The third person helping out with the statistics. The fourth person running the lab. The fifth person happened to just work down the corridor. The sixth person is a friend of the fifth person. Yeah, it gets to a point where you begin to get a bit skeptical.
David: Yeah. This is a systematic review. They started with 723 studies and they selected seven. They had very specific criteria like you said earlier. They needed to be quantitative. It needed to be a randomized control trial. A control before and after study, interrupted time series, or repeated measures studies. It needed good quantitative measurement over time with an intervention.
That was specifically looking at the impact of the WHO (World Health Organization) Checklist not just any sort of checklist, and they had to study postoperative complications and include mortality. They excluded these studies if it only addressed a particular issue. As you said, Drew, if it was a study that looked at the checklist but was only looking at infection rates post-surgery was excluded. It wanted any complications.
There are pages and pages of statistics, most of which I don't understand in the paper, I must admit. But basically, what they worked out was that when they combined all these studies, together there were 8429 post-surgery complications before the checklist and 6769 complications afterward. Somewhere like a 20% reduction across all of the studies when they looked at 30-day mortality rates. Then complications that included things like infection, blood loss, unplanned return to surgery, pneumonia, and those types of complications.
A lot of the studies measured adherence to the checklist, which is one of the other criteria that I didn't mention, is that each of the studies had to assess adherence to the checklist. Not just we have a checklist in our hospital, but the frequency and completeness of people using that checklist and concluded that half of the studies had good completion and half the studies had bad. The ones that had the good completion of the checklist had a great reduction in complications.
This review provided some pretty strong conclusions and statistics, but at the very end—Drew, I'm interested in your thoughts about this—the author says we can't possibly regard this as definitive proof in the absence of high-quality studies. Like any meta-analysis, they said we'd still like to have better studies.
Drew: I suspect that what's going on here is that it's really hard to decide to do different things at once. Once you've decided that better adherence to the checklist leads to better outcomes, then you got to be cautious about taking credit for these studies that have got poor adherence to the checklist.
The trouble here is that they've decided that we can see this apparent pattern that as you do more and more following the checklist, you'll get better and better outcomes. But if we filter out all the ones that had poor adherence, then we don't get our overall underlying figure, statistically significant that having the checklist improves our outcomes.
I don't think we should take cautious statements like that as a denial of a clear pattern. The best evidence is it seems to work but don't assume that's a slam dunk. We can always try to get better evidence than the best current evidence.
David: Yeah, absolutely. If listeners can follow my train of thought on my path of research on this. I went and got the first paper that they looked at impacts on patient safety then a whole range of other factors that are related to patient safety. I went and tried to get that hard numbers by getting the meta-analysis paper which was looking at how significant is that impact on patient safety outcomes. Both of the papers talked about compliance and adherence.
I went and got a third paper which is another systematic review and it was titled, A Systematic Review of the Effectiveness, Compliance, and Critical Factors for Implementation of Safety Checklist in Surgery. If we know the checklist, I think we can say that we know the checklist improves patient safety outcomes, with the caveat of when they use and follow, then we need to understand when and how they used, followed, and what are some of the issues like we mentioned earlier.
This is a really interesting question. If the evidence is strong, that it improves the quality of work and safety, then why isn't compliance really, really easy? This paper was published in the Annals of Surgery. Another systematic review, started with nearly 5000 articles, shortlisted down to 84 they did a full read of, and then 22 were included in the review.
Interestingly in the review, because I was looking at checklist compliance, the findings in these studies range from 12% compliance to 100% compliance. There were studies of hospitals that had 12% completion rates or use rates on the checklist and other hospitals had 100% with an average of 75%. That's a huge range of compliance for very similar checklists and very similar processes in very similar operating environments.
Drew: Or it's a huge range in honesty of people about how often they probably complete the checklist.
David: Yes, it could well be. You raised a really good point of those 22 studies, a number of them were interview-based which is asking people how often they do things. Some of them were real-time observation-based. Some of them were documentation reviews. You're right, Drew. The quality or accuracy of the data, depending on how the data was gathered, can have a big impact here as well.
Drew: Although I think it's fairly indicative in one sense, a lot of these studies, the safety team, or the people responsible for the checklist are also part of doing the study. If people tell you that they hate your checklist, then regardless of whether they're doing it because they actually never complete the checklist or they're just trying to send a message to you about how much they hate doing the checklist, you still got to take that 12% rate as people really don't find this helpful.
David: Yeah, absolutely which is still really interesting. Maybe that's a good question why they don't find it helpful. We are just about to get into that when we do the practical takeaways. In this last paper, 15 of the studies evaluated compliance with the checklist. Really looking at this frequency of using completeness of the checklist in terms of the records point of view.
Here's what they concluded from these studies. The success of the implementation of a checklist and it's compliance levels is higher when it's led by a multidisciplinary team. A team that involves the uses of the checklist, as well as the designs of the checklist, as well as supervisors and managers, like organizations, use working groups, group of change champions or something like that, involving a multidisciplinary team in the implementation process is going to lead to greater compliance rather than when it's just mandated by a single person from a manager or from a safety department.
Discussion about common cause events and how the checklist related to preventing them was really important. It seems that getting people together talking about what goes wrong in surgery, what are the common causes of the things going wrong, and then showing how the checklist checks for those things that can be common causes can really help people understand the connection between the safety work and the safety of work. When people can see that this me checking this is going to mean that this is not going to happen during the process really increases compliance.
Drew: I think this one needs to be read in conjunction with that previous one about the multidisciplinary team because it's not just about one person coming out with a checklist then explains to everyone else why it's important. It's if you can't convince a multidisciplinary team that this belongs on the checklist because they all agree that there's a clear link between this item and a particular accident that they all know about, then you don't get to put it on the checklist. Having that need to explain and justify results in shortage checklists as well as people who understand the checklist better.
David: Yeah, Drew. You described there exactly how our test for consensus is in the safety clutter paper which is, do all stakeholders agree that this adds value and it's necessary? Absolutely. It seems to have a real impact.
This paper talks about the why and how. Why should people use a checklist and how should they complete it. I think that's a really good framework to think about checklists and checklist compliance which is why. Does the person even feel like it's their role to take care of the items in the checklist that they need to check off? Do they feel that it connects to safety risks associated with their work?
If I feel responsible for it and I feel it's important from a risk, then there's my why. But then, the how is really important. Do I know how? Have I got the information I need? Have I got the resources? Have I got the time?
For example, the after six minutes. If I don't get an extra six minutes added to the surgery schedule then it's really hard for me to add the extra six minutes in to do the checklist. I know that pilots feel that way with plane turn arounds, with a delayed landing and then a non-scheduled take-off. Sometimes they just physically don't have the time to do all the checks that they meant to do. That why and how is a good framework for people to think about.
Drew, we might just go straight to practical takeaways if you’re happy to do that.
Drew: Sure. There's one last thing that I'd like to mention about the findings. That is that one thing that you don't get with this study is talking about power dynamics. I think a lot of people in healthcare don't like to acknowledge the class and power issues that arise, particularly when it's around things like surgery where you have multi-disciplinary teams. Where you have doctors. You have a nursing staff. You have allied healthcare staff. You have anesthesiologists who are doctors but are their own special class.
When we talk about socio-technical systems, what we mean is that things like a checklist then become a technology that feeds into these preexisting relationships and how the work happens. We can think of checklists as a very straightforward thing that we are simply marking off items and that acts as a barrier to prevent hazards.
I really suspect what's going on here is a lot more complicated. That this is actually a tool that allows people to interrupt. It's a tool that allows people to talk to people in higher positions of authority, to slow things down if they need to or to stop them pushing ahead when they don't feel safe to push ahead. None of that contradicts the value of the checklist but the value may become in ways more than obvious.
David: Yeah, I think that's a really good perspective. I think power and authority gradients and that is talked a lot about in healthcare, is talked a lot about in aviation, other sectors. It might be a good topic for our future podcast because I know there are some studies in aviation which looked at challenges in the cockpit based on seniority and those demographic type factors and makeup of the crew factors. Checklists can really help provide a common language and a mechanism to get past that power gradient.
Drew: Let's jumpstart into the practical takeaways. The most important one I think is that this is an activity where if used in the right type of situations, in this type of situation we're talking about pre-surgery, checklists can be a good and useful safety tool. I think that's just the first simple takeaway. Don't think this is paperwork. This is [...] and flick, therefore this is not helpful.
David: Drew, let's talk really targetly at the development of a checklist, the items on the checklist, and how it gets integrated into the work and the workplace. Making sure the items on the checklist match the real and agreed safety risks and the process that's being undertaken, not having any redundant items. Having redundant items on there distracts from the important items and we know that from the clutter research that if you actually take away the stuff that's not important and just leave the stuff that is important, then you'll get better quality and better compliance of the stuff that's remaining. So short, simple, straightforward.
If you want to have a team-based understanding of the checklist, one of the risks is if you want to create a conversation on the checklist. Some of the studies have talked about putting the checklist up on the wall in poster size in the operating theater and actually having the operating team stand around the poster and actually go through the checklist together.
Thinking about the medium as well when you develop it. What's on it, but also how that medium is communicated in the way that it's going to be used in the workplace. I didn't mind that as an idea, to actually create a team-based conversation around the checklist rather than one person holding a clipboard.
Drew: I don't know the answer to what works best, but I agree absolutely that checklist design isn't just about thinking what items are there. It’s thinking about how people are going to use this. Who is going to be able to see it as they go through it? Who's going to lead the process? Who's going to be part of the process?
David: So if you're going to put a checklist into part of your process, participation and agreement of the whole team is really important. The benefits, the coaching, the feedback, the education that goes along with it, the time and the resources, all of those things. I think simply, in safety, sometimes we just develop a checklist and throw it over the fence but the research here would recommend you don't do that. You actually take the time to figure out how long it's going to take, how it's going to work into the process, whether people can do it.
I think what I'm learning most in the podcast as I'm going through a lot of these research is the care and attention that you should put into every single thing that you try to do to intervene in the workplace is much greater than I ever probably worried about throughout my career. Just implementing one checklist, five items into surgery or something like that should be done as a really, really big project.
Drew: If you're thinking about that six minutes and how that is valuably spent every time someone goes into surgery, that's a heck of a lot of time. If it's five minutes, if it's seven minutes, that makes a difference not just in the overall efficiency but in whether it's actually going to get done and done properly. Then if you're going to use that time every single time, you really want to get the maximum value out of it.
David: Absolutely, Drew. One of the points I want to give a perspective on when I looked at this WHO Checklist, it actually contains 22 items and they talked about it as this surgical safety checklist as if it's one thing, but it's actually split into three sections. There are seven questions that are done before anesthesia before the patient is put to sleep. Then, they do the next 10 questions on the checklist before they do the first skin incision, so after the anesthesia is taking its effect, they go to the next 10 items, then they do their surgery. Then, there are five items on the checklist which get done before the patient leaves the operating room. Have we accounted for everything? Have we sewn up everything we needed to?
I thought that's really interesting. There are actually three separate checklists designed in that critical phase of the work as a way of planning the risk controls for the next short phase of work. It's different from how we think about checklists in the general workplace. It started in aviation, there's a preflight checklist, and probably a pre-landing checklist, and things like that. But normally we just have a pre-start checklist which is do this once at the start of the day and then get on and do your day's work. I thought this was a really important distinction between a general checklist and a really task-specific checklist.
Drew: What struck me was just how proximate the checks are to the actual critical moment. One of the things we've noticed with things like vehicle checklists is they're built-in to the prestart and that's not necessarily the right time to do it. Some of the things you’re checking about the vehicle are irrelevant or they are actually on site and start doing a particular operation.
Operators might find it hard to do that check until they get to the site. A checklist that's just before you start everything can be fairly irrelevant, but if you think of work, pretty much any job has these really critical moments and if you get things right at those moments, everything else flows neatly.
For surgery, it's just before you put the patient under. It's just before you make the first cut. If you're doing mechanical work, it's making sure that everything is isolated right at that moment just before you physically start work. It's making sure all the tools, test racks and stuff are moved before you start closing up. If you identify when those critical moments are, that's when something like a checklist can really support making things safer.
David: Drew, if we think about safety work versus the safety of work, this is where I think that conversation just there is exactly the distinction and the conversation that we wanted to make with that model and that paper which is that safety gets improved when you actually make physical changes to the task, the tools, or the work activity.
If the checklist is prompting physical checks, physical changes, and physical setup practices for the work, then this is where we're seeing in all of this research, this connection between the safety work process like a checklist, and the safety of work like the patient safety outcomes. I just think we talked a lot about this podcast, but this is almost like the perfect example of what we've been talking about most of the way through.
Drew: Here's a useful test. David, you've been looking at the WHO checklist a fair bit as you went through preparing for this episode. Is there any item on that checklist where if the item can't be ticked, the team can't take immediate action?
David: I suppose that would be the assumption. You mean take immediate action to correct it?
Drew: To correct it or to stop stuff going ahead, is there something they can do? I think that's the danger if you get a checklist wrong is you go through an, okay I got seven items here and four of them are everyone has been properly trained. If the answer is no, there's nothing you can do about it.
David: The answer is very much yes. Something like is all of the surgical equipment sterilized? If it's not or if there's a problem, then the crew can just go and get a fresh lot of sterilized surgical equipment. There is definitely the opportunity and means to get the things in place that are on the checklist. I think that's what you're asking about.
Drew: I think that's a great checklist. If the answer is no, then very quickly the answer will be it is now, I fixed it.
David: Yes, exactly. I know this has been done in a number of other industries like critical steps, critical states through work activities, and how to make sure that the critical things are in place.
Drew, invitations for our listeners. I’d like to know how our listeners are using checklists as part of their safety management activities and success stories, or problems that they're seeing with checklist because we talk a lot about rules, procedures, and other admin things, but until I started reading about this in the last week, I hadn't thought about the checklist as a very specific thing to use. I actually think they're more interesting than procedures.
Drew: I'm inclined to agree. I would love to hear stories, not just about checklists in general but specific checklists and specific items. You want something that you've been really glad was on a checklist because you have the right item and the right check at the right time. You think it saved your life or saved someone else's life. What are your horror story checklists that you think are irrelevant, take up time, don’t help you out.
David: Yeah, I'm sure there are a lot of planes around the world that haven't taken off because something has been wrong in their preflight checklist. I think there are a lot of people who might have had their lives saved without necessarily knowing it, just by checklist. Surgery as well by the look of these numbers that we've talked about today. There's a lot of people who had operations. Maybe some of our listeners have been helped along by a checklist in the operating room.
Drew, today we asked the question, do pre-surgery checklists improve patient safety outcomes? The short answer?
Drew: You’re asking me? Yes, absolutely. Evidence is not definitive, but it all points in the same direction. There are a number of mechanisms, a number of intermediate outcomes that get improved, and a number of long-term morbidity outcomes that get improved.
David: That's it for this week. We hope you found this episode thought-provoking and ultimately useful in shaping the safety of work in your own organization. Send us a review or any comments, or questions, or ideas for future episodes to us at feedback@safetyofwork.com.