On today’s episode of Safety of Work, we discuss how practitioners can find and read research. We thought this was a great topic, as this is often a dilemma for safety practitioners.
Tune in to hear our discussion and insights.
Topics:
Quotes:
“The basic rule for what is legal and what is not, is authors own the text until they submit it to a publisher.”
“Anyone who’s got even just a student account at a university, shares in the subscription. So if they log in, while they’re on campus, then they have free access to a lot of stuff which is paywalled, when they’re off campus.”
“Internal validity is how much within the scope of the paper it has correctly answered the question.”
Resources:
Drew: You're listening to the Safety of Work Podcast episode 34. Today, we're asking the question, how can practitioners find and read research? Let's get started.
Hey, everybody. My name is Drew Rae. I'm here with David Provan and this week we thought we'd address a problem that comes up often for safety practitioners who'd like to engage more in evidence-based practice. That's just where do you look to find research online and how do you get proper access to it once you find it.
This is a question that David and I get asked quite a bit. Sometimes it's really quite direct. People say to us, hey, I want to read this paper, but I tried to go to the website and it's asking me $40 to get a copy of one paper. Sometimes it's not that quite direct. It's a bit more subtle. We find people who are discussing and criticizing ideas online when it's obvious that they haven't been able to find the original material. All they got is a third-hand hand me down version of the research. We thought that since you're listening to this podcast which is all about safety research, that we'll give some tips pertaining to the research itself.
David: You might even be a researcher who's doing full-time research, so hopefully you find this useful no matter what role you're in. We thought we might step through it in three separate questions. We'll talk about how you search for good papers. We'll talk about how you found it in the first place, then how you got hold of a copy of the paper. Then once you get hold of it, the third question is how do you skim read it or review it to know if it's any good and how it might apply to what you're trying to find out.
With that as our high-level framework, you sit on the editorial board of Safety Science. Can you start by telling us what an academic journal actually is and how does it work?
Drew: This one is tricky because it's one of those areas where the deep history of academia gets in the road of what might make any sense. Quite literally, a journal is a periodical that is going to make total sense to anyone who's maybe over the age of about 35 and is going to seem crazy to anyone younger than that.
The way a periodical works is you have sitting in your library or in your office a big set of ring binders. Every week, every month, or every six months a new supplement comes out and gets added into the last spot on the last ring binder. Eventually, you'll just have shelves and shelves of these filled up.
In academia, the way periodicals work is they only update from the end, so their shelves get longer and longer. In fields like the law, they used to actually have corrections. They have to slide one out and slide the new one in various places. That's the way it physically looks. If you go to an old library, they will have places called stacks which are just these really old dusty shelves and just long, long hardbound versions of these periodicals.
David, you and I do PhDs at different times. That's literally how I spent months of my PhDs. It’s wandering through the stacks with a scrap of paper telling me the number I'm looking for, digging out this big thick volume, going through the page I want, taking it down to the photocopier, photocopying it, and that's how I get hold of a journal paper.
David: Yeah, I did do my undergraduate psychology in the mid-90s and I did exactly the same approach. You take the articles home, highlight them, write-up on a piece of paper what you want to say, and then book sometime in the computer room at the universities so you can come and actually type it up.
Drew: The reason I gave that version of what a journal actually is, is just to show how little sympathy I actually have for people who complain about not being able to find PDFs because it is so easy compared to the alternative. The other side of this is how stuff makes it into those periodicals.
The idea is that academic authors write up a paper and they submit it to one of these journals. A journal is just a part of a publisher that creates the periodical. These days, even though each journal works independently, almost all of them are owned by one of about three big publishers. If you hear people complaining about Elsevier—the one that gets most often complained about just because it's the biggest—it's just like a publishing house, and then under that, there are various brands.
Journal of Safety Science, Accident Analysis and Investigation, Reliability Engineering Systems Safety, Journal of Safety Research, they are all owned by Elsevier, but they operate quite independently. They each have their editorial board. You submit a paper to them. The editor-in-chief takes a quick look at it, chucks it out if he thinks it's not well-written, the English is just terrible, or it's a crackpot, and then he passes the rest on to the associate editors.
The associate editor then takes a much closer look at it. They read the paper carefully. They think about who would be good people to comment and criticize it. Very often, 90% of the time at that stage will send it back to the authors and say I'm sorry, this has got some problems that are just too big to send it out for peer review. The other 10% go out to experts who read and comment on them, and send comments back to the authors. The authors respond to those comments who then send them back in.
That's what peer reviewing is and that's what a paper is. It's something that has been through this process. The reason why it's not readily available is really because of these big publishing houses that own the journals. Who actually owns the paper once it has gone through that process is kind of a bit confusing. The authors have some rights over the paper and in particular, the authors have the right to share the version of it that they first submitted. They very often have the right to share later versions of this including the published version privately.
What they can't do is take the final published version and just put it up on a website for everyone else to access. Often when you find that there's a charge, that charge is being issued by the publisher of the journal who's trying to act as a gateway to get to the research.
David: Drew, now that all of these journals are published online, I'm not sure how many people anywhere in the world still receive hard copies of their journals each month or every six months like you said. We can now search for these journals online. We can search through either proper databases and I might get you to talk about those few databases or we can just search in our traditional search engines. There are a number of ways we can go and find these papers now.
Drew: Probably the laziest way to do it—by laziest, I mean this is the way most of us do it most of the time—is we use something called Google Scholar. Google Scholar is just instead of typing google.com you type in scholar.google.com. It works exactly the same as Google except that it's got a bit of a filter to only include things that look like they are journal papers. It doesn't do any actual reputability check. It doesn't do any check as to whether it's a proper academic paper, but it does generate results that look like academic papers, so everything you get probably has a PDF somewhere that you can download.
David: I pretty much exclusively use Google Scholar now, Drew. I did have a period of time where I didn't have access to university libraries or the library catalogs, but Google Scholar if you're looking at a particular topic, I find that you may not get the exact paper you want but you will be able to find PDF versions of papers that are on that topic if you're searching a fairly general topic, so you can start to hone your search that way. Then, we'll talk about where else you can go.
Drew, I suppose when we're looking at databases or at Google Scholar, do we just type in the general topics that we're trying to find? Is that the way you'd approach trying to search for an article that is going to be useful?
Drew: The two most common things that people come back with when they've come looking for a paper is they either say I've gone looking for this. I can't find anything on this topic, or they look for this and there's way too much, I can't find what I'm actually looking for. This is something Google is really good at general search inquiries but is really, really bad at for academic information. In that what it tends to prioritize is very seldom exactly what you're looking for. People who are used to looking for papers have come up with a number of standard tricks that you need to use in order to make sure that the things you're looking for appear near the front.
These are actually things that date back in the early days of search engines that Google made unnecessary. This is one reason why Google became so popular for search engines that you didn't have to know these tips and tricks, but you're still going to use them to make search engines work for you.
A couple of other simple ones. I'm just using things like direct quotes around whole phrases. If you search for safety culture into Google, it sees those two words safety and culture. It doesn't necessarily search for that term safety culture. Whereas if you put quotes around the term, then it looks specifically for that term. If you put a plus sign in front of a term, it says that it has to appear somewhere in the metadata. If you put a minus sign, it says we don't want that to appear.
One of the common things when searching for safety information is you have all these medical journal papers and you realize that actually we don't care about medical information. So you just put a little -medicine, -nursing, -drugs, then suddenly all those papers disappear from your search and the stuff you're looking for appears in front of you.
David: If I wanted to think about how might safety leadership impact safety culture, the best thing to start is I go to Google Scholar and I type “safety leadership” +“safety culture.” That’s going to pull papers for me that talk about safety leadership and safety culture.
Drew: While I'm talking, David, do you want to open up a window in the background and check that this one works for our listeners? But yeah, there's a big difference. In Google, I'm most tempted to type in ‘how does safety leadership influence safety culture,’ and that's unlikely to work in an academic search engine whereas putting “safety leadership” and “safety culture” will tend to help out.
The other thing to do is not to use search engines at all, but to use some other ways of trying to locate papers. David, you mentioned library catalogs, and a lot of library catalogs can be accessed by people who aren't actually members of the library. You need to be a member to get hold of the actual papers, but you can use them like a search engine.
They have some really useful tools built into them so you can very quickly (for example) filter on time and find stuff that's been published very recently. You can filter by academic field so you can just untick the box that's related to medicine, to architecture, or to anything that you're not interested in, and just leave in the fields with safety papers are likely to be located.
The other thing to do is—we'll get on to this a little bit when it comes to getting a hold of papers as well—rather than using searches to find interesting papers, to use networking tools to find a paper. A lot of the journal websites have got recommendation engines built-in. The same way if you've ever been looking for a job and you subscribed to a job search site and it regularly sends you job ads that you might be interested in. You can do the same thing with all the journal sites. You can say, please send me recommendations, and you tell the topics you're interested in and anytime something new gets published, usually you don't get bombarded. You get an email once a week that says here's a bunch of paper recently that you might be interested in.
David: The last way to search is that there are (I suppose) social and professional networking sites for research and researches. Sites such as ResearchGate and Mendeley where lots of academics create individual profiles and they publish either (as you said) drafts of their research that they've submitted to the journals. They might also list all the different publications they have made. That can enable you to contact them directly just like you might send someone on LinkedIn a message and you can find out how to get that paper that way.
Drew: I'm a member of ResearchGate. I don't tend to use Mendeley so much these days, but ResearchGate is particularly good for finding academics who do work that you're interested in. That's the quickest way to find stuff is know who tends to publish in this area, and then every time they put out something new have a look at it. Not only will that immediately might be of interest, but they're likely to have sighted other stuff or reference other stuff that is interesting.
David: Drew, we found our papers now whether we use just plain old Google, or Google Scholar, or database, gone directly to the journal. We've gone through one of these social networking tools. We've [...] out our topics, we found our papers, we've honed in on one, and we go, okay, great. Here's the paper I want to get my hands on. How do I do that if it’s paywalled?
Drew: I was very tempted to order this section from most legal to least legal ways of getting hold of papers. The first thing I just need to quickly explain is that the basic rule for what is legal and what is not is the author's own text until they submit it to a publisher. The version that they first submit to a publisher is called a preprint and the publisher has basically no rights over that preprint. The author is free to share it. They are free to put it up on their website. They'll very often put it up on sharing sites or on their university webpage.
If they haven't made it available, it's probably just because they haven't gone to the effort of making it available rather than being restricted. The version that the publisher actually prints out, that’s in that really nice printed journal-type format, that people might be used to seeing with the two columns, the big heading, the abstract in italics, that has been edited by the journal, so the journal has some rights over it.
Usually, after a couple of years, the author has the right to freely share it or to freely put it leased-up on archive signs, but during those two years, effectively that version is not free for open sharing, but it is free for private sharing for a number of different uses.
The big thing is that there is absolutely nothing wrong with the author directly sending you a copy of the paper and it is usually okay for any individual who has a copy of the paper also to share a copy of the paper with you. What's not okay is putting it up on a publically accessible website. The fact that you can't find a publicly accessible website does not mean that the paper isn't there ready to be shared with you.
David: I suppose, Drew, like you said there from the least legal to the most legal. We're not even going to talk about the ways, but there are sites that illegally distribute research and we're not just going to talk about those.
Drew: Yes. I'm not sure whether you are talking to the audience. They were talking to me, but David, I agree absolutely.
David: You're the one with the job in academia, Drew, so we don't talk about those, but there are plenty of ways to get it without even needing to do that anyway.
Drew: I think the bottom line is you don't even need to go to those sites in order to get a hold of it. This is my sort of order I use and it is very, very seldom that I will get to the end of this list and not have a copy of the paper.
The first thing is just to search Google for the exact title of the paper in quotes. You'll notice that you'll get lots of hits back and those hits are all from different places. One of those places will be the official journal and that place you will not be able to access the paper unless you pay for it. That will usually come up as the first link so you click on the first link, you see that the paper cost money. Don't stop there.
The next couple of places will just be weird indexing services that don't even have a copy of the paper. They'll just refer you back to the journal. But then after that, you start getting hits like the author’s homepage, the author’s home university, or a public archive that has a copy of the paper. Very often you're the fourth, fifth, or sixth hit in Google will be somewhere that has a free PDF of the paper. Just that quick search most of the time will find the PDF.
The second thing you can do is look up the author directly. The first one is to look up the title of the paper. The other one is to look up the author. If they're still working in academia, they will have a homepage at their university. They will have a page somewhere like ResearchGate with a list of their publications. Depending on how proud of their publications they are or the institution's policies, very often, a lot of those will be clickable links directly to a PDF of a paper. Much older academics—a lot of their old papers—don't do this and they're less tech-savvy, so they’re less likely to do it but most recently published research you can access like that.
David: I just did that, while you're talking, I just typed the title of one of our papers and I came up with seven hits. It's on ScienceDirect which is obviously the journal page and it's on ResearchGate. It's on the Griffith University Research Repository. It's also on PsycNet and you can just see on your search results on the right-hand side, the imaginary column where the PDF exists and where the PDF doesn't. Even though that’s still a paywall paper, I think there were at least two ways of getting it straight away.
Drew: Oh, brilliant. The next thing that might not appear obvious to a lot of people is just ask the author. Authors both have permission to share the paper and are very, very willing to give copies of the paper. It's not like a book where the book sells 10,000 copies and the author's not going to give out a free version of it.
We earn nothing from the papers. We don't get paid anytime you pay to download it and we love people reading it. There's lowkey ways to do this. On something like ResearchGate, there's a button which says you're basically asking for a copy which just sends a list off to the authors. Every couple of weeks, I log into my ResearchGate account and there'll be 30 people who have asked for PDFs. Even if it's something I'm not allowed to publically share, I can just privately send off a copy to each of those people.
The other thing is you just send an email. Academics get lots of emails so you want to be polite about it, but just, hey, I'm interested in your research. I've been trying to get a hold of a copy of this paper and I can't. Could you send me one? You'll usually get the paper back.
David: I've tried those ways and always had a good reception. You just need to go. If you know the university that the person's at, you can go to their staff site, you can get the person's work email, and you can just send them an email. Like you said, Drew, people are usually very happy that people are reading their work. It may take a week or two, but you'll get a copy of the paper.
Drew: Yeah, and depending on who it is. You'll get a copy of that paper plus they'll say, by the way you should read this, this, and this as well. You'll find your inbox filled with the reading list.
The final one is just to ask someone who's got access. Anyone who has a user account at a university, the university has subscription to a large number of journals. Not every university has exact access to the same journals, so academics will follow this process with each other. We’ll ask someone at another university, do you have access to this journal? But anyone who's got even just a student account at a university shares in the subscription.
If they log in while they're on campus, then they have free access to a lot of stuff which is paywalled when they're off-campus. Being on campus includes being virtually on campus through VPN and things like that. If all else fails, just ask someone else who might have access and you're likely to get access to a paper pretty readily.
David: That's good advice. Most people will know someone who's doing some kind of study at a university and yeah, make friends with them. If you get to the point, like you said Drew, to get to that stage, that might happen in one of ten papers you're looking for maybe.
Drew, we found the papers and people can get them through those different ways. You got a paper and the title might sound really interesting and good, but what's probably important now is to just talk through what makes a good paper. What makes good research and how can a person who gets hold of a paper go through it to decide whether or not it's useful or they should do anything about what it says?
Drew: We talk a lot on this podcast about specific types of research and how to evaluate that, but I thought it would be nice to have a bit of a general discussion of how you decide very quickly whether something is going to be worth your time or not. David, you do this as much as I do so I'm very interested in your rules of thumb.
One thing I've got into the habit of is every paper has an abstract. It's only 200–300 words, so you read that just to get a sense of what the paper is promising. You know what it promises. If it doesn't promise something useful, probably it's not helpful.
The second thing I do in safety is I flip straight to the method section. I don't do that to evaluate the method. I do that to see if there is a method. If there's no method section, then the paper is not actually a research paper. It's a literature review or it's a discussion paper. It's not going to have original results in.
That's absolutely fine if the title of the paper is, ‘A Literature Review of…’ and that's what I'm looking for. It's like a review of literature. But if it's not claiming to be a literature review and it doesn't have a method section, then it's unlikely to be useful. David, where do you look first?
David: I usually get a lot out of the abstract (I think) and sort of frame through what problem do I think the authors are trying to discuss or solve, and how clear is that problem because they should very early on the abstract say what they are trying to do. Usually, the abstract will then actually refer to a method or not.
I'll be interested if someone says we did a survey of 20 people, or someone said we did 87 interviews. I'll just form a view. Once I know the question they are trying to solve and how they went about trying to solve it, it will probably give me some pretty good insight into whether what they're going to conclude in the paper is something that's going to be interesting and maybe generalizable.
Drew: You mentioned a question there. I would love it if every paper very quickly in the introduction told you what their research question is. Unfortunately, that's not the case. Often you can start reading the introduction and you can get lost in the sea of background information, but if a paper is written well, then the introduction will make clear what the question is that they're trying to answer. If they don't, an easier thing to do is skip right to the conclusions and what they conclude because what they concluded then tells you what the question was.
For me, the introduction or the conclusions, I like to take a guess myself at how I would investigate that. If it claims to be making statements about what is the most risky thing to do in construction, then I would expect it to be doing some sort of analysis of accident statistics. If it's making a claim about leadership, I might expect it to be collecting data from leaders and from followers or something like that.
The next thing to do is just to compare what the method is with what the claims are or what the question is that they're answering, and just check that it's something that you're comfortable with. A lot of the papers that I dismissed are ones where they claimed to be making answers about risk, whereas actually all they've done is basically survey people of what they think the risk is.
David: Yeah, that's not always easy and intuitive, Drew, but as a rough rule of thumb, if the question is trying to say what is actually happening here or why is this happening then you need a research method that's not just superficial. Not just a bunch of surveys or things like that. It needs to be observational. It needs to be embedded (in my opinion anyway). I think if someone's just trying to say how many times does this happen or what do people think, then a survey is absolutely fine.
Drew: Yeah, that's exactly right. The other thing that I encourage people to think about when they're reading papers is the difference between what the authors say and what the research says. That's a mistake that you see a lot of people, particularly students or people who aren't familiar with academic research. They say this paper tells me something, but that hasn't come from the results of the paper. That just came from the introduction or the background to the paper. In other words, it's not original stuff coming from that.
One of the ones I notice all the time because David and I both work with Sidney Dekker and he gets cited a lot. You'll see someone make a bold claim about safety and then the citation is to Sidney Dekker. Immediately, alarm bells go off because Sidney is a theorist. He doesn't tend to do a lot of empirical research.
If someone is citing Sidney as if they're citing a fact, there's a mismatch there. If they're citing him to cite an idea, well that makes total sense. If they're saying, “Some people advocate strongly for restorative justice, cite Dekker,” you think, okay fine. Then they say, "Restorative justice is effective in increasing the organizational learning, cite Dekker,” you think, no. That's the difference between what the author says which is your opinions, ideas, statements, and what the research says, what is the actual data coming out of the investigation.
David: Yeah, and I'd be caught at even preparing for these podcast episodes a few times because I get my head spinning whenever there are four pages of tables, numbers, and statistics in these papers.
So I'll go to what the authors conclude and go, oh, Drew. This will be a really interesting thing to talk about. Here's the finding from this paper, and you'll come back and say, no, no, no. Look at these statistics. It's weak, if anything relationship, just because you can read the statistics better than me.
What you've said there, Drew, being what the author says and what the research actually says is it might take some time and you might need some help, but making sure that the author's conclusion in the paper does actually match what they find with their method.
Drew: The general rule of thumb is just read the paper as narrowly as possible. Authors like to spruce up their claims. They like to talk about broader implications, but it's best to just stick to assuming what the paper shows is what the paper shows.
David: I think that's been a good general discussion there about how to read papers, look for questions, look for the methods that they've used, look for who the authors are, how the claims in the paper match what the results from the research shows.
The question that people have is how do I apply this? Do these findings matter for me and my company? There's a bit of a topic in academic research about the generalizability of results. It's not in our notes here, but I thought it might be good if you gave a bit of a discussion about how you see the generalizability. If it's research in healthcare and I'm in the construction sector, what do I do with the findings from that research?
Drew: I'm not going to get too much into the technical side of it, but I need to introduce one technical term, which is the idea of validity. There are three main types of validity we talked about when we're talking about evaluating papers. Construct validity, internal validity, and external validity.
Construct validity is about when a paper is measuring something. Are they measuring it in the right way? That's the question of something’s making a claim about risk. Are they measuring risks or are they measuring perceptions of risk? Someone is making a claim about leadership, are they actually measuring leadership or are they measuring opinions about leadership? Someone's measuring culture. Are they actually measuring culture, or are they measuring climate organization structure, or something else? That's construct validity.
That one’s probably the hardest one for non-academics to worry about. A lot of the time you just need to trust that the peer reviewers have done their job and that the construct is reasonably reliable.
Internal validity is how much within the scope of the paper it has correctly answered the question. Someone is working out what is the safest way to build scaffolding in construction. Has the paper given a good answer to the safest way to build scaffolding in construction? Have they answered the narrow question well?
External validity is how much does that matter beyond exactly what they did on the paper. In the paper it would have been even more narrow than that. They would have been looking at a certain type of scaffolding in a certain type of construction in a certain company in a certain country. External validity is how much you can go beyond that. Can you go beyond that company? Can you go beyond that exact circumstance? Can you go beyond scaffolding? Can you go beyond construction?
The way usually to do that is to look at how much detail they've given you about the type of situation and can you match that to your own circumstances. If they've been vague about the type of company or vague about the type of work, it's really hard to know how similar yours is. Whereas if they've been really specific, then you can say I'm not in scaffolding and construction.
But the types of things that we're talking about without talking about small teams where the person who leads the team is an expert and the rest of the people are basically unskilled and following along although that's actually pretty similar to the type of work we're doing in quarrying where you've got hired hands but with an expert foreman. Because it's similar enough, probably what they say about leadership and safety rules matches as well.
David: I think the last bit you said there is something that every practitioner can kind of do. It's from that view just by having some thought around the context that the research was conducted within and how that relates to their own context. As you said, it might be very specific because you might be in the same industry and a very similar type of organization to your organization. Or it could be slightly more removed like you said, but it could be about pilots and cockpits, but like you said the context is, in some ways, quite similar to clinicians in an operating theater. Like we know that the transfer of research from different industries goes on quite a lot.
Drew: That's actually something that I got pulled up on by someone a little bit while ago and it sat with me for a while. We tend to make these broad assumptions about industries and the bit I got caught with is when we're talking about something like healthcare, there are very, very different environments. What goes on in a surgical theater might in fact be very, very similar to what goes on in a cockpit.
You could look at research from one, look at research from the other, and say there are all these learnings we can take from one to the other. But to make the claim that healthcare is like aviation is ridiculous because there are parts of healthcare that are nothing like surgery. What goes on in an emergency room is nothing like what happens in planned surgery. What goes on in a palliative care ward is nothing like either of them.
It's really about looking for exactly what they were looking at in this study and were they trying to solve a problem or trying to investigate the situation which is similar to what the problems we face and the situations we experience.
David: We talked about finding an individual paper, getting a hold of it, then reviewing it, but that individual paper will sit within a body of research literature. We know that straight away because all the papers will have a reference list and there might be somewhere between 20 and 200 references in the paper references.
If someone’s reviewed a paper and they’ve got some interesting ideas out of it, that they actually want to look a little bit broader at the body of literature. How might they start from that paper and then move out into the body of literature and find new papers?
Drew: That's a really good question. The way I like to think about this one is forwards, backwards, and sideways. When a paper has a set of references on it, if you follow those references, you're going backwards in time. These are all things that were published before that paper so you're looking at where the information originally came from. That's really important if you want to get to the bottom or origin of ideas.
The paper might mention Safety II. That doesn't mean that it’s the expert's source on Safety II. You look at the references and you find they've cited this book by Erik Hollnagel about Safety II, and you might want to go back to that original source to find out where the ideas come from and get their most original, most clear versions.
Forward citations are like traveling forward in time from the paper. This is looking for other people who have read the paper and then written follow-up stuff. Google Scholar is particularly good for this. If you look at Google Scholar, under any entry it's got cited by a link so that it is cited by tells you everyone else who, since that paper, has not only read it but has written something else that uses that as a foundation.
Particularly for people who are doing research themselves, this is really useful because you can find people who are doing very similar work as you are doing because they've been inspired by the same stuff. They've written follow-ups. You'd be much better to read and follow on the latest stuff than from the original stuff.
Then the idea of going sideways is just to realize that academics are pretty specialized. If they publish one paper about something and particularly if you find it good, if you find that they are readable, interesting, relevant, you think that they are reasonably reliable, then probably those are habits that they have. So, you go sideways. You look for the authors name and you type their name into the search engines or you click on their name in Google Scholar and find out what else they've written. It can be a very quick way of finding something that is more relevant to your precise question is find something that's roughly relevant to find out what else the authors have written.
That's actually a good rule of thumb. If you think it's relevant, you look at what else they've written and there's nothing else that's relevant. Probably they're not actually the expert you thought they were.
David: I like that and I actually used the forward citations quite a bit because it's quite common. Lots of papers are published every year in the field and you get something and all of a sudden I think we've done it a couple of times in this podcast. It might be eight or ten years old.
If you just stop there, I think particularly when we're looking at the VR Episode where we had some research from the early- or mid-2000s, then we picked up a systematic literature review from 2019 or something just by going forward and just by finding what's the latest stuff. I think that's really important.
We haven't gotten here, but I was thinking, people generally read books so I was still thinking of research. If someone wanted to find out about safety leadership and they type safety leadership into Google, how would you advise people about the difference between reading some academic papers on safety leadership versus just picking up a book from Amazon on safety leadership?
Drew: That's a tough question, David. I'm curious what you got in mind by asking the question. Is this an invitation to slag certain people off.
David: No, absolutely not. I suppose underneath one part of that is anyone can write a book and it's not subject to any peer review or any sort process of trying to determine how accurate the information is and how reliable the information might be.
I can write a book on any kind of subject I wanted tomorrow. My comment, I think it was a loaded question, but it was more about books that are easy to get your hands on. If you want to pay $40, you pay $40 and get a book as opposed to an individual paper and they're typically a bit easier to read whereas academic papers can be hard work, but there are compromises and trade-offs with the two different mediums.
Drew: There are a few different types of books and it's really hard to know which type of book it is until you've picked it up and read it. Often you can't even tell from the reviews. Some books are written by people who, I don't want to sound derogatory, but I think the most accurate description would be they are disciples of ideas in safety.
They have maybe an inherited knowledge that's coming from the academic research, but they're not deeply familiar with it and they're trying to take that knowledge and translate it for practitioners. Those sorts of books can be very helpful and often very useful because they're written by people who are on that boundary of. They’re interested enough that they probably read lots of stuff, but they are practical enough that they are trying to turn it into applied knowledge from experience.
I don't want to name names negatively. A good positive example would be the work of Tim Shaw. Very readable, not an academic himself, but understands the academic literature, does spare a bit of time to literature, and translates it into practice. Going a little bit more academic. Someone like Todd Conklin, has a PhD. Writes stuff which is mostly from a practitioner's perspective rather than a research perspective.
Those things are great for engaging in the space of ideas for how to apply them. They're like fellow travelers on the road with you as you try to apply research. You then have books that are written by academics, but they're not academic books.
David, can you think of any great examples there? I'd put some of Sidney Dekker stuff in that category. They're not written for an academic audience. They're written for a practitioner audience. The amount of research content in them is very, very low. They don't have any original research in the book. They might have as much research as one or two papers and lots of storytelling that wraps around that research.
David: I think Sidney’s writing style tends to be heavily referenced and most of his chapters in all of his books have a very strong reference list around them. I think someone like Erik Hollnagel’s work is more stories and ideas. Not story in a fictional sense but more narrating the literature as opposed to directly referencing the literature.
From the question point of view, I think it's just work because we're talking about accessing research and that research makes its way into some books and people can get their hands and people like getting their hands on books, audiobooks, and things like that. I think we just conclude that just be discerning around the idea as you take out any published materials just like you would in the tips we've given you about the academic papers.
Drew: I guess the big bit of a discernment I'd say is be very wary of treating second-hand ideas like they're first-hand ideas. If you want to know about Safety II, particularly if you want to be in a position where you can criticize it and point out the problems with it, then please don't do that unless you've read Erik Hollnagel’s book because everything else is a reinterpretation by other people of that work.
I probably did a bad example because a lot of the stuff that Erik has written is reinterpretations and variations on that original work, but you get the point.
David: Yeah, and I think it was episode 17 where I interviewed Carsten Busch who's done his master's thesis on the original work of Heinrich and gone back to a lot of the original source materials from the 20s through the 40s. I know that Carsten is just on the final stage of his book manuscript that explores the work of Heinrich much more deeply. I think if you go back to that episode, that's his big takeout, which is that a lot of the ways that work is being talked about in the last 30 years is inconsistent with the original ideas and the context of the original.
Drew: I think my favorite version of that is there's a particular paper that I wanted to get my hands on for years and every time I see someone else reference that paper, I think someone has got hold of a copy of it. I can ask them for it, and no one's ever got a copy of it.
This unfortunately happens both in academia and in book-writing. People don't actually go back to the original sources. They get a little bit lazy and what you end up is this mirror reflection of the ideas rather than the ideas themselves.
David: Yeah. Drew, we've been through finding an article, getting your hands on a copy, reviewing, interpreting it, how you think it might apply, where the research sits within the broader context of other types of information mediums. You've written a few practical takeaways here, so I might let you go through these because we always finish our search of practical takeaways and this one is no different, but what's your advice for people in this space?
Drew: Three, I hope are simple and clear takeaways. The first one is if you're a practitioner, probably, if you're a young researcher, definitely, get into a habit of going back to the original sources. Most arguments in safety, most of the disagreements we have are because people have a twisted second-hand version of the research.
David, I know you've got into arguments on LinkedIn. I have, too, with people who you realize after you’ve been arguing for a while, not only have they not read the stuff, they're refusing to read the stuff. You just think, what is the point? You don't actually know what someone has said and yet you're criticizing their work or you're defending their work. Just get into a habit of going back and reading the original sources.
The second takeaway is about how easy that is. Most papers, certainly recent ones, are available for free. Never pay to get hold of a paper and don't give up after a very quick search thinking this isn't available. I do a lot of my writing and certainly all my podcasting sitting at my desk at home. I don't use my university access to get hold of papers. I very, very rarely need to fire up the VPN and pretend I'm at the university to get hold of a PDF because they are all just publically available on the web.
The third one is if you're not used to reading academic research, I think you'd be surprised at just how accessible most of the papers are. Most academic research papers have lots of ideas and background information in and around the topic. Even before you get to the research, they often give you a really good summary of what's been written in a field and a good understanding of that field.
Most of the one that get published are actually reasonably well-written. They are more likely to have errors of English and grammar than they are highfalutin academic language that's making them unreadable.
David: I'd say the safety science discipline, but in some of the institutional work, institutional logics, and organizational study stuff, I didn't understand most of the works that were in the paper, but you're right. In the safety science space most papers are very readable.
Drew: Administrative science and social science does actually get really unreadable. In that case, the books are actually even worse, but safety science definitely usually readable. A lot of organizational stuff that doesn't go down these institutional theory-type line is very, very readable.
David: Drew, do you have some questions for our listeners?
Drew: We've never done a poll of our listeners, so I'm interested how many people actually read papers. How many of you listened to the episode and go read the paper that we've talked about? How many of you trust our summaries of it? And if you don't tend to read papers or you don't read papers as much as you'd like to, what are the big disincentive or obstacles that get in the way? Certainly as an academic, I'd like to know what those disincentives are so we can do things about them.
David: Drew, this week, we asked the question, how can practitioners find research? I think the answer is start with Google. Move forward from there to contacting the author or just phoning a friend, and 99 times out of 100 you should be able to get a copy of the paper you want without having to pay for it. Drew, we've also tried to help our listeners with once they get their hands on that paper, how to know if it's any good or not.
That's it for this week. We hope you found this episode thought-provoking and ultimately useful in shaping the safety of work in your own organization. Please send us any comments, questions, reviews, ideas for future episodes to us at feedback@safetyofwork.com.