This episode examines research on electronic team monitoring AI systems and their psychological impact on workers. David Provan and Drew Rae discuss a July 2025 study from Cognition, Technology and Work by German researchers exploring how perceived invasiveness of AI monitoring affects team members' stress levels and acceptance. The research included two experimental studies testing different levels of monitoring invasiveness and purposes, from basic email counting to biometric stress monitoring.
The findings reveal that invasiveness drives negative reactions more than the stated purpose of monitoring, with participants showing skepticism about AI's ability to accurately measure teamwork quality. The hosts emphasize that even well-intentioned monitoring systems introduce psychosocial hazards and stress, requiring organizations to carefully balance potential benefits against worker well-being impacts when implementing AI-powered team support systems.
Discussion Points:
Quotes:
Drew Rae: "The moment you divide it up and you just try to analyze the human behavior or analyze the automation, you lose the understanding of where the safety is coming from and what's necessary for it to be safe."
David Provan: "We actually don't think about that automation in the context of the overall system and all of the interfaces and everything like that. So we, we look at AI as AI and, you know, deploying. Introducing ai, but we don't do any kind of comprehensive analysis of, you know, what's gonna be all of the flow on implications and interfaces and potentially unintended consequences or the system, not necessarily just the technology or automation itself."
Drew Rae: People are going to have reactions. And those reactions are gonna have a big impact on their willingness for you to do this in the first place...You can't just force it onto them… All the things that you're trying to improve might actually get worse because of the monitoring.
David Provan: "But I think this paper makes a really good argument, which is actually our automated system should be far more flexible than that. So I might be able to adjust, you know, it's functioning. If I know, if I, if I know enough about how it's functioning and why it's functioning, and I realize that the automation can't understand context and situation, then I should be able to make adjustments."
Drew Rae: Most people don't mind if their car is giving them feedback on their driving, but most people don't like it if their car is phoning home to your boss, giving information about your driving.
Resources:
[00:00:00] David Provan - cohost: You are listening to the Safety Work Podcast, episode 1 32. Today we're asking the question, how much should we worry about the invasiveness of team support ai? Let's get started.
[00:00:27] David Provan - cohost: Hey everybody. My name's David Provin, and I'm here with Drew Ray and we are from the Safety Science Innovation Lab at Griffith University in Australia. Welcome to the Safety of Work Podcast. In each episode, we ask an important question in relation to the safety of work or the work of safety, and we examine the evidence surrounding it.
[00:00:43] David Provan - cohost: So Drew, we're continuing our conversation from last week on, uh, AI and automation. So would you like to sort of give a little bit of background to today's episode?
[00:00:52] Drew Rae - cohost: Sure. So last week we talked pretty generally about the relationship between automation and AI and safety, but that leaves a lot of specifics that I thought it was worth getting into.
[00:01:05] Drew Rae - cohost: One of them is this increasing use of AI as workplace monitoring. And there's a few different ways that people do it. Like some people use it with cameras to monitor hazards. But one of the things that's being particularly popular is using it to monitor interactions. So it's happening in a few different ways.
[00:01:26] Drew Rae - cohost: Some of it is things like replacing risk assessment, take five style things with conversations, and then monitoring the quality of those conversations and give feedback. Another one is monitoring teamwork in situations like. Your military teams or surgery teams are in a cockpit and giving feedback, all sorts of monitoring, the tone of voice, the language that's used, how people are interacting.
[00:01:52] Drew Rae - cohost: Even a lot of looking at emails and who people are emailing and the tone of their emails, things like that. Since people are using it, it's something that. Is interesting to sort of think about, okay, so what are the effects when we do that sort of monitoring?
[00:02:05] David Provan - cohost: I think particularly Drew, like we've done employee and team monitoring systems for a long time.
[00:02:11] David Provan - cohost: If we think cockpit voice recorders in aviation or in vehicle monitoring systems in, in heavy vehicles. But now, like you said, with ai. The range of data, you know, mood, sentiment, stress, like a, a whole bunch of sort of more personal data is available to be monitored quite excessively by organizations. So I think it's important that we think about the, I guess, positives and negatives associated with that sort of monitoring.
[00:02:37] Drew Rae - cohost: And then there, there are some genuine positives to this sort of, you know, it's not just workplace surveillance. Often it's very well intentioned to give people feedback to make them perform better. So it's intended as performance enhancing or safety enhancing or improving the effectiveness of training, improving the effectiveness of risk assessments so people see the opportunity for doing it.
[00:03:00] Drew Rae - cohost: But there, there's also some natural skepticism.
[00:03:03] David Provan - cohost: And, and so Drew, um, should we start by introducing the paper and talk a little bit about, uh, I guess making our way through, because we, I'm not sure, it's probably been a few episodes since we've done a more experimental research design. So should we start with the paper?
[00:03:16] Drew Rae - cohost: Okay. So this is published very recently, July, 2025, in the journal Cognition, technology and Work, which is kind of one of the big four safety journals. Uh, the title is Intelligent Automated Systems to support human Teamwork. Perceived Invasiveness, impacts team members', psychological reactions Negatively.
[00:03:39] Drew Rae - cohost: So it's kind of putting the conclusions into the paper. Uh, the authors are not people that we're familiar with. Uh, Greta Entra, Paula Volkman, Olga, all of Annette Kluge, all from, um, Germany. A couple of, um. Associated at universities and research centers. In Baum, Germany and in the research center for trustworthy data science and security.
[00:04:08] David Provan - cohost: And then another, um, university in sen. So, um, not safety researchers, but I guess, um, we're, we're talking about a little bit the safety implications of of, of more technology applications. So,
[00:04:19] Drew Rae - cohost: yeah. Yeah. I haven't done big, deep background on the researchers, but they're all sort of in this experimental psychology of space, like seeing how people.
[00:04:27] Drew Rae - cohost: React to technology doing very much in this experimental mold.
[00:04:32] David Provan - cohost: So we can talk through, um, drew a little bit of the background and underpinning literature, um, that sort of prompted this study, and then we'll talk methods and, and findings and practical takeaways. So should we start with, I guess, a little bit of background into sort of electronic team monitoring systems and some previous research?
[00:04:48] Drew Rae - cohost: Yeah, so, so I mean that they indicate firstly just the obvious that. When you digitalize processes or you bring technology into the workplace, it does have an impact. On our work happens, but als and there's increasing use of, we are doing work that almost automatically creates digital signals. Uh, so like a easy example is we're using email that creates a digital signal signal.
[00:05:15] Drew Rae - cohost: Or you can introduce ways of creating digital signals by recording things or videoing them. So we are creating this data. There's these opportunities to use it for organizational goals, and in this case, the goal is to monitor and improve teamwork by, essentially the main mechanism we do that is it provides feedback, it tells you what you're doing well, tells you what you're not doing well, and that helps you then improve your work in a team.
[00:05:45] Drew Rae - cohost: But people are going to have reactions. And those reactions are gonna have a big impact on their willingness for you to do this in the first place. Because you're of these things really do require the cooperation of employees. You can't just force it onto them. But also, if they've got negative sentiment that's gonna hurt the teamwork, it's gonna hurt their stress levels.
[00:06:07] Drew Rae - cohost: All the things that you're trying to improve might actually get worse because of the monitoring. Um, and the two main things that they point out are just decreased satisfaction so people don't like it and stress. People are aware that they're being watched and that drives up there. Um, levels of discomfort, their workload.
[00:06:24] David Provan - cohost: And I think the, the, the other point to that, drew is also the focus of attention as well. When people are being monitored, and I mean, this is sort of cutting to the end a little bit, but, you know, are very focused on the parameters that are being monitored, associated with the task. And that may conflict with, um, good quality task performance of what the, the work that they're actually doing.
[00:06:42] David Provan - cohost: So. There's sort of work implications, personal psychosocial implications, and there's obviously an intended positive benefit from these technologies as well. So it's um, a few trade offs there.
[00:06:52] Drew Rae - cohost: So, so they do point to like, what do we already know about what tends to affect people is these, these are fairly obvious, but I think they're worth reminding.
[00:07:01] Drew Rae - cohost: So the first one is just if it is team related rather than individual related. People tend to be more happy with being monitored. Because this is very clearly about the collective endeavor. People don't like very individualized monitoring. And then if it is clearly about feedback rather than evaluation, people uh, tend to be less stressed and more happy with technology.
[00:07:25] Drew Rae - cohost: So like a classic example here is in-vehicle monitoring. Most people don't mind if their car is giving them feedback on their driving, but most people don't like it if their car is phoning home to your boss, giving information about your driving. You, those are two quite different purposes,
[00:07:41] David Provan - cohost: and I think through terms of the background to this study, you sort of suggest that a lot of the research over the last decade or so has been focused on that individual, um, electronic individual monitoring.
[00:07:51] David Provan - cohost: So, um, some of those applications that we've spoken about, about the individual and with the increasing use of sort of team monitoring ai. We're sort of tentatively questioning whether or not those same findings for individual monitoring would apply to, to team monitoring. And they sort of suggested that there was, you know, much more limited research in, in team monitoring systems and the effects.
[00:08:14] Drew Rae - cohost: Yes, so, so this isn't a big study and it's got a lot of limitations, but what they're trying to do is just gently push the research out of a space that we. Have a fair bit of knowledge about individuals, move it into that team space, get some tentative findings that they can then use as a basis for much more robust field trials.
[00:08:35] Drew Rae - cohost: And that's the way research often progresses is we start off doing this with fairly token things in the lab, gives us tentative findings that hopefully tell us what is worth spending the extra money. For the proper field research.
[00:08:46] David Provan - cohost: So we ready? Talk about the aim and then the method.
[00:08:49] Drew Rae - cohost: Sure. So the AIM's pretty straightforward.
[00:08:51] Drew Rae - cohost: It's to use electronic team monitoring and to test out these things that we think. We know that as it gets more invasive, people do get more stressed, and that if the purpose is for giving feedback, then people will be a lot more accepting than if the purpose is. To sort of manage and monitor performance.
[00:09:14] David Provan - cohost: So there's two, two studies, um, included in the paper, and we'll talk through each of those in the findings, um, of each and limitations of each of those. And both of those studies involve, like you said, and Drew exposing people to, uh, um, either electronic team monitoring situation, um, not a personal experience, but just, um, just driving a situation.
[00:09:34] David Provan - cohost: And the second study was about then putting them in a electronic team monitoring simulated experience. And in doing both of those things, they're trying to investigate the psychological reactions with a post experience questionnaire. So an online survey that would then ask them about their psychological reactions to, um, what they've just done.
[00:09:53] David Provan - cohost: So
[00:09:54] Drew Rae - cohost: the first study has 156 participants, which is a reasonable number for this. Sort of thing. And they're basically just giving each participant one of three videos that's going to explain to them a workplace monitoring system. So they're not actually experiencing the monitoring, they're having the system explain to it and they're reacting.
[00:10:15] Drew Rae - cohost: And that's fairly realistic. 'cause like that's the way it would start off in a workplace is you'd probably have to do some sort of online training where they explain what's gonna happen. So this is like your first test of how will people react? And the three systems that are explained are different levels of invasiveness.
[00:10:30] Drew Rae - cohost: So the low invasive group, it's gonna monitor the number of emails and phone calls that you make. The medium invasiveness is going to track the content and do some sort of like motivation and mood analysis. Of your emails and phone calls and the highly invasive is gonna wear a digital wristwatch that monitors your location and heart rate.
[00:10:54] David Provan - cohost: So what they did there, drew, is they watched a, a short video where it was a, uh, a HR manager saying, this is you and your team, and this is the, the technology, and this is what it's going to do. And so they're asking people to put themself in the situation of, if, if, if your organization was now, um, like you said, implementing this and you worked in that organization in that role, how would you feel about it?
[00:11:21] Drew Rae - cohost: David? I'm not a huge fan of some of the choices they made here in the design in particular. One of the things that they did, which is exactly the right thing to do, is before they rolled this out to all of the participants, they tested the material just to see whether it was. You're doing what they thought it was doing, and they discovered that their original, highly invasive version, no one really considered to be more invasive than the other two.
[00:11:46] Drew Rae - cohost: So they, in response to that, like early trial, they had to change it, and that's what led them to this digital wristwatch. But the, the trouble is that now you've got. Two team-based interventions and something that is really very individual, which is monitoring your stress using a wearable device, which kind of compromises their original goals of extending this mostly to team situations.
[00:12:10] David Provan - cohost: Yeah, they've manipulated, um, they've got a few confounding variables, so they haven't really been able to isolate out an account for, so, you know, and they, and they say that at the end of the paper and, and in even the second study, there's some. You know, similar types of limitations where they've moved around a few too many things.
[00:12:25] David Provan - cohost: Uh, and I think in some of these early studies when we're trying to explore a field, I don't think we want to have, you know, these, I don't think we wanna manipulate these, this number of things.
[00:12:34] Drew Rae - cohost: Yeah. And you don't wanna judge too harshly. 'cause like, the part of the whole point of doing these things is you don't know in advance what's the best way to research it.
[00:12:43] Drew Rae - cohost: So it's really hard to like, make all of the decisions perfectly on the fly. And it's really easy in hindsight to judge it and say like, why didn't you do it this other way instead? But there are some like fairly basic things like I wouldn't have given this to three separate groups. I would've given all 153 participants each of the three videos and that in a random order.
[00:13:04] Drew Rae - cohost: And that way, at least, your different groups don't have different participants in. So there's always the risk that because the groups are different, any results you see just could be, they've got different people with different reactions. And so the more you can control those sorts of things, the cleaner your results are.
[00:13:19] David Provan - cohost: Yeah. For something like invasiveness, like it's, it's not immediately obvious how to do that. And I probably would've gone for like two conditions rather than three and made them a little bit more extreme. Like the first condition being, okay, it's gonna count how many phone calls you make, how many emails you make, and, and to which members of the team or something like that.
[00:13:37] David Provan - cohost: From a teamwork point of view. And then the other invasive one is gonna be really extreme. It's gonna, you know, consider. The maturity of the language that you are using. It's gonna do your mood and your sentiment and emotional responses and a whole bunch of kind of, um, individual team reaction type interaction things.
[00:13:56] David Provan - cohost: So you can try and paint a picture of something that is not really that invasive, but something that is. You know, I think to the average person, or to most people, something that would feel very, very invasive, because I think that's where AI is right now, is it has the potential to be very, to, I guess, monitor quite invasive data.
[00:14:15] Drew Rae - cohost: Yeah. Should we talk about the second study now and then we'll talk about the results?
[00:14:19] David Provan - cohost: Yeah. Yeah. So, so the second study they, they actually created a, a, I'm gonna call it a simulation. Would you call it a simulation? Simulation.
[00:14:27] Drew Rae - cohost: Yeah, they, they're, they're actually doing a task and they think they're being monitored while they're doing the task.
[00:14:33] David Provan - cohost: So this research was done during COVID, so it was over Zoom virtually. Um, they had 20 participants, um, which as far as I can work out, were in teams of two. So let's assume we've got 10 teams of two. They were giving a given a ta, they were recruited, um, via LinkedIn. It was a convenience sample. They just went out and got participants.
[00:14:52] David Provan - cohost: Um, and that's important I think a little bit later as well. But they gave them a task to work together to design a new fitness tracker for a technology company. So they were given a task and randomly split into two, I guess, different simulations. Um, the first was given that task and they were told that there was going to, that their teamwork was going to be monitored for the purpose of development.
[00:15:17] David Provan - cohost: So for the purpose of giving them feedback on, you know, how they work together as a team and, and what they could do to. Improve their, their teamwork. And they were just paid a flat fee of four 40 euros, I think, to participate in the, in the study. And they were even showing fake graphs and, and feedback and what that monitoring was gonna look like and, and everything like that.
[00:15:38] David Provan - cohost: Then the second group were told, um, that they were also going to be monitored. They had some more invasive, uh, monitoring going on, like eye tracking technology and facial recognition technology. So they increased the scale of the monitoring going on. And they also told that group that the purpose of the monitoring was to evaluate their performance.
[00:15:57] David Provan - cohost: So how Well they did that task as a team and they were told that they would be compensated, um, I guess, um, not given 40 euros, but compensated based on the performance of, of, of their task, you know, as determined by the electronic monitoring system.
[00:16:13] Drew Rae - cohost: Yeah. They, they're actually told that there were a whole heap of other teams that they would be ranked.
[00:16:17] Drew Rae - cohost: And that they would get compensated based on their performance relative to the other teams
[00:16:21] David Provan - cohost: and their performance as determined by the monitoring system.
[00:16:24] Drew Rae - cohost: So hopefully that gives a fairly good indication of the type of thing that we are doing here. So these are in lab experiments, um, because it's during COVID in lab, it means they're sitting online at their computers doing these tasks, but they're designed to gauge how this would work if it was a real system being rolled out.
[00:16:43] Drew Rae - cohost: You obviously, it's a little bit unethical to actually roll out a system that might cause. Stress to your participants just to see whether the intervention would cause stress to the participants. So you've gotta do this as a simulation to start with.
[00:16:54] David Provan - cohost: I think also Drew in terms of like the informed consent and that it's also important that you don't, and, and, and it is quite ethical in, in, in researching and you have to just to get, um, not to confound your results, but people didn't really know what this, what the researchers were doing.
[00:17:09] David Provan - cohost: You know, they were, they were given their example and they were actually told a cover story about, you know, what they were, you know, so the, the cover story for re study number two was that they were working on an interdisciplinary project to try to design some electronic team monitoring software, and that they were testing that software, um, as part of this experiment.
[00:17:28] David Provan - cohost: Um, so they weren't really told what the researchers were doing.
[00:17:32] Drew Rae - cohost: Yeah, we would, it, it is necessary. It's enough for informed consent. Just not enough information that the people know exactly what's being measured and why. So they're not gonna skew the results to deliberately make them look good or bad. Uh, should we move onto the findings?
[00:17:46] David Provan - cohost: Yeah. Do you wanna start us with study one?
[00:17:48] Drew Rae - cohost: Okay, so, so study one, remember they've got these three conditions, the very low invasive, medium invasive, high invasive. They basically find that there's no real difference between the low and medium conditions, which I actually found fairly interesting given that the low one is counting emails.
[00:18:06] Drew Rae - cohost: The medium one is quite intrusive. It's like monitoring the language of emails, and people didn't make a lot of distinction between those two. But they did find that the highly invasive one where they would be wearing a wristwatch, stress sensor was significantly more stressful, uh, was seen as less fair, and they would be less likely to accept it.
[00:18:28] Drew Rae - cohost: If that was proposed in their workplace.
[00:18:31] David Provan - cohost: Yeah. They actually asked, as they asked these questions about how invasive and how fair do you think this technology is, how stre, how stre, how, what level of stress does it introduce to you? And, and so on. But then they also asked these two questions about, you know, would you download the monitoring app and would you turn it.
[00:18:48] David Provan - cohost: And so, you know, as if people were given the choice whether to turn it on or not. And as you just mentioned, drew predictably that the one, the highly invasive system, significantly less participants said, yes, I would, I would turn on that monitoring system.
[00:19:02] Drew Rae - cohost: There. There is just one little tweaky that I find interesting is they predicted that, and this is based on some of the previous research, that the more invasive something is.
[00:19:14] Drew Rae - cohost: Possibly the more fair the evaluation is seen as because it's collecting more data and that eval data is being evaluated more rigorously, whereas in fact, it was the opposite. The higher invaders, everyone was seen as a less fair way of measuring performance. Um, and we're gonna see this in the second study in some of the qualitative feedback they got, was that their participants were really skeptical that exactly what was being measured was the right thing to measure.
[00:19:43] Drew Rae - cohost: So, you know, in this first study, people didn't think that measuring your stress was a good measure of your communication performance. 'cause the other ones are about reading like tone of email and so people are sort of like really worried that the electronic measuring doesn't quite match actually what's being tested for.
[00:20:01] David Provan - cohost: Yeah, I think Drew that surprised me a little bit as well. 'cause um, you know, we all know I've sort of. Individual and team monitoring things like, you know, if you're working a call center, you know, you, you're doing a call every six minutes and, and you know, like this is what this low invasive test was, but we're just gonna, we're gonna track team performance through the number of tasks performed and, you know, basically the frequency of tasks before.
[00:20:23] David Provan - cohost: And that's not perceived as fair. 'cause people say, look, you know, sometimes I need to make a 25 minute phone call to satisfy a customer. And, you know, so that's where this fairness, um, thing comes in. And, you know, I would've thought that the more data that. I think the research has hypothesized that the more data that we get through more invasive data collection, the more of a overarching representation of the work we're going to have.
[00:20:45] David Provan - cohost: And the more that, you know, people who are being monitored will feel yes. I feel like, you know, um, the organization's getting the true picture of work performance. Um, but yeah, I was surprised that, you know, keep collecting data more and more data and people I think, still feel like the organization doesn't know.
[00:21:01] David Provan - cohost: Understand, understand the work.
[00:21:02] Drew Rae - cohost: Mm-hmm. But as, as we get onto the second study where they're actually doing the task and being measured, the qualitative feedback shows that the people are trying to guess and game exactly what is being measured. They're like asking each other, you know, is this measuring the volume of how much we speak?
[00:21:19] Drew Rae - cohost: Is that how well we're communicating? Or is it measuring the tone of our voice? Or who's speaking? Or the balance. And their uncertainty about exactly what's being measured and evaluated is causing them to get stressed and. To be really skeptical about the evaluation. So I, yeah, I think that might be like a general takeaway here is that particularly when we have these systems that you were talking last week about the lack of visibility of what the AI is thinking becomes even super important If we are doing something like getting chat GPT to evaluate the tone of a conversation.
[00:21:52] Drew Rae - cohost: How do we know how it's making that score? There's no transparency. Is it looking for particular words? Is it looking for particular phrases? It's, and so people are going to either game it or they're just gonna think that it's unfair that it's not. Actually matching reality. It's using some sort of hidden scoring system.
[00:22:08] David Provan - cohost: So should we move to study two a little bit more detail?
[00:22:12] Drew Rae - cohost: Yeah. So, so study two, remember, is where they're got two groups. One of them thinks that they're being given feedback. The other one thinks that they're being given a score that will be compared to other teams. One of the things you've gotta check in this sort of experiment is do they actually believe they're being monitored?
[00:22:31] Drew Rae - cohost: Because you, the whole experiment's gonna fail if they don't believe that they're actually, if they think it's fake. And it is just to be clear, in this case, it is fake. They're not actually being scored and monitored. And they found that most of the participants probably believed that it was real.
[00:22:45] David Provan - cohost: Yeah.
[00:22:45] David Provan - cohost: And, and they, they did that in a couple of ways. One is they actually ran the simulation, like I mentioned a little bit before, where they actually showed them, you know, their. Fake performance feedback, graphs and you know, all the different colors and dashboards and dials and what the system was, you know, was, was apparently generating to them even though it was all, all mockups.
[00:23:05] David Provan - cohost: Um, and they also asked 'em the question, the survey about, you know, did, were you aware of the monitoring during the task or you know, was it in your mind when you were performing the. So they, they asked them that as the, at the start of the questionnaire just to sort of confirm as well, um, that people were saying, yes, I was aware of that I was being monitored.
[00:23:21] Drew Rae - cohost: So, so the, the first sort of finding here is that both groups perceived it as really quite invasive now as the authors talk about in their limitations. This is always gonna be a little bit higher in a lab experiment that you know, you are doing an experiment, you're doing this task once you are being surveilled, while you do these tasks.
[00:23:41] Drew Rae - cohost: Once you know you're part of an experiment, all of those are gonna exaggerate feelings of being surveilled. You. Maybe if this is out on a work site, and this is the 300th conversation that the monitor's been turned. On for, you'll forget that the monitor's there, but at least for the purpose of the experiment, the purpose didn't really matter or what data was actually collected being mattered.
[00:24:01] Drew Rae - cohost: People were just very aware that they were talking to the recorder and not just talking to each other.
[00:24:07] David Provan - cohost: Yeah. I think you, just bearing in mind, we're talking about 20 people in, in a single point in time experiment. You know, some, the, some of these statistical, some of these are not statistically significant, some of these differences that we're talking to between the two different conditions.
[00:24:22] Drew Rae - cohost: Yeah. And, and so I, I think that first finding that both groups were very aware that is, doesn't require a statistical significance test. But the next one, which is there was no real difference between the two groups. And this is where statistical significance matters because they didn't actually have enough participants.
[00:24:41] Drew Rae - cohost: To detect a difference, even if a difference existed, and this is a really common mistake people make and it is a mistake, but it's a common mistake people make in these small trial experiments is they say, look, we didn't find any difference between the two groups, as if that's an interesting finding, but they couldn't possibly have found a difference because they just didn't have enough people to find a difference.
[00:25:03] Drew Rae - cohost: Even if there was a reasonable difference, they wouldn't have found it. So yeah, that's a case where the small size of the experiment just lets them down. So I think there are, they do acknowledge their own limitations and there are interesting things we can say about it, but it's not a reliable conclusion to say, oh, it doesn't matter whether it's there for feedback or doesn't matter when there's there for performance.
[00:25:23] Drew Rae - cohost: Monitoring.
[00:25:24] David Provan - cohost: So with that as a caveat, I think, um, there was a few things that I think the researchers were a little bit surprised in and, and, and, you know, different to what they'd hypothesized. So, you know, we say that they're both procedures in invasive, you know, to some extent they're saying that the, the electronic team monitoring software for the purpose of development, which is to give them feedback on their team performance so that they can improve.
[00:25:47] David Provan - cohost: Was sort of accepted lower than the actual performance evaluation. It was kind of as if like, you know, teams felt that it was fine for the organization to evaluate performance, um, but maybe not as fine for them to evaluate their teamwork on how they did the task, which is kind of interesting.
[00:26:03] Drew Rae - cohost: Yeah, and, and particularly I think this is something that is likely to translate to if we try to do this in the real world.
[00:26:10] Drew Rae - cohost: Is teams are complex and people are quite skeptical about the ability of an automated system to monitor the things that really matter for teamwork. And there is some other evidence that bears this out that, you know, we try using things like tone of voice and like tone of voice can differ based on whether you are a man or a woman.
[00:26:29] Drew Rae - cohost: It can dis for whether you're neurodiverse different races and different accents show up as higher levels of stress. People are re right to be skeptical that you can actually like detect their mood from their tone of voice. And it gets even more complex if you don't wanna detect how well a team is communicating based on how often people are talking.
[00:26:47] Drew Rae - cohost: You think how much is missing there? How much body language, how much a team can be communicating without saying much. How much a team can be saying a lot and not actually communicating. You know, the, these things are never gonna be great measures of what you are actually caring about.
[00:27:03] David Provan - cohost: So following the study, sort of Risha sort of talked about two really important technological design characteristics for electronic monitoring being purpose and invasiveness.
[00:27:12] David Provan - cohost: So this idea of, you know, for what purpose are we doing the monitoring and how invasive is it for the individual or the team being monitored? And so they're saying that, you know, through both of these studies, that was really what they were trying to understand by having these different purposes, these different levels of invasiveness, and try to understand the sort of psychological.
[00:27:32] David Provan - cohost: To, um, purpose and invasiveness, and I guess they concluded that they found that regardless of purpose systems are seen to be invasive. Like if, if they're invasive, they're invasive. And it's almost as if, like, even if it is got different purposes, I, I think the research has sort of concluded, at least from, from their work that maybe.
[00:27:53] David Provan - cohost: It doesn't get over the, the psychological stress, even if it's a very well intended system.
[00:28:00] Drew Rae - cohost: Yeah. This, this is the, like as an empirical finding, this is the one that's weakest based on their design, but I think the point that they're making there is actually just so plausible that it's worth making. That, you know, you can't say, oh, we are doing it for feedback.
[00:28:16] Drew Rae - cohost: And have people even trust that claim. You know, once you are collecting data, it is actually the method of collecting data and the capacity for different uses of it that are likely to drive acceptance. And you know, it was very clear in their first study that people thought that this collecting of personal biometric data.
[00:28:33] Drew Rae - cohost: Regardless of its purpose, significantly reduced their acceptance, increased their feeling of, um, stress around it, presumably because they're just suspicious of. What's this data actually gonna be used for?
[00:28:45] David Provan - cohost: And I think that's where, um, particularly in organizations, there's a lot of factors at play, variables at play here in regards to leadership and culture and trust and, and all of these things as well.
[00:28:56] David Provan - cohost: So, you know, it's just that idea that even if we go to our organization, say, Hey, we're going to put in this. Fabulous AI risk assessment facilitation software that's going to, um, as teams are talking in a risk assessment conversation, it's going to prompt them and facilitate and, and identify people who aren't, you know, contributing as much and understand mood, sentiment, and do all these things for the purpose of, you know, doing a better risk assessment.
[00:29:23] David Provan - cohost: Even if we have very clear, positive reasons that won't necessarily negate. The feeling of invasiveness and stress that the monitoring creates.
[00:29:32] Drew Rae - cohost: Yeah, I, I was having a conversation actually this week, David, I, I can't, can't attribute it 'cause it was just a background conversation, but it was about, uh, drug policies and the fact that it would probably be a lot fairer and much more aligned with organizational objectives.
[00:29:50] Drew Rae - cohost: To measure impairment rather than to do drug tests. But unions are very skeptical and pushing back at the idea of the farer system simply because they have a, well, you know, probably well justified from experience. It's just an expansion of the powers of the employer. You are giving them one more surveillance tool.
[00:30:12] Drew Rae - cohost: You know that people are innately skeptical of expanding the number of things that might be tested or the number, number of things that might be measured, even if it would result in something that was, you know, much fairer than just, you know, saliva or urine drug testing, which is pretty invasive.
[00:30:28] David Provan - cohost: Absolutely. And so Drew, should we just recap on a few of the limitations just before we talk? Practical takeaways?
[00:30:34] Drew Rae - cohost: Okay. So, so the biggest one is when you are doing these as an experiment rather than for real. You're never going to quite get people's real attitudes. These, these, these sorts of studies should not be adopted immediately by workplaces.
[00:30:50] Drew Rae - cohost: You've got to use these as suggestions for what it is you would measure in a workplace. It's a step along the research. We shouldn't treat these as final results. Uh, the second one is that be because we've got such a small second study that is trying to like simultaneously look at purpose and invasiveness.
[00:31:09] Drew Rae - cohost: These are really two separate things that ideally we would be separating out and getting people's attitudes to invasiveness using one study attitude to purpose using another study. They're kind of a little bit combined here. And then the other is just whether the people who did this study, they just grabbed some people off LinkedIn, which is quite reasonable.
[00:31:29] Drew Rae - cohost: But any given workplace is probably going to actually have a much more cohesive group. With a very particular attitude to technology and experience and trust in management and the use of the technology. So we shouldn't generalize too much just from, you know, small group of people from LinkedIn.
[00:31:47] David Provan - cohost: And they talked about little things like people volunteering to do a tech technology study, maybe have a different view or feeling about technology and monitoring than maybe the background population.
[00:31:58] David Provan - cohost: And you point about purpose and invasiveness, like in that second study that actually changed the level of invasiveness by adding unit eye tracking and facial recognition. So they made. So the two studies were different in purpose and different in invasiveness as well. So you, like you said, they should have been four different conditions or something like that, and more than, more than 10 teams.
[00:32:19] Drew Rae - cohost: But the, the reason why I think we chose this study and why it is useful is it gives us a set of things that we should be asking ourselves if we are thinking about adopting this technology is it shouldn't just be, what data do we want and what's the best way to collect that data? Yeah, the meta lesson here is we should be thinking about how is the collection of this data going to affect people, and that's a key question in any type of workplace measurement.
[00:32:46] David Provan - cohost: So do we talk practical takeaways,
[00:32:48] Drew Rae - cohost: Andrew? Okay. David, you've got quite a few listed here. I might group a couple of 'em together just for. Brevity. So F first one is just the fairly obvious that there are different levels of invasiveness and there is definitely some relationship between how invasive things are and people's acceptance of it.
[00:33:08] Drew Rae - cohost: You know, when we are doing this on a workplace, we really want to deliberately try to minimize feelings of invasiveness. If we want the monitoring to be accepted and useful, that is like should be a conscious thing is how invasive is. This should be one of our key questions, so, so second one is, you don't sort of get a get outta jail free card by explaining the purpose of it.
[00:33:28] Drew Rae - cohost: Invasive is invasive. That's the right question. Not, oh, but we're collecting this data because it's gonna be really helpful. That's not gonna solve the problem. People might in fact be indifferent to the purpose. What they care about is the invasiveness.
[00:33:42] David Provan - cohost: They also talk a lot about stakeholder management change and implementation.
[00:33:46] David Provan - cohost: Just towards the end of this, this paper and just talking about alignment and, and just I guess the, the work that organizations would need to do to, to introduce, introduce, um, you know, electronic team monitoring. Well, and yeah, like you said, the, the, the finding sort of suggests that even if you've got this amazingly positive purpose, if people feel like.
[00:34:08] David Provan - cohost: Data about them is being sort of collected. Uh, it's, it's still a concern.
[00:34:12] Drew Rae - cohost: The second thing I think we can put together, which is that, and it falls under the heading of people aren't stupid. They recognize that any sort of monitoring is likely to be reductionist by which the, the authors mean. Like it's not measuring everything that's relevant.
[00:34:28] Drew Rae - cohost: It's picking a few key variables. And the fact that you slap the label AI over the top of it doesn't change that suspicion, that it's not measuring the full complexity of the real thing. It's measuring particular things that are salient to the machine, and that's not fair. You know, we're trying to measure teamwork, but what we're actually measuring is volume of conversation.
[00:34:49] Drew Rae - cohost: Who speaks? We claim we are measuring tone and sentiment in emails. What we're actually looking for is a few key words that indicate. Aggression and that's not fair.
[00:34:58] David Provan - cohost: And I think what we, what that fairness, I think also flows into Drew is I guess where people's focus and attention is when they're, when they're performing work or performing a task.
[00:35:08] David Provan - cohost: And like you mentioned in this study too here about, you know, even the discussion explicitly happening about what is this system doing? What's it looking for? Um, and you know, participants fed back that, you know, when they're being monitored, they're really focused on optimizing for the monitoring. So making sure that they're not fully concentrating on doing the task well, but being, I guess either a little bit or a lot distracted by how do I make sure I satisfy the system as opposed to necessarily delivering, you know, the best possible task outcome.
[00:35:37] Drew Rae - cohost: Yeah, it didn't come up in this research, but when these things get deployed in real organizations, people aren't just trying to game. What the system's actually measuring. They come up with folk theories about what they think the system is measuring. So you get people, for example, they think that it's censoring swear words, and so you're not actually measuring swear words at all, but people in your organization start, you know, asterisking out the middles of words as if that's gonna improve.
[00:36:05] Drew Rae - cohost: It's like tone detection. You see automated driving systems, people. Think that it's testing for certain things and they start adapting their behavior to what they think it is scoring.
[00:36:14] David Provan - cohost: And Drew, we mentioned the invasiveness, but I think also that, you know all, I mean this is a huge claim, but I guess in in general terms, that electronic monitoring introduces stress.
[00:36:26] David Provan - cohost: So I guess there is a real practical takeaway for health and safety folk that, you know, particularly with a focus on psychosocial hazards in the workplace, is, you know, to some extent we need to consider any form of electronic monitoring as potentially, um, a psychosocial hazard for people. You know, and, and people may, may have different responses to reactions to that with the research, not just in this paper, but I think that the literature more broadly suggests that that's the case.
[00:36:50] David Provan - cohost: Once I start monitoring parameters of people at work, I introduce stress for 'em.
[00:36:57] Drew Rae - cohost: Yeah. Well, new South Wales has just introduced the hierarchy of controls into their regulations for psychosocial hazards. So like, you know, introduce monitoring. Monitoring is a hazard. Highest level of the hierarchy of controls.
[00:37:10] Drew Rae - cohost: Eliminate the hazard ate, and then this is like, the tension is you, you claim you're doing this for safety, but if overall we care about worker or wellbeing, then part of that package is not just surveillance, but all of the other things we do that adds stress. Um, reporting, monitoring, online training, all of these things can in fact be.
[00:37:33] Drew Rae - cohost: Reducing psychosocial wellbeing, and we just gotta be careful that our overall load of these things that we put onto employees isn't actually causing more harm. Than the harm we're trying to prevent with the data we're collecting.
[00:37:45] David Provan - cohost: I had one incident, maybe just a story, drew, and it's not electronic team monitoring, but experience.
[00:37:50] David Provan - cohost: We've talked in a few times about even like vehicle monitoring systems and you know, an organization has introduced them for safety purposes. So we wanna introduce in vehicle monitoring, driver feedback, um, reduce speeding and all of these types of hazards. So it was a deliberate safety intervention.
[00:38:06] David Provan - cohost: There a seriously miss where, um, a driver almost, um, hit a small child in a, in a country town. And you know, what was attributed, at least to that event, was the person said that they were so worried about speeding through some of these sort of 50 kilometer, 35 mile an hour zones in these towns that they were just watching intently, their speedometer on the dash.
[00:38:30] David Provan - cohost: And a kid had ran out onto the road after a, after a soccer ball. And had just, just missed a major accident and said, you know, I'm, we are spending more time watching the speedometer than we are looking at the windscreen. And I guess that's just a, I know it's not really a story relevant to this study, but it sort of just reinforces the point of stress.
[00:38:50] David Provan - cohost: Um, and, and task. And like we said, even if we're doing something for safety, we need to understand if there's introduced hazards.
[00:38:58] Drew Rae - cohost: Yeah. And, and I think this is just generally the key message of our own decluttering work. Is safety is ne never neutral. It's never, oh, this might do some good. It's always, this might do some good and it might do some harm.
[00:39:12] Drew Rae - cohost: And that's why we need to be really careful about what is the evidence for the good that we think we're doing, and what are the potential harms and are we measuring those as well? And making sure that this trade off is actually always on the right side.
[00:39:25] David Provan - cohost: And the researchers talk about this trade off and this, this, this conflict between the goals of employers and managers, which is all about productivity, organizational effectiveness, and the rights of employees and, and individuals, you know, their psychological wellbeing.
[00:39:37] David Provan - cohost: And they're sort of talking about these, you know, are these two competing interests and how do organizations navigate these, these competing interests between. Individual health and wellbeing and the objectives of the organization and most team monitoring even whether it's developmental or performance, like in the second study in this paper, it's all about organizational effectiveness.
[00:39:57] David Provan - cohost: We want individuals and teams to get better at what they do, and we want the performance outcomes of their work to be better as well.
[00:40:04] Drew Rae - cohost: Yeah, funnily enough, people often communicate better when people aren't surveilling their communication.
[00:40:09] David Provan - cohost: And we're seeing it so much. I mean, I'm sure everyone's on, been on a video conference call in the last week where something's been recorded and, you know, I think it is becoming, you know, a regular part of our day-to-day work.
[00:40:20] David Provan - cohost: Lots and lots more, um, you know, individual and team monitoring going on. And so, you know, I think it's a, it's a important, you know, topic to, to. To think about as we're introducing this. So, so Drew the question we asked, are we ready to move on to, are we ready to finish? Yeah, I think so. So the question we asked this week was, how much should we worry about the invasiveness of team support ai?
[00:40:42] Drew Rae - cohost: Not so much that we don't explore ways of improving AI and using it, but enough that we should be measuring and managing the invasiveness just as much as we are measuring and managing the benefits. To make sure that we're getting that trade off right, that these things have evidence that they work and don't cause harm.
[00:41:03] David Provan - cohost: Great. So that's it for this week. We hope you found this episode thought provoking and ultimately useful in shaping the safety of work in your own organization. Join us further discussion on LinkedIn or send any comments, questions, or ideas for future episodes to feedback@safety.com