Why It Is Crucial to Involve Nurses in AI Development Processes

Patient safety researchers at the Institute for Healthcare Improvement (IHI) have been studying nurse-led technology innovation, including introducing AI into nursing workflow. Healthcare Innovation recently spoke with Patricia McGaffigan, R.N., M.S., vice president of safety at IHI and president of the Certification Board for Professionals in Patient Safety, and Kate Feske-Kirby, an IHI research associate in innovation, about their current research efforts. 

Healthcare Innovation: AI is being introduced into clinical workflows at a rapid pace. You two have given a lot of thought to some of the issues that need to be addressed during this transition regarding nursing. Could you talk about some of the possible unintended consequences that might be introduced into the nursing workflow if AI tools are rolled out too rapidly or without enough input form nurses? 

McGaffigan: The IHI Lucian Leape Institute, a patient safety think tank, has worked with an expert panel and created a core paper from which this work has evolved. We frame it as the hype, the hope and the potential peril of this work. One of the main concerns that people have is grounded in the safety lens and whether AI is really safe for use, particularly for clinical purposes. There are some concerns about whether it will impact safety — for example, if it's used for prior authorizations and other things which clearly have safety implications. 

There are a number of safety implications that come up in terms of whether decision support is accurate and based on evidence, and what the learning data set is that is informing any of the recommendations. There are concerns about whether people will know that decisions are based on the use of artificial intelligence and whether there are transparency requirements that should be provided. Also, are people really prepared now to embrace this technology and make safe decisions about its use, and how will that impact how we skill and build competencies in the future workforce?

HCI: Is there a concern about a loss of nurse autonomy — that their judgment could be overruled?

McGaffigan: Yes. There’s also concern about whether the role of nurses will be replaced because of AI. My personal opinion is that the risk will be if nursing does not embrace and learn to lead the safe and responsible use of AI in our work, because the benefits are equally compelling.

HCI: Kate, did you want to talk about any other issues that your research identified as potential concerns?

Feske-Kirby: One thing that came up quite often, particularly during the Lucian Leape Institute expert panel, was potential depersonalization and disconnection, both between the clinician and their patient and the patient's family, as well as between the clinicians and their work. 

When you have an interaction with the patient, you get to know them. You talk to them about how their day is going. And sometimes the technology that is made to support documentation tasks, which can be very useful in alleviating that burden, might not record that information, or might depersonalize that note so that maybe that clinician/patient relationships might take a bit of a hit there, although we very much believe that nurses and other clinicians have the ability to keep that moving forward. 

Another issue is alert fatigue. Once this is implemented, how will it actually perform in day-to-day clinical tasks? There has already been some effort to implement some of these technologies before, and there haven’t necessarily been efforts at co-design, making sure that key parties are involved in the design, the implementation and the monitoring of these technologies.

HCI: This sounds like a continuation of problems that have been longstanding, as far as nurses feeling like technological deployments are things that happen to them rather than with them. You mentioned co-design. Could nurses by resistant to AI if it feels like they weren't involved in the development of the applications but it's applied to their everyday work?

McGaffigan: In recent presentations that I've done, one as recently as last week and several in recent months with hundreds of nurses and nurse leaders, we’ve done surveys and very few of them have been involved in any way in the meaningful governance or engagement or user-centered approaches to considering and bringing generative AI into their worlds. And that, in and of itself, is a major reason why there will always be resistance to new ideas, whether they're technology-based or not. 

A lot of what we advocate for in this work is making sure that in the governance and oversight and decision-making and testing, we've got nurses who are embedded in all aspects of that work. And to not just think about it as something that we do to get a new technology launched, but to look at this, as we do at IHI, as continuous improvement work. We’re going to continue to learn and appreciate new things, especially because we're seeing the pace of development in AI be so incredibly brisk.

HCI: Viewing it from the outside, it would seem like such an oversight to develop AI governance strategies and teams and not include nurses or nurse leadership.

McGaffigan: We have compelling reasons to state that nurses should be meaningfully involved from stem to stern in this process. The way generative AI applications are developed might be coming from external parties and developers. There’s a lot of physician-led work in this space, and I'm really hopeful we will see more bright spots of opportunity for AI applications that will benefit nurses, such as event listening to help with documentation or scheduling — capabilities that make life for a charge nurse much less complex.

I've talked with AI companies, and I am extremely hopeful that there are a lot of investments going into a very much needed opportunity for generative AI to make a really meaningful difference for the work of nurses to allow them to spend more time with patients. I think it will take a while, but I'm hopeful.

HCI: I understand that IHI and other groups participated in a Learning and Action Network pilot with nurse-led teams at five healthcare organizations. Was the goal to learn how some of these innovative solutions can help the nursing workforce thrive? Were there some key findings from that work you could talk about?

McGaffigan: There were some key findings, and it's such a fun project that was generously funded by the Johnson & Johnson Foundation, where we engaged five different types of healthcare settings that wanted to test new approaches for care delivery innovation with a primary outcome measure in whether or not we could positively impact the thriving of the workforce. 

There were a series of change ideas that fell into different categories, many of which were including and addressing virtual nursing technologies. There was an AI technology that was used to support patient assignments, and another area involved  how we reconfigure care delivery teams and the work of care delivery teams to be able to best serve patients.

You hear all the time that we need to retain nurses and we need to reduce turnover. We would agree with that. What was different and important about this project was moving beyond whether or not we had a nurse at the bedside; we cared about whether that nurse was thriving, because we know from all of our work in safety and quality that the outcomes of patients are inextricably linked with the safety and the well-being and the ability of the workforce to come to work every day and be at their best. 

We were able to measurably appreciate whether or not teams were thriving amid nurse-led care delivery innovation. We did find quantitative improvement in elements of thriving and a balance of qualitative improvements for nurses in terms of time saved, how they felt about coming to work every day, whether they felt they had more time with patients, and whether we were able to meaningfully engage them in their work, in really stewarding redesign of work and systems in which work is done.

HCI: It seems like that kind of analysis could also serve as an alarm bell on the other side — if people weren't thriving as you were doing an implementation, you could stop and make adjustments. But without that measurement, you might not actually realize the impact you're having.

McGaffigan: If you look at why nurses are leaving their jobs, some of it is because they feel that they're not valued and that they are not able to meaningfully contribute to evolving the work of nursing, and they feel grossly understaffed. This model brought the five sites together to be able to learn as one and share ideas, and, more importantly, to be able to test ideas of change before we roll them out. That speaks to a lot of what you have heard about technology rollout. Often it is about going big without really understanding the conditions under which that thrives or fails.

HCI: Your research has identified some guardrails to consider when doing this type of implementation. Kate, could you mention a few of those? 

Feske-Kirby: One involved investing in developing and sustaining skills and competencies for nurses. We want to make sure that nurses are equipped and feel comfortable using new technology, that they can apply it safely and provide quality care. But there's also the question of making sure that they remain skilled without the technology should anything happen. Technical glitches happen. We've had lots of adverse weather conditions that can really impact a hospital or healthcare's ability to sustain technology that uses a lot of energy. 

Another is advocating for and engaging in continuous improvement. I think our phrase was, “nurses are indomitable improvers.” They're really the backbone of this work and thinking about the larger workflow and patient flow through a hospital, to make sure that improvements really touch all corners of the hospital, workforce and patient experience. 

And another guardrail is around making sure that any time saved, any efficiencies that are gleaned from artificial intelligence or use of new technology are repurposed into providing better care and more time with patients.

HCI: A few months ago, I interviewed the dean of the nursing school at Florida State University because they were launching an AI-focused Master of Science Nursing program. Is it important that more nursing schools follow suit and build AI into the curriculum, so that nurses get experience working and innovating with these tools in their training before they get into the workforce?

McGaffigan: You just mentioned something I get super-excited to hear about — it’s how nursing education is changing and evolving to be able to keep up with and hopefully lead and get ahead of technology changes. You just referred to a graduate program, which is fabulous, and yes, I think the inclusion of AI and other technology use into foundational pre-licensure education for all healthcare professionals is super-critical right now. I know that changing curricula and adding curricula is not something that can be easily done overnight, but thinking about how we can evolve the way we approach education of healthcare colleagues and healthcare leaders to be able to ensure we're able to stay ahead of this and provide just-in-time education, as we see the rapid pace of technology unfold, is super-important for workforce capabilities and for ensuring that we have high-quality care.