This paper evaluates frameworks for the procurement, integration, monitoring, and evaluation of AI tools in clinical settings, focusing on mapping these to the Plan-Do-Study-Act (PDSA) cycle. 1️⃣ Plan: ▫ Assessing how AI tools can be incorporated into existing clinical workflows is critical to their successful implementation. ▫ Emphasizing the need for clear documentation and transparency about the AI tool’s functioning to build trust among users. ▫ Addressing ethical considerations, including data privacy and bias, is essential before implementing AI tools. ▫ Evaluating the financial viability of AI tools, including purchase, implementation, and maintenance costs. ▫ Ensuring AI tools meet regulatory standards and are approved by relevant authorities. ▫ Defining legal responsibilities and liabilities in case of adverse events or errors caused by AI tools. 2️⃣ Do: ▫ Successfully incorporating AI tools into the existing processes and assigning roles for their use and maintenance. ▫ Providing adequate training for end-users to effectively operate AI tools. ▫ Ensuring that AI tools are user-friendly and perceived as valuable additions to the clinical workflow. ▫ Involving a multidisciplinary team with the necessary technical and clinical expertise to support AI integration. 3️⃣ Study: ▫ Using predefined metrics like sensitivity, specificity, and accuracy to validate AI tools. ▫ Collecting feedback on user satisfaction and acceptance to identify barriers to effective use. ▫ Measuring the impact of AI tools on patient outcomes, such as morbidity and mortality rates. ▫ Establishing protocols for reporting and analyzing adverse events or system failures. 4️⃣ Act: ▫ Continuously monitoring and adjusting AI tools based on feedback and performance data to enhance their effectiveness. ✍🏻 Sarim Dawar Khan, Zahra Hoodbhoy, Raja MHR, Kim JY, Hogg HDJ, Manji AAA, et al. (2024) Frameworks for procurement, integration, monitoring, and evaluation of artificial intelligence tools in clinical settings: A systematic review. PLOS Digit Health 3(5): e0000514. DOI: 10.1371/journal.pdig.0000514
Need to study measures in all three layers of an AI based technology: foundation layer, technical layer & sociotechical layer.
thx, useful ❕ ✔
Very helpful!
Interesting paper!
Digital Strategist
1moThank you, this will be useful for what I am doing with https://gregory-ms.com/