Feasibility and acceptance of a cloud-based mobile app for antimicrobial stewardship and infection control in Colombian hospitals

Corresponding author: William Joseph Ketcherside ILÚM Health Solutions, LLC; Kenilworth, NJ, USA Email: joeketch@me.com Abstract Infection control and antimicrobial stewardship programs (ICASPs) are essential to reduce the emergence and spread of antimicrobial resistance. The primary objective of this study was to assess the feasibility of extending a commercial off-the-shelf (COTS) software for ICASPs in lowand middle-income countries (LMICs). This project involved three hospitals in Colombia, including Centro Médico Imbanaco, Clínica San Francisco, and DIME Clínica Neurocardiovascular. A COTS platform (ILÚM Health SolutionsTM; Kenilworth, NJ) was extended to function in a range of technology settings, and translatable to almost any language. ICASP features were added, including clinical practice guidelines, hand hygiene (HH) documentation, and isolation precaution (IP) documentation. The platform was delivered as a smartphone mobile application (“app”) for both iOS and Android. The app was successfully implemented at all sites, however, full back-end data integration was not feasible at any site. In contrast to the United States, a suite of surveillance tools and physician-focused decision support without patient data proved to be valuable. Language translation processing occurred quickly and incurred minimal costs. HH and IP compliance tracking were the most used features among ICASP staff; treatment guidelines were most often used by physicians. Use of the app streamlined activities and reduced the time spent on ICASP tasks. Users consistently reported positive impressions including simplicity of design, ease of navigation, and improved efficiency. This ICASP app was feasible in limited-resource settings, highly acceptable to users, and represents an innovative approach to antimicrobial resistance prevention.


not for citation purposes
Feasibility and acceptance of a cloud-based mobile app for AMS and IC Ketcherside et al.

Background
Antimicrobial resistance (AMR) is a global threat that results in increased healthcare costs and poorer patient outcomes. [1][2][3] In low-and middle-income countries (LMICs), antibiotic consumption, healthcareassociated infections, and AMR rates are rising. [4][5][6][7] Professional, non-governmental, and government organizations are concerned about AMR progression in LMICs and seek innovative solutions. [8][9][10] Infection control and antimicrobial stewardship programs (ICASPs) reduce the emergence and spread of AMR. 10, 11 In general, health facilities in LMICs have many barriers to implementation and execution of ICASPs, including limited staff with specialized training, minimal laboratory and diagnostic tools, gaps in data on antimicrobial use and resistance, and lack of a robust health information technology infrastructure. [12][13][14][15] While many LMIC settings lack the informatics infrastructure of high-income countries, mobile cellular technologies and smartphones are often widely available. 16 Therefore, a mobile ICASP software tool designed for use in LMICs could improve infection control (IC) and antimicrobial stewardship (AMS) management. The use of the mobile application ("app") was expected to streamline activities and reduce the time spent on various tasks. The objective of this project was to leverage smartphones to enhance ICASP in LMICs and assess uptake, acceptance, and performance in a range of hospital environments.

Methods
This project was a collaboration between Universidad El Bosque (Bogotá, Colombia), OpGen, Inc.
(Gaithersburg, MD, USA), and ILÚM Health Solutions, LLC (Kenilworth, NJ, USA). The project began on September 30, 2017 and the app went live at all three sites occurred less than 11 months later on August 28-30, 2018. Ethics Review Board approval was granted at each of the three hospitals.

Hospital Sites
Three hospitals in southwestern Colombia participated in this project. These sites were selected because they have robust infectious diseases (ID) and infection prevention committees, all hospitals receive consistent ID consultation through the same physician, and utilize hospital-specific guidelines for antimicrobial use. All sites utilized paper documentation for IC processes.
Centro Médico Imbanaco (CMI) is a 400-bed tertiary care hospital in Cali, Colombia. CMI represents a hightech hospital with bar code scanning, a commercialgrade data center, and a homegrown electronic medical record (EMR). The ICASP has seven practitioners and uses advanced processes. Clínica San Francisco (CSF) is a 133-bed community teaching hospital in Tuluá, Colombia. CSF represents a medium-tech hospital with an internally developed EMR and publishes their local infection treatment guidelines on the hospital intranet. DIME Clínica Neurocardiovascular (DIME) is a 54-bed specialty hospital in Cali, Colombia. DIME represents a low-tech hospital; it has an EMR but lacks integration across departments.

App Description and Features
The project team extended a commercial off the shelf (COTS) app designed for ID management in U.S. acute care hospitals (ILÚM Health Solutions; Kenilworth, NJ). The modified software was designed for flexibility of integration and adaptability of connectivity, including an offline mode. The app was built to function in a range of technology settings with variable data availability and to be translated to other languages. A virtual private network (VPN) was required to access data from the EMR; one was set up between the cloud and each of the sites. The VPN allowed for assessment of the individual hospitals' data feeds and capability to integrate real-time data. Microbiology data were obtained from each hospital to develop site-specific antibiograms.
The app is available for both iOS and Android devices; users downloaded it directly onto their personal smartphones. Content was configured to each hospital. The app gives permission to local champions to update or create new content.
For IC, tracking tools for hand hygiene (HH) and isolation precaution (IP) compliance tracking were built. In both tools, the user answers a series of questions to indicate if the activity observed is compliant with recommended practices. Reference material for IP is also available. ICASP team members not for citation purposes Feasibility and acceptance of a cloud-based mobile app for AMS and IC Ketcherside et al.
are the intended primary user of this feature. ICASP leadership has access to a suite of analytic functions to aggregate monthly and quarterly reports.

For AMS, Treatment Guidelines and Surgical
Prophylaxis Guidelines offer interactive decision support via checkbox selection of patient factors and guide clinicians to an appropriate antibiotic choice per local hospital guidelines. Additional AMS features include medication information for antimicrobials, an antibiogram, and a PDF view of flow diagrams as an alternative to the Treatment Guidelines. Physicians are the intended primary users of these features.
The Messages feature allows the system and privileged users to send notifications to users in a specific role. This feature was extended to allow for collection of data for research questionnaires and end-user satisfaction surveys. Beyond the features available to end-users, the app contains a full suite of System Administration tools for account creation, access controls to features, content editing, and usage reports.

Training
A train-the-trainer approach was used to teach users how to use the app. The project team trained select ICASP team leaders at each site and provided onsite support during go-live. Training materials covering app installation, account activation, and app use were disseminated. Following the go-live, the clinical champions continued to train new users.

Evaluation and Time on Task Analysis
Net Promoter Score ® (NPS ® ), a standard method for measuring customer satisfaction with a product on a ten-point scale, was used to measure initial success of the app. Respondents are grouped as promoters (enthusiasts, score 9-10), passives (satisfied but unenthusiastic, score 7-8), or detractors (unhappy users, score 0-6). NPS is determined by subtracting the percentage of customers who are detractors from the percentage who are promoters. What is generated is a score between -100 and 100. The NPS ® survey was delivered to users via the Messages feature four weeks after go-live.
Perceived ease of use and usability were measured via a modified Technology Acceptance Model (TAM) survey delivered via the Messages feature. 17 The survey consisted of 30 questions using a seven-point scale to assess perceived usefulness, perceived ease of use, workflow changes and improvement, product improvement, and user information.
Usage metrics were automatically collected via the system to track logins and page views. Analysis of feature usage and retention rates were performed at 90 days post-go-live at each site for both individuals and in aggregate. Time on tasks for IC activities was documented by staff before and 90-days postimplementation.

App implementation and technical outcomes
The app was successfully implemented and utilized, however, each institution had obstacles that prohibited the complete back-end connection to realtime data. The project team spent over 200 hours performing the setup of the VPN and associated components to support data integration analysis. The project team learned that VPNs are feasible, however, can be highly resource-consuming. Translating the app to other languages was simple once the internationalization infrastructure was put in place.
The system is now able to provide rapid translations into other languages.

User acquisition, feature usage, and retention rates
From app go-live, new users were continually added to the system at each site through the first 90 days ( Figure 1). This is illustrative of the wider acceptance of the app as it was rolled across the organizations.
User retention represents returned users and repeat usage of the app. Retention rates were calculated per user up to 90 days from initial login. Retention at 90 days is summarized in Table I. Among ICASP staff, retention rates were 100% for all three sites at 90 days. Physician retention rates at all sites ranged from 88-92%.
ICASP staff reported using the HH and IP compliance trackers most often. The most common feedback was related to improved efficiency and reduced time spent on observation tasks. Additionally, users reported they felt the observations were of higher quality because use of one's phone is less obvious than previous monitoring methods. not for citation purposes Feasibility and acceptance of a cloud-based mobile app for AMS and IC Ketcherside et al.
The Treatment Guidelines were the most commonly used AMS feature at all three sites and physician users reported this as the most useful feature. After initial go-live, usage varied widely over time and across institutions. CSF had the most usage of guidelines among the sites. Physicians frequently described how the Treatment Guidelines made them more informed and led to more meaningful conversations when consulting with ID colleagues. Among all aspects of the app, its efficiency was valued most. The most common criticism was a lack of additional guidelines for other infections and populations.

Evaluation and time on task analysis
The resulting NPS ® was +77, indicating there are more satisfied than unsatisfied users. However, this score is remarkably high and indicates a need for repeat testing over time and additional investigative methodologies.
Forty-six survey responses were received, 12 from CMI, 14 from CSF, and 20 from DIME representing 35% of users. Respondents were primarily physicians (34, 74%) and had been in practice for ten years or less (29, 63%). Thirty-one users (67%) agreed that the app provided the tools necessary to perform their job related to ICASP. Most respondents rated the app very highly with regards to perceived usefulness (average score 5.94, range 1-7) and perceived ease of use (average score 6.19, range 3-7). Treatment Guidelines were considered the most useful feature of the app. The most commonly cited barrier to app usage was wireless network connectivity. End-users were also given the opportunity to provide qualitative feedback on the app throughout the evaluation. Users consistently reported positive impressions, specifically the simplicity of design and ease of navigation.
The use of technology and a streamlined process have allowed ICASP staff to make more observations and spend more time on other tasks. The time spent on tasks before and after implementation is presented in Table II.

Discussion
The findings from this project demonstrate the feasibility of a mobile app for ICASPs in LMICs. With this app, a suite of infection control monitoring tools and physician-focused decision support can still have broad impacts on practice, even without integrated real-time patient data. This contrasts with the most common U.S. practices, which rely on data  integration to support clinical decision support and surveillance.
In these sites, clinicians used our app to improve antibiotic use, despite not having access to real-time, patient-specific data. One of the largest AMS needs in LMICs is support for better empiric antibiotic decisions through localized treatment guidelines. 18 By providing such guidance through an easily navigable mobile app, most physician users reported feeling empowered in their antibiotic decisions and confidence in consultations with ID specialists.
For IC teams, the mobile app replaced time-intensive paper processes with efficient electronic ones. The time spent in Data Collection did not decrease at CSF as this site based their collection on a predetermined duration rather than a number of observations as in the other sites. An additional unanticipated benefit from using the app was the ability to conduct observations inconspicuously, allowing staff to report higher quality observations. Per the TAM survey, the high average scores are predictive of high software usage.
The level of effort involved in setting up a similar system depends on several factors: size and complexity of the hospital database, type of VPN system, and the client's desire for data integration.

Future directions
This project revealed information about user needs and demonstrated gains in productivity through automating manual tasks. It also laid the groundwork for further outcomes studies at these three sites, which will explore the clinical and economic impact of this tool.

Conclusion
This novel ICASP mobile app was feasible at hospitals with varying levels of information technology capacity and was highly acceptable to users. Improved efficiency was observed with decreased time on tasks related to HH and IP compliance. Physician users consistently reported positive impressions including simplicity of design, increase confidence in antibiotic decisions, and improved efficiency.
An ICASP monitoring and clinical decision support tool scalable to many settings has the potential to impact the care of patients around the globe. Beyond the experience at these three sites, the findings of this project suggest that hospitals with functional ICASPs in other countries could adopt the tools in this app quickly to realize similar benefits.

Disclaimer
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.