Treating surgical turnover times as statistically independent events when testing interventions and mobile applications

Uddin et al. created a novel Android app that certified registered nurse anesthetists and operating room nurses used to record the beginning and end of turnover times (1). The screen counts down from the allotted time for the turnover (1). Multivariable linear regression was used to model the turnover time in excess of that allotted. Especially at the surgical suite studied, there was positive association between the excess time and preoperative delays. This app seems to us potentially an excellent use of mobile technology. Anesthesia providers walk to multiple locations during turnovers; mobile technology would provide ongoing feedback on the total time. Without feedback, anesthesia providers are poor judges of their personal turnover times when studied by survey (2). Changes can be made to preoperative assessment to reduce turnover time (3); the authors’ app likely will be very beneficial.

The app not only visualized the elapsed time, but also the start and stop times for the turnover. Do the authors know the percentage of the self-reported turnover times less than the period from the last operating room sensor off one patient to first sensor on the next patient? Sensor data for estimating start/end of turnover times has been used for >10 years (4-6).

The authors’ report of the quantification and inferential testing of the efficacy of their app treated turnover times within each of their two groups, ORTimer or no ORTimer, as independent and identically distributed. This assumption can be inaccurate (7-10). For example, Dexter et al. observed “significant positive serial correlation from one turnover to the next turnover (P<10⁻⁴). When the average turnover was calculated for each workday, there was [still] significant correlation… (P=0.04)… Averaging over 4-week periods was sufficient to eliminate this autocorrelation (P=0.58).” Austin et al. reported that “there was significant positive serial correlation from one turnover to the next turnover during the day (runs test; P<0.0001). Averaging turnover time data points over monthly periods was sufficient to eliminate this autocorrelation (P=0.16).” Wang et al. found that longer than allotted turnover times (i.e., as the authors studied) were more common when the number of simultaneous turnovers at the surgical suite exceeded the number of planned turnovers (8). They used structural equation modeling to examine the correlations among cases on each day, analogous to adjusted Pearson correlations. In the setting of these correlations, Monte-Carlo simulation studies of how to analyze such operating room management data found underestimation of P values (i.e., chance of type I error if the correlations are not considered) (11).

Consequently, in the authors’ Table 1 (i.e., their primary result), the standard errors and P values may be significant underestimates. Using runs test based on the median, do each of the four groups in Table 1 lack autocorrelation, unlike Dexter, Austin, and Wang? If autocorrelation is present, then what are the modified means, standard errors, and P values for the Table 1 when suitable batching/binning is used with the selected threshold times? We appreciate that several of the authors are expert in discrete-event simulation. Other methods developed for comparing simulation outputs suitably address autocorrelation (e.g., bootstrapping).

Acknowledgements

Funding: Support was provided solely from institutional and/or departmental sources.

Footnote

Conflicts of Interest: The Division of Management Consulting, Department of Anesthesia, University of Iowa, performs some of the calculations used in this article. F Dexter receives no funds personally other than his salary and allowable expense reimbursements from the University of Iowa and has tenure with no incentive program. He and his family have no financial holdings in any company related to his work, other than indirectly through mutual funds for retirement. Income from the Division’s consulting work is used to fund Division research.

References

1. Uddin M, Allen R, Huynh N, et al. Assessing operating room turnover time via the use of mobile application. mHealth 2018;4:12. [Crossref] [PubMed]
2. Masursky D, Dexter F, Isaacson SA, et al. Surgeons' and anesthesiologists' perceptions of turnover times. Anesth Analg 2011;112:440-4. [Crossref] [PubMed]
3. Epstein RH, Dexter F, Schwenk ES, et al. Bypass of an anesthesiologist-directed preoperative evaluation clinic results in greater first-case tardiness and turnover times. J Clin Anesth 2017;41:112-9. [Crossref] [PubMed]
4. Xiao Y, Hu P, Hu H, et al. An algorithm for processing vital sign monitoring data to remotely identify operating room occupancy in real-time. Anesth Analg 2005;101:823-9. table of contents. [Crossref] [PubMed]
5. Epstein RH, Dexter F, Piotrowski E. Automated correction of room location errors in anesthesia information management systems. Anesth Analg 2008;107:965-71. [Crossref] [PubMed]
6. Epstein RH, Dexter F. Database Quality and Access Issues Relevant to Research Using Anesthesia Information Management System Data. Anesth Analg 2018;127:105-14. [Crossref] [PubMed]
7. Dexter F, Epstein RH, Marcon E, et al. Estimating the incidence of prolonged turnover times and delays by time of day. Anesthesiology 2005;102:1242-8; discussion 6A.
8. Wang J, Dexter F, Yang K. A behavioral study of daily mean turnover times and first case of the day start tardiness. Anesth Analg 2013;116:1333-41. [Crossref] [PubMed]
9. Austin TM, Lam HV, Shin NS, et al. Elective change of surgeon during the OR day has an operationally negligible impact on turnover time. J Clin Anesth 2014;26:343-9. [Crossref] [PubMed]
10. Dexter F. High-quality operating room management research. J Clin Anesth 2014;26:341-2. [Crossref] [PubMed]
11. Dexter F, Marcon E, Epstein RH, et al. Validation of statistical methods to compare cancellation rates on the day of surgery. Anesth Analg 2005;101:465-73. table of contents. [Crossref] [PubMed]