International Journal for Quality in Health Care, 2017, 29(5), 722–727 doi: 10.1093/intqhc/mzx097 Advance Access Publication Date: 14 September 2017 Article Article The role of patient perception of crowding in the determination of real-time patient satisfaction at Emergency Department HAO WANG1, JEFFREY A. KLINE2, BRADFORD E. JACKSON3, RICHARD D. ROBINSON1, MATTHEW SULLIVAN1, MARCUS HOLMES1, KATHERINE A. WATSON1, CHAD D. COWDEN1, JESSICA LAUREANO PHILLIPS3, CHET D. SCHRADER1, JOANNA LEUCK1, and NESTOR R. ZENAROSA1 1 Department of Emergency Medicine, Integrative Emergency Services, John Peter Smith Health Network, 1500S. Main St., Fort Worth, TX 76104, USA, 2Department of Emergency Medicine, Indiana University School of Medicine, 640 Eskenazi Ave, Indianapolis, IN 46202, USA, and 3Center for Outcomes Research, John Peter Smith Health Network, 1500S. Main St., Fort Worth, TX 76104, USA Address reprint requests to: Hao Wang. Tel: +1-817-702-8696; Fax: +1-817-702-1143; E-mail: [email protected] Editorial Decision 3 July 2017; Accepted 4 July 2017 Abstract Objective: To evaluate the associations between real-time overall patient satisfaction and Emergency Department (ED) crowding as determined by patient percepton and crowding estimation tool score in a high-volume ED. Design: A prospective observational study. Setting: A tertiary acute hospital ED and a Level 1 trauma center. Participants: ED patients. Intervention(s): Crowding status was measured by two crowding tools [National Emergency Department Overcrowding Scale (NEDOCS) and Severely overcrowded–Overcrowded–Not overcrowded Estimation Tool (SONET)] and patient perception of crowding surveys administered at discharge. Main outcome measure(s): ED crowding and patient real-time satisfaction. Results: From 29 November 2015 through 11 January 2016, we enrolled 1345 participants. We observed considerable agreement between the NEDOCS and SONET assessment of ED crowding (bias = 0.22; 95% limits of agreement (LOAs): −1.67, 2.12). However, agreement was more variable between patient perceptions of ED crowding with NEDOCS (bias = 0.62; 95% LOA: −5.85, 7.09) and SONET (bias = 0.40; 95% LOA: −5.81, 6.61). Compared to not overcrowded, there were overall inverse associations between ED overcrowding and patient satisfaction (Patient perception OR = 0.49, 95% conﬁdence limit (CL): 0.38, 0.63; NEDOCS OR = 0.78, 95% CL: 0.65, 0.95; SONET OR = 0.82, 95% CL: 0.69, 0.98). Conclusions: While heterogeneity exists in the degree of agreement between objective and patient perceived assessments of ED crowding, in our study we observed that higher degrees of ED crowding at admission might be associated with lower real-time patient satisfaction. Key words: real-time patient satisfaction survey, Emergency Department, patient perception, tool, crowding © The Author 2017. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved. For permissions, please e-mail: [email protected] 722 ED crowding and patient satisfaction • Patient Satisfaction Introduction Emergency Department (ED) overcrowding degrades the quality of emergency care by increased ambulance diversion, increased rate of patients left without being seen, prolonged patient total length of stay, decreased patient satisfaction, etc. [1–3]. Several ED crowding estimation tools have been developed and deployed to evaluate relative ED crowding [4–6]. Among these tools, the National Emergency Department Overcrowding Scale (NEDOCS) was validated in previous studies and is widely used in the USA , while Severely overcrowded–Overcrowded–Not overcrowded Estimation Tool (SONET) was speciﬁcally derived at an extremely high-volume ED setting and has since been externally validated . Both tools reported worsening of the patient care outcomes with increased levels of ED crowding and both were derived based on providers’ perceptions of ED crowding [6, 7]. However, these perceptions determine crowding by viewing an ED as an entire entity and accuracy of the reports from different studies are quite controversial [8–10]. Conversely, patient perception of ED crowding is considered to view crowding speciﬁcally at the individual-level with impressions reﬂecting the patient experience. To date, no study has investigated the potential agreement of ED crowding determined by either estimation tool or patients’ perceptions, Prior research suggests that prolonged ED wait times, as perceived by patients, is associated with decreased patient satisfaction . Similarly, validated ED crowding metrics are predictive of failure to meet patient satisfaction goals  and lower overall ED satisfaction . However, most of these studies were retrospective and patient satisfaction results were obtained through remote survey data gathering and analysis done weeks after the index visit (e.g. Press-Ganey, NRC Picker). Such survey results are typically not available in real-time and, therefore, do not reﬂect perceptions of ED crowding during the index visit. Although it would seem obvious that crowding worsens patient satisfaction, little literature has been published to document this effect. One previous study showed ED crowding— especially prolonged waiting times as perceived by patients—was associated with compromised emergency care. Uncertainty with respect to direct evidence of a link between patient satisfaction and ED crowding was noted due to variabilities among patients, nurses and physicians in a setting yielding relatively poor survey participation (<20%) in the form of perceptions of patients . Another study conducted a realtime ED patient satisfaction survey among high acuity patients in a Pakistan tertiary hospital . Investigators described several justiﬁcations favoring the use of real-time patient satisfaction surveys over mailed surveys as it is easier to provide quality assured service, easier to capture ED patients and easier to reﬂect the direct response between patient satisfaction and ED service provided. Real-time patient satisfaction evaluation has been reported as advantageous in terms of rendering instant feedback for quality improvement, higher completion rates and ease of use [15–17]. Prospective studies of current state real-time ED patient satisfaction as a function of relative ED crowding should yield improved understanding of this dynamic and guide development of quality projects that deliver an enhanced future state . The primary aim of this study was to evaluate the direct association between ED crowding and real-time patient satisfaction. Since there is no gold standard to determine ED crowding and current measurements are subjective, we sought to evaluate the association between ED crowding and real-time patient satisfaction using different ED crowding measurements (tools vs. patient perceptions) despite their accuracy. In addition to this, we assessed potential consistencies 723 in associations when ED crowding was determined by patient perception, as well as, two objective estimation tools. We hypothesized that overcrowding worsened patient satisfaction despite different measurements used. Methods Study design and protocol This was a single-center prospective observational study conducted at an academic ED (John Peter Smith Health Network, Fort Worth, TX) with an annual volume of over 118 000 patients. Each ED crowding tool utilized in this study were initially derived from provider perceptions of crowding. This study sought to assess ED crowding as determined by provider perception captured using two validated estimation tools and patient perceptions as attributed using a real-time patient satisfaction survey. The initial step was to assign an objective ED crowding score to each patient encounter using two ED crowding estimation tools (NEDOCS and SONET) upon patient arrival and registration at ED [4, 6]. The second step was to simultaneously collect patient satisfaction from real-time patient surveys upon their individual dispositions (e.g. upon patient discharge, transferred to other facilities, admitted, etc.). Study participants All ED-registered patients during the period of 29 November 2015 through 11 January 2016 and were subsequently dispositioned (e.g. discharged home or admitted to hospital) from ED were considered eligible for enrollment into this study. Exclusion criteria were: (i) patients who refused to participate in the real-time patient satisfaction survey; (ii) patients who did not participate in the satisfaction survey (e.g. patients who left without being seen or eloped); (iii) patients whose surveyors entered incorrect information (e.g. answers not chosen from predetermined options [menu] or answers unrelated to the questions); (iv) patients who completed <20% of the survey; (v) patients not assigned an initial ED crowding score upon registration and/or (vi) patients not assigned an ED crowding score due to incomplete data. Deﬁnition of ED crowding Three measures of ED crowding were assessed for this study, two objective ED crowding estimation tools (NEDOCS and SONET) derived from provider perception and patient perception of overcrowding. NEDOCS has been externally validated and is nationally recognized as a tool to measure ED crowding [4, 8, 19]. SONET was derived directly from the study ED and determined to be more accurate than NEDOCS for crowding estimation in an extremely high-volume ED setting (>100 000 annual visits) . ED crowding was categorized into three different levels by both NEDOCS and SONET (not overcrowded ≤100, overcrowded >100 to ≤140 and severely overcrowded >140). During the study period, a question regarding ED crowding status was included in the satisfaction survey whereby the patient was asked to rate their perception of relative ED crowding on a Likert scale of 1–10 with 1 being the least and 10 being the worst crowded. To align ED crowding scores as calculated by the crowding estimation tools, patient perceived ED crowding scores were further categorized into the same three levels used to derive SONET as previously reported (i.e. not overcrowded = 1–5, overcrowded = 6–7 and severely overcrowded = 8–10) (i.e. multiples of a constant of 20) . Wang et al. 724 Patient satisfaction survey A real-time patient satisfaction survey currently available in market (Qualitick, Clearwater, FL) was conducted at the end of the ED encounter, prior to patient moving out of ED (Supplemental Table 1). A computer tablet housing the conﬁdential survey was provided to patients and/or their designees in a private setting, away from healthcare providers. The survey only allows the patient to view one question at a time and does not give the option to return to the previous question once the ‘turn page’ button is hit. Expected time for survey completion was <5 min. This brief survey included a priori questions addressing established risks affecting patient satisfaction as published in the literature (i.e. patient satisfaction with provider(s), patient satisfaction with nurse(s), perception of pain control, etc.) [21, 22]. Patient perception of ED crowding was the last question of the survey. The primary outcome of this study, patient satisfaction, was assessed from the following, ‘Overall, how satisﬁed were you with your visit today? (Scale of 1–10, 1: very dissatisﬁed, 10: very satisﬁed).’ reported ED as overcrowded more than NEDOCS (bias = 0.22; 95% LOA: −1.67, 2.12). We found similar differences in agreement between patient perception and both NEDOCS and SONET. Patient’s perceptions of crowding were emphasized at the extremes of the crowding Data analysis Descriptive statistics are presented as frequencies and percentages for categorical variables and median and interquartile ranges (IQRs) for continuous variables. Agreement between ED crowding measures (patient perception, NEDOCS and SONET) were assessed by the between-measure mean difference (bias) and corresponding 95% limits of agreement (LOAs) and presented graphically using Bland–Altman plots . We used three separate fractional logistic regression models with robust variance to estimate odds ratios (ORs) and corresponding 95% conﬁdence limits (CLs) for the association between ED crowding and patient satisfaction with adjustment for a minimal sufﬁcient set of covariates to reduce confounding bias. This set of covariates was identiﬁed using directed acyclic graphs and included age, sex, race/ethnicity and acuity level . Patient satisfaction scores were transformed from the original scale of 1 to 10, to a continuous 0 to 1 data element in order to facilitate statistical modeling. Furthermore, for each measure of ED crowding, we estimated the expected overall patient satisfaction across levels of ED crowding. More pragmatically in the management of ED ﬂow, ED crowding scores were categorized into three levels of crowding (not overcrowded, overcrowded or severely overcrowded). Observations with missing values for relevant covariates were excluded from the analysis. All analysis was performed using Stata 14.0 (College Station, TX). This study was approved by the local Institutional Review Board. Results A total of 13 196 patients were registered in the ED during the study period (29 November 2015 through 11 January 2016), of which, 1746 (13%) were eligible to participate and enrolled in the study (Fig. 1). Of those enrolled, 161 patients were excluded due to incorrect information, 135 for less than 20% survey completion and 105 due to no ED crowding score group assignment upon registration. The ﬁnal 1345 patients were analyzed. Table 1 presents the general characteristics of the enrolled and nonenrolled participants during the study period. In brief, enrolled patients were predominately female, younger (median age 41 years vs. 47 years) and had lower Emergency Severity Index (ESI) scores. The majority of patients were assigned a mid-acuity triage level (ESI-3) and were ultimately discharged. Figure 2 presents the assessments of agreement between the three measures of ED crowding using Bland–Altman plots. There was consistent agreement between SONET and NEDOCS where SONET Figure 1 Study patient enrollment ﬂow diagram. Table 1 Characteristics of enrolled and nonenrolled patients Enrolled Nonenrolled Patients, n (%) 1345 (10.2) 11 851 (89.8) Age (year), median (IQR) 41 (28–54) 47 (33–57) Sex (male), yes, n (%) 595 (44) 5834 (49) Race, n (%) NH White 533 (40) 4451 (38) NH Black 476 (35) 4069 (34) Hispanic 335 (25) 3179 (27) ESI, n (%) ESI-1 17 (1.3) 296 (2.5) ESI-2 379 (28) 2694 (23) ESI-3 849 (63) 6586 (56) ESI-4 87 (6.5) 1974 (17) ESI-5 11 (0.8) 241 (2) Unknown 2 (0.2) 60 (0.5) ED disposition, n (%) Admit 234 (17) 2371 (20) Discharge 1039 (77) 7427 (63) 72 (5) 2053 (17) Othersa Total ED length of stay, minutes 274 (189–381) 230 (144–339) (median, IQR) ED crowding levels determined by NEDOCS, n (%)b Not overcrowded 718 (53) 5748 (49) Overcrowded 386 (29) 3716/ (32) Severely overcrowded 241 (18) 2282 (19) ED crowding levels determined by SONET, n (%)b Not overcrowded 763 (57) 6151 (52) Overcrowded 486 (36) 4804 (41) Severely overcrowded 96 (7) 791 (7) NH, non-Hispanic. a Others refer to other ED dispositions including ED transfer to other facilities, ED sent to Labor & Delivery, ED send to Operating Room or Left without being seen (only applied to nonenrolled patients). Percentage adding up does not equal to 100 due to number rounding up. b ED crowding levels (e.g. not overcrowded, overcrowded and severely overcrowded) were determined upon each patient arriving to and registering at ED. ED crowding and patient satisfaction • Patient Satisfaction spectrum, with more frequent perceptions of ED as less crowded compared to each objective measure. The mean differences in assessment methods were close to 0 with wide limits of agreement for patient perception and both NEDOCS and SONET bias = 0.62; 95% LOA: −5.85, 7.09 and bias = 0.40; 95% LOA: −5.81, 6.61, respectively. The adjusted ORs and 95% CLs for the association between ED crowding and real-time patient satisfaction are presented in Table 2. In terms of the objective measures of overcrowding, ED overcrowded was associated with lower odds of patient satisfaction compared to not overcrowded (NEDOCS = 0.78; 95% CL: 0.65, 0.95; SONET = 0.82; 95% CL: 0.69, 0.98). Moreover, severely 725 overcrowded for objective measures was also associated with lower odds of patient satisfaction (NEDOCS = 0.79; 95% CL: 0.61, 1.01; SONET = 0.78; 95% CL: 0.51, 1.18). Patient perceptions of overcrowded was associated with lower odds of patient satisfaction. The magnitude of the association was greater with the perception of overcrowded (OR = 0.49; 95% CL: 0.38, 0.63) than severely overcrowded (OR = 0.73; 95% CL: 0.56, 0.97). Figure 3 further illustrates the relation between ED crowding and patient satisfaction resulting in an overall downward trend across levels of crowding and patient perceptions exhibiting a nonlinear pattern compared to the objective measures of ED crowding. Figure 2 Assessments of agreement between NEDOCS, SONET and patient perception of ED crowding using Bland–Altman plots. Table 2 Adjusted OR and 95% CLs for the association between ED crowding and real-time patient satisfaction across different assessment modes Emergency department crowding assessments ED crowding Not overcrowded Overcrowded Severely overcrowded Age Sex (male vs. female) Race/ethnicity NH black vs. NH white Hispanic vs. NH white Acuity level ESI 1 ESI 2 ESI 3 ESI 4 ESI 5 Patient perception AOR (LCL, UCL) NEDOCS AOR (LCL, UCL) SONET AOR (LCL, UCL) Reference 0.49 (0.38, 0.63) 0.73 (0.56, 0.97) 1.01 (1.00, 1.01) 1.10 (0.93, 1.30) Reference 0.78 (0.65, 0.95) 0.79 (0.61, 1.01) 1.01 (1.00, 1.01) 1.06 (0.89, 1.25) Reference 0.82 (0.69, 0.98) 0.78 (0.51, 1.18) 1.01 (1.00, 1.01) 1.05 (0.89, 1.25) 1.33 (1.09, 1.63) 1.07 (0.87, 1.31) 1.32 (1.08, 1.62) 1.04 (0.84, 1.29) 1.32 (1.08, 1.61) 1.04 (0.84, 1.28) Reference 1.00 (0.47, 2.13) 1.04 (0.49, 2.21) 1.08 (0.48, 2.43) 0.81 (0.29, 2.26) Reference 0.88 (0.41, 1.87) 0.92 (0.42, 1.92) 0.92 (0.39, 2.01) 0.73 (0.25, 2.03) Reference 0.90 (0.41, 1.93) 0.94 (0.44, 1.99) 0.95 (0.42, 2.14) 0.77 (0.27, 2.18) AOR, adjusted odds ratio; LCL, 95% lower conﬁdence limit; UCL, 95% upper conﬁdence limit. Wang et al. 726 Figure 3 Conditional means of patient satisfaction across NEDOCS, SONET and patient perception of ED crowding. Discussion Our results suggest that ED crowding, regardless of determination method (e.g. patient perceptions, NEDOCS or SONET), was associated with patient satisfaction even after adjustment for potential confounders. There was heterogeneity between the associations of crowding and patient satisfaction, where patient perception of crowding seemed to result in more pronounced impact on overall patient satisfaction than that noted with either objective scores (NEDOCS and SONET). We observed varying degrees of agreement between the three measures of crowding, in which the two objective measures exhibited the strongest agreement. One of the most notable ﬁndings of this study was that patient perception of ED crowding was associated with varying levels of realtime patient satisfaction. Given that no standard measurement has been developed to accurately measure ED crowding, we included patient perceptions of crowding as a study modality. Discrepancies were noted when comparing crowding estimation tools (NEDOCS and SONET) and patient perceptions of ED crowding. This suggests that patient perception of ED crowding may not be a reliable marker, although relative overcrowding as perceived by patients, and families, may negatively impact overall patient satisfaction. To date, no studies have determined whether a direct link exists between patient perceptions of ED crowding and their overall realtime ED satisfaction. Ours is the ﬁrst to investigate the heterogeneity of different ED crowding assessment modalities. Our ﬁndings support the literature in terms of improved understanding of potential associations between ED crowding and real-time patient satisfaction that differs from other traditional satisfaction surveys currently on the market. We believe that interface or integration of real-time patient satisfaction surveys to the Electronic Health Record (EHR) can provide meaningful feedback regarding improved delivery of quality health care. Several factors could inﬂuence the measurement of patient satisfaction, including the manner in which and/or timing whereby the satisfaction survey is administered (e.g. real-time vs. after-care timing; phone vs. Internet vs. mail survey methods; third party vs. healthcare personnel administrator, etc.). Different survey techniques have their advantages and disadvantages. Real-time satisfaction surveys can reduce issues with recall bias and have higher completion rates , two factors which might produce lower reported satisfaction scores. On the other hand, some other risks could result in relatively higher real-time satisfaction scores including fear that the survey may not truly be anonymous; or concerns that after-care arrangements and management may be negatively affected by overly critical survey responses. Consideration of such risks, though not substantially validated, may, in part, explain the overall high satisfaction scores noted in this study. This study is not without limitation. First, this was a prospective observational single-center study in which patients were not randomly selected for participation. The nonrandomization of study participants may have introduced confounders not previously accounted for. Second, we anticipated enrolling all patients during the study period. However, only approximately 10% of ED patients were included in the evaluable sample yielding a smaller sample than originally expected. This was due to numerous factors including: patient refusal to participate; inability to initiate the survey due to less efﬁcient patient ﬂow (e.g. elevated real-time ED census, admitted patents holding in ED, etc.); unavailability of unit clerk or study coordinator; and patient status (i.e. severity of illness). If the perceptions of crowding and patient satisfaction among patients who did not complete the survey differed from our study sample then our estimates would be subject to greater bias. Though selfadministered, mailed surveys are a common methodology to assess patient satisfaction, one may argue that this modality presents challenges, such as obtaining accurate mailing addresses . The study institution serves a unique patient population, consisting primarily of high transit, homeless, under- and uninsured patients, resulting in incomplete EHRs and unreliable patient contact information for address and/or telephone number which make our real-time survey response rate relatively higher than the traditional mailed out satisfaction surveys. Given that the current study is the ﬁrst of its kind, examining the potential association between ED crowding and real- ED crowding and patient satisfaction • Patient Satisfaction time patient satisfaction, we feel the low response rate is acceptable. However, future research is warranted to externally validate the real-time survey tool and its association with ED crowding. Third, the crowding deﬁnition in the patient satisfaction survey is not explained in detail and could further bias the study results. The magnitude and direction of this bias is unclear due to the unknown variability of patient responses; however, we believe that our study does provide useful information to help support future research which may examine associations between patient perceptions and patient reported outcomes. Fourth, no standard ED crowding measurement currently exists. Our study addresses agreement between measurements, not measurement accuracy. Additionally, patient satisfaction results were not compared between real-time and traditional surveys; other factors that might affect overall patient satisfaction (e.g. patients with psychosocial risks) were not investigated in this study; and, we were unable to address different perceptions of crowding relative to varying severity of patient illness and assigned level of acuity. We are still uncertain as to whether a change in the crowding level during an ED visit via planned interventions inﬂuences patient perceptions of crowding or if it could subsequently affect patient satisfaction scores. Fifth, because the concept of utilizing real-time ED patient satisfaction surveys as a metric for describing perceived ED crowding is novel, we did not employ a validated real-time survey tool. Sixth, to ensure we complied with copy right laws afforded to conventional patient satisfaction survey instruments and because data from these tools are not collected in real-time (e.g. Press-Ganey and NRC Picker), a modiﬁed version was employed. Future research, aimed at large-scale, multisite collaboration is needed to validate the use of a modiﬁed, realtime patient satisfaction survey to describe perceived ED crowding among ED patients. In summary, ED overcrowding at admission might be associated with lower real-time patient satisfaction scores. While there may be varying degrees of agreement between objective and patient perceived assessments of crowding, we observed inverse associations with patient satisfaction throughout. Future studies would beneﬁt by focusing on investigation of the (i) association between dynamic changes in ED crowding levels, interventions to minimize ED overcrowding and overall patient satisfaction and (ii) direction of crowding change and its inﬂuence on patient perception of ED crowding with and/or without interventions and overall patient satisfaction. Supplementary material Supplementary material is available at International Journal for Quality in Health Care online. References 1. Pitts SR, Pines JM, Handrigan MT et al. National trends in emergency department occupancy, 2001 to 2008: effect of inpatient admissions versus emergency department practice intensity. Ann Emerg Med 2012;60: 679–86. 2. Twanmoh JR, Cunningham GP. When overcrowding paralyzes an emergency department. Manag Care 2006;15:54–9. 3. McCarthy ML, Aronsky D, Jones ID et al. The emergency department occupancy rate: a simple measure of emergency department crowding? Ann Emerg Med 2008;51:15–24. 24.e1–2. 727 4. Weiss SJ, Derlet R, Arndahl J et al. Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS). Acad Emerg Med 2004; 11:38–50. 5. Bernstein SL, Verghese V, Leung W et al. Development and validation of a new index to measure emergency department crowding. Acad Emerg Med 2003;10:938–42. 6. Wang H, Robinson RD, Garrett JS et al. Use of the SONET score to evaluate high volume emergency department overcrowding: a prospective derivation and validation study. Emerg Med Int 2015;2015:401757. 7. Weiss SJ, Ernst AA, Derlet R et al. Relationship between the National ED Overcrowding Scale and the number of patients who leave without being seen in an academic ED. Am J Emerg Med 2005;23:288–94. 8. Hoot NR, Zhou C, Jones I et al. Measuring and forecasting emergency department crowding in real time. Ann Emerg Med 2007;49:747–55. 9. Weiss SJ, Ernst AA, Nick TG. Comparison of the National Emergency Department Overcrowding Scale and the Emergency Department Work Index for quantifying emergency department crowding. Acad Emerg Med 2006;13:513–8. 10. Jones SS, Allen TL, Flottemesch TJ et al. An independent evaluation of four quantitative emergency department crowding scales. Acad Emerg Med 2006;13:1204–11. 11. Bernstein SL, Aronsky D, Duseja R et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med 2009; 16:1–10. 12. Pines JM, Iyer S, Disbot M et al. The effect of emergency department crowding on patient satisfaction for admitted patients. Acad Emerg Med 2008;15:825–31. 13. Pines JM, Garson C, Baxt WG et al. ED crowding is associated with variable perceptions of care compromise. Acad Emerg Med 2007;14:1176–81. 14. Khursheed M, Fayyaz J, Zia N et al. Real-time patient satisfaction of Emergency Department services in a Tertiary-Care Hospital in Karachi, Pakistan. Emerg Med 2014;4:188. 15. Quinn GP, Jacobsen PB, Albrecht TL et al. Real-time patient satisfaction survey and improvement process. Hosp Top 2004;82:26–32. 16. Indovina K, Keniston A, Reid M et al. Real-time patient experience surveys of hospitalized medical patients. J Hosp Med 2016;11:251–6. 17. Ryan CM, Lee AF, Kazis LE et al. Is real-time feedback of burn-speciﬁc patient-reported outcome measures in clinical settings practical and useful? A pilot study implementing the young adult burn outcome questionnaire. J Burn Care Res 2016;37:64–74. 18. Pham JC, Trueger NS, Hilton J et al. Interventions to improve patientcentered care during times of emergency department crowding. Acad Emerg Med 2011;18:1289–94. 19. Weiss SJ, Ernst AA, Nick TG. Comparison of the National Emergency Department Overcrowding Scale and the Emergency Department Work Index for quantifying emergency department crowding. Acad Emerg Med 2006;13:513–8. 20. Wang H, Robinson RD, Bunch K et al. The inaccuracy of determining overcrowding status by using the national ED overcrowding study tool. Am J Emerg Med 2014;32:1230–6. 21. Topacoglu H, Karcioglu O, Ozucelik N et al. Analysis of factors affecting satisfaction in the emergency department: a survey of 1019 patients. Adv Ther 2004;21:380–8. 22. Demir C, Celik Y. Determinants of patient satisfaction in a military teaching hospital. J Healthc Qual 2002;24:30–4. 23. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–10. 24. Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology 1999;10:37–48. 25. EmCare. Real-time patient satisfaction tool provides better training and support to the ED. 2016. Ref Type: Internet Communication 26. Yarris LM, Fu R, Duby R et al. Comparison of mailed vs. on-site emergency department patient satisfaction surveys. J Emerg Med 2014;47:702–9.