Voice recognition software versus a traditional transcription service for physician charting in the ED

doi:10.1053/ajem.2001.24487

The American Journal of Emergency Medicine

Volume 19, Issue 4, July 2001, Pages 295-298

https://doi.org/10.1053/ajem.2001.24487 Get rights and content

Abstract

This study was conducted to compare real-time voice recognition software to a traditional transcription service. Two emergency department (ED) physicians dictated 47 charts using a voice dictation software program and a traditional transcription service. Accuracy, word per minute dictation time and turnaround time were calculated from the data. The transcription service used in our study was more accurate than the voice recognition program with an accuracy of 99.7 percent versus 98.5 percent for the voice recognition program. The average number of corrections per chart was 2.5 for the voice recognition program and 1.2 for the traditional transcription service. Turnaround time was much better using the computer voice recognition program with an average turnaround time of 3.65 minutes versus a turnaround time of 39.6 minutes for the traditionally transcribed charts. The charts dictated using the voice recognition program were considerably less costly than the manually transcribed charts. In summary, computer voice recognition is nearly as accurate as traditional transcription, it has a much shorter turnaround time and is less expensive than traditional transcription. We recommend its use as a tool for physician charting in the ED. (Am J Emerg Med 2001; 19:295-298. Copyright © 2001 by W.B. Saunders Company)

Section snippets

Methods

A total of 47 charts were dictated by 2 ED physicians at a suburban level 1 trauma center with an annual census of 45,000. One of the physicians was an “advanced” user, having several years of experience with and having dictated several hundred charts with the software. The second physician was a “basic” user with approximately 2 weeks experience and having dictated approximately 20 charts with the software.

Dragon NaturallySpeaking Medical suite version 4 was installed onto a 450 MHz Pentium II

Results

Our data comparing the voice recognition program to the traditional transcription service with regard to accuracy, average number of errors, average turnaround time and word per minute dictation time is listed in Table 1.

. Comparison of Voice Recognition and Traditional Transcription Service Dictations

Empty Cell	Voice Recognition (95% CI)	Transcription (95% CI)	Difference (95% CI)
Accuracy (%)	98.5 (98.2-98.9)	99.7 (99.6-99.8)	1.2 (0.8-1.5)
Average no. errors/chart	2.5 (2-3)	1.2 (0.9-1.5)	1.3 (.67-1.88)
Average

Discussion

There are several ways to create ED records: handwritten charts, handwritten templates, traditional dictation services, and now computer-generated voice recognition systems. Handwritten charts are time-consuming, fatiguing, and often difficult to read. Marill, et al found that handwritten template charts (ie, the T system ) is associated with higher gross billing and physician satisfaction, but no significant decrease in emergency physician total evaluation time.¹ Traditionally transcribed

Conclusion

Computer voice recognition transcription using real-time voice recognition software is an economical and accurate way to generate ED records. The software is nearly as accurate as traditional transcription, it has a much shorter turnaround time and it is less expensive. We recommend it's use as a tool for physician charting in the ED.

Acknowledgements

Special thanks to Douglas Propp, MD for obtaining the hardware for this study, and also special thanks to Myrna Greenfield and Dragon Systems Inc. for donating the software for use in this study.

References (4)

K Marill et al.
Prospective randomized trial of template-assisted versus undirected written recording of physician records in the emergency department
Ann Emerg Med
(1999)
NJ Zemmel et al.
Status of voicetype dictation for windows for the emergency physician
J Emerg Med
(1996)

There are more references available in the full text version of this article.

Cited by (58)

Automatic documentation of professional health interactions: A systematic review
2023, Artificial Intelligence in Medicine
Electronic systems are increasingly present in the healthcare system and are often related to improved medical care. However, the widespread use of these technologies ended up building a relationship of dependence that can disrupt the doctor–patient relationship. In this context, digital scribes are automated clinical documentation systems that capture the physician–patient conversation and then generate the documentation for the appointment, enabling the physician to engage with the patient entirely. We have performed a systematic literature review on intelligent solutions for automatic speech recognition (ASR) with automatic documentation during a medical interview. The scope included only original research on systems that could detect speech and transcribe it in a natural and structured fashion simultaneously with the doctor–patient interaction, excluding speech-to-text-only technologies. The search resulted in a total of 1995 titles, with eight articles remaining after filtering for the inclusion and exclusion criteria. The intelligent models mainly consisted of an ASR system with natural language processing capability, a medical lexicon, and structured text output. None of the articles had a commercially available product at the time of the publication and reported limited real-life experience. So far, none of the applications has been prospectively validated and tested in large-scale clinical studies. Nonetheless, these first reports suggest that automatic speech recognition may be a valuable tool in the future to facilitate medical registration in a faster and more reliable manner. Improving transparency, accuracy, and empathy could drastically change how patients and doctors experience a medical visit. Unfortunately, clinical data on the usability and benefits of such applications is almost non-existent. We believe that future work in this area is necessary and needed.
Physician use of speech recognition versus typing in clinical documentation: A controlled observational study
2020, International Journal of Medical Informatics
Citation Excerpt :
However, the research remains largely heterogenous and often uses different evaluation metrics, making it challenging to compare findings over time and across studies. Many observational studies were conducted over a decade ago and as such may have limited applicability to modern SR software, which shows improved speed and accuracy over earlier systems [18–24]. Among more recent studies, findings regarding the impact of SR on accuracy, efficiency and provider satisfaction remain mixed [25–29].
Speech recognition (SR) is increasingly used directly by clinicians for electronic health record (EHR) documentation. Its usability and effect on quality and efficiency versus other documentation methods remain unclear.
To study usability and quality of documentation with SR versus typing.
In this controlled observational study, each subject participated in two of five simulated outpatient scenarios. Sessions were recorded with Morae® usability software. Two notes were documented into the EHR per encounter (one dictated, one typed) in randomized order. Participants were interviewed about each method’s perceived advantages and disadvantages. Demographics and documentation habits were collected via survey. Data collection occurred between January 8 and February 8, 2019, and data analysis was conducted from February through September of 2019.
Brigham and Women’s Hospital, Boston, Massachusetts, USA.
Ten physicians who had used SR for at least six months.
Documentation time, word count, vocabulary size, number of errors, number of corrections and quality (clarity, completeness, concision, information sufficiency and prioritization).
Dictated notes were longer than typed notes (320.6 vs. 180.8 words; p = 0.004) with more unique words (170.9 vs. 120.4; p = 0.01). Documentation time was similar between methods, with dictated notes taking slightly less time to complete than typed notes. Typed notes had more uncorrected errors per note than dictated notes (2.9 vs. 1.5), although most were minor misspellings. Dictated notes had a higher mean quality score (7.7 vs. 6.6; p = 0.04), were more complete and included more sufficient information.
Participants felt that SR saves them time, increases their efficiency and allows them to quickly document more relevant details. Quality analysis supports the perception that SR allows for more detailed notes, but whether dictation is objectively faster than typing remains unclear, and participants described some scenarios where typing is still preferred. Dictation can be effective for creating comprehensive documentation, especially when physicians like and feel comfortable using SR. Research is needed to further improve integration of SR with EHR systems and assess its impact on clinical practice, workflows, provider and patient experience, and costs.
A clinician survey of using speech recognition for clinical documentation in the electronic health record
2019, International Journal of Medical Informatics
Citation Excerpt :
In this study, most users estimated seeing fewer than 10 errors per document, with less than 25% of those being clinically significant. These estimates are lower than the error rates reported in many formal evaluations of SR accuracy [28–30]. In previous studies, SR error rates (i.e., number of errors divided by the total number of words in the document) have been found to be as low as 0.3% with traditional dictation (i.e., with editing/revision by professional transcriptionists) but as high as 23–52% when clinicians use front-end SR systems (i.e. with editing/revision by clinicians) [23,24].
To assess the role of speech recognition (SR) technology in clinicians’ documentation workflows by examining use of, experience with and opinions about this technology.
We distributed a survey in 2016–2017 to 1731 clinician SR users at two large medical centers in Boston, Massachusetts and Aurora, Colorado. The survey asked about demographic and clinical characteristics, SR use and preferences, perceived accuracy, efficiency, and usability of SR, and overall satisfaction. Associations between outcomes (e.g., satisfaction) and factors (e.g., error prevalence) were measured using ordinal logistic regression.
Most respondents (65.3%) had used their SR system for under one year. 75.5% of respondents estimated seeing 10 or fewer errors per dictation, but 19.6% estimated half or more of errors were clinically significant. Although 29.4% of respondents did not include SR among their preferred documentation methods, 78.8% were satisfied with SR, and 77.2% agreed that SR improves efficiency. Satisfaction was associated positively with efficiency and negatively with error prevalence and editing time. Respondents were interested in further training about using SR effectively but expressed concerns regarding software reliability, editing and workflow.
Compared to other documentation methods (e.g., scribes, templates, typing, traditional dictation), SR has emerged as an effective solution, overcoming limitations inherent in other options and potentially improving efficiency while preserving documentation quality.
While concerns about SR usability and accuracy persist, clinicians expressed positive opinions about its impact on workflow and efficiency. Faster and better approaches are needed for clinical documentation, and SR is likely to play an important role going forward.
Incidence of speech recognition errors in the emergency department
2016, International Journal of Medical Informatics
Citation Excerpt :
Despite the advantages of SR technology, high error rates ranging from 10 to 23% have been observed in clinical documents generated by this technology [2], raising concern about the number of errors being entered into the permanent medical record, their impact on quality of care and the medical liability that may arise. To date, there have been few studies published on the use SR in ED [3–5]. A recent study by Zick et al. evaluated the accuracy and cost savings of traditional voice dictation as compared to a real-time SR software and observed high accuracies of 99.7% and 98.5% respectively [5].
Physician use of computerized speech recognition (SR) technology has risen in recent years due to its ease of use and efficiency at the point of care. However, error rates between 10 and 23% have been observed, raising concern about the number of errors being entered into the permanent medical record, their impact on quality of care and medical liability that may arise. Our aim was to determine the incidence and types of SR errors introduced by this technology in the emergency department (ED).
Level 1 emergency department with 42,000 visits/year in a tertiary academic teaching hospital.
A random sample of 100 notes dictated by attending emergency physicians (EPs) using SR software was collected from the ED electronic health record between January and June 2012. Two board-certified EPs annotated the notes and conducted error analysis independently. An existing classification schema was adopted to classify errors into eight errors types. Critical errors deemed to potentially impact patient care were identified.
There were 128 errors in total or 1.3 errors per note, and 14.8% (n = 19) errors were judged to be critical. 71% of notes contained errors, and 15% contained one or more critical errors. Annunciation errors were the highest at 53.9% (n = 69), followed by deletions at 18.0% (n = 23) and added words at 11.7% (n = 15). Nonsense errors, homonyms and spelling errors were present in 10.9% (n = 14), 4.7% (n = 6), and 0.8% (n = 1) of notes, respectively. There were no suffix or dictionary errors. Inter-annotator agreement was 97.8%.
This is the first estimate at classifying speech recognition errors in dictated emergency department notes. Speech recognition errors occur commonly with annunciation errors being the most frequent. Error rates were comparable if not lower than previous studies. 15% of errors were deemed critical, potentially leading to miscommunication that could affect patient care.
How artificial intelligence is changing health and health care
2023, Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril (2019)
Benefits, barriers, and facilitators of using speech recognition technology in nursing documentation and reporting: A cross-sectional study
2023, Health Science Reports

View all citing articles on Scopus

^☆: Returned November 26, 2000.

^☆☆: Supported in part by software donated by Dragon Systems, Inc., Newton, MA. Hardware for the study was donated by Lutheran General Hospital's Department of Emergency Medicine.

^★: Address reprint requests to Robert G. Zick, MD, MBA, 355 Ridge Ave, Evanston, IL, 60202.

^♢: 0735-6757/01/1904-0011$35.00/0

View full text

Clinical NotesVoice recognition software versus a traditional transcription service for physician charting in the ED☆,☆☆,★,★★,♢