Voice recognition for radiology reporting: Is it good enough?

doi:10.1016/j.crad.2005.07.002

Clinical Radiology

Volume 60, Issue 11, November 2005, Pages 1205-1212

https://doi.org/10.1016/j.crad.2005.07.002 Get rights and content

AIM

To compare the efficiency and accuracy of radiology reports generated by voice recognition (VR) against the traditional tape dictation–transcription (DT) method.

MATERIALS AND METHODS

Two hundred and twenty previously reported computed radiography (CR) and cross-sectional imaging (CSI) examinations were separately entered into the Radiology Information System (RIS) using both VR and DT. The times taken and errors found in the reports were compared using univariate analyses based upon the sign-test, and a general linear model constructed to examine the mean differences between the two methods.

RESULTS

There were significant reductions (p<0.001) in the mean difference in the reporting times using VR compared with DT for the two reporting methods assessed (CR, +67.4; CSI, +122.1 s). There was a significant increase in the mean difference in the actual radiologist times using VR compared with DT in the CSI reports; −14.3 s, p=0.037 (more experienced user); −13.7 s, p=0.014 (less experienced user). There were significantly more total and major errors when using VR compared with DT for CR reports (−0.25 and −0.26, respectively), and in total errors for CSI (−0.75, p<0.001), but no difference in major errors (−0.16, p=0.168). Although there were significantly more errors with VR in the less experienced group of users (mean difference in total errors −0.90, and major errors −0.40, p<0.001), there was no significant difference in the more experienced (p=0.419 and p=0.814, respectively).

CONCLUSIONS

VR is a viable reporting method for experienced users, with a quicker overall report production time (despite an increase in the radiologists' time) and a tendency to more errors for inexperienced users.

Introduction

Over the previous decade, there has been an increase in both the number and complexity of radiology reports. Historically, reports have been produced by several methods: Direct dictation to a secretary, handwritten onto the back of the request form for subsequent typing, and dictation on a tape handset with subsequent transcription (tape dictation–transcription, DT). These methods are time-consuming for the radiologist, expensive in terms of transcriptionist costs, and may lead to inevitable delays in the availability of the reports. Newer methods are being tried and there have been major advances in speech recognition systems. Digital dictation with subsequent speech recognition and corrections done by clerical staff (Philips SP6000) has been reported as a good alternative for report generation when compared with word-processed reports performed by the radiologist.¹ Automated speech recognition (ASR MediSpeak) has been compared with conventional DT in CT report generation. The errors made were comparable, and the report turnaround time was reduced, although the time taken by the radiologist increased.²

Guidelines issued by the Royal College of Radiologists (RCR) suggest that all reports should ideally be proofread and authorized immediately at the time of transcription and whilst reviewing the image being reported,³ in order to minimize the clinical risk from errors. This is not possible with the current working practices of many radiology departments where DT methods are in use. Few would have time to be able to review the images whilst proofreading. The guidelines state that departments need to seek ways of developing immediate transcription methods to minimize this source of risk.

The Norfolk and Norwich University Hospital is a large teaching hospital performing over 250,000 radiology examinations annually. DT is the most frequent method of report transcription, but with this we have experienced delays of up to 7 working days from dictation of the report by the radiologist and its subsequent transcription and availability on the hospital information systems. This was due to clerical shortages and constant interruptions by clinicians chasing reports that were still “on tape”. The situation has changed markedly since the introduction of voice recognition (VR) reporting 3 years ago. An initial comparison of VR and DT reporting here suggested VR had both a time and error advantage, although limited by study size.⁴ We performed a study to compare reporting times and error rates for the two techniques in our department, to assess the efficiency and accuracy of both methods.

Section snippets

Methods and materials

The Norfolk and Norwich University Hospital used a hospital-wide PACS (GE Pathspeed) and McKesson Total Care Radiology Information System (McKesson TC-rad v8.9 RIS). Well-developed software was used (GHG VoicePro for Radiology) to provide an interface between RIS and the VR speech engine (Dragon NaturallySpeaking Professional v.5). GHG customized the VR reporting system to enable RIS to function using both command mode of VR (to navigate around the system), as well as text mode (to talk the

Results

Of the 220 reports used in the study 160 (73%) were for CR, the remaining for CSI. For 99 reports (45%) VR was carried out first. The average word length was 53 words.

Discussion

The present study demonstrates a total reporting time advantage of VR over DT for long reports, and this time advantage remains when adjusted for length of report, irrespective of the reporting method or experience of VR use. The DT method is complex, involving several stages and individuals in the pathway. The VR method incorporates the dictation and transcription into one stage. The report turnaround time (time from the end of the examination to the time the report is available on RIS) is

References (7)

H. Arndt et al.
Anwendung der digitalen Spracherkennung in der radiologischen routine (use of digital speech recognition in diagnostic radiology)
Fortschr Rontgenstr
(1999)
U.K.M. Teichgraber et al.
Digitale Spracherkennung bei der erfassung computertomograthischer befundtexte (automated speech recognition for report generation in computed tomography)
Fortschr Rontgenstr
(1999)
Risk management in clinical radiology
(2002)

There are more references available in the full text version of this article.

Cited by (46)

XReport: An online structured reporting platform for radiologists
2022, SoftwareX
Currently the most widespread way of reporting in radiology is dictation mainly due to performance benefits. The output of this method is plain text, which varies in style (structure, nomenclature, abbreviations, etc.) and content between doctors even when reporting the exact same case. Templated radiology provides a structure for reporting and aims to help in generating more unified reports. We propose a web-based system for creating and using radiological structured reporting templates.
We developed our software based on web technologies. We wrote the system with modular design in mind. We have separate libraries for the different functionalities: a rendering library which renders the templates based on a schema, an editor library which handles template creation, and an evaluator library, which parses, and executes our custom domain specific language, FormScript, which enables dynamic behavior in our templates. We also developed a Single Page Application to create, browse, use and share templating reports. The backend of the application is powered by Firebase from Google.
We deployed our system at a publicly accessible domain at https://app.radiosheets.com.
Benefits of Integrated RIS/PACS/Reporting Due to Automatic Population of Templated Reports
2019, Current Problems in Diagnostic Radiology
Citation Excerpt :
Regarding time savings, report automation allows the radiologist to spend more time per examination reviewing the images and synthesizing findings into a clinically relevant impression, rather than on dictating automatable noninterpretive but required fields. The use of templates, along with experience in VR, has been shown to increase the efficiency of dictation.7,8 No studies, however, have demonstrated the effects of a fully automated template system by which all noninterpretive fields are automatically populated.
With integration of the Radiological Information Systems (RIS), Picture Archiving and Communication systems (PACS), and reporting systems, patient identifiers and examination information can automatically map into examination reports. There are many potential benefits of report automation to radiologists including improvements in efficiency, accuracy, and fatigue. In this article we describe a 2 part study, with the first part being an anonymous survey of radiologists concerning report automation. A total of 13 staff radiologists and 9 radiology residents at a single institution completed an anonymous survey. Respondents were asked if automatic population of examination description, comparison examination data, indications, computed tomography dose, technique, and copy to physician data saved time, decreased fatigue, and increased accuracy. Respondents were asked if a “copy findings” function saved time. The second objective part of the study was a mock examination experiment to assess time savings of report automation and to assess error rates. Of all, 9 radiologists were asked to dictate fields for 8 mock examinations. Subjects were timed and reporting errors monitored. Estimated daily time savings and error rates were calculated assuming a mix of 80 studies. A total of 95% surveyed responded that report automation saved time; 91% that report automation improved accuracy of dictations; 82% that report automation decreased fatigue. Furthermore, 83% of copy finding function users reported time savings. Average time to dictate these prepopulated fields was 51 seconds per study. Average error rate per report was 0.86, with an average of 0.26 errors remaining uncorrected upon report completion. Estimated average time per day saved per radiologist from report automation was 68 minutes. Estimated average corrected errors was 48 per day. Estimated average uncorrected or missed errors was 21 per day. These estimated benefits from report automation result from tight integration of RIS, PACS, and reporting systems.
Adapting a Computerized Medical Dictation System to Prepare Academic Papers in Radiology
2018, Current Problems in Diagnostic Radiology
Everyday radiologists use dictation software to compose clinical reports of imaging findings. The dictation software is tailored for medical use and to the speech pattern of each radiologist. Over the past 10 years we have used dictation software to compose academic manuscripts, correspondence letters, and texts of educational exhibits. The advantages of using voice dictation is faster composition of manuscripts. However, use of such software requires preparation. The purpose of this article is to review the steps of adapting a clinical dictation software for dictating academic manuscripts and detail the advantages and limitations of this technique.
Laterality errors in radiology reports generated with and without voice recognition software: Frequency and clinical significance
2013, Journal of the American College of Radiology
Citation Excerpt :
Previous studies have evaluated the use of voice recognition software in radiology [7,10-18], but the majority focused on cost-effectiveness and reporting time. Of those analyzing accuracy, several found increases in the frequency of errors without the use of voice recognition [10,11,14]. However, these studies evaluated for errors in syntax, grammar, spelling, and so on, and did not specifically address laterality errors, which are more likely to be clinically relevant.
The aim of this study was to determine the incidence, types, and clinical implications of laterality errors and the effect of voice recognition software on the frequency of laterality errors.
All radiology reports generated between January 2007 and April 2011 were retrospectively evaluated to identify revised reports containing laterality errors. Type of error was catalogued with regard to modality, body part, type of discrepancy (major or minor, with discrepancies considered major if the potential existed to affect patient management), duration of time between report finalization and corrected report, clinical significance, and use of voice recognition. The rate of errors causing major and minor discrepancies between voice recognition–generated reports and nonvoice recognition–generated reports was compared.
Among 2,923,094 reports, 1,607 (0.055%) contained corrected laterality errors, and 56 (0.0019% of the total report volume) were major. A total of 584,878 (20%) were generated using voice recognition. The rate of laterality errors leading to major discrepancies in voice recognition–generated reports was 0.00188%, compared with 0.00192% in nonvoice recognition–generated reports (P = .9436). None of the errors led to wrong-sided surgery. However, there were potential adverse effects due to laterality errors in 3 patients with major discrepancies (0.000103% of the total report volume).
Rates of laterality errors were low and, in our population, did not result in wrong-sided surgeries. Rates of laterality errors in reports with major discrepancies were unaffected by voice recognition software, but voice recognition was associated with a significant reduction in the duration of time between report finalization and the issuing of a corrected report.
A novel report generation approach for medical applications: The SISDS methodology and its applications
2013, International Journal of Medical Informatics
Despite exciting innovation in information system technologies, the medical reporting has remained static for a long time. Structured reporting was established to address the deficiencies in report content but has largely failed in its adoption due to concerns over workflow and productivity. The methods used in medical reporting are insufficient in providing with information for statistical processing and medical decision making as well as high quality healthcare.
The aim of this study is to introduce a novel method that enables professionals to efficiently produce medical reports that are less error-prone and can be used in decision support systems without extensive post-processing.
We first present the formal definition of the proposed method, called SISDS, that provides a clear separation between the data, logic and presentation layers. It allows free-text like structured data entry in a structured form, and reduces the cognitive effort by inline editing and dynamically controlling the information flow based on the entered data. Then, we validate the usability and reliability of the method on a real-world testbed in the field of radiology. For this purpose, a sample esophagus report was constructed by a focus group of radiologists and real patient data have been collected using a web-based prototype; these data are then used to build a decision support system with off-the-shelf tools. The usability of the method is assessed by evaluating its acceptability by the users and the accuracy of the resulting decision support system. For reliability, we conducted a controlled experiment comparing the performance of the method to that of transcriptionist-oriented systems in terms of the rate of successful diagnosis and the total time required to enter the data.
The most noticeable observation in the evaluation is that the rate of successful diagnosis improves significantly with the proposed method; in our case study, a success rate of 81.25% has been achieved by using the SISDS method compared to 43.75% for the transcriptionist-oriented system. In addition, the average time required to obtain the final approved reports decreased from 29 min to 14 min. Based on questionnaire responses, the acceptance rate of the SISDS methodology by users is also found to outperform the rates of the current methods.
The empirical results show that the method can effectively help to reduce medical errors, increase data quality, and lead to more accurate decision support. In addition, the dynamic hierarchical data entry model proves to provide a good balance between cognitive load and structured data collection.
The radiology report - Are we getting the message across?
2011, Clinical Radiology
Citation Excerpt :
LaFortune et al. argued that a court could easily find the radiologist responsible for harm done to a patient as a result of a careless unchecked report and adds that the use of phrases such as “dictated but not checked” and other phrases to limit responsibility also reduce the reliability of a report.17 Although voice recognition is a viable method of reporting for experienced users, with a quicker overall report time, this comes at the price of an increase in the radiologists’ time and a tendency to more errors for inexperienced users.43 Other errors (such as grammatical or spelling errors) can also make the radiologist look rather careless, as demonstrated in Table 8, which are all excerpts from verified radiology reports.
The radiology report is the primary method of communication between radiologist and referrer. Despite this, radiologists receive very little formal training regarding the structure of the radiology report and also its importance as a medico-legal document. We present a review of radiology reporting, highlighting the importance of report structure and language with the purpose of helping radiologists improve the clarity, brevity, pertinence, and readability of reports. We encourage radiologists to avoid hedging and strive to improve communication with referring clinicians.

View all citing articles on Scopus

View full text