https://orcid.org/0000-0002-1690-6718
Abstract
The purpose of this project was to improve ease and speed of physician comprehension when interpreting daily laboratory data for patients admitted within the Military Healthcare System (MHS).
A JavaScript program was created to convert the laboratory data obtained via the outpatient electronic medical record (EMR) into a “fishbone diagram” format that is familiar to most physicians. Using a balanced crossover design, 35 internal medicine trainees and staff at Naval Medical Center Portsmouth were asked to complete timed comprehension tests for laboratory data sets formatted in the outpatient EMR’s format and in fishbone diagram format. The number of responses per second and error rate per response were measured for each format. Participants were asked to rate relative ease of use for each format and indicate which format they preferred.
Comprehension speed increased 37% (6.28 seconds per interpretation) with the fishbone diagram format with no observed increase in errors. Using a Likert scale of 1–5 (1 being hard, 5 easy), participants indicated the new format was easier to use (4.14 for fishbone vs 2.14 for table) with 89% expressing preference for the new format.
The publically available web application that converts tabular lab data to fishbone diagram format is currently used 10 000–12 000 times per month across the MHS, delivering significant benefit to the enterprise in terms of time saved and improved physician experience.
This study supports the use of fishbone diagram formatting for laboratory data for inpatients within the MHS.
Lay Summary
Laboratory data in the Military Healthcare System (MHS) are presented in a table format that does not allow for easy comparison of results, includes a large amount of extraneous information, and does not import into the inpatient medical record reliably.To address these issues, a web application was created that reformats the data from table format to a shorthand format called “fishbone diagrams” that is familiar to most physicians. The web application also removes a significant amount of extraneous information, simplifying the data in the process of reformatting it.Thirty-five internal medicine trainees and staff physicians were given 1 min to complete a series of questions regarding a similar set of lab results in each data format in a balanced crossover design. They were also asked to rate how easy they found each format to use and to express overall preference for one of the formats.Participants were able to interpret data 37% faster (6.28 seconds per interpretation) using the fishbone data format with no increase in interpretation errors. Participants consistently rated the new data format as easier to use and 89% of participants preferred it over the table format. The web application is currently used 10 000–12 000 times a month.
BACKGROUND AND SIGNIFICANCE
The use of laboratory tests to diagnose and appropriately treat illness is fundamental to the practice of medicine. As medical technology and understanding of disease have progressed, so have the number of tests available and quantity ordered by physicians.1 Since the computer and electronic medical records (EMRs) entered common use in medical documentation, systems have been developed with the intent of streamlining care, improving physician comprehension, and reducing opportunity for error.2–4
Most EMRs, including those used in the Military Healthcare System (MHS), use a tabular format to report laboratory data. This format has the strength of providing consistent formatting regardless of type of data being presented and provides a way to view a large amount of information at once but can be cumbersome when interpreting single or a small set of lab values quickly. Most physicians use, or are familiar with, a shorthand notation colloquially called “fishbone diagrams” to record complete blood counts (CBC), basic and comprehensive metabolic panels, hepatic function panels, and coagulation panels graphically. This format is typically only employed in handwritten notes.
The military health system uses EMRs, which also report laboratory data in a tabular format. A current limitation of the EMRs used is that laboratory results do not all automatically populate into inpatient notes and so are typically either manually typed into the note or copy/pasted from the outpatient EMR. To improve this process, a Health Insurance Accountability and Portability Act compliant web application was developed that reformats lab values into the more familiar fishbone diagram format when copying laboratory data from outpatient EMR to inpatient EMR. The program also removes a great deal of extraneous symbols and words, leaving only the relevant information desired by the physician. This program was developed in 2016, and since that time has spread via word of mouth to be used at military hospitals across the country with 10 000–12 000 uses each month.
The presumption underpinning this program’s creation was that simpler data presented in predictable, familiar formats would be faster to interpret and require less cognitive effort. To date, this intervention had not been assessed objectively. This project aimed to quantify the impact of this quality improvement initiative on physician comprehension speed and accuracy.
METHODS
Data formats
The 2 data formats used are presented in Figures 1 and 2.
Example of existing outpatient EMR data format. EMR: electronic medical record.
Example of fishbone diagram data format.
The table format consists of 3 columns, lab name, sample type, and lab result in that order. A single result is displayed on each row and results are color coded blue for low and red for high with additional text added after some results to indicate additional information available. The default view does not allow for multiple tests of the same type to be seen side by side, and when copied and pasted from outpatient to inpatient medical record the table format is lost entirely.
The fishbone diagram format is constructed using a simple JavaScript page that accepts copied and pasted results from the outpatient medical record, without personally identifiable information. The program identifies individual lab tests and groups them by date and time and provides the data formatted in fishbone diagrams, if applicable to the result type. All other data are formatted in a list with removal of extraneous text. The program also applies asterisks to values that fall outside the lab reference range.
Study design
Deidentified chemistry and hematology results were presented to participants using the 2 data formats (tabular and fishbone diagram) along with questionnaires requesting identification of individual values and trends. Participants completed the 2 questionnaires in a balanced crossover experiment. After completing both questionnaires, participants were asked to complete a 3-question survey rating perceived ease of use and to indicate overall preference for one of the data formats. The study was conducted during March 2022 at Naval Medical Center Portsmouth in Portsmouth, Virginia.
Participants
A total of 35 participants were recruited at a daily internal medicine residency didactic session. Participants were asked to abstain if they were unfamiliar with either data format.
Patient cases
Each laboratory data format was applied to a pair of basic metabolic panels (BMP) and a pair of CBC labeled as being from sequential days (one CBC and BMP for each day). The laboratory data were identical in quantity and type of information but individual result values used for each data format differed.
Procedure
Before the study, every participant was informed about the project and confirmed familiarity with both data formats. Participants were each given both questionnaires (one for each data format) and a survey with the lab data hidden by a cover sheet. Participants were informed they would have 60 s to answer as many questions as possible about the data set provided and then would answer a set of questions about a set of data. The questions were designed so that each questionnaire requested identical cognitive tasks in the same order. For example, question 3 asked to identify a trend on both questionnaires but one questionnaire asked about anemia, the other about renal dysfunction. The study materials were distributed randomly but were prepared such that 50% of participants had the questionnaire with data formatted using a table as the first questionnaire. The remaining 50% started the questionnaire with data formatted using fishbone diagrams. Participants completed the 2 questionnaires in the assigned order and then completed a 3-question survey.
Outcome measures
Responses were graded manually with incorrect or partially correct answers both counted as an erroneous interpretation. Omitted questions, which were rare, were not considered to have undergone interpretation, and were counted neither toward total interpretations nor as erroneous. For each questionnaire, the number of questions answered and the number of errors committed were recorded.
For the survey results, the ratings for ease of use (1–5 on a Likert scale with 5 being easy) were recorded for each data format. The data format preference of each participant was also recorded.
RESULTS
Thirty-five internal medicine staff, residents, and medical students participated, including 9 medical students, 7 intern physicians, 11 resident physicians, and 6 staff physicians (2 did not provide year of training).
A total of 154 interpretations were made using the table format and 270 interpretations for the fishbone diagram format. The average number of answers per questionnaire was 4.4 for the table format and 7.67 for the fishbone diagram format. Given that each survey was completed in 60 s, the average number of seconds per interpretation was 15.42 for table format and 9.14 for the fishbone diagram format (P < .001).
A total of 16 interpretation errors were made using the table format and 29 using the fishbone diagram format. Despite the higher quantity of errors with the fishbone diagram format, the average number of errors per interpretation was the same between the groups at 0.11 errors per response.
Using a Likert scale of 1–5 (1 being difficult, 5 being easy), respondents rated ease of use of table format an average of 2.14 and rated the fishbone diagram an average of 4.14. When asked which format was preferred, 89% of participants indicated a preference for the fishbone diagram format.
DISCUSSION
This assessment was designed to specifically test 2 different clinical systems in a use that mimics a specific clinical scenario, assessing the labs of an inpatient from 2 sequential days. With the current EMR being employed in this scenario, physicians of all levels of training were faster and no less accurate using a computer-generated version of shorthand notation. The average time per interpretation was 6.28 s faster using fishbone diagram format, a 37% increase in speed. The web application that converts data from tabular to fishbone diagram format is used an average of 10 000–12 000 times per month. With this volume of use, the modest improvement in speed can observed can add up to significant time savings across the enterprise.
These improvements may be largely attributable to the effects of simplification and grouping that the new format provides. In the process of perceiving information, the reader typically starts with separating common elements by relatedness.5 The format tested here does not change the grouping of lab results relative to the other tests being displayed, but it does display the results within a lab panel in a special arrangement that is more familiar to the physician. While perceiving information, the reader also must sift through what is presented and edit out extraneous information such as unnecessary symbols and words, and unhelpful precision (such as a test displayed to the fourth decimal).6 The new data format removes most extraneous information, which likely contributes to the improved comprehension speed observed. Simplifying the data also reduces the amount of work the reader must perform to achieve comprehension, reducing the “cognitive load” of the task.7 The result is a data set that the reader feels is easier to comprehend, which may explain the overwhelming preference among trainees and staff for fishbone diagram formatting relative to tabular format currently used in the outpatient EMR.
While successful at improving speed and ease of use, it is important to note the limitations of this project. First, the project is comparing a new improvement relative to the status quo of a specific data format unique to one clinical system within the military health system. This project is not designed to draw conclusions about physician comprehension in general, but rather to assess which of these 2 specific systems currently in use is superior for comprehension speed and ease of use. This project also examined the use of these data formats in a very specific context: comparing lab results from 2 consecutive days. This is a common task for those practicing inpatient medicine, but the observed improvements in comprehension speed may not hold true when attempting to comprehend larger data sets with lab results that span over more than a few days or that contain more than a handful of basic tests. It is possible that the fishbone diagram would still perform better, given it displays the data in a significantly smaller amount of space (and so more data can been seen simultaneously) but increasingly complex data sets may still be easier to comprehend in table format.
CONCLUSION
The use of fishbone diagram format for a small set laboratory results was associated with significantly improved speed of physician comprehension without increased rate of errors. This study supports the continued use of fishbone diagram formatting for laboratory data for reporting and interpreting daily laboratory results for admitted patients within the MHS.
MILITARY-SPECIFIC DISCLAIMERS
The views expressed in this article reflect the results of research conducted by the author and do not necessarily reflect the official policy or position of the Department of the Navy, Department of Defense, or the United States Government.
I am a military service member. This work was prepared as part of my official duties. Title 17 U.S.C. 105 provides that “Copyright protection under this title is not available for any work of the United States Government.” Title 17 U.S.C. 101 defines a United States Government work as a work prepared by a military service member or employee of the United States Government as part of that person’s official duties.
Research data derived from a Naval Medical Center Portsmouth IRB reviewed protocol deemed not research (EIRB reference 948182).
AUTHOR CONTRIBUTIONS
The listed author is the only contributor to this work and is the point of contact for any correspondence.
CONFLICT OF INTEREST STATEMENT
None declared.
DATA AVAILABILITY
Data available from the Dryad Digital Repository: https://doi.org/10.5061/dryad.mkkwh712w.
