EchoGen: A New Benchmark Study on Generating Conclusions from Echocardiogram Notes.

TitleEchoGen: A New Benchmark Study on Generating Conclusions from Echocardiogram Notes.
Publication TypeJournal Article
Year of Publication2022
AuthorsTang L, Kooragayalu S, Wang Y, Ding Y, Durrett G, Rousseau JF, Peng Y
JournalProc Conf Assoc Comput Linguist Meet
Date Published2022 May

Generating a summary from findings has been recently explored (Zhang et al., 2018, 2020) in note types such as radiology reports that typically have short length. In this work, we focus on echocardiogram notes that is longer and more complex compared to previous note types. We formally define the task of echocardiography conclusion generation (EchoGen) as generating a conclusion given the findings section, with emphasis on key cardiac findings. To promote the development of EchoGen methods, we present a new benchmark, which consists of two datasets collected from two hospitals. We further compare both standard and state-of-the-art methods on this new benchmark, with an emphasis on factual consistency. To accomplish this, we develop a tool to automatically extract concept-attribute tuples from the text. We then propose an evaluation metric, FactComp, to compare concept-attribute tuples between the human reference and generated conclusions. Both automatic and human evaluations show that there is still a significant gap between human-written and machine-generated conclusions on echo reports in terms of factuality and overall quality.

Alternate JournalProc Conf Assoc Comput Linguist Meet
PubMed ID36339656
PubMed Central IDPMC9634991
Grant ListR00 LM013001 / LM / NLM NIH HHS / United States