We present a new text-to-SQL dataset for electronic health records (EHRs). The utterances were collected from 222 hospital staff, including physicians, nurses, insurance review and health records teams, and more. To construct the QA dataset on structured EHR data, we conducted a poll at a university hospital and templatized the responses to create seed questions. Then, we manually linked them to two open-source EHR databases, MIMIC-III and eICU, and included them with various time expressions and held-out unanswerable questions in the dataset, which were all collected from the poll. Our dataset poses a unique set of challenges: the model needs to 1) generate SQL queries that reflect a wide range of needs in the hospital, including simple retrieval and complex operations such as calculating survival rate, 2) understand various time expressions to answer time-sensitive questions in healthcare, and 3) distinguish whether a given question is answerable or unanswerable based on the prediction confidence. We believe our dataset, EHRSQL, could serve as a practical benchmark to develop and assess QA models on structured EHR data and take one step further towards bridging the gap between text-to-SQL research and its real-life deployment in healthcare. EHRSQL is available at https://github.com/glee4810/EHRSQL.
翻译:我们为电子健康记录(EHRs)提供了一个新的文本到SQL数据集。发言来自222名医院工作人员,包括医生、护士、保险审查和健康记录小组等222名医院工作人员,收集了一套独有的挑战:模型需求:1)生成了反映医院广泛需要的SQL查询,包括简单的检索和复杂的操作,如计算生存率,2)了解在保健中回答时间敏感问题的各种时间表达方式,3)根据预测信心区分一个问题是否可以回答或无法回答。我们认为,我们的数据集,即EHRSQL,可以作为开发和评估EHRA结构化数据模型的实用基准,并且进一步迈出了在ERHR-QRQ/QRQQ上构建和在EHR-QRQ/HR-QQ/QRGRM-Q的文本研究之间缩小差距的一步。