TcmYiAnBERT:基于无监督学习的中医医案预训练模型

目的/意义 充分挖掘中医医案中的文本信息,提高中医药信息化程度和中医医案症状术语抽取、关系抽取等下游任务的准确率。方法/过程 通过光学字符识别和爬虫技术获取大量中医医案数据并进行预处理,构建面向中医医案领域预训练数据集,使用BERT模型预训练方法,经过多轮训练得到首个面向中医领域专有预训练模型TcmYiAnBERT,并将该模型开源。结果/结论 中医领域专有预训练模型TcmYiAnBERT在中医命名实体识别任务中比未使用该模型的预训练模型F1值提高2.8个百分点。

TcmYiAnBERT:A Traditional Chinese Medicine Case Pre-training Model Based on Unsupervised Learning

Purpose/Significance To fully mine the text information in traditional Chinese medicine (TCM) medical records, to improve the degree of TCM informatization, and to improve the accuracy of downstream tasks such as symptom term extraction and relationship extraction in TCM medical records.Method/Process A large number of TCM medical case data are obtained through optical character recognition (OCR) technology and crawler technology, and data preprocessing is carried out. A pre-training data set for TCM medical case field is constructed. The first proprietary pre-training model, namely TcmYiAnBERT, for TCM field is obtained through multiple rounds of training by using the BERT model pre-training method, and the model is open source. Result/Conclusion The experiment shows that the recognition accuracy of TCM domain specific pre-training model TcmYiAnBERT in the task of TCM named entity recognition (NER) is 2.8 percentage points higher than that of other pre-training models.