Design and implementation of big data ad-hoc queries and statistics subsystem for miliary electronic health records
10.7644/j.issn.1674-9960.2017.12.014
- VernacularTitle:军人电子健康档案大数据即席查询统计子系统的设计与实现
- Author:
Chen-Yang CHI
1
;
Hai-Bin MENG
;
Dong-Liang QIN
;
Cheng QIAN
;
Dong-Sheng ZHAO
;
Hua-Jian MAO
Author Information
1. 军事科学院军事医学研究院
- Keywords:
military electronic health records;
ad-hoc queries and statistics of big data;
CarbonData columnar storage format;
Spark SQL interactive processing framework;
medical records systems,computerized
- From:
Military Medical Sciences
2017;41(12):1009-1012
- CountryChina
- Language:Chinese
-
Abstract:
Objective To improve the analysis efficiency and interactive experience of the Military Electronic Health Records System(MEHRS)and to realize quick response of ad-hoc queries and statistics in the MEHRS with big data columnar storage and processing technologies.Methods We carried out requirement analysis and functional design of the ad-hoc queries and statistics subsystem of the MEHRS,proposed a three-tier architecture which included the archive storage layer,statistical pretreatment layer and statistical application layer.After the selection and evaluation of big data processing technologies,CarbonData columnar storage was used to store preprocessed data and executed statistics with Spark SQL on the basis of medical business data modeling and preprocessing.Results Five testing tasks were executed on two million archives in the following two subsystems:one with modeless and non-preprocessed MongoDB storage,the other with modeled and preprocessed CarbonData storage.The latter could finish these tasks within seconds and was dozens of times more efficient than the former statistically.Conclusion This study designs and implements a big data technology proposal that satisfies the quick response of ad-hoc queries and statistics in the MEHRS, providing powerful and flexible technical support for big data statistical analysis.
