组学大数据环境下的基因变异信息并行处理与分析

随着第二代测序技术的发展与应用,其产生的测序数据也呈现快速的增长趋势,如何有效、快速、稳定地对海量测序数据进行分析成为生物研究领域迫切的需求。目前许多传统的测序数据分析软件仅支持单一功能,并不具备完整的数据分析能力,应对海量的测序数据时其处理能力也显著不足。为了应对上述问题,本文设计了一款基于Hadoop框架的测序数据分析软件,整合了现今生物研究领域内常用的多款序列分析软件,从而实现了对测序序列数据的自动化分析。该软件输入原始的测序数据后,经过碱基质量控制、序列比对、SNP位点信息提取、突变基因信息生成等几个过程,最终输出详细的突变基因信息报告。该软件实现了自动化的数据分析,提高了数据分析的效率,极大减轻了数据分析人员的工作量。     

浅谈云存储在医院信息化中的应用

目的:探讨云存储在医院信息化建设中,提供可扩展存储资源的应用前景及可能的挑战。方法:使用目前主流的Hadoop开源云平台软件和高性能的Panasas集群硬件设备,提出了一种基于私有云的存储解决方案,并分析云存储在医院信息化中所面临的困难与对策。结果:解决目前医院信息化过程中存储资源不足、管理不便的问题,为日益庞大的医疗数据提供按需扩展的存储空间。结论:随着云存储技术和法规的不断完善,云存储将在医院信息化建设中发挥更加实际的作用。     

基于 Hadoop 的心电数据库存储研究

本文旨在研究远程监控心血管疾病时,产生的心电大数据的存储方式。采用 Hadoop 分布式集群技术,设计基于 HBase 非关系型（NoSQL）心电数据库,测试存储时间和访问效率,解决心电大数据的存储问题。经测试,HBase 数据库的存储时间、访问时间、存储的高可靠性,均满足心电数据的存储要求。本研究为后续进行心血管大数据分析、统计、数据挖掘打下基础。     

医学大数据应用信息系统策划与设计

医学大数据是国家重要的战略性基础数据资源,将应用于精准临床诊疗、决策支持、疾病监测预警与管理、公众健康服务等领域。当前,国内对医学大数据技术的应用成熟度还有待提高,如何将传统的医学数据平稳过渡到大数据体系中,通过数据挖掘等手段对其进行专业的分析来实现数据的“增值”,是当下亟待解决的重要问题。本文通过构建区域级医学大数据应用技术工程实验室,对医学大数据应用信息系统总体架构、数据中心架构等核心功能进行初步策划与设计。     

Hadoop neural network for parallel and distributed feature selection

In this paper, we introduce a theoretical basis for a Hadoop-based neural network for parallel and distributed feature selection in Big Data sets. It is underpinned by an associative memory (binary) neural network which is highly amenable to parallel and distributed processing and fits with the Hadoop paradigm. There are many feature selectors described in the literature which all have various strengths and weaknesses. We present the implementation details of five feature selection algorithms constructed using our artificial neural network framework embedded in Hadoop YARN. Hadoop allows parallel and distributed processing. Each feature selector can be divided into subtasks and the subtasks can then be processed in parallel. Multiple feature selectors can also be processed simultaneously (in parallel) allowing multiple feature selectors to be compared. We identify commonalities among the five features selectors. All can be processed in the framework using a single representation and the overall processing can also be greatly reduced by only processing the common aspects of the feature selectors once and propagating these aspects across all five feature selectors as necessary. This allows the best feature selector and the actual features to select to be identified for large and high dimensional data sets through exploiting the efficiency and flexibility of embedding the binary associative-memory neural network in Hadoop.     

基于Hadoop的电子健康档案云平台设计和实现

阐述基于Hadoop的电子健康档案云平台架构设计,包括服务对象及需求、逻辑架构、软件架构等方面,介绍基于HBase的电子健康档案云平台数据预处理模型,进行实验环境的搭建和配置,通过实验完成Hadoop集群的启动。     

Unstructured medical image query using big data – An epilepsy case study

Big data technologies are critical to the medical field which requires new frameworks to leverage them. Such frameworks would benefit medical experts to test hypotheses by querying huge volumes of unstructured medical data to provide better patient care. The objective of this work is to implement and examine the feasibility of having such a framework to provide efficient querying of unstructured data in unlimited ways. The feasibility study was conducted specifically in the epilepsy field. The proposed framework evaluates a query in two phases. In phase 1, structured data is used to filter the clinical data warehouse. In phase 2, feature extraction modules are executed on the unstructured data in a distributed manner via Hadoop to complete the query. Three modules have been created, volume comparer, surface to volume conversion and average intensity. The framework allows for user-defined modules to be imported to provide unlimited ways to process the unstructured data hence potentially extending the application of this framework beyond epilepsy field. Two types of criteria were used to validate the feasibility of the proposed framework – the ability/accuracy of fulfilling an advanced medical query and the efficiency that Hadoop provides. For the first criterion, the framework executed an advanced medical query that spanned both structured and unstructured data with accurate results. For the second criterion, different architectures were explored to evaluate the performance of various Hadoop configurations and were compared to a traditional Single Server Architecture (SSA). The surface to volume conversion module performed up to 40 times faster than the SSA (using a 20 node Hadoop cluster) and the average intensity module performed up to 85 times faster than the SSA (using a 40 node Hadoop cluster). Furthermore, the 40 node Hadoop cluster executed the average intensity module on 10,000 models in 3 h which was not even practical for the SSA. The current study is limited to epilepsy field and further research and more feature extraction modules are required to show its applicability in other medical domains. The proposed framework advances data-driven medicine by unleashing the content of unstructured medical data in an efficient and unlimited way to be harnessed by medical experts.     

基于Hadoop的中医数据存储平台设计与开发

随着中医技术的广泛应用,中医资料数据飞速涌现出来,利用中医数据存储平台合理管理和存储这些重要的中医资料数据显得极为重要。本文提出了一种基于分布式计算技术进行管理和存储中医资料数据方法,构建了中医资料数据存储平台解决方案,采用Linux集群技术,设计开发一个基于Hadoop的中医数据存储平台。本系统由五大模块组成,有系统管理模块、并行加载存储模块、并行查询模块、数据字典模块、备份恢复模块,能够实现存储中医资料数据。系统模块实现结果表明,该系统安全可靠、易维护、具有良好的可扩展性。     

‘Big data’, Hadoop and cloud computing in genomics

Since the completion of the Human Genome project at the turn of the Century, there has been an unprecedented proliferation of genomic sequence data. A consequence of this is that the medical discoveries of the future will largely depend on our ability to process and analyse large genomic data sets, which continue to expand as the cost of sequencing decreases. Herein, we provide an overview of cloud computing and big data technologies, and discuss how such expertise can be used to deal with biology’s big data sets. In particular, big data technologies such as the Apache Hadoop project, which provides distributed and parallelised data processing and analysis of petabyte (PB) scale data sets will be discussed, together with an overview of the current usage of Hadoop within the bioinformatics community.     

Analysis of microarray leukemia data using an efficient MapReduce-based K-nearest-neighbor classifier

Microarray-based gene expression profiling has emerged as an efficient technique for classification, prognosis, diagnosis, and treatment of cancer. Frequent changes in the behavior of this disease generates an enormous volume of data. Microarray data satisfies both the veracity and velocity properties of big data, as it keeps changing with time. Therefore, the analysis of microarray datasets in a small amount of time is essential. They often contain a large amount of expression, but only a fraction of it comprises genes that are significantly expressed. The precise identification of genes of interest that are responsible for causing cancer are imperative in microarray data analysis. Most existing schemes employ a two-phase process such as feature selection/extraction followed by classification. In this paper, various statistical methods (tests) based on MapReduce are proposed for selecting relevant features. After feature selection, a MapReduce-based K-nearest neighbor (mrKNN) classifier is also employed to classify microarray data. These algorithms are successfully implemented in a Hadoop framework. A comparative analysis is done on these MapReduce-based models using microarray datasets of various dimensions. From the obtained results, it is observed that these models consume much less execution time than conventional models in processing big data.