NSSC OpenIR  > 空间技术部
基于流式计算的空间科学卫星数据实时处理
Alternative TitleReal-time processing of space science satellite data based on stream computing
孙小涓; 石涛; 胡玉新; 佟继周; 李冰; 宋峣
Department空间技术部
Source Publication计算机应用
2019
Volume39Issue:6Pages:1563-1568
ISSN1001-9081
Language中文
Keyword流式计算 数据流 空间科学卫星 数据处理
Abstract针对空间科学卫星探测数据的实时处理要求越来越高的问题,提出一种基于流计算框架的空间科学卫星数据实时处理方法。首先,根据空间科学卫星数据处理特点对数据流进行抽象分析;然后,对各处理单元的输入输出数据结构进行重新定义;最后,基于流计算框架Storm设计数据流处理并行结构,以适应大规模数据并行处理和分布式计算的要求。对应用该方法开发的空间科学卫星数据处理系统进行测试分析,测试结果显示,在相同条件下数据处理时间比原有系统缩短了一半;数据局部性策略比轮询策略具有更高的吞吐率,数据元组吞吐率平均提高29%。可见采用流式计算框架能够大幅缩短数据处理延迟,提高空间科学卫星数据处理系统的实时性。
Other AbstractConcerning the increasingly high real-time processing requirement of space science satellite observed data,a real-time processing method of space science satellite data based on stream computing framework was proposed. Firstly,the data stream was abstractly analyzed according to the data processing characteristics of space science satellite. Then,the input and output data structures of each processing unit were redefined. Finally,the parallel data stream processing structure was designed based on the stream computing framework Storm to meet the requirements of parallel processing and distributed computing of large-scale data. The developed system for space science satellite data processing applying with this method was tested and analyzed. The results show that the data processing time is half of that of the original system under same conditions and the data localization strategy has higher throughput than round-robin strategy with the data tuple throughput increased by 29% on average. It can be seen that the use of stream computing framework can greatly shorten the data processing delay and improve the real-time performance of the space science satellite data processing system.
Indexed ByCSCD
Citation statistics
Document Type期刊论文
Identifierhttp://ir.nssc.ac.cn/handle/122/7229
Collection空间技术部
Recommended Citation
GB/T 7714
孙小涓,石涛,胡玉新,等. 基于流式计算的空间科学卫星数据实时处理[J]. 计算机应用,2019,39(6):1563-1568.
APA 孙小涓,石涛,胡玉新,佟继周,李冰,&宋峣.(2019).基于流式计算的空间科学卫星数据实时处理.计算机应用,39(6),1563-1568.
MLA 孙小涓,et al."基于流式计算的空间科学卫星数据实时处理".计算机应用 39.6(2019):1563-1568.
Files in This Item:
File Name/Size DocType Version Access License
20193961563-1568.pdf(2594KB)期刊论文出版稿开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[孙小涓]'s Articles
[石涛]'s Articles
[胡玉新]'s Articles
Baidu academic
Similar articles in Baidu academic
[孙小涓]'s Articles
[石涛]'s Articles
[胡玉新]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[孙小涓]'s Articles
[石涛]'s Articles
[胡玉新]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.