"Cloud computing architecture technology and practice" 20:2.4.5 serial analysis of large data cloud

2884 people read comment(0) Collection report

2.4.5Big data analysis cloud

Cloud based cloud computing based on the overall structure of the big data analysis of cloud solutions, such as2-33As shown.


chartTwo-Thirty-three Big data analysis cloud solution architecture subsystem combination

Data analysis of cloud solutions for large flow mass static data batch processing, as well as dynamic data stream processing to provide support for the key features of the enterprise and industry application scenarios, the automatic extracting and summarizing information value to realize the business value added. Big data analysis of cloud computing platform supported by the parallel data analysis and mining, can make full use of cloud computing to create maximum value of the underlying capabilities.

In the scene of mass static data batch and high data analysis platform requires a thorough analysis of the accumulated over a long period of time, storage capacity huge historical data (such as if single, logs, system information). Data analysis platform of parallel data processing engine further dependent on the elastic computing clusters, elastic storage service, distributed structured storage service and distributed message queue service, such as the Internet electronic commerce website users, telecom operatorsBSS/OSSSystem, video and entertainment website, search service. The types of services provided by the big data analysis platform include: information database refinement search, log analysis of consumer behavior, analysis of system operation log and intelligent analysis and mining of centralized monitoring signaling information. These large data analysis services for the precise positioning of advertising push, network operation and maintenance optimization, based on the analysis of user consumption trends and other business operators to provide decision support for business operations.

In the large flow dynamic data flow scenarios and its key features in many vehicles in to in a large number of sources of information generated from dynamic events and dynamic data (for example from the telecommunications network real time detection of signaling information, fromGPSLocation information, come from the network terminal in real-time collection of information, etc.), in a relatively short time window, dynamic data pipeline automatic analysis and processing, and provides timely, accurate and intelligent execution strategy decision, for specific target business services (such as the construction of large-scale intelligent transport cloud, cloud logistics network. A large number of persistent storage, which are involved in the intermediate steps of segmentation, merging and mixing, and the last data batch processingI/OCompared to the characteristics of the interaction, the biggest difference is that the data stream processing process more attention to deal with the timeliness and agility control ability, so the processing process is mainly completed in the memory. Stream processing and batch processing can be unified under the same framework engine.

In order to facilitate the majority of third-party application development programming personnel and the cloud computing platform ecosystem partners sufficiently independent from massive data processing and stream processing business of the internal implementation details of the structure can be in parallel between the data analysis engine and concurrent application settingsSQL/classSQLAdaptation and translation layer, which is well known by the developer.SQLOr classSQLStandard language for the operation of massive data.

step on
Guess you're looking for
View comments
* the above user comments only represent their personal views, does not represent the views or position of the CSDN website
    personal data
    • visit71463 times
    • Integral:One thousand three hundred and fifty-one
    • Grade
    • Rank:18670th name
    • original65
    • Reproduced:3
    • Translation:0
    • Comments:18