Subscription programmers magazine RSS CSDN home> Programmers magazine

"Programmer September B Journal: big data core technology and practice" hot market!

Published in19:01 2015-09-15| Time reading| sourceProgrammers magazine| ZeroArticle comments| authorProgrammers magazine

Abstract:Combat big data and technology topics, extract the electricity, game, mobile and other areas of big data applications, covering the current popular data open source technology practice and details, such as Hadoop, spark, docker and with members list these wonderful practice depends on forming the fundamental, bustling open source data ecosystem.

Cover story: big data core technology and Practice

With the development of Internet and mobile Internet, vast amounts of data are generated in human daily life, various agencies began to explore extraction from data and insights to guide practice. Under the stimulation of the demand, over the past few years, large data source ecosystem has been considerable development -- in the entire life cycle of the data, collected from processing, data visualization and storage, various open source framework technology everywhere.

And with these open source technology as the cornerstone of the industry has emerged a series of admirable big data architecture practice, soElectronic Journal of programmers in September B: big data combat and technology"Excerpts from the electricity, mobile, gaming and other industries big data applications, and covers the current popular data open source technology practice and technical details, such as Hadoop, spark, docker etc., for you uncover data in every walk of life, the first in military practice, and big data ecosystem in open tuning techniques.

(1)Open source big data open source ecological overview

Large data source ecological flowers, this paper from the scheduling and resource management, distributed file system, distributed computing category, image processing and machine learning field, all aspects of inventory of today's hot technology.

(2)Connected Commerce eBay big data platform practice

EBay analysis platform infrastructure products for Han Qing, uncover the eBay in recent years development in big data platform and the main practice and now and gradually built up based on Hadoop and enterprise class data warehouse of ecological data.

(3)Youku potatoes big data platform services and application monitoring design and practice sharing: Hold live your platform

Doctors see a complexion is not good, know he may be patient, but to confirm what disease is still need further to the patient to feel the pulse. Then, the pulse platform and how to?

Big data service platform and application level monitoring artifact - the secret mirror,Design original intention is to solve this part of the demand of a service, an application monitoring platform.The secrets of heaven mirror is now responsible for supporting group a hundred large data scenarios, while also providing a minute level indicators monitoring service every day to collect 5 million index data, support for monitoring multi dimension data interactive query of the user, the same day data query millisecond response, across days query a second level of response.

Practice and Discussion on the construction of large data platform (4) micro shop

Micro shop is the world's leading mobile electricity suppliers network, creating a convenient mobile phone shopping environment, there are more than 30 million of the shop owner to use micro shop sales of goods. Micro store big data architect Wang Feng, will focus on large data processing platform for data acquisition, transmission, storage, analysis of the public in the process of basic technology.

(5)See the Spark before he

Seven cattle cloud storage technology director Chen ChaoExpose Spark insider,To share with you some of the most important recent improvements in Spark.

(6) SparkR: a new tool for data scientists

R is one of the most popular programming language and environment in the data scientists, in the Spark to join the R support, the community is more concerned about the topic. This paper invites Intel data team of engineers employed, who is one of the main SparkR contribution, he with we review SparkR project background and an overview of the current characteristics of the overall, describes its architecture and some key technical points and the outlook and summary.

(7) the practice of building Spark cluster based on Mesos

In order to users in the cloud operation spark, several technology founder Wang pujun worked out three needs: first of all, to automate the deployment of spark cluster, while achieving the spark calculation node dynamic stretching; secondly, to allow the user to conveniently use the spark cluster, let developers like the development of stand-alone programs use the same spark cluster for developing and debugging; and finally, to support multi tenant, improve the utilization rate of resources, from spark tasks of different users while running in the cloud, and ensure different spark between tasks cannot interfere. In order to meet the above three points,Wang Pu to share them with a set of Spark based, HDFS, Hue, Zeppelin, Mesos, Marathon, Docker and other open source projects, multi tenant, high availability of Spark cluster.


(1): IEEE SPECTRUM express foreign periodicals

(2) twenty-first Century ten notorious network attacks

(3) the nature of DevOps

(4) the sound of the industry

(5) big data and individual rights and interests


(1)Open source application of the strategy -- by boat to sail or ship to sea

Open source framework is currently a hot topic. Domestic awareness of open source, but also from the primary understanding of the use of free to a higher level of development. In the face of good or not good framework, we naturally think of a topic carefully: by boat to sea or shipbuilding to sea? This paper consists of Tiny open source framework founder Luo fruit for your brilliant way.

(2) HTML5 game framework in the army by Qingqi Phaser

HTML5 game with its low threshold, low cost, simple and interesting, easy to share the characteristics of the majority of players and the industry's attention. This article introduces the HTML5 framework of the game in the army by Qingqi Phaser. Although they are not big, but it is very fast and challenging, Phaser for rapid development and prototyping is perfect.


(1)Project scheduling problem  

Usually the leader gives us a description of something, and then asks, "how long does it take to make such a piece of software?" I hope we like soldiers with "yes" or "no" answer to a question raised by the executive as, immediately in a loud voice shouted "three days" or "20 days" this answer. We sometimes get used to this reality, can not immediately give estimates of their own will be anxious. Senior programmer An Xiaohui with black humor, share their own project management experience.

(2) "siege lion manual decryption"

Share management of their secret shop No. 1, senior director of technology and Huang Zhekeng, in addition to the "little, much, close to home, for technical personnel, understand some as demand for unique mental workers, help managers better management and technical personnel.

Programmers 2015 electronic version of the subscription hot, including: iPad version, Android versionPDF version.

Subscription advisory:

step on

Recent activity


2015 China big data technology conference

In order to better help enterprises in-depth understanding of large data with the latest technology at home and abroad, master more industry data of practical experience, to further promote the big data technology innovation, industry application and talent cultivation, 10-12 December 2015, sponsored by the China Computer Federation (CCF), CO CCF large numbers according to the Expert Committee of the contractor, Chinese Academy of Sciences Institute of computing technology, and DNT branch in Beijing Science and Technology Co., Ltd. and CSDN 2015 China big data technology conference (big data technology conference 2015, BDTC 2015) will Yunnan new crown holiday hotel in Beijing held a grand.

Microblogging attention

Programmer mobile terminal subscription Download