Subscription programmers magazine RSS CSDN home> Programmers magazine

"Programmer" in 2014 2 monthly: big data real time processing

Published in11:07 2014-01-24| Time reading| source"Programmer"| ZeroArticle comments| authorProgrammers magazine

Abstract:This period "big data real-time processing" as the theme, inviting senior technical experts from a number of companies, from the transmission, storage, computing and other many aspects explain data real-time processing process, combined with the practice of the core technology of the impala, presto, storm, Samza, spark streaming etc. the in-depth analysis.

This issue's cover reported to "big data real-time processing" as the theme, inviting from Alibaba, Baidu and Tencent, Intel, Sina, Sohu, Jingdong, percent more than the company's senior technical experts, from the transmission, storage, computing and other many aspects explain data real-time processing, and combined with the practice of the core technology of the impala, presto, storm, Samza, spark streaming etc. the in-depth analysis, gives the specific application scenarios and optimization scheme.

(1) real time and interaction: entanglement between technology and reality

(2)Real time data transmission in the big data Era

(3)Application of HBase in content recommendation engine system

(4) the practice of real-time processing flow type computing platform

(5) the application of Storm in Tencent

(6) the practice of flow calculation based on Samza in Jingdong

(7)Spark Streaming: a massive data stream processing upstart

(8) Presto Realization Analysis

(9) the practice of building a real time big data query system based on Impala

(10) Baidu real time computing system

(11) percentage point real time computing: Architecture and algorithms


(1) express foreign periodicals

(2) Web text selection

(3) news

(4) new tools for new products

(5) the world of things


(1)To the technical people on the management class: control and planning

(2) how to improve the non technical ability of R & D personnel

Corporate R & D core asset is the human resources, human resources management in addition to the attention "organizational structure, responsibilities, performance evaluation, professional training" of the conventional content, but also try to enhance R & D personnel of non technology to, existing human resources play a greater effect. In this paper, the author combined with his years of practical experience, enhance R & D personnel about non technical talent cheats.


(1) the Internet product strategy Nirvana

The Internet industry actually all the time "smoke, war continuously", Internet companies only in the fierce competitive environment in middle school will be the use of strategy, both global and development in the process of strategic development, enhancement of the ecological system user channel control ability, can be the winner's attitude to obtain the absolute market share.

(2) talk about Ali's internal innovation mechanism - Horse Racing

Foundation to evergreen, companies will always be defeated, but can also be beat themselves, which requires to encourage innovation, tolerate mistakes. In order to allow the company to maintain a lasting vitality and innovation capability, Taobao inside the horse racing mechanism to become the source of innovation.

(3) guided design


(1) iBeacons Technology

As IOS 7 the most important one of the new techniques, ibeacons technology have lot of NFC do not have the advantage. For example, it can remote file transfer, support for indoor navigation and mobile payments. At the same time, because it can improve the customer relationship management, its commercial value also to bring unlimited reverie.

(2) fall from the top game developers to see the sense of crisis in the product industry

In many game companies for their "Centennial" intended to fame some game companies because of the subsequent product of inertia and the core members of the left to the dilemma. At the core of the game market, "wealthy" and "grassroots" for their own market and the risk of a miscarriage of justice is the same.

(3) how to catch a free ride of wearable devices - an exclusive interview with Liu Chao CEO

"Le power" is a development of hardware thinking App, in addition to the common application of common B2C business, it also actively explore the feasibility of providing services to get involved in the field of hardware.

(4) the making process and understanding of new exploration exhibits -- "digital map" of the emperor.

(5)Blue ocean of social software"

(6) FreeType, game font parsing tool

Because OpenGL does not provide support for the font, the game engine usually needs to achieve its own analysis of the font and rendering. This article will be from the history of computer fonts, the interpretation of the iOS and Android platforms, such as the extensive use of the font engine Freetype parsing principle and process, and the Cocos2d-x game engine as an example to briefly introduce its use.

cloud computing

(1)Tencent large scale Hadoop cluster practice

TDW is Tencent's largest offline data processing platform. This paper mainly from the needs, challenges, programs and plans for the future, and other aspects, introduces the TDW in the construction of a single large cluster take the jobtracker decentralization and namenode high availability of two optimization schemes.

(2) the public comment on the data structure of the road

(3) SDN controller: Beacon core technology analysis

At present, there are many kinds of SDN controller, and the overall structure and core functions of the controller are also different. The Beacon as a pioneer and guide in the controller, the SDN controller design has played an important guiding role, this paper will design and implementation of Beacon for a detailed analysis.


(1) SOSP Yetan Louse: distributed text

Symposium on operating systems principles (sosp) is the operating system in the field of top conferences, every odd numbered held in this field another important meeting osdi every even numbered years held), in November 2013, in Farmington, Pennsylvania, the United States held the 24th sosp. This paper will lead readers to resolve the most important papers and related research in this conference.

(2) where is the Linux kernel sneezing?

Familiar with Linux users, be sure to know Panic. It is similar to the nature of BSOD (Screen Of Death Blue), but there are many different forms and details of the performance of the. BSOD once generated, the system will enter a stable "stagnation" state, had to restart. And after the panic, sometimes the system can also move, or after a while back, is like sneezing, one after another "ejected stars".

(3) C structure lost package technology


(1)Feedback is everything - Storm Marz, founder of Nathan, an interview with the founder of real time computing system

Marz Nathan is a distributed fault-tolerant real-time computing system - Storm's founder, in July 2011 Twitter acquisition of social media data analysis firms BackType former, he is the chief engineer of BackType. In Twitter, he is responsible for the establishment of flow computing team, a number of key business for the entire firm to provide infrastructure support. In March 2013, he chose to leave Twitter, founded his own company.

(2) review

(3) GEEK

(4) cartoon

Tel: 01051661202-176/381

"Programmer" magazine subscription in 2014 to send gifts activities fiery

step on

Recent activity


2015 China big data technology conference

In order to better help enterprises in-depth understanding of large data with the latest technology at home and abroad, master more industry data of practical experience, to further promote the big data technology innovation, industry application and talent cultivation, 10-12 December 2015, sponsored by the China Computer Federation (CCF), CO CCF large numbers according to the Expert Committee of the contractor, Chinese Academy of Sciences Institute of computing technology, and DNT branch in Beijing Science and Technology Co., Ltd. and CSDN 2015 China big data technology conference (big data technology conference 2015, BDTC 2015) will Yunnan new crown holiday hotel in Beijing held a grand.

Microblogging attention

Programmer mobile terminal subscription Download

Related popular articles