Subscribe cloud computing RSS CSDN home> cloud computing

Micro ticket technology VP Yang Senmiao: annual growth of 4000% behind the big data and cloud computing

Published in15:25 2016-01-05| Time reading| sourceCSDN| ZeroArticle comments| authorZhou Jianding

Abstract:Micro shadow era R & D center vice president of technology Yang Senmiao guest Tencent cloud room ", dialogue Tencent Vice President Zeng Jiaxin, the interpretation of the micro ticket son years growth 4000 behind the big data and cloud computing systems.

Micro ticket son, a recently attracted particular attention online ticketing platform: Valuation of nearly 10 billion, with the merger of Che Guevara, annual growth rate of over 4000%, the monthly live exceeded 2 million, 500 cities nationwide coverage of more than 4500 theater, sunrise votes of 100 million copies. The peak value of 200 million.

Behind this series of data, what kind of support technology? Recently, the micro shadow era R & D center vice president of technology Yang Senmiao guest Tencent cloud room ", dialogue Tencent Vice President Zeng Jiaxin, the interpretation of the micro ticket son years growth 4000 behind the big data and cloud computing systems.

Yang Senmiao said micro ticket son of core is with high data precision marketing and service, by convolutional neural network (CNN) and singular value decomposition (SVD) algorithm to 200tb user data and industry data analysis, micro ticket son can accurately grasp the needs of users, enhance the conversion rate of box office; at the same time, facing the challenge of users during the period of rapid growth in the high peak, high flow, high scalability, security of cloud computing resources is supported by the uninterrupted operation of the necessary conditions.


Tencent cloud vice president Ceng Jiaxin (right) dialogue with Yang Senmiao, vice president of micro ticket (left)

Big data and application of deep learning

Established only a short 500 days, Yang Senmiao believes that the micro ticket can be settled in the 700 million active users of the micro channel, 800 million active users of QQ, big data is its core strengths.

Micro ticket son will according to the optimal user heat the recommendation of theater row piece, first through the data analysis of the user's location and the user in the vicinity of the theater, and nurturing upstream, and theater manager also developed micro ticket professional edition to help them to understand its theater data, to determine the schedule and industry disc. It can be said that our core competitiveness is to use the entire big data for the film industry, performances and sports industry to do service, connection and marketing.

Specifically, the data sources of the micro ticket large data analysis mainly includes 4 categories:

  1. Micro ticket server production log, including the request to access the data, interface calls generated by a variety of log;
  2. Business data, such as data on the movie schedule, the box office, the order of the transaction, every marketing campaign, comments, etc.;
  3. Big data industry, mainly the pan entertainment industry website external data and report research statistics;
  4. Movie community and user interaction data.

Currently produced in the micro ticket son log entries of billions of pieces, business, the cumulative amount of data not less than tens of millions of magnitude and large data is to explosive speed continuous expansion, the current micro ticket son of big data of the amount of data stored has reached 200t.

Micro ticket son through the full integration of these data, for each user, films, performances, physical channels to build a complete picture. The difficulty mainly lies in to establish the relationship between different sources of data, complex types of data, the source is not a result difficult to construct a unified mapping relationship, current micro ticket son by constantly improve the matching relation table supplemented by matching the model, such as semantic matching, posters or publicity photo image similarity matching correction, continuous data fusion to meet business development and application.

algorithm

From a logical point of view, this process contains the two parts of the algorithm.

1 feature engineering algorithm

Feature engineering is to improve the accuracy of the algorithm, the data to do a series of mathematical transformation. This part of the project is particularly important, not only test the mathematical ability of the algorithm engineers, the same test engineering capabilities.

When a huge data set can not be extracted from a single machine, it is needed to design a parallel feature extraction algorithm from the single machine algorithm. Micro ticket son feature extraction engineering the depth of learning (deep learning) technology. This is because the depth of learning natural network topological structure is more likely to parallel and parallel storage can also meet the needs of the huge amount of data storage.

Of course, these algorithms are more to cater to the use of the scene. Such as Convolutional (Neural CNN (Network) and SVD (Value Decomposition Singular), the two dimension reduction methods will be used because of different business scenarios.

2 target result algorithm

The important dimension of the feature engineering algorithm is given, which is given to the "target result algorithm" to deal with.

In addition to conventional machine learning / deep learning algorithm, the micro ticket for the uncertain mathematics related algorithms also have some applications. Because many scenarios, accurate estimation performance is not very good, especially when the independent variable interpretation ability is not strong.

For example, commonly used Regression algorithm for determining the information has a high predictive power, but for uncertain information is relatively weak. For this kind of scene, the micro ticket is used to describe the uncertain information, such as "Entrop (entropy)" and "Lyapunov"".

Computing platform

Computing architecture, because the depth of learning technology breakthrough more sources in pattern recognition, rely on multi platform for expensive CUDA platform (micro ticket son completed "feature extraction" to such a large task, will consume about 500 units of cluster resources). Therefore, in addition to the daily needs of the data storage and extraction of clusters, the algorithm focuses on the micro votes to further the parallel memory computing technology.

Yang Senmiao hopes that deep learning will be able to calculate the cost of parallel in the low cost of memory computing platforms (such as Spark cluster), you can not reduce the input parameters and the size of the neural network, the better to complete the task. In order to offset and high-speed GPU speed difference, micro ticket son used many clusters, and a "flexible" mode, after the end of the training process, automatic cluster expansion is the general pattern so as to avoid the GPU cluster with the same hardware must consume problem.

It is easy to see that the big data platform is not entirely in the cloud. Yang Senmiao is a self built platform for big data and cloud platform data quality:

1 self built big data platform advantages

  • Users according to their own needs planning and construction of a large data platform fit business needs, and platform construction and business development and promote each other.
  • The upgrading of the version of technical institutions, the introduction of new technologies, asset management, automation, operation and maintenance, rights and other systems research and development by the user's own decision, controllable.
  • Big data platform technology team in the framework of each technology will continue to in-depth research, to ensure the stability of the platform and continuous innovation, and enhance the company's technical influence.

2 self built big data platform shortcomings

  • The comprehensive ability of the platform technology team members (learning ability, innovation ability) requires a higher.
  • The specificity of the IT industry, the stability requirements of the team members are higher.

The advantages of the 3 cloud data platform

  • Service providers have a sound solution system, according to the specific needs of users and application scenarios to provide users with the appropriate technical architecture.
  • In each package of the technical architecture is a complete ecological system, in addition to the big data platform itself, also attached to the asset management system, operation and maintenance of automation system, monitoring and alarm system, authentication system, security system and different levels of HA.

4 cloud data platform shortcomings

  • Each set of solutions provided to the customer is often not fully fit user needs, plus the technical architecture of the various components and parts of the custom package, for the user to increase the different levels of learning and maintenance costs.
  • When the user will need some cutting-edge technology or the third party components integrated into Yunda data platform, service providers often requires a long time even tell the user there is no suspense to the R & D program.

Micro ticket according to the business scene to choose self built platform or cloud platform:

  1. The establishment of business data warehouse is still in the local physical cluster, because it involves a large amount of business data, and a lot of calculation and configuration, the accuracy of the data requirements are higher.
  2. The analysis of some competitive products, such as the analysis of the box office, public opinion and user behavior analysis, etc., is done in the cloud. One is because more data sources change rapidly, cloud storage and computing resources application flexible allocation, can quickly respond to the needs; and secondly, micro ticket son all operational databases are in the cloud, so that data transmission is also more convenient.
  3. Some real-time computing and quasi real time business services are mostly carried out in the cloud, so that the maximum degree of shortening due to network or data interaction caused by the delay.
  4. Data transmission, the relationship between data T+1 and T+0 data batch synchronization, millions of data synchronization is completed within 1 minutes, the flow of real-time data processing, second level response.

The follow-up efforts in the direction. Yang Senmiao said, is how to play the fortunately, two aspects of "big data and social" core strengths, "big data to understand social the film performances and other entertainment industry and the audience better connected together, such as Tencent excellent map of face recognition effectively help the micro ticket son of data acquisition, and this can enrich its big data, to achieve a better insight into the.

Cloud computing to ensure seamless expansion

As an online ticketing platform, in the period of rapid growth of users, will inevitably encounter high peak, large flow challenges, how to have a stable, secure server, is undoubtedly the most critical issue. This year the national archives, micro vote share in the movie box office in China accounted for the ratio exceeded 25%, the daily average ticket amount reached more than 100 million peak even higher than 200 million storage amount of votes, service flow than the daily values grow up to four times as much, the rapid expansion of each service unit capacity the 4-6 times.

Yang Senmiao said, the traditionalIDC room mode in infrastructure construction, security management, broadband and fast expansion of the hardware and many other aspects of a lot of drawbacks,Independent purchase of hardware equipment will also bring a lot of IT costs, but also requires a huge operation and maintenance team. Through cooperation with Tencent cloud, in the peak period of time, the micro ticket quickly through the cloud to expand. Not only such, Tencent cloud dynamics can be extended effectively help micro ticket son face daily activities, the rapid expansion and in peak timely cancel the expansion of equipment and save resources.

Tencent micro ticket based on detailed investigation and comparison of several domestic mainstream cloud services company, for flexibility and expansibility, after-sales service response time, platform server performance and price factors comprehensive evaluation made after the decision. Yang Senmiao said, so choose the reasons, is more valued Tencent cloud technology team of professional and technical.

On Tencent cloud platform, with the rapid expansion of micro ticket services, cloud hosting cloudDB load balanced CDN security scanning distributed defense platform provides various functions are quickly getting used to, but each kind of new things in the initial use of time definitely will encounter a problem, coupled with the difference between the use of individual service and traditional IDC way, micro votes in the use process indeed encountered some minor problems, but under the Tencent platform of strong technical support team of fast and efficient communication, quickly solved. Yang Senmiao said,Through Tencent cloud services, micro ticket operation and maintenance team is not much, but each time the peak of the business, can quickly make a response.

In addition, the micro ticket in the cloud, DDOS attacks and other external attacks do not have to worry about their own. Tencent cloud has covered the national 400+ network nodes, as well as one hundred DDOS G protection capabilities, able to effectively help the micro ticket users to solve the problem of increasing the amount of platform users. Not only that, Tencent cloud in the cloud server, CDN acceleration services, cloud monitoring, load balance, etc., but also to provide a high performance, professional and reliable service support for the micro ticket.

summary

In the era of mobile Internet, all walks of life are constantly changing, with big data, cloud computing and other advanced technologies, to be able to stand out in the increasingly fierce competition environment.

Yang Senmiao suggested that entrepreneurs in the mobile Internet era should be bold to embrace new technologies, embrace the convenience of cloud computing. She believes that IT's technological evolution and change is very fast in recent3-5, cloud computing and mobile should have a great demand and become the mainstream.In the future, big data and social will help to better connect the micro ticket users, cloud computing is to provide a more professional micro ticket to provide a more professional mobile Internet +O2O operating services.

For more information, please video on demand:Micro ticket son daily orders 1 million, Cloud Computing supports its rise

top
Zero
step on
Zero