Statement:This article from the SDCC 2016 Database & Architecture Summit (ShangHai Railway Station) of the guests to share, without permission to prohibit any form of reprint.
Author:Li Zhihui, house m CTO, has worked for Alibaba and Intel Asia Pacific R & D center, engaged in large-scale web sites, distributed systems, big data platform aspects of research and development. The author of "large web technology architecture - the core principle and case analysis" of a book.
Commissioning editor:Qian Shuguang, focus on architecture and algorithms for reporting or submission, please email qianshg@PROG3.COM. Otherwise "CSDN senior architect group", with Daniel architect and many well-known Internet companies, welcomed the architect and micro channel qshuguang2008 application into the group. Note the name + company + jobs.
In recent years, the wave of Internet entrepreneurs of all types of Internet Venture Company such as raging like a storm, set up like bamboo shoots after a spring rain. Technology as an important part of the Internet business, but also unprecedented attention. Development of Internet companies are usually explosive growth, in a very short period of time, business scale, user amount of hundreds of thousands of times the growth, the framework of website technology put forward a great challenge.
In this paper, we share in a typical Internet Venture Company, how to quickly respond to business changes, and constantly optimize the system architecture to meet the needs of business. Technical team is how to constantly optimize the rapid development of the restructuring and growth, to adapt to changes in business and technical architecture.
Home meters was established at the end of 2014, is a focus on Campus E-commerce Internet companies. Just over a year's time, the company's business covering nearly 200 cities, more than 1000 colleges and universities, more than 10000 dormitories, daily orders 200 thousand, peak orders 500 thousand.
Corresponding to the size of the technical team from the start of the three engineers, the development of a team of 50 people. Business scale changes as shown in figure 1.
Like all Internet Co, the system architecture is very simple. Four major business systems: buyer systems, vendor systems, supply chain systems, operational support systems, and components of the company's core application systems. Nginx as a front-end web server through load balancing services to mobile App and mobile Web service. As shown in figure 2.
Such a simple system architecture in the early stage of the company and there is no problem, only a few thousand orders in the daily average of the time, the system's processing capacity is not the main problem at that time. But with the rapid growth of users, goods, orders, the pressure of the system is increasing, the main performance is the load pressure of the database is particularly high, the peak of the database to increase the response delay. At that time the company's business objectives are average daily order 200 thousand, peak order 500 thousand, the results of the performance test is the current system architecture is simply not enough to support such an order quantity.
Solution is to increase the cache and the database master. In the application services layer to increase the Redis cache, cache business objects. In front of the use of third party CDN services, cache pictures, JS and other static files. And the data analysis task is migrated to the big data platform to reduce the pressure of the database access. The reconstructed system is shown in Figure 3.
2 version of the architecture of the system performance and processing capacity has been greatly improved, but with the system more and more complex, many functions are repeated development, the development of new requirements for a longer period of time, Bug is also more and more. So it is decided to use the distributed service, which can be used in a distributed way, and can be used in various business systems, so as to improve the efficiency of development and maintenance costs.
Also with the development of business scale, another problem is also highlighted: table of orders of a dramatic increase in the amount of data, according to this speed, order table single table data will soon be bigger than that of database storage limit. At that time, there are two programs, one is a distributed database, the order table is split into a number of physical library. Another is the hot and cold separation, the historical order to migrate to MongoDB, only to provide read-only query operation. Technical team after weighing, and finally chose the second options. The latest system architecture is shown in Figure 4.
Early home meter technical team only three engineers, there is almost no division of labor, each person is a full stack of engineers, where to do what to do. With the gradual development of the company, the size of the technical team is constantly changing, when the team has more than a dozen people, it is necessary to organize the division of labor. The original grouping method is according to the professional functions of the engineer division of labor, that is, the back-end group, APP group, front-end group. As shown in Figure 5, the benefits of such grouping are the same as the professional skills of an engineer in a group, which are more likely to communicate and collaborate with each other.
When the team is small, only a dozen people, the product is not much, only one or two products, such an organization can be effective operation of the. But when the number of dozens of products have five or six, each product need across multiple technical team development, communication and coordination costs are rising rapidly, this time according to the products belong to the division of labor more favorable to improve the development efficiency and organizational structure as shown in Figure 6.
As the team size continues to grow, the division of labor is more detailed, organizational structure also need to make a more precise adjustment, as shown in figure 6.
Technical department established specialized architecture group, performance testing, performance optimization, reconfigurable architecture, process improvement and other business unrelated to technological transformation and the product development team were technical support, technical department firefighters and special forces.
In the cloud computing and Internet technology is very perfect today, for most startups, the technology will not be a barrier to entrepreneurship. If the business model is correct, the market has been recognized and sought after capital, the rapid development of business, even if there are some errors and technical errors, it will not cause major damage to the business development.
Zhai meters in its short history, the rapid development of business constantly of technology put forward a variety of challenges, technical team has confused poor to deal with, but through constant self evolution, experience once again reborn and ashes, and eventually grows.