Sogou Under the big data log -flume (two) with the optimization of high

Had to read the flume optimization articles! ...
Read(15) comments(1)

Sogou -flume (a) introduction and examples of the big data log

There are a lot of people explain what flume is, listening is very tall, but this world is very simple, there is no need to understand this, we have to listen to my definition of flume, flume is already written a set of code, used to transmit data, and file Java for the same, is not sound more easy to some, here we look at its other definitions, it will be found, not so strange. ...
Read(85) comments(5)

Sogou Log --ElasticSearch section (four) --Mapping under big data

Equivalent to the definition of the database table structure, mapping elasticsearch is also very important. Directly related to the performance and the accuracy of the search results. Elasticsearch Java API example of too little, here I will show. In order to explain the definition of mapping, I define a simple model here, type, ID, and catIds 3 properties, focusing on how to use the API java to define the mapping, specific each field should be how to define, here is not to discuss. J...
Read(147) comments(3)

Sogou Large data log --ElasticSearch part (three) Search, --Bulk operation

Actually in a blog, as long as you can read, you should be able to according to the code giving top priority to do the, followed by analogy es operated in batch bulk, search search function, but here is simple.   Bulk index and delete   The Java code of code BulkRequestBuilder = client.prepareBulk (bulkRequest); For (int i=500; I ...
Read(205) comments(3)

Sogou Log --ElasticSearch section (two) under the big data -- combined with the basic operation of Java

Elasticsearch (the name is too long, later referred to as "es) as a search engine, the current can be described as the sun at high noon, almost and Solr neat drive and drive. What he can do, what is the relationship with the cloud, which is no longer described. But es of official documents, especially on the Java client documentation is really less poor, not even a complete additions and deletions to the sample. Here, I'll show. Before starting to explain, or to do a bedding, in order to have an index of the model, we have a custom model, a temporary name is LogModel...
Read(677) comments(5)

Sogou Under the big data log --ElasticSearch part (a) - first

With companies increasingly sharp increase in trading volume, platform log accumulated to the level of a massive, at this time of the retrieval and debugging is a very painful process.
Read(190) comments(2)

Sogou Java cluster optimization - dubbo+zookeeper build high availability of distributed clusters

The development of the Internet, the scale of web applications is expanding, the conventional vertical application architecture has been unable to cope with, distributed service architecture and mobile computing architecture is imperative, Dubbo is a distributed service framework, in which case the birth of. Now the core business is extracted, as an independent service, so that the front end of the application can be more rapid and stable response. First: introduce the background of Dubbo      Before large-scale service, the application may be just through RMI or Hessian, etc....
Read(1019) comments(4)

Sogou Java cluster optimization - the need to understand the memory overflow and memory leaks

We operate some variables, IDE environment to provide us with a very good convenience, JVM for us to encapsulate the excellent automatic garbage collection mechanism, but we still that sentence, people are not reliable, this sentence so that we repeatedly know themselves, the same, the memory of the problem, we have to go deep into research, because this is the inevitable place for large software optimization! ...
Read(488) comments(6)

Sogou IO: I/O to understand the Java model

Listen to IO, I was confused, but is written, read, anything else? Indeed in the world of IO, only to read, but the business is different, different framework, IO will also show different, and in the Internet programming, IO programming, more and more become a bottleneck, we come together to learn, the next IO concept, and then the distinction between blocking and non blocking IO and asynchronous IO distinction, and then introduces the 5 IO model, finally introduces the two and high performance IO IO...
Read(241) comments(2)

Sogou Java cluster optimization - Nginx+tomcat cluster configuration - practice chapter

A good tool, not only to achieve a good function, there is a more important point is that there must be a more simple business logic, so that users in the use of this tool in order to smooth transition, and Nginx software, for the majority of developers precisely meet these two requirements, do not mind, the user will feel, intentions, is to allow users to experience less, can not do without! This is not the thinking of Internet products do? ...
Read(235) comments(8)

Sogou Java cluster optimization - Nginx+tomcat cluster configuration - Theory Chapter

We understand the truth, that is, the growth of data is a problem we must face, because in the changing technology of the moment, the way we collect data and storage and processing technology continues to upgrade, forcing us to face this problem, and, in the increasingly hot data mining today, we also want to pay attention to this problem, change is inevitable! We have thought that in the case of extreme optimization, an ordinary server, you can dig out the potential of the infinite, I have done this, with a 20 thousand yuan of equipment, carrying the concurrent 1 people, but with the technology, we need to be more flexible processing applications, we need more timely response speed, we need to analyze and process data in real time! At this time, we must be in touch with a word...
Read(386) comments(7)

Sogou Java cluster optimization: a single case model under multi thread

In the initial learning design patterns, I great design idea excited, in later works, several integration design patterns, and in the current design, we have detected the foresight of the original design of the model, but there are also some shortcomings, need us to improve, someone said, the world there is no absolute thing, of course, and then the simple things, the environment has changed and will change, today and everybody share mode optimization in multi thread order. The details of the optimization, more like carve a piece of art, we are exposed to an expert, such ideas more strong, so we often judge a personnel to the character and quality, through his contact with the people, friends can be inferred, and excellent computer, as well, we are in contact with the framework, since...
Read(482) comments(9)

Sogou Hadoop parsing --MapReduce

MapReduce, all of the calculations are done locally, in a timely manner to calculate the need for foreign data, but also a good collection, so that we can ensure that the most efficient use of bandwidth, so that we have the ability to handle the data with a linear increase in the number of clusters. ...
Read(488) comments(11)

Sogou Hadoop resolution - the first meeting

Cloud computing, is a joint N machines together to complete a calculation, of course, as the business complexity of the system requirements will certainly become more and more high, so the complexity of the system will be more and more high, so the modular development certainly put on the agenda, so it is divided into several modules, but also in order to solve different problems, let us together to study these modules. ...
Read(442) comments(8)

Sogou Java cluster optimization: data source encryption

Project, security is an eternal topic, we project on sensitive information processing, generally take the way of encryption, and in the previous framework we generally write a class of encryption and decryption of sensitive information, but after jboss6, for encryption to achieve a better interface, today we will explore the secret here: First in an increase of                                         The ciphertext is generated by the following command line:...
Read(287) comments(6)

Sogou Java cluster optimization -- ORM framework query optimization theory

As we all know, the current popular enterprise architecture, ORM has been the most basic part of the architecture design of the underlying operational support for the logic layer, but the fact is always and we expect some deviation, ORM in providing a better operating experience, also lost a part of the native SQL flexibility and efficiency, of course, this problem does not affect our use of ORM framework, but it is to enhance the flow of the site, especially in the multi relational complex query, the performance bottleneck is to mention the part! ...
Read(394) comments(5)

Sogou Java cluster optimization: a preliminary study of multi thread

To use multi thread, we need to understand the next multi thread, in the near future, I wrote the relevant article, in this brief review, a program with data, constitute a process, just like express company is a process, the method is to send courier company, courier like thread, they are sent to express, is responsible for regional differences, then the most simple multi thread is how to achieve it? Simple realization multithreading has a lot of kinds of methods, and Java here mainly has two, one is to continue the thread class. Another is to achieve runable interface, we now...
Read(587) comments(6)

Sogou Java architecture decryption - real time dynamic AOP

In the last post, we come together to form a container, which encapsulates the business. In this way, we will service and assembled into the client, and client is equivalent to our development to use the configuration file, we found out the problem? Is that I can not dynamically change? When the service container is completed, it is not allowed to add a container, which is not allowed, and the container is a compile time function of the container, he lost a lot of flexibility! What about that? The actual is to change the order of the call is good, we look at the results: 1, class diagram comparison...
Read(879) comments(7)

Sogou Java architecture decryption - dual container optimization AOP

In the last blog is proposed, and the optimization is a never-ending process, indeed, with changes in demand, the basic hardware and software upgrade, we more and more it does not consider code capacity, but consider the code quality, but with the deepening of the research to a certain stage, we have to consider the problem of code capacity.
Read(805) comments(5)

Sogou Java architecture decryption - with the interface to transform AOP

Optimization is an endless work, in the AOP on the road, we go very far, but there are still a lot of work, we did not do, for example, the AOP business part of the package into a container, the AOP service part of the transformation into an interface, so that it is not subject to the specific form of the limit! So AOP optimization, and a step forward, but also in line with our thinking of the interface programming, the following and I study how to integrate the interface into the implementation of aop. ...
Read(1042) comments(4)

Sogou Decryption of Java Architecture -- Deep reconstruction AOP

With the in-depth study of AOP, we found that the core concept has become extremely important, these concepts is a map, guidelines for us in the optimization of the road to go farther and farther! ...
Read(546) comments(7)

Sogou Decryption of Java Architecture -- implementation of AOP dynamic proxy

AOP spring depth of the core, to see how this mysterious thing is how to look at a face, how he is from the Jane into the difficult, difficult to! ...
Read(453) comments(10)

Java flexible document export -Freemarker

Projects always need to generate a variety of reports and documents, and word documents are we often use the format, using freemarkerm template is a more convenient way to generate, is a summary of the key steps for your reference First, the production of templates First use Word to do a template, as shown below: (note, there is a table above, I set the border is not visible) and then save as XML files, then open the XML file with the tool, it was used XML Editor firstobject feeling is not as good as notepa...
Read(195) comments(9)
173 data a total of 18 pagesOne Two Three Four Five ... The Library Shadowe
    personal data
    • Visit187162 times
    • Integral:Eight thousand eight hundred and twenty-seven
    • Grade
    • Rank:1083rd name
    • Original169
    • Reproduced:3
    • Translation:0
    • Comments:4111 article
    Blog column
    Latest comments