MasterofProject

2 justjavac JustJavaC17:47 2013.10.19 questions

20 million open house data, how to quickly query (database optimization)

A hotel 20 million data leakage.Http://kfxx.info/

How to quickly check the full record of a name.

The name is not unique, there may be the same.

CSV data about the size of a 3G, if into the database, or 1.5g (MySQL), and 20 million also is not a small number.

50 answers

Ygycomon
Ygycomon   Rxr 09:46 2013.10.21
Has adopted

Index ah, this is what I want to, if you know the table structure, if there is no guide database, put on Linux with cat | grep can also be

If you are willing to spend time, you can put entire data to reconstruct all over again, in accordance with the name or other query fields do partition table, so the query faster, if direct import CSV file did not have the means, only direct index

Avi9111
Avi9111 1 pictures of the data is almost 20 million more than your data, and did not see the picture of 1.5G? Hundreds of m the seen and loaded into memory that is things for seconds, 1 is loaded memory is no problem, and the index just a quick positioning, so 2 is not much relationship index with the memory load.
About a year ago Reply
Huiyuanzaixian
Huiyuanzaixian Index is, of course, naturally we can think of query optimization method., but if there is a very large amount of data to a load all loaded into memory, how should I do?
About a year ago Reply
Loserss
Loserss   10:16 2014.12.29

Index processing, partition processing

A576699534
A576699534   23:46 2015.02.03

Just find the words on the use of social work library query toolHttp://www.weigongkai.com/

Fredrickhu
Fredrickhu   Ds   Rxr 09:24 2014.11.18

Add the appropriate index to the name field and include the column you are interested in using INCLUDE.

Laijieyao
Laijieyao   09:48 2014.11.18

For data, (the answer is not enough to enter the 10 character tips on the good, to help me delete the contents of my input why)

J754379117
J754379117   16:17 2014.11.18

Create index in the name field, not a little faster. I have to 2000W data import test, pro test is feasible!

J754379117
J754379117   16:17 2014.11.18

Create index in the name field, not a little faster. I have to 2000W data import test, pro test is feasible!

91program
91program   Ds   Rxr 19:08 2014.11.18

If you want to support fuzzy query, simple index will not work. But in any case, using the index is a way to solve the.

ZhangxiangDavaid
ZhangxiangDavaid   10:49 2014.11.19

The fastest is all in memory

Gbj_2010
Gbj_2010   11:08 2014.11.19

Can use redis database to MySQL data are written to redis, in the use of query reids, efficiency is very high, only but when writing in fees for some time

Huiyuanzaixian
Huiyuanzaixian Do you mean to say that all of the data in the database is written to the redis? This workload is a little big, let alone to do the above query optimization?
About a year ago Reply
A total of 50 data One Three Four ... Shadowe
User default icon Csdn
Upload medium...
Upload pictures
illustrate