注意:測(cè)試方法或者推算思路,可能有問題,歡迎批評(píng)與指正。
Mongodb的版本:
執(zhí)行命令mongod --version得到 "version" :
"3.0.1",
|
myNote
|
Master
|
slave
|
Test-server
|
|
硬件配置:
|
|
|
2核心
2個(gè)邏輯cpu
主頻2.2GHz
|
6核心
24個(gè)邏輯cpu
主頻2.0GHz
|
|
4核心
8個(gè)邏輯cpu
2.1GHz
|
|
測(cè)試單個(gè)請(qǐng)求的處理時(shí)間
|
|
數(shù)據(jù)量為3W個(gè)相同點(diǎn)
|
109ms
|
199ms
|
|
212ms
|
|
數(shù)據(jù)量為2W個(gè)相同點(diǎn)
|
76ms
|
131ms
|
|
129ms
|
|
數(shù)據(jù)量為1W個(gè)相同點(diǎn)
|
36ms
|
66ms
|
|
63ms
|
|
數(shù)據(jù)量為1W個(gè)相同點(diǎn), 測(cè)試并發(fā)多請(qǐng)求的處理時(shí)間 。(極端人口密集的情況)
|
并發(fā)10
|
149ms
|
88ms
|
|
122ms
|
|
并發(fā)20
|
308ms
|
135ms
|
|
213ms
|
|
并發(fā) 30
|
454ms
|
189ms
|
|
333ms
|
|
并發(fā)
50
|
778ms
|
246ms
|
|
503ms
|
|
并發(fā)
100
|
1634ms
|
585ms
|
|
724ms
|
|
并發(fā)
200
|
|
1035ms
|
|
1375ms
|
|
數(shù)據(jù)量為100W個(gè)均勻分布的點(diǎn),測(cè)試并發(fā)多請(qǐng)求的處理時(shí)間。(均勻分布的情況)
|
|
并發(fā)
1000
|
|
10ms以下。測(cè)試機(jī)cpu耗盡。服務(wù)端cpu峰值20%
|
|
|
|
推算:能處理并發(fā)為1000*5*24約12W/S;
參考資料:1億條記錄的MongoDB數(shù)據(jù)庫(kù)隨機(jī)查詢性能測(cè)試 http://www.jb51.net/article/44693.htm
|
|
數(shù)據(jù)量為1W個(gè)點(diǎn)(180,180) ,測(cè)試搜索(0,0)附近的點(diǎn)的處理時(shí)間.(極端人口稀疏的情況)
|
|
1W
|
|
59ms
|
|
|
|
10W
|
|
603ms
|
|
|
|
20W
|
|
962ms
|
|
|
|
30W
|
|
1241ms
|
|
|
|
結(jié)論:
寫性能:寫數(shù)據(jù)的速度不穩(wěn)定,表現(xiàn)為寫入一萬條數(shù)據(jù)
可能耗時(shí)280至320毫秒;寫的極限是,2M/s,頻率約2W/s;
讀性能:讀數(shù)據(jù)的速度表現(xiàn)穩(wěn)定,并發(fā)一千下,每個(gè)請(qǐng)求的處理耗時(shí)在10ms以下;24個(gè)cpu的峰值為20%,推論出頻率約10W/s;
#!/usr/bin/python
import pymongo
from pymongo import MongoClient, GEO2D
import time
import random
from bson.son import SON
import threading
from multiprocessing import Process
x_min_num=30
x_max_num=31
y_min_num=179
y_max_num=180
def mongodb_connect():
connection = pymongo.Connection("192.168.0.102",27017)
#db = connection.admin
#db.authenticate("superuser","pwd") #用戶認(rèn)證
db = connection.test
return db
def mongodb_save(collections,document):
collections.insert(document)
'''
db.users.ensureIndex({"gps":"2d"},{"min":-180,"max":180})
'''
def mongodb_createCollection():
print "mongodb_createCollection"
db = mongodb_connect()
my_collection = db.users
my_collection.create_index([("gps", GEO2D)])
#my_collection.create_index("mike")
# my_collection.create_index([("mike", pymongo.DESCENDING), ("eliot", pymongo.ASCENDING)])
#my_collection.ensure_index([("gps",pymongo.GEO2D),("min",-180),("max",180)],background=True)
def mongodb_destroyCollection():
db = mongodb_connect()
my_collection = db.users
my_collection.drop()
print "mongodb_destroyCollection"
#my_collection.drop_index([("gps", GEO2D)])
def mongodb_insert_1w_data():
db = mongodb_connect()
my_collection = db.users
doc_list=list()
for i in xrange(10000):
doc=dict()
x=random.uniform(x_min_num,x_max_num)
y=random.uniform(y_min_num,y_max_num)
doc["account"]=str(100+i)
doc["gps"]=[ x,y ]
doc_list.append(doc)
begin_time = time.time()
my_collection.insert(doc_list)
end_time = time.time()
print "my_collection.insert_many done:",end_time-begin_time
def mongodb_findNear1km( location ):
db = mongodb_connect()
my_collection = db.users
begin_time = time.time()
doc_list = my_collection.find( {"gps": {"$near": location}} ).limit(100)
end_time = time.time()
# for doc in doc_list:
# print "gps:",doc["gps"]
# print "----------------"
#
# print "mongodb_findNear1km begin_time:",begin_time
# print "mongodb_findNear1km end_time:",end_time
#print "mongodb_findNear1km done:",end_time-begin_time ,begin_time,end_time
class WorkerThread(threading.Thread):
def __init__(self,worker_index):
threading.Thread.__init__(self)
self.worker_index=worker_index
def run(self):
print "*"*20
x=random.uniform(x_min_num,x_max_num)
y=random.uniform(y_min_num,y_max_num)
location= [x, y]
mongodb_findNear1km(location)
print "worker:",self.worker_index,location
def mongodb_insert_some_data(how_many):
begin_time = time.time()
for i in xrange(how_many):
mongodb_insert_1w_data()
end_time = time.time()
print "mongodb_insert_some_data done:",end_time-begin_time
def mongodb_test_concurrentQueryRequest(concurrent_num):
print "mongodb_test_concurrentQueryRequest:", concurrent_num
for i in xrange(concurrent_num):
worker = WorkerThread(i)
worker.setDaemon(True)
worker.start()
def sleeper(name, seconds):
mongodb_test_concurrentQueryRequest(100)
time.sleep(1)
def testMore(num):
process_list=list()
for i in xrange(num):
p = Process(target=sleeper, args=('bob', 5))
p.start()
process_list.append(p)
#p.join()
for p in process_list:
p.join()
def mongodb_putdata():
mongodb_destroyCollection()
mongodb_createCollection()
mongodb_insert_some_data(10)
def testone():
x=random.uniform(x_min_num,x_max_num)
y=random.uniform(y_min_num,y_max_num)
location= [x, y]
mongodb_findNear1km(location)
def main():
print "main"
mongodb_putdata()
#testone()
return
#testMore(1)
mongodb_test_concurrentQueryRequest(100)
count=0
while 1:
time.sleep(1)
count +=1
#testone()
print count
if count > 100:
print "count 100"
break
main()