@ @flyinweb
1. 2. 3. 4. 5. 6. 7.
?
RTOLAP(Realtime OLAP) Grid Computing In-memory database
5 / > 1 > 10 > 20 >>>>>>>>>>>>>>>
QPS > 300 AVG Query Scan Row > 300 AVG Query Compute Column > 50
cool.taobao.com jianghu.taobao.com
· · · ·
· SQL ·Schema Free · ·
///
Redis ID
Tokyo Cabinet
Prom Service
ID
Hbase
Redis
Prom Service
-
2.(Hbase) 3.(Hbase)
-
-
Garuda &
WEB ITier
REST
ISV ITier
MYSQL
MN-1 ()
MN-.. ()
JDBC CN-1 ()
JDBC CN-... ()
LCN-1 ()
LCN-... ()
PANGU/HDFS
()
ZK
CCN
ZK MCN
REST 1/2
Fixed/Free Schema Partition/TableGroup Join
/
Table TableSchema
Int Short
TableSchema ColumnSchema ....
/ ColumnSchema
Partition · Interval · Range · Hash TableGroup · Join · PartitionGroup
Garuda DBx TableGroupx Table Partition
/()
· @ memory
· @ disk
Index array(abstract)
· Hash · B+Tree
tree<T,int[]>
SSD
· Skiplist · Bitmap
skiplist<T,int[]> SSD
hashmap<T,int[]> SSD
· String · PForDelta(7%)
unique<T,int>
memory
()
()
(ms)
(ms)
Array
5
200,000 100
5
200,000 1000
10 177 112994350 5 772 259067358
Hashmap
5
200,000 100
5
200,000 1000
653 1143 17497813 533 10838 18453589
SkipList B+ tree
5
200,000 100
28959 41853
477863
5
200,000 1000
47439
36179 5
552799
5
200,000 100
3922 6112 3272251
5
200,000 1000
4261 58458 3421260
16core E5620 @2.40GHZ,24GB
Master SQL
master
Localnode
SQL
Lcn1
Lcn2
Lcn3
Lcn4
Lcn5
TableGroup
+batch
group by partition_key AVG
sum/count Limit
limit * Order by rand() limit n
rand(cardinality*limit+2) Distinct
Bitset+Bloom Filter
TableGroup Join
M:N +
M:N count
-------------------------------------WHERE keyword contains (iphone',') ------------------------WHERE cpv in (x1',x2')
1
2
3
LIRS Evicted Factor Object Type/Object Size Object Domain
SSD
DISK
Memory
Master LIRS SQL cardinality Partition result
Day 1 Query 1
06/01
06/02
06/03
06/04
06/05
06/07
06/08
Day 2 Query 2
06/02 06/03 06/04 06/05 06/07 06/08 06/09
Monitor /
Buffer Failover Rotate (// ) ....
Failover Rotate (T4)
Heartbeat
OLD
JobID
FailOver
Materiality formless
+ > 5 * 7 30 3 4 QPS > 300 <2s
SNSJianghu.taobao.com
100,000,000 1 QPS > 600 < 90ms
MVCC (DB/USER/) hadoop SSD
! Q&A