Skip to content

Commit

Permalink
LRU: test-lru.rb improved in different ways.
Browse files Browse the repository at this point in the history 
1. Scan keys with pause to account for actual LRU precision.
2. Test cross-DB with 100 keys allocated in DB1.
3. Output results that don't fluctuate depending on number of keys.
4. Output results in percentage to make more sense.
5. Save file instead of outputting to STDOUT.
6. Support running multiple times with average of outputs.
7. Label each square (DIV) with its ID as HTML title.
  • Loading branch information
@antirez
antirez committed Jul 11, 2016
1 parent eee878c commit 32a5494
Show file tree
Hide file tree
Showing 2 changed files with 180 additions and 98 deletions.
10 changes: 8 additions & 2 deletions utils/lru/README
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,17 @@ Redis approximated LRU algorithm against the theoretical output of true
LRU algorithm.

In order to use the program you need to recompile Redis setting the define
REDIS_LRU_CLOCK_RESOLUTION to 1, by editing redis.h.
REDIS_LRU_CLOCK_RESOLUTION to 1, by editing the file server.h.
This allows to execute the program in a fast way since the 1 ms resolution
is enough for all the objects to have a different enough time stamp during
the test.

The program is executed like this:

ruby test-lru.rb > /tmp/lru.html
ruby test-lru.rb /tmp/lru.html

You can optionally specify a number of times to run, so that the program
will output averages of different runs, by adding an additional argument.
For instance in order to run the test 10 times use:

ruby test-lru.rb /tmp/lru.html 10
268 changes: 172 additions & 96 deletions utils/lru/test-lru.rb
View file
Original file line number Diff line number Diff line change
@@ -1,112 +1,188 @@
require 'rubygems'
require 'redis'

r = Redis.new
r.config("SET","maxmemory","2000000")
r.config("SET","maxmemory-policy","allkeys-lru")
r.config("SET","maxmemory-samples",5)
r.config("RESETSTAT")
r.flushall

puts <<EOF
<html>
<body>
<style>
.box {
width:5px;
height:5px;
float:left;
margin: 1px;
}
$runs = []; # Remember the error rate of each run for average purposes.

def testit(filename)
r = Redis.new
r.config("SET","maxmemory","2000000")
r.config("SET","maxmemory-policy","allkeys-lru")
r.config("SET","maxmemory-samples",10)
r.config("RESETSTAT")
r.flushall

html = ""
html << <<EOF
<html>
<body>
<style>
.box {
width:5px;
height:5px;
float:left;
margin: 1px;
}
.old {
border: 1px black solid;
}
.new {
border: 1px green solid;
}
.otherdb {
border: 1px red solid;
}
.ex {
background-color: #666;
}
</style>
<pre>
EOF

.old {
border: 1px black solid;
}
# Fill the DB up to the first eviction.
oldsize = r.dbsize
id = 0
while true
id += 1
r.set(id,"foo")
newsize = r.dbsize
break if newsize == oldsize # A key was evicted? Stop.
oldsize = newsize
end

.new {
border: 1px green solid;
}
inserted = r.dbsize
first_set_max_id = id
html << "#{r.dbsize} keys inserted"

# Access keys sequentially, so that in theory the first part will be expired
# and the latter part will not, according to perfect LRU.

STDERR.puts "Access keys sequentially"
(1..first_set_max_id).each{|id|
r.get(id)
sleep 0.001
STDERR.print(".") if (id % 150) == 0
}
STDERR.puts

# Insert more 50% keys. We expect that the new keys will rarely be expired
# since their last access time is recent compared to the others.
#
# Note that we insert the first 100 keys of the new set into DB1 instead
# of DB0, so that we can try how cross-DB eviction works.
half = inserted/2
html << "Insert enough keys to evict half the keys we inserted"
add = 0

otherdb_start_idx = id+1
otherdb_end_idx = id+100
while true
add += 1
id += 1
if id >= otherdb_start_idx && id <= otherdb_end_idx
r.select(1)
r.set(id,"foo")
r.select(0)
else
r.set(id,"foo")
end
break if r.info['evicted_keys'].to_i >= half
end

.ex {
background-color: #666;
}
</style>
<pre>
html << "#{add} additional keys added."
html << "#{r.dbsize} keys in DB"

# Check if evicted keys respect LRU
# We consider errors from 1 to N progressively more serious as they violate
# more the access pattern.

errors = 0
e = 1
error_per_key = 100000.0/first_set_max_id
half_set_size = first_set_max_id/2
maxerr = 0
(1..(first_set_max_id/2)).each{|id|
if id >= otherdb_start_idx && id <= otherdb_end_idx
r.select(1)
exists = r.exists(id)
r.select(0)
else
exists = r.exists(id)
end
if id < first_set_max_id/2
thiserr = error_per_key * ((half_set_size-id).to_f/half_set_size)
maxerr += thiserr
errors += thiserr if exists
elsif id >= first_set_max_id/2
thiserr = error_per_key * ((id-half_set_size).to_f/half_set_size)
maxerr += thiserr
errors += thiserr if !exists
end
}
errors = errors*100/maxerr

STDERR.puts "Test finished with #{errors}% error! Generating HTML on stdout."

html << "#{errors}% error!"
html << "</pre>"
$runs << errors

# Generate the graphical representation
(1..id).each{|id|
# Mark first set and added items in a different way.
c = "box"
if id >= otherdb_start_idx && id <= otherdb_end_idx
c << " otherdb"
elsif id <= first_set_max_id
c << " old"
else
c << " new"
end

# Add class if exists
if id >= otherdb_start_idx && id <= otherdb_end_idx
r.select(1)
exists = r.exists(id)
r.select(0)
else
exists = r.exists(id)
end

c << " ex" if exists
html << "<div title=\"#{id}\" class=\"#{c}\"></div>"
}

# Close HTML page

html << <<EOF
</body>
</html>
EOF

# Fill
oldsize = r.dbsize
id = 0
while true
id += 1
r.set(id,"foo")
newsize = r.dbsize
break if newsize == oldsize
oldsize = newsize
f = File.open(filename,"w")
f.write(html)
f.close
end

inserted = r.dbsize
first_set_max_id = id
puts "#{r.dbsize} keys inserted"

# Access keys sequentially

puts "Access keys sequentially"
(1..first_set_max_id).each{|id|
r.get(id)
# sleep 0.001
}
def print_avg
avg = ($runs.reduce {|a,b| a+b}) / $runs.length
puts "#{$runs.length} runs, AVG is #{avg}"
end

# Insert more 50% keys. We expect that the new keys
half = inserted/2
puts "Insert enough keys to evict half the keys we inserted"
add = 0
while true
add += 1
id += 1
r.set(id,"foo")
break if r.info['evicted_keys'].to_i >= half
if ARGV.length < 1
STDERR.puts "Usage: ruby test-lru.rb <html-output-filename> [num-runs]"
exit 1
end

puts "#{add} additional keys added."
puts "#{r.dbsize} keys in DB"

# Check if evicted keys respect LRU
# We consider errors from 1 to N progressively more serious as they violate
# more the access pattern.

errors = 0
e = 1
edecr = 1.0/(first_set_max_id/2)
(1..(first_set_max_id/2)).each{|id|
e -= edecr if e > 0
e = 0 if e < 0
if r.exists(id)
errors += e
end
}
filename = ARGV[0]
numruns = 1

puts "#{errors} errors!"
puts "</pre>"

# Generate the graphical representation
(1..id).each{|id|
# Mark first set and added items in a different way.
c = "box"
if id <= first_set_max_id
c << " old"
else
c << " new"
end
numruns = ARGV[1].to_i if ARGV.length == 2

# Add class if exists
c << " ex" if r.exists(id)
puts "<div class=\"#{c}\"></div>"
numruns.times {
testit(filename)
print_avg if numruns != 1
}

# Close HTML page

puts <<EOF
</body>
</html>
EOF

0 comments on commit 32a5494

Please sign in to comment.