Skip to content

Commit

Permalink
new rake task to record benchmarks, new 2.0 baseline benchmarks
Browse files Browse the repository at this point in the history
  • Loading branch information
jeremydmiller committed May 3, 2017
1 parent 4110b93 commit 85efb12
Show file tree
Hide file tree
Showing 17 changed files with 263 additions and 0 deletions.
12 changes: 12 additions & 0 deletions benchmarks/2.0.baseline/results/BulkLoading-report-default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2 Gen 0=834.5588 Allocated=7.74 MB

Method | Mean | StdDev |
-------------------- |------------ |---------- |
BulkInsertDocuments | 196.4910 ms | 9.4850 ms |
14 changes: 14 additions & 0 deletions benchmarks/2.0.baseline/results/BulkLoading-report-github.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
``` ini

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2 Gen 0=834.5588 Allocated=7.74 MB

```
Method | Mean | StdDev |
-------------------- |------------ |---------- |
BulkInsertDocuments | 196.4910 ms | 9.4850 ms |
2 changes: 2 additions & 0 deletions benchmarks/2.0.baseline/results/BulkLoading-report.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Method,Job,AnalyzeLaunchVariance,EvaluateOverhead,MaxStdErrRelative,MinInvokeCount,MinIterationTime,RemoveOutliers,Affinity,Jit,Platform,Runtime,AllowVeryLargeObjects,Concurrent,CpuGroups,Force,Server,Clock,EngineFactory,Toolchain,InvocationCount,IterationTime,LaunchCount,RunStrategy,TargetCount,UnrollFactor,WarmupCount,Mean,StdDev,Gen 0,Allocated
BulkInsertDocuments,Default,False,Default,Default,Default,Default,Default,15,RyuJit,X64,Clr,False,True,False,True,False,Default,Default,Default,1,Default,Default,Default,Default,16,2,196.4910 ms,9.4850 ms,834.5588,7.74 MB
31 changes: 31 additions & 0 deletions benchmarks/2.0.baseline/results/BulkLoading-report.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8' />
<title>BulkLoading</title>

<style type="text/css">
table { border-collapse: collapse; display: block; width: 100%; overflow: auto; }
td, th { padding: 6px 13px; border: 1px solid #ddd; }
tr { background-color: #fff; border-top: 1px solid #ccc; }
tr:nth-child(even) { background: #f8f8f8; }
</style>
</head>
<body>
<pre><code>
BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
</code></pre>
<pre><code>WarmupCount=2 Gen 0=834.5588 Allocated=7.74 MB
</code></pre>

<table>
<thead><tr><th> Method</th><th> Mean</th><th>StdDev</th>
</tr>
</thead><tbody><tr><td>BulkInsertDocuments</td><td>196.4910 ms</td><td>9.4850 ms</td>
</tr></tbody></table>
</body>
</html>
12 changes: 12 additions & 0 deletions benchmarks/2.0.baseline/results/DocumentActions-report-default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2 Gen 0=415.7609 Allocated=4.46 MB

Method | Mean | StdDev |
---------------- |----------- |---------- |
InsertDocuments | 43.0381 ms | 1.5788 ms |
14 changes: 14 additions & 0 deletions benchmarks/2.0.baseline/results/DocumentActions-report-github.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
``` ini

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2 Gen 0=415.7609 Allocated=4.46 MB

```
Method | Mean | StdDev |
---------------- |----------- |---------- |
InsertDocuments | 43.0381 ms | 1.5788 ms |
2 changes: 2 additions & 0 deletions benchmarks/2.0.baseline/results/DocumentActions-report.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Method,Job,AnalyzeLaunchVariance,EvaluateOverhead,MaxStdErrRelative,MinInvokeCount,MinIterationTime,RemoveOutliers,Affinity,Jit,Platform,Runtime,AllowVeryLargeObjects,Concurrent,CpuGroups,Force,Server,Clock,EngineFactory,Toolchain,InvocationCount,IterationTime,LaunchCount,RunStrategy,TargetCount,UnrollFactor,WarmupCount,Mean,StdDev,Gen 0,Allocated
InsertDocuments,Default,False,Default,Default,Default,Default,Default,15,RyuJit,X64,Clr,False,True,False,True,False,Default,Default,Default,1,Default,Default,Default,Default,16,2,43.0381 ms,1.5788 ms,415.7609,4.46 MB
31 changes: 31 additions & 0 deletions benchmarks/2.0.baseline/results/DocumentActions-report.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8' />
<title>DocumentActions</title>

<style type="text/css">
table { border-collapse: collapse; display: block; width: 100%; overflow: auto; }
td, th { padding: 6px 13px; border: 1px solid #ddd; }
tr { background-color: #fff; border-top: 1px solid #ccc; }
tr:nth-child(even) { background: #f8f8f8; }
</style>
</head>
<body>
<pre><code>
BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
</code></pre>
<pre><code>WarmupCount=2 Gen 0=415.7609 Allocated=4.46 MB
</code></pre>

<table>
<thead><tr><th> Method</th><th>Mean</th><th>StdDev</th>
</tr>
</thead><tbody><tr><td>InsertDocuments</td><td>43.0381 ms</td><td>1.5788 ms</td>
</tr></tbody></table>
</body>
</html>
12 changes: 12 additions & 0 deletions benchmarks/2.0.baseline/results/EventActions-report-default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2 Allocated=1.88 MB

Method | Mean | StdDev |
------------- |----------- |---------- |
AppendEvents | 44.3330 ms | 0.8700 ms |
14 changes: 14 additions & 0 deletions benchmarks/2.0.baseline/results/EventActions-report-github.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
``` ini

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2 Allocated=1.88 MB

```
Method | Mean | StdDev |
------------- |----------- |---------- |
AppendEvents | 44.3330 ms | 0.8700 ms |
2 changes: 2 additions & 0 deletions benchmarks/2.0.baseline/results/EventActions-report.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Method,Job,AnalyzeLaunchVariance,EvaluateOverhead,MaxStdErrRelative,MinInvokeCount,MinIterationTime,RemoveOutliers,Affinity,Jit,Platform,Runtime,AllowVeryLargeObjects,Concurrent,CpuGroups,Force,Server,Clock,EngineFactory,Toolchain,InvocationCount,IterationTime,LaunchCount,RunStrategy,TargetCount,UnrollFactor,WarmupCount,Mean,StdDev,Allocated
AppendEvents,Default,False,Default,Default,Default,Default,Default,15,RyuJit,X64,Clr,False,True,False,True,False,Default,Default,Default,1,Default,Default,Default,Default,16,2,44.3330 ms,0.8700 ms,1.88 MB
31 changes: 31 additions & 0 deletions benchmarks/2.0.baseline/results/EventActions-report.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8' />
<title>EventActions</title>

<style type="text/css">
table { border-collapse: collapse; display: block; width: 100%; overflow: auto; }
td, th { padding: 6px 13px; border: 1px solid #ddd; }
tr { background-color: #fff; border-top: 1px solid #ccc; }
tr:nth-child(even) { background: #f8f8f8; }
</style>
</head>
<body>
<pre><code>
BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
</code></pre>
<pre><code>WarmupCount=2 Allocated=1.88 MB
</code></pre>

<table>
<thead><tr><th>Method</th><th>Mean</th><th>StdDev</th>
</tr>
</thead><tbody><tr><td>AppendEvents</td><td>44.3330 ms</td><td>0.8700 ms</td>
</tr></tbody></table>
</body>
</html>
14 changes: 14 additions & 0 deletions benchmarks/2.0.baseline/results/LinqActions-report-default.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2

Method | Mean | StdDev | Gen 0 | Gen 1 | Allocated |
------------------ |---------------- |-------------- |---------- |-------- |---------- |
CreateLinqCommand | 189.7593 us | 2.3160 us | - | - | 42.64 kB |
RunLinqQuery | 124,248.2148 us | 1,785.5957 us | 4975.0000 | - | 34.13 MB |
CompiledQueries | 29,998.1235 us | 719.6757 us | 958.3333 | 83.3333 | 8.71 MB |
16 changes: 16 additions & 0 deletions benchmarks/2.0.baseline/results/LinqActions-report-github.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
``` ini

BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0

WarmupCount=2

```
Method | Mean | StdDev | Gen 0 | Gen 1 | Allocated |
------------------ |---------------- |-------------- |---------- |-------- |---------- |
CreateLinqCommand | 189.7593 us | 2.3160 us | - | - | 42.64 kB |
RunLinqQuery | 124,248.2148 us | 1,785.5957 us | 4975.0000 | - | 34.13 MB |
CompiledQueries | 29,998.1235 us | 719.6757 us | 958.3333 | 83.3333 | 8.71 MB |
4 changes: 4 additions & 0 deletions benchmarks/2.0.baseline/results/LinqActions-report.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Method,Job,AnalyzeLaunchVariance,EvaluateOverhead,MaxStdErrRelative,MinInvokeCount,MinIterationTime,RemoveOutliers,Affinity,Jit,Platform,Runtime,AllowVeryLargeObjects,Concurrent,CpuGroups,Force,Server,Clock,EngineFactory,Toolchain,InvocationCount,IterationTime,LaunchCount,RunStrategy,TargetCount,UnrollFactor,WarmupCount,Mean,StdDev,Gen 0,Gen 1,Allocated
CreateLinqCommand,Default,False,Default,Default,Default,Default,Default,15,RyuJit,X64,Clr,False,True,False,True,False,Default,Default,Default,1,Default,Default,Default,Default,16,2,189.7593 us,2.3160 us,-,-,42.64 kB
RunLinqQuery,Default,False,Default,Default,Default,Default,Default,15,RyuJit,X64,Clr,False,True,False,True,False,Default,Default,Default,1,Default,Default,Default,Default,16,2,"124,248.2148 us","1,785.5957 us",4975.0000,-,34.13 MB
CompiledQueries,Default,False,Default,Default,Default,Default,Default,15,RyuJit,X64,Clr,False,True,False,True,False,Default,Default,Default,1,Default,Default,Default,Default,16,2,"29,998.1235 us",719.6757 us,958.3333,83.3333,8.71 MB
33 changes: 33 additions & 0 deletions benchmarks/2.0.baseline/results/LinqActions-report.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<!DOCTYPE html>
<html lang='en'>
<head>
<meta charset='utf-8' />
<title>LinqActions</title>

<style type="text/css">
table { border-collapse: collapse; display: block; width: 100%; overflow: auto; }
td, th { padding: 6px 13px; border: 1px solid #ddd; }
tr { background-color: #fff; border-top: 1px solid #ccc; }
tr:nth-child(even) { background: #f8f8f8; }
</style>
</head>
<body>
<pre><code>
BenchmarkDotNet=v0.10.1, OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4980HQ CPU 2.80GHz, ProcessorCount=4
Frequency=10000000 Hz, Resolution=100.0000 ns, Timer=UNKNOWN
[Host] : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
Job-DNINOD : Clr 4.0.30319.42000, 64bit RyuJIT-v4.6.1076.0
</code></pre>
<pre><code>WarmupCount=2
</code></pre>

<table>
<thead><tr><th> Method</th><th> Mean</th><th> StdDev</th><th>Gen 0</th><th>Gen 1</th><th>Allocated</th>
</tr>
</thead><tbody><tr><td>CreateLinqCommand</td><td>189.7593 us</td><td>2.3160 us</td><td> -</td><td>-</td><td>42.64 kB</td>
</tr><tr><td>RunLinqQuery</td><td>124,248.2148 us</td><td>1,785.5957 us</td><td>4975.0000</td><td>-</td><td>34.13 MB</td>
</tr><tr><td>CompiledQueries</td><td>29,998.1235 us</td><td>719.6757 us</td><td>958.3333</td><td>83.3333</td><td>8.71 MB</td>
</tr></tbody></table>
</body>
</html>
19 changes: 19 additions & 0 deletions rakefile.rb
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,25 @@
desc 'Run Benchmarks'
task :benchmarks => [:restore] do
sh 'dotnet run --project src/MartenBenchmarks --configuration Release'


end

desc 'Record Benchmarks'
task :recordbenchmarks do
if !ENV['profile'].nil?
dir = 'benchmarks/' + ENV['profile']
if Dir.exists? dir
Dir.rmdir dir
end

puts 'Creating directory ' + dir
Dir.mkdir dir;

cp_r 'BenchmarkDotNet.Artifacts/results', dir


end
end


Expand Down

0 comments on commit 85efb12

Please sign in to comment.