Which is the fastest?

This project aims to be a load benchmarking suite, no more, no less

Measuring response times (routing times) for each framework (middleware).

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️

Results are not production-ready yet

⚠️⚠️⚠️⚠️⚠️⚠️⚠️⚠️

Additional purposes :

• Helping decide between languages, depending on use case
• Learning languages, best practices, devops culture ...
• Having fun ❤️

Requirements

• Crystal as built-in tools are made in this language
• Docker as frameworks are isolated into containers
• wrk as benchmarking tool, >= 4.1.0
• postgresql to store data, >= 10

ℹ️ you need wrk stable

git clone --branch 4.1.0 https://github.com/wg/wrk

⚠️ docker is used for development purpose, production results will be computed on DigitalOcean ⚠️

Usage

• Install all dependencies

shards install

• Build internal tools

shards build

• Create and initialize the database

createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sql

Docker can be used to set up the database:

docker run -it --rm -d \
  -p 5432:5432 \
  -e POSTGRES_DB=benchmark \
  -e POSTGRES_HOST_AUTH_METHOD=trust \
  -v /tmp/pg-data:/var/lib/postgresql/data \
  --name pg postgres:12-alpine

Wait several seconds for the container to start, then inject the dump:

docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"

After creating the database, export its URL:

export DATABASE_URL="postgresql://postgres@localhost/benchmark"

• Make configuration

bin/make config

• Build containers

jobs are either languages (example : crystal) or frameworks (example : router.cr)

bin/neph [job1] [job2] [job3] ...

• Export all results readme

bin/db to_readme

Results

ℹ️ Updated on 2020-03-21 ℹ️

Benchmarking with wrk

• Threads : 8
• Timeout : 8
• Duration : 15s (seconds)

ℹ️ Sorted by max req/s on concurrency 64 ℹ️

	Language	Framework	Speed (64)	Speed (256)	Speed (512)	Speed (1024)	Speed (2048)
1	nim (1.0)	httpbeast (0.2)	183 311	196 756	198 580	195 282	195 309
2	javascript (13.7)	sifrr (0.0)	180 473	193 511	193 323	189 919	190 660
3	javascript (13.7)	nanoexpress-pro (1.1)	174 528	186 721	182 928	187 247	182 666
4	go (1.14)	fasthttp (1.9)	163 475	171 750	173 918	169 992	169 576
5	javascript (13.7)	nanoexpress (1.1)	163 443	180 210	180 052	174 355	177 034
6	go (1.14)	fasthttprouter (0.1)	157 613	166 134	168 271	163 778	163 430
7	go (1.14)	router (0.7)	157 600	165 914	168 197	164 120	163 804
8	go (1.14)	atreugo (10.4)	156 760	164 770	167 327	163 331	162 710
9	go (1.14)	fiber (1.8)	156 693	166 800	164 681	155 386	154 025
10	crystal (0.33)	toro (0.4)	155 952	164 143	160 897	152 242	151 612
11	go (1.14)	gorouter-fasthttp (4.4)	155 382	163 425	165 682	161 294	160 839
12	java (8)	rapidoid (5.5)	155 300	169 051	171 010	167 053	167 072
13	crystal (0.33)	router.cr (0.2)	155 294	162 885	160 659	151 157	151 055
14	crystal (0.33)	spider-gazelle (2.3)	151 418	158 300	155 352	145 977	145 891
15	php (7.4)	workerman (4.0)	146 977	157 464	158 882	157 283	157 006
16	crystal (0.33)	grip (0.28)	145 763	152 517	148 582	140 198	139 117
17	crystal (0.33)	kemal (0.26)	144 654	151 945	148 037	139 799	138 193
18	nim (1.0)	jester (0.4)	144 057	154 516	155 335	151 553	151 745
19	crystal (0.33)	amber (0.33)	136 702	142 972	138 890	129 404	129 430
20	crystal (0.33)	lucky (0.18)	135 172	139 221	134 528	123 838	125 426
21	crystal (0.33)	orion (2.1)	132 718	136 796	131 319	120 902	120 148
22	crystal (0.33)	athena (0.8)	126 854	129 440	123 129	110 867	110 924
23	java (8)	act (1.8)	119 062	130 618	130 711	127 746	127 284
24	go (1.14)	rte (0.0)	111 097	110 181	113 492	113 325	113 070
25	go (1.14)	httprouter (1.3)	109 322	108 482	111 757	111 644	111 172
26	go (1.14)	chi (4.0)	104 669	103 122	106 196	106 042	105 482
27	go (1.14)	gorouter (4.4)	104 012	107 875	110 677	109 771	109 521
28	go (1.14)	aero (1.3)	103 642	103 223	105 915	104 967	104 813
29	go (1.14)	violetear (7.0)	102 144	101 539	105 017	105 104	104 802
30	go (1.14)	echo (4.1)	100 257	98 774	101 913	102 211	102 004
31	go (1.14)	goroute (0.0)	100 126	98 379	101 378	101 962	101 742
32	go (1.14)	kami (2.2)	99 758	103 430	104 985	103 298	103 001
33	ruby (2.7)	agoo (2.12)	98 417	120 215	125 072	125 176	124 597
34	go (1.14)	gorilla-mux (1.7)	97 314	95 604	98 713	99 235	98 902
35	go (1.14)	beego (1.12)	97 016	100 360	103 549	103 221	102 790
36	csharp (8.0)	aspnetcore (3.1)	96 821	104 852	105 447	102 601	103 044
37	go (1.14)	gin (1.5)	96 127	99 270	101 595	101 793	101 530
38	go (1.14)	webgo (3.0)	94 837	93 571	96 997	97 541	97 266
39	c (99)	kore (3.3)	91 570	157 941	157 570	156 352	147 974
40	javascript (13.7)	polkadot (1.0)	89 819	100 738	101 326	96 351	96 820
41	cpp (14/17)	drogon (1.0)	88 286	91 531	92 457	93 056	92 279
42	javascript (13.7)	0http (2.2)	84 696	93 695	95 814	91 279	89 801
43	go (1.14)	air (0.15)	83 223	87 023	89 244	89 249	89 032
44	javascript (13.7)	restana (4.1)	83 198	90 425	89 107	85 369	85 132
45	javascript (13.7)	rayo (1.3)	83 171	87 320	84 916	82 269	81 566
46	javascript (13.7)	polka (0.5)	82 224	87 898	85 434	82 197	79 916
47	go (1.14)	gf (1.11)	76 779	81 949	83 564	83 615	83 386
48	java (8)	javalin (3.5)	74 824	78 927	78 738	75 161	76 548
49	scala (2.12)	akkahttp (10.1)	73 386	79 793	76 646	75 118	75 791
50	python (3.8)	falcon (2.0)	71 383	77 827	77 713	76 403	76 363
51	java (8)	spring-boot (2.1)	71 016	77 060	74 929	69 287	70 178
52	javascript (13.7)	foxify (0.1)	66 075	67 872	63 758	63 988	64 060
53	javascript (13.7)	muneem (2.4)	65 400	70 335	68 807	66 296	66 441
54	php (7.4)	one (2.0)	64 729	69 020	69 116	68 283	68 064
55	elixir (1.1)	cowboy_stream (2.7)	64 575	65 454	64 879	63 311	63 329
56	kotlin (1.3)	ktor (1.2)	64 476	76 911	78 410	74 786	74 168
57	javascript (13.7)	fastify (2.13)	64 457	71 281	68 226	67 351	66 360
58	swift (5.1)	perfect (3.1)	63 350	71 527	76 342	76 440	78 058
59	go (1.14)	mars (1.0)	61 654	64 963	67 779	67 705	67 658
60	javascript (13.7)	koa (2.11)	61 285	63 945	62 035	60 821	61 104
61	java (8)	micronaut (1.2)	60 876	67 770	69 199	66 553	68 230
62	python (3.8)	bottle (0.12)	59 913	62 900	62 565	61 756	61 534
63	c (11)	agoo-c (0.7)	57 393	174 634	184 324	184 173	180 275
64	python (3.8)	apidaora (0.14)	57 197	63 628	63 178	60 701	60 556
65	javascript (13.7)	iotjs-express (0.0)	57 097	61 558	58 565	57 126	56 895
66	rust (1.41)	nickel (0.11)	56 885	57 541	55 657	56 255	56 277
67	java (8)	spring-framework (5.2)	56 684	63 914	63 482	61 890	61 913
68	clojure (1.1)	coast (1.0)	53 714	54 828	54 633	54 534	54 453
69	scala (2.12)	http4s (0.18)	51 511	58 460	59 343	60 945	60 666
70	javascript (13.7)	express (4.17)	50 565	54 086	53 157	51 729	50 140
71	python (3.8)	blacksheep (0.2)	50 474	54 762	54 147	52 265	52 404
72	javascript (13.7)	moleculer (0.14)	50 398	51 874	50 782	50 014	49 994
73	php (7.4)	hyperf (1.0)	48 727	50 737	50 701	50 077	50 125
74	swift (5.1)	kitura (2.8)	48 422	47 788	48 416	47 470	46 984
75	python (3.8)	asgineer (0.7)	48 404	52 459	51 854	50 124	50 058
76	swift (5.1)	vapor (3.3)	47 878	47 992	48 277	47 921	47 503
77	swift (5.1)	kitura-nio (2.8)	47 606	49 185	47 492	46 829	46 688
78	python (3.8)	pyramid (1.1)	46 768	52 451	51 911	51 646	51 428
79	cpp (11)	evhtp (1.2)	46 284	46 113	45 953	45 889	45 240
80	python (3.8)	starlette (0.13)	45 734	49 321	48 812	47 171	47 261
81	python (3.8)	hug (2.6)	44 720	46 672	46 373	45 550	45 887
82	javascript (13.7)	hapi (19.1)	43 557	45 374	44 272	44 197	43 951
83	elixir (1.1)	cowboy (2.7)	42 503	43 536	43 083	42 197	42 043
84	php (7.4)	imi (1.0)	41 478	43 033	43 406	43 153	43 198
85	javascript (13.7)	restify (8.5)	41 372	43 070	41 746	41 730	41 701
86	php (7.4)	sw-fw-less (preview)	41 250	43 279	42 975	42 389	42 317
87	rust (1.41)	gotham (0.4)	41 192	49 295	50 829	52 961	52 792
88	ruby (2.7)	hanami-api (0.1)	40 547	42 158	39 707	39 165	38 947
89	python (3.8)	emmett (2.0.0b1)	39 108	42 006	41 665	40 296	40 464
90	ruby (2.7)	syro (3.2)	38 923	43 244	41 170	40 297	40 501
91	ruby (2.7)	roda (3.3)	38 352	39 554	37 995	37 246	37 771
92	elixir (1.1)	plug (1.8)	37 651	38 396	37 850	37 076	36 999
93	php (7.4)	swoft (2.0)	36 264	37 011	36 682	36 155	36 234
94	ruby (2.7)	cuba (3.9)	35 267	35 484	34 455	34 074	34 164
95	dart (2.7)	aqueduct (3.2)	33 335	32 357	32 878	31 809	31 940
96	elixir (1.1)	phoenix (1.4)	30 789	32 803	32 406	31 796	31 751
97	python (3.8)	responder (2.0)	29 727	31 424	31 028	29 822	29 869
98	python (3.8)	fastapi (0.52)	29 508	31 294	31 023	29 737	29 749
99	fsharp (4.7)	suave (2.5)	29 114	27 415	33 419	32 469	31 665
100	ruby (2.7)	rack-routing (0.0)	28 066	28 381	27 896	27 830	27 984
101	python (3.8)	molten (1.0)	26 925	27 693	27 209	27 223	26 973
102	python (3.8)	clastic (19.9)	26 092	26 556	26 420	26 408	26 310
103	ruby (2.7)	camping (2.1)	25 803	25 475	25 122	25 043	25 280
104	python (3.8)	aiohttp (3.6)	25 730	28 028	27 905	26 842	27 124
105	python (3.8)	masonite (2.3)	24 025	24 833	24 668	24 355	23 845
106	python (3.8)	flask (1.1)	23 811	24 706	24 393	24 018	24 235
107	javascript (13.7)	turbo_polka (0.3)	23 061	21 118	20 503	20 470	20 405
108	python (3.8)	sanic (19.12)	21 529	21 840	21 195	20 542	20 394
109	rust (1.41)	iron (0.6)	18 322	18 492	18 389	18 508	18 260
110	php (7.4)	spiral (2.4)	17 639	18 028	18 125	17 986	18 077
111	ruby (2.7)	sinatra (2.0)	16 170	16 094	16 052	16 030	16 010
112	go (1.14)	gramework (1.7)	14 956	15 430	15 386	15 330	15 397
113	ruby (2.7)	grape (1.3)	14 870	14 746	14 698	14 641	14 657
114	ruby (2.7)	flame (4.18)	13 393	13 114	13 117	13 071	13 033
115	ruby (2.7)	hanami (1.3)	12 026	11 978	11 957	11 934	11 980
116	python (3.8)	quart (0.11)	11 238	12 166	11 400	10 867	10 627
117	swift (5.1)	swifter (1.4)	10 854	11 141	11 074	10 809	11 210
118	python (3.8)	django (3.0)	10 204	10 348	10 289	10 238	10 250
119	php (7.4)	ubiquity (2.3)	9 104	9 010	9 127	51 769	52 415
120	python (3.8)	cherrypy (18.5)	8 042	8 353	8 254	8 202	8 219
121	php (7.4)	one-fpm (2.0)	7 868	7 844	7 774	43 856	43 118
122	python (3.8)	tornado (6.0)	7 627	10 514	10 184	10 092	10 118
123	php (7.4)	phalcon (4.0)	7 463	7 471	7 524	52 154	48 362
124	php (7.4)	hamlet (3.2)	7 459	7 463	7 515	43 808	44 781
125	php (7.4)	chubbyphp (2.8)	5 605	5 570	5 591	44 556	42 802
126	crystal (0.33)	onyx (0.5)	4 794	4 898	4 917	4 950	4 918
127	php (7.4)	slim (4.4)	4 612	4 622	4 736	44 003	41 856
128	php (7.4)	yii (2.0)	4 326	4 362	4 416	43 618	40 804
129	php (7.4)	lumen (7.0)	4 304	4 317	4 375	43 353	41 555
130	ruby (2.7)	rails (6.0)	3 981	3 783	3 774	3 748	3 730
131	php (7.4)	symfony (4.3)	3 607	3 628	3 669	42 672	38 941
132	php (7.4)	mezzio (3.2)	2 812	2 832	2 899	41 735	40 543
133	python (3.8)	cyclone (1.3)	2 425	2 449	2 420	2 409	2 393
134	julia (1.3)	merly (0.2)	2 131	5 475	4 352	3 467	1 783
135	python (3.8)	klein (19.6)	1 641	1 653	1 624	1 608	1 620
136	python (3.8)	nameko (2.12)	1 529	1 498	1 463	1 460	1 468
137	php (7.4)	laminas (3.1)	1 426	1 457	1 509	39 093	38 381
138	perl (5.3)	dancer2 (2.0)	887	702	1 357	1 625	486
139	php (7.4)	basicphp (0.9)	482	515	2 500	35 114	33 763
140	php (7.4)	laravel (7.2)	274	158	2 278	23 201	21 877

How to contribute ?

In any way you want ...

• Request a framework addition
• Report a bug (on any implementation)
• Suggest an idea
• ...

Any kind of idea is ❤️

Contributors

• Taichiro Suzuki - Author | Maintainer
• OvermindDL1 - Maintainer
• Marwan Rabbâa - Maintainer