The Participants

The objective of the benchmark is not testing deployment (like uwsgi vs gunicorn and etc) but instead test the frameworks itself.

The Methodic

Results bellow were received with Amazon EC2 t2.medium server. I've used a WRK utility with params:

$ wrk -d30s -t12 -c400 [URL]

Most of the frameworks (except Tornado) were running from Gunicorn HTTP Server with 2 workers and Meinheld worker class for WSGI applications. Example:

$ gunicorn [APP] --workers=2 --worker-class=meinheld.gmeinheld.MeinheldWorker

The sources of the applications for tests can be found at here.

The benchmark has a three kind of tests:

  1. JSON test. Serialize a object to JSON and return `application/json` response.
  2. Remote test. Load and return http response from a remote server.
  3. Complete test. Load some data from DB using ORM, insert a new object, sort and render to template.

The Results

Created with Highcharts 4.1.5Values (ms)Encode a object to JSON and return as responselower is betterpyramid 43.49 msaiohttp 103.10 msflask 71.59 msmuffin 115.09 msdjango 112.19 mstornado 149.84 msbottle 26.23 msfalcon 19.81 mspyramidaiohttpflaskmuffindjangotornadobottlefalconmin50%75%90%050100150200Highcharts.com
Name50% (ms)75% (ms)Avg (ms)Req/sNon 200-xTimeouts
Falcon19.2419.8119.1920677.13  
Bottle24.7726.2325.0615761.45  
Pyramid41.7543.4941.639402.69  
Flask64.3271.5965.686023.4  
Aiohttp91.67103.1108.014093.41  
Django103.2112.19103.363696.9  
Muffin108.07115.09171.563575.36  
Tornado138.24149.84136.872829.72  
Created with Highcharts 4.1.5Values (ms)Load a response from remote server and return as responselower is betterpyramid 10518.92 msaiohttp 369.08 msflask 11690.80 msmuffin 428.75 msdjango 12954.23 mstornado 2069.25 msbottle 9911.84 msfalcon 12976.84 mspyramidaiohttpflaskmuffindjangotornadobottlefalconmin50%75%90%0k5k10k15k20kHighcharts.com
Name50% (ms)75% (ms)Avg (ms)Req/sNon 200-xTimeouts
Aiohttp358.08369.08338.941120.27 
Muffin372.95428.75376.981019.76 
Tornado1994.392069.251928.31194.37 
Pyramid3295.110518.926673.7819.35 338
Falcon3196.2312976.846696.1719.28 328
Flask3306.8811690.86824.8819.16 363
Bottle3363.749911.846403.9219.09 335
Django3317.6412954.236918.6418.96 300
Created with Highcharts 4.1.5Values (ms)Load data from database with ORM and render to templatelower is betterpyramid 601.49 msaiohttp 156.90 msflask 1649.89 msmuffin 485.40 msdjango 1976.44 mstornado 988.86 msbottle 630.17 msfalcon 805.35 mspyramidaiohttpflaskmuffindjangotornadobottlefalconmin50%75%90%0k2k4k6k8kHighcharts.com
Name50% (ms)75% (ms)Avg (ms)Req/sNon 2xxTimeouts
Aiohttp151.78156.9254.751004.8268%236
Muffin420.14485.41552.7819.62 
Bottle613.5630.171062.86451.34 178
Tornado937.37988.86910.06418.36 
Falcon766.75805.351457.99350.26 81
Pyramid562.44601.49812.43248.42 235
Flask1032.631649.891465.25222.78 496
Django1610.461976.442632.3688.57 42

Conclusion

Nothing here, just some measures for you.