rfc:better_benchmarks
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
rfc:better_benchmarks [2009/04/21 09:40] – Add tool suggestion pbiggar | rfc:better_benchmarks [2017/09/22 13:28] (current) – external edit 127.0.0.1 | ||
---|---|---|---|
Line 92: | Line 92: | ||
* [[http:// | * [[http:// | ||
* Short, doesnt take command line parameters, is varied, but simple. | * Short, doesnt take command line parameters, is varied, but simple. | ||
- | * Many benchmarks are copied from the language shootout | + | * Many benchmarks are copied from the language shootout |
- | * This seems to be the only test with data that needs to be run-through. This makes it a somewhat better test than the others, it seems. | + | * Some tests come with input data |
==== Large benchmarks | ==== Large benchmarks | ||
Line 144: | Line 144: | ||
* Not very useful if it is IO bound | * Not very useful if it is IO bound | ||
* Lots of data sets: can use any PHP package | * Lots of data sets: can use any PHP package | ||
+ | * NOTE: Seems to require graphviz | ||
* [[http:// | * [[http:// | ||
Line 157: | Line 158: | ||
* The data sets are built into the applications | * The data sets are built into the applications | ||
* The sunspider ports are probably easier to work with | * The sunspider ports are probably easier to work with | ||
- | |||
- | * 3d-raytrace (ignoring the actual drawing from JS) | ||
* Sunspider: http:// | * Sunspider: http:// | ||
Line 169: | Line 168: | ||
* [[http:// | * [[http:// | ||
* [[http:// | * [[http:// | ||
+ | * [[http:// | ||
+ | ===== Benchmarking in other languages ===== | ||
+ | |||
+ | * Python: http:// | ||
+ | |||
+ | * Ruby: http:// | ||
+ | |||
+ | ===== Desired Benchmark Features ===== | ||
+ | |||
+ | ==== For command-line apps applications ==== | ||
+ | * Run-time | ||
+ | * Memory usage | ||
+ | * Hardware performance counters (if available), using PAPI | ||
+ | * Simulated hardware statistics, using cachegrind | ||
+ | * These should be combined into a single representative number, using some hardware model. | ||
+ | * Ability to compare all of these over two runs | ||
+ | * Support for other PHP implementations | ||
+ | * Benchmark characterisation: | ||
+ | * This is hard to do properly, so best to do it badly and give coarse grained information. | ||
+ | |||
+ | *TODO* | ||
+ | |||
+ | ==== For web-apps ==== | ||
+ | |||
+ | * Requests per second | ||
+ | * Memory usage | ||
+ | * Bottleneck (is it scripting, DB, or network) | ||
+ | * Total time for request | ||
====== Status ====== | ====== Status ====== | ||
- | | + | ===== Completed Tests ===== |
- | * Currently working on porting DeltaBlue - Ólafur W (07.04.2009) | + | |
+ | | ||
+ | * deltablue (04.05.2009) | ||
+ | * crypto | ||
+ | * whirl & j4p5 | ||
+ | * crypto-md5 (29.06.2009) | ||
+ | * richards (29.06.2009) | ||
+ | * Crypto-AES | ||
+ | |||
+ | ===== Still unfinished ===== | ||
====== Collaboration ====== | ====== Collaboration ====== | ||
Line 202: | Line 239: | ||
* Ólafur Waage - olafurw [at] gmail.com | * Ólafur Waage - olafurw [at] gmail.com | ||
* Michiaki Tatsubori - mich [at] acm.org | * Michiaki Tatsubori - mich [at] acm.org | ||
+ | * Alexander Hjalmarsson - hjalle [at] sgh.se | ||
+ | * Davide Mendolia - idaf1er [at] gmail.com | ||
rfc/better_benchmarks.txt · Last modified: 2017/09/22 13:28 by 127.0.0.1