From dc19b708c56a168b4c9dcf5d1bf7c2cae0a04c71 Mon Sep 17 00:00:00 2001 From: Sam T Date: Sun, 31 Mar 2013 15:49:46 +0400 Subject: add bench --- bench/Main.hs | 67 +++++ bench/comparison.html | 806 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 873 insertions(+) create mode 100644 bench/Main.hs create mode 100644 bench/comparison.html (limited to 'bench') diff --git a/bench/Main.hs b/bench/Main.hs new file mode 100644 index 0000000..279842f --- /dev/null +++ b/bench/Main.hs @@ -0,0 +1,67 @@ +{-# LANGUAGE PackageImports #-} +module Main (main) where + +import Control.DeepSeq +import Data.Maybe +import Data.Attoparsec.ByteString as Atto +import Data.ByteString as B +import Data.ByteString.Lazy as BL +import Criterion.Main +import System.Environment + +import "bencode" Data.BEncode as A +import Data.AttoBencode as B +import Data.AttoBencode.Parser as B +import "bencoding" Data.BEncode as C + +instance NFData A.BEncode where + rnf (A.BInt i) = rnf i + rnf (A.BString s) = rnf s + rnf (A.BList l) = rnf l + rnf (A.BDict m) = rnf m + +instance NFData B.BValue where + rnf (B.BInt i) = rnf i + rnf (B.BString s) = rnf s + rnf (B.BList l) = rnf l + rnf (B.BDict d) = rnf d + +instance NFData C.BEncode where + rnf (C.BInteger i) = rnf i + rnf (C.BString s) = rnf s + rnf (C.BList l) = rnf l + rnf (C.BDict d) = rnf d + +getRight :: Either String a -> a +getRight (Left e) = error e +getRight (Right x) = x + +main :: IO () +main = do + (path : args) <- getArgs + torrentFile <- B.readFile path + let lazyTorrentFile = fromChunks [torrentFile] + + case rnf (torrentFile, lazyTorrentFile) of + () -> return () + + withArgs args $ + defaultMain + [ bench "decode/bencode" $ nf A.bRead lazyTorrentFile + , bench "decode/AttoBencode" $ nf (getRight . Atto.parseOnly bValue) torrentFile + , bench "decode/b-encode" $ nf (getRight . C.decode) torrentFile + + , let Just v = A.bRead lazyTorrentFile in + bench "encode/bencode" $ nf A.bPack v + , let Right v = Atto.parseOnly bValue torrentFile in + bench "encode/AttoBencode" $ nf B.encode v + , let Right v = C.decode torrentFile in + bench "encode/b-encode" $ nf C.encode v + + , bench "decode+encode/bencode" $ nf (A.bPack . fromJust . A.bRead) + lazyTorrentFile + , bench "decode+encode/AttoBencode" $ nf (B.encode . getRight . Atto.parseOnly bValue) + torrentFile + , bench "decode+encode/b-encode" $ nf (C.encode . getRight . C.decode) + torrentFile + ] diff --git a/bench/comparison.html b/bench/comparison.html new file mode 100644 index 0000000..a613cea --- /dev/null +++ b/bench/comparison.html @@ -0,0 +1,806 @@ + + + + + criterion report + + + + + + + + +
+
+

criterion performance measurements

+ +

overview

+ +

want to understand this report?

+ +
+ +

decode/bencode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time2.1863382999994313e-42.1937343849211334e-42.2049485730881596e-4
Standard deviation3.2937910958181315e-64.590843178568767e-66.750357183347315e-6
+ + +

Outlying measurements have moderate + (0.14197240019484916%) + effect on estimated standard deviation.

+ +

decode/AttoBencode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time2.187692583553376e-52.201425493709624e-52.241606971574208e-5
Standard deviation4.662239278481744e-71.0963252840598192e-62.3666466731813973e-6
+ + +

Outlying measurements have moderate + (0.4842496881368733%) + effect on estimated standard deviation.

+ +

decode/b-encode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time1.282347230067164e-51.3550721529107661e-51.6246955776066507e-5
Standard deviation1.5982100082029483e-66.233023270045819e-61.4128011325087792e-5
+ + +

Outlying measurements have severe + (0.9892762013650902%) + effect on estimated standard deviation.

+ +

encode/bencode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time1.4411505527701358e-51.4534557291439606e-51.4914383550126746e-5
Standard deviation3.8302057349891327e-71.0067970172105852e-62.2149203967007296e-6
+ + +

Outlying measurements have severe + (0.6458150544551003%) + effect on estimated standard deviation.

+ +

encode/AttoBencode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time1.831606229322615e-51.838992762692543e-51.8495424146026025e-5
Standard deviation3.3635788025947543e-74.478271892551609e-75.91450793126853e-7
+ + +

Outlying measurements have moderate + (0.18048812021751406%) + effect on estimated standard deviation.

+ +

encode/b-encode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time8.655272731286693e-68.66695453836302e-68.682124711670457e-6
Standard deviation5.402771267923524e-86.772790614661658e-89.053197956983826e-8
+ + +

Outlying measurements have no + (9.899999999999942e-3%) + effect on estimated standard deviation.

+ +

decode+encode/bencode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time2.389043013473133e-42.4096515486528587e-42.4453794897257616e-4
Standard deviation9.295450653777997e-61.3613324768426126e-52.429004193978993e-5
+ + +

Outlying measurements have severe + (0.5446476041541671%) + effect on estimated standard deviation.

+ +

decode+encode/AttoBencode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time4.390556836078146e-54.412782356268218e-54.445992661874833e-5
Standard deviation1.0201549401119706e-61.3726195141175464e-61.9415018031670644e-6
+ + +

Outlying measurements have moderate + (0.2673392111143691%) + effect on estimated standard deviation.

+ +

decode+encode/b-encode

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time2.1623848928976316e-52.1709573653639946e-52.1848333266677047e-5
Standard deviation3.8526427612676155e-75.47401627275277e-78.120960293222424e-7
+ + +

Outlying measurements have moderate + (0.19016961700791796%) + effect on estimated standard deviation.

+ + +

understanding this report

+ +

In this report, each function benchmarked by criterion is assigned + a section of its own. In each section, we display two charts, each + with an x axis that represents measured execution time. + These charts are active; if you hover your mouse over data points + and annotations, you will see more details.

+ +
    +
  • The chart on the left is a + kernel + density estimate (also known as a KDE) of time + measurements. This graphs the probability of any given time + measurement occurring. A spike indicates that a measurement of a + particular time occurred; its height indicates how often that + measurement was repeated.
    • + +
  • The chart on the right is the raw data from which the kernel + density estimate is built. Measurements are displayed on + the y axis in the order in which they occurred.
    • +
+ +

Under the charts is a small table displaying the mean and standard + deviation of the measurements. We use a statistical technique + called + the bootstrap + to provide confidence intervals on our estimates of these values. + The bootstrap-derived upper and lower bounds on the mean and + standard deviation let you see how accurate we believe those + estimates to be. (Hover the mouse over the table headers to see + the confidence levels.)

+ +

A noisy benchmarking environment can cause some or many + measurements to fall far from the mean. These outlying + measurements can have a significant inflationary effect on the + estimate of the standard deviation. We calculate and display an + estimate of the extent to which the standard deviation has been + inflated by outliers.

+ + + +
+
+ + + -- cgit v1.2.3