criterion performance measurements
overview
want to understand this report?
uniplate/simplify/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.4075075728633604e-5 | 3.414117493620788e-5 | 3.419604954351435e-5 |
| Standard deviation | 1.6768107552908869e-7 | 2.0377870408385176e-7 | 2.610045733693887e-7 |
Outlying measurements have no (7.142487448889845e-3%) effect on estimated standard deviation.
uniplate/simplify/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.233738975389385e-5 | 7.258000799316799e-5 | 7.296490565706847e-5 |
| Standard deviation | 6.441042477683648e-7 | 1.038951754389264e-6 | 1.7480886652532348e-6 |
Outlying measurements have slight (8.387036564385976e-2%) effect on estimated standard deviation.
uniplate/variables/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.0191849187478016e-5 | 2.0231439289307102e-5 | 2.0291676453320092e-5 |
| Standard deviation | 1.1422629182239008e-7 | 1.629443714458935e-7 | 2.3181560721279873e-7 |
Outlying measurements have no (6.622222222222315e-3%) effect on estimated standard deviation.
uniplate/variables/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.811584939532622e-5 | 6.821506761991086e-5 | 6.830876169071053e-5 |
| Standard deviation | 2.832220639381815e-7 | 3.3352541981665046e-7 | 4.074948140517933e-7 |
Outlying measurements have no (7.936000000000113e-3%) effect on estimated standard deviation.
uniplate/zeros/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.514983564762956e-5 | 1.5160102160771418e-5 | 1.5168452670653368e-5 |
| Standard deviation | 2.9766323906447786e-8 | 3.3948007468062174e-8 | 3.948712561759532e-8 |
Outlying measurements have no (6.369165023011078e-3%) effect on estimated standard deviation.
uniplate/zeros/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.69755244661296e-5 | 6.707255318338321e-5 | 6.717244901399297e-5 |
| Standard deviation | 2.804028006901511e-7 | 3.35600967548415e-7 | 4.0997358974586223e-7 |
Outlying measurements have no (7.935999999999999e-3%) effect on estimated standard deviation.
uniplate/constFold/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.4382279596170335e-5 | 5.4457053215625805e-5 | 5.453912749501716e-5 |
| Standard deviation | 2.3217373316581523e-7 | 2.733773581394395e-7 | 3.389348827627498e-7 |
Outlying measurements have no (7.691845441980859e-3%) effect on estimated standard deviation.
uniplate/constFold/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.9586663612012954e-4 | 2.9616996265664764e-4 | 2.9657238872469706e-4 |
| Standard deviation | 9.465463726818072e-7 | 1.2250442920181797e-6 | 1.6181012657327342e-6 |
Outlying measurements have slight (1.0415512465373956e-2%) effect on estimated standard deviation.
uniplate/rename/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.024348101703741e-5 | 5.03165422672329e-5 | 5.040241821596497e-5 |
| Standard deviation | 2.306491395662061e-7 | 2.71239634973365e-7 | 3.383142921631374e-7 |
Outlying measurements have no (7.575316123769021e-3%) effect on estimated standard deviation.
uniplate/rename/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.72785087861133e-4 | 3.748763717362824e-4 | 3.7979297315116685e-4 |
| Standard deviation | 2.745979977664433e-6 | 9.7628631668862e-6 | 1.7198532474936448e-5 |
Outlying measurements have moderate (0.17975591803962843%) effect on estimated standard deviation.
uniplate/symbols/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.6180309708111072e-5 | 2.6206971106613014e-5 | 2.6251483031944836e-5 |
| Standard deviation | 6.593779413738662e-8 | 1.1277698455046093e-7 | 2.0348930033036475e-7 |
Outlying measurements have no (6.896219135802323e-3%) effect on estimated standard deviation.
uniplate/symbols/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.2953380206625253e-4 | 2.2984421137627715e-4 | 2.3017715355573966e-4 |
| Standard deviation | 9.081313251005663e-7 | 1.157644863706322e-6 | 1.6470158392917997e-6 |
Outlying measurements have no (9.9e-3%) effect on estimated standard deviation.
uniplate/bill/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.8480067564558225e-5 | 2.8530417443276177e-5 | 2.858578045466083e-5 |
| Standard deviation | 1.4231176146483187e-7 | 1.7019248544062535e-7 | 2.2015786413166602e-7 |
Outlying measurements have no (6.944104846202904e-3%) effect on estimated standard deviation.
uniplate/bill/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.312143343919977e-4 | 3.3184382130946793e-4 | 3.3255549706840673e-4 |
| Standard deviation | 1.8411064591221253e-6 | 2.1869692529331023e-6 | 2.6509331182554715e-6 |
Outlying measurements have slight (1.0637067869117882e-2%) effect on estimated standard deviation.
uniplate/increase800/generic-plate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.80918587629011e-5 | 7.821531386803701e-5 | 7.83683850656784e-5 |
| Standard deviation | 3.9870858416009823e-7 | 4.6060708354491864e-7 | 5.412694203975959e-7 |
Outlying measurements have no (8.12953507121752e-3%) effect on estimated standard deviation.
uniplate/increase800/uniplate
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.47933741003123e-4 | 4.483855630383306e-4 | 4.4881676404212394e-4 |
| Standard deviation | 1.1957274700556056e-6 | 1.4742686943668512e-6 | 2.055302515796853e-6 |
Outlying measurements have slight (1.13621350244418e-2%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.