Skip to content

BLD: test_perf.py, add --base-pickle --target-pickle options to test_perf #3734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
2 commits merged into from Jun 1, 2013
Merged

BLD: test_perf.py, add --base-pickle --target-pickle options to test_perf #3734

2 commits merged into from Jun 1, 2013

Conversation

ghost
Copy link

@ghost ghost commented Jun 1, 2013

Until now test_perf either generated a single commit report for HEAD,
or fell back to vbench to compare two commits. with this change, can now
compare results between saved results of test_perf -H invocations.

Flow:

  • Use build_cache (cdev from Tips and tricks for pandas devs #3156) to jump to desired commit using build_cache
  • checkout current vb_suite from upstream/master
  • use test_perf -H -d <commit>-foo.pickle to save timings to file, for
    target and base commits
  • Use test_perf options --base-pickle, --target-pickle to generate comparison report.

cc @jreback

λ ./test_perf.sh --base-pickle 31ecaa9-0.10.1.pickle --target-pickle f9eea30-0.11.0.pickle
This script compares the performance of two commits.
Make sure the python 'vbench' library is installed.

Setting the BUILD_CACHE_DIR env var to a temp directory will
potentially speed up subsequent runs.

base_commit not specified, Assuming base_pickle is named <commit>-foo.*
target_commit not specified, Assuming target_pickle is named <commit>-foo.*
f9eea30
Verifying specified commits exist in repo...
31ecaa9: RLS: set released to true
f9eea30: RLS: Version 0.11
*** 
Invoked with :
--ncalls: 3
--repeats: 3


-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
frame_reindex_columns                        |   0.3260 |   0.2900 |   1.1241 |
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------

Ratio < 1.0 means the target commit is faster then the baseline.
Seed used: 1234

Target [f9eea30] : RLS: Version 0.11
Base   [31ecaa9] : RLS: set released to true

ghost pushed a commit that referenced this pull request Jun 1, 2013
 BLD: test_perf.py, add --base-pickle --target-pickle options to test_perf
@ghost ghost merged commit 63d8998 into pandas-dev:master Jun 1, 2013
@ghost ghost deleted the PR_test_perf_pickle_compare branch June 1, 2013 19:50
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants