Skip to content

Benchmark_Driver produces benchmark log in format which compare_perf_tests doesn't understand #82605

Open
@valeriyvan

Description

@valeriyvan

Description

Command

utils/build-script --benchmark \
  --swift-darwin-supported-archs "$(uname -m)" \
  --release-debuginfo --swift-disable-dead-stripping \
  --bootstrapping=hosttools

produces benchmark log which looks like:

<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 1, "name": "Ackermann2", "num_samples": 200, "min": 769.0, "median": 786.0, "max": 1953.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 2, "name": "AngryPhonebook", "num_samples": 200, "min": 735.0, "median": 735.0, "max": 763.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 3, "name": "AngryPhonebook.ASCII2", "num_samples": 200, "min": 29.0, "median": 29.0, "max": 30.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 4, "name": "AngryPhonebook.ASCII2.Small", "num_samples": 200, "min": 331.0, "median": 332.0, "max": 388.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 5, "name": "AngryPhonebook.Armenian", "num_samples": 200, "min": 345.0, "median": 345.0, "max": 357.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 6, "name": "AngryPhonebook.Armenian.Small", "num_samples": 200, "min": 134.0, "median": 134.0, "max": 144.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 7, "name": "AngryPhonebook.Cyrillic", "num_samples": 200, "min": 368.0, "median": 369.0, "max": 376.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 8, "name": "AngryPhonebook.Cyrillic.Small", "num_samples": 200, "min": 142.0, "median": 142.0, "max": 146.0})>
<bound method PerformanceTestResult.json of PerformanceTestResult({"number": 9, "name": "AngryPhonebook.Strasse", "num_samples": 200, "min": 328.0, "median": 329.0, "max": 334.0})>
...

Reproduction

utils/build-script --benchmark \
  --swift-darwin-supported-archs "$(uname -m)" \
  --release-debuginfo --swift-disable-dead-stripping \
  --bootstrapping=hosttools

Expected behavior

Benchmark log should be written in one of supported formats.

Environment

swiftc -version
swift-driver version: 1.120.5 Apple Swift version 6.1.2 (swiftlang-6.1.2.1.2 clang-1700.0.13.5)
Target: arm64-apple-macosx15

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugA deviation from expected or documented behavior. Also: expected but undesirable behavior.triage neededThis issue needs more specific labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions