-
Notifications
You must be signed in to change notification settings - Fork 201
Few benchmarks for predicate evaluation #291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
jmschonfeld
merged 37 commits into
swiftlang:main
from
ordo-one:feature/predicate-benchmark
Nov 7, 2023
Merged
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
900e474
fix(patch): [sc-3840] Use branch instead of hash to be possible to us…
ser-0xff c8b9fa1
Merge branch 'apple:main' into feature/sc-3840/use-new-swift-foundati…
hassila 7a11c1a
fix(patch): [sc-3840] Refer branch instead of commit hash.
ser-0xff 5e1da58
fix(patch): [sc-3840] Refer dependencies using branches to avoid SPM …
ser-0xff 90b6914
Merge pull request #2 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff ccf8872
fix(patch): Rename ordo/swift-foundation to ordo/package-swift-founda…
ser-0xff 83585f0
Merge pull request #3 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff 33d35df
Bump dependency version.
ser-0xff cac962f
Merge pull request #4 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff 9e05c66
fix(patch): Bump dependency version
ser-0xff ee5f4db
Merge pull request #5 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff ca7b5ac
Bump dependency version.
ser-0xff 8f02d74
Merge pull request #6 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff cf8c2cb
Merge branch 'apple:main' into main
hassila ca9daea
Benchmark for predicates.
ser-0xff 974e2b1
Cosmetic fix.
ser-0xff c0e70fd
Rollback changes in the package.
ser-0xff 12e57df
Make benchmark as a separate package.
ser-0xff 73b8aa4
Change directory structure.
ser-0xff 48477b3
Some more reasonable benchmark defaults.
hassila 48f42c1
Merge branch 'feature/predicate-benchmark' of github.com:ordo-one/swi…
hassila 2436ea2
Add conditions with computed properties.
ser-0xff a8fcd02
Cosmetic fix.
ser-0xff b5730a1
Cosmetic fix.
ser-0xff e97f65b
Add nested computed property benchmarks
ser-0xff 472ce2b
Update Predicates.swift
hassila 80a84c3
Address PR feedback
hassila f0d9cdf
Add link to failed compilation
hassila 6a4a773
Bump tools to 5.9 for the benchmarks, so we can depend on macOS .v14 …
hassila 05a33a1
Match parent project platforms exactly, reverted required if #availab…
hassila 90e3ae6
Dynamically calculate test number to make it easier to add additional…
hassila 532493d
Use zero numbering for test cases
hassila ea831a4
Simplify naming and remove counters, makes filtering benchmarks easie…
hassila d0b8f45
Add multivariate predicate test
hassila 2729432
Polish multivariate test
hassila e484eb9
Use same naming for multivariate test as for single.
hassila 285ab31
Fix naming for variadic tests, decrease run time to 3 seconds as we s…
hassila File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,103 @@ | ||
//===----------------------------------------------------------------------===// | ||
// | ||
// This source file is part of the Swift.org open source project | ||
// | ||
// Copyright (c) 2022-2023 Apple Inc. and the Swift project authors | ||
// Licensed under Apache License v2.0 with Runtime Library Exception | ||
// | ||
// See https://swift.org/LICENSE.txt for license information | ||
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors | ||
// | ||
//===----------------------------------------------------------------------===// | ||
|
||
import Benchmark | ||
import func Benchmark.blackHole | ||
import FoundationEssentials | ||
|
||
let benchmarks = { | ||
Benchmark.defaultConfiguration.maxIterations = 1_000_000_000 | ||
Benchmark.defaultConfiguration.maxDuration = .seconds(3) | ||
Benchmark.defaultConfiguration.scalingFactor = .kilo | ||
Benchmark.defaultConfiguration.metrics = .arc + [.cpuTotal, .wallClock, .mallocCountTotal, .throughput] // use ARC to see traffic | ||
// Benchmark.defaultConfiguration.metrics = [.cpuTotal, .wallClock, .mallocCountTotal, .throughput] // skip ARC as it has some overhead | ||
// Benchmark.defaultConfiguration.metrics = .all // Use all metrics to easily see which ones are of interest for this benchmark suite | ||
if #available(macOS 14, *) { | ||
|
||
let monster = Monster(name: "Orc", level: 80, hp: 100, mana: 0, weapon: .sword(Sword(p1: 1, p2: 2, p3: 3, p4: 4, p5: 5))) | ||
|
||
var predicateTests : [(String, Predicate<Monster>)] = [] | ||
|
||
predicateTests.append(("predicateTrivialCondition", #Predicate<Monster> { monster in | ||
true | ||
})) | ||
|
||
predicateTests.append(("predicateKeypathPropertyCondition", #Predicate<Monster> { monster in | ||
(monster.level == 80) | ||
})) | ||
|
||
predicateTests.append(("predicateKeypathComputedPropertyCondition", #Predicate<Monster> { monster in | ||
(monster.levelComputed == 80) | ||
})) | ||
|
||
predicateTests.append(("predicateKeypathNestedComputedPropertyCondition", #Predicate<Monster> { monster in | ||
(monster.weaponP1 == 1) | ||
})) | ||
|
||
predicateTests.append(("predicateThreeKeypathNestedComputedPropertyCondition", #Predicate<Monster> { monster in | ||
((monster.weaponP1 == 1) && | ||
(monster.weaponP2 == 2) && | ||
(monster.weaponP3 == 3)) | ||
})) | ||
|
||
// This test disabled, as enabling it will make compilation fail due to https://github.com/apple/swift/issues/69277 | ||
// predicateTests.append(("predicateFiveKeypathNestedComputedPropertyCondition", #Predicate<Monster> { monster in | ||
// ((monster.weaponP1 == 1) && | ||
// (monster.weaponP2 == 2) && | ||
// (monster.weaponP3 == 3) && | ||
// (monster.weaponP4 == 4) && | ||
// (monster.weaponP5 == 5)) | ||
// })) | ||
|
||
predicateTests.forEach { (testDescription, predicate) in | ||
Benchmark(testDescription) { benchmark in | ||
var matched = 0 | ||
|
||
for _ in benchmark.scaledIterations { | ||
if try predicate.evaluate(monster) { | ||
matched += 1 | ||
} | ||
} | ||
|
||
guard matched == benchmark.scaledIterations.count else { | ||
fatalError("Internal error: wrong number of matched monsters") | ||
} | ||
} | ||
} | ||
|
||
var variadicPredicateTests : [(String, Predicate<Monster, Monster>)] = [] | ||
let monster2 = Monster(name: "Orc", level: 80, hp: 100, mana: 0, weapon: .sword(Sword(p1: 1, p2: 2, p3: 3, p4: 4, p5: 5))) | ||
|
||
variadicPredicateTests.append(("predicateVariadicThreeKeypathNestedComputedPropertyCondition", | ||
#Predicate<Monster, Monster> { monster, monster2 in | ||
((monster.weaponP1 == 1) && | ||
(monster.weaponP2 == 2) && | ||
(monster2.weaponP2 == 2)) | ||
})) | ||
|
||
variadicPredicateTests.forEach { (testDescription, predicate) in | ||
Benchmark(testDescription) { benchmark in | ||
var matched = 0 | ||
|
||
for _ in benchmark.scaledIterations { | ||
if try predicate.evaluate(monster, monster2) { | ||
matched += 1 | ||
} | ||
} | ||
|
||
guard matched == benchmark.scaledIterations.count else { | ||
fatalError("Internal error: wrong number of matched monsters") | ||
} | ||
} | ||
} | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
//===----------------------------------------------------------------------===// | ||
// | ||
// This source file is part of the Swift.org open source project | ||
// | ||
// Copyright (c) 2022-2023 Apple Inc. and the Swift project authors | ||
// Licensed under Apache License v2.0 with Runtime Library Exception | ||
// | ||
// See https://swift.org/LICENSE.txt for license information | ||
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors | ||
// | ||
//===----------------------------------------------------------------------===// | ||
|
||
struct Mace { | ||
let p1: Int | ||
let p2: Int | ||
let p3: Int | ||
let p4: Int | ||
let p5: Int | ||
} | ||
|
||
struct Sword { | ||
let p1: Int | ||
let p2: Int | ||
let p3: Int | ||
let p4: Int | ||
let p5: Int | ||
} | ||
|
||
struct Lightsaber { | ||
let p1: Int | ||
let p2: Int | ||
let p3: Int | ||
let p4: Int | ||
let p5: Int | ||
} | ||
|
||
enum Weapon { | ||
case mace(Mace) | ||
case sword(Sword) | ||
case lightsaber(Lightsaber) | ||
|
||
var p1: Int { | ||
switch self { | ||
case let .mace(mace): | ||
mace.p1 | ||
case let .sword(sword): | ||
sword.p1 | ||
case let .lightsaber(lighsaber): | ||
lighsaber.p1 | ||
} | ||
} | ||
|
||
var p2: Int { | ||
switch self { | ||
case let .mace(mace): | ||
mace.p2 | ||
case let .sword(sword): | ||
sword.p2 | ||
case let .lightsaber(lighsaber): | ||
lighsaber.p2 | ||
} | ||
} | ||
|
||
var p3: Int { | ||
switch self { | ||
case let .mace(mace): | ||
mace.p3 | ||
case let .sword(sword): | ||
sword.p3 | ||
case let .lightsaber(lighsaber): | ||
lighsaber.p3 | ||
} | ||
} | ||
|
||
var p4: Int { | ||
switch self { | ||
case let .mace(mace): | ||
mace.p4 | ||
case let .sword(sword): | ||
sword.p4 | ||
case let .lightsaber(lighsaber): | ||
lighsaber.p4 | ||
} | ||
} | ||
|
||
var p5: Int { | ||
switch self { | ||
case let .mace(mace): | ||
mace.p5 | ||
case let .sword(sword): | ||
sword.p5 | ||
case let .lightsaber(lighsaber): | ||
lighsaber.p5 | ||
} | ||
} | ||
} | ||
|
||
struct Monster { | ||
let name: String | ||
var level: Int | ||
var hp: Int | ||
var mana: Int | ||
var weapon: Weapon? | ||
var levelComputed: Int { level } | ||
var weaponP1: Int? { weapon?.p1 } | ||
var weaponP2: Int? { weapon?.p2 } | ||
var weaponP3: Int? { weapon?.p3 } | ||
var weaponP4: Int? { weapon?.p4 } | ||
var weaponP5: Int? { weapon?.p5 } | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
// swift-tools-version: 5.9 | ||
|
||
import PackageDescription | ||
|
||
let package = Package( | ||
name: "benchmarks", | ||
platforms: [.macOS("13.3"), .iOS("16.4"), .tvOS("16.4"), .watchOS("9.4")], // Should match parent project | ||
dependencies: [ | ||
.package(path: "../"), | ||
.package(url: "https://github.com/ordo-one/package-benchmark.git", from: "1.11.1"), | ||
], | ||
targets: [ | ||
.executableTarget( | ||
name: "PredicateBenchmarks", | ||
dependencies: [ | ||
.product(name: "FoundationEssentials", package: "swift-foundation"), | ||
.product(name: "Benchmark", package: "package-benchmark"), | ||
], | ||
path: "Benchmarks/Predicates", | ||
jmschonfeld marked this conversation as resolved.
Show resolved
Hide resolved
|
||
plugins: [ | ||
.plugin(name: "BenchmarkPlugin", package: "package-benchmark") | ||
] | ||
), | ||
] | ||
) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.