Skip to content

Few benchmarks for predicate evaluation #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Nov 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
900e474
fix(patch): [sc-3840] Use branch instead of hash to be possible to us…
ser-0xff Oct 3, 2023
c8b9fa1
Merge branch 'apple:main' into feature/sc-3840/use-new-swift-foundati…
hassila Oct 3, 2023
7a11c1a
fix(patch): [sc-3840] Refer branch instead of commit hash.
ser-0xff Oct 3, 2023
5e1da58
fix(patch): [sc-3840] Refer dependencies using branches to avoid SPM …
ser-0xff Oct 3, 2023
90b6914
Merge pull request #2 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff Oct 3, 2023
ccf8872
fix(patch): Rename ordo/swift-foundation to ordo/package-swift-founda…
ser-0xff Oct 4, 2023
83585f0
Merge pull request #3 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff Oct 4, 2023
33d35df
Bump dependency version.
ser-0xff Oct 4, 2023
cac962f
Merge pull request #4 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff Oct 4, 2023
9e05c66
fix(patch): Bump dependency version
ser-0xff Oct 4, 2023
ee5f4db
Merge pull request #5 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff Oct 4, 2023
ca7b5ac
Bump dependency version.
ser-0xff Oct 4, 2023
8f02d74
Merge pull request #6 from ordo-one/feature/sc-3840/use-new-swift-fou…
ser-0xff Oct 4, 2023
cf8c2cb
Merge branch 'apple:main' into main
hassila Oct 11, 2023
ca9daea
Benchmark for predicates.
ser-0xff Oct 11, 2023
974e2b1
Cosmetic fix.
ser-0xff Oct 11, 2023
c0e70fd
Rollback changes in the package.
ser-0xff Oct 11, 2023
12e57df
Make benchmark as a separate package.
ser-0xff Oct 11, 2023
73b8aa4
Change directory structure.
ser-0xff Oct 11, 2023
48477b3
Some more reasonable benchmark defaults.
hassila Oct 11, 2023
48f42c1
Merge branch 'feature/predicate-benchmark' of github.com:ordo-one/swi…
hassila Oct 11, 2023
2436ea2
Add conditions with computed properties.
ser-0xff Oct 12, 2023
a8fcd02
Cosmetic fix.
ser-0xff Oct 12, 2023
b5730a1
Cosmetic fix.
ser-0xff Oct 12, 2023
e97f65b
Add nested computed property benchmarks
ser-0xff Oct 12, 2023
472ce2b
Update Predicates.swift
hassila Oct 12, 2023
80a84c3
Address PR feedback
hassila Nov 6, 2023
f0d9cdf
Add link to failed compilation
hassila Nov 6, 2023
6a4a773
Bump tools to 5.9 for the benchmarks, so we can depend on macOS .v14 …
hassila Nov 6, 2023
05a33a1
Match parent project platforms exactly, reverted required if #availab…
hassila Nov 7, 2023
90e3ae6
Dynamically calculate test number to make it easier to add additional…
hassila Nov 7, 2023
532493d
Use zero numbering for test cases
hassila Nov 7, 2023
ea831a4
Simplify naming and remove counters, makes filtering benchmarks easie…
hassila Nov 7, 2023
d0b8f45
Add multivariate predicate test
hassila Nov 7, 2023
2729432
Polish multivariate test
hassila Nov 7, 2023
e484eb9
Use same naming for multivariate test as for single.
hassila Nov 7, 2023
285ab31
Fix naming for variadic tests, decrease run time to 3 seconds as we s…
hassila Nov 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions Benchmarks/Benchmarks/Predicates/Predicates.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2022-2023 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

import Benchmark
import func Benchmark.blackHole
import FoundationEssentials

let benchmarks = {
Benchmark.defaultConfiguration.maxIterations = 1_000_000_000
Benchmark.defaultConfiguration.maxDuration = .seconds(3)
Benchmark.defaultConfiguration.scalingFactor = .kilo
Benchmark.defaultConfiguration.metrics = .arc + [.cpuTotal, .wallClock, .mallocCountTotal, .throughput] // use ARC to see traffic
// Benchmark.defaultConfiguration.metrics = [.cpuTotal, .wallClock, .mallocCountTotal, .throughput] // skip ARC as it has some overhead
// Benchmark.defaultConfiguration.metrics = .all // Use all metrics to easily see which ones are of interest for this benchmark suite
if #available(macOS 14, *) {

let monster = Monster(name: "Orc", level: 80, hp: 100, mana: 0, weapon: .sword(Sword(p1: 1, p2: 2, p3: 3, p4: 4, p5: 5)))

var predicateTests : [(String, Predicate<Monster>)] = []

predicateTests.append(("predicateTrivialCondition", #Predicate<Monster> { monster in
true
}))

predicateTests.append(("predicateKeypathPropertyCondition", #Predicate<Monster> { monster in
(monster.level == 80)
}))

predicateTests.append(("predicateKeypathComputedPropertyCondition", #Predicate<Monster> { monster in
(monster.levelComputed == 80)
}))

predicateTests.append(("predicateKeypathNestedComputedPropertyCondition", #Predicate<Monster> { monster in
(monster.weaponP1 == 1)
}))

predicateTests.append(("predicateThreeKeypathNestedComputedPropertyCondition", #Predicate<Monster> { monster in
((monster.weaponP1 == 1) &&
(monster.weaponP2 == 2) &&
(monster.weaponP3 == 3))
}))

// This test disabled, as enabling it will make compilation fail due to https://github.com/apple/swift/issues/69277
// predicateTests.append(("predicateFiveKeypathNestedComputedPropertyCondition", #Predicate<Monster> { monster in
// ((monster.weaponP1 == 1) &&
// (monster.weaponP2 == 2) &&
// (monster.weaponP3 == 3) &&
// (monster.weaponP4 == 4) &&
// (monster.weaponP5 == 5))
// }))

predicateTests.forEach { (testDescription, predicate) in
Benchmark(testDescription) { benchmark in
var matched = 0

for _ in benchmark.scaledIterations {
if try predicate.evaluate(monster) {
matched += 1
}
}

guard matched == benchmark.scaledIterations.count else {
fatalError("Internal error: wrong number of matched monsters")
}
}
}

var variadicPredicateTests : [(String, Predicate<Monster, Monster>)] = []
let monster2 = Monster(name: "Orc", level: 80, hp: 100, mana: 0, weapon: .sword(Sword(p1: 1, p2: 2, p3: 3, p4: 4, p5: 5)))

variadicPredicateTests.append(("predicateVariadicThreeKeypathNestedComputedPropertyCondition",
#Predicate<Monster, Monster> { monster, monster2 in
((monster.weaponP1 == 1) &&
(monster.weaponP2 == 2) &&
(monster2.weaponP2 == 2))
}))

variadicPredicateTests.forEach { (testDescription, predicate) in
Benchmark(testDescription) { benchmark in
var matched = 0

for _ in benchmark.scaledIterations {
if try predicate.evaluate(monster, monster2) {
matched += 1
}
}

guard matched == benchmark.scaledIterations.count else {
fatalError("Internal error: wrong number of matched monsters")
}
}
}
}
}
110 changes: 110 additions & 0 deletions Benchmarks/Benchmarks/Predicates/PredicatesTypes.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift.org open source project
//
// Copyright (c) 2022-2023 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
// See https://swift.org/CONTRIBUTORS.txt for the list of Swift project authors
//
//===----------------------------------------------------------------------===//

struct Mace {
let p1: Int
let p2: Int
let p3: Int
let p4: Int
let p5: Int
}

struct Sword {
let p1: Int
let p2: Int
let p3: Int
let p4: Int
let p5: Int
}

struct Lightsaber {
let p1: Int
let p2: Int
let p3: Int
let p4: Int
let p5: Int
}

enum Weapon {
case mace(Mace)
case sword(Sword)
case lightsaber(Lightsaber)

var p1: Int {
switch self {
case let .mace(mace):
mace.p1
case let .sword(sword):
sword.p1
case let .lightsaber(lighsaber):
lighsaber.p1
}
}

var p2: Int {
switch self {
case let .mace(mace):
mace.p2
case let .sword(sword):
sword.p2
case let .lightsaber(lighsaber):
lighsaber.p2
}
}

var p3: Int {
switch self {
case let .mace(mace):
mace.p3
case let .sword(sword):
sword.p3
case let .lightsaber(lighsaber):
lighsaber.p3
}
}

var p4: Int {
switch self {
case let .mace(mace):
mace.p4
case let .sword(sword):
sword.p4
case let .lightsaber(lighsaber):
lighsaber.p4
}
}

var p5: Int {
switch self {
case let .mace(mace):
mace.p5
case let .sword(sword):
sword.p5
case let .lightsaber(lighsaber):
lighsaber.p5
}
}
}

struct Monster {
let name: String
var level: Int
var hp: Int
var mana: Int
var weapon: Weapon?
var levelComputed: Int { level }
var weaponP1: Int? { weapon?.p1 }
var weaponP2: Int? { weapon?.p2 }
var weaponP3: Int? { weapon?.p3 }
var weaponP4: Int? { weapon?.p4 }
var weaponP5: Int? { weapon?.p5 }
}
25 changes: 25 additions & 0 deletions Benchmarks/Package.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// swift-tools-version: 5.9

import PackageDescription

let package = Package(
name: "benchmarks",
platforms: [.macOS("13.3"), .iOS("16.4"), .tvOS("16.4"), .watchOS("9.4")], // Should match parent project
dependencies: [
.package(path: "../"),
.package(url: "https://github.com/ordo-one/package-benchmark.git", from: "1.11.1"),
],
targets: [
.executableTarget(
name: "PredicateBenchmarks",
dependencies: [
.product(name: "FoundationEssentials", package: "swift-foundation"),
.product(name: "Benchmark", package: "package-benchmark"),
],
path: "Benchmarks/Predicates",
plugins: [
.plugin(name: "BenchmarkPlugin", package: "package-benchmark")
]
),
]
)