Import Icinga Notifications internal/utils

[oxzi]

This PR imports the internal/utils package from Icinga Notifications into utils for general purpose functions and into database for database-related utility functions. The commit history was kept, but messages were slightly altered since the original messages often referred changes outside the internal/utils package.

My motivation for this merge was to use a utility function already implemented, but residing in the wrong project's utility package. Since the IGL is the place for common code, I would like to get rid of utility packages outside this repository.

Steps performed to create this PR

Create Patches

Within the icinga-notifications repository.

$ cd internal/utils
$ git format-patch --relative --root -- ./
0001-Introduce-utils-package.patch
0002-Move-helper-function-to-utils-package.patch
0003-Implement-incremental-config-updates-for-rules.patch
0004-Add-utils-ToDBInt-function.patch
0005-Use-sqlx.Tx-for-event-processing.patch
0006-Use-ctx-everywhere-cache-object-only-when-the-tx-suc.patch
0007-Trigger-time-based-escalations.patch
0008-Rename-and-simplify-restoreAttrsFor-function.patch
0009-Make-sure-to-release-semaphore-in-utils-ForEachRow.patch
0010-utils.IterateOrderedMap-for-plugin.FormatMessage.patch
0011-Switch-to-icinga-go-library.patch
0012-Drop-unused-Remove-utils-functions.patch
0013-Restore-all-muted-objects-without-an-incident-on-dae.patch
0014-Make-logging-more-consistent.patch
0015-Use-utils.IterateOrderedMap-as-intended-via-for-rang.patch
$ cd -
$ mv internal/utils/*.patch .
$ ls *.patch

Apply Patches

Within this repository.

First, create a githook(5) to verify each applied patch works.
Plot twist: each patch will fail now.

$ cat .git/hooks/pre-applypatch
#!/usr/bin/env bash
#
# An example hook script to verify what is about to be committed
# by applypatch from an e-mail message.
#
# The hook should exit with non-zero status after issuing an
# appropriate message if it wants to stop the commit.
#
# To enable this hook, rename this file to "pre-applypatch".

. git-sh-setup
precommit="$(git rev-parse --git-path hooks/pre-commit)"
test -x "$precommit" && exec "$precommit" ${1+"$@"}

go build
go test -race -count=1 ./...

Start applying patches:

$ git am -3 --directory utils/ ../icinga-notifications/*.patch

Each patch now requires some manual intervention, e.g., making it compile or manually move some code to ./database/util.go.

If the current patch works, build it manually once and continue afterwards.

$ git add utils/ database/ && git am --continue

The 0011-Switch-to-icinga-go-library.patch patch was skipped since it does not made sense in this context.

Verifying Methods

To make sure that nothing was lost, I have manually diffed each function.

@@ -1,5 +1,5 @@
 // BuildInsertStmtWithout builds an insert stmt without the provided column.
-func BuildInsertStmtWithout(db *database.DB, into interface{}, withoutColumn string) string {
+func BuildInsertStmtWithout(db *DB, into interface{}, withoutColumn string) string {
        columns := db.BuildColumns(into)
        for i, column := range columns {
                if column == withoutColumn {
@@ -11,7 +11,7 @@

        return fmt.Sprintf(
                `INSERT INTO "%s" ("%s") VALUES (%s)`,
-               database.TableName(into), strings.Join(columns, `", "`),
+               TableName(into), strings.Join(columns, `", "`),
                fmt.Sprintf(":%s", strings.Join(columns, ", :")),
        )
 }

@@ -3,7 +3,7 @@
 // A new transaction is started on db which is then passed to fn. After fn returns, the transaction is
 // committed unless an error was returned. If fn returns an error, that error is returned, otherwise an
 // error is returned if a database operation fails.
-func RunInTx(ctx context.Context, db *database.DB, fn func(tx *sqlx.Tx) error) error {
+func RunInTx(ctx context.Context, db *DB, fn func(tx *sqlx.Tx) error) error {
        tx, err := db.BeginTxx(ctx, nil)
        if err != nil {
                return err

@@ -1,7 +1,7 @@
 // InsertAndFetchId executes the given query and fetches the last inserted ID.
 func InsertAndFetchId(ctx context.Context, tx *sqlx.Tx, stmt string, args any) (int64, error) {
        var lastInsertId int64
-       if tx.DriverName() == database.PostgreSQL {
+       if tx.DriverName() == PostgreSQL {
                preparedStmt, err := tx.PrepareNamedContext(ctx, stmt+" RETURNING id")
                if err != nil {
                        return 0, err

@@ -1,7 +1,7 @@
 // ExecAndApply applies the provided restoreFunc callback for each successfully retrieved row of the specified type.
 // Returns error on any database failure or fails to acquire the table semaphore.
-func ExecAndApply[Row any](ctx context.Context, db *database.DB, stmt string, args []interface{}, restoreFunc func(*Row)) error {
-       table := database.TableName(new(Row))
+func ExecAndApply[Row any](ctx context.Context, db *DB, stmt string, args []interface{}, restoreFunc func(*Row)) error {
+       table := TableName(new(Row))
        sem := db.GetSemaphoreForTable(table)
        if err := sem.Acquire(ctx, 1); err != nil {
                return errors.Wrapf(err, "cannot acquire semaphore for table %q", table)

@@ -1,7 +1,7 @@
 // ForEachRow applies the provided restoreFunc callback for each successfully retrieved row of the specified type.
 // It will bulk SELECT the data from the database scoped to the specified ids and scans into the provided Row type.
 // Returns error on any database failure or fails to acquire the table semaphore.
-func ForEachRow[Row, Id any](ctx context.Context, db *database.DB, idColumn string, ids []Id, restoreFunc func(*Row)) error {
+func ForEachRow[Row, Id any](ctx context.Context, db *DB, idColumn string, ids []Id, restoreFunc func(*Row)) error {
        subject := new(Row)
        query := fmt.Sprintf("%s WHERE %q IN (?)", db.BuildSelectStmt(subject, subject), idColumn)
        stmt, args, err := sqlx.In(query, ids)

No diff for ToDBString, ToDBInt, and IterateOrderedMap.

Reword Commit Messages

Since most commit messages referred a certain change and the changes in the util.go file was a drive-by change, I have reworded each commit message and linked the original commit on GitHub.

Cleanup

As further specified in #127 (comment), some functions and commits were dropped again.

[oxzi]

Additional changes after a note from @lippserd:

• Dropped RunInTx as the almost identical DB.ExecTx method was already added in Introduce DB#ExecTx() method #63
• Dropped InsertAndFetchId in favor for InsertObtainID from Introduce DB#InsertObtainID() function #64

Furthermore, I have removed the two commits adding Remove{If,Nils} and removing them again.

[oxzi]

I have successfully tested Icinga Notifications with this changes in Icinga/icinga-notifications#310 and a replace statement in the go.mod file.