Skip to content

Conversation

ceshine
Copy link
Contributor

@ceshine ceshine commented Sep 5, 2025

Overview of the Changes

This pull request introduces a major modernization of the project's tooling, dependency management, and testing infrastructure. The build system has been migrated from setup.py to the standard pyproject.toml, and the CI/CD pipeline has been overhauled to use uv for significantly faster dependency management. A comprehensive test suite for the callback modules has been added, greatly improving code reliability.

Furthermore, the request includes numerous enhancements and bug fixes across the library, such as updating the TelegramCallback to work with modern libraries, improving the clarity and type safety of custom metrics, and migrating to the latest PyTorch LRScheduler API.

Key Changes

  • Project Modernization & Tooling

    • Build System: Migrated from setup.py to pyproject.toml, adopting the modern standard for Python packaging.
    • CI/CD Pipeline: The CircleCI configuration has been completely revamped. It now uses uv for dependency installation, which is significantly faster than the previous conda/pip setup. A ruff linting step has also been added to enforce code quality.
    • Code Formatting & Linting: Introduced ruff for code formatting and linting, with configuration added in a new .pre-commit-config.yaml file to automate style guide adherence.
  • Comprehensive Testing

    • A new test file, test_callbacks.py, has been created, adding extensive unit and integration tests for all callbacks.
    • Tests for TelegramCallback now use unittest.mock.AsyncMock to avoid making real network calls.
    • Integration tests for LookaheadCallback and LookaheadModelCheckpoint verify correct parameter swapping and checkpointing behavior.
    • Augmentation callback tests (MixUp, CutMix, SnapMix) have been added to ensure their transformations are applied correctly.
  • Callback Improvements

    • TelegramCallback:
      • Updated to use event_loop.run_until_complete for compatibility with the latest python-telegram-bot library.
      • Now intelligently skips sending metrics during the initial sanity validation run.
      • Reads metrics from trainer.logged_metrics for better compatibility with newer PyTorch Lightning versions.
      • The on_exception hook has been re-enabled to report training failures.
    • LookaheadCallback & LookaheadModelCheckpoint:
      • Now explicitly assert that the attached optimizer is an instance of Lookahead, preventing incorrect usage and silent failures.
    • RandomAugmentationChoiceCallback:
      • Fixed a typing error and replaced np.random.choice with random.choices for better performance and consistency.
  • Metrics Module Refinements

    • The base GlobalMetric class has been updated to align with recent changes in torchmetrics, removing the deprecated compute_on_step parameter.
    • The internal state has been renamed from target to targets for improved clarity.
    • AUC and FBeta metrics now issue warnings on initialization to clarify their specialized behavior (coalescing classes) and recommend standard torchmetrics alternatives for general use cases.
    • Code within compute methods has been refactored for better readability and type safety using explicit casts.
  • Other Notable Enhancements

    • ScreenLogger: Enhanced to sanitize hyperparameters, ensuring all values are JSON-serializable before being pretty-printed to the console.
    • Learning Rate Schedulers: Migrated from the deprecated _LRScheduler to the public torch.optim.lr_scheduler.LRScheduler API.
    • Documentation: The README.md file has been updated with a disclaimer, uv installation instructions, and a new "Release Notes" section.

New Dependencies Added

  • Build System:
    • hatchling: Added as the build backend in pyproject.toml.
  • Development:
    • pytest, pytest-env: Formalized as development dependencies for running the new test suite.
  • Optional:
    • python-telegram-bot: Defined as an optional dependency under the [telegram] extra, so it is only installed when needed.

Disclosure: the above pull request was written with the assistance of Gemini 2.5 Pro

- Add [project.optional-dependencies] with a telegram extra (python-telegram-bot>=22.3).
- Remove the telegram entry from [dependency-groups].
- Consumers should now install via extras, e.g. pip install .[telegram].

Commit message generated by GPT-5
- Add _sanitize_value and sanitize_dict_like_object to ensure
  hyperparams/metrics are JSON-serializable.
- Pretty-print sanitized hyperparams in log_hyperparams.
- Add @OverRide annotations and save_dir property.
- Minor: adjust val_* metric spacing and refine type hints.

Commit message generated by GPT-5
- Replace _LRScheduler with LRScheduler in imports, bases, and type hints
- BaseLRScheduler now derives from LRScheduler
- MultiStageScheduler now sets optimizer from first sub-scheduler,
  avoids super().__init__, and calls _initial_step() for initial LRs

Commit message generated by GPT-5
- Do not report eval metrics during sanity validation (global_step == 0).
- Read metrics from trainer.logged_metrics instead of logger_connector for compatibility
  with newer PyTorch lightning.
- Remove unnecessary deepcopy import and copying.
- Update unit tests to use .kwargs, rename test class.
- Add integration test verifying start, per-epoch metrics, and finish messages.
- Document why on_exception is disabled (event loop closes on shutdown).

Commite message partly generated by GPT-5
- Replace asyncio.run(...) with event_loop.run_until_complete(...)
  for sending Telegram messages.
- Re-enable on_exception to notify on training errors.

Commit message generated by GPT-5y
@ceshine ceshine self-assigned this Sep 5, 2025
and improve docs/warnings

- Rename internal metric state from `target` to `targets` for clarity
and consistency - Add typing casts around `self.preds`/`self.targets` to
satisfy static type checkers - Standardize `process_group` type hints to
`Any` with `None` default - AUC: - Add initialization warning clarifying
specialized behavior and directing to `BinaryAUROC` - Refactor compute
to use local lists/tensors and clearer variable names -
SpearmanCorrelation: - Refactor compute with explicit casts and local
vars - Add pyright ignore where necessary for Spearman’s correlation
access - FBeta: - Expand docstring with binary and multiclass behavior
caveats - Add warning clarifying specialized behavior and recommending
torchmetrics’ FBeta for general cases - Refactor compute with explicit
casts and clearer naming

These changes improve clarity, type safety, and user guidance while
preserving runtime behavior.

Commit message generated by GPT-5
@ceshine ceshine marked this pull request as ready for review October 8, 2025 06:25
@ceshine ceshine merged commit 380b33d into veritable-tech:main Oct 8, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant