Skip to content

feat: experiments logging improvements #2049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Feb 8, 2025
Merged

Conversation

pd93
Copy link
Member

@pd93 pd93 commented Feb 8, 2025

Closes #1979

Refactors the experiments package slightly so that experiments only need to be specified in one place now. Also moved the print functions to the logger to avoid importing anything in the experiments package. This stops circular imports.

This way of doing things allows us to easily iterate over all experiments which means I was able to easily add a function that validates all experiments. If an experiment is inactive (has no enabled values) and a user tries to specify a value, a warning will now be output.

Finally, I've consolidated how we read any Task-namespaced environment variables into the new env.GetTaskVar function that will always prefix TASK_ to any variables read. The logger now only reads these variables once instead of every time a call to print is made.

@pd93 pd93 changed the title experiments logging improvements feat: experiments logging improvements Feb 8, 2025
Copy link
Member

@andreynering andreynering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@pd93
Copy link
Member Author

pd93 commented Feb 8, 2025

@andreynering Have added your suggestion and also added some tests for the experiments and did a little bit more refactoring. We'll now get warnings in two cases:

  1. A user attempted to enable an experiment with no valid values

    ❯ TASK_X_ANY_VARIABLES=1 taskd
    task: Experiment "ANY_VARIABLES" is inactive and cannot be enabled
    
  2. A user attempted to enable an experiment with a value that is not in the allowed values

    ❯ TASK_X_MAP_VARIABLES=99 taskd
    task: Experiment "MAP_VARIABLES" has an invalid value "99" (allowed values: 1, 2)
    

@pd93 pd93 merged commit 6ce798e into main Feb 8, 2025
14 checks passed
@pd93 pd93 deleted the experiments-logging-improvements branch February 8, 2025 23:02
pd93 added a commit that referenced this pull request Feb 8, 2025
vmaerten pushed a commit that referenced this pull request Feb 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exit with error when an experiment is specified
2 participants