Skip to content

change repo urls of some of our existing libraries #4512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

change repo urls of some of our existing libraries #4512

wants to merge 4 commits into from

Conversation

einspunktnull
Copy link
Contributor

No description provided.

@github-actions GitHub Actions
Copy link
Contributor

Hi @einspunktnull.
Your pull request has been detected as something other than a Library Manager submission.
A maintainer will need to review it before it can be merged.

If you intended to submit a library, please check the instructions and update your pull request if necessary:
https://github.com/arduino/library-registry/blob/main/README.md#instructions

@github-actions github-actions bot added the topic: modification Change existing list entry label May 11, 2024
@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/voltmeter:

Linting library in voltmeter
WARNING: No license file found.                                                                                                                 
         See:                                                                                                                                   
         https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository
         (Rule LD002)                                                                                                                           

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in voltmeter/examples/Voltmeter25V

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/timeout-callback:

Linting library in timeout-callback
WARNING: A commonly misspelled word was found in the library.properties paragraph field. Suggested correction: Timeout  
         is a minimalistic library to call a callback function after a user-specific time in milliseconds. Timeout is   
         non-blocking and designed to use minimal memory requirements.                                                  
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format         
         (Rule LP035)                                                                                                   

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in timeout-callback/examples/OneSecond

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/simple-serial-protocol-arduino:

Linting library in simple-serial-protocol-arduino
WARNING: library.properties name value SimpleSerialProtocol is longer than the recommended length of 16 characters.
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format    
         (Rule LP010)                                                                                              

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in simple-serial-protocol-arduino/examples/echo_example

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/average-value:

Linting library in average-value
WARNING: No license file found.                                                                                                                 
         See:                                                                                                                                   
         https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository
         (Rule LD002)                                                                                                                           

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in average-value/examples/AverageValue

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/interval-callback:

Linting library in interval-callback
WARNING: A commonly misspelled word was found in the library.properties paragraph field. Suggested correction:          
         IntervalCallback is a minimalistic library to call a callback function repetively after a user-specific time in
         milliseconds. Interval is non-blocking and designed to use minimal memory requirements.                        
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format         
         (Rule LP035)                                                                                                   

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in interval-callback/examples/EverySeconds

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions github-actions bot requested a review from per1234 May 11, 2024 10:11
This was referenced May 18, 2024
Open
Open
Open
Open
Open
per1234
per1234 reviewed May 18, 2024
View reviewed changes
Copy link
Contributor

@per1234 per1234 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @einspunktnull. I am confirming your request.

Unlike submissions, this type of request is still handled manually. The Library Manager index maintainer occasionally comes through to clear all the outstanding requests. They will merge this pull request when it is done. You will get a notification from GitHub at that time.

After that, you can monitor the indexing of the libraries via their dedicated logs web page.
Information about that here:
https://github.com/arduino/library-registry/blob/main/FAQ.md#can-i-check-on-library-releases-being-added-to-library-manager

@per1234 per1234 added topic: URL change Change library repository URL status: pending backend Depends on backend maintenance operations and removed topic: modification Change existing list entry labels May 18, 2024
@github-actions github-actions bot added topic: modification Change existing list entry and removed topic: URL change Change library repository URL status: pending backend Depends on backend maintenance operations labels May 19, 2024
@github-actions GitHub Actions
Copy link
Contributor

Hi @einspunktnull.
Your pull request has been detected as something other than a Library Manager submission.
A maintainer will need to review it before it can be merged.

If you intended to submit a library, please check the instructions and update your pull request if necessary:
https://github.com/arduino/library-registry/blob/main/README.md#instructions

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/timeout-callback:

Linting library in timeout-callback
WARNING: A commonly misspelled word was found in the library.properties paragraph field. Suggested correction: Timeout  
         is a minimalistic library to call a callback function after a user-specific time in milliseconds. Timeout is   
         non-blocking and designed to use minimal memory requirements.                                                  
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format         
         (Rule LP035)                                                                                                   

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in timeout-callback/examples/OneSecond

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/simple-serial-protocol-arduino:

Linting library in simple-serial-protocol-arduino
WARNING: library.properties name value SimpleSerialProtocol is longer than the recommended length of 16 characters.
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format    
         (Rule LP010)                                                                                              

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in simple-serial-protocol-arduino/examples/echo_example

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/average-value:

Linting library in average-value
WARNING: No license file found.                                                                                                                 
         See:                                                                                                                                   
         https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository
         (Rule LD002)                                                                                                                           

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in average-value/examples/AverageValue

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/interval-callback:

Linting library in interval-callback
WARNING: A commonly misspelled word was found in the library.properties paragraph field. Suggested correction:          
         IntervalCallback is a minimalistic library to call a callback function repetively after a user-specific time in
         milliseconds. Interval is non-blocking and designed to use minimal memory requirements.                        
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format         
         (Rule LP035)                                                                                                   

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in interval-callback/examples/EverySeconds

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/voltmeter:

Linting library in voltmeter
WARNING: No license file found.                                                                                                                 
         See:                                                                                                                                   
         https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository
         (Rule LD002)                                                                                                                           

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in voltmeter/examples/Voltmeter25V

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions github-actions bot requested a review from per1234 May 19, 2024 06:51
@per1234 per1234 removed their request for review May 19, 2024 06:52
@per1234 per1234 added topic: URL change Change library repository URL status: pending backend Depends on backend maintenance operations and removed topic: modification Change existing list entry labels May 19, 2024
@github-actions GitHub Actions
Copy link
Contributor

Hi @einspunktnull.
Your pull request has been detected as something other than a Library Manager submission.
A maintainer will need to review it before it can be merged.

If you intended to submit a library, please check the instructions and update your pull request if necessary:
https://github.com/arduino/library-registry/blob/main/README.md#instructions

@github-actions github-actions bot added topic: modification Change existing list entry and removed topic: URL change Change library repository URL status: pending backend Depends on backend maintenance operations labels May 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/average-value:

Linting library in average-value
WARNING: No license file found.                                                                                                                 
         See:                                                                                                                                   
         https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository
         (Rule LD002)                                                                                                                           

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in average-value/examples/AverageValue

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/interval-callback:

Linting library in interval-callback
WARNING: A commonly misspelled word was found in the library.properties paragraph field. Suggested correction:          
         IntervalCallback is a minimalistic library to call a callback function repetively after a user-specific time in
         milliseconds. Interval is non-blocking and designed to use minimal memory requirements.                        
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format         
         (Rule LP035)                                                                                                   

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in interval-callback/examples/EverySeconds

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/simple-serial-protocol-arduino:

Linting library in simple-serial-protocol-arduino
WARNING: library.properties name value SimpleSerialProtocol is longer than the recommended length of 16 characters.
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format    
         (Rule LP010)                                                                                              

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in simple-serial-protocol-arduino/examples/echo_example

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/timeout-callback:

Linting library in timeout-callback
WARNING: A commonly misspelled word was found in the library.properties paragraph field. Suggested correction: Timeout  
         is a minimalistic library to call a callback function after a user-specific time in milliseconds. Timeout is   
         non-blocking and designed to use minimal memory requirements.                                                  
         See: https://arduino.github.io/arduino-cli/latest/library-specification/#libraryproperties-file-format         
         (Rule LP035)                                                                                                   

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in timeout-callback/examples/OneSecond

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions GitHub Actions
Copy link
Contributor

Arduino Lint has suggestions for possible improvements to https://github.com/yesbotics/voltmeter:

Linting library in voltmeter
WARNING: No license file found.                                                                                                                 
         See:                                                                                                                                   
         https://docs.github.com/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository
         (Rule LD002)                                                                                                                           

Linter results for project: 0 ERRORS, 1 WARNINGS

-------------------

Linting sketch in voltmeter/examples/Voltmeter25V

Linter results for project: no errors or warnings

-------------------

Linter results for projects: 0 ERRORS, 1 WARNINGS

@github-actions github-actions bot requested a review from per1234 May 22, 2024 10:40
@per1234 per1234 removed their request for review May 22, 2024 13:58
@per1234 per1234 added topic: URL change Change library repository URL status: pending backend Depends on backend maintenance operations and removed topic: modification Change existing list entry labels May 22, 2024
@MatteoPologruto MatteoPologruto self-assigned this May 23, 2024
@MatteoPologruto
Copy link
Contributor

Hello @einspunktnull @per1234, I'm reporting the following errors:

2024/05/23 14:54:12 Loaded 6939 libraries from DB
Changing URL of library AverageValue from https://gitlab.com/yesbotics/libs/arduino/average-value.git to https://github.com/yesbotics/average-value.git
error: While backing up library release archive: stat /home/ubuntu/workspace/libraries/gitlab.com/arduino/AverageValue-1.0.0.zip: no such file or directory
Original files were restored.

2024/05/23 14:55:49 Loaded 6939 libraries from DB
Changing URL of library IntervalCallback from https://gitlab.com/yesbotics/libs/arduino/interval-callback.git to https://github.com/yesbotics/interval-callback.git
error: While backing up library release archive: stat /home/ubuntu/workspace/libraries/gitlab.com/arduino/IntervalCallback-1.0.0.zip: no such file or directory
Original files were restored.

2024/05/23 14:57:40 Loaded 6939 libraries from DB
Changing URL of library TimeoutCallback from https://gitlab.com/yesbotics/libs/arduino/timeout-callback.git to https://github.com/yesbotics/timeout-callback.git
error: While backing up library release archive: stat /home/ubuntu/workspace/libraries/gitlab.com/arduino/TimeoutCallback-1.0.0.zip: no such file or directory
Original files were restored.

2024/05/23 14:58:34 Loaded 6939 libraries from DB
Changing URL of library Voltmeter from https://gitlab.com/yesbotics/libs/arduino/voltmeter.git to https://github.com/yesbotics/voltmeter.git
error: While backing up library release archive: stat /home/ubuntu/workspace/libraries/gitlab.com/arduino/Voltmeter-1.0.0.zip: no such file or directory
Original files were restored.

2024/05/23 14:59:33 Loaded 6939 libraries from DB
Changing URL of library SimpleSerialProtocol from https://gitlab.com/yesbotics/simple-serial-protocol/simple-serial-protocol-arduino.git to https://github.com/yesbotics/simple-serial-protocol-arduino.git
error: While backing up library release archive: stat /home/ubuntu/workspace/libraries/gitlab.com/simple-serial-protocol/SimpleSerialProtocol-2.1.0.zip: no such file or directory
Original files were restored.

@umbynos umbynos added the status: waiting for information More information must be provided before work can proceed label Jul 29, 2024
@per1234
Copy link
Contributor

per1234 commented Jul 8, 2025

@MatteoPologruto I performed an investigation into the unexpected failure of the libraries-repository-engine modify --repo-url command for the requested URL change requests and found the cause and the solution:

Cause

The cause of the problem is a change in the behavior of libraries-repository-engine that I inadvertently introduced in arduino/libraries-repository-engine@d0f3044.

Prior to arduino/libraries-repository-engine@d0f3044, archives were stored under a folder structure named according to the final two components of the parent of the repository URL. After the change, the name format was changed to <host>/<final parent>.

For a GitHub repository, or a GitLab repository under a personal account or in the root of a group, the two different approaches to determining the archive folder structure are equivalent. For example, https://github.com/arduino-libraries/Servo.git release archives are stored under <LibrariesFolder>/github.com/arduino-libraries either way. However, they are not for repositories under a GitLab subgroup.

For example, prior to arduino/libraries-repository-engine@d0f3044 release archives for https://gitlab.com/yesbotics/libs/arduino/average-value.git were stored under <LibrariesFolder>/libs/arduino. But when we run a modify --repo-url command using a version of libraries-repository-engine from after arduino/libraries-repository-engine@d0f3044, it expects to find them under <LibrariesFolder>/gitlab.com/arduino.

This only affected maintenance operations because the "DB" is stored persistently, so the generated libraries index had the correct libraries[*].url value despite the change in the libraries-repository-engine behavior.

So the fault you encountered only occurs when a modify --repo-url or remove command is performed under the following conditions:

I did a survey to determine the full scope of the impact of the regression and found that it happens the libraries that are the subject of this request were the only ones affected by the regression.

Even though the change in behavior is a regression for this specific scenario, it was actually an improvement going forward because an archive path like <LibrariesFolder>/gitlab.com/arduino is more correct than the former <LibrariesFolder>/libs/arduino. So it doesn't make sense to change the behavior of libraries-repository-engine back to how it was prior to arduino/libraries-repository-engine@d0f3044. In addition, a significant number of archives are now stored under paths specific to the new behavior, so changing it back would cause as much trouble as leaving it.

You might have noticed that there is still a bug in how the archive paths are determined for GitLab subgroup repos, since the correct archive folder structure should have a form like <LibrariesFolder>/gitlab.com/yesbotics rather than the current <LibrariesFolder>/gitlab.com/arduino. I submitted a report about that at arduino/libraries-repository-engine#334. However, that is not relevant to this task because that bug was present in libraries-repository-engine from the start, so there that component of the archive path hasn't changed over time.

Solution

Here is an overview of the sequence of operations that will resolve this must perform to :

  1. Move archive files to the modern path.
  2. Remove the libraries.
  3. Merge URL change PRs
  4. Sync

1. Move Archive Files

This operation is performed solely in order to allow the next step to succeed (remove commands also fail if the release archive files are not found at the expected path).

Please run the following commands on the "backend" machine:

mv /home/ubuntu/workspace/libraries/libs/arduino /home/ubuntu/workspace/libraries/gitlab.com

mv /home/ubuntu/workspace/libraries/yesbotics/simple-serial-protocol /home/ubuntu/workspace/libraries/gitlab.com

2. Remove Libraries

Please perform a library removal operation on the backend machine.

The reason for doing this is that, even though the previous operation moved the library release archive files to the modern paths, the "DB" entries for those library releases still have Releases[*].URL values reflecting the previous path, which will result in the generated libraries index having incorrect libraries[*].url values for those releases.

Although this could be resolved by manually updating the data in the "DB", I think it is easier to just remove the libraries, which will result in the correct entries being automatically generated on the next run of the sync job. I verified that this will not result in the loss of any releases (the releases that are present in the current "DB" are all present in the library repositories on GitHub).

So please perform the standard library removal procedure (i.e., libraries-repository-engine remove) on the backend machine for the following libraries:

  • AverageValue
  • IntervalCallback
  • SimpleSerialProtocol
  • TimeoutCallback
  • Voltmeter

3. Merge PRs

Next, please merge the URL change PRs here in the arduino/library-registry repo:

This will result in the next sync job indexing the libraries using the registry data with the new URLs.

4. Sync

This is the standard libraries-repository-engine sync command.

This will index the libraries once again, generating the DB entries with the correct Releases[*].URL values.

No manual action is required for this operation as it will happen as a matter of course via the scheduled job.

@per1234 per1234 removed the status: waiting for information More information must be provided before work can proceed label Jul 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@per1234 per1234 per1234 left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@MatteoPologruto MatteoPologruto

Labels
status: pending backend Depends on backend maintenance operations topic: URL change Change library repository URL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@einspunktnull @MatteoPologruto @per1234 @umbynos