Skip to content

Update and Check loadListing for input dir provenance #1986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 144 commits into from
Mar 27, 2024
Merged

Update and Check loadListing for input dir provenance #1986

merged 144 commits into from
Mar 27, 2024

Conversation

ElderMedic
Copy link
Collaborator

merged latest main in for update
fixed loadListing for provenance files copying

mr-c and others added 30 commits June 5, 2023 19:50
@mr-c
pass through QEMU_LD_PREFIX by default (common-workflow-language#1849)
0f58a96 
For systems using qemu-user-static to run foreign binaries outside of
containers
@dependabot
Update flake8-bugbear requirement from <23.6 to <23.7 (common-workflo…
d7cde2e 
…w-language#1856)

Updates the requirements on [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear) to permit the latest version.
- [Release notes](https://github.com/PyCQA/flake8-bugbear/releases)
- [Commits](PyCQA/flake8-bugbear@16.4.1...23.6.5)

---
updated-dependencies:
- dependency-name: flake8-bugbear
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mr-c
gh-actions: keep running even if codecov upload fails (common-workflo…
d73e104 
…w-language#1857)
@dependabot
Bump sphinx-rtd-theme from 1.2.1 to 1.2.2 (common-workflow-language#1858
ac4cd0c 
)

Bumps [sphinx-rtd-theme](https://github.com/readthedocs/sphinx_rtd_theme) from 1.2.1 to 1.2.2.
- [Changelog](https://github.com/readthedocs/sphinx_rtd_theme/blob/master/docs/changelog.rst)
- [Commits](readthedocs/sphinx_rtd_theme@1.2.1...1.2.2)

---
updated-dependencies:
- dependency-name: sphinx-rtd-theme
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mr-c
dockerfile: pin graphviz to before 8.x
8aecdcc
@mr-c
pin to compatible galaxy-util
9e1eb1d
@dependabot @mr-c
Update pytest requirement from <7.4,>=6.2 to >=6.2,<7.5
15a4f64 
Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.0...7.4.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@mr-c
*requirements.txt; remove spaces to enable re-use in other contexts
f88d752 
Like `pip install $(grep -v mypy mypy-requirements.txt)`
@mr-c
confirm enhanced TypeDSL from Schema Salad v1.3 doesn't leak through
e4be144
@mr-c
tox: skip py3.6 for lint
263a554
@mr-c
required outputs with secondaryFiles fix for containers (common-workf…
0f7367b 
…low-language#1870)

* required outputs with secondaryFiles fix for containers

* improve error message

* leave no files behind
@dependabot
Update flake8-bugbear requirement from <23.7 to <23.8 (common-workflo…
48a75cc 
…w-language#1877)

Updates the requirements on [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear) to permit the latest version.
- [Release notes](https://github.com/PyCQA/flake8-bugbear/releases)
- [Commits](PyCQA/flake8-bugbear@16.4.1...23.7.10)

---
updated-dependencies:
- dependency-name: flake8-bugbear
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@mr-c
Update Python version support document; drop Python 3.7 (EOL) (common…
8fc75e1 
…-workflow-language#1872)

* refresh python version support

* CI: Drop Python 3.7 from testing
@dependabot @mr-c
Update black requirement from ~=23.3 to ~=23.7
dff81dc 
Updates the requirements on [black](https://github.com/psf/black) to permit the latest version.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.3.0...23.7.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@mr-c
drop py3.7 lint checks
f23d3e0
@mr-c
enable pickling of cwltool.workflow.Workflow (common-workflow-languag…
1573509 
…e#1878)
@mr-c
start testing with Python 3.12
a9cea86
@mr-c
formatting fix
2fedbd6
@mr-c
py version support policy: latest from Debian stable or Ubuntu LTS
8075ae4 
Allow for any supported version of Python, not just the default Python version

Stop tracking RHEL

This would mean that we can then drop Python 3.6 support
@mr-c
code: drop Python 3.6-3.7 support
9c768e9
@mr-c
drop msvcrt: we don't support MS Windows anyhow
73ddf02
@dependabot @mr-c
Update rdflib requirement from <6.4,>=4.2.2 to >=4.2.2,<7.1
5a645df 
Updates the requirements on [rdflib](https://github.com/RDFLib/rdflib) to permit the latest version.
- [Release notes](https://github.com/RDFLib/rdflib/releases)
- [Changelog](https://github.com/RDFLib/rdflib/blob/main/CHANGELOG.md)
- [Commits](RDFLib/rdflib@4.2.2...7.0.0)

---
updated-dependencies:
- dependency-name: rdflib
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@jmfernandez @mr-c
Fixed race condition which happens when a job runs "too fast",
0baf100 
When a podman process finishes even before reaching the monitoring
method, a deadlock happens, as no one is updating `process.returncode`
and spawned process is in zombie state (so, no signal is sent).

This fix adds a `process.poll()` call, so it gives the chance to fill in
`process.returncode`.
@dependabot @mr-c
Bump sphinx-rtd-theme from 1.2.2 to 1.3.0
05058d9 
Bumps [sphinx-rtd-theme](https://github.com/readthedocs/sphinx_rtd_theme) from 1.2.2 to 1.3.0.
- [Changelog](https://github.com/readthedocs/sphinx_rtd_theme/blob/master/docs/changelog.rst)
- [Commits](readthedocs/sphinx_rtd_theme@1.2.2...1.3.0)

---
updated-dependencies:
- dependency-name: sphinx-rtd-theme
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@mr-c
Cope with narrower terminal
8b90b6e
@mr-c
Use python3 in test CWL tool description
5301c1e
@mr-c
bump minimum required schema_salad version to reflect reality.
bd89c56
@jmfernandez @mr-c
Use docker/podman/udocker inspect instead of docker images.
385161d 
This approach avoids the issue about podman using "canonical"
image tags just using the "inspect" subcommand.
@jmfernandez @mr-c
Removed additional udocker/podman pull.
f919784 
This udocker/podman pull command was being done even when
`--disable-pull` parameter is passed to `cwltool`, and it
should not be needed, as all container image pulls should
have happened at this point.
@jmfernandez @mr-c
Narrowed the exceptions to be caught, as of @mr-c indication.
179855c 
Also, cosmetic changes to fix some flake8 complaints.
dependabot bot and others added 28 commits January 26, 2024 07:26
@dependabot
Update black requirement from ~=23.12 to ~=24.1
3732321 
Updates the requirements on [black](https://github.com/psf/black) to permit the latest version.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.12.0...24.1.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@mr-c
apply black 2024 formatting
80c9ec0
@mr-c
ignore black 2024 formatting commit
2f5f336
@mr-c
Merge pull request common-workflow-language#1970 from common-workflow…
d8f0eb2 
…-language/dependabot/pip/black-approx-eq-24.1

Update black requirement from ~=23.12 to ~=24.1
@dependabot @mr-c
Update pytest requirement from <7.5,>=6.2 to >=6.2,<8.1
61765e1 
Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.0...8.0.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot @mr-c
Update flake8-bugbear requirement from <23.13 to <24.2
3ac795c 
Updates the requirements on [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear) to permit the latest version.
- [Release notes](https://github.com/PyCQA/flake8-bugbear/releases)
- [Commits](PyCQA/flake8-bugbear@16.4.1...24.1.15)

---
updated-dependencies:
- dependency-name: flake8-bugbear
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@mr-c
address flake8-bugbear suggestions
35eeab4
@dependabot @mr-c
Bump codecov/codecov-action from 3 to 4
36e0da4 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3 to 4.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@v3...v4)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@mr-c
Fix codecov v4 configuration
fe1e8a9
@lihl2014 @mr-c
add the --shm-size parameter at runtime (common-workflow-language#1972)
7b8f174 
Co-authored-by: lihl <lihl@1123>
Co-authored-by: Michael R. Crusoe <[email protected]>
@dependabot @mr-c
Update galaxy-tool-util requirement
9875cf4 
Updates the requirements on [galaxy-tool-util](https://github.com/galaxyproject/galaxy) to permit the latest version.
- [Release notes](https://github.com/galaxyproject/galaxy/releases)
- [Commits](galaxyproject/galaxy@galaxy-tool-util-22.1.3...v23.2.1)

---
updated-dependencies:
- dependency-name: galaxy-tool-util
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot @mr-c
Update black requirement from ~=24.1 to ~=24.2
814d6d1 
Updates the requirements on [black](https://github.com/psf/black) to permit the latest version.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@24.1.0...24.2.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@svonworl @mr-c
Append "cwltool" to HTTP User-Agent string (common-workflow-language#…
ca16314 
…1977)

Also appends the program name, if not "cwltool"

Should work for calrissian, reana-cwl-runner, and arvados-cwl-runner, but not
toil-cwl-runner which doesn't call the cwltool.main.main() function.

Probably works for streamflow, but I haven't tested that.

---------

Co-authored-by: Michael R. Crusoe <[email protected]>
@dependabot @mr-c
Update pytest requirement from <8.1,>=6.2 to >=6.2,<8.2
60ddadb 
Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@6.2.0...8.1.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@mr-c
conformance testing: set cwltest 2.5 as minimum version for Pytest 8.…
70ab9a5 
…1.x compat.
@dependabot @mr-c
Update flake8-bugbear requirement from <24.2 to <24.3
442ac81 
Updates the requirements on [flake8-bugbear](https://github.com/PyCQA/flake8-bugbear) to permit the latest version.
- [Release notes](https://github.com/PyCQA/flake8-bugbear/releases)
- [Commits](PyCQA/flake8-bugbear@16.4.1...24.2.6)

---
updated-dependencies:
- dependency-name: flake8-bugbear
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot @mr-c
Bump mypy from 1.8.0 to 1.9.0 (common-workflow-language#1982)
6c9e6fb 
* Bump mypy from 1.8.0 to 1.9.0

Bumps [mypy](https://github.com/python/mypy) from 1.8.0 to 1.9.0.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](python/mypy@v1.8.0...1.9.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update pyproject.toml

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Michael R. Crusoe <[email protected]>
@mr-c
biocontainers test is flaky, automatically retry it.
3b39796
@mr-c
setuptools: silence package discovery warnings
c972913
@mr-c
Makefile: check-python3 is unneeded, call python3 explictly.
d7a49fd 
the "dist", "clean", and "release-test" targets don't need a virtualenv
so as long as there is a python3 on the PATH they will work
@mr-c
biocontainer tests: use version corresponding to v2 Docker Image Format
c34c7a1
@mr-c
pytest: use the worksteal scheduler and run on all logical cores, not…
98c0bca 
… just physical.
@ElderMedic
Merge branch 'main' into prov_data_input_output for updates
29eee5f
@ElderMedic
fix merge err
5fc090b
@ElderMedic
fix loadListing of dir for prov
18a9bd6
@ElderMedic
misc change of comments
fc0e331
@ElderMedic
fix text, remove vscode space files
627026c
@ElderMedic
remove redundant texts
96d1e20
@mr-c
Copy link
Member

mr-c commented Mar 27, 2024

Thank you @ElderMedic ; I invited you to join this repo so you can directly work on the prov_data_input_output branch

@ElderMedic ElderMedic merged commit c7963b3 into common-workflow-language:prov_data_input_output Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.