Skip to content

some general fixes/additions for next release #3026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ compute resources to the global community, alleviating the technical burdens,
such as familiarity with the command line or access to compute power, that are
typically limiting for researchers studying microbial ecology.

Qiita is currently in alpha status. We are very open to community
Qiita is currently in beta status. We are very open to community
contributions and feedback. If you're interested in contributing to Qiita,
see `CONTRIBUTING.md <https://github.com/biocore/qiita/blob/master/CONTRIBUTING.md>`__.
If you'd like to report bugs or request features, you can do that in the
Expand All @@ -43,9 +43,7 @@ Current features

* Target gene data: we support deblur against GreenGenes (13_8) and close
reference picking against GreenGenes (13_8) and Silva.
* Metagenoic/Shotgun data: we support Shogun processing. Note that this data
is suitable for download and further down stream analyses but we don't recommend
meta-analysis within Qiita (only single study).
* Metagenomic and Metatranscriptomic data: we support Shogun processing.
* biom files can be added as new preparation templates for downstream
analyses; however, this cannot be made public.

Expand Down
25 changes: 25 additions & 0 deletions qiita_db/processing_job.py
Original file line number Diff line number Diff line change
Expand Up @@ -821,6 +821,31 @@ def external_id(self, value):
qdb.sql_connection.TRN.add(sql, [value, self.id])
qdb.sql_connection.TRN.execute()

@property
def release_validator_job(self):
"""Retrieves the release validator job

Returns
-------
qiita_db.processing_job.ProcessingJob or None
The release validator job of this job
"""
rvalidator = None
with qdb.sql_connection.TRN:
sql = """SELECT processing_job_id
FROM qiita.processing_job
WHERE command_id in (
SELECT command_id
FROM qiita.software_command
WHERE name = 'release_validators')
AND command_parameters->>'job' = %s"""
qdb.sql_connection.TRN.add(sql, [self.id])
results = qdb.sql_connection.TRN.execute_fetchflatten()
if results:
rvalidator = ProcessingJob(results[0])

return rvalidator

def submit(self, parent_job_id=None, dependent_jobs_list=None):
"""Submits the job to execution
This method has the ability to submit itself, as well as a list of
Expand Down
17 changes: 15 additions & 2 deletions qiita_db/test/test_processing_job.py
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,10 @@ def test_complete_type(self):
qdb.artifact.Artifact(exp_artifact_count).filepaths])

def test_complete_success(self):
# Note that here we are submitting and creating other multiple jobs;
# thus here is the best place to test any intermediary steps/functions
# of the job creation, submission, exectution, and completion.
#
# This first part of the test is just to test that by default the
# naming of the output artifact will be the name of the output
fd, fp = mkstemp(suffix='_table.biom')
Expand All @@ -457,8 +461,16 @@ def test_complete_success(self):
'artifact_type': 'BIOM'}}
job = _create_job()
job._set_status('running')

# here we can test that job.release_validator_job hasn't been created
# yet so it has to be None
self.assertIsNone(job.release_validator_job)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add another test for the case where the return value is not None?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha! I knew I was forgetting something ... thank you!

job.complete(True, artifacts_data=artifacts_data)
self._wait_for_job(job)
# let's check for the job that released the validators
self.assertIsNotNone(job.release_validator_job)
self.assertEqual(job.release_validator_job.parameters.values['job'],
job.id)
# Retrieve the job that is performing the validation:
validators = list(job.validator_jobs)
self.assertEqual(len(validators), 1)
Expand Down Expand Up @@ -858,8 +870,9 @@ def test_raise_if_not_in_construction_error(self):
tester._raise_if_not_in_construction()

def test_submit(self):
# In order to test a success, we need to actually run the jobs, which
# will mean to run split libraries, for example.
# The submit method is being tested in test_complete_success via
# a job, its release validators and validators submissions.
# Leaving this note here in case it's helpful for future development
pass

def test_from_default_workflow(self):
Expand Down