Skip to content

bpo-29603 New Queue.unfinished() method #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

bpo-29603 New Queue.unfinished() method #192

wants to merge 3 commits into from

Conversation

slytomcat
Copy link

Class queue.Queue control the number of unfinished tasks via method task_done(). But it is only possible to get the information about all task done (via join() method).
I'm sure that exposing the number of unfinished tasks (unfinished_tasks class variable) can be very useful in many situations when you need more control over the process.

But it is not good idea to provide write access to this internal variable (as it controls internal queue class status). Better way - provide RO access via class method like qsize or empty. It can be look like this:

    def unfinished(self):
        return self.unfinished_tasks

One example of this method usage: there is not optimal function _adjust_thread_count in concurrent.futures.ThreadPoolExecutor with following comment:

    # TODO(bquinlan): Should avoid creating new threads if there are more
    # idle threads than items in the work queue.

It can be easily done with following condition:

    if self._work_queue.unfinished() <= len(self._threads):
        return

http://bugs.python.org/issue29603

@slytomcat slytomcat changed the title 29603 New Queue.unfinished() method bpo-29603 New Queue.unfinished() method Feb 20, 2017
@slytomcat
Copy link
Author

Example of usage this new method can be seen here: https://github.com/slytomcat/cpython/commit/ea313986d86c083e81624fbc8a7ab6a75784e15d

@slytomcat
Copy link
Author

I'm closing it after discussion in http://bugs.python.org/issue29603

@slytomcat slytomcat closed this Feb 22, 2017
akruis pushed a commit to akruis/cpython that referenced this pull request Dec 29, 2018
…code

Refactor the finalisation code of stackless.tasklet and
stackless.channel.
- For tasklets, use a PEP 442 finalizer;
- for channels a simple tp_clear function is sufficient.

Like every PEP 442 other finalizer, the new finalizer runs only once for
each tasklet.
akruis pushed a commit to akruis/cpython that referenced this pull request Dec 29, 2018
In order to prevent an endless loop in the tasklet-finalizer, try at
most 10 times to kill a tasklet.
akruis pushed a commit to akruis/cpython that referenced this pull request Dec 29, 2018
Move the handling of tasklets with non-trivial C-state from
tasklet_traverse to tasklet_finalize. If a tasklet still has a
non-trivial C-state after killing the tasklet, the finalizer appends the
tasklet to gc.garbage. Now de-allocation caused by ref-counting behaves
identically to de-allocation caused by GC.
The gc-patch introduced by
stackless-dev/stackless_historic@35ba34f18 is
now obsolete.
akruis pushed a commit to akruis/cpython that referenced this pull request Dec 29, 2018
Revert the patch, that added the function PyObject_GC_Collectable(). It
was introduced by 
stackless-dev/stackless_historic@35ba34f18.
See bpo-9141 for a discussion of this interesting patch.
akruis pushed a commit to akruis/cpython that referenced this pull request Jan 12, 2019
…code

Refactor the finalisation code of stackless.tasklet and
stackless.channel.
- For tasklets, use a PEP 442 finalizer;
- for channels a simple tp_clear function is sufficient.

Like every PEP 442 other finalizer, the new finalizer runs only once for
each tasklet.
akruis pushed a commit to akruis/cpython that referenced this pull request Jan 12, 2019
In order to prevent an endless loop in the tasklet-finalizer, try at
most 10 times to kill a tasklet.
akruis pushed a commit to akruis/cpython that referenced this pull request Jan 12, 2019
Move the handling of tasklets with non-trivial C-state from
tasklet_traverse to tasklet_finalize. If a tasklet still has a
non-trivial C-state after killing the tasklet, the finalizer appends the
tasklet to gc.garbage. Now de-allocation caused by ref-counting behaves
identically to de-allocation caused by GC.
The gc-patch introduced by
stackless-dev/stackless_historic@35ba34f18 is
now obsolete.
akruis pushed a commit to akruis/cpython that referenced this pull request Jan 12, 2019
Revert the patch, that added the function PyObject_GC_Collectable(). It
was introduced by 
stackless-dev/stackless_historic@35ba34f18.
See bpo-9141 for a discussion of this interesting patch.
jaraco pushed a commit that referenced this pull request Dec 2, 2022
Ensure we're sending in a string instead of a list.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants