chore: add _prepare_and_schedule_batch function in PyExecutor #6365
Conversation
Signed-off-by: junq <[email protected]>
📝 WalkthroughWalkthroughA new private method Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes 📜 Recent review detailsConfiguration used: .coderabbit.yaml 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
✨ Finishing Touches
🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
/bot run |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
tensorrt_llm/_torch/pyexecutor/py_executor.py (1)
803-845: Good consolidation of batch preparation logic with room for improvementThe new method effectively consolidates duplicate logic from the executor loops, reducing code duplication and improving maintainability. The early return pattern and error handling are well implemented.
Consider adding a docstring to document this critical method:
+ def _prepare_and_schedule_batch(self): + """ + Prepares and schedules a batch of requests for processing. + + Returns: + tuple: (scheduled_batch, iter_stats) if processing should continue, + (None, None) if processing should stop. + """ - def _prepare_and_schedule_batch(self):Also consider making the warning message more descriptive:
- logger.warning( - "num_fitting_reqs=0 and fitting_disagg_gen_init_requests is empty, may not have enough kvCache" - ) + logger.warning( + "No fitting requests and no disaggregated generation init requests available. " + "This may indicate insufficient KV cache resources or all requests are waiting for resources." + )
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tensorrt_llm/_torch/pyexecutor/py_executor.py(3 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py
📄 CodeRabbit Inference Engine (CODING_GUIDELINES.md)
**/*.py: Python code should conform to Python 3.8+.
Indent Python code with 4 spaces. Do not use tabs.
Always maintain the namespace when importing in Python, even if only one class or function from a module is used.
Python filenames should use snake_case (e.g., some_file.py).
Python classes should use PascalCase (e.g., class SomeClass).
Python functions and methods should use snake_case (e.g., def my_awesome_function():).
Python local variables should use snake_case. Prefix k for variable names that start with a number (e.g., k_99th_percentile).
Python global variables should use upper snake_case and prefix G (e.g., G_MY_GLOBAL).
Python constants should use upper snake_case (e.g., MY_CONSTANT).
Avoid shadowing variables declared in an outer scope in Python.
Initialize all externally visible members of a Python class in the constructor.
For interfaces that may be used outside a Python file, prefer docstrings over comments.
Comments in Python should be reserved for code within a function, or interfaces that are local to a file.
Use Google style docstrings for Python classes and functions, which can be parsed by Sphinx.
Attributes and variables in Python can be documented inline; attribute docstrings will be rendered under the docstring for the class.
Avoid using reflection in Python when functionality can be easily achieved without it.
When using try-except blocks in Python, limit the except to the smallest set of errors possible.
When using try-except blocks to handle multiple possible variable types in Python, keep the body of the try as small as possible, using the else block to implement the logic.
Files:
tensorrt_llm/_torch/pyexecutor/py_executor.py
**/*.{cpp,h,hpp,cc,cxx,cu,py}
📄 CodeRabbit Inference Engine (CODING_GUIDELINES.md)
All TensorRT-LLM Open Source Software code should contain an NVIDIA copyright header that includes the current year. This includes .cpp, .h, .cu, .py, and any other source files which are compiled or interpreted.
Files:
tensorrt_llm/_torch/pyexecutor/py_executor.py
🔇 Additional comments (2)
tensorrt_llm/_torch/pyexecutor/py_executor.py (2)
858-860: Clean integration of the consolidated methodThe replacement of complex inline logic with the new
_prepare_and_schedule_batchmethod call is well executed. The proper handling of return values and early exit condition maintains the existing behavior while improving code readability.
964-966: Consistent integration pattern maintainedThe integration follows the same clean pattern as in
_executor_loop, ensuring consistency across both executor implementations. The refactoring successfully eliminates code duplication while preserving existing functionality.
|
PR_Github #13001 [ run ] triggered by Bot |
|
PR_Github #13001 [ run ] completed with state |
Signed-off-by: junq <[email protected]>
|
/bot run |
|
PR_Github #13121 [ run ] triggered by Bot |
|
PR_Github #13121 [ run ] completed with state |
|
/bot run |
|
PR_Github #13137 [ run ] triggered by Bot |
|
PR_Github #13137 [ run ] completed with state |
…#6365) Signed-off-by: junq <[email protected]> Signed-off-by: Shreyas Misra <[email protected]>
…#6365) Signed-off-by: junq <[email protected]> Signed-off-by: Ransiki Zhang <[email protected]>
…#6365) Signed-off-by: junq <[email protected]> Signed-off-by: Lanyu Liao <[email protected]>
Summary by CodeRabbit