File tree Expand file tree Collapse file tree 2 files changed +3
-2
lines changed Expand file tree Collapse file tree 2 files changed +3
-2
lines changed Original file line number Diff line number Diff line change @@ -13,6 +13,7 @@ Kernels for LLM sampling.
13
13
sampling_from_probs
14
14
top_p_sampling_from_probs
15
15
top_k_sampling_from_probs
16
+ min_p_sampling_from_probs
16
17
top_k_top_p_sampling_from_probs
17
18
top_p_renorm_prob
18
19
top_k_renorm_prob
Original file line number Diff line number Diff line change @@ -219,7 +219,7 @@ def min_p_sampling_from_probs(
219
219
min_p : torch .Tensor ,
220
220
deterministic : bool = True ,
221
221
) -> Tuple [torch .Tensor , torch .Tensor ]:
222
- r"""Fused GPU kernel for min_p sampling from probabilities,
222
+ r"""Fused GPU kernel for ` min_p sampling <https://arxiv.org/abs/2407.01082>`_ from probabilities,
223
223
224
224
this operator implements GPU-based rejection sampling without explicit sorting.
225
225
@@ -235,7 +235,7 @@ def min_p_sampling_from_probs(
235
235
where the first dimension is the maximum number of rounds for rejection sampling.
236
236
Expected to be uniformly distributed in ``[0, 1)``.
237
237
min_p: torch.Tensor
238
- The threshold for min_p sampling for each request, shape ``(batch_size,)``.
238
+ The :math:`p_{\text{base}}` in min_p sampling for each request, shape ``(batch_size,)``.
239
239
deterministic: bool
240
240
Whether to use deterministic kernel implementation, default is ``True``.
241
241
You can’t perform that action at this time.
0 commit comments