Skip to content

feat(http/retry): add a unit test suite to PeekTrailersBody<B> #3556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jan 22, 2025
Merged

feat(http/retry): add a unit test suite to PeekTrailersBody<B> #3556

merged 3 commits into from
Jan 22, 2025

Conversation

cratelyn
Copy link
Member

@cratelyn cratelyn commented Jan 22, 2025 •
edited
Loading

linkerd/linkerd2#8733 tracks the upgrade to hyper 1.0.

the PeekTrailersBody<B> body middleware, provided by the linkerd-http-retry crate, is uniquely positioned to be adversely affected by the changes to the Body trait. this type, in order to attempt to peek an inner B-typed body's HTTP/2 trailers, has some subtle and nuanced edge cases.

the intent of this type is to only yield the asynchronous task responsible for reading the body once. depending on what the inner body yields, and when, this can result in combinations of: no data frames and no trailers, no data frames with trailers, one data frame and no trailers, one data frame with trailers, or two data frames. moreover, depending on which of these are yielded, the body will call .await some scenarios, and only poll functions once in others.

migrating this to the Frame<T> and poll_frame() style of the 1.0 Body interface, away from the 0.4 interface that provides distinct poll_data() and poll_trailers() methods, is fundamentally tricky.

this branch is most importantly focused on introducing a unit test suite to PeekTrailersBody<B>, to help provide assurance that our body middleware behaves as expected across this migration boundary.

along the way, some documentation is added, and some minor changes are made to the control flow of the polling logic to carve a path for later changes related to polling.

@cratelyn cratelyn mentioned this pull request Jan 22, 2025
Closed
@cratelyn cratelyn marked this pull request as ready for review January 22, 2025 15:58
@cratelyn cratelyn requested a review from a team as a code owner January 22, 2025 15:58
@cratelyn
Copy link
Member Author

a note on process: pending approval, my intent is to land this PR as three commits: one containing the docs additions, one containing the refactor tweaks, and one including the test suite. this will avoid conflating all of this in a commit called "add a unit test suite".

@cratelyn cratelyn force-pushed the kate/http-retry.add-peek-trailers-test-suite branch from c160f63 to 1e1b62e Compare January 22, 2025 17:33
@cratelyn
Copy link
Member Author

cratelyn commented Jan 22, 2025 •
edited
Loading

a snapshot of this branch as originally submitted for review can be found at https://github.com/linkerd/linkerd2-proxy/tree/kate/http-retry.add-peek-trailers-test-suite.archive-pre-squash.

compared to main: main...kate/http-retry.add-peek-trailers-test-suite.archive-pre-squash

@cratelyn
docs(http/retry): document PeekTrailersBody<B> interfaces
8169b3e 
this is a squashed commit containing the following:

---

docs(http/retry): document `PeekTrailersBody::inner`

Signed-off-by: katelyn martin <[email protected]>

docs(http/retry): document `PeekTrailersBody::peek_trailers()`

Signed-off-by: katelyn martin <[email protected]>

docs(http/retry): document `PeekTrailersBody::no_trailers()`

Signed-off-by: katelyn martin <[email protected]>
@cratelyn
refactor(http/retry): refactor PeekTrailersBody<B> logic
89497b4 
this is a squashed commit containing the following:

---

refactor(http/retry): decompose `WithPeekTrailersBody<B>` type alias

this commit breaks this large type out into two halves.

this is a purely cosmetic change.

Signed-off-by: katelyn martin <[email protected]>

refactor(http/retry): `PeekTrailersBody` is pin projected

we must pass our `Pin<T>`'edness down to the inner `B`-typed body for
`PeekTrailersBody` to itself implement `http_body::Body`.

this commit tweaks the existing code to rely on the `pin-project`
library. this generates a `project()` method to pin inner fields whose
`poll_data()` and `poll_trailers()` functions we delegate to.

this is a noöp change.

Signed-off-by: katelyn martin <[email protected]>

refactor(http/retry): defer construction of `PeekTrailersBody<B>`

this commit refactors the polling logic in
`PeekTrailersBody<B>::read_response`.

this commit makes some subtle changes with the migration to hyper 1.0 in
mind, to make this function more easily portable to the new signature of
`http_body::Body`.

see linkerd/linkerd2#8733 for more
information.

this commit defers the `Self` construction of the `PeekTrailersBody`
body. this means that the control flow does not need to reach through to
e.g. `body.inner` to poll the inner body being peeked. additionally, it
provides us with `let` bindings for the first data frame yielded, and
the trailers frame yielded thereafter.

this is largely cosmetic, but will make it easier to deal with the
additional matching we'll need to do when there is a single polling
function that yields us `Frame<D>` objects.

Signed-off-by: katelyn martin <[email protected]>

refactor(http/retry): `PeekTrailersBody` transforms `B`

this is a small structural change to the
`PeekTrailersBody::read_response()` function to facilitate writing some
unit tests.

rather than transforming a `Response<B>` into a
`Response<PeekTrailersBody<B>>`, we hoist the `Response::into_parts()`
and `Response::from_parts()` calls up. `read_response()` is renamed to
`read_body()`.

Signed-off-by: katelyn martin <[email protected]>
@cratelyn
feat(http/retry): unit test suite for PeekTrailersBody<B>
9be9042 
`PeekTrailersBody<B>` contains some subtle edge cases related to the
number of DATA frames yielded by the inner body, and how persistent it
will be about polling for TRAILERS frames.

for example, if it yields a DATA frame, it will not await trailers being
available, but it *will* do so if the inner body does not yield a DATA
frame. if a DATA frame is yielded, it will check for a TRAILERS frame,
but it must be immmediately available.

this is all subtle, and particularly subject to change with the upgrade
to http-body 1.0's frame-oriented `Body` interface.

so, this commit introduces a test suite for `PeekTrailersBody<B>`. it
includes assertions to confirm when the peek middleware can and cannot
observe the trailers.

some `TODO(kate)` comments are left where issues exist.

Signed-off-by: katelyn martin <[email protected]>
@cratelyn cratelyn force-pushed the kate/http-retry.add-peek-trailers-test-suite branch from 1e1b62e to 9be9042 Compare January 22, 2025 17:43
@cratelyn cratelyn enabled auto-merge (rebase) January 22, 2025 17:44
@cratelyn cratelyn merged commit 399ab1d into main Jan 22, 2025
15 checks passed
@cratelyn cratelyn deleted the kate/http-retry.add-peek-trailers-test-suite branch January 22, 2025 17:56
hawkw
hawkw reviewed Jan 22, 2025
View reviewed changes
@cratelyn cratelyn mentioned this pull request Jan 22, 2025
Merged
cratelyn added a commit that referenced this pull request Jan 23, 2025
@cratelyn
refactor(http/retry): PeekTrailersBody<B> uses BodyExt::frame()
a29df3f 
this commit reworks `PeekTrailersBody<B>`.

the most important goal here is replacing the control flow of
`read_body()`, which polls a body using `BodyExt` future combinators
`data()` and `frame()` for up to two frames, with varying levels of
persistence depending on outcomes.

to quote #3556:

> the intent of this type is to only yield the asynchronous task
> responsible for reading the body once. depending on what the inner
> body yields, and when, this can result in combinations of: no data
> frames and no trailers, no data frames with trailers, one data frame
> and no trailers, one data frame with trailers, or two data frames.
> moreover, depending on which of these are yielded, the body will call
> .await some scenarios, and only poll functions once in others.
>
> migrating this to the Frame<T> and poll_frame() style of the 1.0 Body
> interface, away from the 0.4 interface that provides distinct
> poll_data() and poll_trailers() methods, is fundamentally tricky.

this means that `PeekTrailersBody<B>` is notably difficult to port
across the http-body 0.4/1.0 upgrade boundary.

this body middleware must navigate a number of edge conditions, and once
it _has_ obtained a `Frame<T>`, make use of conversion methods to
ascertain whether it is a data or trailers frame, due to the fact that
its internal enum representation is not exposed publicly. one it has
done all of that, it must do the same thing once more to examine the
second frame.

this commit uses the compatibility facilities and backported `Frame<T>`
introduced in the previous commit, and rewrites this control flow using
a form of the `BodyExt::frame()` combinator.

this means that this middleware is forward-compatible with http-body
1.0, which will dramatically simplify the remaining migration work to be
done in #3504.

see linkerd/linkerd2#8733 for more information
and other links related to this ongoing migration work.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Jan 24, 2025
@cratelyn
feat(http/retry): model PeekTrailersBody<B> with Frame<T> (#3559)
dd4fbcd 
this branch contains a sequence of commits that refactor `PeekTrailersBody<B>`.

this branch is specifically focused on making this body middleware
forward-compatible with the 1.0 interface(s) of `http_body::Body` and
`http_body_util::BodyExt`.

it does this in two main steps: (1) temporarily vendoring `http_body::Frame<T>`
and providing a compatibility shim that provides a `frame()` method for a body,
and (2) modeling `PeekTrailersBody<B>` and its peeking logic in terms of this
`Frame<'a, T>` future.

---

* feat(http/retry): add `Frame<T>` compatibility facilities

this commit introduces a `compat` submodule to `linkerd-http-retry`.

this helps us frontrun the task of replacing all of the finicky control
flow in `PeekTrailersBody<B>` using the antiquated `data()` and
`trailers()` future combinators. instead, we can perform our peeking
in terms of an approximation of `http_body_util::BodyExt::frame()`.

to accomplish this, this commit vendors a copy of the `Frame<T>` type.
we can use this to preemptively model our peek body in terms of this
type, and move to the "real" version of it when we're upgrading in
pr #3504.

additionally, this commit includes a type called
`ForwardCompatibleBody<B>`, and a variant of the `Frame<'a, T>`
combinator. these are a bit boilerplate-y, admittedly, but the pleasant
part of this is that we have, in effect, migrated the trickiest body
middleware in advance of #3504. once we upgrade to http-body 1.0, all of
these types can be removed.

https://docs.rs/http-body-util/latest/http_body_util/trait.BodyExt.html#method.frame
https://docs.rs/http-body-util/0.1.2/src/http_body_util/combinators/frame.rs.html#10

Signed-off-by: katelyn martin <[email protected]>

* refactor(http/retry): `PeekTrailersBody<B>` uses `BodyExt::frame()`

this commit reworks `PeekTrailersBody<B>`.

the most important goal here is replacing the control flow of
`read_body()`, which polls a body using `BodyExt` future combinators
`data()` and `frame()` for up to two frames, with varying levels of
persistence depending on outcomes.

to quote #3556:

> the intent of this type is to only yield the asynchronous task
> responsible for reading the body once. depending on what the inner
> body yields, and when, this can result in combinations of: no data
> frames and no trailers, no data frames with trailers, one data frame
> and no trailers, one data frame with trailers, or two data frames.
> moreover, depending on which of these are yielded, the body will call
> .await some scenarios, and only poll functions once in others.
>
> migrating this to the Frame<T> and poll_frame() style of the 1.0 Body
> interface, away from the 0.4 interface that provides distinct
> poll_data() and poll_trailers() methods, is fundamentally tricky.

this means that `PeekTrailersBody<B>` is notably difficult to port
across the http-body 0.4/1.0 upgrade boundary.

this body middleware must navigate a number of edge conditions, and once
it _has_ obtained a `Frame<T>`, make use of conversion methods to
ascertain whether it is a data or trailers frame, due to the fact that
its internal enum representation is not exposed publicly. one it has
done all of that, it must do the same thing once more to examine the
second frame.

this commit uses the compatibility facilities and backported `Frame<T>`
introduced in the previous commit, and rewrites this control flow using
a form of the `BodyExt::frame()` combinator.

this means that this middleware is forward-compatible with http-body
1.0, which will dramatically simplify the remaining migration work to be
done in #3504.

see linkerd/linkerd2#8733 for more information
and other links related to this ongoing migration work.

Signed-off-by: katelyn martin <[email protected]>

* refactor(http/retry): mock body enforces `poll_trailers()` contract

this commit addresses a `TODO` note, and tightens the enforcement of a
rule defined by the v0.4 signature of the `Body` trait.

this commit changes the mock body type, used in tests, so that it will
panic if the caller improperly polls for a trailers frame before the
final data frame has been yielded.

previously, a comment indicated that we were "fairly sure" this was
okay. while that may have been acceptable in practice, the changes in
the previous commit mean that we now properly respect these rules.

thus, a panic can be placed here, to enforce that "[is] only be called
once `poll_data()` returns `None`", per the documentation.

Signed-off-by: katelyn martin <[email protected]>

* refactor(http/retry): rename `PeekTrailersBody::Buffered`

<#3559 (comment)>

this is a nicer name than `Unknown` for this case. not to mention, we'll
want that name shortly to address the possibility of unknown frame
variants.

Signed-off-by: katelyn martin <[email protected]>

* refactor(http/retry): encapsulate `Inner<B>` enum variants

this commit makes the inner enum variants private.

#3559 (comment)

Signed-off-by: katelyn martin <[email protected]>

* refactor(http/retry): gracefully ignore unknown frames

#3559 (comment)

Signed-off-by: katelyn martin <[email protected]>

---------

Signed-off-by: katelyn martin <[email protected]>
@cratelyn cratelyn mentioned this pull request Jan 24, 2025
Merged
cratelyn added a commit that referenced this pull request Mar 10, 2025
@cratelyn
chore(deps): upgrade to hyper 1.x
0a8c4a5 
note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Mar 11, 2025
@cratelyn
chore(deps): upgrade to hyper 1.x
b6af8db 
note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Mar 12, 2025
@cratelyn
chore(deps): upgrade to hyper 1.x
4b1d279 
note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Mar 12, 2025
@cratelyn
chore(deps): upgrade to hyper 1.x
1e641bf 
note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Mar 14, 2025
@cratelyn
chore(deps): upgrade to hyper 1.x
ea77f0e 
note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Mar 18, 2025
@cratelyn
chore(deps): upgrade to hyper 1.x
abfd48b 
note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Mar 18, 2025
@cratelyn
chore(deps): upgrade to hyper 1.x
3992d74 
note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Mar 21, 2025
@cratelyn
chore(deps)!: upgrade to hyper 1.x (#3504)
76d4181 
## ✨ chore(deps): upgrade to hyperium 1.x crates

this branch performs an exciting upgrade for our proxy.

this branch upgrades a number of our dependencies so that we use the 1.0
release family of the `hyper` http framework, and its ecosystem. see the
[v1.0 announcement][hyper-v1] for more information.

this branch upgrades the following dependencies:

* `h2`: 0.3 -> 0.4
* `http`: 0.2 -> 1
* `http-body`: 0.4 -> 1
* `hyper`: 0.14.32 -> 1
* `prost`: 0.12 -> 0.13
* `prost-build`: 0.12 -> 0.13
* `prost-types`: 0.12 -> 0.13
* `tonic`: 0.10 -> 0.12
* `tonic-build`: 0.10 -> 0.12

a `hyper-util` dependency is added, which provides among other things,
legacy-compatible interfaces such as `hyper_util::client::legacy::Client`, or
glue to use `hyper` with the tokio runtime.

see <https://docs.rs/hyper-util/latest/hyper_util/> for more information.

a `http-body-util` dependency is added, which provides a `BodyExt` trait and a
channel-backed body for use in unit tests. the `deprecated` feature flag that
was active on our `0.14` hyper dependency has been removed, along with the
`stream` and `runtime` feature flags.

the `linkerd2-proxy-api` dependency is updated. see:
<linkerd/linkerd2-proxy-api#421>

### 📝 notes for review

bear particular attention to changes involving `http_body::Body` middleware.

the change from two separate `poll_data()` and `poll_trailers()` functions,
to a single `poll_frame()` method, induces some subtle changes to various
pieces of middleware.

also bear in mind that failing to set a timer, in our case
`hyper_util::rt::TokioTimer`, can cause http/2 clients, or http/1 and http/2
servers, to panic. make sure that any uses of
`hyper::server::conn::http1::Builder`, `hyper::client::conn::http1::Builder`,
or `hyper::client::conn::http2::Builder` install a timer.

### ❗ breaking change: `l5d-proxy-error` values

the `l5d-proxy-error` header can be examined to observe the cause of proxy
errors encountered when sending meshed traffic. by virtue of this using a newer
`hyper` client in the proxy, some error messages may in turn look different.
for example, an error like `"connect timed out after 1s"` may now appear as
`"client error (Connect)"`.

### 📚 other notes

this work, by virtue of touching so many parts of the system, is carried out
in distinct commits. an initial commit upgrades the dependencies at th
workspace level. subsequent commits will not compile if the `--workspace` flag
is provided, but the intent of this branch is to update each crate
individually.

use commands like, e.g. `cargo check --tests -p linkerd-proxy-http` to build
particular crates at intermediate commits within this branch.

this commit is also only the final leaf in an _extended_ line of work. this
has been done to mitigate the effort of reviewing this change, and the risk of
churn in the event of any unanticipated errors. see the top-level comment in
linkerd/linkerd2#8733 for an overview of all of the
work that brought us to this juncture.

[hyper-v1]: https://seanmonstar.com/blog/hyper-v1/

---

* chore(deps): upgrade to hyper 1.x

note: this commit will not compile, code changes are intentionally
elided from this commit.

this commit upgrades hyper, http, tonic, prost, related dependencies,
and their assorted cargo features.

see <linkerd/linkerd2#8733>.

see also:

* #3379
* #3380
* #3382
* #3405
* hyperium/hyper#3796
* #3411
* #3421
* #3427
* #3428
* #3432
* #3433
* #3444
* #3445
* #3454
* #3455
* #3456
* #3457
* #3461
* #3459
* #3465
* #3466
* #3467
* #3468
* linkerd/linkerd2-proxy-api#421
* linkerd/linkerd2#13492
* linkerd/linkerd2#13493
* hyperium/hyper#3816
* #3472
* #3473
* #3479
* tokio-rs/tokio#7059
* #3509
* hyperium/http-body#140
* #3515
* hyperium/http-body#141
* #3530
* #3531
* #3540
* #3556
* #3558
* #3559
* #3564
* #3567
* #3573
* #3583
* hyperium/http-body#144
* #3585
* #3586
* #3597
* #3598
* #3611
* #3614
* #3615
* #3616
* #3647
* #3651
* #3653
* #3654
* #3655
* #3656
* #3657
* #3660
* #3671
* #3672
* #3673
* #3676
* hyperium/http-body#147
* #3692
* #3699
* #3700
* #3701
* #3708
* linkerd/drain-rs#36
* #3715
* #3717
* eminence/procfs#340

---

squash:

chore(deps): add hyper-util workspace dependency

chore(deps): add http-body-util workspace dependency

chore(deps): upgrade linkerd2-proxy-api

this commit represents main as of linkerd/linkerd2-proxy-api#421.

Signed-off-by: katelyn martin <[email protected]>

* chore(http/box): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(hyper-balance): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/retain): ugrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/stream-timeouts): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/classify): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/upgrade): upgrade to hyper 1.x

NOTE: there is a comment noting that the upgrade middleware does not
expect to be cloneable. it is unfortunately, however, at odds with the
new bounds expected of extensions.

so, `Http11Upgrade` is now Clone'able, but a comment is left in place
noting this weakened invariant.

it's worth investigating how upgrades have changed since, in more
detail, but for the current moment we are interested in being
especially conservative about changing behavior, and focusing on api
changes like `Body::poll_frame(..)`.

Signed-off-by: katelyn martin <[email protected]>

* chore(metrics): upgrade to hyper 1.x

a brief note; this commit happened to tickle an unfortunate sharp edge
in `BoxBody` and `Full`'s respective constructors. type inference could
not figure out how to construct the body, so we refrain from boxing the
response body now.

Signed-off-by: katelyn martin <[email protected]>

* chore(http/metrics): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/prom): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/insert): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/retry): deprecate linkerd-http-body-compat

Signed-off-by: katelyn martin <[email protected]>

* chore(mock/http-body): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(http/retry): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(proxy/tap): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(proxy/http): update to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(app/core): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(app/test): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(app/admin): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(app/outbound): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(app/inbound): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(app/integration): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(app): upgrade to hyper 1.x

Signed-off-by: katelyn martin <[email protected]>

* chore(transport-header): update generated code

Signed-off-by: katelyn martin <[email protected]>

* chore(spiffe-proto): update generated code

Signed-off-by: katelyn martin <[email protected]>

* chore(opencensus-proto): update generated code

Signed-off-by: katelyn martin <[email protected]>

* chore(opentelemetry-proto): update generated code

Signed-off-by: katelyn martin <[email protected]>

* chore(deny.toml): update cargo-deny directives

this commit updates the contents of `deny.toml`.

Signed-off-by: katelyn martin <[email protected]>

* chore: `compile` has been renamed to `compile_protos`

this addresses deprecation warnings, updating calls to a function that
has since been renamed.

Signed-off-by: katelyn martin <[email protected]>

* chore(deps): remove `linkerd-http-body-compat` dependencies

this commit removes this crate, which we added to future proof code for
this upgrade, from its dependents.

Signed-off-by: katelyn martin <[email protected]>

* chore(http/body-compat): remove `linkerd-http-body-compat` crate

Signed-off-by: katelyn martin <[email protected]>

* chore(deps): update to drain 0.2.1

see linkerd/drain-rs#41.

Signed-off-by: katelyn martin <[email protected]>

---------

Signed-off-by: katelyn martin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@olix0r olix0r olix0r approved these changes

+1 more reviewer

@hawkw hawkw hawkw left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@olix0r olix0r

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@cratelyn @olix0r @hawkw