Hold times for successful payments #3801

joostjager · 2025-05-26T11:55:36Z

Apply the hold time reporting mechanism as implemented in #2256 to the success case. Spec in lightning/bolts#1044

ldk-reviews-bot · 2025-05-26T11:55:39Z

👋 Thanks for assigning @valentinewallace as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

codecov · 2025-05-27T12:02:51Z

Codecov Report

Attention: Patch coverage is 85.88710% with 210 lines in your changes missing coverage. Please review.

Project coverage is 88.82%. Comparing base (257ebad) to head (5519b4c).

Files with missing lines	Patch %	Lines
lightning/src/ln/channelmanager.rs	84.72%	116 Missing and 7 partials ⚠️
lightning/src/ln/channel.rs	84.44%	75 Missing and 11 partials ⚠️
lightning/src/ln/payment_tests.rs	88.88%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3801      +/-   ##
==========================================
- Coverage   88.82%   88.82%   -0.01%     
==========================================
  Files         165      165              
  Lines      119075   119955     +880     
  Branches   119075   119955     +880     
==========================================
+ Hits       105769   106550     +781     
- Misses      10986    11101     +115     
+ Partials     2320     2304      -16

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

valentinewallace

Still going through a first pass, made it to fcc1a8c

lightning/src/ln/channel.rs

lightning/src/ln/channelmanager.rs

lightning/src/ln/channel.rs

lightning/src/ln/msgs.rs

lightning/src/ln/channel.rs

ldk-reviews-bot · 2025-07-01T21:42:05Z

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

joostjager · 2025-07-02T11:04:07Z

First batch of comments addressed diff. Push below is just a rebase.

valentinewallace

Finished a pass, will take another look tomorrow! Lot to take in here having not reviewed the previous PRs

lightning/src/ln/channel.rs

lightning/src/ln/channelmanager.rs

valentinewallace · 2025-07-02T20:12:42Z

lightning/src/ln/channelmanager.rs

+
+				// Create new attribution data as the final hop. Always report a zero hold time, because reporting a
+				// non-zero value will not make a difference in the penalty that may be applied by the sender.
+				let attribution_data = process_fulfill_attribution_data(


nit: to me a clearer name would be add_hold_time_from_fulfill?

I tried to keep the name somewhat in line with process_failure_packet. In your suggestion, what does "from fulfill" mean exactly?

Ah okay. Currently sounds to me like something the recipient would call when they receive the final update_fulfill_htlc to extract the hold times. It's a pretty similar name to process_onion_success atm.

what does "from fulfill" mean exactly?

Basically meant from_update_fulfill_htlc. ISTM we're not really processing the attribution data, more like adding to it? I get wanting to be consistent with the other pre-existing method though.

Of course "process" is the vaguest possible word. Now renamed process_onion_success to decode_fulfill_attribution_data. Not consistent with the existing failure case naming, but I think more descriptive within the scope of this PR.

lightning/src/ln/channel.rs

valentinewallace · 2025-07-02T20:34:27Z

lightning/src/ln/channel.rs

 			let mut holding_cell_failures =
-				holding_cell_htlc_updates.iter_mut().filter_map(|upd| {
-					if let HTLCUpdateAwaitingACK::FailHTLC {
+				holding_cell_htlc_updates.iter_mut().filter_map(|upd| match upd {


Honestly this ser code makes me nervous. Is it possible to test a round-trip ser with some failures and some claims, or extend an existing test around holding cell htlcs?

The test do_channel_holding_cell_serialize is hitting these two match arms. I don't see a simple round-trip test for FundedChannel. Looked into it, but that would require a lot of instantiation of all the fields inside FundedChannel. Let's discuss what the best move is here.

Extended blinding_point_skimmed_fee_malformed_ser with None and Some attribution_data.

lightning/src/ln/channel.rs

fuzz/src/process_onion_failure.rs

lightning/src/ln/onion_utils.rs

lightning/src/ln/channelmanager.rs

valentinewallace · 2025-07-03T21:05:54Z

lightning/src/ln/onion_utils.rs

+			// Shift attribution data to prepare for processing the next hop.
+			attribution_data.shift_left();
+		} else {
+			log_debug!(


We hit this in a bunch of places in tests, should that be the case / why is that?

I've looked at the tests that hit this and it looks to be a mix of onchain resolution (where there is no attribution data), and restart / data loss (where we also may not recover attribution data because it is not critical).

lightning/src/ln/onion_utils.rs

valentinewallace · 2025-07-03T21:16:21Z

lightning/src/ln/onion_utils.rs

+				route_hop_idx
+			);
+
+			break;


It seems like we'll return a subset of the hold times in this case? Would it be better to continue and get the hold times that we can? I'm wondering if one node on the path can cause us to miss parsing some hold times here.

It would only work in the specific case where an upstream node correctly shifted and encrypted and also didn't corrupt any of the received data, but added invalid hmacs. I think that would need to be a software bug? I thought it is not worth doing Option just for that. But open to different views.

valentinewallace · 2025-07-03T21:17:11Z

lightning/src/ln/payment_tests.rs

+	let (_, path_events) =
+		claim_payment_along_route(ClaimAlongRouteArgs::new(&nodes[0], route, preimage.unwrap()));
+
+	// If we've delayed, check that this is reflected in the hold times.


Can we also check that the hold times are decreasing along the path?

lightning/src/ln/channel.rs

lightning/src/ln/onion_utils.rs

valentinewallace · 2025-07-03T21:43:23Z

lightning/src/ln/channelmanager.rs

+
+				// Create new attribution data as the final hop. Always report a zero hold time, because reporting a
+				// non-zero value will not make a difference in the penalty that may be applied by the sender.
+				let attribution_data = process_fulfill_attribution_data(


Ah okay. Currently sounds to me like something the recipient would call when they receive the final update_fulfill_htlc to extract the hold times. It's a pretty similar name to process_onion_success atm.

what does "from fulfill" mean exactly?

Basically meant from_update_fulfill_htlc. ISTM we're not really processing the attribution data, more like adding to it? I get wanting to be consistent with the other pre-existing method though.

Preparation for reuse of the logic.

Need to store AttributionData as part of the inbound HTLC removal reason so that it can be used in the upstream UpdateFulfillHTLC message.

Necessary to preserve attribution data when the HTLC is in the holding cell.

Adds hold time reporting for the final and intermediate nodes.

AttributionData is needed as part of the outbound HTLC outcome when revoke_and_ack has happened and the AttributionData is decoded to get the hold times for inclusion in the PaymentPathSuccessful event.

Prepare for inspecting hold times in PaymentPathSuccessful.

joostjager · 2025-07-04T08:08:41Z

Comments addressed diff

Hold times are surfaced via the PaymentPathSuccessful event.

ldk-reviews-bot · 2025-07-07T00:00:40Z

🔔 1st Reminder

Hey @valentinewallace @carlaKC! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-07-07T00:00:43Z

🔔 1st Reminder

Hey @valentinewallace @carlaKC! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

joostjager changed the title ~~Successful payment hold times~~ Hold times for successful payments May 26, 2025

joostjager force-pushed the fulfill-hold-times branch 7 times, most recently from badc803 to e913312 Compare May 27, 2025 09:52

joostjager mentioned this pull request May 27, 2025

Simplify UpdateFulfillFetch::NewClaim #3803

Merged

joostjager force-pushed the fulfill-hold-times branch from e913312 to 69a99b4 Compare May 27, 2025 11:53

joostjager force-pushed the fulfill-hold-times branch from 69a99b4 to 1f56e04 Compare May 27, 2025 12:24

joostjager mentioned this pull request May 27, 2025

Success hold times lightning/bolts#1261

Closed

joostjager mentioned this pull request Jun 5, 2025

Attribution data (feature 36/37) lightning/bolts#1044

Open

joostjager force-pushed the fulfill-hold-times branch 8 times, most recently from bace463 to 1c64a50 Compare June 26, 2025 13:52

joostjager marked this pull request as ready for review June 26, 2025 14:16

joostjager marked this pull request as draft June 26, 2025 14:17

joostjager added this to Weekly Goals Jun 26, 2025

joostjager self-assigned this Jun 26, 2025

joostjager force-pushed the fulfill-hold-times branch 2 times, most recently from 233cbfc to f5668f5 Compare June 30, 2025 09:36

valentinewallace reviewed Jul 1, 2025

View reviewed changes

joostjager force-pushed the fulfill-hold-times branch from df8ed65 to 5b219eb Compare July 2, 2025 10:57

joostjager requested a review from valentinewallace July 2, 2025 10:58

Rustfmt parts of channel.rs and channelmanager.rs

506b14e

joostjager force-pushed the fulfill-hold-times branch from 5b219eb to d6985fd Compare July 2, 2025 11:08

valentinewallace reviewed Jul 2, 2025

View reviewed changes

joostjager mentioned this pull request Jul 3, 2025

Attribution Data and HTLC Hold Times #3908

Open

8 tasks

joostjager force-pushed the fulfill-hold-times branch from d6985fd to 5519b4c Compare July 3, 2025 10:57

joostjager requested review from valentinewallace and carlaKC July 3, 2025 11:22

valentinewallace reviewed Jul 3, 2025

View reviewed changes

joostjager added 10 commits July 4, 2025 09:58

Rustfmt touch ups

3e78f49

Small refactors in onion_utils

09b8ee8

Extract hold time calculation

c98b105

Preparation for reuse of the logic.

Add AttributionData to UpdateFulfillHTLC message

d6ba09e

Add AttributionData to InboundHTLCRemovalReason::Fulfill

e786c73

Need to store AttributionData as part of the inbound HTLC removal reason so that it can be used in the upstream UpdateFulfillHTLC message.

Add AttributionData to HTLCUpdateAwaitingACK::ClaimHTLC

1710f91

Necessary to preserve attribution data when the HTLC is in the holding cell.

Hold time reporting

0224789

Adds hold time reporting for the final and intermediate nodes.

Remove code duplication in FundedChannel::read

7737b28

Add AttributionData to OutboundHTLCOutcome::Success

04e9e5e

AttributionData is needed as part of the outbound HTLC outcome when revoke_and_ack has happened and the AttributionData is decoded to get the hold times for inclusion in the PaymentPathSuccessful event.

Return path events from expect_payment_sent

e96fe1c

Prepare for inspecting hold times in PaymentPathSuccessful.

joostjager force-pushed the fulfill-hold-times branch from 5519b4c to 3aed96e Compare July 4, 2025 08:08

Decode hold times as a sender

b05cc65

Hold times are surfaced via the PaymentPathSuccessful event.

joostjager force-pushed the fulfill-hold-times branch from 3aed96e to b05cc65 Compare July 4, 2025 08:11

joostjager requested a review from valentinewallace July 4, 2025 08:12

Hold times for successful payments #3801

Are you sure you want to change the base?

Hold times for successful payments #3801

Uh oh!

Conversation

joostjager commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ldk-reviews-bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

valentinewallace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ldk-reviews-bot commented Jul 1, 2025

Uh oh!

joostjager commented Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

valentinewallace left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

joostjager commented May 26, 2025 •

edited

Loading

ldk-reviews-bot commented May 26, 2025 •

edited

Loading

codecov bot commented May 27, 2025 •

edited

Loading

joostjager commented Jul 2, 2025 •

edited

Loading

joostjager Jul 4, 2025 •

edited

Loading

joostjager commented Jul 4, 2025 •

edited

Loading