[Ready for Review] Track in-flight solving tx to delay HTLC failure update until enough confirmations #305

ariard · 2019-02-01T02:37:02Z

WIP, need to implement block_disconnected, fix tests (was expected they break) and surely prune duplicate stuff in is_resolving_htlc_output.

ariard · 2019-02-07T02:41:19Z

All cases should be covered, just need to complete tests:

Delayed-Failures-until-maturation-cases :

Revoked Remote HTLC: claim_htlc_outputs_shared_tx/claim_htlc_outpus_single_tx/test_simple_commitment_revoked_fail_backward/do_test_commitment_revoked_fail_backward_exhaustive
Remote Dust HTLC: test_fail_backwards_latest_remote_announce/test_fail_backwards_previous_remote_announce
Remote Non-Dust HTLC: test_fail_backwards_latest_remote_announce/test_fail_backwards_previous_remote_announce
Local Dust HTLC: test_failure_delay_dust_htlc_local_commitment
Local Non-Dust HTLC:
test_htlc_on_chain_timeout/test_duplicate_payment_hash_one_failure_one_success

Canceled-Failures :

Revoked Remote HTLC: test_sweep_outbound_htlc_failure_update
Remote Dust HTLC: test_sweep_outbound_htlc_failure_update
Remote Non-Dust HTLC: test_sweep_outbound_htlc_failure_update
Local Dust HTLC: test_sweep_outbound_htlc_failure_update
Local Non-Dust HTLC: test_sweep_outbound_htlc_failure_update

ariard · 2019-02-13T01:59:26Z

Heey, should be good now. Modify a lot of tests due to introduction of delay in both ChannelMonitor and ChannelManager. Hope I don't change semantic of them while fixing

ariard · 2019-02-28T02:54:13Z

Rebased, and fix full_stack_target break

TheBlueMatt

Thanks so much for tackling this! It generally looks pretty good, and I'm happy to take it as-is as an intermediate step, but I think it would be nice to go ahead and start by tracking per-tx-that-claims instead of just height. I'm pleasantly surprised by how little diff this required, or maybe you just did it really well :p.

TheBlueMatt · 2019-03-22T20:09:30Z

src/ln/channelmonitor.rs

@@ -172,10 +172,6 @@ impl<Key : Send + cmp::Eq + hash::Hash> ChainListener for SimpleManyChannelMonit
 						// In case of reorg we may have htlc outputs solved in a different way so
 						// we prefer to keep claims but don't store duplicate updates for a given
 						// (payment_hash, HTLCSource) pair.
-						// TODO: Note that we currently don't really use this as ChannelManager


Hmm, because the pending updates are stored in ChannelMonitor instead of ManyChannelMonitor (so that we can persist them), the de-duplicating code here should never trigger - it should be safe to always just push (if we want we can add debug_assert checks that its unique so that we'll catch it in fuzzing if its actually hit, but no reason to take the runtime performance penalty here).

Okay, move the de-duplicating code at ChannelMonitor, a little bit simpler (even if not most efficient way due to HahsMap<HTLCSource, update> being non-implementable due to Hash trait absent on SecretKey)

TheBlueMatt · 2019-03-22T20:59:40Z

src/ln/channelmonitor.rs

-									log_trace!(self, "Failing HTLC with payment_hash {} from {} remote commitment tx due to broadcast of revoked remote commitment transaction", log_bytes!(htlc.payment_hash.0), $commitment_tx);
-									htlc_updated.push(((**source).clone(), None, htlc.payment_hash.clone()));
+									log_trace!(self, "Failing HTLC with payment_hash {} from {} remote commitment tx due to broadcast of revoked remote commitment transaction, waiting confirmation until {} height", log_bytes!(htlc.payment_hash.0), $commitment_tx, height + HTLC_FAIL_ANTI_REORG_DELAY - 1);
+									match self.htlc_updated_waiting_threshold_conf.entry(height + HTLC_FAIL_ANTI_REORG_DELAY - 1) {


Hmm, we don't really want to just wait for some number of blocks, we really want to wait for min(htlc_expiry - some buffer(I presume HTLC_FAIL_TIMEOUT_BLOCKS), the txn in txn_to_broadcast are confirmed with at least HTLC_FAIL_ANTI_REORG_DELAY confirmations). This requires some more tracking, but I think this enables some really nice additional features - ideally we'd track the txn_to_broadcast here (or maybe the data that went into creating them) and be able to recreate them with a bumped fee as time passes.

Separating the tracking to per-HTLC-by-tx also allows us to fail backwards HTLCs that are near-expiry while holding on to others as we await confirmation (and separate out feerates for each class).

Well, there L1192, its revoked HTLC outputs which are concerned, so we really care for maturation of revoked commitment transaction, their htlc_expiry may already been expired ? And in other places it's from broadcasting of failure-trigger tx we start timer, so don't get you on min(htlc_expiry - some_buffer) ?

Timely failure of inbound HTLC may triggered by some per-tx-that-claims (like a HTLC-timeout) but IMO it's 2 different problems : waiting delay A for maturation of failure-trigger tx and waiting delay B before bumping timeout/claim tx. If so, shouldn't be hard to track txn_broadcast in check_spend_local_tx/check_spend_remote_tx, and at height timer expiration regenerate tx with higher fee, but seems to me it merits its own PR.

ariard · 2019-03-27T02:19:17Z

Rebased, bf7f26a differs slightly from its previous state due to the fact I forgot at first to prune channel closing in case of block disconnected

Broadcasting a commitment tx means that we have to fail inbound HTLC in backward channel. Doing it prematurely would put us at risk in case of reorg. So we delay passing failure update upstream until solving tx mature to HTLC_FAIL_ANTI_ REORG_DELAY. Requirements differ if HTLC is a revoked/non-revoked dust/ non-revoked non-dust one. Add connect_blocks in test_utils to fix broken tests due to anti-reorg delay enforcement Remove anti-duplicate htlc update stuff in ManySimpleChannelMonitor

Modify ChainListener API by adding height field to block_disconnect

Fix tests broken by introduced change

Add test_failure_delay_htlc_local_commitment Move some bits of check_spend_remote as we need to fail dust HTLCs which can be spread on both prev/lastest local commitment tx

Add pruning of waiting-conf channel closing at block_disconnect Fix tests broken by introduced change

ariard · 2019-07-19T22:05:46Z

Closing, #336 is doing the job

ariard force-pushed the 2018-12-track-in-flight-preconfirmed-tx branch 3 times, most recently from 9104d6f to 98fc5dc Compare February 5, 2019 01:46

ariard changed the title ~~[WIP] Track in-flight solving tx to delay HTLC failure update until safety confirmations~~ [WIP] Track in-flight solving tx to delay HTLC failure update until enough confirmations Feb 5, 2019

ariard force-pushed the 2018-12-track-in-flight-preconfirmed-tx branch 2 times, most recently from 9b536ec to 3cf25dd Compare February 7, 2019 02:37

ariard force-pushed the 2018-12-track-in-flight-preconfirmed-tx branch 5 times, most recently from b45a1b0 to 6da8ae6 Compare February 13, 2019 01:54

ariard changed the title ~~[WIP] Track in-flight solving tx to delay HTLC failure update until enough confirmations~~ [Ready for Review]] Track in-flight solving tx to delay HTLC failure update until enough confirmations Feb 13, 2019

ariard changed the title ~~[Ready for Review]] Track in-flight solving tx to delay HTLC failure update until enough confirmations~~ [Ready for Review] Track in-flight solving tx to delay HTLC failure update until enough confirmations Feb 13, 2019

ariard force-pushed the 2018-12-track-in-flight-preconfirmed-tx branch 2 times, most recently from a2fabca to c5588f7 Compare February 14, 2019 01:53

ariard force-pushed the 2018-12-track-in-flight-preconfirmed-tx branch from 01b9794 to 6f061d0 Compare February 28, 2019 02:02

TheBlueMatt reviewed Mar 22, 2019

View reviewed changes

TheBlueMatt mentioned this pull request Mar 25, 2019

BOLT5: fix claim backward revoked htlc success #322

Closed

ariard force-pushed the 2018-12-track-in-flight-preconfirmed-tx branch 2 times, most recently from 432d95e to 5598bcd Compare March 27, 2019 02:11

Antoine Riard added 6 commits March 28, 2019 20:49

Implement block_disconnect for pruning of waiting-conf HTLC updates

1ff9395

Modify ChainListener API by adding height field to block_disconnect

Delay failure of non-dust HTLC-outputs until solving timeout tx matures

44bbd4c

Fix tests broken by introduced change

Fail back dust HTLC of local commitment tx after enough confirmations

1f2c3f5

Add test_failure_delay_htlc_local_commitment Move some bits of check_spend_remote as we need to fail dust HTLCs which can be spread on both prev/lastest local commitment tx

Implement delay channel closing in ChannelManager until maturation

d4f5c59

Add pruning of waiting-conf channel closing at block_disconnect Fix tests broken by introduced change

Add block_disconnecting tests to cancel HTLC failure updates

669824f

ariard force-pushed the 2018-12-track-in-flight-preconfirmed-tx branch from 5598bcd to 669824f Compare March 29, 2019 00:55

ariard closed this Jul 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Ready for Review] Track in-flight solving tx to delay HTLC failure update until enough confirmations #305

[Ready for Review] Track in-flight solving tx to delay HTLC failure update until enough confirmations #305

Uh oh!

ariard commented Feb 1, 2019

Uh oh!

ariard commented Feb 7, 2019 •

edited

Loading

Uh oh!

ariard commented Feb 13, 2019

Uh oh!

ariard commented Feb 28, 2019

Uh oh!

TheBlueMatt left a comment

Uh oh!

TheBlueMatt Mar 22, 2019

Uh oh!

ariard Mar 27, 2019

Uh oh!

TheBlueMatt Mar 22, 2019

Uh oh!

ariard Mar 26, 2019

Uh oh!

ariard commented Mar 27, 2019 •

edited

Loading

Uh oh!

ariard commented Jul 19, 2019

Uh oh!

Uh oh!

[Ready for Review] Track in-flight solving tx to delay HTLC failure update until enough confirmations #305

[Ready for Review] Track in-flight solving tx to delay HTLC failure update until enough confirmations #305

Uh oh!

Conversation

ariard commented Feb 1, 2019

Uh oh!

ariard commented Feb 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ariard commented Feb 13, 2019

Uh oh!

ariard commented Feb 28, 2019

Uh oh!

TheBlueMatt left a comment

Choose a reason for hiding this comment

Uh oh!

TheBlueMatt Mar 22, 2019

Choose a reason for hiding this comment

Uh oh!

ariard Mar 27, 2019

Choose a reason for hiding this comment

Uh oh!

TheBlueMatt Mar 22, 2019

Choose a reason for hiding this comment

Uh oh!

ariard Mar 26, 2019

Choose a reason for hiding this comment

Uh oh!

ariard commented Mar 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ariard commented Jul 19, 2019

Uh oh!

Uh oh!

ariard commented Feb 7, 2019 •

edited

Loading

ariard commented Mar 27, 2019 •

edited

Loading