profile
viewpoint
Rusty Russell rustyrussell Blockstream http://rusty.ozlabs.org GPG: 15EE 8D6C AB0E 7F0C F999 BFCB D920 0E6C D1AD B8F1 Rusty Russell <rusty@rustcorp.com.au>

lightningnetwork/lightning-rfc 1167

Lightning Network Specifications

rustyrussell/ccan 798

The C Code Archive Network

lightningd/plugins 99

Community curated plugins for c-lightning

rustyrussell/bitcoin-iterate 95

Simple fast iterator to extract data from bitcoind's blockchain files.

rustyrussell/bitcoin-storage-guide 78

Rusty's Remarkable Unreliable Guide To Bitcoin Storage

cdecker/lightning-integration 67

Lightning Integration Testing Framework

rustyrussell/bitcoin-iblt 8

IBLT benchmarking for bitcoin

create barnchrustyrussell/lightning

branch : fix-bashism

created branch time in 3 hours

push eventElementsProject/lightning

niftynei

commit sha e66937e012d9d8f979a0a8e1d14a47bcce23788d

mfc: Add a `commitment_feerate` optional parameter to multifundchannel Technically there *are* two feerates that we need to know: - the feerate to use for the funding transaction, and - the feerate to tell our peer to use for our commitment txs/htlc txs As written, `multifundchannel` uses the same feerate for both. This optional parameter will allow us to differentiate between the two, which will be exceedingly handy for anchor output worlds. ;) FIXME: test this Changelog-Added: JSON API: `multifundchannel` has a new optional argument, 'commitment_feerate', which can be used to differentiate between the funding feerate and the channel's initial commitment feerate

view details

niftynei

commit sha f8c4cc73ae3d6cb7522bd5b5b65d2f842d9aa28c

mfc: use declared constants reduce, reuse, recycle for a greener world

view details

niftynei

commit sha 26bc4f5239b944edfd9cd54be5951cfa7d2d76f7

tx,bugfix: correct signature length estimate 71-bytes for a signature already includes the sighash byte. 2-bytes 30 44 (DER- prefix thing) 34-bytes 02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r) 34-bytes 02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s) 1-byte 01 (sighash) == 71-bytes

view details

niftynei

commit sha ede5f5be3cc6544bdef39db51b8a39f1821bfccc

mfc: blackbox test for commitment vs funding tx feerates Liquid is excluded because the mempool entry doesn't contain a 'weight' field.

view details

push time in 4 hours

PR merged ElementsProject/lightning

Reviewers
Add a `commitment feerate` parameter to multifundchannel

This lets a caller of multifundchannel specify a separate feerate for the channel's initial commitment transaction. This separates it from the feerate used for the funding transaction.

Writing the test for this uncovered an off by one in how we were calculating signature lengths.

Note that this conforms with the interface for openchannel_init, the new RPC that will initiate a v2 channel open (which contains two separate feerates, one for the funding tx and another for the commitment txs)

+121 -49

0 comment

8 changed files

niftynei

pr closed time in 4 hours

PullRequestReviewEvent

push eventElementsProject/lightning

niftynei

commit sha ae825fff26bc76d5273f47a797ac3f8eae10ab32

opening: use correct dust_limit for reserve floor Fixes #4140 Reported-By: @PsySc0rpi0n Changelog-Fixed: openingd now uses the correct dust limit for determining the allowable floor for a channel open (affects fundee only)

view details

push time in 4 hours

PR merged ElementsProject/lightning

opening: use correct dust_limit for reserve floor

Fixes #4140

Reported-By: @PsySc0rpi0n

+6 -6

1 comment

1 changed file

niftynei

pr closed time in 4 hours

issue closedElementsProject/lightning

Using wrong dust_limit as floor for channel_reserve

We should be using the opener's dust_limit as the channel_reserve floor, not the local dust_limit.

image

https://github.com/ElementsProject/lightning/blob/master/openingd/openingd.c#L211-L214

https://github.com/ElementsProject/lightning/blob/master/openingd/openingd.c#L896-L915

Encountered by @PsySc0rpi0n

closed time in 4 hours

niftynei

push eventElementsProject/lightning

niftynei

commit sha d1c7c7815968d6a70b785ac57c4172574d1e7fcf

channeld-df: actually check serial_id of input when setting sigs We're about to totally upset the order that sigs are set on our PSBTs for new channel opens, making it such that our peer's sigs may arrive before ours do. We can no longer rely on the 'set witness means this is our input' since there's no guarantee that our input sigs have been added yet, so we check the serial_id and only set the stack on their (odd) inputs.

view details

niftynei

commit sha bdf1cc2f93f848a30aa73d7e1e6958eb021d0f1c

channeld-df: only send our sigs if we've got them

view details

niftynei

commit sha c6ad4f9b2002655278d833ef2369085658305e9d

channel.psbt: make non-const We update it in the next patch, which technically breaks this contract. So we shouldn't have the contract of const on this in the first place then.

view details

niftynei

commit sha f9aab50ee88cf76d72eaea7124eeebb685d29b6c

dual-fund: rework where we send our tx-sigs message, allow peers in Prior to this patch update, we expected a client to call `openchannel_signed` before checking for peer's tx-sigs messages on the wire. When moving to a 'multifundchannel' approach, we'll need to be able to collect sigs from our peers before sending our tx_sigs message. There's no strict ordering on when tx-sigs messages are sent/received, so this is fine. To do this, we go ahead and start up channeld as soon as commitment_sigs are secured, so that we process incoming tx-sigs from our peers as soon as we get them.

view details

niftynei

commit sha daa55d1221e46b3e5b592b823fc24d8c80dae9af

df: add notification for receiving peer's funding tx sigs This will allow us to build complex, multi-peer transactions, with easeTM! Changelog-Added: EXPERIMENTAL, Plugins: `openchannel_peer_sigs` notification, which contains a peer's signatures for the funding transaction (`opt_dual_fund`)

view details

niftynei

commit sha 9d412718df6b40960cd1bd5dda44c9a5ac331f18

psbt: save the index of the change on the 'parent' Note that for removals, the index will be on the original; for additions, the index will be on the new. Yes this is implicit.

view details

niftynei

commit sha c6d4bd676fe5d5de4ee3b46eaddc40e53a95d029

dual-open,openchannel_update: include the index of the funding output This allows us to do correct reporting via multiopenchannel :)

view details

niftynei

commit sha d535a27104aae4a316fc013e9feb64beeeab5cd5

df, bugfix: wait til after we've saved the channel to do this this cleans up `cmd` and we're not done with it yet (we need it for saving the channel updates to the database)

view details

niftynei

commit sha 1b3a9be416cebda0b0b928451786be24f6d37c74

df, channeld: cleanup how psbt signalling works We used to send our tx_sigs before we got to channeld existing. We changed how this worked so that multifundchannel could live, but failed to clean up the logic of what "having a psbt around" means wrt channeld and messaging our peer. The general idea is that we want to send `tx_signatures` to our peer on reconnect until they've sent us `funding_locked`. Note that it's an error to - send `funding_locked` without having sent `tx_signatures` - send `tx_signatures` after sending `funding_locked` We use the 'finalized' state of the peer's inputs/outputs to help signal where we are in receiving their sigs -- but this doesn't work at all for opens where the peer doesn't contribute inputs at all. This isn't really a huge deal, but it does mean that if we receive a peer's `tx_sigs` more than once (will happen for a reconnect before `funding_locked`), then we'll issue a notification about receiving their sigs multiple times. /shrug

view details

push time in 7 hours

PR merged ElementsProject/lightning

Reviewers
cleanups for some dual-funding related code, III of III

This one's actually a big logical change. Prior to this (in #4069), we would wait until the user provided the signatures for openchannel_signed before we opened up a channeld and sent them to our peer.

The problem with this is that we wouldn't receive the peer's sigs until after we've sent our sigs. To make this work with multifundchannel (coming soon, watch this spaceTM), we need to be able to collect all of our peer's sigs first, collate them for all the other peer's we're signing with, and then present the collected sigs (ours plus all the other nodes) to the peers.

To do this, we go ahead and jump directly into channeld after we've collected the commitment_sigs from our peer. We can now call openchannel_signed at our leisure -- ideally as soon as you've got all the signatures you need!

Note that this impacts our reconnect logic in channeld; we'll send the tx_sigs and funding_locked on reconnect until we've gotten a funding_locked from the peer.

Right, so to summarize

  • openchannel_signed is no longer a blocker for starting channeld
  • There's some logic in channeld now around whether we've sent sigs to our peer yet or not and whether we've gotten them back from the peer (or not)
  • If the peer sends us funding_locked but we haven't gotten tx_sigs from them yet, we'll close the channel
  • There's now a notification when we receive sigs from the peer

Built on top of #4134, this commit set starts at 22bf555

+384 -152

0 comment

12 changed files

niftynei

pr closed time in 7 hours

push eventElementsProject/lightning

Rusty Russell

commit sha 1bf3eebbf61577cd6c9c1bcdeeb3333f1ca58889

dijkstra: fix heap ordering. We were always ordering heap by distance, not score (which are different if we are routing by cheapest, not shortest!). This simplifies our callbacks, too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 30bf6706b7738779f5dd920a8c22a6d340854792

route: return NULL if destination is unreachable. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha bb9ad57a031af510d8cec96fdef231888e9e7860

gossip_store: don't copy old delete markers on startup compact. So we don't have to handle them at load time, either. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 52c465fef07de6eccff46dda79e28cd946dfc61a

common/gossmap: fix gossmap_node_get_announce() on unannounced nodes. We would return junk before. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha b80342b9284839ca88730643005199cc7852ae1f

gossmap: implement feature tests Faster than pulling the announce msg and parsing. We need this to test if the node supports TLV. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha c6625943b567c11bacb86e176f4a82af3e4ccf5e

pytest: test that route can see private channels. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 83aea6b2bbf40bd85bd4f842ba949686556160f7

gossip_store: make private channels more similar to channel_announcement Instead of a boutique message, use a "real" channel_announcement for private channels (with fake sigs and pubkeys). This makes it far easier for gossmap to handle local channels. Backwards compatible update, since we update old stores. We also fix devtools/dump-gossipstore to know about the tombstone markers. Since we increment our channel_announce count for local channels now, the stats in the tests changed too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 639eddf840f85c696a495a17ad11645a6d577eb8

common/gossmap: digest private channel information too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 92f2461b5d939b507219932561b2c589062802e6

plugins/pay: fix leak on failed new payments. Start with attaching the payment to cmd (in case of failure), then steal onto the plugin itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha eadf2c91feba6ee027c5a910c42e9f2602026966

libplugin-pay: incorporate gossip store. So we can use this for routing determinations. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha b470ae2c73e2414387d5502eae766230278c80a6

plugins/libplugin-pay: use gossmap. This is a fairly direct translation. Even so, it should be faster in most cases, and and we can do more sophisticated things if we want. This also handles disabled channels better. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.

view details

Rusty Russell

commit sha f3bd57a0886ff75a3647202ee25486ef8cec2f80

common: cleanups suggested by Christian Decker's review. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 7 hours

PR merged ElementsProject/lightning

Pay: use gossmap gossip pay-plugin

This is a simple translation, but it involves a routing fix and some enhancements.

Building on this to get more route information is a job for @cdecker.

(Moving getroute et-al out to a plugin is still in my todo list, as is random routing!)

+857 -374

4 comments

39 changed files

rustyrussell

pr closed time in 7 hours

push eventElementsProject/lightning

niftynei

commit sha 6926711f4578d6c1e29e02e4ffd5b560e18742b0

dualopend, nit: move some lines to be within 80chars Random tidy of a few lines to be <=80 characters

view details

niftynei

commit sha 8bf9b4132b2708579ee85f7ae2121b38b731c485

df: simplify `check_balances`, add spec quotes `check_balances` had a weird interface because it was meant to be able to be used at any 'intermediate' point to verify that a single side had a valid inputs/output balance. This was worse than useless. Now it just straight checks for both sides' balances are correct and that both sides pay their fees. Called after transaction is constructed.

view details

niftynei

commit sha 5c04ff1ad795ec4f6facd5726c80e3e40e9ac8a7

df: Pass the serial_id of the funding output to openchannel_init caller This is handy/necessary for getting multifundchannel to work, as we need to know what output to tell all the other peers about. Changelog-Added: Experimental!! JSON-RPC: openchannel_init returns a field `funding_serial` that indicates the serial_id of the funding output in the provided PSBT

view details

niftynei

commit sha 02c3f114056069c2306e004d153bd5edc6fd1391

df, bugfix: dont free the `uc` yet we free it later, which is a problem if we also free it here.

view details

niftynei

commit sha a97e61244221454efbed9c1080b33928d3cfff72

df, bugfix: set the reserve correctly on the channel We weren't passing it through to channeld. Gotta set it on the uc->channel_config for it to all *just work* TM

view details

niftynei

commit sha 6d650064a0b1ddba8a7be14325895d4549806da0

df, nit: make this error message a little bit more informative

view details

niftynei

commit sha 4a1843a1515e52cd13513c259ce10bb1e87db1ae

df, bugfix: use tal_wally around an allocation

view details

push time in 9 hours

PR merged ElementsProject/lightning

Reviewers
cleanups for some dual-funding related code, II of III

This one's mostly small fixes found while running some tests. Plus we completely rework check_balances, the method that needed some help anyway.

Based on #4133, start at 3262c3a

+322 -134

2 comments

7 changed files

niftynei

pr closed time in 9 hours

pull request commentElementsProject/lightning

cleanups for some dual-funding related code, II of III

Ack https://github.com/ElementsProject/lightning/commit/dc9bcfb4263782638ad71e0b0a59f35a78ed1819

niftynei

comment created time in 9 hours

PullRequestReviewEvent

pull request commentElementsProject/lightning

opening: use correct dust_limit for reserve floor

Changelog-Fixed?

niftynei

comment created time in a day

push eventniftynei/lightning

niftynei

commit sha 4034d0c306c4d3f1ac6b387bca64152bd79e419c

psbt: have the unknown map 'add' be a 'set' instead

view details

niftynei

commit sha 3674de9865f29a235f1ea3b43f5c1cf47fca4400

json: add channel_id helper

view details

niftynei

commit sha b6a7b52a3e8e6687f16d82774a98e87985770f8d

json nit: use const for json_add_psbt

view details

niftynei

commit sha b4773203bb3d78a99c182a4a2eb78b7cbafec376

psbt-finalized: hoist method to common

view details

niftynei

commit sha 9d4afd588002a7335065153aba729ebd45c267d7

psbt: hoist up `psbt_add_serials`, so we can use it elsewhere We're going to use this in multifundchannel.

view details

niftynei

commit sha 8317957db237a36ed178ca06dac5586c491be352

feerate: remove duplicate method is dupe of `bitcoin_tx_core_weight`

view details

niftynei

commit sha cdfb82533626aa0158d552b3d13d7249bad4be7c

nit: move changeset_get_next to inside EXPERIMENTAL_FEATURES flag These reference the wires that are flagged behind EXPERIMENTAL_FEATURES, moving this here makes things compile ok.

view details

niftynei

commit sha 4508584b212299e078366c55be30c44bbdcc50cb

dualfund: rearrange things so that the wire-dependent calls are separate There's a few structs/wire calls that only exist under experimental features. These were in a common file that was shared/used a bunch of places but this causes problems. Here we move one of the problematic methods back into `openingd`, as it's only used locally and then isolate the references to the `witness_stack` in a new `common/psbt_internal` file. This lets us remove the iff EXP_FEATURES inclusion switches in most of the Makefiles.

view details

niftynei

commit sha ef484d2c1a4d9754c0a4652ceeb4fc8ebe203521

dualopend, nit: move some lines to be within 80chars Random tidy of a few lines to be <=80 characters

view details

niftynei

commit sha 95cd2d33263f2ac7924034635bb94a25691a5c39

df: simplify `check_balances`, add spec quotes `check_balances` had a weird interface because it was meant to be able to be used at any 'intermediate' point to verify that a single side had a valid inputs/output balance. This was worse than useless. Now it just straight checks for both sides' balances are correct and that both sides pay their fees. Called after transaction is constructed.

view details

niftynei

commit sha 482f9508a1affe18515a9425b01bba28b7af0caf

df: Pass the serial_id of the funding output to openchannel_init caller This is handy/necessary for getting multifundchannel to work, as we need to know what output to tell all the other peers about. Changelog-Added: Experimental!! JSON-RPC: openchannel_init returns a field `funding_serial` that indicates the serial_id of the funding output in the provided PSBT

view details

niftynei

commit sha 62101c238770b52785169fc12273d1d5909d2120

df, bugfix: dont free the `uc` yet we free it later, which is a problem if we also free it here.

view details

niftynei

commit sha 555e3b39f6b5f01675301a45d2b15e6029a665c5

df, bugfix: set the reserve correctly on the channel We weren't passing it through to channeld. Gotta set it on the uc->channel_config for it to all *just work* TM

view details

niftynei

commit sha ab652841af391f271fba014633f5ab254489d558

df, nit: make this error message a little bit more informative

view details

niftynei

commit sha dc9bcfb4263782638ad71e0b0a59f35a78ed1819

df, bugfix: use tal_wally around an allocation

view details

push time in a day

push eventrustyrussell/lightning

niftynei

commit sha 4034d0c306c4d3f1ac6b387bca64152bd79e419c

psbt: have the unknown map 'add' be a 'set' instead

view details

niftynei

commit sha 3674de9865f29a235f1ea3b43f5c1cf47fca4400

json: add channel_id helper

view details

niftynei

commit sha b6a7b52a3e8e6687f16d82774a98e87985770f8d

json nit: use const for json_add_psbt

view details

niftynei

commit sha b4773203bb3d78a99c182a4a2eb78b7cbafec376

psbt-finalized: hoist method to common

view details

niftynei

commit sha 9d4afd588002a7335065153aba729ebd45c267d7

psbt: hoist up `psbt_add_serials`, so we can use it elsewhere We're going to use this in multifundchannel.

view details

niftynei

commit sha 8317957db237a36ed178ca06dac5586c491be352

feerate: remove duplicate method is dupe of `bitcoin_tx_core_weight`

view details

niftynei

commit sha cdfb82533626aa0158d552b3d13d7249bad4be7c

nit: move changeset_get_next to inside EXPERIMENTAL_FEATURES flag These reference the wires that are flagged behind EXPERIMENTAL_FEATURES, moving this here makes things compile ok.

view details

niftynei

commit sha 4508584b212299e078366c55be30c44bbdcc50cb

dualfund: rearrange things so that the wire-dependent calls are separate There's a few structs/wire calls that only exist under experimental features. These were in a common file that was shared/used a bunch of places but this causes problems. Here we move one of the problematic methods back into `openingd`, as it's only used locally and then isolate the references to the `witness_stack` in a new `common/psbt_internal` file. This lets us remove the iff EXP_FEATURES inclusion switches in most of the Makefiles.

view details

Rusty Russell

commit sha 949e9b016f3b62dda9a950f5c1f06ca647df1448

dijkstra: fix heap ordering. We were always ordering heap by distance, not score (which are different if we are routing by cheapest, not shortest!). This simplifies our callbacks, too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha c652c4f9471f0302e0f03da4ee9e05bd4466ae72

route: return NULL if destination is unreachable. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 5641b8f7f4c52d0b3d50770d5540ce7f651217ed

gossip_store: don't copy old delete markers on startup compact. So we don't have to handle them at load time, either. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 3f29d3beaa754195229a811e137963037178f56d

common/gossmap: fix gossmap_node_get_announce() on unannounced nodes. We would return junk before. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha c6750ce513f5ddbb9c8035d2fe498783c0291ddc

gossmap: implement feature tests Faster than pulling the announce msg and parsing. We need this to test if the node supports TLV. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha bfcbe3e9e85e3d7881714b86026edc70a7a63124

pytest: test that route can see private channels. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 0478606329834cd2ec25504f1913de1dc755e280

gossip_store: make private channels more similar to channel_announcement Instead of a boutique message, use a "real" channel_announcement for private channels (with fake sigs and pubkeys). This makes it far easier for gossmap to handle local channels. Backwards compatible update, since we update old stores. We also fix devtools/dump-gossipstore to know about the tombstone markers. Since we increment our channel_announce count for local channels now, the stats in the tests changed too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 927fefe3060d3cc0189a2a845966718a90d8ae77

common/gossmap: digest private channel information too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha c5411bd1e16aaab2d5fcbd1e566716a94b67e017

plugins/pay: fix leak on failed new payments. Start with attaching the payment to cmd (in case of failure), then steal onto the plugin itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 9513318d2e5f0d298ef5989d16d68b401bdcae91

libplugin-pay: incorporate gossip store. So we can use this for routing determinations. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha f73ac7f5d9beab28db6d84900fcc67f18474fa53

plugins/libplugin-pay: use gossmap. This is a fairly direct translation. Even so, it should be faster in most cases, and and we can do more sophisticated things if we want. This also handles disabled channels better. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.

view details

Rusty Russell

commit sha 64673c45d5e0c3b9447b80e2f6ed5aab7e725579

common: cleanups suggested by Christian Decker's review. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in a day

push eventElementsProject/lightning

niftynei

commit sha 4034d0c306c4d3f1ac6b387bca64152bd79e419c

psbt: have the unknown map 'add' be a 'set' instead

view details

niftynei

commit sha 3674de9865f29a235f1ea3b43f5c1cf47fca4400

json: add channel_id helper

view details

niftynei

commit sha b6a7b52a3e8e6687f16d82774a98e87985770f8d

json nit: use const for json_add_psbt

view details

niftynei

commit sha b4773203bb3d78a99c182a4a2eb78b7cbafec376

psbt-finalized: hoist method to common

view details

niftynei

commit sha 9d4afd588002a7335065153aba729ebd45c267d7

psbt: hoist up `psbt_add_serials`, so we can use it elsewhere We're going to use this in multifundchannel.

view details

niftynei

commit sha 8317957db237a36ed178ca06dac5586c491be352

feerate: remove duplicate method is dupe of `bitcoin_tx_core_weight`

view details

niftynei

commit sha cdfb82533626aa0158d552b3d13d7249bad4be7c

nit: move changeset_get_next to inside EXPERIMENTAL_FEATURES flag These reference the wires that are flagged behind EXPERIMENTAL_FEATURES, moving this here makes things compile ok.

view details

niftynei

commit sha 4508584b212299e078366c55be30c44bbdcc50cb

dualfund: rearrange things so that the wire-dependent calls are separate There's a few structs/wire calls that only exist under experimental features. These were in a common file that was shared/used a bunch of places but this causes problems. Here we move one of the problematic methods back into `openingd`, as it's only used locally and then isolate the references to the `witness_stack` in a new `common/psbt_internal` file. This lets us remove the iff EXP_FEATURES inclusion switches in most of the Makefiles.

view details

push time in a day

PR merged ElementsProject/lightning

Reviewers
cleanups for some dual-funding related code, I of III

Bunch of code move + rework patches for the next big dual-funding patchset

Built on top of #4055, start at 927277a

+365 -324

1 comment

25 changed files

niftynei

pr closed time in a day

push eventrustyrussell/lightning

niftynei

commit sha e2a6fd711296aca262d846743d1dbc44e34456d7

common: pull up `param_psbt` Usable other places, not just in wallet

view details

niftynei

commit sha 3c98bec693404c0db6ffb5367e616d20d3fb2456

db: add helper for db_col_psbt We had db_bind_psbt; this is the missing symmetric response to that

view details

niftynei

commit sha 82c0b48215e0786f536bc8ff555ef89952ccc82d

wires: towire/fromwire for wally_tx We're eventually moving away from 'bitcoin_tx

view details

niftynei

commit sha 7a5a7a0ac50ae6aee726ddfc8fc148c615450ae2

psbt_open: method for adding a witness_stack to a psbt input Set a stack of witnesses onto a PSBT input

view details

niftynei

commit sha 93045945be14a325f09aa9b5bcbb4d98bba03ccb

psbt: add helper to set the non-witness utxo for an input We need this info for every dual-funded open

view details

niftynei

commit sha 9af6e83393cc5c6e2b1004d6cb85492033f187c2

dual-fund: check the max feerate also, not just min feerate We cap beneath our max feerate

view details

niftynei

commit sha 9c1675fcb9db652a8c792ffefdd28d70f8bd4342

dual-fund: remove flag for option_anchor_outputs It's assumed true

view details

niftynei

commit sha d0702e05dd462f9147576f90e5a4cfceba6e60d3

psbt: add the full transaction for the utxo in a psbt We need this for dual funding, since the interactive tx construction protocol requires the full tx to send places. We add it to all PSBTs (if we have it), here

view details

niftynei

commit sha 99293844a100d09f1c415a49458b6f061fbe6837

dualopend: use status_failed, not peer_failed

view details

niftynei

commit sha 2618ef10c38b6437cb3b37094f0fb4c0cb569fb1

tx_roles: pull up roles, rename We're going to use these elsewhere/more widely so having them in common and using a more generic name is a obvious place to start.

view details

niftynei

commit sha 06c41a05474eee8891610005bfb275574e5705c3

dualfund: opener, openchannel_init command (1/3) There are 3 commands for opening a channel with dualfunding. `openchannel_init` is the first of these. It initializes the open-channel dialog, and stops once we've run out of updates (input/outputs) to send to the peer.

view details

niftynei

commit sha 3a405c33e6a835cf41aa25216cbe7e833079b3c3

dualfund: Pass in expected remote's serial parity Now that we've got the opener in progress, we need to be able to toggle which parity to check a remote's serial_ids for

view details

niftynei

commit sha a7f29f30dba7b04d311465902716a6902f44b3fa

df-open: pathway for getting a commit back from peer Goes all the way back to where we save it to the database and return whatever command kicked this off

view details

niftynei

commit sha b2ec5a9f451a9d347f5836efad889d5cd351f9bd

peer_channeld: pass over PSBT, remove second message We need the PSBT to create the finalized tx from once the peer's tx_signatures are received. Since we're passing the PSBT, we no longer need the secondary message to be passed, as it was derived from the PSBT. Also removes now unused witness serialization code

view details

niftynei

commit sha 537eeab20872e4e3cbc2e5ed8d5352e663d15d6e

df-open: add a 'open_commands' list to stash pending opens around in `openchannel_signed` commands hang out across the openingd/channeld boundary -- we don't return until we've successfully broadcast the transaction (or timed out waiting for them to send a tx_sigs back).

view details

niftynei

commit sha 8858ae4f3d55e870f5f57b8000357fd3e09def15

df-open: commands to update a PSBT or submit a signed PSBT `openchannel_signed` and `openchannel_update` which allow a user to continue a openchannel or kick off the completion of a openchannel. `openchannel_update` should be called until it returns with `commitments_secured`.

view details

niftynei

commit sha 0df818c53cffe00804d4bb4d53f3837cd618c86c

df-open: preliminary handling for tx_sigs message Missing some thing still (like persistence and broadcasting the tx)

view details

niftynei

commit sha 254ea37702531be4a971813a008c4688b99178e3

psbt: method for extracting witness stacks

view details

niftynei

commit sha 46641951faa2e3c851ac81c3296d5babdcc3b4bd

dual-open: use tx_roles, not side, as switch It's easier to reason about

view details

niftynei

commit sha aa1b8296c7c5ea4228b8ae77bda83a800aca65d2

peer_control: move open_command up to where channeld can get it, also include a method for finding a pending/available open_command for a channel

view details

push time in a day

push eventElementsProject/lightning

niftynei

commit sha e2a6fd711296aca262d846743d1dbc44e34456d7

common: pull up `param_psbt` Usable other places, not just in wallet

view details

niftynei

commit sha 3c98bec693404c0db6ffb5367e616d20d3fb2456

db: add helper for db_col_psbt We had db_bind_psbt; this is the missing symmetric response to that

view details

niftynei

commit sha 82c0b48215e0786f536bc8ff555ef89952ccc82d

wires: towire/fromwire for wally_tx We're eventually moving away from 'bitcoin_tx

view details

niftynei

commit sha 7a5a7a0ac50ae6aee726ddfc8fc148c615450ae2

psbt_open: method for adding a witness_stack to a psbt input Set a stack of witnesses onto a PSBT input

view details

niftynei

commit sha 93045945be14a325f09aa9b5bcbb4d98bba03ccb

psbt: add helper to set the non-witness utxo for an input We need this info for every dual-funded open

view details

niftynei

commit sha 9af6e83393cc5c6e2b1004d6cb85492033f187c2

dual-fund: check the max feerate also, not just min feerate We cap beneath our max feerate

view details

niftynei

commit sha 9c1675fcb9db652a8c792ffefdd28d70f8bd4342

dual-fund: remove flag for option_anchor_outputs It's assumed true

view details

niftynei

commit sha d0702e05dd462f9147576f90e5a4cfceba6e60d3

psbt: add the full transaction for the utxo in a psbt We need this for dual funding, since the interactive tx construction protocol requires the full tx to send places. We add it to all PSBTs (if we have it), here

view details

niftynei

commit sha 99293844a100d09f1c415a49458b6f061fbe6837

dualopend: use status_failed, not peer_failed

view details

niftynei

commit sha 2618ef10c38b6437cb3b37094f0fb4c0cb569fb1

tx_roles: pull up roles, rename We're going to use these elsewhere/more widely so having them in common and using a more generic name is a obvious place to start.

view details

niftynei

commit sha 06c41a05474eee8891610005bfb275574e5705c3

dualfund: opener, openchannel_init command (1/3) There are 3 commands for opening a channel with dualfunding. `openchannel_init` is the first of these. It initializes the open-channel dialog, and stops once we've run out of updates (input/outputs) to send to the peer.

view details

niftynei

commit sha 3a405c33e6a835cf41aa25216cbe7e833079b3c3

dualfund: Pass in expected remote's serial parity Now that we've got the opener in progress, we need to be able to toggle which parity to check a remote's serial_ids for

view details

niftynei

commit sha a7f29f30dba7b04d311465902716a6902f44b3fa

df-open: pathway for getting a commit back from peer Goes all the way back to where we save it to the database and return whatever command kicked this off

view details

niftynei

commit sha b2ec5a9f451a9d347f5836efad889d5cd351f9bd

peer_channeld: pass over PSBT, remove second message We need the PSBT to create the finalized tx from once the peer's tx_signatures are received. Since we're passing the PSBT, we no longer need the secondary message to be passed, as it was derived from the PSBT. Also removes now unused witness serialization code

view details

niftynei

commit sha 537eeab20872e4e3cbc2e5ed8d5352e663d15d6e

df-open: add a 'open_commands' list to stash pending opens around in `openchannel_signed` commands hang out across the openingd/channeld boundary -- we don't return until we've successfully broadcast the transaction (or timed out waiting for them to send a tx_sigs back).

view details

niftynei

commit sha 8858ae4f3d55e870f5f57b8000357fd3e09def15

df-open: commands to update a PSBT or submit a signed PSBT `openchannel_signed` and `openchannel_update` which allow a user to continue a openchannel or kick off the completion of a openchannel. `openchannel_update` should be called until it returns with `commitments_secured`.

view details

niftynei

commit sha 0df818c53cffe00804d4bb4d53f3837cd618c86c

df-open: preliminary handling for tx_sigs message Missing some thing still (like persistence and broadcasting the tx)

view details

niftynei

commit sha 254ea37702531be4a971813a008c4688b99178e3

psbt: method for extracting witness stacks

view details

niftynei

commit sha 46641951faa2e3c851ac81c3296d5babdcc3b4bd

dual-open: use tx_roles, not side, as switch It's easier to reason about

view details

niftynei

commit sha aa1b8296c7c5ea4228b8ae77bda83a800aca65d2

peer_control: move open_command up to where channeld can get it, also include a method for finding a pending/available open_command for a channel

view details

push time in a day

PR merged ElementsProject/lightning

Reviewers
Opener side of the draft dual-funding spec

A few more commits, to make the opener side of the dual-funding piece work.

The opener side requires 3 new RPC calls:

  • openchannel_init
  • openchannel_update
  • openchannel_signed

Each of these take a PSBT. An opener initiates an v2 open by calling openchannel_init with a PSBT that includes enough funding for their desired feerate and a funding output, which will be added to the provided PSBT with the amount the caller specified plus however much the peer adds. This method will return a PSBT with partial additions from the peer.

Next, you'll update the PSBT with any additions/changes you have, then call openchannel_update with the updated (or identical) PSBT. Once both sides have stopped updating the PSBT, openchannel_update will return with commitments_secured set to true. (Until this point it will return commitments_secured as false).

The next step is to pass a PSBT with partial_signatures for every input you're contributing to this funding transaction to openchannel_signed, which will collect the transaction signatures from the peer, build the final transaction and broadcast it.

For a working example of this see tests/plugins/df_opener.py.

FIXME: the df_opener plugin uses wallycore which isn't part of the build/pip install process, which is why the provided test is flagged off. @jgriffiths any word on when it might appear on pypi?

~FIXME: needs docs for new RPCs~

+3550 -374

2 comments

93 changed files

niftynei

pr closed time in a day

pull request commentElementsProject/lightning

Pay: use gossmap

Ah, tests without COMPAT enabled. OK, fixed.

rustyrussell

comment created time in a day

push eventrustyrussell/lightning

Rusty Russell

commit sha 34bb80b80ea15c7add97f08f6ba69716767fb56b

gossip_store: make private channels more similar to channel_announcement Instead of a boutique message, use a "real" channel_announcement for private channels (with fake sigs and pubkeys). This makes it far easier for gossmap to handle local channels. Backwards compatible update, since we update old stores. We also fix devtools/dump-gossipstore to know about the tombstone markers. Since we increment our channel_announce count for local channels now, the stats in the tests changed too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha c604ea2062bf1f88667514200af74e2dd38d9e47

common/gossmap: digest private channel information too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 5e47c1adbddc73edb4ef80f90694f10850520eba

plugins/pay: fix leak on failed new payments. Start with attaching the payment to cmd (in case of failure), then steal onto the plugin itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha c97236a1e67763715b3ce9f6eb33a48795e51244

libplugin-pay: incorporate gossip store. So we can use this for routing determinations. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 8d64304ff57d188524d1fa6ae5b5076fb2b13ec1

plugins/libplugin-pay: use gossmap. This is a fairly direct translation. Even so, it should be faster in most cases, and and we can do more sophisticated things if we want. This also handles disabled channels better. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.

view details

Rusty Russell

commit sha 1f48e7ebdfc097ef0094cb9052a10f2fe9336677

common: cleanups suggested by Christian Decker's review. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in a day

push eventrustyrussell/lightning

Rusty Russell

commit sha 79a21755c4a84f6cd114f4a433f84e538fd7681d

pytest: fix flaky test if one side hasn't processed close yet. ``` @unittest.skipIf(TEST_NETWORK != 'regtest', "External wallet support doesn't work with elements yet.") def test_funding_close_upfront(node_factory, bitcoind): ... # check that you can provide a closing address upfront addr = l1.rpc.newaddr()['bech32'] > _fundchannel(l1, l2, amt_normal, addr) ... pyln.client.lightning.RpcError: RPC call failed: method: fundchannel_start, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 100000, 'announce': True, 'close_to': 'bcrt1qctx2k9cu9fd7nk449mzphqjcvvpyc4rxh6826x'}, error: {'code': -1, 'message': 'They sent error channel 2a1ca624cd1127761cb7a4395df2c3fd6d0abb3732c1f85a5345b0da716540d0: Multiple channels unsupported'} ``` Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Christian Decker

commit sha 9f8c9d64064b2509b31bf48df65918dc855efce2

pytest: Check for connection close and reset errors in timeout test I get both errors from time to time, seems like a timing issue.

view details

Christian Decker

commit sha d5d41ef1a0efad1e87acd36c9b33789f127dd487

pyln: Remove any logging handlers at teardown to avoid logging error Inspired by https://github.com/pytest-dev/pytest/issues/5502#issuecomment-647157873

view details

Christian Decker

commit sha 30ecb172046b0eae61a23c195156ba0cd533fa36

make: Include the $PYTEST_OPTS when running pytest

view details

Christian Decker

commit sha c4f4f4744c7a4da9d4d5556f2a405adc7fd44f87

travis: Allow environment to specify $PYTEST_PAR

view details

Christian Decker

commit sha aed21e2afa8c03f61d3870e6ee515dbd6e68ab76

travis: Consolidate pytest options to cli options We were merging pytest.ini options with command line options, which is a bit weird.

view details

Christian Decker

commit sha ae9753df3a1abe34caf0ebec0a41edf5946f4676

travis: Specify pytest-rerunfailures==9.1 to avoid regression Seems the recently released version 9.1.1 has regressed, and isn't actually rerunning failed tests. Pinning it to 9.1 seems to work however.

view details

Rusty Russell

commit sha a297c1b9a506db2b1a000d1f1c730aafe61639ba

pytest: fix overzealous removal of directories on failure. We were always deleting the directories. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Christian Decker

commit sha 556725c5ff1b8bb1454676dfec658f43e91bd925

pyln: Add logging handler that passes records to lightningd It is often pretty usefuk to use the builtin logging module to debug things, including libraries that a plugin may use. This adds a simple `PluginLogHandler` that maps the python logging levels to the `lightningd` logging levels, and formats the record in a way that it doesn't clutter up the `lightningd` logs (no duplicate timestamps and levels). This allow us to tweak the log level that is reported to `lightningd` simply using the following ```python3 import logging logging.basicConfig(level=logging.DEBUG) ``` Notice that in order for the logs to be displayed on the terminal or the logfile, both the logging level in the plugin _and_ the `--log-level` `lightningd` is running need to be adjusted (the python logging level only controls which messages get forwarded to `lightningd`, it does not have the power to overrule `lightningd` about what to actually display). I chose `logging.INFO` as the default, since libraries have a tendency to spew out everything in `logging.DEBUG` mode Changelog-Added: pyln: Plugins have been integrated with the `logging` module for easier debugging and error reporting.

view details

Christian Decker

commit sha b1aed933e64bc7c1dc1db820618e32cf4b6c3a7f

pyln: Plugin methods and hooks refuse to set results twice We had a couple of instances where a plugin would be killed by `lightningd` because we were returning a result of an exception twice, and it was hard to trace down the logic error in the user plugin that caused that. This patch adds a traceback the first time we return a result/exception, and raise an exception with a stacktrace of the first termination when a second one comes in. This can still terminate the plugin, but the programmer gets a clear indication where the result was set, and can potentially even recover from it. Changelog-Added: pyln: Plugin method and hook requests prevent the plugin developer from accidentally setting the result multiple times, and will raise an exception detailing where the result was first set.

view details

Rusty Russell

commit sha 6195d953ccd62449ce2fb3ac6c5641cab4355168

plugins: use "slow" feerate for mutual close negotiation. We're rarely in a hurry here, and bitcoind is aggressive with fees. You can always spend this output if you really have to, using CPFP. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Changed: Protocol: mutual closing feerate reduced to "slow" to avoid overpaying.

view details

Rusty Russell

commit sha 4ba9ad66bca4bd9900ba981177ecf92d0f04e62d

options: remove unused 'commit-fee-min/max' options. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Jan Sarenik

commit sha 90acf5072d0bee2f71e5e9789b36bdeca9ccabe4

pyln: utils:BitcoinD: add -wallet="test" option Per https://github.com/bitcoin/bitcoin/pull/15454

view details

Rusty Russell

commit sha ec868d4acb4f607e8729b7ad404c38100369ba24

lightningd: fix crash when we try to send fail_htlc msg to onchaind. Great report from whitslack on this crash at startup: ``` 2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1) 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d ``` The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN, because onchaind has started but not responded to init yet (which it does once it has analyzed the commitment tx). Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg to channel A's onchaind as if it were channeld. Explicitly check if it's channeld, rather than trying to see if it's onchaind. Fixes: #4114 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: crash: assertion fail at restart when source and destination channels of an HTLC are both onchain.

view details

Rusty Russell

commit sha d5c91a347aa33322373ffd5d6cd1aaf07ba50185

channeld: log broken message if we receive HTLCs out of order. We didn't care, but other implementations (particularly lnd) do. And it does violate the spec. (We need to use skip not xfail on the test which catches this, since xfail doesn't seem to stop errors reported by cleanup) (Includes Christian's typo fix!) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha d04597cbb68ea14755c9ddcffa7519fcbb7cb8f8

pytest: add test for HTLC ordering. [ Includes tidyups by Christian Decker ] Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha a3c30441d31e95e70a5e102d53dd21f45bf13e74

channeld: order htlcs by id before retransmission. We had one report of this, and then Eugene and Roasbeef of Lightning Labs confirmed it; they saw misordered HTLCs on reconnection too. Since we didn't enforce this when we receive HTLCs, we never noticed :( Fixes: #3920 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)

view details

grubles

commit sha 94f84d38436a6bbab0b0b3fe17703390761927c9

Update configure OpenBSD's `sha256sum` equivalent is `sha256`. This adds support in `configure` for it.

view details

Jan Sarenik

commit sha b8c972a0e2dea71fa9a70aab4e408db87deaa988

doc: Add missing signet to --help and man

view details

Jan Sarenik

commit sha 2dd6b82dfbd596934ac0f500dbbaf6994af17701

doc: reorder --mainnet before --testnet

view details

push time in a day

push eventrustyrussell/lightning

Rusty Russell

commit sha a6aecb9d1feb3449bcf02556e4f8b68da2bb9492

dijkstra: fix heap ordering. We were always ordering heap by distance, not score (which are different if we are routing by cheapest, not shortest!). This simplifies our callbacks, too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 5e97bcab11514013d2d0703d0c530a9f521433a0

route: return NULL if destination is unreachable. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 76b529b38d041623d2ff2c660b0f73cfcb1935da

gossip_store: don't copy old delete markers on startup compact. So we don't have to handle them at load time, either. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 7dd353d0f8b49e6f6755b5911c1bf0b0667fe332

common/gossmap: fix gossmap_node_get_announce() on unannounced nodes. We would return junk before. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 382dd2124f7ebd4ad568f324da6e44894e7cf187

gossmap: implement feature tests Faster than pulling the announce msg and parsing. We need this to test if the node supports TLV. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 6ff1400975578edf0b1c75977f9e7cab3bd0786e

pytest: test that route can see private channels. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha f6130685272a5534382d02bc358bc86d89161032

gossip_store: make private channels more similar to channel_announcement Instead of a boutique message, use a "real" channel_announcement for private channels (with fake sigs and pubkeys). This makes it far easier for gossmap to handle local channels. Backwards compatible update, since we update old stores. We also fix devtools/dump-gossipstore to know about the tombstone markers. Since we increment our channel_announce count for local channels now, the stats in the tests changed too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 6c4e1e0b64588135a281559ab0487c4cde4a710a

common/gossmap: digest private channel information too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 7e4b2456ee1f8dd8dae4c2f724e25d5081853fb0

plugins/pay: fix leak on failed new payments. Start with attaching the payment to cmd (in case of failure), then steal onto the plugin itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 7321ca8f527db9b4d50dbd91ccbd6b29ff3ba95b

libplugin-pay: incorporate gossip store. So we can use this for routing determinations. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha a0ff1efa43fc06ba26f60248a0249893686972d8

plugins/libplugin-pay: use gossmap. This is a fairly direct translation. Even so, it should be faster in most cases, and and we can do more sophisticated things if we want. This also handles disabled channels better. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.

view details

Rusty Russell

commit sha 764819ad72c3557011caf74e07c94094209bbf5b

common: cleanups suggested by Christian Decker's review. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 2 days

Pull request review commentElementsProject/lightning

Pay: use gossmap

 u8 *gossmap_node_get_announce(const tal_t *ctx, 	map_copy(map, n->nann_off, msg, len); 	return msg; }++/* BOLT #7:+ * 1. type: 256 (`channel_announcement`)+ * 2. data:+ *     * [`signature`:`node_signature_1`]+ *     * [`signature`:`node_signature_2`]+ *     * [`signature`:`bitcoin_signature_1`]+ *     * [`signature`:`bitcoin_signature_2`]+ *     * [`u16`:`len`]+ *     * [`len*byte`:`features`]+ *     * [`chain_hash`:`chain_hash`]+ *     * [`short_channel_id`:`short_channel_id`]+ *     * [`point`:`node_id_1`]+ *     * [`point`:`node_id_2`]+ */+int gossmap_chan_has_feature(const struct gossmap *map,+			     const struct gossmap_chan *c,+			     int fbit)+{+	/* Note that first two bytes are message type */+	const size_t feature_len_off = 2 + (64 + 64 + 64 + 64);+	size_t feature_len;++	feature_len = map_be16(map, c->cann_off + feature_len_off);++	if (map_feature_set(map, OPTIONAL_FEATURE(fbit),+			    c->cann_off + feature_len_off + 2, feature_len))+		return OPTIONAL_FEATURE(fbit);+	if (map_feature_set(map, COMPULSORY_FEATURE(fbit),

Yes, it's fairly cheap, but since this is the only way we call it...

rustyrussell

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentElementsProject/lightning

Pay: use gossmap

 struct dijkstra { 	/* NULL means it's been visited already. */ 	const struct gossmap_node **heapptr; +	/* How we decide "best" */

Good idea, I'll do a followon cleanup commit.

rustyrussell

comment created time in 2 days

PullRequestReviewEvent

Pull request review commentElementsProject/lightning

openingd/: Fail `fundchannel_start` if we already are, or will become, the fundee.

 static void opening_funder_finished(struct subd *openingd, const u8 *resp, 	tal_free(fc->uc); } +static void+opening_funder_failed_cancel_commands(struct uncommitted_channel *uc,

@niftynei is right here; C is always written bottom to top. Sometimes judicious use of pre-declaration can reduce diff size, but best practice is usually a separate preparatory commit to reorder.

ZmnSCPxj

comment created time in 2 days

PullRequestReviewEvent

push eventrustyrussell/lightning-rfc

Rusty Russell

commit sha 984b6e7cd89a2da4aa2c8cf177c60ae5bbf4f899

fixup! Invoice amount is either based on offer amount * offer currency * invoice quantity, or (no offer amount) the invreq amount field. Also ban the nonsensical periodic proportional amount if there's no specified amount. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 4 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void handle_peer_funding_locked(struct peer *peer, const u8 *msg) 					   &peer->channel_id));  	peer->funding_locked[REMOTE] = true;+	if (peer->psbt)+		peer->psbt = tal_free(peer->psbt);

Don't need to if (peer->psbt) here. tal_free(NULL) is a noop.

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void announce_channel(struct peer *peer) 	send_channel_update(peer, 0); } +static enum tx_role our_tx_role(struct peer *peer)+{+	return peer->channel->opener == LOCAL ?+		TX_INITIATOR : TX_ACCEPTER;+}++static enum tx_role their_tx_role(struct peer *peer)

const...

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void sendfunding_done(struct bitcoind *bitcoind UNUSED, 	tal_free(cs); } -static void handle_funding_tx(struct channel *channel,-			      const u8 *msg)+static void send_funding_tx(struct channel *channel,+			    const struct wally_tx *wtx TAKES) {-	struct channel_send *cs; 	struct lightningd *ld = channel->peer->ld;+	struct channel_send *cs;  	cs = tal(channel, struct channel_send); 	cs->channel = channel;--	if (!fromwire_channeld_funding_tx(tmpctx, msg, &cs->wtx)) {-		channel_internal_error(channel,-				       "bad channeld_funding_tx: %s",-				       tal_hex(tmpctx, msg));-		return;-	}+	if (taken(wtx))+		cs->wtx = tal_steal(cs, wtx);+	else+		cs->wtx = wtx;

If we're not supposed to take it, we should almost certainly clone it?

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void handle_funding_tx(struct channel *channel, 			   sendfunding_done, cs); } +static void peer_tx_sigs_msg(struct channel *channel, const u8 *msg)+{+	struct wally_psbt *psbt;+	const struct wally_tx *wtx;+	struct lightningd *ld = channel->peer->ld;++	if (!fromwire_channeld_funding_sigs(tmpctx, msg, &psbt)) {+		channel_internal_error(channel,+				       "bad channeld_funding_sigs: %s",+				       tal_hex(tmpctx, msg));+		return;+	}++	tal_wally_start();+	if (wally_psbt_combine(cast_const(struct wally_psbt *,+					  channel->psbt),

This breaks the rules; channel->psbt must not be const if we want to alter it. Casting away just removes the warning, but it's still no legal to alter it.

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void announce_channel(struct peer *peer) 	send_channel_update(peer, 0); } +static enum tx_role our_tx_role(struct peer *peer)

const...

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void send_onionmsg(struct peer *peer, const u8 *msg) 						    tlvs))); } +static void handle_send_tx_sigs(struct peer *peer, const u8 *msg)+{+	const struct wally_psbt *psbt;+	struct bitcoin_txid txid;++	if (!fromwire_channeld_send_tx_sigs(tmpctx, msg,+					    cast_const2(struct wally_psbt **,+						       &psbt)))

just de-const the declaration?

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void peer_reconnect(struct peer *peer,  #if EXPERIMENTAL_FEATURES 	/* Send our tx_sigs again */-	if (peer->psbt && !peer->funding_locked[REMOTE]) {+	enum tx_role role = peer->channel->opener == LOCAL+		? TX_INITIATOR : TX_ACCEPTER;

Hmm, maybe make an inline function enum tx_role our_role(enum side opener)?

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, III of III

 static void handle_tx_sigs(struct peer *peer, const u8 *msg) 	struct witness_stack **ws; 	const struct wally_tx *wtx; 	size_t j = 0;+	enum tx_role role = peer->channel->opener == REMOTE+		? TX_INITIATOR : TX_ACCEPTER;

This is their role, right? If they opened the channel, they're the initiator...

niftynei

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentElementsProject/lightning

cleanups for some dual-funding related code, II of III

 static bool find_txout(struct wally_psbt *psbt, const u8 *wscript, u16 *funding_ 	return false; } -static bool check_balances(struct state *state,+static bool check_balances(const tal_t *ctx,+			   struct state *state, 			   struct wally_psbt *psbt,-			   bool check_opener_balance,-			   bool do_full_tx_check,-			   u32 feerate_per_kw_funding)+			   u32 feerate_per_kw_funding,+			   char **err_reason)

Better interface would be to return char *, and NULL on success.

niftynei

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

push eventElementsProject/lightning

niftynei

commit sha 628df74e26ca8a6b775e6736e71d207c1678c17e

mfc: allow a 'close_to' address to be passed in This will allow us to let `fundchannel` handle a close to address, also

view details

niftynei

commit sha 4926c25bb5cd327aa9bfbc4d4156a63f168b24b3

fundchannel: take a 'close_to' address Finally, extends the 'close_to' functionality up to the flagship 'open a channel' command. Changelog-Added: JSON-API `fundchannel` now accepts an optional 'close_to' param, a bitcoin address that the channel funding should be sent to on close. Requires `opt_upfront_shutdownscript`

view details

niftynei

commit sha ee329f08de783b2cdc51a7e909fc0f7a2d816baf

test, fundchannel: pass kwargs down to fundchannel command Allows us to more easily pass through args to `fundchannel` while still using the utility function's funding + open confirmation logics

view details

niftynei

commit sha de34f08b820bd2b63412b6f5913a9025ea3bf98e

tests,fundchannel: return the result from the fundchannel rpc call We need this so we can verify the 'close_to' result

view details

niftynei

commit sha 8f253b2adcb46cfc5faf93b8c9c5cbdc49d7ce06

test: simplify test for close_to Since `fundchannel` now supports the 'close_to' argument, we can remove all the logic needed to call fundchannel_start here. Underneath, we're still calling `fundchannel_start`, we're just one (or two, if you count multifundchannel) call levels away from it now.

view details

push time in 5 days

PR merged ElementsProject/lightning

Reviewers
Expose `close_to` up through `multifundchannel` + `fundchannel`

We've had close_to available on fundchannel_start for a while now. This wires it all the way through to general fundchannel users.

+145 -102

2 comments

14 changed files

niftynei

pr closed time in 5 days

pull request commentElementsProject/lightning

Expose `close_to` up through `multifundchannel` + `fundchannel`

Ack https://github.com/ElementsProject/lightning/pull/4132/commits/9650a869ffdd1b932ddfe0124308bd4e5e801267

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

Opener side of the draft dual-funding spec

 static void accepter_got_offer(struct subd *dualopend, 	plugin_hook_call_openchannel2(dualopend->ld, payload); } +static struct command_result *json_open_channel_init(struct command *cmd,+						     const char *buffer,+						     const jsmntok_t *obj UNNEEDED,+						     const jsmntok_t *params)+{+	struct funding_channel *fc = tal(cmd, struct funding_channel);+	struct node_id *id;+	struct peer *peer;+	struct channel *channel;+	bool *announce_channel;+	u32 *feerate_per_kw_funding;+	u32 *feerate_per_kw;+	struct amount_sat *amount, psbt_val;+	struct wally_psbt *psbt;++	u8 *msg = NULL;++	fc->cmd = cmd;+	fc->cancels = tal_arr(fc, struct command *, 0);+	fc->uc = NULL;+	fc->inflight = false;++	if (!param(fc->cmd, buffer, params,+		   p_req("id", param_node_id, &id),+		   p_req("amount", param_sat, &amount),+		   p_req("initialpsbt", param_psbt, &psbt),+		   p_opt("commitment_feerate", param_feerate, &feerate_per_kw),+		   p_opt("funding_feerate", param_feerate, &feerate_per_kw_funding),+		   p_opt_def("announce", param_bool, &announce_channel, true),+		   p_opt("close_to", param_bitcoin_address, &fc->our_upfront_shutdown_script),+		   NULL))+		return command_param_failed();++	psbt_val = AMOUNT_SAT(0);+	for (size_t i = 0; i < psbt->num_inputs; i++) {+		struct amount_sat in_amt = psbt_input_get_amount(psbt, i);+		if (!amount_sat_add(&psbt_val, psbt_val, in_amt))+			return command_fail(cmd, JSONRPC2_INVALID_PARAMS,+					    "Overflow in adding PSBT input values. %s",+					    type_to_string(tmpctx, struct wally_psbt, psbt));+	}++	/* If they don't pass in at least enough in the PSBT to cover+	 * their amount, nope */+	if (!amount_sat_greater(psbt_val, *amount))+		return command_fail(cmd, FUND_CANNOT_AFFORD,+				    "Provided PSBT cannot afford funding of "+				    "amount %s. %s",+				    type_to_string(tmpctx, struct amount_sat, amount),+				    type_to_string(tmpctx, struct wally_psbt, psbt));++	fc->funding = *amount;+	if (!feerate_per_kw) {+		feerate_per_kw = tal(cmd, u32);+		/* Anchors exist, set the commitment feerate to min */+		*feerate_per_kw = feerate_min(cmd->ld, NULL);+	}+	if (!feerate_per_kw_funding) {+		feerate_per_kw_funding = tal(cmd, u32);+		*feerate_per_kw_funding = opening_feerate(cmd->ld->topology);+		if (!*feerate_per_kw_funding)+			return command_fail(cmd, LIGHTNINGD,+					    "`funding_feerate` not specified and fee "+					    "estimation failed");+	}++	if (!topology_synced(cmd->ld->topology)) {+		return command_fail(cmd, FUNDING_STILL_SYNCING_BITCOIN,+				    "Still syncing with bitcoin network");+	}++	peer = peer_by_id(cmd->ld, id);+	if (!peer) {+		return command_fail(cmd, FUNDING_UNKNOWN_PEER, "Unknown peer");+	}++	channel = peer_active_channel(peer);+	if (channel) {+		return command_fail(cmd, LIGHTNINGD, "Peer already %s",+				    channel_state_name(channel));+	}++	if (!peer->uncommitted_channel) {+		return command_fail(cmd, FUNDING_PEER_NOT_CONNECTED,+				    "Peer not connected");+	}++	if (peer->uncommitted_channel->fc) {+		return command_fail(cmd, LIGHTNINGD, "Already funding channel");+	}++#if EXPERIMENTAL_FEATURES+	if (!feature_negotiated(cmd->ld->our_features,+			        peer->their_features,+				OPT_DUAL_FUND)) {+		return command_fail(cmd, FUNDING_V2_NOT_SUPPORTED,+				    "v2 openchannel not supported "+				    "by peer");+	}+#endif /* EXPERIMENTAL_FEATURES */

Don't need this #if, since this entire file is experimental-only.

niftynei

comment created time in 5 days

Pull request review commentElementsProject/lightning

Opener side of the draft dual-funding spec

 struct wally_psbt; struct wally_psbt_input; struct wally_psbt_output; struct wally_map;+#if EXPERIMENTAL_FEATURES+struct witness_element;+#endif /* EXPERIMENTAL_FEATURES */

#if still unnecessary.

niftynei

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentElementsProject/lightning

Opener side of the draft dual-funding spec

 bool psbt_has_required_fields(struct wally_psbt *psbt)  	return true; }++#if EXPERIMENTAL_FEATURES+void psbt_input_set_final_witness_stack(struct wally_psbt_input *in,+					struct witness_element **elements)

const struct witness_element **?

niftynei

comment created time in 5 days

PullRequestReviewEvent

push eventrustyrussell/lightning-rfc

Rusty Russell

commit sha cbfbad352a519f27be5d6a45855366696d478d5c

fixup! recurrence_sig must be a separate field: no point mirroring it into the invoice.

view details

Rusty Russell

commit sha c91878937bcfbf5aa169c874eab40fb4cd314432

fixup! Since we insist that periods be requested in order, we need to distinguish whether first period begins at basetime, or that simply anchors when a period can begin. We still want to start with invoice request counter 0, so we have a separate offset field for this case. The paywindow needs to be more flexible though, and not limited to when we have a base, so make it a separate field. The signature is also changed to a standard signature which covers the entire invoice request rather than a boutique one (why not?). Examples are put in for period calculations, as they are not simple (hint: see mktime() for how to implement this, but make sure you set to UTC first).

view details

push time in 5 days

Pull request review commentElementsProject/lightning

Fuzz testing integration

 const char *fmt_amount_msat_btc(const tal_t *ctx, 				const struct amount_msat *msat, 				bool append_unit) {+	if (msat->millisatoshis == 0)+		return tal_fmt(ctx, append_unit ? "0btc" : "0");

yeah, caller might tal_free it...

darosior

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentElementsProject/lightning

Expose `close_to` up through `multifundchannel` + `fundchannel`

 param_destinations_array(struct command *cmd, const char *name, 			   p_opt_def("announce", param_bool, &announce, true), 			   p_opt_def("push_msat", param_msat, &push_msat, 				     AMOUNT_MSAT(0)),+			   /* FIXME: do address validation here */

Remove FIXME? Or turn into a note that opening will fail if it's not valid anyway...

niftynei

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventElementsProject/lightning

niftynei

commit sha 60be62a3bbc25dc849430baaaed4d61c1b4720b5

openingd: patch test_opening_tiny_channel under EXP_FEAT `test_opening_tiny_channel` fails if EXPERIMENTAL_FEATURES is on because we don't include the anchor in our reserve if we're the channel opener. Seems fine to include in all cases?

view details

push time in 5 days

PR merged ElementsProject/lightning

openingd: patch test_opening_tiny_channel under EXP_FEAT

test_opening_tiny_channel fails if EXPERIMENTAL_FEATURES is on because we don't include the anchor in our reserve if we're the channel opener.

Seems fine to include in all cases?

+0 -1

0 comment

1 changed file

niftynei

pr closed time in 5 days

PullRequestReviewEvent

push eventElementsProject/lightning

grubles

commit sha 94f84d38436a6bbab0b0b3fe17703390761927c9

Update configure OpenBSD's `sha256sum` equivalent is `sha256`. This adds support in `configure` for it.

view details

push time in 6 days

PR merged ElementsProject/lightning

Add sha256 check to configure for OpenBSD

OpenBSD's sha256sum equivalent is sha256. This adds support in configure for it.

+3 -1

1 comment

1 changed file

grubles

pr closed time in 6 days

issue closedElementsProject/lightning

lnd c-lightning incompatibility

When restoring HTLC's, c-lightning will send the HTLC's out of order (i.e. not monotonically increasing) which lnd doesn't expect. Example log output from lnd:

[alice] 2020-08-10 19:21:10.499 [DBG] PEER: Received UpdateAddHTLC(chan_id=cde2d57f1167394d462596f96a776159f92f2b26b3ecfa6bfc39bf5bbabf52cb, id=2938, amt=74788 mSAT, expiry=597, hash=bef6cbdcd234f054169c3e9e1b8ebd50ee8af7d35e70f2c405ade505c8268f19) from 0276086490698c35dd4bb59139f2c4f7fb362e4a28896ad1ab1152fb0c4830a6d6@127.0.0.1:53056
[alice] 2020-08-10 19:21:10.499 [DBG] PEER: Received UpdateAddHTLC(chan_id=cde2d57f1167394d462596f96a776159f92f2b26b3ecfa6bfc39bf5bbabf52cb, id=2914, amt=58650 mSAT, expiry=597, hash=bef6cbdcd234f054169c3e9e1b8ebd50ee8af7d35e70f2c405ade505c8268f19) from 0276086490698c35dd4bb59139f2c4f7fb362e4a28896ad1ab1152fb0c4830a6d6@127.0.0.1:53056
[alice] 2020-08-10 19:21:10.499 [DBG] PEER: Received UpdateAddHTLC(chan_id=cde2d57f1167394d462596f96a776159f92f2b26b3ecfa6bfc39bf5bbabf52cb, id=2801, amt=88584 mSAT, expiry=597, hash=bef6cbdcd234f054169c3e9e1b8ebd50ee8af7d35e70f2c405ade505c8268f19) from 0276086490698c35dd4bb59139f2c4f7fb362e4a28896ad1ab1152fb0c4830a6d6@127.0.0.1:53056
[alice] 2020-08-10 19:21:10.499 [DBG] PEER: Received UpdateAddHTLC(chan_id=cde2d57f1167394d462596f96a776159f92f2b26b3ecfa6bfc39bf5bbabf52cb, id=2943, amt=69839 mSAT, expiry=597, hash=bef6cbdcd234f054169c3e9e1b8ebd50ee8af7d35e70f2c405ade505c8268f19) from 0276086490698c35dd4bb59139f2c4f7fb362e4a28896ad1ab1152fb0c4830a6d6@127.0.0.1:53056
[alice] 2020-08-10 19:21:10.499 [DBG] PEER: Received UpdateAddHTLC(chan_id=cde2d57f1167394d462596f96a776159f92f2b26b3ecfa6bfc39bf5bbabf52cb, id=2959, amt=134633 mSAT, expiry=597, hash=cc2c6ce50b9b50ee4a482f5493e0e8e00dd5c9f470b8e410960dfef1ee668bf7) from 0276086490698c35dd4bb59139f2c4f7fb362e4a28896ad1ab1152fb0c4830a6d6@127.0.0.1:53056
[alice] 2020-08-10 19:21:10.499 [DBG] HSWC: ChannelLink(551:1:0): loaded 0 fwd pks
[alice] 2020-08-10 19:21:10.499 [DBG] PEER: Received UpdateAddHTLC(chan_id=cde2d57f1167394d462596f96a776159f92f2b26b3ecfa6bfc39bf5bbabf52cb, id=2777, amt=81882 mSAT, expiry=597, hash=bef6cbdcd234f054169c3e9e1b8ebd50ee8af7d35e70f2c405ade505c8268f19) from 0276086490698c35dd4bb59139f2c4f7fb362e4a28896ad1ab1152fb0c4830a6d6@127.0.0.1:53056
[alice] 2020-08-10 19:21:10.499 [ERR] HSWC: ChannelLink(551:1:0): failing link: unable to handle upstream add HTLC: ID 2787 on HTLC add does not match expected next ID 2776 with error: invalid update
[alice] 2020-08-10 19:21:10.499 [ERR] HSWC: ChannelLink(551:1:0): link failed, exiting htlcManager
[alice] 2020-08-10 19:21:10.499 [INF] HSWC: ChannelLink(551:1:0): exited

If c-lightning logs are needed, I can send those over. Thanks.

closed time in 7 days

Crypt-iQ

push eventElementsProject/lightning

Rusty Russell

commit sha d5c91a347aa33322373ffd5d6cd1aaf07ba50185

channeld: log broken message if we receive HTLCs out of order. We didn't care, but other implementations (particularly lnd) do. And it does violate the spec. (We need to use skip not xfail on the test which catches this, since xfail doesn't seem to stop errors reported by cleanup) (Includes Christian's typo fix!) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha d04597cbb68ea14755c9ddcffa7519fcbb7cb8f8

pytest: add test for HTLC ordering. [ Includes tidyups by Christian Decker ] Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha a3c30441d31e95e70a5e102d53dd21f45bf13e74

channeld: order htlcs by id before retransmission. We had one report of this, and then Eugene and Roasbeef of Lightning Labs confirmed it; they saw misordered HTLCs on reconnection too. Since we didn't enforce this when we receive HTLCs, we never noticed :( Fixes: #3920 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)

view details

push time in 7 days

PR merged ElementsProject/lightning

Fix HTLCs rexmit out of order lnd-compat

As noted in this morning's spec call, I tracked it down.

+63 -1

1 comment

3 changed files

rustyrussell

pr closed time in 7 days

push eventrustyrussell/lightning

Rusty Russell

commit sha 10a36cf3bbd3551731ec6127507a9c67f57984e7

pytest: add test for HTLC ordering. [ Includes tidyups by Christian Decker ] Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 7f4b11e8cd4bf9289f730c27f42c3715d22b4117

channeld: order htlcs by id before retransmission. We had one report of this, and then Eugene and Roasbeef of Lightning Labs confirmed it; they saw misordered HTLCs on reconnection too. Since we didn't enforce this when we receive HTLCs, we never noticed :( Fixes: #3920 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)

view details

push time in 7 days

push eventElementsProject/lightning

Rusty Russell

commit sha ec868d4acb4f607e8729b7ad404c38100369ba24

lightningd: fix crash when we try to send fail_htlc msg to onchaind. Great report from whitslack on this crash at startup: ``` 2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1) 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d ``` The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN, because onchaind has started but not responded to init yet (which it does once it has analyzed the commitment tx). Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg to channel A's onchaind as if it were channeld. Explicitly check if it's channeld, rather than trying to see if it's onchaind. Fixes: #4114 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: crash: assertion fail at restart when source and destination channels of an HTLC are both onchain.

view details

push time in 7 days

PR merged ElementsProject/lightning

lightningd: fix crash when we try to send fail_htlc msg to onchaind. crash onchaind

Great report from whitslack on this crash at startup:

2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1)
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d

The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN, because onchaind has started but not responded to init yet (which it does once it has analyzed the commitment tx).

Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg to channel A's onchaind as if it were channeld.

Explicitly check if it's channeld, rather than trying to see if it's onchaind.

Fixes: #4114 Signed-off-by: Rusty Russell rusty@rustcorp.com.au

+1 -1

1 comment

1 changed file

rustyrussell

pr closed time in 7 days

issue closedElementsProject/lightning

Assertion failed (FATAL SIGNAL 6) in subd_send_msg

lightningd crashed due to a failed assertion at lightningd/subd.c:750:

https://github.com/ElementsProject/lightning/blob/v0.9.1/lightningd/subd.c#L750-L751

Now I get the same crash whenever I try to restart it. Now I am going to have to hack around the assertion because I can't wait around with my node offline.

2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1)
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:480 (sd_msg_read) 0x559fb67a172c
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x559fb67f96db
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x559fb67f96db
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x559fb67f96db
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:445 (io_loop) 0x559fb67fb673
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x559fb677f5ce
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1015 (main) 0x559fb676ec62
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../csu/libc-start.c:314 (__libc_start_main) 0x7f61cdc93e29
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x559fb676f659
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff

closed time in 7 days

whitslack

issue commentlightningnetwork/lightning-rfc

reply_channel_range is now too strict.

... except this introduces an ambiguity :(

If you split the final queried block, receiver will think that the reply is complete. Then you send another reply...

rustyrussell

comment created time in 7 days

issue openedlightningnetwork/lightning-rfc

reply_channel_range is now too strict.

We insist that block numbers increase, eg. replies should be (first_block=100, num_blocks=10), (first_block=110, num_blocks=10).

This requires implementations to split on block boundaries, and also has the (previously raised but currently purely theoretical) problem that a single block with more than 8k channels would be unsendable without compressed encoding.

Moreover, lnd simply packs in channel ids to the limit, thus it can repeat a block on the edge case, and allows this explicitly:

syncer.go:736

		// If we've previously received a reply for this query, look at
		// its last block to ensure the current reply properly follows
		// it.
		if g.prevReplyChannelRange != nil {
			prevReply := g.prevReplyChannelRange
			prevReplyLastHeight := prevReply.LastBlockHeight()

			// The current reply can either start from the previous
			// reply's last block, if there are still more channels
			// for the same block, or the block after.
			if msg.FirstBlockHeight != prevReplyLastHeight &&
				msg.FirstBlockHeight != prevReplyLastHeight+1 {

				return fmt.Errorf("first block of reply %v "+
					"does not continue from last block of "+
					"previous %v", msg.FirstBlockHeight,
					prevReplyLastHeight)
			}
		}
	}

So I think we should allow this: in particular, c-lightning will probably start doing the same thing, packing up to 8k descriptors into a reply chunk, then compressing. (We currently do a binary split when a block doesn't fit, which is pretty dumb).

created time in 7 days

pull request commentElementsProject/lightning

Fix HTLCs rexmit out of order

OK, wove @cdecker fixes into commits, so self-ack:

Ack https://github.com/ElementsProject/lightning/commit/f417840779ddea4441870dd6efa9916ec0091715

rustyrussell

comment created time in 7 days

push eventrustyrussell/lightning

Rusty Russell

commit sha 306edea7258ee5e170a1a1d2c81ca43724932773

channeld: log broken message if we receive HTLCs out of order. We didn't care, but other implementations (particularly lnd) do. And it does violate the spec. (We need to use skip not xfail on the test which catches this, since xfail doesn't seem to stop errors reported by cleanup) (Includes Christian's typo fix!) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 3d1956a002afd47dc112fc73f43ee9b8be486222

pytest: add test for HTLC ordering. [ Includes tidyups by Christian Decker ] Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha f417840779ddea4441870dd6efa9916ec0091715

channeld: order htlcs by id before retransmission. We had one report of this, and then Eugene and Roasbeef of Lightning Labs confirmed it; they saw misordered HTLCs on reconnection too. Since we didn't enforce this when we receive HTLCs, we never noticed :( Fixes: #3920 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)

view details

push time in 7 days

push eventrustyrussell/lightning

Rusty Russell

commit sha fed7c715ca3da771e8729c6d73994b8281d7e654

Apply suggestions from code review Co-authored-by: Christian Decker <decker.christian@gmail.com>

view details

push time in 7 days

push eventrustyrussell/lightning

Rusty Russell

commit sha 4144fef413fe604d16a6eaeb57ae45716ca1f587

lightningd: fix crash when we try to send fail_htlc msg to onchaind. Great report from whitslack on this crash at startup: ``` 2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1) 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d ``` The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN, because onchaind has started but not responded to init yet (which it does once it has analyzed the commitment tx). Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg to channel A's onchaind as if it were channeld. Explicitly check if it's channeld, rather than trying to see if it's onchaind. Fixes: #4114 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: crash: assertion fail at restart when source and destination channels of an HTLC are both onchain.

view details

push time in 7 days

PR opened ElementsProject/lightning

Reviewers
Fix HTLCs rexmit out of order lnd-compat

As noted in this morning's spec call, I tracked it down.

+63 -1

0 comment

3 changed files

pr created time in 8 days

push eventrustyrussell/lightning

Rusty Russell

commit sha d5243d47134faaf892a423817ed11a91b10bf712

channeld: order htlcs by id before retransmission. We had one report of this, and then Eugene and Roasbeef of Lightning Labs confirmed it; they saw misordered HTLCs on reconnection too. Since we didn't enforce this when we receive HTLCs, we never noticed :( Fixes: #3920 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: Protocol: fixed retransmission order of multiple new HTLCs (causing channel close with LND)

view details

push time in 8 days

PR opened ElementsProject/lightning

Reviewers
pytest: fix overzealous removal of directories on failure. testing

We were always deleting the directories.

Signed-off-by: Rusty Russell rusty@rustcorp.com.au

+1 -1

0 comment

1 changed file

pr created time in 8 days

create barnchrustyrussell/lightning

branch : guilt/htlcs-rexmit-out-of-order

created branch time in 8 days

create barnchrustyrussell/lightning

branch : pytest-dirclean-fix

created branch time in 8 days

PR opened ElementsProject/lightning

lightningd: fix crash when we try to send fail_htlc msg to onchaind. crash onchaind

Great report from whitslack on this crash at startup:

2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1)
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d

The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN, because onchaind has started but not responded to init yet (which it does once it has analyzed the commitment tx).

Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg to channel A's onchaind as if it were channeld.

Explicitly check if it's channeld, rather than trying to see if it's onchaind.

Fixes: #4114 Signed-off-by: Rusty Russell rusty@rustcorp.com.au

+1 -1

0 comment

1 changed file

pr created time in 8 days

push eventrustyrussell/lightning

Rusty Russell

commit sha 1d69f0fa70758c98ebcd3fcc1358a0f3e8545957

lightningd: fix crash when we try to send fail_htlc msg to onchaind. Great report from whitslack on this crash at startup: ``` 2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1) 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d ``` The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN, because onchaind has started but not responded to init yet (which it does once it has analyzed the commitment tx). Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg to channel A's onchaind as if it were channeld. Explicitly check if it's channeld, rather than trying to see if it's onchaind. Fixes: #4114 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 8 days

push eventrustyrussell/lightning

Rusty Russell

commit sha 748aa3a91d0546b1c36d32d8a54d0fb80837e25b

lightningd: fix crash when we try to send fail_htlc msg to onchaind. Great report from whitslack on this crash at startup: ``` 2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1) 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d 2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d ``` The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN, because onchaind has started but not responded to init yet (which it does once it has analyzed the commitment tx). Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg to channel A's onchaind as if it were channeld. Explicitly check if it's channeld, rather than trying to see if it's onchaind. Fixes: #4114 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 8 days

create barnchrustyrussell/lightning

branch : fix-onchaind-crash

created branch time in 8 days

issue commentElementsProject/lightning

Assertion failed (FATAL SIGNAL 6) in subd_send_msg

Great report, thanks!

Your fix is fine. I'll do the proper one now.

whitslack

comment created time in 8 days

pull request commentElementsProject/lightning

Minor cleanups

Ack https://github.com/ElementsProject/lightning/pull/4121/commits/b402e2bad9fc3f8d4512617c2595b4d6fca36b6e

But I've restarted the elements test twice, and it's still failing. Not sure why...

cdecker

comment created time in 8 days

issue commentlightningnetwork/lightning-rfc

commit_sig, revoke_and_ack ordering in channel reestablish flow

Thanks for this!

The problem is that we generally treat "receive sig, send revocation" as an atomic operation. By acknowledging that we've received the sig, but not immediately sending the revocation, we can break this and Badness Happens.

The answer is indeed, to send the revocation first for this case. But this is not the only case

c-lightning indeed remembers what it sent before:

	/* We have to re-send in the same order we sent originally:
	 * revoke_and_ack (usually) alters our next commitment. */
	if (retransmit_revoke_and_ack && !peer->last_was_revoke)
		resend_revoke(peer);

	if (retransmit_commitment_signed)
		resend_commitment(peer, peer->last_sent_commit);

	/* This covers the case where we sent revoke after commit. */
	if (retransmit_revoke_and_ack && peer->last_was_revoke)
		resend_revoke(peer);
Crypt-iQ

comment created time in 8 days

pull request commentlightningnetwork/lightning-rfc

BOLT 4: link to BOLT 1 for tlv_payload format

Well, the 'tlv_stream' vs 'tlv_record' nomenclature came a bit later (the tlvs here stands for tlv_stream`).

We could use tlv_stream: instead of tlvs: everywhere, but then tools/extract-formats.py would need updating.

bitonic-cjp

comment created time in 8 days

PR opened ElementsProject/lightning

Reviewers
pytest: fix flaky test if one side hasn't processed close yet. testing
    @unittest.skipIf(TEST_NETWORK != 'regtest', "External wallet support doesn't work with elements yet.")
    def test_funding_close_upfront(node_factory, bitcoind):
...

        # check that you can provide a closing address upfront
        addr = l1.rpc.newaddr()['bech32']
>       _fundchannel(l1, l2, amt_normal, addr)
...
pyln.client.lightning.RpcError: RPC call failed: method: fundchannel_start, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 100000, 'announce': True, 'close_to': 'bcrt1qctx2k9cu9fd7nk449mzphqjcvvpyc4rxh6826x'}, error: {'code': -1, 'message': 'They sent error channel 2a1ca624cd1127761cb7a4395df2c3fd6d0abb3732c1f85a5345b0da716540d0: Multiple channels unsupported'}

Changelog-None

+9 -0

0 comment

1 changed file

pr created time in 9 days

create barnchrustyrussell/lightning

branch : fix-flaky-test-3

created branch time in 9 days

push eventrustyrussell/lightning

Rusty Russell

commit sha f6130685272a5534382d02bc358bc86d89161032

gossip_store: make private channels more similar to channel_announcement Instead of a boutique message, use a "real" channel_announcement for private channels (with fake sigs and pubkeys). This makes it far easier for gossmap to handle local channels. Backwards compatible update, since we update old stores. We also fix devtools/dump-gossipstore to know about the tombstone markers. Since we increment our channel_announce count for local channels now, the stats in the tests changed too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 6c4e1e0b64588135a281559ab0487c4cde4a710a

common/gossmap: digest private channel information too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 7e4b2456ee1f8dd8dae4c2f724e25d5081853fb0

plugins/pay: fix leak on failed new payments. Start with attaching the payment to cmd (in case of failure), then steal onto the plugin itself. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 7321ca8f527db9b4d50dbd91ccbd6b29ff3ba95b

libplugin-pay: incorporate gossip store. So we can use this for routing determinations. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha a0ff1efa43fc06ba26f60248a0249893686972d8

plugins/libplugin-pay: use gossmap. This is a fairly direct translation. Even so, it should be faster in most cases, and and we can do more sophisticated things if we want. This also handles disabled channels better. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Changed: plugins: `pay` will now try disabled channels as a last resort.

view details

push time in 9 days

push eventrustyrussell/lightning

Rusty Russell

commit sha d8e07734df463a9a669193c22312be9c83466cbd

lightningd: clean up close code now force is always true. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha de7eac042bdf0f205dcdb9f0745ceac6ccc596cb

common: add routines for log level names. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 783b9da6ae0006be808e046329ebd29a29c4167e

ccan: update to latest version, get json_out_finished update. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 3be837f0ed7e68f905ca452779818b8274775c5a

common/json_stream: add generic double-cr helper. And make caller of json_stream_forward_change_id use it, since we're going to reuse that. Also call json_out_finished here, so next object doesn't have a "," prepended. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 25c8356142646ac9b80d71a7b95eaddd4d10fc1f

JSON-RPC: notifications command. This lets callers enable notifications; we won't send any if they don't. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Added: JSON-RPC: `notifications` command to enable notifications.

view details

Rusty Russell

commit sha 223e0a2bbe52b597f2a82f351253b1235559d4f0

lightningd: forward notifications from plugins if enabled. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 28aad3143b9fef00e5fce5a87424c78f4fbb494f

libplugin: support for sending notifications. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Added: libplugin: routines to send notification updates and progress.

view details

Rusty Russell

commit sha 1e27ed976b6ef8648462029283e2dd3af4271372

libplugin: ignore incoming notifications. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 1b796d9cd108a6c2b5e29302aed50b9a1a355ac1

pyln: handle (ignore) notifications, and add notify_msg to send them. We also sanity check that response id matches our request. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Added: pyln: pyln.client handles and can send progress notifications.

view details

Rusty Russell

commit sha 70dbc47bda0574171580e666837ebc176f298949

lightning-cli: print notifications (with '# ' prefix) if received. This can be suppressed with -N. Note that we wull get an error with older lightningd, but we ignore it anyway. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Added: cli: print notifications and progress bars if commands provide them.

view details

Rusty Russell

commit sha ee55c8cd227334e51c5edbe2b75412d02add9cef

pytest: add notifications to tests. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 3a3f0156ca7ebc59bbfee76d1b933f5ceaa725fd

lightningd: infrastructure for internal notifications. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha a0671737461c7deefdd2eddd03cf38f3e916daa2

close: add notification for slow closes. For compatibility, we only do this if `allow-deprecated-apis` is false for now. Otherwise scripts parsing should use `grep -v '^# '` or start using `-N none`. Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false) Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon Fixes: #3925 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 9 days

pull request commentElementsProject/lightning

Pay: use gossmap

Ya trivial rebase.

rustyrussell

comment created time in 9 days

push eventrustyrussell/lightning

Christian Decker

commit sha 05934724dd556de3fddcf7c91c91bae5d45fa10f

pytest: Don't start 3 nodes in test_gossip_ratelimit We really are just interested in their on-chain footprint, so actually starting the nodes is pointless overhead, and caused a lot of flakyness due to the output ordering sometimes not matching up. We now just use the `bitcoind` API to fund, sign and send a raw transaction that matches the stashed gossip messages.

view details

Christian Decker

commit sha 1563bbc2fa5aac542ed280c9ee67be6a365b4bd1

pytest: Cleanup test_gossip_ratelimit Drive-by code cleanups I stumbled over while investigating the issue fixed in the previous commit.

view details

Christian Decker

commit sha 8d8b8077938deff865105b7bb8ad2488a55607ee

pytest: Make LightningNode.fund_channel more resilient It was really flaky, especially under `test_mpp_interference_2`, most likely due to multiple calls to `fund_channel`. This commit looks for the specific txids in the `listfunds` output and the `getrawmempool` output, avoiding strange artifacts from multiple calls.

view details

Christian Decker

commit sha 66dc3ed6651766fd1a69892e9ef0a99c7202147f

pyln: Add pytest to type ignores Reported-by: Rusty Russell <@rustyrussell>

view details

Christian Decker

commit sha b6053d7cd669dda1b274ec3175a47161eaa7556c

Fix pytest major version They released version 6, which is causing some trouble apparently. So fix it to v5.

view details

Christian Decker

commit sha 0de7a289cc1636ac3b17edb876329a80c2348a3c

travis: Add blinker dependency This is a new, not explicitly mentioned, dependency for pytest-sentry

view details

Christian Decker

commit sha 81569de709ce815aea4afe9701536f40737c1cc7

pytest: Stabilize test_setchannelfee_{restart,zero} In both cases the flakyness arises from the destination not knowing about the modified fees of the forwarding node, thus including the outdated details in the routehint, and the sender being unlucky and always trying with the routehint anyway. The long-term solutions to this is going to be #4111, this commit just reduces the flakyness to get back to business.

view details

Christian Decker

commit sha 9021bb26d1ed587c96fe2256b26b8c9f21c64745

pyln: Decode process output once before storing it

view details

Christian Decker

commit sha ee0ee046ff62788d92496047f64629a2a0804950

pytest: Stabilize test_penalty_htlc_tx_timeout lightning-5 can sometimes see itself sweeping the unilateral output resulting in this weird line: ```log HTLC already resolved by SELF when we found preimage ```

view details

Christian Decker

commit sha 6f870dfe399461fc556da4a8dea29eeef62b91ad

pytest: Don't give up on the first psql connection error Since we start a new instance of postgres for each test we may end up swamped and the startup can take a bit longer. So let's loop until we get a success.

view details

Rusty Russell

commit sha 2d897be10a45760c501cd5b58d84741cfd0e0980

pytest: fix experimental test. Broken back in 3b8c0a7397cc0694117dfe63ecb2bff0b6957a4b which changed the behavior of funding not to mark as immediately spent. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

fiatjaf

commit sha 9c838cf95388d0d083c0cc7bd60d25a8678ba281

pyln-client: listpayments -> listpays

view details

Rusty Russell

commit sha 711133e3c7c232095deae428c95cb1e49ffa05ef

pytest: fix flaky test. l2 only gives up 100 blocks after it ignores the HTLC, so sometimes this test fails. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Michael Schmoock

commit sha 011590b20ef57a8bd17cfa6c77063fa4841a0590

fix: broken SQL statement in wallet db_set_utxo I discovered this accidentally when using the `tests/plugins/dblog.py` plugin on another testcase: tests/test_connection.py::test_fail_unconfirmed There the plugin/hook crashes because it can't execute the statement: ```json { "jsonrpc": "2.0", "id": 34, "error": { "code": -32600, "message": "Error while processing db_write: unrecognized token: \"174WHERE\"", "traceback": "Traceback (most recent call last):\n File \"/home/will/projects/lightning.git/contrib/pyln-client/pyln/client/plugin.py\", line 535, in _dispatch_request\n result = self._exec_func(method.func, request)\n File \"/home/will/projects/lightning.git/contrib/pyln-client/pyln/client/plugin.py\", line 520, in _exec_func\n return func(*ba.args, **ba.kwargs)\n File \"/home/will/projects/lightning.git/tests/plugins/dblog.py\", line 45, in db_write\n plugin.conn.execute(c)\nsqlite3.OperationalError: unrecognized token: \"174WHERE\"\n" } } ``` Changelog-Fixed: plugin: Regression with SQL statement expansion that could result in invalid statements being passed to the `db_write` hook.

view details

Christian Decker

commit sha 7d6892abba946497151e20a4c4b9f82d8ac33b42

pytest: Verification mode of expanded stmts for the db_write hook This can be used to find instances of unparseable expanded SQL statements like the one fixed in #4090

view details

Christian Decker

commit sha 77ca07e91da75285c3e168024f3c4c183a3f83d6

db: Fix statement expansion bugs found through dblog mode

view details

Christian Decker

commit sha ad049c36c119d1e283ac270d73b60d811c509c59

travis: Enable TEST_CHECK_DBSTMTS=1 mode for one of the configs

view details

Rusty Russell

commit sha a6aecb9d1feb3449bcf02556e4f8b68da2bb9492

dijkstra: fix heap ordering. We were always ordering heap by distance, not score (which are different if we are routing by cheapest, not shortest!). This simplifies our callbacks, too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 5e97bcab11514013d2d0703d0c530a9f521433a0

route: return NULL if destination is unreachable. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

Rusty Russell

commit sha 76b529b38d041623d2ff2c660b0f73cfcb1935da

gossip_store: don't copy old delete markers on startup compact. So we don't have to handle them at load time, either. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 9 days

push eventrustyrussell/lightning

Rusty Russell

commit sha 19989f0c021d294bb3b8217bf0cfb89f4585e57d

close: add notification for slow closes. For compatibility, we only do this if `allow-deprecated-apis` is false for now. Otherwise scripts parsing should use `grep -v '^# '` or start using `-N none`. Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false) Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon Fixes: #3925 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

view details

push time in 9 days

more