lightningd: migrate events from bookkeeper at startup.
What changed, and why it matters
This commit moves the bookkeeper accounting data from a separate plugin database into the main lightningd database at startup. It adds a one-time migration that reads the old accounts.sqlite3 file and copies its records into new internal tables. The change is mostly a refactor of where data lives, but it touches startup code, database migrations, and option handling. There is no direct evidence in the commit that this fixes an active security vulnerability; it appears to be architectural cleanup.
Treat this as a regular code-review item rather than an urgent security patch. Reviewers should verify that the migration cannot be triggered or abused via crafted legacy database files or option paths, and that the temporary chdir and external DB open do not introduce path-traversal or symlink issues. The FIXME stub for migrate_setup_coinmoves should be completed and tested before release. No immediate user action is required unless the project flags this commit as security-relevant.
Security signals we found
Database migration code runs at startup with full node privileges
Migration opens and reads an external/legacy database file path controlled by user options
Migration abort()s on unexpected tag strings or amount conversion failures
Migration calls fatal() if the legacy database version is not exactly 17
New code chdirs into the legacy bookkeeper directory during migration
User descriptions from the legacy DB are copied into the datastore without explicit length/sanitization checks visible in the diff
No explicit input validation on legacy DB contents beyond column type binding
Evidence from the diff
Core Lightning is migrating the bookkeeper plugin’s accounting records into the main wallet DB. The commit introduces wallet/account_migration.c, which opens the legacy accounts.sqlite3 (or a user-supplied DSN), reads chain_events and channel_events, and inserts them into the new chain_moves and channel_moves tables. It also preserves user descriptions in the datastore. The –bookkeeper-dir and –bookkeeper-db options are now consumed by lightningd itself (hidden) instead of the plugin. Several internal helpers (wait_index_increment, db_bind_mvt_account_id, etc.) were refactored to accept an explicit struct db* so the migration can operate before the wallet is fully initialized. A stub migrate_setup_coinmoves() is left as FIXME for nodes that never had a bookkeeper DB.
Changed components
lightningd startup and option handlingwallet database layer (wallet/db.c, wallet/wallet.c)bookkeeper plugin (plugins/bkpr/bookkeeper.c)wait-index subsystem (lightningd/wait.c)new wallet/account_migration.c migration modulechain_moves and channel_moves tablesdatastore (utxo/payment descriptions)Inspect captured patch +710 / −98
diff --git a/common/coin_mvt.c b/common/coin_mvt.c
index fd89444a..a31a8f3e 100644
--- a/common/coin_mvt.c
+++ b/common/coin_mvt.c
@@ -155,7 +155,7 @@ const char *mvt_tag_str(enum mvt_tag tag)
return mvt_tags[tag];
}
-static void tag_set(struct mvt_tags *tags, enum mvt_tag tag)
+void mvt_tag_set(struct mvt_tags *tags, enum mvt_tag tag)
{
u64 bitnum = mvt_tag_in_db(tag);
assert(bitnum < NUM_MVT_TAGS);
@@ -394,10 +394,10 @@ struct chain_coin_mvt *new_coin_channel_open_proposed(const tal_t *ctx,
/* If we're the opener, add to the tag list */
if (is_opener)
- tag_set(&tags, MVT_OPENER);
+ mvt_tag_set(&tags, MVT_OPENER);
if (is_leased)
- tag_set(&tags, MVT_LEASED);
+ mvt_tag_set(&tags, MVT_LEASED);
mvt = new_chain_coin_mvt(ctx, channel, NULL, time_now().ts.tv_sec,
NULL, out, NULL, 0,
@@ -423,10 +423,10 @@ struct chain_coin_mvt *new_coin_channel_open(const tal_t *ctx,
/* If we're the opener, add to the tag list */
if (is_opener)
- tag_set(&tags, MVT_OPENER);
+ mvt_tag_set(&tags, MVT_OPENER);
if (is_leased)
- tag_set(&tags, MVT_LEASED);
+ mvt_tag_set(&tags, MVT_LEASED);
mvt = new_chain_coin_mvt(ctx, channel, NULL, time_now().ts.tv_sec,
NULL, out, NULL, blockheight,
@@ -684,10 +684,10 @@ struct mvt_tags mk_mvt_tags_(enum mvt_tag tag, ...)
va_list ap;
struct mvt_tags ret = { 0 };
- tag_set(&ret, tag);
+ mvt_tag_set(&ret, tag);
va_start(ap, tag);
while ((tag = va_arg(ap, enum mvt_tag)) != 999)
- tag_set(&ret, mvt_tag_in_db(tag));
+ mvt_tag_set(&ret, mvt_tag_in_db(tag));
va_end(ap);
return ret;
}
diff --git a/common/coin_mvt.h b/common/coin_mvt.h
index dd16ca40..feef4883 100644
--- a/common/coin_mvt.h
+++ b/common/coin_mvt.h
@@ -130,6 +130,9 @@ static inline struct mvt_tags tag_to_mvt_tags(enum mvt_tag tag)
return tags;
}
+/* Add a tag */
+void mvt_tag_set(struct mvt_tags *tags, enum mvt_tag tag);
+
/* Extract the primary tag */
enum mvt_tag primary_mvt_tag(struct mvt_tags tags);
diff --git a/doc/lightningd-config.5.md b/doc/lightningd-config.5.md
index b57e9cb1..7a6d9fcd 100644
--- a/doc/lightningd-config.5.md
+++ b/doc/lightningd-config.5.md
@@ -293,18 +293,6 @@ authenticate with username `user` and password `pass`, and then use the
database `db_name`. The database must exist, but the schema will be managed
automatically by `lightningd`.
-* **bookkeeper-dir**=*DIR* [plugin `bookkeeper`]
-
- Directory to keep the accounts.sqlite3 database file in.
-Defaults to lightning-dir.
-
-* **bookkeeper-db**=*DSN* [plugin `bookkeeper`]
-
- Identify the location of the bookkeeper data. This is a fully qualified data source
-name, including a scheme such as `sqlite3` or `postgres` followed by the
-connection parameters.
-Defaults to `sqlite3://accounts.sqlite3` in the `bookkeeper-dir`.
-
* **encrypted-hsm**
If set, you will be prompted to enter a password used to encrypt the `hsm_secret`.
diff --git a/lightningd/coin_mvts.c b/lightningd/coin_mvts.c
index 3ceac59e..cc206b62 100644
--- a/lightningd/coin_mvts.c
+++ b/lightningd/coin_mvts.c
@@ -258,13 +258,14 @@ void json_add_channel_mvt_fields(struct json_stream *stream,
}
static u64 coinmvt_index_inc(struct lightningd *ld,
+ struct db *db,
enum wait_subsystem subsys,
const struct mvt_account_id *account,
struct amount_msat credit,
struct amount_msat debit,
enum wait_index idx)
{
- return wait_index_increment(ld, subsys, idx,
+ return wait_index_increment(ld, db, subsys, idx,
"account", account->channel ? fmt_channel_id(tmpctx, &account->channel->cid) : account->alt_account,
"=credit_msat", tal_fmt(tmpctx, "%"PRIu64, credit.millisatoshis), /* Raw: JSON output */
"=debit_msat", tal_fmt(tmpctx, "%"PRIu64, debit.millisatoshis), /* Raw: JSON output */
@@ -272,21 +273,23 @@ static u64 coinmvt_index_inc(struct lightningd *ld,
}
u64 chain_mvt_index_created(struct lightningd *ld,
+ struct db *db,
const struct mvt_account_id *account,
struct amount_msat credit,
struct amount_msat debit)
{
- return coinmvt_index_inc(ld, WAIT_SUBSYSTEM_CHAINMOVES,
+ return coinmvt_index_inc(ld, db, WAIT_SUBSYSTEM_CHAINMOVES,
account, credit, debit,
WAIT_INDEX_CREATED);
}
u64 channel_mvt_index_created(struct lightningd *ld,
+ struct db *db,
const struct mvt_account_id *account,
struct amount_msat credit,
struct amount_msat debit)
{
- return coinmvt_index_inc(ld, WAIT_SUBSYSTEM_CHANNELMOVES,
+ return coinmvt_index_inc(ld, db, WAIT_SUBSYSTEM_CHANNELMOVES,
account, credit, debit,
WAIT_INDEX_CREATED);
}
diff --git a/lightningd/coin_mvts.h b/lightningd/coin_mvts.h
index 33d9701e..11f4fd47 100644
--- a/lightningd/coin_mvts.h
+++ b/lightningd/coin_mvts.h
@@ -4,6 +4,9 @@
#include <common/coin_mvt.h>
+struct htlc_in;
+struct htlc_out;
+struct json_stream;
struct lightningd;
struct account_balance {
@@ -54,11 +57,13 @@ void json_add_channel_mvt_fields(struct json_stream *stream,
/* For db code to get incremental ids */
u64 chain_mvt_index_created(struct lightningd *ld,
+ struct db *db,
const struct mvt_account_id *account,
struct amount_msat credit,
struct amount_msat debit);
u64 channel_mvt_index_created(struct lightningd *ld,
+ struct db *db,
const struct mvt_account_id *account,
struct amount_msat credit,
struct amount_msat debit);
diff --git a/lightningd/forwards.c b/lightningd/forwards.c
index 2c001e59..ecb14094 100644
--- a/lightningd/forwards.c
+++ b/lightningd/forwards.c
@@ -19,7 +19,8 @@ static u64 forward_index_inc(struct lightningd *ld,
const struct short_channel_id *out_channel,
enum wait_index idx)
{
- return wait_index_increment(ld, WAIT_SUBSYSTEM_FORWARD, idx,
+ return wait_index_increment(ld, ld->wallet->db,
+ WAIT_SUBSYSTEM_FORWARD, idx,
"status", forward_status_name(status),
"in_channel", fmt_short_channel_id(tmpctx, in_channel),
"=in_htlc_id", tal_fmt(tmpctx, "%"PRIu64, in_htlc_id),
diff --git a/lightningd/lightningd.c b/lightningd/lightningd.c
index 452cee38..1c3fb7e4 100644
--- a/lightningd/lightningd.c
+++ b/lightningd/lightningd.c
@@ -271,6 +271,8 @@ static struct lightningd *new_lightningd(const tal_t *ctx)
ld->try_reexec = false;
ld->recover_secret = NULL;
ld->db_upgrade_ok = NULL;
+ ld->old_bookkeeper_dir = NULL;
+ ld->old_bookkeeper_db = NULL;
/* --invoices-onchain-fallback */
ld->unified_invoices = false;
diff --git a/lightningd/lightningd.h b/lightningd/lightningd.h
index 306b8c29..ce8b62b0 100644
--- a/lightningd/lightningd.h
+++ b/lightningd/lightningd.h
@@ -383,6 +383,10 @@ struct lightningd {
char *wallet_dsn;
+ /* For migration from old accounts.db */
+ char *old_bookkeeper_dir;
+ char *old_bookkeeper_db;
+
bool encrypted_hsm;
/* What (additional) messages the HSM accepts */
u32 *hsm_capabilities;
diff --git a/lightningd/options.c b/lightningd/options.c
index 252fcb54..d746c047 100644
--- a/lightningd/options.c
+++ b/lightningd/options.c
@@ -1603,6 +1603,16 @@ static void register_opts(struct lightningd *ld)
"Re-enable a long-deprecated API (which will be removed entirely next version!)");
opt_register_logging(ld);
+ /* Old bookkeeper migration flags. */
+ opt_register_early_arg("--bookkeeper-dir",
+ opt_set_talstr, NULL,
+ &ld->old_bookkeeper_dir,
+ opt_hidden);
+ opt_register_early_arg("--bookkeeper-db",
+ opt_set_talstr, NULL,
+ &ld->old_bookkeeper_db,
+ opt_hidden);
+
dev_register_opts(ld);
}
diff --git a/lightningd/pay.c b/lightningd/pay.c
index f7a26781..44931d70 100644
--- a/lightningd/pay.c
+++ b/lightningd/pay.c
@@ -2118,7 +2118,8 @@ static u64 sendpay_index_inc(struct lightningd *ld,
enum payment_status status,
enum wait_index idx)
{
- return wait_index_increment(ld, WAIT_SUBSYSTEM_SENDPAY, idx,
+ return wait_index_increment(ld, ld->wallet->db,
+ WAIT_SUBSYSTEM_SENDPAY, idx,
"status", payment_status_to_string(status),
"=partid", tal_fmt(tmpctx, "%"PRIu64, partid),
"=groupid", tal_fmt(tmpctx, "%"PRIu64, groupid),
diff --git a/lightningd/peer_htlcs.c b/lightningd/peer_htlcs.c
index 23519012..454eb8dc 100644
--- a/lightningd/peer_htlcs.c
+++ b/lightningd/peer_htlcs.c
@@ -3043,7 +3043,8 @@ static u64 htlcs_index_inc(struct lightningd *ld,
enum htlc_state hstate,
enum wait_index idx)
{
- return wait_index_increment(ld, WAIT_SUBSYSTEM_HTLCS, idx,
+ return wait_index_increment(ld, ld->wallet->db,
+ WAIT_SUBSYSTEM_HTLCS, idx,
"state", htlc_state_name(hstate),
"short_channel_id", fmt_short_channel_id(tmpctx, channel_scid_or_local_alias(channel)),
"direction", owner == LOCAL ? "out": "in",
diff --git a/lightningd/wait.c b/lightningd/wait.c
index ed3c0972..4d9066f6 100644
--- a/lightningd/wait.c
+++ b/lightningd/wait.c
@@ -128,6 +128,7 @@ static void json_add_index(struct command *cmd,
}
static u64 wait_index_bump(struct lightningd *ld,
+ struct db *db,
enum wait_subsystem subsystem,
enum wait_index index,
u64 num,
@@ -142,7 +143,7 @@ static u64 wait_index_bump(struct lightningd *ld,
/* FIXME: We can optimize this! It's always the max of the fields in
* the table, *unless* we delete one. So we can lazily write this on
* delete, and fix it up to MAX() when we startup. */
- db_set_intvar(ld->wallet->db,
+ db_set_intvar(db,
tal_fmt(tmpctx, "last_%s_%s_index",
wait_subsystem_name(subsystem),
wait_index_name(index)),
@@ -172,6 +173,7 @@ static u64 wait_index_bump(struct lightningd *ld,
}
u64 wait_index_increment(struct lightningd *ld,
+ struct db *db,
enum wait_subsystem subsystem,
enum wait_index index,
...)
@@ -180,7 +182,7 @@ u64 wait_index_increment(struct lightningd *ld,
u64 ret;
va_start(ap, index);
- ret = wait_index_bump(ld, subsystem, index, 1, ap);
+ ret = wait_index_bump(ld, db, subsystem, index, 1, ap);
va_end(ap);
return ret;
@@ -198,7 +200,7 @@ void wait_index_increase(struct lightningd *ld,
return;
va_start(ap, num);
- wait_index_bump(ld, subsystem, index, num, ap);
+ wait_index_bump(ld, ld->wallet->db, subsystem, index, num, ap);
va_end(ap);
}
diff --git a/lightningd/wait.h b/lightningd/wait.h
index 6e0a08ab..d60abfe2 100644
--- a/lightningd/wait.h
+++ b/lightningd/wait.h
@@ -37,6 +37,7 @@ const char *wait_subsystem_name(enum wait_subsystem subsystem);
/**
* wait_index_increment - increment an index, tell waiters.
* @ld: the lightningd
+ * @db: the database (usually ld->wallet->db, except really early)
* @subsystem: subsystem for index
* @index: which index
* ...: name/value pairs, followed by NULL.
@@ -48,6 +49,7 @@ const char *wait_subsystem_name(enum wait_subsystem subsystem);
* Returns the updated index value (always > 0).
*/
u64 LAST_ARG_NULL wait_index_increment(struct lightningd *ld,
+ struct db *db,
enum wait_subsystem subsystem,
enum wait_index index,
...);
diff --git a/plugins/bkpr/bookkeeper.c b/plugins/bkpr/bookkeeper.c
index e150b7a7..1312924d 100644
--- a/plugins/bkpr/bookkeeper.c
+++ b/plugins/bkpr/bookkeeper.c
@@ -1513,27 +1513,6 @@ static const char *init(struct command *init_cmd, const char *b, const jsmntok_t
struct bkpr *bkpr = bkpr_of(p);
be64 index;
- /* Options processing makes NULL the owner of options. Steal them */
- tal_steal(bkpr, bkpr->datadir);
- tal_steal(bkpr, bkpr->db_dsn);
-
- /* Switch to bookkeeper-dir, if specified */
- if (bkpr->datadir && chdir(bkpr->datadir) != 0) {
- if (mkdir(bkpr->datadir, 0700) != 0 && errno != EEXIST)
- plugin_err(p,
- "Unable to create 'bookkeeper-dir'=%s",
- bkpr->datadir);
- if (chdir(bkpr->datadir) != 0)
- plugin_err(p,
- "Unable to switch to 'bookkeeper-dir'=%s",
- bkpr->datadir);
- }
-
- /* No user suppled db_dsn, set one up here */
- if (!bkpr->db_dsn)
- bkpr->db_dsn = tal_fmt(bkpr, "sqlite3://accounts.sqlite3");
-
- plugin_log(p, LOG_DBG, "Setting up database at %s", bkpr->db_dsn);
bkpr->accounts = init_accounts(bkpr, init_cmd);
bkpr->onchain_fees = init_onchain_fees(bkpr, init_cmd);
bkpr->descriptions = init_descriptions(bkpr, init_cmd);
@@ -1565,22 +1544,11 @@ int main(int argc, char *argv[])
/* No datadir is default */
bkpr = tal(NULL, struct bkpr);
- bkpr->datadir = NULL;
- bkpr->db_dsn = NULL;
-
plugin_main(argv, init, take(bkpr), PLUGIN_STATIC, true, NULL,
commands, ARRAY_SIZE(commands),
notifs, ARRAY_SIZE(notifs),
NULL, 0,
NULL, 0,
- plugin_option("bookkeeper-dir",
- "string",
- "Location for bookkeeper records.",
- charp_option, NULL, &bkpr->datadir),
- plugin_option("bookkeeper-db",
- "string",
- "Location of the bookkeeper database",
- charp_option, NULL, &bkpr->db_dsn),
NULL);
return 0;
diff --git a/plugins/bkpr/bookkeeper.h b/plugins/bkpr/bookkeeper.h
index 8c5ceba5..5358f2cd 100644
--- a/plugins/bkpr/bookkeeper.h
+++ b/plugins/bkpr/bookkeeper.h
@@ -16,9 +16,6 @@ struct bkpr {
/* Where we're up to in listchainmoves, listchannelmoves */
u64 chainmoves_index, channelmoves_index;
-
- char *db_dsn;
- char *datadir;
};
/* Helper to ignore returns from datastore */
diff --git a/tests/test_bookkeeper.py b/tests/test_bookkeeper.py
index 22c3a696..30b004e9 100644
--- a/tests/test_bookkeeper.py
+++ b/tests/test_bookkeeper.py
@@ -916,8 +916,6 @@ def test_bookkeeper_custom_notifs(node_factory, chainparams):
assert acct_fee == Millisatoshi(fee)
-# FIXME: Restore once bookkeeper migrating to core.
-@pytest.mark.xfail(strict=True)
@unittest.skipIf(TEST_NETWORK != 'regtest', "Snapshots are bitcoin regtest.")
@unittest.skipIf(os.getenv('TEST_DB_PROVIDER', 'sqlite3') != 'sqlite3', "uses snapshots")
def test_migration(node_factory, bitcoind):
@@ -1045,24 +1043,24 @@ def test_migration(node_factory, bitcoind):
'payment_id': '7ccef7e9fabbf4a841af44b1fc7319bc70ce98697b77ce6dacffa84bebcd4350',
'tag': 'invoice',
'type': 'channel'},
- {'account': 'wallet',
- 'credit_msat': 1004927000,
+ {'account': 'be7f3755c04abec58212fe9287898c76364d1a0d12a1828bf9fc3ac4a8b25a67',
+ 'credit_msat': 4927000,
'currency': 'bcrt',
'debit_msat': 0,
'tag': 'onchain_fee',
'txid': '675ab2a8c43afcf98b82a1120d1a4d36768c898792fe1282c5be4ac055377fbe',
'type': 'onchain_fee'},
{'account': 'wallet',
- 'credit_msat': 0,
+ 'credit_msat': 1004927000,
'currency': 'bcrt',
- 'debit_msat': 1004927000,
+ 'debit_msat': 0,
'tag': 'onchain_fee',
'txid': '675ab2a8c43afcf98b82a1120d1a4d36768c898792fe1282c5be4ac055377fbe',
'type': 'onchain_fee'},
- {'account': 'be7f3755c04abec58212fe9287898c76364d1a0d12a1828bf9fc3ac4a8b25a67',
- 'credit_msat': 4927000,
+ {'account': 'wallet',
+ 'credit_msat': 0,
'currency': 'bcrt',
- 'debit_msat': 0,
+ 'debit_msat': 1004927000,
'tag': 'onchain_fee',
'txid': '675ab2a8c43afcf98b82a1120d1a4d36768c898792fe1282c5be4ac055377fbe',
'type': 'onchain_fee'}]
diff --git a/wallet/Makefile b/wallet/Makefile
index c3707dde..b79ce3c6 100644
--- a/wallet/Makefile
+++ b/wallet/Makefile
@@ -1,6 +1,7 @@
#! /usr/bin/make
WALLET_LIB_SRC := \
+ wallet/account_migration.c \
wallet/db.c \
wallet/invoices.c \
wallet/psbt_fixup.c \
@@ -30,6 +31,7 @@ ALL_C_HEADERS += $(WALLET_HDRS)
# The following files contain SQL-annotated statements that we need to extact
WALLET_SQL_FILES := \
$(DB_SQL_FILES) \
+ wallet/account_migration.c \
wallet/db.c \
wallet/invoices.c \
wallet/wallet.c \
diff --git a/wallet/account_migration.c b/wallet/account_migration.c
new file mode 100644
index 00000000..3e4807de
--- /dev/null
+++ b/wallet/account_migration.c
@@ -0,0 +1,537 @@
+/* All this code is to read the old accounts.db file from bookkeeper
+ * and copy the moves table */
+#include "config.h"
+#include <ccan/tal/path/path.h>
+#include <ccan/tal/str/str.h>
+#include <db/bindings.h>
+#include <db/common.h>
+#include <db/exec.h>
+#include <db/utils.h>
+#include <lightningd/coin_mvts.h>
+#include <lightningd/lightningd.h>
+#include <unistd.h>
+#include <wallet/account_migration.h>
+#include <wallet/wallet.h>
+
+/* These functions and definitions copied almost exactly from old
+ * plugins/bkpr/{recorder.c,chain_event.h,channel_event.h}
+ */
+struct chain_event {
+
+ /* Id of this chain event in the database */
+ u64 db_id;
+
+ /* db_id of account this event belongs to */
+ u64 acct_db_id;
+
+ /* Name of the account this belongs to */
+ char *acct_name;
+
+ /* Name of account this originated from */
+ char *origin_acct;
+
+ /* Tag describing the event */
+ const char *tag;
+
+ /* Is the node's wallet ignoring this? */
+ bool ignored;
+
+ /* Is this chain output stealable? If so
+ * we'll need to watch it for longer */
+ bool stealable;
+
+ /* Is this chain event because of a splice
+ * confirmation? */
+ bool splice_close;
+
+ /* Is this a rebalance event? */
+ bool rebalance;
+
+ /* Amount we received in this event */
+ struct amount_msat credit;
+
+ /* Amount we paid in this event */
+ struct amount_msat debit;
+
+ /* Total 'amount' of output on this chain event */
+ struct amount_msat output_value;
+
+ /* What token are the credit/debits? */
+ const char *currency;
+
+ /* What time did the event happen */
+ u64 timestamp;
+
+ /* What block did the event happen */
+ u32 blockheight;
+
+ /* What txo did this event concern */
+ struct bitcoin_outpoint outpoint;
+
+ /* What tx was the outpoint spent in (if spent) */
+ struct bitcoin_txid *spending_txid;
+
+ /* Sometimes chain events resolve payments */
+ struct sha256 *payment_id;
+
+ /* Desc of event (maybe useful for printing notes) */
+ const char *desc;
+
+ /* Added: close_count */
+ u32 output_count;
+
+ /* Added: peer_id */
+ struct node_id *peer_id;
+
+ /* Added: did we open this account? */
+ bool we_opened;
+};
+
+static struct chain_event *stmt2chain_event(const tal_t *ctx, struct db_stmt *stmt)
+{
+ struct chain_event *e = tal(ctx, struct chain_event);
+ e->db_id = db_col_u64(stmt, "e.id");
+ e->acct_db_id = db_col_u64(stmt, "e.account_id");
+ e->acct_name = db_col_strdup(e, stmt, "a.name");
+
+ if (!db_col_is_null(stmt, "e.origin"))
+ e->origin_acct = db_col_strdup(e, stmt, "e.origin");
+ else
+ e->origin_acct = NULL;
+
+ e->tag = db_col_strdup(e, stmt, "e.tag");
+
+ e->credit = db_col_amount_msat(stmt, "e.credit");
+ e->debit = db_col_amount_msat(stmt, "e.debit");
+ e->output_value = db_col_amount_msat(stmt, "e.output_value");
+
+ e->currency = db_col_strdup(e, stmt, "e.currency");
+ e->timestamp = db_col_u64(stmt, "e.timestamp");
+ e->blockheight = db_col_int(stmt, "e.blockheight");
+
+ db_col_txid(stmt, "e.utxo_txid", &e->outpoint.txid);
+ e->outpoint.n = db_col_int(stmt, "e.outnum");
+
+ if (!db_col_is_null(stmt, "e.payment_id")) {
+ e->payment_id = tal(e, struct sha256);
+ db_col_sha256(stmt, "e.payment_id", e->payment_id);
+ } else
+ e->payment_id = NULL;
+
+ if (!db_col_is_null(stmt, "e.spending_txid")) {
+ e->spending_txid = tal(e, struct bitcoin_txid);
+ db_col_txid(stmt, "e.spending_txid", e->spending_txid);
+ } else
+ e->spending_txid = NULL;
+
+ e->ignored = db_col_int(stmt, "e.ignored") == 1;
+ e->stealable = db_col_int(stmt, "e.stealable") == 1;
+
+ if (!db_col_is_null(stmt, "e.ev_desc"))
+ e->desc = db_col_strdup(e, stmt, "e.ev_desc");
+ else
+ e->desc = NULL;
+
+ e->splice_close = db_col_int(stmt, "e.spliced") == 1;
+ e->output_count = db_col_int(stmt, "a.closed_count");
+ if (!db_col_is_null(stmt, "a.peer_id")) {
+ e->peer_id = tal(e, struct node_id);
+ db_col_node_id(stmt, "a.peer_id", e->peer_id);
+ } else
+ e->peer_id = NULL;
+
+ e->we_opened = db_col_int(stmt, "a.we_opened");
+
+ /* Note that they would have never executed the final migration from
+ * "common: remove "ignored" tag", in this PR, so we do that now:
+ * {SQL("UPDATE chain_events"
+ * " SET account_id = (SELECT id FROM accounts WHERE name = 'external')"
+ * " WHERE account_id = (SELECT id FROM accounts WHERE name = 'wallet')"
+ * " AND ignored = 1"), NULL},
+ */
+ if (e->ignored && streq(e->acct_name, ACCOUNT_NAME_WALLET))
+ e->acct_name = ACCOUNT_NAME_EXTERNAL;
+
+ return e;
+}
+
+static struct chain_event **find_chain_events(const tal_t *ctx,
+ struct db_stmt *stmt TAKES)
+{
+ struct chain_event **results;
+
+ db_query_prepared(stmt);
+ if (stmt->error)
+ db_fatal(stmt->db, "find_chain_events err: %s", stmt->error);
+ results = tal_arr(ctx, struct chain_event *, 0);
+ while (db_step(stmt)) {
+ struct chain_event *e = stmt2chain_event(results, stmt);
+ tal_arr_expand(&results, e);
+ }
+
+ if (taken(stmt))
+ tal_free(stmt);
+
+ return results;
+}
+
+static struct chain_event **list_chain_events(const tal_t *ctx, struct db *db)
+{
+ struct db_stmt *stmt;
+
+ stmt = db_prepare_v2(db, SQL("SELECT"
+ " e.id"
+ ", e.account_id"
+ ", a.name"
+ ", e.origin"
+ ", e.tag"
+ ", e.credit"
+ ", e.debit"
+ ", e.output_value"
+ ", e.currency"
+ ", e.timestamp"
+ ", e.blockheight"
+ ", e.utxo_txid"
+ ", e.outnum"
+ ", e.spending_txid"
+ ", e.payment_id"
+ ", e.ignored"
+ ", e.stealable"
+ ", e.ev_desc"
+ ", e.spliced"
+ ", a.closed_count"
+ ", a.peer_id"
+ ", a.we_opened"
+ " FROM chain_events e"
+ " LEFT OUTER JOIN accounts a"
+ " ON e.account_id = a.id"
+ " ORDER BY e.timestamp, e.id;"));
+
+ return find_chain_events(ctx, take(stmt));
+}
+
+struct channel_event {
+
+ /* Id of this chain event in the database */
+ u64 db_id;
+
+ /* db_id of account this event belongs to */
+ u64 acct_db_id;
+
+ /* Name of the account this belongs to */
+ char *acct_name;
+
+ /* Tag describing the event */
+ const char *tag;
+
+ /* Amount we received in this event */
+ struct amount_msat credit;
+
+ /* Amount we paid in this event */
+ struct amount_msat debit;
+
+ /* Total 'fees' related to this channel event */
+ struct amount_msat fees;
+
+ /* What token are the credit/debits? */
+ const char *currency;
+
+ /* Payment identifier (typically the preimage hash) */
+ struct sha256 *payment_id;
+
+ /* Some payments share a payment_id, and are differentiable via id */
+ u32 part_id;
+
+ /* What time did the event happen */
+ u64 timestamp;
+
+ /* Description, usually from invoice */
+ const char *desc;
+
+ /* ID of paired event, iff is a rebalance */
+ u64 *rebalance_id;
+};
+
+static struct channel_event *stmt2channel_event(const tal_t *ctx, struct db_stmt *stmt)
+{
+ struct channel_event *e = tal(ctx, struct channel_event);
+
+ e->db_id = db_col_u64(stmt, "e.id");
+ e->acct_db_id = db_col_u64(stmt, "e.account_id");
+ e->acct_name = db_col_strdup(e, stmt, "a.name");
+
+ e->tag = db_col_strdup(e, stmt, "e.tag");
+
+ e->credit = db_col_amount_msat(stmt, "e.credit");
+ e->debit = db_col_amount_msat(stmt, "e.debit");
+ e->fees = db_col_amount_msat(stmt, "e.fees");
+
+ e->currency = db_col_strdup(e, stmt, "e.currency");
+ if (!db_col_is_null(stmt, "e.payment_id")) {
+ e->payment_id = tal(e, struct sha256);
+ db_col_sha256(stmt, "e.payment_id", e->payment_id);
+ } else
+ e->payment_id = NULL;
+ e->part_id = db_col_int(stmt, "e.part_id");
+ e->timestamp = db_col_u64(stmt, "e.timestamp");
+
+ if (!db_col_is_null(stmt, "e.ev_desc"))
+ e->desc = db_col_strdup(e, stmt, "e.ev_desc");
+ else
+ e->desc = NULL;
+
+ if (!db_col_is_null(stmt, "e.rebalance_id")) {
+ e->rebalance_id = tal(e, u64);
+ *e->rebalance_id = db_col_u64(stmt, "e.rebalance_id");
+ } else
+ e->rebalance_id = NULL;
+
+ return e;
+}
+
+static struct channel_event **list_channel_events(const tal_t *ctx,
+ struct db *db)
+
+{
+ struct db_stmt *stmt;
+ struct channel_event **results;
+
+ stmt = db_prepare_v2(db, SQL("SELECT"
+ " e.id"
+ ", e.account_id"
+ ", a.name"
+ ", e.tag"
+ ", e.credit"
+ ", e.debit"
+ ", e.fees"
+ ", e.currency"
+ ", e.payment_id"
+ ", e.part_id"
+ ", e.timestamp"
+ ", e.ev_desc"
+ ", e.rebalance_id"
+ " FROM channel_events e"
+ " LEFT OUTER JOIN accounts a"
+ " ON a.id = e.account_id"
+ " ORDER BY e.timestamp, e.id;"));
+ db_query_prepared(stmt);
+
+ results = tal_arr(ctx, struct channel_event *, 0);
+ while (db_step(stmt)) {
+ struct channel_event *e = stmt2channel_event(results, stmt);
+ tal_arr_expand(&results, e);
+ }
+ tal_free(stmt);
+
+ return results;
+}
+/* end stolen code */
+
+static void acct_db_error(struct lightningd *ld, bool fatal, const char *fmt, va_list ap)
+{
+ va_list ap2;
+
+ fmt = tal_fmt(tmpctx, "bookkeper migration: %s", fmt);
+ va_copy(ap2, ap);
+ logv(ld->log, LOG_BROKEN, NULL, true, fmt, ap);
+
+ if (fatal)
+ fatal_vfmt(fmt, ap2);
+ va_end(ap2);
+}
+
+void migrate_from_account_db(struct lightningd *ld, struct db *db)
+{
+ const char *olddir = NULL;
+ const char *db_dsn;
+ struct db *account_db;
+ struct chain_event **chain_events;
+ struct channel_event **channel_events;
+ size_t descriptions_migrated = 0;
+ struct db_stmt *stmt;
+ int version;
+
+ /* Initialize wait indices: we're going to use it to generate ids. */
+ load_indexes(db, ld->indexes);
+
+ /* Switch to bookkeeper-dir, if specified */
+ if (ld->old_bookkeeper_dir) {
+ olddir = path_cwd(NULL);
+ if (chdir(ld->old_bookkeeper_dir) != 0)
+ fatal("Unable to switch to 'bookkeeper-dir'=%s",
+ ld->old_bookkeeper_dir);
+ }
+
+ /* No user suppled db_dsn, set one up here */
+ db_dsn = ld->old_bookkeeper_db;
+ if (!db_dsn)
+ db_dsn = "sqlite3://accounts.sqlite3";
+
+ /* If we can't open it, we ignore it */
+ account_db = db_open(NULL, db_dsn, ld->developer, false, acct_db_error, ld);
+ if (!account_db) {
+ migrate_setup_coinmoves(ld, db);
+ goto out;
+ }
+
+ /* Load events */
+ db_begin_transaction(account_db);
+ version = db_get_version(account_db);
+ /* -1 means empty database (Postgres usually). */
+ if (version == -1) {
+ db_commit_transaction(account_db);
+ tal_free(account_db);
+ migrate_setup_coinmoves(ld, db);
+ goto out;
+ }
+ /* Last migration was 24.08. Migrate there first if this happens. */
+ if (version != 17)
+ fatal("Cannot migrate account database version %i", version);
+ chain_events = list_chain_events(tmpctx, account_db);
+ channel_events = list_channel_events(tmpctx, account_db);
+ db_commit_transaction(account_db);
+ tal_free(account_db);
+
+ for (size_t i = 0; i < tal_count(chain_events); i++) {
+ const struct chain_event *ev = chain_events[i];
+ struct mvt_account_id *account = tal(ev, struct mvt_account_id);
+ struct mvt_tags tags;
+ enum mvt_tag tag;
+ struct amount_sat output_sat;
+ u64 id;
+
+ stmt = db_prepare_v2(db,
+ SQL("INSERT INTO chain_moves ("
+ " id,"
+ " account_channel_id,"
+ " account_nonchannel_id,"
+ " tag_bitmap,"
+ " credit_or_debit,"
+ " timestamp,"
+ " utxo,"
+ " spending_txid,"
+ " peer_id,"
+ " payment_hash,"
+ " block_height,"
+ " output_sat,"
+ " originating_channel_id,"
+ " originating_nonchannel_id,"
+ " output_count) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"));
+ set_mvt_account_id(account, NULL, ev->acct_name);
+ id = chain_mvt_index_created(ld, db, account, ev->credit, ev->debit);
+ db_bind_u64(stmt, id);
+ if (!mvt_tag_parse(ev->tag, strlen(ev->tag), &tag))
+ abort();
+ tags = tag_to_mvt_tags(tag);
+ if (tag == MVT_CHANNEL_OPEN && ev->we_opened)
+ mvt_tag_set(&tags, MVT_OPENER);
+ if (ev->splice_close)
+ mvt_tag_set(&tags, MVT_SPLICE);
+ if (ev->stealable)
+ mvt_tag_set(&tags, MVT_STEALABLE);
+ db_bind_mvt_account_id(stmt, db, account);
+ db_bind_mvt_tags(stmt, tags);
+ db_bind_credit_debit(stmt, ev->credit, ev->debit);
+ db_bind_u64(stmt, ev->timestamp);
+ db_bind_outpoint(stmt, &ev->outpoint);
+ if (ev->spending_txid)
+ db_bind_txid(stmt, ev->spending_txid);
+ else
+ db_bind_null(stmt);
+ if (ev->peer_id)
+ db_bind_node_id(stmt, ev->peer_id);
+ else
+ db_bind_null(stmt);
+ if (ev->payment_id)
+ db_bind_sha256(stmt, ev->payment_id);
+ else
+ db_bind_null(stmt);
+ db_bind_int(stmt, ev->blockheight);
+ if (!amount_msat_to_sat(&output_sat, ev->output_value))
+ abort();
+ db_bind_amount_sat(stmt, output_sat);
+ if (ev->origin_acct) {
+ struct mvt_account_id *orig_account = tal(ev, struct mvt_account_id);
+ set_mvt_account_id(orig_account, NULL, ev->origin_acct);
+ db_bind_mvt_account_id(stmt, db, orig_account);
+ } else {
+ db_bind_null(stmt);
+ db_bind_null(stmt);
+ }
+ if (ev->output_count > 0)
+ db_bind_int(stmt, ev->output_count);
+ else
+ db_bind_null(stmt);
+ db_exec_prepared_v2(take(stmt));
+
+ /* Put descriptions into datastore for bookkeeper */
+ if (ev->desc) {
+ log_debug(ld->log, "Adding utxo description '%s' to %s",
+ ev->desc, fmt_bitcoin_outpoint(tmpctx, &ev->outpoint));
+ wallet_datastore_save_utxo_description(db, &ev->outpoint, ev->desc);
+ descriptions_migrated++;
+ }
+ }
+
+ for (size_t i = 0; i < tal_count(channel_events); i++) {
+ const struct channel_event *ev = channel_events[i];
+ struct mvt_account_id *account = tal(ev, struct mvt_account_id);
+ enum mvt_tag tag;
+ u64 id;
+
+ stmt = db_prepare_v2(db,
+ SQL("INSERT INTO channel_moves ("
+ " id,"
+ " account_channel_id,"
+ " account_nonchannel_id,"
+ " credit_or_debit,"
+ " tag_bitmap,"
+ " timestamp,"
+ " payment_hash,"
+ " payment_part_id,"
+ " payment_group_id,"
+ " fees) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?);"));
+ set_mvt_account_id(account, NULL, ev->acct_name);
+ id = channel_mvt_index_created(ld, db, account, ev->credit, ev->debit);
+ db_bind_u64(stmt, id);
+ db_bind_mvt_account_id(stmt, db, account);
+ db_bind_credit_debit(stmt, ev->credit, ev->debit);
+ if (!mvt_tag_parse(ev->tag, strlen(ev->tag), &tag))
+ abort();
+ db_bind_mvt_tags(stmt, tag_to_mvt_tags(tag));
+ db_bind_u64(stmt, ev->timestamp);
+ if (ev->payment_id)
+ db_bind_sha256(stmt, ev->payment_id);
+ else
+ db_bind_null(stmt);
+ if (ev->part_id) {
+ db_bind_u64(stmt, ev->part_id);
+ /* Unf. this was not recorded! */
+ db_bind_u64(stmt, 0);
+ } else {
+ db_bind_null(stmt);
+ db_bind_null(stmt);
+ }
+ db_bind_amount_msat(stmt, ev->fees);
+ db_exec_prepared_v2(take(stmt));
+
+ /* Put descriptions into datastore for bookkeeper */
+ if (ev->desc && ev->payment_id) {
+ wallet_datastore_save_payment_description(db, ev->payment_id, ev->desc);
+ descriptions_migrated++;
+ }
+ }
+
+ log_info(ld->log, "bookkeeper migration complete: migrated %zu chainmoves, %zu channelmoves, %zu descriptions",
+ tal_count(chain_events),
+ tal_count(channel_events),
+ descriptions_migrated);
+
+out:
+ if (olddir) {
+ if (chdir(olddir) != 0)
+ fatal("Unable to switch to back to %s",
+ olddir);
+ tal_free(olddir);
+ }
+}
diff --git a/wallet/account_migration.h b/wallet/account_migration.h
new file mode 100644
index 00000000..a4af4060
--- /dev/null
+++ b/wallet/account_migration.h
@@ -0,0 +1,10 @@
+#ifndef LIGHTNING_WALLET_ACCOUNT_MIGRATION_H
+#define LIGHTNING_WALLET_ACCOUNT_MIGRATION_H
+#include "config.h"
+
+struct lightningd;
+struct db;
+
+/* Some migrations are so epic they get their own file. Not in a good way. */
+void migrate_from_account_db(struct lightningd *ld, struct db *db);
+#endif /* LIGHTNING_WALLET_ACCOUNT_MIGRATION_H */
diff --git a/wallet/db.c b/wallet/db.c
index faae6161..6057952d 100644
--- a/wallet/db.c
+++ b/wallet/db.c
@@ -18,6 +18,7 @@
#include <lightningd/plugin_hook.h>
#include <sodium/randombytes.h>
#include <stddef.h>
+#include <wallet/account_migration.h>
#include <wallet/db.h>
#include <wallet/psbt_fixup.h>
#include <wire/peer_wire.h>
@@ -1091,6 +1092,7 @@ static struct migration dbmigrations[] = {
")"), NULL},
/* We do a lookup before each append, to avoid duplicates */
{SQL("CREATE INDEX chain_moves_utxo_idx ON chain_moves (utxo)"), NULL},
+ {NULL, migrate_from_account_db},
};
/**
diff --git a/wallet/invoices.c b/wallet/invoices.c
index 8122cac5..1655c8f9 100644
--- a/wallet/invoices.c
+++ b/wallet/invoices.c
@@ -777,13 +777,14 @@ static u64 invoice_index_inc(struct lightningd *ld,
invstrname = "bolt11";
- return wait_index_increment(ld, WAIT_SUBSYSTEM_INVOICE, idx,
- "status", state ? invoice_status_str(*state) : NULL,
- /* We don't want to add more JSON escapes here! */
- "=label", label ? tal_fmt(tmpctx, "\"%s\"", label->s) : NULL,
- invstrname, invstring,
- "description", description,
- NULL);
+ return wait_index_increment(ld, ld->wallet->db,
+ WAIT_SUBSYSTEM_INVOICE, idx,
+ "status", state ? invoice_status_str(*state) : NULL,
+ /* We don't want to add more JSON escapes here! */
+ "=label", label ? tal_fmt(tmpctx, "\"%s\"", label->s) : NULL,
+ invstrname, invstring,
+ "description", description,
+ NULL);
}
void invoice_index_deleted(struct lightningd *ld,
diff --git a/wallet/test/run-db.c b/wallet/test/run-db.c
index a010a1b0..ce3e2b9f 100644
--- a/wallet/test/run-db.c
+++ b/wallet/test/run-db.c
@@ -38,6 +38,7 @@ void bitcoind_getrawblockbyheight_(const tal_t *ctx UNNEEDED,
{ fprintf(stderr, "bitcoind_getrawblockbyheight_ called!\n"); abort(); }
/* Generated stub for chain_mvt_index_created */
u64 chain_mvt_index_created(struct lightningd *ld UNNEEDED,
+ struct db *db UNNEEDED,
const struct mvt_account_id *account UNNEEDED,
struct amount_msat credit UNNEEDED,
struct amount_msat debit UNNEEDED)
@@ -53,6 +54,7 @@ void channel_gossip_update(struct channel *channel UNNEEDED)
{ fprintf(stderr, "channel_gossip_update called!\n"); abort(); }
/* Generated stub for channel_mvt_index_created */
u64 channel_mvt_index_created(struct lightningd *ld UNNEEDED,
+ struct db *db UNNEEDED,
const struct mvt_account_id *account UNNEEDED,
struct amount_msat credit UNNEEDED,
struct amount_msat debit UNNEEDED)
@@ -415,6 +417,10 @@ void plugin_hook_db_sync(struct db *db UNNEEDED)
{
}
+void migrate_from_account_db(struct lightningd *ld UNNEEDED, struct db *db UNNEEDED)
+{
+}
+
static struct db *create_test_db(void)
{
struct db *db;
diff --git a/wallet/test/run-wallet.c b/wallet/test/run-wallet.c
index 31c8f91b..421bb6a7 100644
--- a/wallet/test/run-wallet.c
+++ b/wallet/test/run-wallet.c
@@ -99,12 +99,14 @@ void broadcast_tx_(const tal_t *ctx UNNEEDED,
{ fprintf(stderr, "broadcast_tx_ called!\n"); abort(); }
/* Generated stub for chain_mvt_index_created */
u64 chain_mvt_index_created(struct lightningd *ld UNNEEDED,
+ struct db *db UNNEEDED,
const struct mvt_account_id *account UNNEEDED,
struct amount_msat credit UNNEEDED,
struct amount_msat debit UNNEEDED)
{ fprintf(stderr, "chain_mvt_index_created called!\n"); abort(); }
/* Generated stub for channel_mvt_index_created */
u64 channel_mvt_index_created(struct lightningd *ld UNNEEDED,
+ struct db *db UNNEEDED,
const struct mvt_account_id *account UNNEEDED,
struct amount_msat credit UNNEEDED,
struct amount_msat debit UNNEEDED)
@@ -1316,6 +1318,10 @@ u32 get_block_height(const struct chain_topology *topo UNNEEDED)
return 0;
}
+void migrate_from_account_db(struct lightningd *ld UNNEEDED, struct db *db UNNEEDED)
+{
+}
+
/**
* mempat -- Set the memory to a pattern
*
diff --git a/wallet/wallet.c b/wallet/wallet.c
index c649696c..46201216 100644
--- a/wallet/wallet.c
+++ b/wallet/wallet.c
@@ -246,7 +246,7 @@ struct wallet *wallet_new(struct lightningd *ld, struct timers *timers)
}
/* Get id for move_accounts; create if necessary */
-static u64 move_accounts_id(struct wallet *wallet, const char *name)
+u64 move_accounts_id(struct db *db, const char *name)
{
struct db_stmt *stmt;
u64 ret;
@@ -256,7 +256,7 @@ static u64 move_accounts_id(struct wallet *wallet, const char *name)
* SQLite 3.35+ (released 2021), but not with INSERT OR
* IGNORE. So we do this in two steps (it likely exists) */
stmt = db_prepare_v2(
- wallet->db,
+ db,
SQL("SELECT id FROM move_accounts WHERE name = ?"));
db_bind_text(stmt, name);
db_query_prepared(stmt);
@@ -269,7 +269,7 @@ static u64 move_accounts_id(struct wallet *wallet, const char *name)
tal_free(stmt);
/* Does not exist, so create */
- stmt = db_prepare_v2(wallet->db,
+ stmt = db_prepare_v2(db,
SQL("INSERT INTO move_accounts (name) VALUES (?)"));
db_bind_text(stmt, name);
db_exec_prepared_v2(stmt);
@@ -3021,7 +3021,7 @@ void wallet_channel_close(struct wallet *w,
/* Update all accouting records to use channel_id string, instead of
* referring to dbid. This is robust if we delete in future, and saves
* a lookup in the load path. */
- new_move_id = move_accounts_id(w, fmt_channel_id(tmpctx, &chan->cid));
+ new_move_id = move_accounts_id(w->db, fmt_channel_id(tmpctx, &chan->cid));
stmt = db_prepare_v2(w->db, SQL("UPDATE chain_moves "
"SET account_channel_id=?,"
" account_nonchannel_id=? "
@@ -6270,11 +6270,11 @@ void wallet_datastore_update(struct wallet *w, const char **key, const u8 *data)
db_exec_prepared_v2(take(stmt));
}
-void wallet_datastore_create(struct wallet *w, const char **key, const u8 *data)
+static void db_datastore_create(struct db *db, const char **key, const u8 *data)
{
struct db_stmt *stmt;
- stmt = db_prepare_v2(w->db,
+ stmt = db_prepare_v2(db,
SQL("INSERT INTO datastore VALUES (?, ?, 0);"));
db_bind_datastore_key(stmt, key);
@@ -6282,6 +6282,11 @@ void wallet_datastore_create(struct wallet *w, const char **key, const u8 *data)
db_exec_prepared_v2(take(stmt));
}
+void wallet_datastore_create(struct wallet *w, const char **key, const u8 *data)
+{
+ db_datastore_create(w->db, key, data);
+}
+
static void db_datastore_remove(struct db *db, const char **key)
{
struct db_stmt *stmt;
@@ -6292,6 +6297,40 @@ static void db_datastore_remove(struct db *db, const char **key)
db_exec_prepared_v2(take(stmt));
}
+void wallet_datastore_save_utxo_description(struct db *db,
+ const struct bitcoin_outpoint *outpoint,
+ const char *desc)
+{
+ const char **key;
+
+ key = tal_arr(tmpctx, const char *, 4);
+ key[0] = "bookkeeper";
+ key[1] = "description";
+ key[2] = "utxo";
+ key[3] = fmt_bitcoin_outpoint(key, outpoint);
+
+ /* In case it's a duplicate, remove first */
+ db_datastore_remove(db, key);
+ db_datastore_create(db, key, tal_dup_arr(key, u8, (u8 *)desc, strlen(desc), 0));
+}
+
+void wallet_datastore_save_payment_description(struct db *db,
+ const struct sha256 *payment_hash,
+ const char *desc)
+{
+ const char **key;
+
+ key = tal_arr(tmpctx, const char *, 4);
+ key[0] = "bookkeeper";
+ key[1] = "description";
+ key[2] = "payment";
+ key[3] = fmt_sha256(key, payment_hash);
+
+ /* In case it's a duplicate, remove first */
+ db_datastore_remove(db, key);
+ db_datastore_create(db, key, tal_dup_arr(key, u8, (u8 *)desc, strlen(desc), 0));
+}
+
void wallet_datastore_remove(struct wallet *w, const char **key)
{
db_datastore_remove(w->db, key);
@@ -6869,9 +6908,9 @@ struct issued_address_type *wallet_list_addresses(const tal_t *ctx, struct walle
return addresseslist;
}
-static void db_bind_credit_debit(struct db_stmt *stmt,
- struct amount_msat credit,
- struct amount_msat debit)
+void db_bind_credit_debit(struct db_stmt *stmt,
+ struct amount_msat credit,
+ struct amount_msat debit)
{
if (amount_msat_is_zero(debit))
db_bind_amount_msat(stmt, credit);
@@ -6882,20 +6921,20 @@ static void db_bind_credit_debit(struct db_stmt *stmt,
}
}
-static void db_bind_mvt_account_id(struct db_stmt *stmt,
- struct lightningd *ld,
- const struct mvt_account_id *account)
+void db_bind_mvt_account_id(struct db_stmt *stmt,
+ struct db *db,
+ const struct mvt_account_id *account)
{
if (account->channel) {
db_bind_u64(stmt, account->channel->dbid);
db_bind_null(stmt);
} else {
db_bind_null(stmt);
- db_bind_u64(stmt, move_accounts_id(ld->wallet, account->alt_account));
+ db_bind_u64(stmt, move_accounts_id(db, account->alt_account));
}
}
-static void db_bind_mvt_tags(struct db_stmt *stmt, struct mvt_tags tags)
+void db_bind_mvt_tags(struct db_stmt *stmt, struct mvt_tags tags)
{
assert(mvt_tags_valid(tags));
db_bind_u64(stmt, tags.bits);
@@ -6919,11 +6958,11 @@ void wallet_save_channel_mvt(struct lightningd *ld,
" payment_part_id,"
" payment_group_id,"
" fees) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?);"));
- id = channel_mvt_index_created(ld,
+ id = channel_mvt_index_created(ld, ld->wallet->db,
&chan_mvt->account,
chan_mvt->credit, chan_mvt->debit);
db_bind_u64(stmt, id);
- db_bind_mvt_account_id(stmt, ld, &chan_mvt->account);
+ db_bind_mvt_account_id(stmt, ld->wallet->db, &chan_mvt->account);
db_bind_credit_debit(stmt, chan_mvt->credit, chan_mvt->debit);
db_bind_mvt_tags(stmt, chan_mvt->tags);
db_bind_u64(stmt, chan_mvt->timestamp);
@@ -7030,11 +7069,11 @@ void wallet_save_chain_mvt(struct lightningd *ld,
" originating_channel_id,"
" originating_nonchannel_id,"
" output_count) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"));
- id = chain_mvt_index_created(ld,
+ id = chain_mvt_index_created(ld, ld->wallet->db,
&chain_mvt->account,
chain_mvt->credit, chain_mvt->debit);
db_bind_u64(stmt, id);
- db_bind_mvt_account_id(stmt, ld, &chain_mvt->account);
+ db_bind_mvt_account_id(stmt, ld->wallet->db, &chain_mvt->account);
db_bind_mvt_tags(stmt, chain_mvt->tags);
db_bind_credit_debit(stmt, chain_mvt->credit, chain_mvt->debit);
db_bind_u64(stmt, chain_mvt->timestamp);
@@ -7054,7 +7093,7 @@ void wallet_save_chain_mvt(struct lightningd *ld,
db_bind_int(stmt, chain_mvt->blockheight);
db_bind_amount_sat(stmt, chain_mvt->output_val);
if (chain_mvt->originating_acct) {
- db_bind_mvt_account_id(stmt, ld, chain_mvt->originating_acct);
+ db_bind_mvt_account_id(stmt, ld->wallet->db, chain_mvt->originating_acct);
} else {
db_bind_null(stmt);
db_bind_null(stmt);
@@ -7459,3 +7498,10 @@ void wallet_begin_old_close_rescan(struct lightningd *ld)
bitcoind_getrawblockbyheight(missing, ld->topology->bitcoind, earliest_block,
mutual_close_p2pkh_catch, missing);
}
+
+/* An existing node without accounting. Fill in what we have so far. */
+void migrate_setup_coinmoves(struct lightningd *ld, struct db *db)
+{
+ /* FIXME: implement! */
+}
+
diff --git a/wallet/wallet.h b/wallet/wallet.h
index c7108a1a..584f17b0 100644
--- a/wallet/wallet.h
+++ b/wallet/wallet.h
@@ -1903,6 +1903,23 @@ struct channel_coin_mvt *wallet_channel_move_extract(const tal_t *ctx,
struct lightningd *ld,
u64 *id);
+/* For bookkeeper migration */
+void db_bind_mvt_tags(struct db_stmt *stmt, struct mvt_tags tags);
+void db_bind_mvt_account_id(struct db_stmt *stmt,
+ struct db *db,
+ const struct mvt_account_id *account);
+void db_bind_credit_debit(struct db_stmt *stmt,
+ struct amount_msat credit,
+ struct amount_msat debit);
+u64 move_accounts_id(struct db *db, const char *name);
+void wallet_datastore_save_utxo_description(struct db *db,
+ const struct bitcoin_outpoint *outpoint,
+ const char *desc);
+void wallet_datastore_save_payment_description(struct db *db,
+ const struct sha256 *payment_hash,
+ const char *desc);
+void migrate_setup_coinmoves(struct lightningd *ld, struct db *db);
+
/**
* wallet_memleak_scan - Check for memleaks in wallet.
*/
Why this scored 24/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.