bookkeeper: implement flexible "bkpr-report" command.
What changed, and why it matters
This commit adds a new bookkeeping command, bkpr-report, that lets users format their node's income events into custom text or CSV output. It is a feature addition, not a security fix. There is no indication in the commit or supplied references that it addresses a vulnerability or security issue.
No security action required. Treat as a normal feature addition. If reviewing further, focus on format-string parser robustness and CSV escaping correctness, but the supplied diff shows no obvious security defects.
Security signals we found
No security-relevant signals detected in the commit message, diff, or supplied references.
New user-facing RPC command with input parsing; no evidence of unsafe memory handling or injection vulnerabilities in the supplied diff.
CSV escaping is implemented for tag values, which is a correctness feature rather than a vulnerability fix.
Evidence from the diff
The patch implements a new JSON-RPC command bkpr-report in the bookkeeper plugin. It introduces a small format-string parser with tag substitution, nested fallback syntax ({tag?fallback}), and CSV escaping. The command reads existing income events from the bookkeeper database and emits formatted strings. No changes to consensus, cryptography, network protocol, or access control are present. The code is purely additive and includes unit tests and Python integration tests.
Changed components
plugins/bkpr/report.cplugins/bkpr/report.hplugins/bkpr/bookkeeper.cplugins/bkpr/bookkeeper.hdoc/schemas/bkpr-report.jsontests/test_bookkeeper.pyInspect captured patch +1436 / −11
diff --git a/contrib/msggen/msggen/schema.json b/contrib/msggen/msggen/schema.json
index cb9e0070..b1c9c67b 100644
--- a/contrib/msggen/msggen/schema.json
+++ b/contrib/msggen/msggen/schema.json
@@ -5112,6 +5112,132 @@
}
]
},
+ "bkpr-report.json": {
+ "$schema": "../rpc-schema-draft.json",
+ "type": "object",
+ "added": "v26.04",
+ "rpc": "bkpr-report",
+ "title": "Formatting for all income impacting events",
+ "description": [
+ "The **bkpr-report** RPC command creates a list of all income impacting events that the bookkeeper plugin has recorded for this node.",
+ "",
+ "It has the useful property that when used with `lightning-cli` and the `escape=csv` it can directly produce valid CSV files."
+ ],
+ "request": {
+ "required": [],
+ "additionalProperties": false,
+ "properties": {
+ "format": {
+ "type": "string",
+ "description": [
+ "This format string is used for each income event (note that `lightning-cli` can get confused if the format begins and ends with `{` and `}`, so you may need to add a space). The following tags in braces are replaced:",
+ "{account}: account name (channel id, or 'wallet')",
+ "{tag}: event tag. This will be one of:",
+ " * `deposit`: an onchain send to the wallet by `outpoint`.",
+ " * `htlc_fulfill`: an onchain HTLC fulfill (due to unilaterally closed channel) at `outpoint`.",
+ " * `invoice`: either incoming (positive credit) or outgoing (positive debit) payment.",
+ " * `invoice_fee`: the routing fee paid to pay an outgoing invoice",
+ " * `journal_entry`: an accounting fixup, caused by loss of data (or, a node which predates bookkeeper)",
+ " * `lease_fee`: a fee paid or received to lease a channel via the experimental liquidity advertisement option",
+ " * `onchain_fee,`: a miner fee paid to open/close a channel, or make a bitcoin payment. The `txid` will correspond to a `withdrawal` `outpoint` for withdrawl",
+ " * `pushed`: an amount pushed to or from us on channel open.",
+ " * `rebalance_fee`: routing fee paid for sending a payment to ourselves.",
+ " * `routed`: credit gained from routing a payment",
+ " * `withdrawal`: debit from an onchain spend.",
+ "{description}: description as provided in the invoice, if present",
+ "{credit}: credit amount in BTC",
+ "{debit}: debit amount in BTC",
+ "{fees}: fee amount in BTC",
+ "{localtime}: event timestamp in local time as YYYY-MM-DD HH:MM:SS",
+ "{utctime}: event timestamp in UTC as YYYY-MM-DD HH:MM:SS",
+ "{outpoint}: outpoint, if present",
+ "{txid}: txid, if present",
+ "{payment_id}: payment hash, if present",
+ "{bkpr-currency}: value of bkpr-currency, if any",
+ "{currencyrate}: exchange rate for 1 BTC at that event time, if available",
+ "{creditdebit}: +credit or -debit (or 0) in BTC",
+ "{currencycredit}: credit amount converted into bkpr-currency",
+ "{currencydebit}: debit amount converted into bkpr-currency",
+ "{currencycreditdebit}: +credit or -debit (or 0) in bkpr-currency",
+ "",
+ "If a field is unavailable, it expands to an empty string.",
+ "",
+ "You can provide fallback with ?, including more variable:",
+ " * {outpoint?NONE}",
+ " * {payment_id?txid: {txid?UNKNOWN}}",
+ "The first one the outpoint, or NONE if that is not available. ",
+ "The second prints the payment_id, or if that is not available, the string 'txid: ' followed by the txid, or if that is not available, 'txid: UNKNOWN'.",
+ "",
+ "The text after ? is used only if that tag would otherwise be empty.",
+ "",
+ "To include a literal {, write {{."
+ ]
+ },
+ "headers": {
+ "type": "array",
+ "description": [
+ "strings to place at the top of the output (useful when creating CSV files directly)."
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "escape": {
+ "type": "string",
+ "description": [
+ "How to handle the formatted output fields: if set to `csv` it will handle fields with commas or double-quotes correctly for that format. Note that text is never escaped (you need to do that), only {} tags."
+ ],
+ "items": {
+ "type": "string"
+ },
+ "default": "none"
+ },
+ "start_time": {
+ "type": "u32",
+ "description": [
+ "UNIX timestamp (in seconds) that filters events after the provided timestamp."
+ ],
+ "default": "zero"
+ },
+ "end_time": {
+ "type": "u32",
+ "description": [
+ "UNIX timestamp (in seconds) that filters events up to and at the provided timestamp."
+ ],
+ "default": "max-int"
+ }
+ }
+ },
+ "response": {
+ "required": [
+ "report"
+ ],
+ "additionalProperties": false,
+ "properties": {
+ "format-hint": {},
+ "report": {
+ "type": "array",
+ "items": {
+ "type": "string",
+ "description": [
+ "The headers one per entry, followed by the formatted strings for each income event"
+ ]
+ }
+ }
+ }
+ },
+ "author": [
+ "Rusty Russell [rusty@rustcorp.com.au](mailto:rusty@rustcorp.com.au) is mainly responsible."
+ ],
+ "see_also": [
+ "lightning-bkpr-listaccountevents(7)",
+ "lightning-bkpr-listbalances(7)",
+ "lightningd-config(5)"
+ ],
+ "resources": [
+ "Main web site: [https://github.com/ElementsProject/lightning](https://github.com/ElementsProject/lightning)"
+ ]
+ },
"blacklistrune.json": {
"$schema": "../rpc-schema-draft.json",
"type": "object",
diff --git a/doc/Makefile b/doc/Makefile
index 858b366f..5f1b8306 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -30,6 +30,7 @@ MARKDOWNPAGES := doc/addgossip.7 \
doc/bkpr-listaccountevents.7 \
doc/bkpr-listbalances.7 \
doc/bkpr-listincome.7 \
+ doc/bkpr-report.7 \
doc/blacklistrune.7 \
doc/cancelrecurringinvoice.7 \
doc/check.7 \
diff --git a/doc/index.rst b/doc/index.rst
index ba126b1c..f4fae7c6 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -38,6 +38,7 @@ Core Lightning Documentation
bkpr-listaccountevents <bkpr-listaccountevents.7.md>
bkpr-listbalances <bkpr-listbalances.7.md>
bkpr-listincome <bkpr-listincome.7.md>
+ bkpr-report <bkpr-report.7.md>
blacklistrune <blacklistrune.7.md>
cancelrecurringinvoice <cancelrecurringinvoice.7.md>
check <check.7.md>
diff --git a/doc/schemas/bkpr-report.json b/doc/schemas/bkpr-report.json
new file mode 100644
index 00000000..2eed4048
--- /dev/null
+++ b/doc/schemas/bkpr-report.json
@@ -0,0 +1,126 @@
+{
+ "$schema": "../rpc-schema-draft.json",
+ "type": "object",
+ "added": "v26.04",
+ "rpc": "bkpr-report",
+ "title": "Formatting for all income impacting events",
+ "description": [
+ "The **bkpr-report** RPC command creates a list of all income impacting events that the bookkeeper plugin has recorded for this node.",
+ "",
+ "It has the useful property that when used with `lightning-cli` and the `escape=csv` it can directly produce valid CSV files."
+ ],
+ "request": {
+ "required": [],
+ "additionalProperties": false,
+ "properties": {
+ "format": {
+ "type": "string",
+ "description": [
+ "This format string is used for each income event (note that `lightning-cli` can get confused if the format begins and ends with `{` and `}`, so you may need to add a space). The following tags in braces are replaced:",
+ "{account}: account name (channel id, or 'wallet')",
+ "{tag}: event tag. This will be one of:",
+ " * `deposit`: an onchain send to the wallet by `outpoint`.",
+ " * `htlc_fulfill`: an onchain HTLC fulfill (due to unilaterally closed channel) at `outpoint`.",
+ " * `invoice`: either incoming (positive credit) or outgoing (positive debit) payment.",
+ " * `invoice_fee`: the routing fee paid to pay an outgoing invoice",
+ " * `journal_entry`: an accounting fixup, caused by loss of data (or, a node which predates bookkeeper)",
+ " * `lease_fee`: a fee paid or received to lease a channel via the experimental liquidity advertisement option",
+ " * `onchain_fee,`: a miner fee paid to open/close a channel, or make a bitcoin payment. The `txid` will correspond to a `withdrawal` `outpoint` for withdrawl",
+ " * `pushed`: an amount pushed to or from us on channel open.",
+ " * `rebalance_fee`: routing fee paid for sending a payment to ourselves.",
+ " * `routed`: credit gained from routing a payment",
+ " * `withdrawal`: debit from an onchain spend.",
+ "{description}: description as provided in the invoice, if present",
+ "{credit}: credit amount in BTC",
+ "{debit}: debit amount in BTC",
+ "{fees}: fee amount in BTC",
+ "{localtime}: event timestamp in local time as YYYY-MM-DD HH:MM:SS",
+ "{utctime}: event timestamp in UTC as YYYY-MM-DD HH:MM:SS",
+ "{outpoint}: outpoint, if present",
+ "{txid}: txid, if present",
+ "{payment_id}: payment hash, if present",
+ "{bkpr-currency}: value of bkpr-currency, if any",
+ "{currencyrate}: exchange rate for 1 BTC at that event time, if available",
+ "{creditdebit}: +credit or -debit (or 0) in BTC",
+ "{currencycredit}: credit amount converted into bkpr-currency",
+ "{currencydebit}: debit amount converted into bkpr-currency",
+ "{currencycreditdebit}: +credit or -debit (or 0) in bkpr-currency",
+ "",
+ "If a field is unavailable, it expands to an empty string.",
+ "",
+ "You can provide fallback with ?, including more variable:",
+ " * {outpoint?NONE}",
+ " * {payment_id?txid: {txid?UNKNOWN}}",
+ "The first one the outpoint, or NONE if that is not available. ",
+ "The second prints the payment_id, or if that is not available, the string 'txid: ' followed by the txid, or if that is not available, 'txid: UNKNOWN'.",
+ "",
+ "The text after ? is used only if that tag would otherwise be empty.",
+ "",
+ "To include a literal {, write {{."
+ ]
+ },
+ "headers": {
+ "type": "array",
+ "description": [
+ "strings to place at the top of the output (useful when creating CSV files directly)."
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "escape": {
+ "type": "string",
+ "description": [
+ "How to handle the formatted output fields: if set to `csv` it will handle fields with commas or double-quotes correctly for that format. Note that text is never escaped (you need to do that), only {} tags."
+ ],
+ "items": {
+ "type": "string"
+ },
+ "default": "none"
+ },
+ "start_time": {
+ "type": "u32",
+ "description": [
+ "UNIX timestamp (in seconds) that filters events after the provided timestamp."
+ ],
+ "default": "zero"
+ },
+ "end_time": {
+ "type": "u32",
+ "description": [
+ "UNIX timestamp (in seconds) that filters events up to and at the provided timestamp."
+ ],
+ "default": "max-int"
+ }
+ }
+ },
+ "response": {
+ "required": [
+ "report"
+ ],
+ "additionalProperties": false,
+ "properties": {
+ "format-hint": {},
+ "report": {
+ "type": "array",
+ "items": {
+ "type": "string",
+ "description": [
+ "The headers one per entry, followed by the formatted strings for each income event"
+ ]
+ }
+ }
+ }
+ },
+ "author": [
+ "Rusty Russell [rusty@rustcorp.com.au](mailto:rusty@rustcorp.com.au) is mainly responsible."
+ ],
+ "see_also": [
+ "lightning-bkpr-listaccountevents(7)",
+ "lightning-bkpr-listbalances(7)",
+ "lightningd-config(5)"
+ ],
+ "resources": [
+ "Main web site: [https://github.com/ElementsProject/lightning](https://github.com/ElementsProject/lightning)"
+ ]
+}
diff --git a/plugins/bkpr/Makefile b/plugins/bkpr/Makefile
index 0647a546..b49b2492 100644
--- a/plugins/bkpr/Makefile
+++ b/plugins/bkpr/Makefile
@@ -13,6 +13,7 @@ BOOKKEEPER_PLUGIN_SRC := \
plugins/bkpr/onchain_fee.c \
plugins/bkpr/rebalances.c \
plugins/bkpr/recorder.c \
+ plugins/bkpr/report.c \
plugins/bkpr/sql.c
BOOKKEEPER_SRC := $(BOOKKEEPER_PLUGIN_SRC) $(BOOKKEEPER_DB_QUERIES)
diff --git a/plugins/bkpr/bookkeeper.c b/plugins/bkpr/bookkeeper.c
index ea0ec8ca..41c6e34f 100644
--- a/plugins/bkpr/bookkeeper.c
+++ b/plugins/bkpr/bookkeeper.c
@@ -33,6 +33,7 @@
#include <plugins/bkpr/onchain_fee.h>
#include <plugins/bkpr/rebalances.h>
#include <plugins/bkpr/recorder.h>
+#include <plugins/bkpr/report.h>
#include <plugins/libplugin.h>
#include <sys/stat.h>
#include <unistd.h>
@@ -48,8 +49,8 @@ struct bkpr *bkpr_of(struct plugin *plugin)
return plugin_get_data(plugin, struct bkpr);
}
-static const struct currencyrate *covering_currencyrate(const struct bkpr *bkpr,
- u64 timestamp)
+const struct currencyrate *covering_currencyrate(const struct bkpr *bkpr,
+ u64 timestamp)
{
/* We look for the previous entry, then check duration covers this. */
u64 ts = timestamp + 1;
@@ -70,18 +71,27 @@ static u64 ratefactor(const struct iso4217_name_and_divisor *currency)
const char *currencyrate_str(const tal_t *ctx,
const struct bkpr *bkpr,
- u64 timestamp)
+ u64 timestamp,
+ const struct amount_msat *msat)
{
const struct currencyrate *crate;
- u64 mul, intpart, fracpart;
+ u64 mul, intpart, fracpart, raw_rate;
crate = covering_currencyrate(bkpr, timestamp);
if (!crate)
return NULL;
mul = ratefactor(bkpr->currency);
- intpart = crate->raw_rate / mul;
- fracpart = crate->raw_rate % mul;
+ if (msat) {
+ unsigned __int128 v;
+ v = (unsigned __int128)msat->millisatoshis * crate->raw_rate /* Raw: 128-bit math */;
+ raw_rate = v / MSAT_PER_BTC;
+ } else {
+ raw_rate = crate->raw_rate;
+ }
+
+ intpart = raw_rate / mul;
+ fracpart = raw_rate % mul;
if (bkpr->currency->minor_unit == 0)
return tal_fmt(ctx, "%"PRIu64, intpart);
@@ -97,7 +107,7 @@ void json_add_currencyrate(struct json_stream *result,
const struct bkpr *bkpr,
u64 timestamp)
{
- const char *str = currencyrate_str(NULL, bkpr, timestamp);
+ const char *str = currencyrate_str(NULL, bkpr, timestamp, NULL);
if (str)
json_add_primitive(result, fieldname, take(str));
@@ -470,6 +480,48 @@ static struct command_result *json_dump_income(struct command *cmd,
return refresh_moves(cmd, do_dump_income, info);
}
+static struct command_result *param_escaped_string_array(struct command *cmd,
+ const char *name,
+ const char *buffer,
+ const jsmntok_t *tok,
+ const char ***arr)
+{
+ size_t i;
+ const jsmntok_t *s;
+
+ if (tok->type != JSMN_ARRAY)
+ return command_fail_badparam(cmd, name, buffer, tok,
+ "should be an array");
+ *arr = tal_arr(cmd, const char *, tok->size);
+ json_for_each_arr(i, s, tok) {
+ struct command_result *ret;
+
+ ret = param_escaped_string(cmd, name, buffer, s, &(*arr)[i]);
+ if (ret)
+ return ret;
+ tal_steal((*arr)[i], *arr);
+ }
+ return NULL;
+}
+
+static struct command_result *json_bkpr_report(struct command *cmd,
+ const char *buf,
+ const jsmntok_t *params)
+{
+ struct report_info *info = tal(cmd, struct report_info);
+
+ if (!param(cmd, buf, params,
+ p_req("format", param_report_format, &info->format),
+ p_opt("headers", param_escaped_string_array, &info->headers),
+ p_opt_def("escape", param_escape_format, &info->escapes, REPORT_FMT_NONE),
+ p_opt_def("start_time", param_u64, &info->start_time, 0),
+ p_opt_def("end_time", param_u64, &info->end_time, SQLITE_MAX_UINT),
+ NULL))
+ return command_param_failed();
+
+ return refresh_moves(cmd, do_bkpr_report, info);
+}
+
struct list_income_info {
bool *consolidate_fees;
u64 *start_time, *end_time;
@@ -1765,6 +1817,10 @@ static const struct plugin_command commands[] = {
"bkpr-channelsapy",
json_channel_apy
},
+ {
+ "bkpr-report",
+ json_bkpr_report,
+ },
{
"bkpr-editdescriptionbypaymentid",
json_edit_desc_payment_id
diff --git a/plugins/bkpr/bookkeeper.h b/plugins/bkpr/bookkeeper.h
index 95cf9356..dc023858 100644
--- a/plugins/bkpr/bookkeeper.h
+++ b/plugins/bkpr/bookkeeper.h
@@ -44,10 +44,16 @@ struct bkpr {
/* Get bkpr struct for the plugin */
struct bkpr *bkpr_of(struct plugin *plugin);
-/* Get currency rate for this timestamp, as string, or NULL. */
+/* Get currency rate for this timestamp, as string, or NULL.
+ * If msat is non-NULL, amount for that number of msat (otherwise 1 btc)*/
const char *currencyrate_str(const tal_t *ctx,
const struct bkpr *bkpr,
- u64 timestamp);
+ u64 timestamp,
+ const struct amount_msat *msat);
+
+/* Get the struct currencyrate covering this timestamp, if any. */
+const struct currencyrate *covering_currencyrate(const struct bkpr *bkpr,
+ u64 timestamp);
/* Add optional currencyrate for this timestamp */
void json_add_currencyrate(struct json_stream *result,
diff --git a/plugins/bkpr/incomestmt.c b/plugins/bkpr/incomestmt.c
index 2b5cb334..470fc5d1 100644
--- a/plugins/bkpr/incomestmt.c
+++ b/plugins/bkpr/incomestmt.c
@@ -6,7 +6,6 @@
#include <ccan/tal/str/str.h>
#include <common/clock_time.h>
#include <common/coin_mvt.h>
-#include <common/json_parse_simple.h>
#include <common/json_stream.h>
#include <inttypes.h>
#include <plugins/bkpr/account.h>
diff --git a/plugins/bkpr/incomestmt.h b/plugins/bkpr/incomestmt.h
index 0de3c00d..fc9391f5 100644
--- a/plugins/bkpr/incomestmt.h
+++ b/plugins/bkpr/incomestmt.h
@@ -3,8 +3,14 @@
#include "config.h"
#include <ccan/tal/tal.h>
+#include <common/amount.h>
+#include <common/json_parse_simple.h>
#include <stdio.h>
+struct bkpr;
+struct command;
+struct json_stream;
+
struct income_event {
const char *acct_name;
const char *tag;
diff --git a/plugins/bkpr/report.c b/plugins/bkpr/report.c
new file mode 100644
index 00000000..754f29a8
--- /dev/null
+++ b/plugins/bkpr/report.c
@@ -0,0 +1,456 @@
+#include "config.h"
+#include <bitcoin/tx.h>
+#include <ccan/array_size/array_size.h>
+#include <ccan/json_escape/json_escape.h>
+#include <ccan/mem/mem.h>
+#include <ccan/tal/str/str.h>
+#include <common/iso4217.h>
+#include <common/json_command.h>
+#include <common/json_param.h>
+#include <common/json_stream.h>
+#include <common/utils.h>
+#include <inttypes.h>
+#include <plugins/bkpr/bookkeeper.h>
+#include <plugins/bkpr/incomestmt.h>
+#include <plugins/bkpr/report.h>
+#include <plugins/libplugin.h>
+
+static const char *report_fmt_acct_name(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ return e->acct_name;
+}
+
+static const char *report_fmt_tag(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ return e->tag;
+}
+
+static const char *report_fmt_desc(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ return e->desc;
+}
+
+static const char *report_fmt_credit(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ return fmt_amount_msat_btc(ctx, e->credit, false);
+}
+
+static const char *report_fmt_debit(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ return fmt_amount_msat_btc(ctx, e->debit, false);
+}
+
+static const char *report_fmt_fees(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ return fmt_amount_msat_btc(ctx, e->fees, false);
+}
+
+static const char *report_fmt_localtime(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ time_t t = e->timestamp;
+ struct tm tm;
+ char buf[100];
+
+ localtime_r(&t, &tm);
+ strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S", &tm);
+ return tal_strdup(ctx, buf);
+}
+
+static const char *report_fmt_utctime(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ time_t t = e->timestamp;
+ struct tm tm;
+ char buf[100];
+
+ gmtime_r(&t, &tm);
+ strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S", &tm);
+ return tal_strdup(ctx, buf);
+}
+
+static const char *report_fmt_outpoint(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ if (!e->outpoint)
+ return NULL;
+ return fmt_bitcoin_outpoint(ctx, e->outpoint);
+}
+
+static const char *report_fmt_txid(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ if (!e->txid)
+ return NULL;
+ return fmt_bitcoin_txid(ctx, e->txid);
+}
+
+static const char *report_fmt_payment_id(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ if (!e->payment_id)
+ return NULL;
+ return fmt_sha256(ctx, e->payment_id);
+}
+
+static const char *report_fmt_bkpr_currency(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr,
+ const struct income_event *e UNNEEDED)
+{
+ return bkpr->currency ? bkpr->currency->name : NULL;
+}
+
+static const char *report_fmt_bkpr_currencyrate(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr,
+ const struct income_event *e)
+{
+ return currencyrate_str(ctx, bkpr, e->timestamp, NULL);
+}
+
+static const char *report_fmt_credit_debit(const tal_t *ctx,
+ const struct bkpr *bkpr UNNEEDED,
+ const struct income_event *e)
+{
+ if (!amount_msat_is_zero(e->credit))
+ return tal_fmt(ctx, "+%s",
+ fmt_amount_msat_btc(tmpctx, e->credit, false));
+ if (!amount_msat_is_zero(e->debit))
+ return tal_fmt(ctx, "-%s",
+ fmt_amount_msat_btc(tmpctx, e->debit, false));
+ return "0";
+}
+
+static const char *report_fmt_currency_credit(const tal_t *ctx,
+ const struct bkpr *bkpr,
+ const struct income_event *e)
+{
+ return currencyrate_str(ctx, bkpr, e->timestamp, &e->credit);
+}
+
+static const char *report_fmt_currency_debit(const tal_t *ctx,
+ const struct bkpr *bkpr,
+ const struct income_event *e)
+{
+ return currencyrate_str(ctx, bkpr, e->timestamp, &e->debit);
+}
+
+static const char *report_fmt_currency_credit_debit(const tal_t *ctx,
+ const struct bkpr *bkpr,
+ const struct income_event *e)
+{
+ const char *s;
+
+ if (!amount_msat_is_zero(e->credit)) {
+ s = currencyrate_str(tmpctx, bkpr, e->timestamp, &e->credit);
+ return s ? tal_fmt(ctx, "+%s", s) : NULL;
+ }
+ if (!amount_msat_is_zero(e->debit)) {
+ s = currencyrate_str(tmpctx, bkpr, e->timestamp, &e->debit);
+ return s ? tal_fmt(ctx, "-%s", s) : NULL;
+ }
+ return "0";
+}
+
+struct report_tag {
+ const char *name;
+ const char *(*fmt)(const tal_t *ctx,
+ const struct bkpr *bkpr,
+ const struct income_event *e);
+};
+
+static const struct report_tag report_tags[] = {
+ { "account", report_fmt_acct_name },
+ { "tag", report_fmt_tag },
+ { "description", report_fmt_desc },
+ { "credit", report_fmt_credit },
+ { "debit", report_fmt_debit },
+ { "fees", report_fmt_fees },
+ { "localtime", report_fmt_localtime },
+ { "utctime", report_fmt_utctime },
+ { "outpoint", report_fmt_outpoint },
+ { "txid", report_fmt_txid },
+ { "payment_id", report_fmt_payment_id },
+ { "bkpr-currency", report_fmt_bkpr_currency },
+ { "currencyrate", report_fmt_bkpr_currencyrate },
+ { "creditdebit", report_fmt_credit_debit },
+ { "currencycredit", report_fmt_currency_credit },
+ { "currencydebit", report_fmt_currency_debit },
+ { "currencycreditdebit", report_fmt_currency_credit_debit },
+};
+
+static const struct report_tag *
+find_report_tag(const char *name, size_t len)
+{
+ for (size_t i = 0; i < ARRAY_SIZE(report_tags); i++) {
+ if (memeqstr(name, len, report_tags[i].name))
+ return &report_tags[i];
+ }
+ return NULL;
+}
+
+struct report_format {
+ /* Produces a string: NULL means simply copy. */
+ const char *(**fmt)(const tal_t *ctx,
+ const struct bkpr *bkpr,
+ const struct income_event *e);
+ const char **str;
+ /* If fmt returns NULL, evaluate these instead. */
+ struct report_format **alt;
+};
+
+static void add_literal(struct report_format *f,
+ const char **start, const char *end)
+{
+ if (*start != end) {
+ tal_arr_expand(&f->fmt, NULL);
+ tal_arr_expand(&f->str,
+ tal_strndup(f->str, *start, end - *start));
+ tal_arr_expand(&f->alt, NULL);
+ *start = end;
+ }
+}
+
+static struct report_format *
+parse_report_format(const tal_t *ctx,
+ const char **start,
+ char term,
+ const char **err)
+{
+ const char *p;
+ struct report_format *f;
+
+ f = tal(ctx, struct report_format);
+ f->fmt = tal_arr(f, typeof(*f->fmt), 0);
+ f->str = tal_arr(f, const char *, 0);
+ f->alt = tal_arr(f, struct report_format *, 0);
+
+ p = *start;
+ while (*p != term) {
+ struct report_format *alt;
+ const struct report_tag *rt;
+
+ if (*p == '\0') {
+ *err = tal_fmt(ctx, "Unterminated tag");
+ return tal_free(f);
+ }
+
+ if (*p != '{') {
+ p++;
+ continue;
+ }
+
+ /* Escaped '{{' => literal '{' */
+ if (p[1] != term && p[1] == '{') {
+ char *lit;
+
+ lit = tal_strndup(f->str, *start, p - *start);
+ lit = tal_strcat(tmpctx, take(lit), "{");
+ tal_arr_expand(&f->fmt, NULL);
+ tal_arr_expand(&f->str, lit);
+ tal_arr_expand(&f->alt, NULL);
+ p += 2;
+ *start = p;
+ continue;
+ }
+
+ /* Emit preceding literal, if any. */
+ add_literal(f, start, p);
+
+ const char *endtag = p + 1 + strcspn(p+1, "?}");
+ if (*endtag == '\0') {
+ *err = tal_fmt(ctx, "Unterminated tag %s", p + 1);
+ return tal_free(f);
+ }
+
+ rt = find_report_tag(p + 1, endtag - (p + 1));
+ if (!rt) {
+ *err = tal_fmt(ctx,
+ "Unknown tag %.*s",
+ (int)(endtag - (p + 1)), p + 1);
+ return tal_free(f);
+ }
+
+ if (*endtag == '?') {
+ *start = endtag + 1;
+ alt = parse_report_format(f, start, '}', err);
+ if (!alt) {
+ /* Steal error upwards! */
+ tal_steal(ctx, *err);
+ return tal_free(f);
+ }
+ /* Consume final } */
+ (*start)++;
+ } else {
+ assert(*endtag == '}');
+ alt = NULL;
+ *start = endtag + 1;
+ }
+
+ tal_arr_expand(&f->fmt, rt->fmt);
+ tal_arr_expand(&f->str, NULL);
+ tal_arr_expand(&f->alt, alt);
+
+ p = *start;
+ }
+
+ add_literal(f, start, p);
+ return f;
+}
+
+struct command_result *param_report_format(struct command *cmd,
+ const char *name,
+ const char *buffer,
+ const jsmntok_t *tok,
+ struct report_format **format)
+{
+ const char *err, *start;
+ struct command_result *ret;
+
+ ret = param_escaped_string(cmd, name, buffer, tok, &start);
+ if (ret)
+ return ret;
+
+ *format = parse_report_format(cmd, &start, '\0', &err);
+ if (!*format)
+ return command_fail_badparam(cmd, name, buffer, tok, err);
+
+ return NULL;
+}
+
+struct command_result *param_escape_format(struct command *cmd,
+ const char *name,
+ const char *buffer,
+ const jsmntok_t *tok,
+ enum escape_format **escape)
+{
+ *escape = tal(cmd, enum escape_format);
+ if (json_tok_streq(buffer, tok, "csv")) {
+ **escape = REPORT_FMT_CSV;
+ } else if (json_tok_streq(buffer, tok, "none")) {
+ **escape = REPORT_FMT_NONE;
+ } else
+ return command_fail_badparam(cmd, name, buffer, tok,
+ "should be `csv` or `none`");
+ return NULL;
+}
+
+static char *escape_value(const tal_t *ctx,
+ const char *val TAKES,
+ enum escape_format esc)
+{
+ bool needs_quotes = false;
+ char *ret, *out;
+ const char *p;
+
+ switch (esc) {
+ case REPORT_FMT_NONE:
+ return tal_strdup(ctx, val);
+
+ case REPORT_FMT_CSV:
+ for (p = val; *p; p++) {
+ if (*p == ',' || *p == '"' || *p == '\n' || *p == '\r') {
+ needs_quotes = true;
+ break;
+ }
+ }
+
+ if (!needs_quotes)
+ return tal_strdup(ctx, val);
+
+ /* Worst case: doubling length plus " around plus nul term */
+ ret = tal_arr(ctx, char, 2 + strlen(val) * 2 + 1);
+ out = ret;
+ *(out++) = '"';
+ /* Quotes get doubled */
+ for (p = val; *p; p++) {
+ if (*p == '"')
+ *(out++) = '"';
+ *(out++) = *p;
+ }
+ *(out++) = '"';
+ *(out++) = '\0';
+ if (taken(val))
+ tal_free(val);
+ return ret;
+ }
+ abort();
+}
+
+static char *format_event(const tal_t *ctx,
+ const struct report_format *fmt,
+ enum escape_format esc,
+ const struct bkpr *bkpr,
+ const struct income_event *e)
+{
+ char *out = tal_strdup(ctx, "");
+
+ for (size_t i = 0; i < tal_count(fmt->fmt); i++) {
+ const char *v;
+
+ if (fmt->fmt[i] == NULL) {
+ assert(fmt->str[i] != NULL);
+ out = tal_strcat(ctx, take(out), fmt->str[i]);
+ continue;
+ }
+
+ v = fmt->fmt[i](tmpctx, bkpr, e);
+ if (v) {
+ v = escape_value(tmpctx, v, esc);
+ out = tal_strcat(ctx, take(out), v);
+ continue;
+ }
+
+ if (fmt->alt[i]) {
+ char *alt = format_event(tmpctx, fmt->alt[i], esc, bkpr, e);
+ out = tal_strcat(ctx, take(out), alt);
+ }
+ }
+
+ return out;
+}
+
+struct command_result *do_bkpr_report(struct command *cmd,
+ struct report_info *info)
+{
+ const struct bkpr *bkpr = bkpr_of(cmd->plugin);
+ struct income_event **events;
+ struct json_stream *js;
+
+ events = list_income_events(tmpctx, bkpr, cmd,
+ *info->start_time,
+ *info->end_time,
+ true);
+
+ js = jsonrpc_stream_success(cmd);
+ json_array_start(js, "report");
+ for (size_t i = 0; i < tal_count(info->headers); i++)
+ json_add_string(js, NULL, info->headers[i]);
+ for (size_t i = 0; i < tal_count(events); i++) {
+ char *line;
+
+ line = format_event(tmpctx, info->format, *info->escapes,
+ bkpr, events[i]);
+ json_add_string(js, NULL, line);
+ }
+ json_array_end(js);
+ /* Tell cli this is simple enough to be formatted flat for humans */
+ json_add_string(js, "format-hint", "simple");
+ return command_finished(cmd, js);
+}
diff --git a/plugins/bkpr/report.h b/plugins/bkpr/report.h
new file mode 100644
index 00000000..630c0c2a
--- /dev/null
+++ b/plugins/bkpr/report.h
@@ -0,0 +1,30 @@
+#ifndef LIGHTNING_PLUGINS_BKPR_REPORT_H
+#define LIGHTNING_PLUGINS_BKPR_REPORT_H
+#include "config.h"
+#include <ccan/short_types/short_types.h>
+#include <common/json_parse_simple.h>
+
+struct command;
+
+enum escape_format {
+ REPORT_FMT_NONE,
+ REPORT_FMT_CSV,
+};
+
+struct report_info {
+ struct report_format *format;
+ const char **headers;
+ enum escape_format *escapes;
+ u64 *start_time, *end_time;
+};
+
+struct command_result *do_bkpr_report(struct command *cmd,
+ struct report_info *info);
+
+struct command_result *param_report_format(struct command *cmd, const char *name,
+ const char *buffer, const jsmntok_t *tok,
+ struct report_format **format);
+struct command_result *param_escape_format(struct command *cmd, const char *name,
+ const char *buffer, const jsmntok_t *tok,
+ enum escape_format **escape);
+#endif /* LIGHTNING_PLUGINS_BKPR_REPORT_H */
diff --git a/plugins/bkpr/test/run-format_event.c b/plugins/bkpr/test/run-format_event.c
new file mode 100644
index 00000000..36256659
--- /dev/null
+++ b/plugins/bkpr/test/run-format_event.c
@@ -0,0 +1,250 @@
+/* plugins/bkpr/test/run-report.c */
+#include "config.h"
+#include <assert.h>
+#include <ccan/str/str.h>
+#include <ccan/tal/str/str.h>
+#include <common/amount.h>
+#include <common/json_parse.h>
+#include <common/setup.h>
+#include <plugins/bkpr/bookkeeper.h>
+
+#include "../report.c"
+
+/* AUTOGENERATED MOCKS START */
+/* Generated stub for bkpr_of */
+struct bkpr *bkpr_of(struct plugin *plugin UNNEEDED)
+{ fprintf(stderr, "bkpr_of called!\n"); abort(); }
+/* Generated stub for command_check_done */
+struct command_result *command_check_done(struct command *cmd)
+
+{ fprintf(stderr, "command_check_done called!\n"); abort(); }
+/* Generated stub for command_check_only */
+bool command_check_only(const struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_check_only called!\n"); abort(); }
+/* Generated stub for command_deprecated_in_ok */
+bool command_deprecated_in_ok(struct command *cmd UNNEEDED,
+ const char *param UNNEEDED,
+ const char *depr_start UNNEEDED,
+ const char *depr_end UNNEEDED)
+{ fprintf(stderr, "command_deprecated_in_ok called!\n"); abort(); }
+/* Generated stub for command_dev_apis */
+bool command_dev_apis(const struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_dev_apis called!\n"); abort(); }
+/* Generated stub for command_fail */
+struct command_result *command_fail(struct command *cmd UNNEEDED, enum jsonrpc_errcode code UNNEEDED,
+ const char *fmt UNNEEDED, ...)
+
+{ fprintf(stderr, "command_fail called!\n"); abort(); }
+/* Generated stub for command_filter_ptr */
+struct json_filter **command_filter_ptr(struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_filter_ptr called!\n"); abort(); }
+/* Generated stub for command_finished */
+struct command_result *command_finished(struct command *cmd UNNEEDED, struct json_stream *response)
+
+{ fprintf(stderr, "command_finished called!\n"); abort(); }
+/* Generated stub for command_log */
+void command_log(struct command *cmd UNNEEDED, enum log_level level UNNEEDED,
+ const char *fmt UNNEEDED, ...)
+
+{ fprintf(stderr, "command_log called!\n"); abort(); }
+/* Generated stub for command_set_usage */
+void command_set_usage(struct command *cmd UNNEEDED, const char *usage UNNEEDED)
+{ fprintf(stderr, "command_set_usage called!\n"); abort(); }
+/* Generated stub for command_usage_only */
+bool command_usage_only(const struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_usage_only called!\n"); abort(); }
+/* Generated stub for currencyrate_str */
+const char *currencyrate_str(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr UNNEEDED,
+ u64 timestamp UNNEEDED,
+ const struct amount_msat *msat UNNEEDED)
+{ fprintf(stderr, "currencyrate_str called!\n"); abort(); }
+/* Generated stub for jsonrpc_stream_success */
+struct json_stream *jsonrpc_stream_success(struct command *cmd)
+
+{ fprintf(stderr, "jsonrpc_stream_success called!\n"); abort(); }
+/* Generated stub for list_income_events */
+struct income_event **list_income_events(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr UNNEEDED,
+ struct command *cmd UNNEEDED,
+ u64 start_time UNNEEDED,
+ u64 end_time UNNEEDED,
+ bool consolidate_fees UNNEEDED)
+{ fprintf(stderr, "list_income_events called!\n"); abort(); }
+/* AUTOGENERATED MOCKS END */
+
+static void test_format_event_literal_only(void)
+{
+ struct report_format *f = tal(tmpctx, struct report_format);
+ struct income_event *e = tal(tmpctx, struct income_event);
+ struct bkpr *bkpr = tal(tmpctx, struct bkpr);
+ char *s;
+
+ f->fmt = tal_arr(f, typeof(*f->fmt), 1);
+ f->str = tal_arr(f, const char *, 1);
+ f->alt = tal_arr(f, struct report_format *, 1);
+
+ f->fmt[0] = NULL;
+ f->str[0] = tal_strdup(f->str, "hello");
+ f->alt[0] = NULL;
+
+ s = format_event(tmpctx, f, REPORT_FMT_NONE, bkpr, e);
+ assert(streq(s, "hello"));
+}
+
+static void test_format_event_simple_tag(void)
+{
+ struct report_format *f = tal(tmpctx, struct report_format);
+ struct income_event *e = tal(tmpctx, struct income_event);
+ struct bkpr *bkpr = tal(tmpctx, struct bkpr);
+ char *s;
+
+ e->tag = "invoice";
+
+ f->fmt = tal_arr(f, typeof(*f->fmt), 3);
+ f->str = tal_arr(f, const char *, 3);
+ f->alt = tal_arr(f, struct report_format *, 3);
+
+ f->fmt[0] = NULL;
+ f->str[0] = tal_strdup(f->str, "tag=");
+ f->alt[0] = NULL;
+
+ f->fmt[1] = report_fmt_tag;
+ f->str[1] = NULL;
+ f->alt[1] = NULL;
+
+ f->fmt[2] = NULL;
+ f->str[2] = tal_strdup(f->str, "!");
+ f->alt[2] = NULL;
+
+ s = format_event(tmpctx, f, REPORT_FMT_NONE, bkpr, e);
+ assert(streq(s, "tag=invoice!"));
+}
+
+static void test_format_event_fallback(void)
+{
+ struct report_format *f = tal(tmpctx, struct report_format);
+ struct report_format *alt = tal(f, struct report_format);
+ struct income_event *e = tal(tmpctx, struct income_event);
+ struct bkpr *bkpr = tal(tmpctx, struct bkpr);
+ char *s;
+
+ alt->fmt = tal_arr(alt, typeof(*alt->fmt), 1);
+ alt->str = tal_arr(alt, const char *, 1);
+ alt->alt = tal_arr(alt, struct report_format *, 1);
+ alt->fmt[0] = NULL;
+ alt->str[0] = tal_strdup(alt->str, "NONE");
+ alt->alt[0] = NULL;
+
+ f->fmt = tal_arr(f, typeof(*f->fmt), 1);
+ f->str = tal_arr(f, const char *, 1);
+ f->alt = tal_arr(f, struct report_format *, 1);
+
+ f->fmt[0] = report_fmt_payment_id;
+ f->str[0] = NULL;
+ f->alt[0] = alt;
+
+ e->payment_id = NULL;
+ s = format_event(tmpctx, f, REPORT_FMT_NONE, bkpr, e);
+ assert(streq(s, "NONE"));
+}
+
+static void test_format_event_nested_fallback(void)
+{
+ struct report_format *f = tal(tmpctx, struct report_format);
+ struct report_format *alt1 = tal(f, struct report_format);
+ struct report_format *alt2 = tal(f, struct report_format);
+ struct income_event *e = tal(tmpctx, struct income_event);
+ struct bkpr *bkpr = tal(tmpctx, struct bkpr);
+ char *s;
+
+ e->txid = NULL;
+ e->desc = NULL;
+
+ alt2->fmt = tal_arr(alt2, typeof(*alt2->fmt), 1);
+ alt2->str = tal_arr(alt2, const char *, 1);
+ alt2->alt = tal_arr(alt2, struct report_format *, 1);
+ alt2->fmt[0] = NULL;
+ alt2->str[0] = tal_strdup(alt2->str, "LAST");
+ alt2->alt[0] = NULL;
+
+ alt1->fmt = tal_arr(alt1, typeof(*alt1->fmt), 1);
+ alt1->str = tal_arr(alt1, const char *, 1);
+ alt1->alt = tal_arr(alt1, struct report_format *, 1);
+ alt1->fmt[0] = report_fmt_txid;
+ alt1->str[0] = NULL;
+ alt1->alt[0] = alt2;
+
+ f->fmt = tal_arr(f, typeof(*f->fmt), 1);
+ f->str = tal_arr(f, const char *, 1);
+ f->alt = tal_arr(f, struct report_format *, 1);
+ f->fmt[0] = report_fmt_desc;
+ f->str[0] = NULL;
+ f->alt[0] = alt1;
+
+ s = format_event(tmpctx, f, REPORT_FMT_NONE, bkpr, e);
+ assert(streq(s, "LAST"));
+}
+
+static void test_format_event_csv_escape_value(void)
+{
+ struct report_format *f = tal(tmpctx, struct report_format);
+ struct income_event *e = tal(tmpctx, struct income_event);
+ struct bkpr *bkpr = tal(tmpctx, struct bkpr);
+ char *s;
+
+ e->desc = "hello, \"world\"";
+
+ f->fmt = tal_arr(f, typeof(*f->fmt), 1);
+ f->str = tal_arr(f, const char *, 1);
+ f->alt = tal_arr(f, struct report_format *, 1);
+
+ f->fmt[0] = report_fmt_desc;
+ f->str[0] = NULL;
+ f->alt[0] = NULL;
+
+ s = format_event(tmpctx, f, REPORT_FMT_CSV, bkpr, e);
+ assert(streq(s, "\"hello, \"\"world\"\"\""));
+}
+
+static void test_format_event_missing_no_fallback(void)
+{
+ struct report_format *f = tal(tmpctx, struct report_format);
+ struct income_event *e = tal(tmpctx, struct income_event);
+ struct bkpr *bkpr = tal(tmpctx, struct bkpr);
+ char *s;
+
+ e->payment_id = NULL;
+
+ f->fmt = tal_arr(f, typeof(*f->fmt), 3);
+ f->str = tal_arr(f, const char *, 3);
+ f->alt = tal_arr(f, struct report_format *, 3);
+
+ f->fmt[0] = NULL;
+ f->str[0] = tal_strdup(f->str, "A");
+ f->alt[0] = NULL;
+
+ f->fmt[1] = report_fmt_payment_id;
+ f->str[1] = NULL;
+ f->alt[1] = NULL;
+
+ f->fmt[2] = NULL;
+ f->str[2] = tal_strdup(f->str, "B");
+ f->alt[2] = NULL;
+
+ s = format_event(tmpctx, f, REPORT_FMT_NONE, bkpr, e);
+ assert(streq(s, "AB"));
+}
+
+int main(int argc, char *argv[])
+{
+ common_setup(argv[0]);
+ test_format_event_literal_only();
+ test_format_event_simple_tag();
+ test_format_event_fallback();
+ test_format_event_nested_fallback();
+ test_format_event_csv_escape_value();
+ test_format_event_missing_no_fallback();
+
+ common_shutdown();
+}
diff --git a/plugins/bkpr/test/run-parse_report.c b/plugins/bkpr/test/run-parse_report.c
new file mode 100644
index 00000000..eb5e91cd
--- /dev/null
+++ b/plugins/bkpr/test/run-parse_report.c
@@ -0,0 +1,245 @@
+#include "config.h"
+#include "../report.c"
+#include <common/setup.h>
+#include <common/utils.h>
+#include <stdio.h>
+
+/* AUTOGENERATED MOCKS START */
+/* Generated stub for bkpr_of */
+struct bkpr *bkpr_of(struct plugin *plugin UNNEEDED)
+{ fprintf(stderr, "bkpr_of called!\n"); abort(); }
+/* Generated stub for command_check_done */
+struct command_result *command_check_done(struct command *cmd)
+
+{ fprintf(stderr, "command_check_done called!\n"); abort(); }
+/* Generated stub for command_check_only */
+bool command_check_only(const struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_check_only called!\n"); abort(); }
+/* Generated stub for command_deprecated_in_ok */
+bool command_deprecated_in_ok(struct command *cmd UNNEEDED,
+ const char *param UNNEEDED,
+ const char *depr_start UNNEEDED,
+ const char *depr_end UNNEEDED)
+{ fprintf(stderr, "command_deprecated_in_ok called!\n"); abort(); }
+/* Generated stub for command_dev_apis */
+bool command_dev_apis(const struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_dev_apis called!\n"); abort(); }
+/* Generated stub for command_fail */
+struct command_result *command_fail(struct command *cmd UNNEEDED, enum jsonrpc_errcode code UNNEEDED,
+ const char *fmt UNNEEDED, ...)
+
+{ fprintf(stderr, "command_fail called!\n"); abort(); }
+/* Generated stub for command_filter_ptr */
+struct json_filter **command_filter_ptr(struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_filter_ptr called!\n"); abort(); }
+/* Generated stub for command_finished */
+struct command_result *command_finished(struct command *cmd UNNEEDED, struct json_stream *response)
+
+{ fprintf(stderr, "command_finished called!\n"); abort(); }
+/* Generated stub for command_log */
+void command_log(struct command *cmd UNNEEDED, enum log_level level UNNEEDED,
+ const char *fmt UNNEEDED, ...)
+
+{ fprintf(stderr, "command_log called!\n"); abort(); }
+/* Generated stub for command_set_usage */
+void command_set_usage(struct command *cmd UNNEEDED, const char *usage UNNEEDED)
+{ fprintf(stderr, "command_set_usage called!\n"); abort(); }
+/* Generated stub for command_usage_only */
+bool command_usage_only(const struct command *cmd UNNEEDED)
+{ fprintf(stderr, "command_usage_only called!\n"); abort(); }
+/* Generated stub for currencyrate_str */
+const char *currencyrate_str(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr UNNEEDED,
+ u64 timestamp UNNEEDED,
+ const struct amount_msat *msat UNNEEDED)
+{ fprintf(stderr, "currencyrate_str called!\n"); abort(); }
+/* Generated stub for jsonrpc_stream_success */
+struct json_stream *jsonrpc_stream_success(struct command *cmd)
+
+{ fprintf(stderr, "jsonrpc_stream_success called!\n"); abort(); }
+/* Generated stub for list_income_events */
+struct income_event **list_income_events(const tal_t *ctx UNNEEDED,
+ const struct bkpr *bkpr UNNEEDED,
+ struct command *cmd UNNEEDED,
+ u64 start_time UNNEEDED,
+ u64 end_time UNNEEDED,
+ bool consolidate_fees UNNEEDED)
+{ fprintf(stderr, "list_income_events called!\n"); abort(); }
+/* AUTOGENERATED MOCKS END */
+
+static void test_parse_report_format_simple(void)
+{
+ struct report_format *f;
+ const char *err, *start = "hello {tag} world";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(f);
+ assert(streq(start, ""));
+
+ assert(tal_count(f->fmt) == 3);
+ assert(f->fmt[0] == NULL);
+ assert(streq(f->str[0], "hello "));
+ assert(f->alt[0] == NULL);
+
+ assert(f->fmt[1] == report_fmt_tag);
+ assert(f->str[1] == NULL);
+ assert(f->alt[1] == NULL);
+
+ assert(f->fmt[2] == NULL);
+ assert(streq(f->str[2], " world"));
+ assert(f->alt[2] == NULL);
+}
+
+static void test_parse_report_format_escaped_open_brace(void)
+{
+ struct report_format *f;
+ const char *err, *start = "a{{b";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(f);
+ assert(streq(start, ""));
+
+ assert(tal_count(f->fmt) == 2);
+ assert(f->fmt[0] == NULL);
+ assert(f->fmt[1] == NULL);
+ assert(tal_count(f->str) == 2);
+ assert(streq(f->str[0], "a{"));
+ assert(streq(f->str[1], "b"));
+ assert(tal_count(f->alt) == 2);
+ assert(f->alt[0] == NULL);
+ assert(f->alt[1] == NULL);
+}
+
+static void test_parse_report_format_alt_simple(void)
+{
+ struct report_format *f;
+ const char *err, *start = "{description?NONE}";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(f);
+ assert(streq(start, ""));
+
+ assert(tal_count(f->fmt) == 1);
+ assert(f->fmt[0] == report_fmt_desc);
+ assert(f->str[0] == NULL);
+ assert(f->alt[0] != NULL);
+
+ assert(tal_count(f->alt[0]->fmt) == 1);
+ assert(f->alt[0]->fmt[0] == NULL);
+ assert(streq(f->alt[0]->str[0], "NONE"));
+ assert(f->alt[0]->alt[0] == NULL);
+}
+
+static void test_parse_report_format_alt_nested(void)
+{
+ struct report_format *f, *a1, *a2;
+ const char *err, *start = "{description?{txid?{outpoint?NONE}}}";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(f);
+ assert(streq(start, ""));
+
+ assert(tal_count(f->fmt) == 1);
+ assert(f->fmt[0] == report_fmt_desc);
+ assert(f->str[0] == NULL);
+ assert(f->alt[0] != NULL);
+
+ a1 = f->alt[0];
+ assert(tal_count(a1->fmt) == 1);
+ assert(a1->fmt[0] == report_fmt_txid);
+ assert(a1->str[0] == NULL);
+ assert(a1->alt[0] != NULL);
+
+ a2 = a1->alt[0];
+ assert(tal_count(a2->fmt) == 1);
+ assert(a2->fmt[0] == report_fmt_outpoint);
+ assert(a2->str[0] == NULL);
+ assert(a2->alt[0] != NULL);
+
+ assert(tal_count(a2->alt[0]->fmt) == 1);
+ assert(a2->alt[0]->fmt[0] == NULL);
+ assert(streq(a2->alt[0]->str[0], "NONE"));
+ assert(a2->alt[0]->alt[0] == NULL);
+}
+
+static void test_parse_report_format_alt_with_suffix(void)
+{
+ struct report_format *f, *a1;
+ const char *err, *start = "{description?{txid}X}Y";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(f);
+ assert(streq(start, ""));
+
+ assert(tal_count(f->fmt) == 2);
+ assert(f->fmt[0] == report_fmt_desc);
+ assert(f->alt[0] != NULL);
+ assert(f->fmt[1] == NULL);
+ assert(streq(f->str[1], "Y"));
+
+ a1 = f->alt[0];
+ assert(tal_count(a1->fmt) == 2);
+ assert(a1->fmt[0] == report_fmt_txid);
+ assert(a1->alt[0] == NULL);
+ assert(a1->fmt[1] == NULL);
+ assert(streq(a1->str[1], "X"));
+}
+
+static void test_parse_report_format_unknown_tag(void)
+{
+ struct report_format *f;
+ const char *err, *start = "{nope}";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(!f);
+ assert(err);
+ assert(strstr(err, "Unknown tag nope"));
+}
+
+static void test_parse_report_format_unterminated_tag(void)
+{
+ struct report_format *f;
+ const char *err, *start = "{description";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(!f);
+ assert(err);
+ assert(strstr(err, "Unterminated tag"));
+}
+
+static void test_parse_report_format_unterminated_nested_alt(void)
+{
+ struct report_format *f;
+ const char *err, *start = "{description?{txid}";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(!f);
+ assert(err);
+ assert(strstr(err, "Unterminated tag"));
+}
+
+static void test_parse_report_format_unknown_nested_tag(void)
+{
+ struct report_format *f;
+ const char *err, *start = "{description?{nope}}";
+
+ f = parse_report_format(tmpctx, &start, '\0', &err);
+ assert(!f);
+ assert(err);
+ assert(strstr(err, "Unknown tag nope"));
+}
+
+int main(int argc, char *argv[])
+{
+ common_setup(argv[0]);
+ test_parse_report_format_simple();
+ test_parse_report_format_escaped_open_brace();
+ test_parse_report_format_alt_simple();
+ test_parse_report_format_alt_nested();
+ test_parse_report_format_alt_with_suffix();
+ test_parse_report_format_unknown_tag();
+ test_parse_report_format_unterminated_tag();
+ test_parse_report_format_unterminated_nested_alt();
+ test_parse_report_format_unknown_nested_tag();
+ common_shutdown();
+}
diff --git a/tests/test_bookkeeper.py b/tests/test_bookkeeper.py
index d2a2e714..c62210e6 100644
--- a/tests/test_bookkeeper.py
+++ b/tests/test_bookkeeper.py
@@ -6,9 +6,13 @@ from utils import (
sync_blockheight, wait_for, only_one, first_channel_id, TIMEOUT
)
+from datetime import datetime
from pathlib import Path
+import csv
+import io
import os
import pytest
+import subprocess
import time
import unittest
@@ -1174,7 +1178,7 @@ def test_migration_no_bkpr(node_factory, bitcoind):
@unittest.skipIf(TEST_NETWORK != 'regtest', "External wallet support doesn't work with elements yet.")
-def test_listincome_timebox(node_factory, bitcoind):
+def test_listincome_and_report_timebox(node_factory, bitcoind):
l1 = node_factory.get_node()
addr = l1.rpc.newaddr()['p2tr']
@@ -1203,6 +1207,16 @@ def test_listincome_timebox(node_factory, bitcoind):
incomes = l1.rpc.bkpr_listincome(end_time=first_one)['income_events']
assert [i for i in incomes if i['timestamp'] > first_one] == []
+ # Test bkpr-report time bracketing too.
+ report = l1.rpc.bkpr_report(format="{localtime},{tag},{creditdebit}", end_time=first_one)['report']
+ assert [r for r in report if datetime.strptime(r.split(',')[0], "%Y-%m-%d %H:%M:%S").timestamp() > first_one] == []
+
+ first_entries = l1.rpc.bkpr_report(format="{localtime},{tag},{creditdebit}", end_time=first_one)['report']
+ last_entries = l1.rpc.bkpr_report(format="{localtime},{tag},{creditdebit}", start_time=first_one)['report']
+ all_entries = l1.rpc.bkpr_report(format="{localtime},{tag},{creditdebit}")['report']
+
+ assert first_entries + last_entries == all_entries
+
@unittest.skipIf(TEST_NETWORK != 'regtest', "Snapshots are bitcoin regtest.")
@unittest.skipIf(os.getenv('TEST_DB_PROVIDER', 'sqlite3') != 'sqlite3', "uses snapshots")
@@ -1225,3 +1239,111 @@ def test_bkpr_parallel(node_factory, bitcoind, executor):
acctevents_after = l1.rpc.bkpr_listaccountevents()
assert acctevents_after == acctevents_before
+
+
+def test_bkpr_report_tags_and_fallback(node_factory):
+ l1, l2 = node_factory.line_graph(2, opts={'bkpr-currency': 'USD'})
+
+ inv = l2.rpc.invoice(100000, "test_bkpr_report_tags_and_fallback", 'desc with "quotes"')
+ l1.rpc.pay(inv["bolt11"])
+ wait_for(lambda: only_one(l1.rpc.listpeerchannels(l2.info['id'])['channels'])['htlcs'] == [])
+
+ res = l1.rpc.call(
+ "bkpr-report",
+ {
+ "format": "{tag}|{account}|{outpoint?NONE}|{txid?NONE}|{payment_id?NONE}|{bkpr-currency?NONE}|{currencyrate?NONE}",
+ "headers": ['tag",account,outpoint,txid,payment_id,bkpr-currency,currencyrate', ",,,,,,"],
+ },
+ )
+
+ assert "report" in res
+ # Header lines copied literally (including ")
+ assert res["report"][0] == 'tag",account,outpoint,txid,payment_id,bkpr-currency,currencyrate'
+ assert res["report"][1] == ",,,,,,"
+ assert res["format-hint"] == "simple"
+
+ rows = [line.split("|") for line in res["report"][2:]]
+ assert all(len(r) == 7 for r in rows)
+
+ # Currency fields should be populated in this setup.
+ assert all(r[5] == "USD" for r in rows)
+ for r in rows:
+ if r[6] != "NONE":
+ assert float(r[6]) > 0
+
+ # At least one fallback should have been used for channel events.
+ assert any(r[2] == "NONE" or r[3] == "NONE" or r[4] == "NONE" for r in rows)
+
+ # Fancier fields should work, too
+ res = l1.rpc.bkpr_report(format="{tag}|{account}|{credit}|{debit}|{creditdebit}|{currencycredit}|{currencydebit}|{currencycreditdebit}")
+ rows = [line.split("|") for line in res["report"]]
+
+ for r in rows:
+ assert len(r) == 8
+ # Credit or debit?
+ if float(r[2]) > 0:
+ assert r[4] == '+' + r[2]
+ assert float(r[5]) > 0
+ assert r[6] == '0.00'
+ assert r[7] == '+' + r[5]
+ else:
+ assert r[4] == '-' + r[3]
+ assert float(r[6]) > 0
+ assert r[5] == '0.00'
+ assert r[7] == '-' + r[6]
+
+
+def test_bkpr_report_invoice(node_factory):
+ l1, l2 = node_factory.line_graph(2, opts={'bkpr-currency': 'USD'})
+ inv = l2.rpc.invoice(123456, "test", "test_bkpr_report_invoice")['bolt11']
+ l1.rpc.xpay(inv)
+
+ # Make sure bookkeeper saw the event.
+ wait_for(lambda: any([e['tag'] == 'invoice' for e in l1.rpc.bkpr_listincome()['income_events']]))
+
+ # This should fail!
+ with pytest.raises(RpcError, match=r'Unknown tag acc'):
+ l1.rpc.bkpr_report(headers=["Tag,Account,Description,Credit,Debit,BTC/USD,Credit (USD),Debit(USD)"], format="{tag},{acc},{description},{creditdebit},", escape='csv')
+
+ lines = l1.rpc.bkpr_report(headers=["Tag,Account,Description,Credit,Debit,BTC/USD,Credit (USD),Debit(USD)"], format="{tag},{account},{description},{creditdebit},", escape='csv')['report']
+
+ invline = only_one([line for line in lines if line.startswith('invoice,')])
+
+ cid = only_one(l1.rpc.listpeerchannels()['channels'])['channel_id']
+ assert invline == f"invoice,{cid},test_bkpr_report_invoice,-0.00000123456,"
+
+ # Test nested tags while we're here!
+ lines = l1.rpc.bkpr_report(format="{tag},{account},{description},{outpoint},{txid},{description?{outpoint?txid: {txid?UNKNOWN}}},{creditdebit}", escape='csv')['report']
+ for l in lines[1:]:
+ parts = l.split(',')
+ if parts[2] != '':
+ assert parts[5] == parts[2]
+ else:
+ if parts[3] != '':
+ assert parts[5] == parts[3]
+ else:
+ assert parts[5] == 'txid: ' + parts[4]
+
+
+def test_bkpr_report_lightning_cli_csv(node_factory):
+ l1, l2 = node_factory.line_graph(2)
+
+ # Give desc something awkward so CSV escaping matters if it shows up.
+ inv = l2.rpc.invoice(100000, "test_bkpr_report_lightning_cli_csv", 'hello, "csv"')
+ l1.rpc.pay(inv["bolt11"])
+ wait_for(lambda: only_one(l1.rpc.listpeerchannels(l2.info['id'])['channels'])['htlcs'] == [])
+
+ # Single-column CSV is enough to validate the CLI path and escaping.
+ res = subprocess.check_output(["cli/lightning-cli",
+ f"--network={TEST_NETWORK}",
+ f"--lightning-dir={l1.daemon.lightning_dir}",
+ "-k",
+ "bkpr-report",
+ 'format={description?"hello, ""fallback"""},{tag},',
+ "escape=csv"],
+ text=True)
+
+ # Must parse cleanly as CSV, one row per returned line.
+ parsed = [next(csv.reader(io.StringIO(line))) for line in res.splitlines()]
+ assert parsed
+ assert all(len(row) == 3 for row in parsed)
Why this scored 18/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.