What changed, and why it matters
This commit removes the 'output blackballing' feature from the Monero wallet software. Blackballing let users mark certain blockchain outputs as 'spent' so their own wallet would avoid picking them as decoys in ring signatures. The change deletes the supporting utility, the wallet commands, the API methods, and the database table used to store these marks. It is a feature-removal cleanup, not a fix for an active security vulnerability in the code itself.
Treat as a routine feature-removal commit. Reviewers should confirm that no other code still references the deleted blackball API or database table, and that downstream documentation and release notes are updated to announce the removal of the monero-blockchain-mark-spent-outputs tool and the mark_output_spent commands.
Security signals we found
Large feature deletion (12 files, -2259 lines) with no replacement logic
Removed local LMDB blackball table and associated read/write/query operations
Removed wallet mixin-selection logic that skipped user-marked 'spent' outputs
No new input parsing, network handling, or cryptographic code introduced
Evidence from the diff
The commit deletes the monero-blockchain-mark-spent-outputs utility (blockchain_blackball.cpp), removes CLI commands mark_output_spent/mark_output_unspent/is_output_spent from simplewallet, removes blackballOutput/unblackballOutput/blackballOutputs from the wallet API, and strips the blackball LMDB table and worker logic from ringdb. In wallet2’s get_outs() mixin selection loop it also removes the two-pass logic that first avoided then allowed blackballed outputs. No new code paths are added; the change is purely subtractive. The removed feature relied on a local, user-populated database of suspected-spent outputs and was never consensus-critical.
Changed components
src/blockchain_utilities/blockchain_blackball.cppsrc/blockchain_utilities/CMakeLists.txtsrc/simplewallet/simplewallet.cppsrc/simplewallet/simplewallet.hsrc/wallet/api/wallet.cppsrc/wallet/api/wallet.hsrc/wallet/api/wallet2_api.hsrc/wallet/ringdb.cppsrc/wallet/ringdb.hsrc/wallet/wallet2.cppsrc/wallet/wallet2.htests/unit_tests/ringdb.cppInspect captured patch +3 / −2259
diff --git a/src/blockchain_utilities/CMakeLists.txt b/src/blockchain_utilities/CMakeLists.txt
index 66c65c4..dc219be 100644
--- a/src/blockchain_utilities/CMakeLists.txt
+++ b/src/blockchain_utilities/CMakeLists.txt
@@ -57,19 +57,6 @@ monero_private_headers(blockchain_export
${blockchain_export_private_headers})
-set(blockchain_blackball_sources
- blockchain_blackball.cpp
- )
-
-set(blockchain_blackball_private_headers
- bootstrap_file.h
- blocksdat_file.h
- bootstrap_serialization.h
- )
-
-monero_private_headers(blockchain_blackball
- ${blockchain_blackball_private_headers})
-
set(blockchain_usage_sources
blockchain_usage.cpp
)
@@ -181,29 +168,6 @@ set_property(TARGET blockchain_export
OUTPUT_NAME "monero-blockchain-export")
install(TARGETS blockchain_export DESTINATION bin)
-monero_add_executable(blockchain_blackball
- ${blockchain_blackball_sources}
- ${blockchain_blackball_private_headers})
-
-target_link_libraries(blockchain_blackball
- PRIVATE
- wallet
- cryptonote_core
- blockchain_db
- version
- epee
- ${Boost_FILESYSTEM_LIBRARY}
- ${Boost_SYSTEM_LIBRARY}
- ${Boost_THREAD_LIBRARY}
- ${CMAKE_THREAD_LIBS_INIT}
- ${EXTRA_LIBRARIES})
-
-set_property(TARGET blockchain_blackball
- PROPERTY
- OUTPUT_NAME "monero-blockchain-mark-spent-outputs")
-install(TARGETS blockchain_blackball DESTINATION bin)
-
-
monero_add_executable(blockchain_usage
${blockchain_usage_sources}
${blockchain_usage_private_headers})
diff --git a/src/blockchain_utilities/blockchain_blackball.cpp b/src/blockchain_utilities/blockchain_blackball.cpp
deleted file mode 100644
index 38c4706..0000000
--- a/src/blockchain_utilities/blockchain_blackball.cpp
+++ /dev/null
@@ -1,1712 +0,0 @@
-// Copyright (c) 2014-2024, The Monero Project
-//
-// All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without modification, are
-// permitted provided that the following conditions are met:
-//
-// 1. Redistributions of source code must retain the above copyright notice, this list of
-// conditions and the following disclaimer.
-//
-// 2. Redistributions in binary form must reproduce the above copyright notice, this list
-// of conditions and the following disclaimer in the documentation and/or other
-// materials provided with the distribution.
-//
-// 3. Neither the name of the copyright holder nor the names of its contributors may be
-// used to endorse or promote products derived from this software without specific
-// prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
-// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-// MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
-// THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
-// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
-// STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
-// THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-#include <boost/range/adaptor/transformed.hpp>
-#include <boost/algorithm/string.hpp>
-#include <boost/filesystem.hpp>
-#include "common/unordered_containers_boost_serialization.h"
-#include "common/command_line.h"
-#include "common/varint.h"
-#include "serialization/crypto.h"
-#include "cryptonote_core/tx_pool.h"
-#include "cryptonote_core/cryptonote_core.h"
-#include "cryptonote_core/blockchain.h"
-#include "blockchain_db/blockchain_db.h"
-#include "wallet/ringdb.h"
-#include "version.h"
-
-#undef MONERO_DEFAULT_LOG_CATEGORY
-#define MONERO_DEFAULT_LOG_CATEGORY "bcutil"
-
-namespace po = boost::program_options;
-using namespace epee;
-using namespace cryptonote;
-
-static uint64_t records_per_sync = 200;
-static uint64_t db_flags = 0;
-static MDB_dbi dbi_relative_rings;
-static MDB_dbi dbi_outputs;
-static MDB_dbi dbi_processed_txidx;
-static MDB_dbi dbi_spent;
-static MDB_dbi dbi_per_amount;
-static MDB_dbi dbi_ring_instances;
-static MDB_dbi dbi_stats;
-static MDB_env *env = NULL;
-
-struct output_data
-{
- uint64_t amount;
- uint64_t offset;
- output_data(): amount(0), offset(0) {}
- output_data(uint64_t a, uint64_t i): amount(a), offset(i) {}
- bool operator==(const output_data &other) const { return other.amount == amount && other.offset == offset; }
-};
-
-//
-// relative_rings: key_image -> vector<uint64_t>
-// outputs: 128 bits -> set of key images
-// processed_txidx: string -> uint64_t
-// spent: amount -> offset
-// ring_instances: vector<uint64_t> -> uint64_t
-// stats: string -> arbitrary
-//
-
-static bool parse_db_sync_mode(std::string db_sync_mode)
-{
- std::vector<std::string> options;
- boost::trim(db_sync_mode);
- boost::split(options, db_sync_mode, boost::is_any_of(" :"));
-
- for(const auto &option : options)
- MDEBUG("option: " << option);
-
- // default to fast:async:1
- uint64_t DEFAULT_FLAGS = DBF_FAST;
-
- if(options.size() == 0)
- {
- // default to fast:async:1
- db_flags = DEFAULT_FLAGS;
- }
-
- bool safemode = false;
- if(options.size() >= 1)
- {
- if(options[0] == "safe")
- {
- safemode = true;
- db_flags = DBF_SAFE;
- }
- else if(options[0] == "fast")
- {
- db_flags = DBF_FAST;
- }
- else if(options[0] == "fastest")
- {
- db_flags = DBF_FASTEST;
- records_per_sync = 1000; // default to fastest:async:1000
- }
- else
- db_flags = DEFAULT_FLAGS;
- }
-
- if(options.size() >= 2 && !safemode)
- {
- char *endptr;
- uint64_t bps = strtoull(options[1].c_str(), &endptr, 0);
- if (*endptr == '\0')
- records_per_sync = bps;
- }
-
- return true;
-}
-
-static std::string get_default_db_path()
-{
- boost::filesystem::path dir = tools::get_default_data_dir();
- // remove .bitmonero, replace with .shared-ringdb
- dir = dir.remove_filename();
- dir /= ".shared-ringdb";
- return dir.string();
-}
-
-static std::string get_cache_filename(boost::filesystem::path filename)
-{
- if (!boost::filesystem::is_directory(filename))
- filename.remove_filename();
- return filename.string();
-}
-
-static int compare_hash32(const MDB_val *a, const MDB_val *b)
-{
- const uint32_t *va = (const uint32_t*) a->mv_data;
- const uint32_t *vb = (const uint32_t*) b->mv_data;
- for (int n = 7; n >= 0; n--)
- {
- if (va[n] == vb[n])
- continue;
- return va[n] < vb[n] ? -1 : 1;
- }
-
- return 0;
-}
-
-int compare_uint64(const MDB_val *a, const MDB_val *b)
-{
- const uint64_t va = *(const uint64_t *)a->mv_data;
- const uint64_t vb = *(const uint64_t *)b->mv_data;
- return (va < vb) ? -1 : va > vb;
-}
-
-static int compare_double64(const MDB_val *a, const MDB_val *b)
-{
- const uint64_t va = *(const uint64_t*) a->mv_data;
- const uint64_t vb = *(const uint64_t*) b->mv_data;
- if (va == vb)
- {
- const uint64_t va = ((const uint64_t*) a->mv_data)[1];
- const uint64_t vb = ((const uint64_t*) b->mv_data)[1];
- return va < vb ? -1 : va > vb;
- }
- return va < vb ? -1 : va > vb;
-}
-
-static int resize_env(const char *db_path)
-{
- MDB_envinfo mei;
- MDB_stat mst;
- int ret;
-
- size_t needed = 1000ul * 1024 * 1024; // at least 1000 MB
-
- ret = mdb_env_info(env, &mei);
- if (ret)
- return ret;
- ret = mdb_env_stat(env, &mst);
- if (ret)
- return ret;
- uint64_t size_used = mst.ms_psize * mei.me_last_pgno;
- uint64_t mapsize = mei.me_mapsize;
- if (size_used + needed > mei.me_mapsize)
- {
- try
- {
- boost::filesystem::path path(db_path);
- boost::filesystem::space_info si = boost::filesystem::space(path);
- if(si.available < needed)
- {
- MERROR("!! WARNING: Insufficient free space to extend database !!: " << (si.available >> 20L) << " MB available");
- return ENOSPC;
- }
- }
- catch(...)
- {
- // print something but proceed.
- MWARNING("Unable to query free disk space.");
- }
-
- mapsize += needed;
- }
- return mdb_env_set_mapsize(env, mapsize);
-}
-
-static void init(std::string cache_filename)
-{
- MDB_txn *txn;
- bool tx_active = false;
- int dbr;
-
- MINFO("Creating spent output cache in " << cache_filename);
-
- tools::create_directories_if_necessary(cache_filename);
-
- int flags = 0;
- if (db_flags & DBF_FAST)
- flags |= MDB_NOSYNC;
- if (db_flags & DBF_FASTEST)
- flags |= MDB_NOSYNC | MDB_WRITEMAP | MDB_MAPASYNC;
-
- dbr = mdb_env_create(&env);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LDMB environment: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_env_set_maxdbs(env, 7);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to set max env dbs: " + std::string(mdb_strerror(dbr)));
- const std::string actual_filename = get_cache_filename(cache_filename);
- dbr = mdb_env_open(env, actual_filename.c_str(), flags, 0664);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open rings database file '"
- + actual_filename + "': " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_txn_begin(env, NULL, 0, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- epee::misc_utils::auto_scope_leave_caller txn_dtor = epee::misc_utils::create_scope_leave_handler([&](){if (tx_active) mdb_txn_abort(txn);});
- tx_active = true;
-
- dbr = mdb_dbi_open(txn, "relative_rings", MDB_CREATE | MDB_INTEGERKEY, &dbi_relative_rings);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- mdb_set_compare(txn, dbi_relative_rings, compare_hash32);
-
- dbr = mdb_dbi_open(txn, "outputs", MDB_CREATE | MDB_INTEGERKEY, &dbi_outputs);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- mdb_set_compare(txn, dbi_outputs, compare_double64);
-
- dbr = mdb_dbi_open(txn, "processed_txidx", MDB_CREATE, &dbi_processed_txidx);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_dbi_open(txn, "spent", MDB_CREATE | MDB_INTEGERKEY | MDB_DUPSORT | MDB_DUPFIXED, &dbi_spent);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- mdb_set_dupsort(txn, dbi_spent, compare_uint64);
-
- dbr = mdb_dbi_open(txn, "per_amount", MDB_CREATE | MDB_INTEGERKEY, &dbi_per_amount);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- mdb_set_compare(txn, dbi_per_amount, compare_uint64);
-
- dbr = mdb_dbi_open(txn, "ring_instances", MDB_CREATE, &dbi_ring_instances);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_dbi_open(txn, "stats", MDB_CREATE, &dbi_stats);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn creating/opening database: " + std::string(mdb_strerror(dbr)));
- tx_active = false;
-}
-
-static void close()
-{
- if (env)
- {
- mdb_dbi_close(env, dbi_relative_rings);
- mdb_dbi_close(env, dbi_outputs);
- mdb_dbi_close(env, dbi_processed_txidx);
- mdb_dbi_close(env, dbi_per_amount);
- mdb_dbi_close(env, dbi_spent);
- mdb_dbi_close(env, dbi_ring_instances);
- mdb_dbi_close(env, dbi_stats);
- mdb_env_close(env);
- env = NULL;
- }
-}
-
-static std::string compress_ring(const std::vector<uint64_t> &ring, std::string s = "")
-{
- const size_t sz = s.size();
- s.resize(s.size() + 12 * ring.size());
- char *ptr = (char*)s.data() + sz;
- for (uint64_t out: ring)
- tools::write_varint(ptr, out);
- if (ptr > s.data() + sz + 12 * ring.size())
- throw std::runtime_error("varint output overflow");
- s.resize(ptr - s.data());
- return s;
-}
-
-static std::string compress_ring(uint64_t amount, const std::vector<uint64_t> &ring)
-{
- char s[12], *ptr = s;
- tools::write_varint(ptr, amount);
- if (ptr > s + sizeof(s))
- throw std::runtime_error("varint output overflow");
- return compress_ring(ring, std::string(s, ptr-s));
-}
-
-static std::vector<uint64_t> decompress_ring(const std::string &s)
-{
- std::vector<uint64_t> ring;
- int read = 0;
- for (std::string::const_iterator i = s.begin(); i != s.cend(); std::advance(i, read))
- {
- uint64_t out;
- std::string tmp(i, s.cend());
- read = tools::read_varint(tmp.begin(), tmp.end(), out);
- CHECK_AND_ASSERT_THROW_MES(read > 0 && read <= 256, "Internal error decompressing ring");
- ring.push_back(out);
- }
- return ring;
-}
-
-static bool for_all_transactions(const std::string &filename, uint64_t &start_idx, uint64_t &n_txes, const std::function<bool(const cryptonote::transaction_prefix&)> &f)
-{
- MDB_env *env;
- MDB_dbi dbi;
- MDB_txn *txn;
- MDB_cursor *cur;
- int dbr;
- bool tx_active = false;
- MDB_val k;
- MDB_val v;
-
- dbr = mdb_env_create(&env);
- if (dbr) throw std::runtime_error("Failed to create LDMB environment: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_env_set_maxdbs(env, 2);
- if (dbr) throw std::runtime_error("Failed to set max env dbs: " + std::string(mdb_strerror(dbr)));
- const std::string actual_filename = filename;
- dbr = mdb_env_open(env, actual_filename.c_str(), 0, 0664);
- if (dbr) throw std::runtime_error("Failed to open rings database file '"
- + actual_filename + "': " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);
- if (dbr) throw std::runtime_error("Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- epee::misc_utils::auto_scope_leave_caller txn_dtor = epee::misc_utils::create_scope_leave_handler([&](){if (tx_active) mdb_txn_abort(txn);});
- tx_active = true;
-
- dbr = mdb_dbi_open(txn, "txs_pruned", MDB_INTEGERKEY, &dbi);
- if (dbr)
- dbr = mdb_dbi_open(txn, "txs", MDB_INTEGERKEY, &dbi);
- if (dbr) throw std::runtime_error("Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_cursor_open(txn, dbi, &cur);
- if (dbr) throw std::runtime_error("Failed to create LMDB cursor: " + std::string(mdb_strerror(dbr)));
- MDB_stat stat;
- dbr = mdb_stat(txn, dbi, &stat);
- if (dbr) throw std::runtime_error("Failed to query m_block_info: " + std::string(mdb_strerror(dbr)));
- n_txes = stat.ms_entries;
-
- bool fret = true;
-
- k.mv_size = sizeof(uint64_t);
- k.mv_data = &start_idx;
- MDB_cursor_op op = MDB_SET;
- while (1)
- {
- int ret = mdb_cursor_get(cur, &k, &v, op);
- op = MDB_NEXT;
- if (ret == MDB_NOTFOUND)
- break;
- if (ret)
- throw std::runtime_error("Failed to enumerate transactions: " + std::string(mdb_strerror(ret)));
-
- if (k.mv_size != sizeof(uint64_t))
- throw std::runtime_error("Bad key size");
- const uint64_t idx = *(uint64_t*)k.mv_data;
- if (idx < start_idx)
- continue;
-
- cryptonote::transaction_prefix tx;
- blobdata bd;
- bd.assign(reinterpret_cast<char*>(v.mv_data), v.mv_size);
- binary_archive<false> ba{epee::strspan<std::uint8_t>(bd)};
- bool r = do_serialize(ba, tx);
- CHECK_AND_ASSERT_MES(r, false, "Failed to parse transaction from blob");
-
- start_idx = *(uint64_t*)k.mv_data;
- if (!f(tx)) {
- fret = false;
- break;
- }
- }
-
- mdb_cursor_close(cur);
- dbr = mdb_txn_commit(txn);
- if (dbr) throw std::runtime_error("Failed to commit db transaction: " + std::string(mdb_strerror(dbr)));
- tx_active = false;
- mdb_dbi_close(env, dbi);
- mdb_env_close(env);
- return fret;
-}
-
-static bool for_all_transactions(const std::string &filename, const uint64_t &start_idx, uint64_t &n_txes, const std::function<bool(bool, uint64_t, const cryptonote::transaction_prefix&)> &f)
-{
- MDB_env *env;
- MDB_dbi dbi_blocks, dbi_txs;
- MDB_txn *txn;
- MDB_cursor *cur_blocks, *cur_txs;
- int dbr;
- bool tx_active = false;
- MDB_val k;
- MDB_val v;
-
- dbr = mdb_env_create(&env);
- if (dbr) throw std::runtime_error("Failed to create LDMB environment: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_env_set_maxdbs(env, 3);
- if (dbr) throw std::runtime_error("Failed to set max env dbs: " + std::string(mdb_strerror(dbr)));
- const std::string actual_filename = filename;
- dbr = mdb_env_open(env, actual_filename.c_str(), 0, 0664);
- if (dbr) throw std::runtime_error("Failed to open rings database file '"
- + actual_filename + "': " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);
- if (dbr) throw std::runtime_error("Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- epee::misc_utils::auto_scope_leave_caller txn_dtor = epee::misc_utils::create_scope_leave_handler([&](){if (tx_active) mdb_txn_abort(txn);});
- tx_active = true;
-
- dbr = mdb_dbi_open(txn, "blocks", MDB_INTEGERKEY, &dbi_blocks);
- if (dbr) throw std::runtime_error("Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_dbi_open(txn, "txs_pruned", MDB_INTEGERKEY, &dbi_txs);
- if (dbr) throw std::runtime_error("Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_cursor_open(txn, dbi_blocks, &cur_blocks);
- if (dbr) throw std::runtime_error("Failed to create LMDB cursor: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_cursor_open(txn, dbi_txs, &cur_txs);
- if (dbr) throw std::runtime_error("Failed to create LMDB cursor: " + std::string(mdb_strerror(dbr)));
-
- MDB_stat stat;
- dbr = mdb_stat(txn, dbi_blocks, &stat);
- if (dbr) throw std::runtime_error("Failed to query txs stat: " + std::string(mdb_strerror(dbr)));
- uint64_t n_blocks = stat.ms_entries;
- dbr = mdb_stat(txn, dbi_txs, &stat);
- if (dbr) throw std::runtime_error("Failed to query txs stat: " + std::string(mdb_strerror(dbr)));
- n_txes = stat.ms_entries;
-
- bool fret = true;
-
- MDB_cursor_op op_blocks = MDB_FIRST;
- MDB_cursor_op op_txs = MDB_FIRST;
- uint64_t tx_idx = 0;
- while (1)
- {
- int ret = mdb_cursor_get(cur_blocks, &k, &v, op_blocks);
- op_blocks = MDB_NEXT;
- if (ret == MDB_NOTFOUND)
- break;
- if (ret)
- throw std::runtime_error("Failed to enumerate blocks: " + std::string(mdb_strerror(ret)));
-
- if (k.mv_size != sizeof(uint64_t))
- throw std::runtime_error("Bad key size");
- uint64_t height = *(const uint64_t*)k.mv_data;
- blobdata bd;
- bd.assign(reinterpret_cast<char*>(v.mv_data), v.mv_size);
- block b;
- if (!parse_and_validate_block_from_blob(bd, b))
- throw std::runtime_error("Failed to parse block from blob retrieved from the db");
-
- ret = mdb_cursor_get(cur_txs, &k, &v, op_txs);
- if (ret)
- throw std::runtime_error("Failed to fetch transaction " + string_tools::pod_to_hex(get_transaction_hash(b.miner_tx)) + ": " + std::string(mdb_strerror(ret)));
- op_txs = MDB_NEXT;
-
- bool last_block = height == n_blocks - 1;
- if (start_idx <= tx_idx++ && !f(last_block && b.tx_hashes.empty(), height, b.miner_tx))
- {
- fret = false;
- break;
- }
- for (size_t i = 0; i < b.tx_hashes.size(); ++i)
- {
- const crypto::hash& txid = b.tx_hashes[i];
- ret = mdb_cursor_get(cur_txs, &k, &v, op_txs);
- if (ret)
- throw std::runtime_error("Failed to fetch transaction " + string_tools::pod_to_hex(txid) + ": " + std::string(mdb_strerror(ret)));
- if (start_idx <= tx_idx++)
- {
- cryptonote::transaction_prefix tx;
- bd.assign(reinterpret_cast<char*>(v.mv_data), v.mv_size);
- CHECK_AND_ASSERT_MES(parse_and_validate_tx_prefix_from_blob(bd, tx), false, "Failed to parse transaction from blob");
- if (!f(last_block && i == b.tx_hashes.size() - 1, height, tx))
- {
- fret = false;
- break;
- }
- }
- }
- if (!fret)
- break;
- }
-
- mdb_cursor_close(cur_blocks);
- mdb_cursor_close(cur_txs);
- mdb_txn_commit(txn);
- tx_active = false;
- mdb_dbi_close(env, dbi_blocks);
- mdb_dbi_close(env, dbi_txs);
- mdb_env_close(env);
- return fret;
-}
-
-static uint64_t find_first_diverging_transaction(const std::string &first_filename, const std::string &second_filename)
-{
- MDB_env *env[2];
- MDB_dbi dbi[2];
- MDB_txn *txn[2];
- MDB_cursor *cur[2];
- int dbr;
- bool tx_active[2] = { false, false };
- uint64_t n_txes[2];
- MDB_val k;
- MDB_val v[2];
-
- epee::scope_guard txn_dtor([&tx_active, &txn](){
- for (int i = 0; i < 2; ++i)
- if (tx_active[i])
- mdb_txn_abort(txn[i]);
- });
- for (int i = 0; i < 2; ++i)
- {
- dbr = mdb_env_create(&env[i]);
- if (dbr) throw std::runtime_error("Failed to create LDMB environment: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_env_set_maxdbs(env[i], 2);
- if (dbr) throw std::runtime_error("Failed to set max env dbs: " + std::string(mdb_strerror(dbr)));
- const std::string actual_filename = i ? second_filename : first_filename;
- dbr = mdb_env_open(env[i], actual_filename.c_str(), 0, 0664);
- if (dbr) throw std::runtime_error("Failed to open rings database file '"
- + actual_filename + "': " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_txn_begin(env[i], NULL, MDB_RDONLY, &txn[i]);
- if (dbr) throw std::runtime_error("Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- tx_active[i] = true;
-
- dbr = mdb_dbi_open(txn[i], "txs_pruned", MDB_INTEGERKEY, &dbi[i]);
- if (dbr)
- dbr = mdb_dbi_open(txn[i], "txs", MDB_INTEGERKEY, &dbi[i]);
- if (dbr) throw std::runtime_error("Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_cursor_open(txn[i], dbi[i], &cur[i]);
- if (dbr) throw std::runtime_error("Failed to create LMDB cursor: " + std::string(mdb_strerror(dbr)));
- MDB_stat stat;
- dbr = mdb_stat(txn[i], dbi[i], &stat);
- if (dbr) throw std::runtime_error("Failed to query m_block_info: " + std::string(mdb_strerror(dbr)));
- n_txes[i] = stat.ms_entries;
- }
-
- if (n_txes[0] == 0 || n_txes[1] == 0)
- throw std::runtime_error("No transaction in the database");
- uint64_t lo = 0, hi = std::min(n_txes[0], n_txes[1]) - 1;
- while (lo <= hi)
- {
- uint64_t mid = (lo + hi) / 2;
-
- k.mv_size = sizeof(uint64_t);
- k.mv_data = (void*)∣
- dbr = mdb_cursor_get(cur[0], &k, &v[0], MDB_SET);
- if (dbr) throw std::runtime_error("Failed to query transaction: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_cursor_get(cur[1], &k, &v[1], MDB_SET);
- if (dbr) throw std::runtime_error("Failed to query transaction: " + std::string(mdb_strerror(dbr)));
- if (v[0].mv_size == v[1].mv_size && !memcmp(v[0].mv_data, v[1].mv_data, v[0].mv_size))
- lo = mid + 1;
- else
- hi = mid - 1;
- }
-
- for (int i = 0; i < 2; ++i)
- {
- mdb_cursor_close(cur[i]);
- dbr = mdb_txn_commit(txn[i]);
- if (dbr) throw std::runtime_error("Failed to query transaction: " + std::string(mdb_strerror(dbr)));
- tx_active[i] = false;
- mdb_dbi_close(env[i], dbi[i]);
- mdb_env_close(env[i]);
- }
- return hi;
-}
-
-static std::vector<uint64_t> canonicalize(const std::vector<uint64_t> &v)
-{
- std::vector<uint64_t> c;
- c.reserve(v.size());
- c.push_back(v[0]);
- for (size_t n = 1; n < v.size(); ++n)
- {
- if (v[n] != 0)
- c.push_back(v[n]);
- }
- if (c.size() < v.size())
- {
- MINFO("Ring has duplicate member(s): " <<
- boost::join(v | boost::adaptors::transformed([](uint64_t out){return std::to_string(out);}), " "));
- }
- return c;
-}
-
-static uint64_t get_num_spent_outputs()
-{
- MDB_txn *txn;
- bool tx_active = false;
-
- int dbr = mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- epee::misc_utils::auto_scope_leave_caller txn_dtor = epee::misc_utils::create_scope_leave_handler([&](){if (tx_active) mdb_txn_abort(txn);});
- tx_active = true;
-
- MDB_cursor *cur;
- dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open cursor for spent outputs: " + std::string(mdb_strerror(dbr)));
- MDB_val k, v;
- mdb_size_t count = 0, tmp;
-
- MDB_cursor_op op = MDB_FIRST;
- while (1)
- {
- dbr = mdb_cursor_get(cur, &k, &v, op);
- op = MDB_NEXT_NODUP;
- if (dbr == MDB_NOTFOUND)
- break;
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get first/next spent output: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_cursor_count(cur, &tmp);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to count entries: " + std::string(mdb_strerror(dbr)));
- count += tmp;
- }
-
- mdb_cursor_close(cur);
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn: " + std::string(mdb_strerror(dbr)));
- tx_active = false;
-
- return count;
-}
-
-static bool add_spent_output(MDB_cursor *cur, const output_data &od)
-{
- MDB_val k = {sizeof(od.amount), (void*)&od.amount};
- MDB_val v = {sizeof(od.offset), (void*)&od.offset};
- int dbr = mdb_cursor_put(cur, &k, &v, MDB_NODUPDATA);
- if (dbr == MDB_KEYEXIST)
- return false;
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to add spent output: " + std::string(mdb_strerror(dbr)));
- return true;
-}
-
-static bool is_output_spent(MDB_cursor *cur, const output_data &od)
-{
- MDB_val k = {sizeof(od.amount), (void*)&od.amount};
- MDB_val v = {sizeof(od.offset), (void*)&od.offset};
- int dbr = mdb_cursor_get(cur, &k, &v, MDB_GET_BOTH);
- CHECK_AND_ASSERT_THROW_MES(!dbr || dbr == MDB_NOTFOUND, "Failed to get spent output: " + std::string(mdb_strerror(dbr)));
- bool spent = dbr == 0;
- return spent;
-}
-
-static std::vector<output_data> get_spent_outputs(MDB_txn *txn)
-{
- MDB_cursor *cur;
- int dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open cursor for spent outputs: " + std::string(mdb_strerror(dbr)));
- MDB_val k, v;
- mdb_size_t count = 0;
- dbr = mdb_cursor_get(cur, &k, &v, MDB_FIRST);
- if (dbr != MDB_NOTFOUND)
- {
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get first spent output: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_cursor_count(cur, &count);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to count entries: " + std::string(mdb_strerror(dbr)));
- }
- std::vector<output_data> outs;
- outs.reserve(count);
- while (1)
- {
- outs.push_back({*(const uint64_t*)k.mv_data, *(const uint64_t*)v.mv_data});
- dbr = mdb_cursor_get(cur, &k, &v, MDB_NEXT);
- if (dbr == MDB_NOTFOUND)
- break;
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get next spent output: " + std::string(mdb_strerror(dbr)));
- }
- mdb_cursor_close(cur);
- return outs;
-}
-
-static void get_per_amount_outputs(MDB_txn *txn, uint64_t amount, uint64_t &total, uint64_t &spent)
-{
- MDB_cursor *cur;
- int dbr = mdb_cursor_open(txn, dbi_per_amount, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open cursor for per amount outputs: " + std::string(mdb_strerror(dbr)));
- MDB_val k, v;
- k.mv_size = sizeof(uint64_t);
- k.mv_data = (void*)&amount;
- dbr = mdb_cursor_get(cur, &k, &v, MDB_SET);
- if (dbr == MDB_NOTFOUND)
- {
- total = spent = 0;
- }
- else
- {
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get per amount outputs: " + std::string(mdb_strerror(dbr)));
- total = ((const uint64_t*)v.mv_data)[0];
- spent = ((const uint64_t*)v.mv_data)[1];
- }
- mdb_cursor_close(cur);
-}
-
-static void inc_per_amount_outputs(MDB_txn *txn, uint64_t amount, uint64_t total, uint64_t spent)
-{
- MDB_cursor *cur;
- int dbr = mdb_cursor_open(txn, dbi_per_amount, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open cursor for per amount outputs: " + std::string(mdb_strerror(dbr)));
- MDB_val k, v;
- k.mv_size = sizeof(uint64_t);
- k.mv_data = (void*)&amount;
- dbr = mdb_cursor_get(cur, &k, &v, MDB_SET);
- if (dbr == 0)
- {
- total += ((const uint64_t*)v.mv_data)[0];
- spent += ((const uint64_t*)v.mv_data)[1];
- }
- else
- {
- CHECK_AND_ASSERT_THROW_MES(dbr == MDB_NOTFOUND, "Failed to get per amount outputs: " + std::string(mdb_strerror(dbr)));
- }
- uint64_t data[2] = {total, spent};
- v.mv_size = 2 * sizeof(uint64_t);
- v.mv_data = (void*)data;
- dbr = mdb_cursor_put(cur, &k, &v, 0);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to write record for per amount outputs: " + std::string(mdb_strerror(dbr)));
- mdb_cursor_close(cur);
-}
-
-static uint64_t get_processed_txidx(const std::string &name)
-{
- MDB_txn *txn;
- bool tx_active = false;
-
- int dbr = mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- epee::misc_utils::auto_scope_leave_caller txn_dtor = epee::misc_utils::create_scope_leave_handler([&](){if (tx_active) mdb_txn_abort(txn);});
- tx_active = true;
-
- uint64_t height = 0;
- MDB_val k, v;
- k.mv_data = (void*)name.c_str();
- k.mv_size = name.size();
- dbr = mdb_get(txn, dbi_processed_txidx, &k, &v);
- if (dbr != MDB_NOTFOUND)
- {
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get processed height: " + std::string(mdb_strerror(dbr)));
- height = *(const uint64_t*)v.mv_data;
- }
-
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn: " + std::string(mdb_strerror(dbr)));
- tx_active = false;
-
- return height;
-}
-
-static void set_processed_txidx(MDB_txn *txn, const std::string &name, uint64_t height)
-{
- MDB_val k, v;
- k.mv_data = (void*)name.c_str();
- k.mv_size = name.size();
- v.mv_data = (void*)&height;
- v.mv_size = sizeof(height);
- int dbr = mdb_put(txn, dbi_processed_txidx, &k, &v, 0);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to set processed height: " + std::string(mdb_strerror(dbr)));
-}
-
-static bool get_relative_ring(MDB_txn *txn, const crypto::key_image &ki, std::vector<uint64_t> &ring)
-{
- MDB_val k, v;
- k.mv_data = (void*)&ki;
- k.mv_size = sizeof(ki);
- int dbr = mdb_get(txn, dbi_relative_rings, &k, &v);
- if (dbr == MDB_NOTFOUND)
- return false;
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get relative ring: " + std::string(mdb_strerror(dbr)));
- ring = decompress_ring(std::string((const char*)v.mv_data, v.mv_size));
- return true;
-}
-
-static void set_relative_ring(MDB_txn *txn, const crypto::key_image &ki, const std::vector<uint64_t> &ring)
-{
- const std::string sring = compress_ring(ring);
- MDB_val k, v;
- k.mv_data = (void*)&ki;
- k.mv_size = sizeof(ki);
- v.mv_data = (void*)sring.c_str();
- v.mv_size = sring.size();
- int dbr = mdb_put(txn, dbi_relative_rings, &k, &v, 0);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to set relative ring: " + std::string(mdb_strerror(dbr)));
-}
-
-static std::string keep_under_511(const std::string &s)
-{
- if (s.size() <= 511)
- return s;
- crypto::hash hash;
- crypto::cn_fast_hash(s.data(), s.size(), hash);
- return std::string((const char*)&hash, 32);
-}
-
-static uint64_t get_ring_instances(MDB_txn *txn, uint64_t amount, const std::vector<uint64_t> &ring)
-{
- const std::string sring = keep_under_511(compress_ring(amount, ring));
- MDB_val k, v;
- k.mv_data = (void*)sring.data();
- k.mv_size = sring.size();
- int dbr = mdb_get(txn, dbi_ring_instances, &k, &v);
- if (dbr == MDB_NOTFOUND)
- return 0;
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get ring instances: " + std::string(mdb_strerror(dbr)));
- return *(const uint64_t*)v.mv_data;
-}
-
-static uint64_t get_ring_subset_instances(MDB_txn *txn, uint64_t amount, const std::vector<uint64_t> &ring)
-{
- uint64_t instances = get_ring_instances(txn, amount, ring);
- if (ring.size() > 11)
- return instances;
-
- uint64_t extra = 0;
- std::vector<uint64_t> subset;
- subset.reserve(ring.size());
- for (uint64_t mask = 1; mask < (((uint64_t)1) << ring.size()) - 1; ++mask)
- {
- subset.resize(0);
- for (size_t i = 0; i < ring.size(); ++i)
- if ((mask >> i) & 1)
- subset.push_back(ring[i]);
- extra += get_ring_instances(txn, amount, subset);
- }
- return instances + extra;
-}
-
-static uint64_t inc_ring_instances(MDB_txn *txn, uint64_t amount, const std::vector<uint64_t> &ring)
-{
- const std::string sring = keep_under_511(compress_ring(amount, ring));
- MDB_val k, v;
- k.mv_data = (void*)sring.data();
- k.mv_size = sring.size();
-
- int dbr = mdb_get(txn, dbi_ring_instances, &k, &v);
- CHECK_AND_ASSERT_THROW_MES(!dbr || dbr == MDB_NOTFOUND, "Failed to get ring instances: " + std::string(mdb_strerror(dbr)));
-
- uint64_t count;
- if (dbr == MDB_NOTFOUND)
- count = 1;
- else
- count = 1 + *(const uint64_t*)v.mv_data;
-
- v.mv_data = &count;
- v.mv_size = sizeof(count);
- dbr = mdb_put(txn, dbi_ring_instances, &k, &v, 0);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to set ring instances: " + std::string(mdb_strerror(dbr)));
-
- return count;
-}
-
-static std::vector<crypto::key_image> get_key_images(MDB_txn *txn, const output_data &od)
-{
- MDB_val k, v;
- k.mv_data = (void*)&od;
- k.mv_size = sizeof(od);
- int dbr = mdb_get(txn, dbi_outputs, &k, &v);
- CHECK_AND_ASSERT_THROW_MES(!dbr || dbr == MDB_NOTFOUND, "Failed to get output: " + std::string(mdb_strerror(dbr)));
- if (dbr == MDB_NOTFOUND)
- return {};
- CHECK_AND_ASSERT_THROW_MES(v.mv_size % 32 == 0, "Unexpected record size");
- std::vector<crypto::key_image> key_images;
- key_images.reserve(v.mv_size / 32);
- const crypto::key_image *ki = (const crypto::key_image*)v.mv_data;
- for (size_t n = 0; n < v.mv_size / 32; ++n)
- key_images.push_back(*ki++);
- return key_images;
-}
-
-static void add_key_image(MDB_txn *txn, const output_data &od, const crypto::key_image &ki)
-{
- MDB_val k, v;
- k.mv_data = (void*)&od;
- k.mv_size = sizeof(od);
- int dbr = mdb_get(txn, dbi_outputs, &k, &v);
- CHECK_AND_ASSERT_THROW_MES(!dbr || dbr == MDB_NOTFOUND, "Failed to get output");
- std::string data;
- if (!dbr)
- {
- CHECK_AND_ASSERT_THROW_MES(v.mv_size % 32 == 0, "Unexpected record size");
- data = std::string((const char*)v.mv_data, v.mv_size);
- }
- data += std::string((const char*)&ki, sizeof(ki));
-
- v.mv_data = (void*)data.data();
- v.mv_size = data.size();
- dbr = mdb_put(txn, dbi_outputs, &k, &v, 0);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to set outputs: " + std::string(mdb_strerror(dbr)));
-}
-
-static bool get_stat(MDB_txn *txn, const char *key, uint64_t &data)
-{
- MDB_val k, v;
- k.mv_data = (void*)key;
- k.mv_size = strlen(key);
- int dbr = mdb_get(txn, dbi_stats, &k, &v);
- if (dbr == MDB_NOTFOUND)
- return false;
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to get stat record");
- CHECK_AND_ASSERT_THROW_MES(v.mv_size == sizeof(uint64_t), "Unexpected record size");
- data = *(const uint64_t*)v.mv_data;
- return true;
-}
-
-static void set_stat(MDB_txn *txn, const char *key, uint64_t data)
-{
- MDB_val k, v;
- k.mv_data = (void*)key;
- k.mv_size = strlen(key);
- v.mv_data = (void*)&data;
- v.mv_size = sizeof(uint64_t);
- int dbr = mdb_put(txn, dbi_stats, &k, &v, 0);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to set stat record");
-}
-
-static void inc_stat(MDB_txn *txn, const char *key)
-{
- uint64_t data;
- if (!get_stat(txn, key, data))
- data = 0;
- ++data;
- set_stat(txn, key, data);
-}
-
-static void open_db(const std::string &filename, MDB_env **env, MDB_txn **txn, MDB_cursor **cur, MDB_dbi *dbi)
-{
- tools::create_directories_if_necessary(filename);
-
- int flags = MDB_RDONLY;
- if (db_flags & DBF_FAST)
- flags |= MDB_NOSYNC;
- if (db_flags & DBF_FASTEST)
- flags |= MDB_NOSYNC | MDB_WRITEMAP | MDB_MAPASYNC;
-
- int dbr = mdb_env_create(env);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LDMB environment: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_env_set_maxdbs(*env, 1);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to set max env dbs: " + std::string(mdb_strerror(dbr)));
- const std::string actual_filename = filename;
- MINFO("Opening monero blockchain at " << actual_filename);
- dbr = mdb_env_open(*env, actual_filename.c_str(), flags, 0664);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open rings database file '"
- + actual_filename + "': " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_txn_begin(*env, NULL, MDB_RDONLY, txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_dbi_open(*txn, "output_amounts", MDB_CREATE | MDB_INTEGERKEY | MDB_DUPSORT | MDB_DUPFIXED, dbi);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- mdb_set_dupsort(*txn, *dbi, compare_uint64);
-
- dbr = mdb_cursor_open(*txn, *dbi, cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB cursor: " + std::string(mdb_strerror(dbr)));
-}
-
-static void close_db(MDB_env *env, MDB_txn *txn, MDB_cursor *cur, MDB_dbi dbi)
-{
- mdb_txn_abort(txn);
- mdb_cursor_close(cur);
- mdb_dbi_close(env, dbi);
- mdb_env_close(env);
-}
-
-static void get_num_outputs(MDB_txn *txn, MDB_cursor *cur, MDB_dbi dbi, uint64_t &pre_rct, uint64_t &rct)
-{
- uint64_t amount = 0;
- MDB_val k = { sizeof(amount), (void*)&amount }, v;
- int dbr = mdb_cursor_get(cur, &k, &v, MDB_SET);
- if (dbr == MDB_NOTFOUND)
- {
- rct = 0;
- }
- else
- {
- if (dbr) throw std::runtime_error("Record 0 not found: " + std::string(mdb_strerror(dbr)));
- mdb_size_t count = 0;
- dbr = mdb_cursor_count(cur, &count);
- if (dbr) throw std::runtime_error("Failed to count records: " + std::string(mdb_strerror(dbr)));
- rct = count;
- }
- MDB_stat s;
- dbr = mdb_stat(txn, dbi, &s);
- if (dbr) throw std::runtime_error("Failed to count records: " + std::string(mdb_strerror(dbr)));
- if (s.ms_entries < rct) throw std::runtime_error("Inconsistent records: " + std::string(mdb_strerror(dbr)));
- pre_rct = s.ms_entries - rct;
-}
-
-static crypto::hash get_genesis_block_hash(const std::string &filename)
-{
- MDB_env *env;
- MDB_dbi dbi;
- MDB_txn *txn;
- int dbr;
- bool tx_active = false;
-
- dbr = mdb_env_create(&env);
- if (dbr) throw std::runtime_error("Failed to create LDMB environment: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_env_set_maxdbs(env, 1);
- if (dbr) throw std::runtime_error("Failed to set max env dbs: " + std::string(mdb_strerror(dbr)));
- const std::string actual_filename = filename;
- dbr = mdb_env_open(env, actual_filename.c_str(), 0, 0664);
- if (dbr) throw std::runtime_error("Failed to open rings database file '"
- + actual_filename + "': " + std::string(mdb_strerror(dbr)));
-
- dbr = mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);
- if (dbr) throw std::runtime_error("Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- epee::misc_utils::auto_scope_leave_caller txn_dtor = epee::misc_utils::create_scope_leave_handler([&](){if (tx_active) mdb_txn_abort(txn);});
- tx_active = true;
-
- dbr = mdb_dbi_open(txn, "block_info", MDB_INTEGERKEY | MDB_DUPSORT | MDB_DUPFIXED, &dbi);
- mdb_set_dupsort(txn, dbi, compare_uint64);
- if (dbr) throw std::runtime_error("Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- uint64_t zero = 0;
- MDB_val k = { sizeof(uint64_t), (void*)&zero}, v;
- dbr = mdb_get(txn, dbi, &k, &v);
- if (dbr) throw std::runtime_error("Failed to retrieve genesis block: " + std::string(mdb_strerror(dbr)));
- crypto::hash genesis_block_hash = *(const crypto::hash*)(((const uint64_t*)v.mv_data) + 5);
- mdb_dbi_close(env, dbi);
- mdb_txn_abort(txn);
- mdb_env_close(env);
- tx_active = false;
- return genesis_block_hash;
-}
-
-static std::vector<std::pair<uint64_t, uint64_t>> load_outputs(const std::string &filename)
-{
- std::vector<std::pair<uint64_t, uint64_t>> outputs;
- uint64_t amount = std::numeric_limits<uint64_t>::max();
- FILE *f;
-
- f = fopen(filename.c_str(), "r");
- if (!f)
- {
- MERROR("Failed to load outputs from " << filename << ": " << strerror(errno));
- return {};
- }
- while (1)
- {
- char s[256];
- if (!fgets(s, sizeof(s), f))
- {
- MERROR("Error reading from " << filename << ": " << strerror(errno));
- break;
- }
- if (feof(f))
- break;
- const size_t len = strlen(s);
- if (len > 0 && s[len - 1] == '\n')
- s[len - 1] = 0;
- if (!s[0])
- continue;
- uint64_t offset, num_offsets;
- if (sscanf(s, "@%" PRIu64, &amount) == 1)
- {
- continue;
- }
- if (amount == std::numeric_limits<uint64_t>::max())
- {
- MERROR("Bad format in " << filename);
- continue;
- }
- if (sscanf(s, "%" PRIu64 "*%" PRIu64, &offset, &num_offsets) == 2 && num_offsets < std::numeric_limits<uint64_t>::max() - offset)
- {
- while (num_offsets-- > 0)
- outputs.push_back(std::make_pair(amount, offset++));
- }
- else if (sscanf(s, "%" PRIu64, &offset) == 1)
- {
- outputs.push_back(std::make_pair(amount, offset));
- }
- else
- {
- MERROR("Bad format in " << filename);
- continue;
- }
- }
- fclose(f);
- return outputs;
-}
-
-static bool export_spent_outputs(MDB_cursor *cur, const std::string &filename)
-{
- FILE *f = fopen(filename.c_str(), "w");
- if (!f)
- {
- MERROR("Failed to open " << filename << ": " << strerror(errno));
- return false;
- }
-
- uint64_t pending_amount = std::numeric_limits<uint64_t>::max();
- std::vector<uint64_t> pending_offsets;
- MDB_val k, v;
- MDB_cursor_op op = MDB_FIRST;
- while (1)
- {
- int dbr = mdb_cursor_get(cur, &k, &v, op);
- if (dbr == MDB_NOTFOUND)
- break;
- op = MDB_NEXT;
- if (dbr)
- {
- fclose(f);
- MERROR("Failed to enumerate spent outputs: " << mdb_strerror(dbr));
- return false;
- }
- const uint64_t amount = *(const uint64_t*)k.mv_data;
- const uint64_t offset = *(const uint64_t*)v.mv_data;
- if (!pending_offsets.empty() && (amount != pending_amount || pending_offsets.back()+1 != offset))
- {
- if (pending_offsets.size() == 1)
- fprintf(f, "%" PRIu64 "\n", pending_offsets.front());
- else
- fprintf(f, "%" PRIu64 "*%zu\n", pending_offsets.front(), pending_offsets.size());
- pending_offsets.clear();
- }
- if (pending_amount != amount)
- {
- fprintf(f, "@%" PRIu64 "\n", amount);
- pending_amount = amount;
- }
- pending_offsets.push_back(offset);
- }
- if (!pending_offsets.empty())
- {
- if (pending_offsets.size() == 1)
- fprintf(f, "%" PRIu64 "\n", pending_offsets.front());
- else
- fprintf(f, "%" PRIu64 "*%zu\n", pending_offsets.front(), pending_offsets.size());
- pending_offsets.clear();
- }
- fclose(f);
- return true;
-}
-
-int main(int argc, char* argv[])
-{
- TRY_ENTRY();
-
- epee::string_tools::set_module_name_and_folder(argv[0]);
-
- uint32_t log_level = 0;
-
- tools::on_startup();
-
- boost::filesystem::path output_file_path;
-
- po::options_description desc_cmd_only("Command line options");
- po::options_description desc_cmd_sett("Command line options and settings options");
- const command_line::arg_descriptor<std::string> arg_blackball_db_dir = {
- "spent-output-db-dir", "Specify spent output database directory",
- get_default_db_path(),
- };
- const command_line::arg_descriptor<std::string> arg_log_level = {"log-level", "0-4 or categories", ""};
- const command_line::arg_descriptor<bool> arg_rct_only = {"rct-only", "Only work on ringCT outputs", false};
- const command_line::arg_descriptor<bool> arg_check_subsets = {"check-subsets", "Check ring subsets (very expensive)", false};
- const command_line::arg_descriptor<bool> arg_verbose = {"verbose", "Verbose output)", false};
- const command_line::arg_descriptor<std::vector<std::string> > arg_inputs = {"inputs", "Path to Monero DB, and path to any fork DBs"};
- const command_line::arg_descriptor<std::string> arg_db_sync_mode = {
- "db-sync-mode"
- , "Specify sync option, using format [safe|fast|fastest]:[nrecords_per_sync]."
- , "fast:1000"
- };
- const command_line::arg_descriptor<std::string> arg_extra_spent_list = {"extra-spent-list", "Optional list of known spent outputs",""};
- const command_line::arg_descriptor<std::string> arg_export = {"export", "Filename to export the backball list to"};
- const command_line::arg_descriptor<bool> arg_force_chain_reaction_pass = {"force-chain-reaction-pass", "Run the chain reaction pass even if no new blockchain data was processed"};
- const command_line::arg_descriptor<bool> arg_historical_stat = {"historical-stat", "Report historical stat of spent outputs for every 10000 blocks window"};
-
- command_line::add_arg(desc_cmd_sett, arg_blackball_db_dir);
- command_line::add_arg(desc_cmd_sett, arg_log_level);
- command_line::add_arg(desc_cmd_sett, arg_rct_only);
- command_line::add_arg(desc_cmd_sett, arg_check_subsets);
- command_line::add_arg(desc_cmd_sett, arg_verbose);
- command_line::add_arg(desc_cmd_sett, arg_db_sync_mode);
- command_line::add_arg(desc_cmd_sett, arg_extra_spent_list);
- command_line::add_arg(desc_cmd_sett, arg_export);
- command_line::add_arg(desc_cmd_sett, arg_force_chain_reaction_pass);
- command_line::add_arg(desc_cmd_sett, arg_historical_stat);
- command_line::add_arg(desc_cmd_sett, arg_inputs);
- command_line::add_arg(desc_cmd_only, command_line::arg_help);
-
- po::options_description desc_options("Allowed options");
- desc_options.add(desc_cmd_only).add(desc_cmd_sett);
-
- po::positional_options_description positional_options;
- positional_options.add(arg_inputs.name, -1);
-
- po::variables_map vm;
- bool r = command_line::handle_error_helper(desc_options, [&]()
- {
- auto parser = po::command_line_parser(argc, argv).options(desc_options).positional(positional_options);
- po::store(parser.run(), vm);
- po::notify(vm);
- return true;
- });
- if (! r)
- return 1;
-
- if (command_line::get_arg(vm, command_line::arg_help))
- {
- std::cout << "Monero '" << MONERO_RELEASE_NAME << "' (v" << MONERO_VERSION_FULL << ")" << ENDL << ENDL;
- std::cout << desc_options << std::endl;
- return 1;
- }
-
- mlog_configure(mlog_get_default_log_path("monero-blockchain-mark-spent-outputs.log"), true);
- if (!command_line::is_arg_defaulted(vm, arg_log_level))
- mlog_set_log(command_line::get_arg(vm, arg_log_level).c_str());
- else
- mlog_set_log(std::string(std::to_string(log_level) + ",bcutil:INFO").c_str());
-
- LOG_PRINT_L0("Starting...");
-
- output_file_path = command_line::get_arg(vm, arg_blackball_db_dir);
- bool opt_rct_only = command_line::get_arg(vm, arg_rct_only);
- bool opt_check_subsets = command_line::get_arg(vm, arg_check_subsets);
- bool opt_verbose = command_line::get_arg(vm, arg_verbose);
- bool opt_force_chain_reaction_pass = command_line::get_arg(vm, arg_force_chain_reaction_pass);
- bool opt_historical_stat = command_line::get_arg(vm, arg_historical_stat);
- std::string opt_export = command_line::get_arg(vm, arg_export);
- std::string extra_spent_list = command_line::get_arg(vm, arg_extra_spent_list);
- std::vector<std::pair<uint64_t, uint64_t>> extra_spent_outputs = extra_spent_list.empty() ? std::vector<std::pair<uint64_t, uint64_t>>() : load_outputs(extra_spent_list);
-
-
- std::string db_sync_mode = command_line::get_arg(vm, arg_db_sync_mode);
- if (!parse_db_sync_mode(db_sync_mode))
- {
- MERROR("Invalid db sync mode: " << db_sync_mode);
- return 1;
- }
-
- const std::vector<std::string> inputs = command_line::get_arg(vm, arg_inputs);
- if (inputs.empty())
- {
- LOG_PRINT_L0("No inputs given");
- return 1;
- }
-
- const std::string cache_dir = (output_file_path / "spent-outputs-cache").string();
- init(cache_dir);
-
- LOG_PRINT_L0("Scanning for spent outputs...");
-
- const uint64_t start_blackballed_outputs = get_num_spent_outputs();
-
- tools::ringdb ringdb(output_file_path.string(), epee::string_tools::pod_to_hex(get_genesis_block_hash(inputs[0])));
-
- bool stop_requested = false;
- tools::signal_handler::install([&stop_requested](int type) {
- stop_requested = true;
- });
-
- int dbr = resize_env(cache_dir.c_str());
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to resize LMDB database: " + std::string(mdb_strerror(dbr)));
-
- // open first db
- MDB_env *env0;
- MDB_txn *txn0;
- MDB_dbi dbi0;
- MDB_cursor *cur0;
- open_db(inputs[0], &env0, &txn0, &cur0, &dbi0);
-
- std::vector<output_data> work_spent;
-
- if (opt_historical_stat)
- {
- if (!start_blackballed_outputs)
- {
- MINFO("Spent outputs database is empty. Either you haven't run the analysis mode yet, or there is really no output marked as spent.");
- goto skip_secondary_passes;
- }
- MDB_txn *txn;
- int dbr = mdb_txn_begin(env, NULL, 0, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- MDB_cursor *cur;
- dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB cursor: " + std::string(mdb_strerror(dbr)));
-
- const uint64_t STAT_WINDOW = 10000;
- uint64_t outs_total = 0;
- uint64_t outs_spent = 0;
- std::unordered_map<uint64_t, uint64_t> outs_per_amount;
- uint64_t start_idx = 0, n_txes;
- uint64_t prev_height = 0;
- for_all_transactions(inputs[0], start_idx, n_txes, [&](bool last_tx, uint64_t height, const cryptonote::transaction_prefix &tx)->bool
- {
- if (height != prev_height)
- {
- if (height % 100 == 0) std::cout << "\r" << height << ": " << (100.0f * outs_spent / outs_total) << "% ( " << outs_spent << " / " << outs_total << " ) \r" << std::flush;
- if (height % STAT_WINDOW == 0)
- {
- uint64_t window_front = (height / STAT_WINDOW - 1) * STAT_WINDOW;
- uint64_t window_back = window_front + STAT_WINDOW - 1;
- LOG_PRINT_L0(window_front << "-" << window_back << ": " << (100.0f * outs_spent / outs_total) << "% ( " << outs_spent << " / " << outs_total << " )");
- outs_total = outs_spent = 0;
- }
- }
- prev_height = height;
- for (const auto &out: tx.vout)
- {
- ++outs_total;
- CHECK_AND_ASSERT_THROW_MES(out.target.type() == typeid(txout_to_key), "Out target type is not txout_to_key: height=" + std::to_string(height));
- uint64_t out_global_index = outs_per_amount[out.amount]++;
- if (is_output_spent(cur, output_data(out.amount, out_global_index)))
- ++outs_spent;
- }
- if (last_tx)
- {
- uint64_t window_front = (height / STAT_WINDOW) * STAT_WINDOW;
- uint64_t window_back = height;
- LOG_PRINT_L0(window_front << "-" << window_back << ": " << (100.0f * outs_spent / outs_total) << "% ( " << outs_spent << " / " << outs_total << " )");
- }
- if (stop_requested)
- {
- MINFO("Stopping scan...");
- return false;
- }
- return true;
- });
- mdb_cursor_close(cur);
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn creating/opening database: " + std::string(mdb_strerror(dbr)));
- goto skip_secondary_passes;
- }
-
- if (!extra_spent_outputs.empty())
- {
- MINFO("Adding " << extra_spent_outputs.size() << " extra spent outputs");
- MDB_txn *txn;
- int dbr = mdb_txn_begin(env, NULL, 0, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- MDB_cursor *cur;
- dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB cursor: " + std::string(mdb_strerror(dbr)));
-
- std::vector<std::pair<uint64_t, uint64_t>> blackballs;
- for (const std::pair<uint64_t, uint64_t> &output: extra_spent_outputs)
- {
- if (!is_output_spent(cur, output_data(output.first, output.second)))
- {
- blackballs.push_back(output);
- if (add_spent_output(cur, output_data(output.first, output.second)))
- inc_stat(txn, output.first ? "pre-rct-extra" : "rct-ring-extra");
- }
- }
- if (!blackballs.empty())
- {
- ringdb.blackball(blackballs);
- blackballs.clear();
- }
- mdb_cursor_close(cur);
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn creating/opening database: " + std::string(mdb_strerror(dbr)));
- }
-
- for (size_t n = 0; n < inputs.size(); ++n)
- {
- const std::string canonical = boost::filesystem::canonical(inputs[n]).string();
- uint64_t start_idx = get_processed_txidx(canonical);
- if (n > 0 && start_idx == 0)
- {
- start_idx = find_first_diverging_transaction(inputs[0], inputs[n]);
- LOG_PRINT_L0("First diverging transaction at " << start_idx);
- }
- LOG_PRINT_L0("Reading blockchain from " << inputs[n] << " from " << start_idx);
- MDB_txn *txn;
- int dbr = mdb_txn_begin(env, NULL, 0, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- MDB_cursor *cur;
- dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB cursor: " + std::string(mdb_strerror(dbr)));
- size_t records = 0;
- const std::string filename = inputs[n];
- std::vector<std::pair<uint64_t, uint64_t>> blackballs;
- uint64_t n_txes;
- for_all_transactions(filename, start_idx, n_txes, [&](const cryptonote::transaction_prefix &tx)->bool
- {
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- const bool miner_tx = tx.vin.size() == 1 && tx.vin[0].type() == typeid(txin_gen);
- for (const auto &in: tx.vin)
- {
- if (in.type() != typeid(txin_to_key))
- continue;
- const auto &txin = boost::get<txin_to_key>(in);
- if (opt_rct_only && txin.amount != 0)
- continue;
-
- const std::vector<uint64_t> absolute = cryptonote::relative_output_offsets_to_absolute(txin.key_offsets);
- if (n == 0)
- for (uint64_t out: absolute)
- add_key_image(txn, output_data(txin.amount, out), txin.k_image);
-
- std::vector<uint64_t> relative_ring;
- std::vector<uint64_t> new_ring = canonicalize(txin.key_offsets);
- const uint32_t ring_size = txin.key_offsets.size();
- const uint64_t instances = inc_ring_instances(txn, txin.amount, new_ring);
- uint64_t pa_total = 0, pa_spent = 0;
- if (!opt_rct_only)
- get_per_amount_outputs(txn, txin.amount, pa_total, pa_spent);
- if (n == 0 && ring_size == 1)
- {
- const std::pair<uint64_t, uint64_t> output = std::make_pair(txin.amount, absolute[0]);
- if (opt_verbose)
- {
- MINFO("Marking output " << output.first << "/" << output.second << " as spent, due to being used in a 1-ring");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- }
- blackballs.push_back(output);
- if (add_spent_output(cur, output_data(txin.amount, absolute[0])))
- inc_stat(txn, txin.amount ? "pre-rct-ring-size-1" : "rct-ring-size-1");
- }
- else if (n == 0 && instances == new_ring.size())
- {
- for (size_t o = 0; o < new_ring.size(); ++o)
- {
- const std::pair<uint64_t, uint64_t> output = std::make_pair(txin.amount, absolute[o]);
- if (opt_verbose)
- {
- MINFO("Marking output " << output.first << "/" << output.second << " as spent, due to being used in " << new_ring.size() << " identical " << new_ring.size() << "-rings");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- }
- blackballs.push_back(output);
- if (add_spent_output(cur, output_data(txin.amount, absolute[o])))
- inc_stat(txn, txin.amount ? "pre-rct-duplicate-rings" : "rct-duplicate-rings");
- }
- }
- else if (n == 0 && !opt_rct_only && pa_spent + 1 == pa_total)
- {
- for (size_t o = 0; o < pa_total; ++o)
- {
- const std::pair<uint64_t, uint64_t> output = std::make_pair(txin.amount, o);
- if (opt_verbose)
- {
- MINFO("Marking output " << output.first << "/" << output.second << " as spent, due to as many outputs of that amount being spent as exist so far");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- }
- blackballs.push_back(output);
- if (add_spent_output(cur, output_data(txin.amount, o)))
- inc_stat(txn, txin.amount ? "pre-rct-full-count" : "rct-full-count");
- }
- }
- else if (n == 0 && opt_check_subsets && get_ring_subset_instances(txn, txin.amount, new_ring) >= new_ring.size())
- {
- for (size_t o = 0; o < new_ring.size(); ++o)
- {
- const std::pair<uint64_t, uint64_t> output = std::make_pair(txin.amount, absolute[o]);
- if (opt_verbose)
- {
- MINFO("Marking output " << output.first << "/" << output.second << " as spent, due to being used in " << new_ring.size() << " subsets of " << new_ring.size() << "-rings");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- }
- blackballs.push_back(output);
- if (add_spent_output(cur, output_data(txin.amount, absolute[o])))
- inc_stat(txn, txin.amount ? "pre-rct-subset-rings" : "rct-subset-rings");
- }
- }
- else if (n > 0 && get_relative_ring(txn, txin.k_image, relative_ring))
- {
- MDEBUG("Key image " << txin.k_image << " already seen: rings " <<
- boost::join(relative_ring | boost::adaptors::transformed([](uint64_t out){return std::to_string(out);}), " ") <<
- ", " << boost::join(txin.key_offsets | boost::adaptors::transformed([](uint64_t out){return std::to_string(out);}), " "));
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- if (relative_ring != txin.key_offsets)
- {
- MDEBUG("Rings are different");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- const std::vector<uint64_t> r0 = cryptonote::relative_output_offsets_to_absolute(relative_ring);
- const std::vector<uint64_t> r1 = cryptonote::relative_output_offsets_to_absolute(txin.key_offsets);
- std::vector<uint64_t> common;
- for (uint64_t out: r0)
- {
- if (std::find(r1.begin(), r1.end(), out) != r1.end())
- common.push_back(out);
- }
- if (common.empty())
- {
- MERROR("Rings for the same key image are disjoint");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- }
- else if (common.size() == 1)
- {
- const std::pair<uint64_t, uint64_t> output = std::make_pair(txin.amount, common[0]);
- if (opt_verbose)
- {
- MINFO("Marking output " << output.first << "/" << output.second << " as spent, due to being used in rings with a single common element");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- }
- blackballs.push_back(output);
- if (add_spent_output(cur, output_data(txin.amount, common[0])))
- inc_stat(txn, txin.amount ? "pre-rct-key-image-attack" : "rct-key-image-attack");
- }
- else
- {
- MDEBUG("The intersection has more than one element, it's still ok");
- std::cout << "\r" << start_idx << "/" << n_txes << " \r" << std::flush;
- for (const auto &out: r0)
- if (std::find(common.begin(), common.end(), out) != common.end())
- new_ring.push_back(out);
- new_ring = cryptonote::absolute_output_offsets_to_relative(new_ring);
- }
- }
- }
- if (n == 0)
- {
- set_relative_ring(txn, txin.k_image, new_ring);
- if (!opt_rct_only)
- inc_per_amount_outputs(txn, txin.amount, 0, 1);
- }
- }
- set_processed_txidx(txn, canonical, start_idx+1);
- if (!opt_rct_only)
- {
- for (const auto &out: tx.vout)
- {
- uint64_t amount = out.amount;
- if (miner_tx && tx.version >= 2)
- amount = 0;
-
- if (opt_rct_only && amount != 0)
- continue;
- if (out.target.type() != typeid(txout_to_key))
- continue;
- inc_per_amount_outputs(txn, amount, 1, 0);
- }
- }
-
- ++records;
- if (records >= records_per_sync)
- {
- if (!blackballs.empty())
- {
- ringdb.blackball(blackballs);
- blackballs.clear();
- }
- mdb_cursor_close(cur);
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn creating/opening database: " + std::string(mdb_strerror(dbr)));
- int dbr = resize_env(cache_dir.c_str());
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to resize LMDB database: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_txn_begin(env, NULL, 0, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB cursor: " + std::string(mdb_strerror(dbr)));
- records = 0;
- }
-
- if (stop_requested)
- {
- MINFO("Stopping scan...");
- return false;
- }
- return true;
- });
- mdb_cursor_close(cur);
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn creating/opening database: " + std::string(mdb_strerror(dbr)));
- LOG_PRINT_L0("blockchain from " << inputs[n] << " processed till tx idx " << start_idx);
- if (stop_requested)
- break;
- }
-
- if (stop_requested)
- goto skip_secondary_passes;
-
- if (opt_force_chain_reaction_pass || get_num_spent_outputs() > start_blackballed_outputs)
- {
- MDB_txn *txn;
- dbr = mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- work_spent = get_spent_outputs(txn);
- mdb_txn_abort(txn);
- }
-
- while (!work_spent.empty())
- {
- LOG_PRINT_L0("Secondary pass on " << work_spent.size() << " spent outputs");
-
- int dbr = resize_env(cache_dir.c_str());
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to resize LMDB database: " + std::string(mdb_strerror(dbr)));
-
- MDB_txn *txn;
- dbr = mdb_txn_begin(env, NULL, 0, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- MDB_cursor *cur;
- dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB cursor: " + std::string(mdb_strerror(dbr)));
-
- std::vector<std::pair<uint64_t, uint64_t>> blackballs;
- std::vector<output_data> scan_spent = std::move(work_spent);
- work_spent.clear();
- for (const output_data &od: scan_spent)
- {
- std::vector<crypto::key_image> key_images = get_key_images(txn, od);
- for (const crypto::key_image &ki: key_images)
- {
- std::vector<uint64_t> relative_ring;
- CHECK_AND_ASSERT_THROW_MES(get_relative_ring(txn, ki, relative_ring), "Relative ring not found");
- std::vector<uint64_t> absolute = cryptonote::relative_output_offsets_to_absolute(relative_ring);
- size_t known = 0;
- uint64_t last_unknown = 0;
- for (uint64_t out: absolute)
- {
- output_data new_od(od.amount, out);
- if (is_output_spent(cur, new_od))
- ++known;
- else
- last_unknown = out;
- }
- if (known == absolute.size() - 1)
- {
- const std::pair<uint64_t, uint64_t> output = std::make_pair(od.amount, last_unknown);
- if (opt_verbose)
- {
- MINFO("Marking output " << output.first << "/" << output.second << " as spent, due to being used in a " <<
- absolute.size() << "-ring where all other outputs are known to be spent");
- }
- blackballs.push_back(output);
- if (add_spent_output(cur, output_data(od.amount, last_unknown)))
- inc_stat(txn, od.amount ? "pre-rct-chain-reaction" : "rct-chain-reaction");
- work_spent.push_back(output_data(od.amount, last_unknown));
- }
- }
-
- if (stop_requested)
- {
- MINFO("Stopping secondary passes. Secondary passes are not incremental, they will re-run fully.");
- return 0;
- }
- }
- if (!blackballs.empty())
- {
- ringdb.blackball(blackballs);
- blackballs.clear();
- }
- mdb_cursor_close(cur);
- dbr = mdb_txn_commit(txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to commit txn creating/opening database: " + std::string(mdb_strerror(dbr)));
- }
-
-skip_secondary_passes:
- uint64_t diff = get_num_spent_outputs() - start_blackballed_outputs;
- LOG_PRINT_L0(std::to_string(diff) << " new outputs marked as spent, " << get_num_spent_outputs() << " total outputs marked as spent");
-
- MDB_txn *txn;
- dbr = mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- uint64_t pre_rct = 0, rct = 0;
- get_num_outputs(txn0, cur0, dbi0, pre_rct, rct);
- MINFO("Total pre-rct outputs: " << pre_rct);
- MINFO("Total rct outputs: " << rct);
- static const struct { const char *key; uint64_t base; } stat_keys[] = {
- { "pre-rct-ring-size-1", pre_rct }, { "rct-ring-size-1", rct },
- { "pre-rct-duplicate-rings", pre_rct }, { "rct-duplicate-rings", rct },
- { "pre-rct-subset-rings", pre_rct }, { "rct-subset-rings", rct },
- { "pre-rct-full-count", pre_rct }, { "rct-full-count", rct },
- { "pre-rct-key-image-attack", pre_rct }, { "rct-key-image-attack", rct },
- { "pre-rct-extra", pre_rct }, { "rct-ring-extra", rct },
- { "pre-rct-chain-reaction", pre_rct }, { "rct-chain-reaction", rct },
- };
- for (const auto &key: stat_keys)
- {
- uint64_t data;
- if (!get_stat(txn, key.key, data))
- data = 0;
- float percent = key.base ? 100.0f * data / key.base : 0.0f;
- MINFO(key.key << ": " << data << " (" << percent << "%)");
- }
- mdb_txn_abort(txn);
-
- if (!opt_export.empty())
- {
- MDB_txn *txn;
- int dbr = mdb_txn_begin(env, NULL, 0, &txn);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- MDB_cursor *cur;
- dbr = mdb_cursor_open(txn, dbi_spent, &cur);
- CHECK_AND_ASSERT_THROW_MES(!dbr, "Failed to open LMDB cursor: " + std::string(mdb_strerror(dbr)));
- export_spent_outputs(cur, opt_export);
- mdb_cursor_close(cur);
- mdb_txn_abort(txn);
- }
-
- LOG_PRINT_L0("Blockchain spent output data exported OK");
- close_db(env0, txn0, cur0, dbi0);
- close();
- return 0;
-
- CATCH_ENTRY("Error", 1);
-}
diff --git a/src/simplewallet/simplewallet.cpp b/src/simplewallet/simplewallet.cpp
index 50c61d4..c64b5bb 100644
--- a/src/simplewallet/simplewallet.cpp
+++ b/src/simplewallet/simplewallet.cpp
@@ -272,9 +272,6 @@ namespace
const char* USAGE_SET_RING("set_ring <filename> | ( <key_image> absolute|relative <index> [<index>...] )");
const char* USAGE_UNSET_RING("unset_ring <txid> | ( <key_image> [<key_image>...] )");
const char* USAGE_SAVE_KNOWN_RINGS("save_known_rings");
- const char* USAGE_MARK_OUTPUT_SPENT("mark_output_spent <amount>/<offset> | <filename> [add]");
- const char* USAGE_MARK_OUTPUT_UNSPENT("mark_output_unspent <amount>/<offset>");
- const char* USAGE_IS_OUTPUT_SPENT("is_output_spent <amount>/<offset>");
const char* USAGE_FREEZE("freeze <key_image>");
const char* USAGE_THAW("thaw <key_image>");
const char* USAGE_FROZEN("frozen <key_image>");
@@ -2000,152 +1997,6 @@ bool simple_wallet::unset_ring(const std::vector<std::string> &args)
return true;
}
-bool simple_wallet::blackball(const std::vector<std::string> &args)
-{
- uint64_t amount = std::numeric_limits<uint64_t>::max(), offset, num_offsets;
- if (args.size() == 0)
- {
- PRINT_USAGE(USAGE_MARK_OUTPUT_SPENT);
- return true;
- }
-
- try
- {
- if (sscanf(args[0].c_str(), "%" PRIu64 "/%" PRIu64, &amount, &offset) == 2)
- {
- m_wallet->blackball_output(std::make_pair(amount, offset));
- }
- else if (epee::file_io_utils::is_file_exist(args[0]))
- {
- std::vector<std::pair<uint64_t, uint64_t>> outputs;
- char str[256];
-
- std::unique_ptr<FILE, tools::close_file> f(fopen(args[0].c_str(), "r"));
- if (f)
- {
- while (!feof(f.get()))
- {
- if (!fgets(str, sizeof(str), f.get()))
- break;
- const size_t len = strlen(str);
- if (len > 0 && str[len - 1] == '\n')
- str[len - 1] = 0;
- if (!str[0])
- continue;
- if (sscanf(str, "@%" PRIu64, &amount) == 1)
- {
- continue;
- }
- if (amount == std::numeric_limits<uint64_t>::max())
- {
- fail_msg_writer() << tr("First line is not an amount");
- return true;
- }
- if (sscanf(str, "%" PRIu64 "*%" PRIu64, &offset, &num_offsets) == 2 && num_offsets <= std::numeric_limits<uint64_t>::max() - offset)
- {
- while (num_offsets--)
- outputs.push_back(std::make_pair(amount, offset++));
- }
- else if (sscanf(str, "%" PRIu64, &offset) == 1)
- {
- outputs.push_back(std::make_pair(amount, offset));
- }
- else
- {
- fail_msg_writer() << tr("Invalid output: ") << str;
- return true;
- }
- }
- f.reset();
- bool add = false;
- if (args.size() > 1)
- {
- if (args[1] != "add")
- {
- fail_msg_writer() << tr("Bad argument: ") + args[1] + ": " + tr("should be \"add\"");
- return true;
- }
- add = true;
- }
- m_wallet->set_blackballed_outputs(outputs, add);
- }
- else
- {
- fail_msg_writer() << tr("Failed to open file");
- return true;
- }
- }
- else
- {
- fail_msg_writer() << tr("Invalid output key, and file doesn't exist");
- return true;
- }
- }
- catch (const std::exception &e)
- {
- fail_msg_writer() << tr("Failed to mark output spent: ") << e.what();
- }
-
- return true;
-}
-
-bool simple_wallet::unblackball(const std::vector<std::string> &args)
-{
- std::pair<uint64_t, uint64_t> output;
- if (args.size() != 1)
- {
- PRINT_USAGE(USAGE_MARK_OUTPUT_UNSPENT);
- return true;
- }
-
- if (sscanf(args[0].c_str(), "%" PRIu64 "/%" PRIu64, &output.first, &output.second) != 2)
- {
- fail_msg_writer() << tr("Invalid output");
- return true;
- }
-
- try
- {
- m_wallet->unblackball_output(output);
- }
- catch (const std::exception &e)
- {
- fail_msg_writer() << tr("Failed to mark output unspent: ") << e.what();
- }
-
- return true;
-}
-
-bool simple_wallet::blackballed(const std::vector<std::string> &args)
-{
- std::pair<uint64_t, uint64_t> output;
- if (args.size() != 1)
- {
- PRINT_USAGE(USAGE_IS_OUTPUT_SPENT);
- return true;
- }
-
- if (sscanf(args[0].c_str(), "%" PRIu64 "/%" PRIu64, &output.first, &output.second) != 2)
- {
- fail_msg_writer() << tr("Invalid output");
- return true;
- }
-
- try
- {
- if (m_wallet->is_output_blackballed(output))
- message_writer() << tr("Spent: ") << output.first << "/" << output.second;
- else
- message_writer() << tr("Not spent: ") << output.first << "/" << output.second;
- }
- catch (const std::exception &e)
- {
- fail_msg_writer() << tr("Failed to check whether output is spent: ") << e.what();
- }
-
- return true;
-}
-
bool simple_wallet::save_known_rings(const std::vector<std::string> &args)
{
fail_msg_writer() << tr("save_known_rings is deprecated");
@@ -3629,18 +3480,6 @@ simple_wallet::simple_wallet()
boost::bind(&simple_wallet::on_command, this, &simple_wallet::save_known_rings, _1),
tr(USAGE_SAVE_KNOWN_RINGS),
tr("Save known rings to the shared rings database"));
- m_cmd_binder.set_handler("mark_output_spent",
- boost::bind(&simple_wallet::on_command, this, &simple_wallet::blackball, _1),
- tr(USAGE_MARK_OUTPUT_SPENT),
- tr("Mark output(s) as spent so they never get selected as fake outputs in a ring"));
- m_cmd_binder.set_handler("mark_output_unspent",
- boost::bind(&simple_wallet::on_command, this, &simple_wallet::unblackball, _1),
- tr(USAGE_MARK_OUTPUT_UNSPENT),
- tr("Marks an output as unspent so it may get selected as a fake output in a ring"));
- m_cmd_binder.set_handler("is_output_spent",
- boost::bind(&simple_wallet::on_command, this, &simple_wallet::blackballed, _1),
- tr(USAGE_IS_OUTPUT_SPENT),
- tr("Checks whether an output is marked as spent"));
m_cmd_binder.set_handler("freeze",
boost::bind(&simple_wallet::on_command, this, &simple_wallet::freeze, _1),
tr(USAGE_FREEZE),
diff --git a/src/simplewallet/simplewallet.h b/src/simplewallet/simplewallet.h
index 651a7a0..0a0f779 100644
--- a/src/simplewallet/simplewallet.h
+++ b/src/simplewallet/simplewallet.h
@@ -247,9 +247,6 @@ namespace cryptonote
bool set_ring(const std::vector<std::string>& args);
bool unset_ring(const std::vector<std::string>& args);
bool save_known_rings(const std::vector<std::string>& args);
- bool blackball(const std::vector<std::string>& args);
- bool unblackball(const std::vector<std::string>& args);
- bool blackballed(const std::vector<std::string>& args);
bool freeze(const std::vector<std::string>& args);
bool thaw(const std::vector<std::string>& args);
bool frozen(const std::vector<std::string>& args);
diff --git a/src/wallet/api/wallet.cpp b/src/wallet/api/wallet.cpp
index e398c1a..ecf1458 100644
--- a/src/wallet/api/wallet.cpp
+++ b/src/wallet/api/wallet.cpp
@@ -2604,88 +2604,6 @@ bool WalletImpl::useForkRules(uint8_t version, int64_t early_blocks) const
return m_wallet->use_fork_rules(version,early_blocks);
}
-bool WalletImpl::blackballOutputs(const std::vector<std::string> &outputs, bool add)
-{
- std::vector<std::pair<uint64_t, uint64_t>> raw_outputs;
- raw_outputs.reserve(outputs.size());
- uint64_t amount = std::numeric_limits<uint64_t>::max(), offset, num_offsets;
- for (const std::string &str: outputs)
- {
- if (sscanf(str.c_str(), "@%" PRIu64, &amount) == 1)
- continue;
- if (amount == std::numeric_limits<uint64_t>::max())
- {
- setStatusError("First line is not an amount");
- return true;
- }
- if (sscanf(str.c_str(), "%" PRIu64 "*%" PRIu64, &offset, &num_offsets) == 2 && num_offsets <= std::numeric_limits<uint64_t>::max() - offset)
- {
- while (num_offsets--)
- raw_outputs.push_back(std::make_pair(amount, offset++));
- }
- else if (sscanf(str.c_str(), "%" PRIu64, &offset) == 1)
- {
- raw_outputs.push_back(std::make_pair(amount, offset));
- }
- else
- {
- setStatusError(tr("Invalid output: ") + str);
- return false;
- }
- }
- bool ret = m_wallet->set_blackballed_outputs(raw_outputs, add);
- if (!ret)
- {
- setStatusError(tr("Failed to mark outputs as spent"));
- return false;
- }
- return true;
-}
-
-bool WalletImpl::blackballOutput(const std::string &amount, const std::string &offset)
-{
- uint64_t raw_amount, raw_offset;
- if (!epee::string_tools::get_xtype_from_string(raw_amount, amount))
- {
- setStatusError(tr("Failed to parse output amount"));
- return false;
- }
- if (!epee::string_tools::get_xtype_from_string(raw_offset, offset))
- {
- setStatusError(tr("Failed to parse output offset"));
- return false;
- }
- bool ret = m_wallet->blackball_output(std::make_pair(raw_amount, raw_offset));
- if (!ret)
- {
- setStatusError(tr("Failed to mark output as spent"));
- return false;
- }
- return true;
-}
-
-bool WalletImpl::unblackballOutput(const std::string &amount, const std::string &offset)
-{
- uint64_t raw_amount, raw_offset;
- if (!epee::string_tools::get_xtype_from_string(raw_amount, amount))
- {
- setStatusError(tr("Failed to parse output amount"));
- return false;
- }
- if (!epee::string_tools::get_xtype_from_string(raw_offset, offset))
- {
- setStatusError(tr("Failed to parse output offset"));
- return false;
- }
- bool ret = m_wallet->unblackball_output(std::make_pair(raw_amount, raw_offset));
- if (!ret)
- {
- setStatusError(tr("Failed to mark output as unspent"));
- return false;
- }
- return true;
-}
-
bool WalletImpl::getRing(const std::string &key_image, std::vector<uint64_t> &ring) const
{
crypto::key_image raw_key_image;
diff --git a/src/wallet/api/wallet.h b/src/wallet/api/wallet.h
index d48d7f1..a5f1d91 100644
--- a/src/wallet/api/wallet.h
+++ b/src/wallet/api/wallet.h
@@ -216,9 +216,6 @@ public:
virtual bool parse_uri(const std::string &uri, std::string &address, std::string &payment_id, uint64_t &amount, std::string &tx_description, std::string &recipient_name, std::vector<std::string> &unknown_parameters, std::string &error) override;
virtual std::string make_uri(const std::string &address, const std::string &payment_id, uint64_t amount, const std::string &tx_description, const std::string &recipient_name, std::string &error) const override;
virtual std::string getDefaultDataDir() const override;
- virtual bool blackballOutputs(const std::vector<std::string> &outputs, bool add) override;
- virtual bool blackballOutput(const std::string &amount, const std::string &offset) override;
- virtual bool unblackballOutput(const std::string &amount, const std::string &offset) override;
virtual bool getRing(const std::string &key_image, std::vector<uint64_t> &ring) const override;
virtual bool getRings(const std::string &txid, std::vector<std::pair<std::string, std::vector<uint64_t>>> &rings) const override;
virtual bool setRing(const std::string &key_image, const std::vector<uint64_t> &ring, bool relative) override;
diff --git a/src/wallet/api/wallet2_api.h b/src/wallet/api/wallet2_api.h
index 2bedcc7..ff4bf8e 100644
--- a/src/wallet/api/wallet2_api.h
+++ b/src/wallet/api/wallet2_api.h
@@ -1082,15 +1082,6 @@ struct Wallet
*/
virtual void setOffline(bool offline) = 0;
virtual bool isOffline() const = 0;
-
- //! blackballs a set of outputs
- virtual bool blackballOutputs(const std::vector<std::string> &outputs, bool add) = 0;
-
- //! blackballs an output
- virtual bool blackballOutput(const std::string &amount, const std::string &offset) = 0;
-
- //! unblackballs an output
- virtual bool unblackballOutput(const std::string &amount, const std::string &offset) = 0;
//! gets the ring used for a key image, if any
virtual bool getRing(const std::string &key_image, std::vector<uint64_t> &ring) const = 0;
diff --git a/src/wallet/ringdb.cpp b/src/wallet/ringdb.cpp
index 8802f73..2afd508 100644
--- a/src/wallet/ringdb.cpp
+++ b/src/wallet/ringdb.cpp
@@ -57,13 +57,6 @@ static int compare_hash32(const MDB_val *a, const MDB_val *b)
return 0;
}
-static int compare_uint64(const MDB_val *a, const MDB_val *b)
-{
- const uint64_t va = *(const uint64_t*) a->mv_data;
- const uint64_t vb = *(const uint64_t*) b->mv_data;
- return va < vb ? -1 : va > vb;
-}
-
static std::string compress_ring(const std::vector<uint64_t> &ring, uint64_t tag)
{
std::string s;
@@ -201,8 +194,6 @@ static size_t get_ring_data_size(size_t n_entries)
return n_entries * (32 + 1024); // highball 1kB for the ring data to make sure
}
-enum { BLACKBALL_BLACKBALL, BLACKBALL_UNBLACKBALL, BLACKBALL_QUERY, BLACKBALL_CLEAR};
-
namespace tools
{
@@ -234,10 +225,6 @@ ringdb::ringdb(std::string filename, const std::string &genesis):
THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
mdb_set_compare(txn, dbi_rings, compare_hash32);
- dbr = mdb_dbi_open(txn, ("blackballs2-" + genesis).c_str(), MDB_CREATE | MDB_INTEGERKEY | MDB_DUPSORT | MDB_DUPFIXED, &dbi_blackballs);
- THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to open LMDB dbi: " + std::string(mdb_strerror(dbr)));
- mdb_set_dupsort(txn, dbi_blackballs, compare_uint64);
-
dbr = mdb_txn_commit(txn);
THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to commit txn creating/opening database: " + std::string(mdb_strerror(dbr)));
tx_active = false;
@@ -253,7 +240,6 @@ void ringdb::close()
if (env)
{
mdb_dbi_close(env, dbi_rings);
- mdb_dbi_close(env, dbi_blackballs);
mdb_env_close(env);
env = NULL;
}
@@ -439,103 +425,4 @@ bool ringdb::set_ring(const crypto::chacha_key &chacha_key, const crypto::key_im
return set_rings(chacha_key, rings, relative);
}
-bool ringdb::blackball_worker(const std::vector<std::pair<uint64_t, uint64_t>> &outputs, int op)
-{
- MDB_txn *txn;
- MDB_cursor *cursor;
- int dbr;
- bool tx_active = false;
- bool ret = true;
-
- THROW_WALLET_EXCEPTION_IF(outputs.size() > 1 && op == BLACKBALL_QUERY, tools::error::wallet_internal_error, "Blackball query only makes sense for a single output");
-
- dbr = resize_env(env, filename.c_str(), 32 * 2 * outputs.size()); // a pubkey, and some slack
- THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to set env map size: " + std::string(mdb_strerror(dbr)));
- dbr = mdb_txn_begin(env, NULL, 0, &txn);
- THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to create LMDB transaction: " + std::string(mdb_strerror(dbr)));
- epee::misc_utils::auto_scope_leave_caller txn_dtor = epee::misc_utils::create_scope_leave_handler([&](){if (tx_active) mdb_txn_abort(txn);});
- tx_active = true;
-
- dbr = mdb_cursor_open(txn, dbi_blackballs, &cursor);
- THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to create cursor for blackballs table: " + std::string(mdb_strerror(dbr)));
-
- MDB_val key, data;
- for (const std::pair<uint64_t, uint64_t> &output: outputs)
- {
- key.mv_data = (void*)&output.first;
- key.mv_size = sizeof(output.first);
- data.mv_data = (void*)&output.second;
- data.mv_size = sizeof(output.second);
-
- switch (op)
- {
- case BLACKBALL_BLACKBALL:
- MDEBUG("Marking output " << output.first << "/" << output.second << " as spent");
- dbr = mdb_cursor_put(cursor, &key, &data, MDB_NODUPDATA);
- if (dbr == MDB_KEYEXIST)
- dbr = 0;
- break;
- case BLACKBALL_UNBLACKBALL:
- MDEBUG("Marking output " << output.first << "/" << output.second << " as unspent");
- dbr = mdb_cursor_get(cursor, &key, &data, MDB_GET_BOTH);
- if (dbr == 0)
- dbr = mdb_cursor_del(cursor, 0);
- break;
- case BLACKBALL_QUERY:
- dbr = mdb_cursor_get(cursor, &key, &data, MDB_GET_BOTH);
- THROW_WALLET_EXCEPTION_IF(dbr && dbr != MDB_NOTFOUND, tools::error::wallet_internal_error, "Failed to lookup in blackballs table: " + std::string(mdb_strerror(dbr)));
- ret = dbr != MDB_NOTFOUND;
- if (dbr == MDB_NOTFOUND)
- dbr = 0;
- break;
- case BLACKBALL_CLEAR:
- break;
- default:
- THROW_WALLET_EXCEPTION(tools::error::wallet_internal_error, "Invalid blackball op");
- }
- THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to query blackballs table: " + std::string(mdb_strerror(dbr)));
- }
-
- mdb_cursor_close(cursor);
-
- if (op == BLACKBALL_CLEAR)
- {
- dbr = mdb_drop(txn, dbi_blackballs, 0);
- THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to clear blackballs table: " + std::string(mdb_strerror(dbr)));
- }
-
- dbr = mdb_txn_commit(txn);
- THROW_WALLET_EXCEPTION_IF(dbr, tools::error::wallet_internal_error, "Failed to commit txn blackballing output to database: " + std::string(mdb_strerror(dbr)));
- tx_active = false;
- return ret;
-}
-
-bool ringdb::blackball(const std::vector<std::pair<uint64_t, uint64_t>> &outputs)
-{
- return blackball_worker(outputs, BLACKBALL_BLACKBALL);
-}
-
-bool ringdb::blackball(const std::pair<uint64_t, uint64_t> &output)
-{
- std::vector<std::pair<uint64_t, uint64_t>> outputs(1, output);
- return blackball_worker(outputs, BLACKBALL_BLACKBALL);
-}
-
-bool ringdb::unblackball(const std::pair<uint64_t, uint64_t> &output)
-{
- std::vector<std::pair<uint64_t, uint64_t>> outputs(1, output);
- return blackball_worker(outputs, BLACKBALL_UNBLACKBALL);
-}
-
-bool ringdb::blackballed(const std::pair<uint64_t, uint64_t> &output)
-{
- std::vector<std::pair<uint64_t, uint64_t>> outputs(1, output);
- return blackball_worker(outputs, BLACKBALL_QUERY);
-}
-
-bool ringdb::clear_blackballs()
-{
- return blackball_worker(std::vector<std::pair<uint64_t, uint64_t>>(), BLACKBALL_CLEAR);
-}
-
}
diff --git a/src/wallet/ringdb.h b/src/wallet/ringdb.h
index 13412cd..45045c2 100644
--- a/src/wallet/ringdb.h
+++ b/src/wallet/ringdb.h
@@ -53,19 +53,9 @@ namespace tools
bool set_ring(const crypto::chacha_key &chacha_key, const crypto::key_image &key_image, const std::vector<uint64_t> &outs, bool relative);
bool set_rings(const crypto::chacha_key &chacha_key, const std::vector<std::pair<crypto::key_image, std::vector<uint64_t>>> &rings, bool relative);
- bool blackball(const std::pair<uint64_t, uint64_t> &output);
- bool blackball(const std::vector<std::pair<uint64_t, uint64_t>> &outputs);
- bool unblackball(const std::pair<uint64_t, uint64_t> &output);
- bool blackballed(const std::pair<uint64_t, uint64_t> &output);
- bool clear_blackballs();
-
- private:
- bool blackball_worker(const std::vector<std::pair<uint64_t, uint64_t>> &outputs, int op);
-
private:
std::string filename;
MDB_env *env;
MDB_dbi dbi_rings;
- MDB_dbi dbi_blackballs;
};
}
diff --git a/src/wallet/wallet2.cpp b/src/wallet/wallet2.cpp
index b89370f..bb488e5 100644
--- a/src/wallet/wallet2.cpp
+++ b/src/wallet/wallet2.cpp
@@ -8864,45 +8864,6 @@ bool wallet2::find_and_save_rings(bool force)
return true;
}
-bool wallet2::blackball_output(const std::pair<uint64_t, uint64_t> &output)
-{
- if (!m_ringdb)
- return false;
- try { return m_ringdb->blackball(output); }
- catch (const std::exception &e) { return false; }
-}
-
-bool wallet2::set_blackballed_outputs(const std::vector<std::pair<uint64_t, uint64_t>> &outputs, bool add)
-{
- if (!m_ringdb)
- return false;
- try
- {
- bool ret = true;
- if (!add)
- ret &= m_ringdb->clear_blackballs();
- ret &= m_ringdb->blackball(outputs);
- return ret;
- }
- catch (const std::exception &e) { return false; }
-}
-
-bool wallet2::unblackball_output(const std::pair<uint64_t, uint64_t> &output)
-{
- if (!m_ringdb)
- return false;
- try { return m_ringdb->unblackball(output); }
- catch (const std::exception &e) { return false; }
-}
-
-bool wallet2::is_output_blackballed(const std::pair<uint64_t, uint64_t> &output) const
-{
- if (!m_ringdb)
- return false;
- try { return m_ringdb->blackballed(output); }
- catch (const std::exception &e) { return false; }
-}
-
bool wallet2::lock_keys_file()
{
if (m_wallet_file.empty())
@@ -8987,8 +8948,6 @@ bool wallet2::tx_add_fake_output(std::vector<std::vector<tools::wallet2::get_out
return false;
}
valid_public_keys_cache.insert(rct::rct2pk(mask));
-// if (is_output_blackballed(output_public_key)) // don't add blackballed outputs
-// return false;
outs.back().push_back(item);
return true;
}
@@ -9378,24 +9337,13 @@ void wallet2::get_outs(std::vector<std::vector<tools::wallet2::get_outs_entry>>
std::unordered_map<const char*, std::set<uint64_t>> picks;
// while we still need more mixins
- uint64_t num_usable_outs = num_outs;
- bool allow_blackballed = false;
- MDEBUG("Starting gamma picking with " << num_outs << ", num_usable_outs " << num_usable_outs
+ MDEBUG("Starting gamma picking with " << num_outs
<< ", requested_outputs_count " << requested_outputs_count);
while (num_found < requested_outputs_count)
{
// if we've gone through every possible output, we've gotten all we can
- if (seen_indices.size() == num_usable_outs)
- {
- // there is a first pass which rejects blackballed outputs, then a second pass
- // which allows them if we don't have enough non blackballed outputs to reach
- // the required amount of outputs (since consensus does not care about blackballed
- // outputs, we still need to reach the minimum ring size)
- if (allow_blackballed)
- break;
- MINFO("Not enough output not marked as spent, we'll allow outputs marked as spent");
- allow_blackballed = true;
- num_usable_outs = num_outs;
+ if (seen_indices.size() == num_outs) {
+ break;
}
// get a random output index from the DB. If we've already seen it,
@@ -9471,11 +9419,6 @@ void wallet2::get_outs(std::vector<std::vector<tools::wallet2::get_outs_entry>>
if (seen_indices.count(i))
continue;
- if (!allow_blackballed && is_output_blackballed(std::make_pair(amount, i))) // don't add blackballed outputs
- {
- --num_usable_outs;
- continue;
- }
seen_indices.emplace(i);
picks[type].insert(i);
diff --git a/src/wallet/wallet2.h b/src/wallet/wallet2.h
index 36e5082..8030e44 100644
--- a/src/wallet/wallet2.h
+++ b/src/wallet/wallet2.h
@@ -1735,11 +1735,6 @@ private:
bool unset_ring(const crypto::hash &txid);
[[deprecated]] bool find_and_save_rings(bool force = true);
- bool blackball_output(const std::pair<uint64_t, uint64_t> &output);
- bool set_blackballed_outputs(const std::vector<std::pair<uint64_t, uint64_t>> &outputs, bool add = false);
- bool unblackball_output(const std::pair<uint64_t, uint64_t> &output);
- bool is_output_blackballed(const std::pair<uint64_t, uint64_t> &output) const;
-
void freeze(size_t idx);
void thaw(size_t idx);
bool frozen(size_t idx) const;
diff --git a/tests/unit_tests/ringdb.cpp b/tests/unit_tests/ringdb.cpp
index cd403f9..97db426 100644
--- a/tests/unit_tests/ringdb.cpp
+++ b/tests/unit_tests/ringdb.cpp
@@ -60,23 +60,13 @@ static crypto::key_image generate_key_image()
return key_image;
}
-static std::pair<uint64_t, uint64_t> generate_output()
-{
- return std::make_pair(rand(), rand());
-}
-
-
static crypto::chacha_key get_KEY_1() { static const crypto::chacha_key KEY_1 = generate_chacha_key(); return KEY_1; }
static crypto::chacha_key get_KEY_2() { static const crypto::chacha_key KEY_2 = generate_chacha_key(); return KEY_2; }
static crypto::key_image get_KEY_IMAGE_1() { static const crypto::key_image KEY_IMAGE_1 = generate_key_image(); return KEY_IMAGE_1; }
-static std::pair<uint64_t, uint64_t> get_OUTPUT_1() { static const std::pair<uint64_t, uint64_t> OUTPUT_1 = generate_output(); return OUTPUT_1; }
-static std::pair<uint64_t, uint64_t> get_OUTPUT_2() { static const std::pair<uint64_t, uint64_t> OUTPUT_2 = generate_output(); return OUTPUT_2; }
#define KEY_1 get_KEY_1()
#define KEY_2 get_KEY_2()
#define KEY_IMAGE_1 get_KEY_IMAGE_1()
-#define OUTPUT_1 get_OUTPUT_1()
-#define OUTPUT_2 get_OUTPUT_2()
class RingDB: public tools::ringdb
{
@@ -142,58 +132,3 @@ TEST(ringdb, different_genesis)
ASSERT_TRUE(ringdb.set_ring(KEY_1, KEY_IMAGE_1, outs, false));
ASSERT_FALSE(ringdb.get_ring(KEY_2, KEY_IMAGE_1, outs2));
}
-
-TEST(spent_outputs, not_found)
-{
- RingDB ringdb;
- ASSERT_TRUE(ringdb.blackball(OUTPUT_1));
- ASSERT_FALSE(ringdb.blackballed(OUTPUT_2));
-}
-
-TEST(spent_outputs, found)
-{
- RingDB ringdb;
- ASSERT_TRUE(ringdb.blackball(OUTPUT_1));
- ASSERT_TRUE(ringdb.blackballed(OUTPUT_1));
-}
-
-TEST(spent_outputs, vector)
-{
- RingDB ringdb;
- std::vector<std::pair<uint64_t, uint64_t>> outputs;
- outputs.push_back(std::make_pair(0, 1));
- outputs.push_back(std::make_pair(10, 3));
- outputs.push_back(std::make_pair(10, 4));
- outputs.push_back(std::make_pair(10, 8));
- outputs.push_back(std::make_pair(20, 0));
- outputs.push_back(std::make_pair(20, 1));
- outputs.push_back(std::make_pair(30, 5));
- ASSERT_TRUE(ringdb.blackball(outputs));
- ASSERT_TRUE(ringdb.blackballed(std::make_pair(0, 1)));
- ASSERT_FALSE(ringdb.blackballed(std::make_pair(10, 2)));
- ASSERT_TRUE(ringdb.blackballed(std::make_pair(10, 3)));
- ASSERT_TRUE(ringdb.blackballed(std::make_pair(10, 4)));
- ASSERT_FALSE(ringdb.blackballed(std::make_pair(10, 5)));
- ASSERT_TRUE(ringdb.blackballed(std::make_pair(10, 8)));
- ASSERT_TRUE(ringdb.blackballed(std::make_pair(20, 0)));
- ASSERT_TRUE(ringdb.blackballed(std::make_pair(20, 1)));
- ASSERT_FALSE(ringdb.blackballed(std::make_pair(20, 2)));
- ASSERT_TRUE(ringdb.blackballed(std::make_pair(30, 5)));
-}
-
-TEST(spent_outputs, mark_as_unspent)
-{
- RingDB ringdb;
- ASSERT_TRUE(ringdb.blackball(OUTPUT_1));
- ASSERT_TRUE(ringdb.unblackball(OUTPUT_1));
- ASSERT_FALSE(ringdb.blackballed(OUTPUT_1));
-}
-
-TEST(spent_outputs, clear)
-{
- RingDB ringdb;
- ASSERT_TRUE(ringdb.blackball(OUTPUT_1));
- ASSERT_TRUE(ringdb.clear_blackballs());
- ASSERT_FALSE(ringdb.blackballed(OUTPUT_1));
-}
-
Why this scored 22/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.