Maand: juni 2020

I have the merkle root, using this value, how to get the block hash or its height?
There is this merkle root available in getblock RPC output:

./bitcoin-cli getblock 0320d6c1bd3c4cd2a08d6f76acb50b06a3ed766b058d247fbda3147aecfef388
{
  "tx": [
    "bccf4c873984245694f64263a3392c4d67c6a6f60efe4ed53aa4965f6d8b7dc0"
  ],
  "hash": "0320d6c1bd3c4cd2a08d6f76acb50b06a3ed766b058d247fbda3147aecfef388",
  "confirmations": 2,
  "size": 180,
  "height": 102,
  "version": 536870912,
  "versionHex": "20000000",
  "merkleroot": "bccf4c873984245694f64263a3392c4d67c6a6f60efe4ed53aa4965f6d8b7dc0",
  "num_tx": 1,
  "time": 1593523883,
  "mediantime": 1593467398,
  "nonce": 1,
  "bits": "207fffff",
  "difficulty": 4.656542373906925e-10,
  "chainwork": "00000000000000000000000000000000000000000000000000000000000000ce",
  "previousblockhash": "1664a604a6c8e603e311c6759f7693343ad77896c53de1bcef37de04dbcbbbc5",
  "nextblockhash": "1924c8db1b986f398a12d5481fcc293913a2dab2ee7fbcb0e131d7941e5201c2"
}

using different transaction hashes, I got the merkle root value, now I need to know which block has this merkle root as in getblock.

While I’m studying about the Bitcoin, I’ve wondered how many merkle paths should be contained in merkle block.

From the point of view of Merkle path, it is generated as the form of containing hashes of counterpart stepping up from bottom to top.

For example, if I have 4 transactions(let me use tx1, tx2, tx3, and tx4) in a block, then the hashes of them(h1, h2, h3, and h4) will be located at the bottom as the leaves. The parents of them will be (h12 and h34), after all, the merkle roots will be (h1234). So, if a SPV node want to verify that the tx2 is in the block, the full node can offer h1, h34 as a merkle path (as well as block header), so that they can prove as the step of hashing will lead them to the merkle roots which is contained in block header.

However, If I requests to verify multiple transactions in a block (e.g. using Bloom filter), then what would be contained into the merkle block (which contains merkle path and block header)?

Does they have multiple merkle paths for each transactions which are interested in, so that the transactions can be verified respectively? or adapted merkle path to be included which encompass all transactions related? (e.g. if we want to verify tx1, tx3 then generating h2, h3; though this exmample seems not to be guaranteed to show the order of transactions)

In the docs it says: The checksum for a descriptor without one can be computed using the getdescriptorinfo RPC.

I would like to build a recovery tool where many descriptors are imported, calling getdescriptorinfo for each is cumbersome. Manually calculating the checksum would save a lot of time and network calls.

Do I just double sha256 the entire descriptor and convert the first 4 bytes to hex?

Thanks 🙂

EDIT: I found this, seems quite a bit more complex than a double hash… https://github.com/bitcoin/bitcoin/blob/master/src/script/descriptor.cpp#L95

Background:

For a research project, I am trying to periodically clear the content of my Bitcoin node’s mempool. For this purpose I have created a new thread in init.cpp using the boost threadgroup (I have experimented with this successfully with a different project in the past and I am sure this is not the cause of the problem) in the Bitcoin Core software. In the new thread, I periodically call a function func (defined below), the sole purpose of which is to clear the content of the node’s mempool.

void func()
{
    LOCK2(cs_main, mempool.cs);
    AssertLockHeld(cs_main);
    AssertLockHeld(mempool.cs);
    mempool.clear();
}

I check for and lock critical sections that I think I should before I modify the mempool (the code above mimics what happens when a new transaction is added to the mempool; I basically looked at what happens in the NetMsgType::TX branch of ProcessMessage in net_processing.cpp and then AcceptToMemoryPoolWorker in validation.cpp and mimic that).

Issue:

When I run Bitcoin after successfully compiling my code, I sometimes immediately (i.e., as soon as the GUI window pops up) and sometimes much later in the execution face a segfault that causes the program to crash.

ubuntu@root:~$ bitcoin-qt -mpctimeout=1
Segmentation fault (core dumped)

In the code above, the function func is executed every 1 minute (represented by mpctimeout=1).

Question:

What could be causing the segfault? Could it possibly be race conditions when trying to get a lock on the mempool or execution of the main thread? It is my understanding that LOCK2 should keep trying to get a lock on the critical sections when they become available in case they are already locked by another thread/process. Since these locks only exist within the scope of the function func, they should ideally be released when returning from the function.

Edit # 1:

Some debugging shows that the segfault doesn’t occur while func is being executed but while the thread is put in sleep for mpctimeout amount of time. This leaves me wondering whether clearing the mempool has some long-term effects? I have made sure that the lock is released by verifying that transactions are indeed accepted to the mempool while the new thread is asleep. The segfault still occurs even if the mpctimeout value is large (e.g., 1 hour).

FWIW, I use boost::this_thread::sleep_for to put the thread to sleep. I assume this works without fault and that the segfault is instead caused in the long run due to clearing the mempool. Any tips?

Edit # 2:

Below is the output of debugging the binary through gdb:

(gdb) run
Starting program: /usr/local/bin/bitcoin-qt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffe9900700 (LWP 25643)]
[New Thread 0x7fffdf37f700 (LWP 25644)]
[New Thread 0x7fffdeb7e700 (LWP 25645)]
[New Thread 0x7fffdd0e7700 (LWP 25646)]
[New Thread 0x7fffc7cd2700 (LWP 25648)]
[New Thread 0x7fffc74d1700 (LWP 25649)]
[New Thread 0x7fffc6cd0700 (LWP 25650)]
[New Thread 0x7fffc64cf700 (LWP 25651)]
[New Thread 0x7fffc5cce700 (LWP 25652)]
[New Thread 0x7fffc54cd700 (LWP 25653)]
[New Thread 0x7fffc4ccc700 (LWP 25654)]
[New Thread 0x7fffa7fff700 (LWP 25655)]
[New Thread 0x7fffa77fe700 (LWP 25656)]
[New Thread 0x7fffa6ffd700 (LWP 25657)]
[New Thread 0x7fffa67fc700 (LWP 25658)]
[New Thread 0x7fffa5ffb700 (LWP 25659)]
[New Thread 0x7fffa57fa700 (LWP 25660)]
[New Thread 0x7fffa4f63700 (LWP 25661)]
[New Thread 0x7fff8dffd700 (LWP 25662)]
[New Thread 0x7fff8d7fc700 (LWP 25663)]
[New Thread 0x7fff8cffb700 (LWP 25664)]
[New Thread 0x7fff7bfff700 (LWP 25665)]
[New Thread 0x7fff7b7fe700 (LWP 25666)]
[New Thread 0x7fff7affd700 (LWP 25667)]
[New Thread 0x7fff7a7fc700 (LWP 25668)]
[New Thread 0x7fff79ffb700 (LWP 25669)]
[New Thread 0x7fff797fa700 (LWP 25670)]
[New Thread 0x7fff78ff9700 (LWP 25671)]
[New Thread 0x7fff53fff700 (LWP 25672)]
[New Thread 0x7fff537fe700 (LWP 25673)]
[New Thread 0x7fff4a174700 (LWP 25677)]
[New Thread 0x7ffed1a7b700 (LWP 25678)]
[Thread 0x7ffed1a7b700 (LWP 25678) exited]
[New Thread 0x7ffed127a700 (LWP 25679)]
[New Thread 0x7ffed0a79700 (LWP 25680)]
[New Thread 0x7ffecbfff700 (LWP 25681)]
[New Thread 0x7ffecb7fe700 (LWP 25682)]
[New Thread 0x7ffecaffd700 (LWP 25683)]
[New Thread 0x7ffeca7fc700 (LWP 25684)]
[New Thread 0x7ffec9ffb700 (LWP 25685)]
[New Thread 0x7ffec97fa700 (LWP 25686)]
[New Thread 0x7ffec8ff9700 (LWP 25687)]
[Thread 0x7ffed127a700 (LWP 25679) exited]
[Thread 0x7ffecb7fe700 (LWP 25682) exited]

Thread 40 "bitcoin-msghand" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffec9ffb700 (LWP 25685)]
__gnu_cxx::__atomic_add_dispatch (__val=1, __mem=0x1a45fe36190ee438)
    at /usr/include/c++/7/ext/atomicity.h:96
96        __atomic_add(__mem, __val);
(gdb)

I suppose that the segfault is caused when trying to update an atomic variable.

Edit # 3:

gdb backtrace

#0  0x0000555555854178 in __gnu_cxx::__atomic_add_dispatch (__val=1, __mem=0xf) at /usr/include/c++/7/ext/atomicity.h:96
#1  0x0000555555854178 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_add_ref_copy() (this=0x7) at /usr/include/c++/7/bits/shared_ptr_base.h:138
#2  0x0000555555854178 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count(std::__shared_count<(__gnu_cxx::_Lock_policy)2> const&) (__r=..., this=0x7ffef5ff71b8)
    at /usr/include/c++/7/bits/shared_ptr_base.h:691
#3  0x0000555555854178 in std::__shared_ptr<CTransaction const, (__gnu_cxx::_Lock_policy)2>::__shared_ptr(std::__shared_ptr<CTransaction const, (__gnu_cxx::_Lock_policy)2> const&)
    (this=0x7ffef5ff71b0) at /usr/include/c++/7/bits/shared_ptr_base.h:1121
#4  0x0000555555854178 in std::shared_ptr<CTransaction const>::shared_ptr(std::shared_ptr<CTransaction const> const&) (this=0x7ffef5ff71b0)
    at /usr/include/c++/7/bits/shared_ptr.h:119
#5  0x0000555555854178 in CTxMemPoolEntry::GetSharedTx() const (this=0x7ffee12106a0) at ./txmempool.h:101
#6  0x0000555555854178 in PartiallyDownloadedBlock::InitData(CBlockHeaderAndShortTxIDs const&, std::vector<std::pair<uint256, std::shared_ptr<CTransaction const> >, std::allocator<std::pair<uint256, std::shared_ptr<CTransaction const> > > > const&) (this=this@entry=0x7ffef5ff9f30, cmpctblock=..., extra_txn=std::vector of length 100, capacity 100 = {...})
    at blockencodings.cpp:114
#7  0x00005555556a2d4a in ProcessMessage(CNode*, std::__cxx11::string const&, CDataStream&, int64_t, CChainParams const&, CConnman*, std::atomic<bool> const&, bool)
    (pfrom=pfrom@entry=0x7ffedc0010d0, strCommand="cmpctblock", vRecv=..., nTimeReceived=1593549860206352, chainparams=..., connman=0x7fffbc04ba20, interruptMsgProc=..., enable_bip61=false) at net_processing.cpp:3051
#8  0x00005555556a6f01 in PeerLogicValidation::ProcessMessages(CNode*, std::atomic<bool>&) (this=<optimized out>, pfrom=0x7ffedc0010d0, interruptMsgProc=...) at net_processing.cpp:3629
#9  0x000055555566b19e in CConnman::ThreadMessageHandler() (this=0x7fffbc04ba20) at net.cpp:1966

I am investigating on privacy violations of the current communication protocol between lightweight clients and full nodes in the Bitcoin network, which utilizes Bloom filters. I run a full node on my Machine.

First, I want to make sure that a sufficient number of lightweight wallets are connected to the node. Second I want to save the communication history of each session between the full node and an SPV node. Specifically, filterload, filteradd, transactions, merkleblock, etc. (According to bip-0037)

Many Thanks!

Because on windows it is so difficult to load the blockchain into the BitcoinCore wallet. Arrives in 2016 the loading rate increases days. In ubuntu18.04 it is so fast and does not slow down after 2016. When it slows down and checks on windows with bitcoind.exe after 2016 and it finds four errors in the blockchain; how should I proceed?

When I was reading bitcoin-paper-errata-and-details.md written by David A. Harding, I realized that there’s probably a common misunderstanding or over-simplification about Merkle tree pruning. What Nick ODell had said might be a live example:

• A leaf (transaction) can be pruned when all of its outputs have been spent.

This once seemed to be true for me, until I read what David had written:

there is currently no way in Bitcoin to prove that a transaction has not been spent

I’m not sure whether I have grasped it, so firstly I made a diagram to illustrate (part of) my understanding to this problem:

Still, I don’t think merely this problem can kill the whole idea of Merkle tree pruning yet, I think it just means that "the reclaimable disk capacity is much lower than expectation". In other words, if I’m not mistaken, Nick ODell’s claim could be "corrected" like:

• A leaf (transaction) can be pruned when all of its outputs have been spent, and all of its previous transactions have been pruned.

However, I then think that, even if the "corrected" claim is taken into consideration, the idea of Merkle tree pruning still doesn’t seem to be feasible/useful:

1. Even if the problem mentioned above is avoided, a malicious node can still deceive the new full node by hiding/picking some merkle branches. A malicious node can lie about the actual ownership of coins (spent/unspent state) without breaking the Merkle tree structure at all. In other words, a new full node joining the network still needs to download & verify everything, otherwise, it could be deceived by a malicious node.

2. If a full node needs to enable pruning to reduce disk space requirement for itself, directly reading/modifying the blockchain files seems to be much less efficient than the current implementation that the UTXO set is completely separated from the blockchain storage, so that a full node (no matter it’s pruning or not) only needs to query and update the UTXO set database during the downloading & validation process. The blockchain itself doesn’t need to be touched once again for validation purposes at all, which is the reason why the old blocks can be simply deleted when "pruning" (not Merkle tree pruning) is enabled.

However, I’m still not sure about this conclusion. Is this related to the idea of fraud proofs, in the sense that as long as there’s still at least one honest full node, the new node would be able to spot which piece of data is the correct one? What if the UTXO set is also committed to the blockchain? What if some more commitments like the block height of previous transaction are also added to the blockchain?

Furthermore, I’ve heard that the Mimblewimble protocol enables secure blockchain pruning. I’m also curious how Mimblewimble could achieve this, and whether similar goal could be eventually achieved in Bitcoin?

So I want to write a bitcoin script so that after a certain time period (say for example after 30 mins), the verification fails and transaction is marked invalid even if it gets broadcasted. Time locks are used so that a transaction is considered valid after a certain time period. I want the opposite to happen. So can I just enter some bogus lines of code after OP_CHECKLOCKTIMEVERIFY so that script check intentionally fails? Or is there any opcode by which I can achieve the same functionality?

I need single integer converted from public BTC address. Is it enough to Base58Check decode the address and I will get hex/binary format of that integer or do I need to do something more?

What will be the format of Base58Check decoded data? Will there be any checksums, additional bytes?

I can write a script in Bash/Perl/Python if I would have instructions.