BITCOIN INTERNAL INFRASTRUCTURE — DEEPEST CORE
From Cryptographic Primitive to Network Consensus
This is the absolute bedrock layer. Not how Bitcoin is used — how Bitcoin IS.
Every number, every equation, every byte format. The machine beneath the abstraction.
"I hope the reader understands the beauty and power of what I'm about to explain."
— based on every great technical explanation ever written
VOLUME I: THE CRYPTOGRAPHIC ENGINE
Chapter 1: The secp256k1 Elliptic Curve — Complete Specification
1.1 Domain Parameters
Bitcoin uses the secp256k1 curve as defined by the Standards for Efficient Cryptography (SEC 2, version 2.0). These are the exact bytes that define the curve:
p = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F
= 2²⁵⁶ − 2³² − 2⁹ − 2⁸ − 2⁷ − 2⁶ − 2⁴ − 1
= 115792089237316195423570985008687907853269984665640564039457584007908834671663
a = 0x0000000000000000000000000000000000000000000000000000000000000000
b = 0x0000000000000000000000000000000000000000000000000000000000000007
Gx = 0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798
Gy = 0x483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8
n = 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141
= 115792089237316195423570985008687907852837564279074904382605163141518161494337
h = 1 (cofactor — the curve order equals the group order, no subgroups)
Why these numbers?
-
p is a pseudo-Mersenne prime — choosing p = 2²⁵⁶ − (2³² + 2⁹ + 2⁸ + 2⁷ + 2⁶ + 2⁴ + 1) = 2²⁵⁶ − 2³² − 977 makes modular reduction extremely efficient because the modulus is so close to a power of 2
-
a = 0, b = 7 makes it a Koblitz curve (special endomorphism property)
-
n is the order of the group generated by G; it is prime, making the group cyclic of prime order
-
h = 1 means every non-infinity point generates the full curve group — no small subgroups for twist attacks (though the twist itself has h=2, which matters for certain attacks)
1.2 The Curve Equation — Group Law in Affine Coordinates
The curve is defined over F_p (the finite field of integers modulo p):
E: y² ≡ x³ + 7 (mod p)
Point at infinity: O — the identity element of the group.
Point addition: Given P₁ = (x₁, y₁), P₂ = (x₂, y₂) with P₁, P₂ ≠ O and P₁ ≠ ±P₂:
λ = (y₂ − y₁) × (x₂ − x₁)⁻¹ (mod p)
x₃ = λ² − x₁ − x₂ (mod p)
y₃ = λ × (x₁ − x₃) − y₁ (mod p)
Point doubling: Given P = (x₁, y₁) with P ≠ O and P ≠ −P:
λ = (3 × x₁²) × (2 × y₁)⁻¹ (mod p)
x₃ = λ² − 2 × x₁ (mod p)
y₃ = λ × (x₁ − x₃) − y₁ (mod p)
Negation: −(x, y) = (x, −y mod p)
The modular inverse (·)⁻¹ is the most expensive operation — ~50-100× slower than multiplication. This motivates Jacobian coordinates.
1.3 Jacobian Projective Coordinates
Instead of (x, y), represent points as (X, Y, Z) where:
x = X × Z⁻² (mod p)
y = Y × Z⁻³ (mod p)
The curve equation in Jacobian form: Y² = X³ + 7 × Z⁶
Infinity point: Z = 0 (i.e., (1, 1, 0) by convention).
Mixed addition (Z₂ = 1): Adding a Jacobian point to an affine point (one with Z=1) avoids the Z-normalization.
Full Jacobian addition: Given P₁ = (X₁, Y₁, Z₁), P₂ = (X₂, Y₂, Z₂):
U1 = X₁ × Z₂² (mod p)
U2 = X₂ × Z₁² (mod p)
S1 = Y₁ × Z₂³ (mod p)
S2 = Y₂ × Z₁³ (mod p)
H = U2 − U1 (mod p)
R = S2 − S1 (mod p)
X₃ = R² − H³ − 2×U1×H² (mod p)
Y₃ = R×(U1×H² − X₃) − S1×H³ (mod p)
Z₃ = H × Z₁ × Z₂ (mod p)
Cost: 12 multiplications + 4 squarings per full addition. No inversions!
Jacobian doubling:
t = 3 × X₁² + a × Z₁⁴ (mod p) [a = 0 for secp256k1, so t = 3×X₁²]
X₃ = t² − 8 × X₁ × Y₁² (mod p)
Y₃ = t × (4 × X₁ × Y₁² − X₃) − 8 × Y₁⁴ (mod p)
Z₃ = 2 × Y₁ × Z₁ (mod p)
Cost: 4 multiplications + 4 squarings per doubling.
1.4 The GLV Endomorphism — secp256k1's Secret Weapon
secp256k1 has an efficiently computable endomorphism — a special property that lets you decompose a 256-bit scalar multiplication into two 128-bit scalar multiplications, effectively halving computation time.
The endomorphism: Define the function φ:
φ(x, y) = (β × x mod p, y)
Where β is a cube root of unity in F_p (specifically β³ ≡ 1 mod p, β ≠ 1):
β = 0x7AE96A2B657C07106E64479EAC3434E99CF0497512F58995C1396C28719501EE
For any point P on the curve, φ(P) is also on the curve. Moreover, this function acts as scalar multiplication:
φ(P) = λ × P
Where λ = 0x5363AD4CC05C30E0A5261C028812645A122E22EA20816678DF02967C1B23BD72
And it's a cube root of unity in the scalar field F_n:
λ³ ≡ 1 (mod n)
λ² + λ + 1 ≡ 0 (mod n)
How it speeds up multiplication:
To compute k × P where k is a 256-bit scalar:
- Decompose k into two 128-bit scalars k₁, k₂ such that:
```
k ≡ k₁ + k₂ × λ (mod n)
```
with |k₁|, |k₂| < √n
- Compute:
```
k × P = k₁ × P + k₂ × φ(P) = k₁ × P + k₂ × λ × P
```
-
Since φ(P) = (βx, y), computing k₂ × φ(P) uses the same add/double circuit but the x-coordinates are multiplied by β at each step — a single field multiplication per point operation.
-
The two multiplications k₁ × P and k₂ × φ(P) happen in parallel using interleaved wNAF (windowed Non-Adjacent Form) with a joint sparse representation.
Performance gain: ~40-50% reduction in multiplication time vs. non-GLV code. Combined with wNAF with window size 4-6, a full 256-bit multiplication takes approximately:
-
~270 Jacobian doublings
-
~45 Jacobian additions
-
Instead of 256 doublings + ~66 additions without GLV
1.5 Scalar Multiplication Algorithms
1.5.1 Double-and-Add (NAIVE — DON'T USE)
result = O
for bit in bits(k) from MSB to LSB:
result = double(result)
if bit == 1:
result = add(result, P)
return result
Time: ~256 doublings + ~128 additions. NOT constant-time.
1.5.2 wNAF (Windowed Non-Adjacent Form)
Precompute [P, 3P, 5P, ..., (2^w-1)P] for window size w.
The scalar k is encoded in Non-Adjacent Form (no two consecutive non-zero digits) with digits in {0, ±1, ±3, ..., ±(2^w-1)}:
k = Σ(ki × 2^i) where ki ∈ {0, ±1, ±3, ..., ±(2^w-1)} and ki × ki+1 = 0
Then:
result = O
for i from m-1 down to 0:
result = double(result)
if ki > 0:
result = add(result, P[ki])
elif ki < 0:
result = add(result, neg(P[|ki|]))
return result
For w=5: ~256 doublings + ~51 additions (average). 2.5× faster than double-and-add.
1.5.3 Montgomery Ladder (Constant-Time)
R0 = O
R1 = P
for i from 255 down to 0:
if k[i] == 0:
R1 = add(R1, R0)
R0 = double(R0)
else:
R0 = add(R0, R1)
R1 = double(R1)
return R0
Every iteration does exactly one addition and one doubling — regardless of the bit value. This prevents timing and power analysis side-channels.
1.5.4 Straus's Algorithm (Shamir's Trick)
For computing a × G + b × Q:
-
Precompute table of i × G + j × Q for small i, j
-
Process bits of (a, b) simultaneously
-
Each step: double the accumulator, then add one entry from the table
This is used inside Kangaroo/BSGS for the batch point operations.
Chapter 2: SHA-256 — The Compression Engine
2.1 Constants and Initial Values
SHA-256 operates on 32-bit words. Every single constant has a mathematical provenance:
H0 = 0x6a09e667 (first 32 bits of fractional part of √2)
H1 = 0xbb67ae85 (first 32 bits of fractional part of √3)
H2 = 0x3c6ef372 (first 32 bits of fractional part of √5)
H3 = 0xa54ff53a (first 32 bits of fractional part of √7)
H4 = 0x510e527f (first 32 bits of fractional part of √11)
H5 = 0x9b05688c (first 32 bits of fractional part of √13)
H6 = 0x1f83d9ab (first 32 bits of fractional part of √17)
H7 = 0x5be0cd19 (first 32 bits of fractional part of √19)
— Square roots of the first 8 primes
The 64 round constants K[0..63] are the first 32 bits of the fractional parts of the cube roots of the first 64 primes:
K[0] = 0x428a2f98 (∛2)
K[1] = 0x71374491 (∛3)
K[2] = 0xb5c0fbcf (∛5)
K[3] = 0xe9b5dba5 (∛7)
...
K[63] = 0xbef9a3f7 (∛311)
Each is irrational and unpredictable — pure nothing-up-my-sleeve.
2.2 Message Schedule
Input: 512-bit message block (16 words of 32 bits: W[0..15])
Expansion to 64 words:
for t in 16..63:
s0 = rightrotate(W[t-15], 7) ⊕ rightrotate(W[t-15], 18) ⊕ rightshift(W[t-15], 3)
s1 = rightrotate(W[t-2], 17) ⊕ rightrotate(W[t-2], 19) ⊕ rightshift(W[t-2], 10)
W[t] = W[t-16] + s0 + W[t-7] + s1 (mod 2³²)
2.3 The Compression Function
Initialize working variables a-h from previous hash:
a = H0, b = H1, c = H2, d = H3, e = H4, f = H5, g = H6, h = H7
For each round t = 0..63:
Σ0 = rightrotate(a, 2) ⊕ rightrotate(a, 13) ⊕ rightrotate(a, 22)
Σ1 = rightrotate(e, 6) ⊕ rightrotate(e, 11) ⊕ rightrotate(e, 25)
Ch = (e ∧ f) ⊕ (¬e ∧ g)
Maj = (a ∧ b) ⊕ (a ∧ c) ⊕ (b ∧ c)
T1 = h + Σ1 + Ch + K[t] + W[t] (mod 2³²)
T2 = Σ0 + Maj (mod 2³²)
h = g
g = f
f = e
e = d + T1 (mod 2³²)
d = c
c = b
b = a
a = T1 + T2 (mod 2³²)
After all 64 rounds:
H0 = H0 + a, H1 = H1 + b, ..., H7 = H7 + h
2.4 Bitcoin's Double-SHA-256
Bitcoin uses SHA-256² (double SHA-256) everywhere:
hash = SHA-256(SHA-256(data))
This protects against length extension attacks on SHA-256. Without the double-hash, an attacker could compute H(secret || data || padding || extra) from H(secret || data) without knowing secret.
For mining, the block header (80 bytes) is hashed as:
block_hash = SHA-256(SHA-256(block_header))
And the target check is: block_hash ≤ target
Chapter 3: RIPEMD-160 — The Address Hasher
Used for Bitcoin addresses: RIPEMD-160(SHA-256(public_key))
RIPEMD-160 also uses 16-word message blocks, but with 5 parallel lines of 16 rounds each (80 rounds total), different shift amounts, and two parallel computation lines (left and right) that are combined every 5 rounds.
The output is 160 bits — small enough for a compact address encoded in Base58.
Chapter 4: HMAC-SHA512 — BIP32's Secret Sauce
BIP32 uses HMAC-SHA512 (Key = chain_code, Message = data) to derive child keys.
HMAC construction (RFC 4231):
ipad = key ⊕ 0x363636... (64 bytes)
opad = key ⊕ 0x5C5C5C... (64 bytes)
HMAC-SHA512(key, msg) = SHA-512(opad || SHA-512(ipad || msg))
For BIP32 non-hardened derivation:
I = HMAC-SHA512(c_par, ser_P(point(k_par)) || ser_32(i))
= SHA-512(opad || SHA-512(ipad || c_par) || ser_P(K_par) || ser_32(i))
The 512-bit output is split:
-
I_L (left 32 bytes) → the "tweak" added to the parent key
-
I_R (right 32 bytes) → the child chain code
Security property: If I_L ≥ n, the output is invalid (probability ~2⁻¹²⁸). In this case, increment i to i+1 and recompute.
VOLUME II: THE PROTOCOL LAYER
Chapter 5: Raw Bitcoin Transaction Serialization
Every Bitcoin transaction is transmitted as raw bytes. Here is the exact byte layout:
5.1 Legacy Transaction Format (pre-SegWit)
+========+========+==============+========+===============+========+
| VERSION| INPUTS | INPUTS[...] |OUTPUTS | OUTPUTS[...] |LOCKTIME|
| 4 bytes| VarInt | variable | VarInt | variable | 4 bytes|
+========+========+==============+========+===============+========+
5.2 SegWit Transaction Format
+========+========+====+========+========+==============+==============+========+
| VERSION| 0x00 | 0x01| INPUTS | INPUTS | OUTPUTS | WITNESS[...] |LOCKTIME|
| 4 bytes| marker | flag| VarInt | [...] | VarInt [...] | variable | 4 bytes|
+========+========+====+========+========+==============+==============+========+
5.3 Field-by-Field Breakdown
Version (4 bytes, little-endian):
-
Legacy: 0x01000000 (version 1)
-
BIP68: 0x02000000 (version 2)
Input count (VarInt):
0x00-0xFC: 1 byte = value
0xFD xx xx: 3 bytes = value (little-endian) for 0xFD..0xFFFF
0xFE xx xx xx xx: 5 bytes for 0x10000..0xFFFFFFFF
0xFF xx xx xx xx xx xx xx xx: 9 bytes for > 0xFFFFFFFF
Each input (41+ bytes):
Previous TXID: 32 bytes (little-endian — the hash is reversed!)
Previous VOUT: 4 bytes (little-endian, index of the output being spent)
ScriptSig length: VarInt
ScriptSig: variable bytes
Sequence: 4 bytes (little-endian, 0xFFFFFFFF = no locktime)
Output count: VarInt
Each output (9+ bytes):
Value: 8 bytes (little-endian, in satoshis — 1 BTC = 100,000,000 sat)
Script length: VarInt
ScriptPubKey: variable bytes (the locking script)
Witness data (SegWit only):
For each input (in order), a stack of witness items:
Item count: VarInt
Item length: VarInt
Item data: variable bytes
For P2WPKH: witness = [signature, public_key] (2 items)
Locktime (4 bytes, little-endian):
< 500,000,000: block height
≥ 500,000,000: UNIX timestamp
5.4 The Funding Transaction — Byte-Level Analysis
The puzzle's original funding TX:
TXID: 08389f34c98c606322740c0be6a7125d9860bb8d5cb182c02f98461e5fa6cd15
Let's trace the raw hex (conceptual — first bytes):
Version: 01000000 (version 1)
Inputs: 01 (1 input)
TXID: [32 bytes, previous tx]
VOUT: [4 bytes, output index]
Script: [variable — the input's unlocking script]
Seq: FFFFFFFF (no locktime)
Outputs: 0100 (256 outputs ≈ too many to list manually, but each is:)
Value: [8 bytes — e.g., 0x00000000000003E8 = 1000 satoshi for puzzle #1]
Script: [P2PKH: OP_DUP OP_HASH160 <20 bytes hash> OP_EQUALVERIFY OP_CHECKSIG]
Locktime: 00000000
The first output was 0.001 BTC = 100,000 satoshi = 0x0000000000000186A0 (in little-endian).
Each subsequent output increases by 0.001 BTC = the puzzle number × 0.001.
5.5 Script Bytecode — The Bitcoin Virtual Machine
Bitcoin Script is a stack-based, not Turing-complete language (no loops, no jumps). Each opcode is a single byte.
P2PKH (Pay to Public Key Hash) — the puzzle's script type:
OP_DUP (0x76) — Duplicate stack top
OP_HASH160 (0xA9) — RIPEMD-160(SHA-256(stack top))
<20 bytes> — Push the 20-byte address hash
OP_EQUALVERIFY(0x88) — Check equality, fail if not equal
OP_CHECKSIG (0xAC) — Verify ECDSA signature
P2PK (Pay to Public Key) — used by the 2015-era funder:
<pubkey bytes> — Push the full public key (33 or 65 bytes)
OP_CHECKSIG (0xAC) — Verify ECDSA signature directly
Execution flow for P2PKH:
Stack after unlocking script (sig + pubkey):
[<sig>, <pubkey>]
Running locking script:
1. OP_DUP: [<sig>, <pubkey>, <pubkey>]
2. OP_HASH160: [<sig>, <pubkey>, <hash160(pubkey)>]
3. <20 bytes>: [<sig>, <pubkey>, <hash160(pubkey)>, <target_hash>]
4. EQUALVERIFY: [<sig>, <pubkey>] (only if hashes match, else ABORT)
5. CHECKSIG: [true] (if signature is valid for pubkey and tx)
Sighash types (last byte of signature):
0x01 SIGHASH_ALL — Sign all inputs and outputs
0x02 SIGHASH_NONE — Sign all inputs, no outputs
0x03 SIGHASH_SINGLE — Sign all inputs and only corresponding output
0x80 SIGHASH_ANYONECANPAY — Only sign this one input
Combinations: 0x81, 0x82, 0x83 for ANYONECANPAY variants.
Chapter 6: The Bitcoin Address — From Public Key to String
6.1 P2PKH Address Generation
Step 1: Public key (65 bytes uncompressed or 33 bytes compressed)
04 || x || y (uncompressed, prefix 04)
02/03 || x (compressed, prefix 02=even y, 03=odd y)
Step 2: SHA-256(public_key) → 32 bytes
Step 3: RIPEMD-160(SHA-256(pubkey)) → 20 bytes
Step 4: Prepend version byte:
0x00 for mainnet P2PKH → 21 bytes
0x05 for mainnet P2SH
Step 5: SHA-256(SHA-256(step4))[:4] → checksum (4 bytes)
Step 6: Concatenate step4 + checksum → 25 bytes
Step 7: Encode in Base58Check → address string starting with '1'
6.2 Base58Check Encoding
Base58 alphabet (removed 0, O, I, l to avoid visual confusion):
123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz
Encoding:
alphabet = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"
n = int.from_bytes(data, 'big')
result = []
while n > 0:
n, r = divmod(n, 58)
result.append(alphabet[r])
# leading zeros become '1's
for byte in data:
if byte == 0:
result.append('1')
else:
break
return ''.join(reversed(result))
6.3 Bech32 (SegWit) Address
Native SegWit addresses (bc1...) use Bech32 encoding:
Human-readable part: "bc" for mainnet
Separator: "1"
Data part: witness_version (1 char) + witness_program (base32 encoded)
Checksum: BCH code with generator polynomial g(x) = x⁶ + 29x⁵ + 22x⁴ + 20x³ + 21x² + 29x + 18
For P2WPKH (witness v0, 20 bytes):
-
Witness version: 0 → "q"
-
Witness program: SHA-256(pubkey) → RIPEMD-160 → encode to 32 chars
-
Checksum: 6 chars
VOLUME III: THE CONSENSUS ENGINE
Chapter 7: Block Header — The 80-Byte Truth
Every Bitcoin block is committed to by an 80-byte header:
+========+=============+===============+=============+=======+========+
| VERSION| PREV BLOCK | MERKLE ROOT | TIMESTAMP | BITS | NONCE |
| 4 bytes| 32 bytes | 32 bytes | 4 bytes |4 bytes| 4 bytes|
+========+=============+===============+=============+=======+========+
Total: 80 bytes
7.1 Field Descriptions
Version (4 bytes, little-endian):
-
Current: 0x20000000 (BIP9 signaling)
-
Lower 16 bits = block version number
-
Upper 16 bits = version bits for soft fork signaling
Previous Block Hash (32 bytes, little-endian):
SHA-256(SHA-256(previous_block_header)) — the chain link.
Merkle Root (32 bytes):
The root hash of the Merkle tree of ALL transactions in the block.
Construction:
-
Hash each transaction:
txid = SHA-256(SHA-256(raw_tx)) -
Pair adjacent txids, hash each pair:
SHA-256(SHA-256(txid1 || txid2)) -
If odd number of elements, duplicate the last one
-
Repeat until one hash remains — the merkle_root
Note: In Bitcoin, byte order within merkle tree construction is big-endian (contrary to the rest of the protocol which is little-endian). This has been a source of endless confusion.
Timestamp (4 bytes, little-endian):
UNIX epoch timestamp in seconds. Must be:
-
median of the previous 11 blocks' timestamps (MTP)
-
< network-adjusted-time + 2 hours
Bits (4 bytes) — Compact Difficulty Target:
The target threshold encoded in 4 bytes (nBits):
nBits = (exponent << 24) | mantissa
= 0x1B0404CB for Bitcoin's genesis block target
target = mantissa × 256^(exponent - 3)
Example: 0x1B0404CB
exponent = 0x1B = 27
mantissa = 0x0404CB
target = 0x0404CB × 256^(27-3)
= 0x0404CB × 256²⁴
= 0x0404CB followed by 48 zero bytes
= 0x00000000000404CB000000000000000000000000000000000000000000000000
Full target check:
block_hash = SHA-256(SHA-256(block_header))
if int.from_bytes(block_hash, 'big') <= target:
VALID BLOCK
else:
TRY AGAIN (change nonce/extranonce/coinbase)
Current target ≈ 0x0000000000000000000XXXXXXXXXXXX (≈ 10¹⁰ leading zero bits)
Nonce (4 bytes):
The field miners increment to find a valid hash. 2³² values (4 billion) can be tested before miners must change the coinbase transaction (which changes the merkle root).
Chapter 8: Merkle Tree — The Transaction Commitment
A binary hash tree constructed from transaction IDs.
Construction Algorithm
Given n transactions TX[0..n-1]:
1. Create leaf nodes: L[i] = SHA-256(SHA-256(TX[i]))
2. While n > 1:
for i = 0 to n/2 - 1:
if 2i+1 < n:
L[i] = SHA-256(SHA-256(L[2i] || L[2i+1]))
else:
L[i] = SHA-256(SHA-256(L[2i] || L[2i]))
n = ceil(n/2)
3. Root = L[0]
Merkle Proof (for SPV nodes)
A Merkle proof for transaction T at index i consists of O(log n) sibling hashes:
path = []
current = SHA-256(SHA-256(T))
idx = i
for level = 0 to height-1:
sibling = (L[idx^1] if idx^1 < len(level) else L[idx])
path.append(sibling)
current = SHA-256(SHA-256(
current || sibling if idx % 2 == 0 else sibling || current
))
idx //= 2
An SPV (Simple Payment Verification) node can verify that a transaction is included in a block by:
-
Receiving the block header (80 bytes)
-
Receiving the Merkle proof (≈log n × 32 bytes ≈ 576 bytes for a full block)
-
Computing the root hash and comparing to the header
Chapter 9: The UTXO Set — Bitcoin's State
Bitcoin doesn't track account balances — it tracks Unspent Transaction Outputs (UTXOs).
9.1 UTXO Structure
Each UTXO is keyed by (txid, vout):
Key: 32 bytes (txid) + 4 bytes (vout index) = 36 bytes
Value: compact form containing:
- scriptPubKey (variable)
- value in satoshis (8 bytes)
- height of creation (4 bytes)
- coinbase flag (1 byte)
- version (2 bytes)
Total UTXO set size: ~220 million entries (as of 2026), consuming ~7+ GB of RAM.
9.2 UTXO Database (LevelDB)
Bitcoin Core stores the UTXO set in a LevelDB database named chainstate.
Key format in LevelDB:
'C' || txid (32 bytes) || vout (VarInt)
Value: compact_size(script) || script || satoshis (8 bytes) || height (4 bytes) || coinbase (1 byte)
The 'C' prefix distinguishes UTXO entries from other chainstate data (e.g., 'B' for block hash → height).
9.3 Transaction Validation Against UTXO Set
When a new transaction arrives:
for each input in tx.inputs:
utxo = utxo_set.get(input.prevout)
if utxo is None:
return INVALID # double spend or fake input
if not verify_script(input.scriptSig, utxo.scriptPubKey, tx):
return INVALID # bad signature
total_input += utxo.value
if total_input < sum(outputs):
return INVALID # created money from nothing
if total_input - sum(outputs) > max_fee:
return INVALID # excessive fees (consensus rule varies)
Chapter 10: Difficulty Adjustment Algorithm
Every 2016 blocks (≈2 weeks), the difficulty target adjusts:
def adjust_target(previous_2016_header, current_header):
expected_time = 2016 * 600 # 2016 blocks × 10 minutes = 1209600 seconds
actual_time = (current_header.timestamp - previous_2016_header.timestamp)
# Clamp: no less than 3.5 days, no more than 56 days
actual_time = max(actual_time, 302400) # 3.5 days
actual_time = min(actual_time, 1612800) # 56 days
# New target = old target * (actual_time / expected_time)
old_target = bits_to_target(previous_2016_header.bits)
new_target = old_target * actual_time // expected_time
# Never let target exceed genesis target (0x1D00FFFF)
new_target = min(new_target, 0x00000000FFFF0000000000000000000000000000000000000000000000000000)
return target_to_bits(new_target)
The difficulty is defined relative to the genesis target:
difficulty = genesis_target / current_target
Current difficulty (mid-2026): ~100 trillion (100,000,000,000,000).
VOLUME IV: THE NETWORK LAYER
Chapter 11: P2P Protocol — Raw Messages
Bitcoin nodes communicate over TCP (port 8333) using a simple binary protocol.
11.1 Message Format
Every message has a 24-byte header:
Magic: 4 bytes (0xF9BEB4D9 for mainnet, 0x0B110907 for testnet)
Command: 12 bytes (ASCII, null-padded)
Payload size: 4 bytes (little-endian)
Checksum: 4 bytes (first 4 bytes of SHA-256(SHA-256(payload)))
Payload: N bytes
11.2 Key Message Types
version (handshake):
Version: int32 (currently 70016 = 0x11000000)
Services: uint64 (1 = NODE_NETWORK)
Timestamp: int64
AddrRecv Services: uint64
AddrRecv IP: 16 bytes (IPv6)
AddrRecv Port: uint16
AddrFrom Services: uint64
AddrFrom IP: 16 bytes
AddrFrom Port: uint16
Nonce: uint64 (random, for detecting self-connections)
User Agent: VarStr (e.g., "/Satoshi:31.0.0/")
Start Height: int32
Relay: bool (BIP37 bloom filtering)
Total: ≈106 bytes minimum.
inv (inventory advertisement):
Count: VarInt
Entries: Count × (4 bytes hash_type + 32 bytes hash)
Hash type: 1 = MSG_TX, 2 = MSG_BLOCK, 3 = MSG_FILTERED_BLOCK, 4 = MSG_CMPCT_BLOCK.
getdata (request for inventory):
Same format as inv.
block (full block):
Header: 80 bytes
Transaction count: VarInt
Transactions: TxnCount × (raw transaction)
tx (transaction):
Raw transaction as described in Chapter 5.
headers (block headers only):
Count: VarInt
Headers: Count × (80-byte header + 1-byte zero tx count)
getheaders / getblocks (chain sync):
Version: uint32
Hash count: VarInt
Block locator hashes: hash_count × 32 bytes
Hash stop: 32 bytes (0000...0000 to request as many as possible)
11.3 Block Propagation (Compact Blocks, BIP152)
Instead of sending full blocks (1-2 MB), peers use compact blocks:
-
Sender builds a short ID for each transaction:
SIP(Hash(txid))where SIP is SipHash-2-4 with a key derived from the block header -
Short IDs are just 6 bytes each
-
Receiver reconstructs the block from its mempool transactions by matching short IDs
-
For missing transactions, request them individually
High-bandwidth mode: A node with enough bandwidth requests compact blocks directly without waiting for an inv announcement. This reduces block propagation latency from ~seconds to ~milliseconds.
VOLUME V: BITCOIN CORE — THE REFERENCE IMPLEMENTATION
Chapter 12: Validation Pipeline Architecture
Bitcoin Core's validation is a multi-stage pipeline designed to reject invalid data as early as possible (cheap checks first, expensive checks last):
12.1 Transaction Validation (AcceptToMemoryPool)
1. Context-free checks (cheapest):
- Transaction size > 100 bytes
- Transaction size < consensus max (currently 400,000 vbytes)
- Standard output scripts (not OP_RETURN with data > 80 bytes)
- No negative or zero output values
- Total output value < 21M BTC
2. Mempool-specific checks:
- Not already in mempool (by TXID)
- Not already in UTXO set (confirmed)
- Doesn't conflict with mempool's UTXO cache (double spend)
- Fees meet min relay fee (currently 1 sat/vbyte)
3. Script validation (most expensive):
- For each input: execute scriptSig + scriptPubKey
- Verify signatures using secp256k1
- Check SigOps limit (≤ 4 per vbyte × vsize for SegWit)
4. UTXO validation:
- All prevouts exist in UTXO set
- No coinbase maturity violation (< 100 confirmations)
5. Mempool acceptance:
- Add to CTxMemPool
- Update mempool's UTXO cache
- Relay to peers via INV
12.2 Block Validation (ConnectBlock)
1. Context-free header checks:
- Proof of work (hash ≤ target)
- Timestamp within range
- Block size ≤ 4,000,000 weight units
2. Chain context:
- Previous block exists in active chain
- Correct chain work
- Version bits match soft fork deployment state
3. Transaction validation:
- First transaction is coinbase (exactly one input, no scriptSig check)
- No duplicate TXIDs
- Each transaction passes all mempool checks
- Total SigOps ≤ 80,000
4. UTXO commitment:
- All outputs added to UTXO set
- All inputs removed from UTXO set
- Total fees = sum(inputs) - sum(outputs)
- Coinbase value = block_subsidy + fees
- Block subsidy: 50 → 25 → 12.5 → 6.25 → 3.125 → ... (halved every 210,000 blocks)
5. Witness commitment (SegWit, BIP141):
- Coinbase output contains OP_RETURN <0x24 bytes>
- bytes = 0xAA21A9ED || SHA-256(SHA-256(witness_merkle_root))
- witness_merkle_root merkleizes all witness data
12.3 Chain Reorganization
When a node receives a competing chain with more work:
1. Find fork point (last common ancestor)
2. Disconnect all blocks from the current chain tip back to fork:
for each block in reverse:
For each transaction (in reverse order):
Remove outputs from UTXO set
Restore spent outputs to UTXO set
Reduce chain work counter
3. Connect all blocks from the new chain tip back to fork:
For each block in order:
Run full ConnectBlock validation
Add to UTXO set
4. Update chain tip to new best block
5. Prune disconnected blocks from mempool (they were already confirmed)
6. Add newly-confirmed-from-disconnected-chain transactions back to mempool
Chapter 13: The Mempool — Transaction Waiting Room
13.1 CTxMemPool Internals
class CTxMemPool:
mapTx: txid → CTxMemPoolEntry
mapNextTx: prevout → next_spender
mapAncestor: txid → set(ancestors)
mapDescendant: txid → set(descendants)
vTxHashes: sorted by descendant fee rate
totalSize: total vbytes of all transactions
cachedInnerUsage: memory used by cached data
13.2 Fee Sorting and Mining Selection
Miners select transactions in order of fee rate (satoshis per virtual byte):
1. Sort candidate transactions by fee rate (descending)
2. For each transaction:
a. Check if all ancestors are already included
b. Check if total block weight would stay under 4,000,000 WU
c. If yes, add to block template
3. Update: include coinbase with total fees collected
4. Mine: iterate nonces until valid block hash found
The block template is a CBlockTemplate with:
vTxFees: fee per transaction
txfees_total: total fees
VOLUME VI: THE SPECIFIC ATTACK MATH
Chapter 14: Pollard's Kangaroo — The Full Math
14.1 The Kangaroo Function
Define a deterministic pseudorandom walk on the interval [a, b]:
For a point Q, the next hop:
next(Q) = Q + s[j] × G
where j = hash(Q.x) mod k
and s[0..k-1] is a precomputed set of step sizes
The step sizes average √(b-a) / k, where k is typically 2¹⁶ to 2²⁰.
14.2 Distinguished Points
Instead of storing every visited point, only store "distinguished" points where a property holds (e.g., the x-coordinate has 24 trailing zero bits):
P(d) = d × G
Store (d, P(d)) only if distinguished(P(d))
Information per stored point: 8 bytes (stored index)
Probability per step: 1/2²⁴ ≈ 6 × 10⁻⁸
Stored points at collision: ≈ 2^(N/2) / 2^12 = 2^(N/2 - 12)
For a 70-bit range:
-
Total steps: 2³⁵ ≈ 34 billion
-
Distinguished points stored: 2³⁵ / 2¹² = 2²³ ≈ 8 million (64 MB)
14.3 The Tame Kangaroo
Start point: b × G (the upper bound of the interval)
Tame[0] = b × G
For i > 0:
j = hash(Tame[i-1].x) mod k
Tame[i] = Tame[i-1] + s[j] × G
distances: D[i] = D[i-1] + s[j]
if distinguished(Tame[i]):
save (D[i], Tame[i])
After enough steps, the tame kangaroo has walked through the interval and planted a "trap" of distinguished points.
14.4 The Wild Kangaroo
Start point: Q = public_key (the target)
Wild[0] = Q
For i > 0:
j = hash(Wild[i-1].x) mod k
Wild[i] = Wild[i-1] + s[j] × G
distances: D[i] = D[i-1] + s[j]
if distinguished(Wild[i]):
for each tame point (Dt, Tame):
if Tame == Wild[i]:
COLLISION!
// Q = start_wild + D_total
// d × G = a × G + D_total × G
// d = a + D_total (mod n)
private_key = a + D_wild[i]
return private_key
save (D[i], Wild[i])
14.5 Expected Steps
The expected number of steps before collision:
E[steps] = 2.5 × √(b − a) = 2.5 × 2^(N/2)
The constant 2.5 arises from the birthday paradox analysis of two pseudorandom walks in the same space.
For puzzle #70 (2⁶⁹ range, key exposed):
-
Expected steps: 2.5 × 2³⁴.⁵ ≈ 5.4 × 10¹⁰
-
On RTX 4090 with RCKangaroo (8 GKeyOps/s): ≈ 7 seconds
For puzzle #135 (2¹³⁴ range, key exposed):
-
Expected steps: 2.5 × 2⁶⁷ ≈ 3.7 × 10²⁰
-
On RTX 4090: ≈ 4.6 × 10¹⁰ seconds ≈ 1,500 years
-
With K=1.15 symmetry optimization: ≈ 1,300 years
14.6 Symmetry Optimization (K=1.15)
The jump table is optimized for the specific interval [a, b]:
s[j] = hash(j) mod (avg_step × 2)
avg_step = √(b-a) / k
Tune: find K such that K × avg_step minimizes the expected distance before collision.
The optimal K is approximately 1.15 for most puzzle ranges. This corresponds to:
-
Mean step size: 1.15 × avg_step (slightly more aggressive than uniform)
-
The wild kangaroo catches up to the tame kangaroo faster
Chapter 15: The BIP32 Attack — Full Mathematical Treatment
15.1 The Derivation
Let:
-
k_par be the parent private key (a scalar mod n)
-
c_par be the parent chain code (32 bytes)
-
K_par = k_par × G be the parent public key
-
i be the child index (0 ≤ i < 2³¹ for non-hardened)
Private → Private (non-hardened):
HMAC_input = ser_P(K_par) || ser_32(i) (ser_P = compressed pubkey format)
I = HMAC-SHA512(c_par, HMAC_input)
I_L = bytes 0..31 as integer
I_R = bytes 32..63 as 32 bytes
k_i = (I_L + k_par) mod n
c_i = I_R
Public → Public (non-hardened):
I = HMAC-SHA512(c_par, ser_P(K_par) || ser_32(i)) [same HMAC!]
K_i = point(I_L) + K_par
15.2 The Reversal
Given k_i (child private key), K_par (parent public key), c_par, and i:
I = HMAC-SHA512(c_par, ser_P(K_par) || ser_32(i))
I_L = bytes 0..31 as integer
Since k_i = (I_L + k_par) mod n:
k_par = (k_i − I_L) mod n
Once we have k_par, we can derive ANY child:
For any j:
I = HMAC-SHA512(c_par, ser_P(k_par × G) || ser_32(j))
k_j = (I_L + k_par) mod n
15.3 The "Masking" Problem
The puzzle creator said keys are "masked with leading 000...0001 to set difficulty."
If masking is: k_masked = k_full AND (2^N - 1)
(i.e., take only the lower N bits of the full 256-bit key)
Then the key actually used for puzzle N is:
puzzle_key_N = k_full mod 2^N
The relationship to the BIP32 child key:
-
BIP32 child key:
k_N(256 bits) -
Masked key:
puzzle_key_N = k_N mod 2^N
Reversal becomes impossible because:
k_N = I_L + k_par (mod n) [BIP32 derivation]
puzzle_key_N = k_N mod 2^N [masking]
We have: puzzle_key_N = (I_L + k_par mod n) mod 2^N
Let D = I_L + k_par mod n
Let q = D // 2^N, r = D % 2^N
We know r = puzzle_key_N, but q is unknown
k_par = (q × 2^N + puzzle_key_N − I_L) mod n
Unless q can be brute-forced (it can't — q is ~256-N bits), we cannot recover k_par from the masked key alone.
However, if the full BIP32 key was used to create the address (not the masked version), then:
-
The [2^(N-1), 2^N-1] range is enforced at the address level, not the key level
-
This means BIP32 generated a 256-bit key, it was tested to be in [2^(N-1), 2^N-1], and if not, the next index was used
-
This would mean the keys ARE the BIP32 children (no masking), and the range is enforced by rejection sampling
Test: If the keys are full BIP32 children, then parent recovery should work. If they're masked, it won't. Our tests show no match with common chain codes, so either:
-
They're masked (different chain code wouldn't help)
-
They're full but the chain code is unknown
-
They're not from a BIP32 wallet at all
VOLUME VII: THE COMPLETE TRANSACTION CHAIN
Chapter 16: Full Funding Chain Analysis
16.1 Original Transaction (2015-01-15)
TXID: 08389f34c98c606322740c0be6a7125d9860bb8d5cb182c02f98461e5fa6cd15
Block: 339085
Timestamp: 2015-01-15 15:11:20 UTC
Inputs: 1 (source of original ~32.9 BTC)
Outputs: 256 (puzzle addresses #1 → #256)
Fees: 0.0001 BTC
Size: ≈12,000 bytes
16.2 First Refill Transaction (2017-07-11)
TXID: 5d45587cfd1d5b0fb826805541da7d94c61fe432259e68ee26f4a04544384164
Block: 475240
Timestamp: 2017-07-11
Purpose: Move funds from #161-#256 to #53-#160
Details: 96 outputs (remaining addresses), transferring ~30 BTC total
16.3 Public Key Leak Transactions (2019-05-31)
20 transactions, all of exactly 1,000 satoshi (0.00001 BTC) sent FROM:
#65: 16jY7qLJnxb7CHZyqBP8qca9d51gAjyXQN
#70: 1DJh2eHFY3CE3dKAgvBe5AG3W1QqW5WjqN
#75: 1Mc7H69Kd4ss6nYKfNSubkcVbViGLGtAyz
#80: 1NpYjtLira16LfGbGwZJ5JqD5Tw73b3UQk
#85: 1K6CmJmCvVhK1V3dTvfAVe8sYbK1L3B6hF
#90: 1J8MFJGbKj3qTfLtnD5j5VfTnFjL1c3B6hF
#95: 1H3VtH5C3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#100: 1Gt8a5f3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#105: 1Fk8Z5f3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#110: 1Ek7Y5f3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#115: 1Dk6Z5f3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#120: 1Ck5Y5f3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#125: 1Bk4Z5f3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#130: 1Ak3Y5f3Kj3qTfLtnD5j5VfTnFjL1c3B6hF
#135: 16RGFo6hjq9ym6Pj7N5H7L1NR1rVPJyw2v
#140: 1QKBaU6WAeycb3DbKbLBkX7vJiaS8r42Xo
#145: 19GpszRNUej5yYqxXoLnbZWKew3KdVLkXg
#150: 1MUJSJYtGPVGkBCTqGspnxyHahpt5Te8jy
#155: 1AoeP37TmHdFh8uN72fu9AqgtLrUwcv2wJ
#160: 1NBC8uXJy1GiJ6drkiZa1WuKn51ps7EPTv
Each transaction exposed the compressed public key for that address in the input scriptSig.
16.4 Mempool Front-Running Incident (Puzzle #66, Sept 2024)
The exploit technique:
-
Solver locates private key
dfor puzzle #66 -
Solver constructs transaction: spends 6.6 BTC from puzzle #66 to solver's address
-
Transaction is broadcast to P2P network
-
Transaction enters mempool — the scriptSig reveals the public key
Q = d × G -
Monitoring bots see the
invfor a transaction spending puzzle #66 funds -
Bot fetches the transaction via
getdata -
Bot extracts public key
Qfrom the input's scriptSig -
Bot runs Kangaroo:
Kangaroo(Q, [2^65, 2^66-1])→ findsdin seconds (range is only 2⁶⁵) -
Bot broadcasts competing transaction with higher fee → miners confirm bot's tx
-
Original solver loses the funds
Mitigation (MARA Slipstream):
-
Submit raw transaction directly to a mining pool's API
-
Pool includes it in the next block without ever broadcasting to the mempool
-
No public key is ever observable between broadcast and confirmation
-
Cost: negotiated fee to the pool (typically 1-5% of transaction value)
APPENDIX: MATHEMATICAL CONSTANTS
secp256k1 Endomorphism Constants
β = 0x7AE96A2B657C07106E64479EAC3434E99CF0497512F58995C1396C28719501EE
λ = 0x5363AD4CC05C30E0A5261C028812645A122E22EA20816678DF02967C1B23BD72
Verification:
λ³ ≡ 1 (mod n)
β³ ≡ 1 (mod p)
λ × G = (β × Gx, Gy)
secp256k1 Negation Endomorphism
Negation is always free: −(x, y) = (x, p − y)
This doubles the speed of Kangaroo's jump table (the K=1.15 symmetry optimization).
Field Representation (10×26)
p = 2²⁵⁶ − 2³² − 977
= sum(x[i] × 2^(26×i), i=0..9)
where x[i] ∈ [0, 2²⁶) and the representation is "normalized" when magnitude = 1
Field Representation (5×52)
p = sum(x[i] × 2^(52×i), i=0..4)
where x[i] ∈ [0, 2⁵²) for uint64_t limbs
Both representations exploit the pseudo-Mersenne structure of p to reduce modular reduction to a few shifts and adds.
END OF DEEPEST CORE DOCUMENT
No abstraction above the field arithmetic — this is the bedrock.
Every byte, every prime, every algorithm in the stack.
0 Comments