This is a twitter thread I made, but decided to publish it here, so hence why the sentences are kept short and the word format.
The Difficulty Adjustment is the great missing innovation that made Bitcoin work.
While it ensures that Bitcoin’s supply is predictable it also is a:
Since its inception Bitcoin adjusted its difficulty 373 times.
For the 1st year of its existence, the difficulty did not increase, the 1st “big” change taking place on 2010-01-01 ~ block 32,256.
The biggest changes are: – upwards being 300% – downwards 25%
The Bitcoin Difficulty tells us how hard it is to find a hash that will make for a valid block.
At the current difficulty of 30,977,051,760,460 1 in every
13,315,579,227,696,403,823,071,581 SHA256 hash will be a valid block.
Mining is a random and unpredictable process.
Even more so miners can leave and join the network at any time.
The difficulty regulates the speed at which new blocks are added on to the blockchain.
Point #2 will be explained further.
Miners hash all the block data with a random nonce until they find a block hash that is smaller than a certain Target.
There is a direct relation between Target and Difficulty.
The smaller the target, the harder it is to mine.
The Target number is stored in a block in the field Bits in a compressed format.
The fact that the Target itself is hashed with the rest of the block data makes the difficulty forgery-proof.*
The Difficulty is usually presented in decimal format.
Target = Max Target / Difficulty
Max Target: the target, if difficulty is 1 (lowest possible difficulty)
This value was picked by Satoshi.
A difficulty of 420, means that it will take 420 more tries(on average) to find a valid hash than with the easiest difficulty of 1.
As mentioned The Genesis Block started with difficulty 1.
Ever since every 2016 blocks, the Difficulty is recalculated.
When a block is mined, the miner also needs to add a timestamp to the block.
This Timestamp is used to recalculate the difficulty each time.
We want a new block to be found ~10 minutes, so after 2016 blocks we check what was the time between blocks.
Expected time: 2016 blocks x 10 minutes = 20,160 minutes.
If the blocks were generated faster, let’s say 8 minutes, then we get 2016 x 9 = 18,144 20160 / 16128 = 1.25
New Difficulty = 1.25 * Old Difficulty
The hashrate is NOT known!
We use the timestamps in blocks to estimate how much hash rate is working on the network.
This is not really an exact science.
But the timestamps themselves are not an exact science.
As mentioned we need some reference of time in order to make sure we adjust for a block to come every 10 minutes.
When a block is mined the miner adds the time inside.
But what if he has the wrong time or he is lying?
Why would miners lie?
It is in the miner’s interest to say that blocks took longer to produce.
This will have the effect of lowering the next difficulty.
To mitigate this attack Bitcoin has 3 rules:
This is a consensus rule.
MAX_FUTURE_BLOCK_TIME – is used.
This is a policy rule.
This is a local client rule.
Rule #1 – ensures that the blockchain advances forward.
Rule #2 – ensures that the chain does not move too forward.
Rule #3 is an extra safeguard.
Most importantly these rules tell us that managing time in Bitcoin is very hard and imprecise.
Even though the difficulty adjusts every 2016 blocks, it only takes into account 2015 blocks.
This is because Satoshi introduced a off-by-1 bug.
This is still present in the code today, as fixing it could result in a hard-fork.
Instead of checking how much time has passed between the last blocks of each 2016 period, the code compares the time between the 1st and the 2016th block of each period.
Between 2016 things there are 2015 gaps.
As a consequence Blocks are NOT actually 10 minutes, but 10.005 minutes.
The 0.3 seconds are not a big deal but bug allows for 2 very weird things to occur.📷
You may expect retargeting interval to be (2 weeks)/2015, but it is even weirder.
If the Bitcoin hash rate is perfectly constant, and blocks have exact timestamps the difficulty would adjust once every 2 weeks, 20 minutes, 1.191658seconds.
As Pieter Wuille points out https://twitter.com/pwuille/status/1098651788343955456
The 2nd effect is the Time Warp Attack.
Given the rules mentioned above:
Within these rules, a miner could lie that it took longer to generate blocks.
The off-by-1 one makes the last block of the current period to not overlap with the 1st block of new period.
An attacker could do this indefinitely and bring the difficulty to 1.
However, this attack would be very visible and would require miner coordination from miners.
Besides the temporal restrictions, the difficulty can not change by more than a factor of 4.
This seems quite a peculiar restriction but it serves a good purpose.
This rule further protects against the attack mentioned.
Also, it is possible to isolate a node from the network and send it blocks with false time stamps.
The node would be tricked into lowering its difficulty and the attacker will be able to create false blocks easier.
Why 2016 blocks? 2 weeks / 10 minutes = 2016
Why not less than 2016?
It would allow for similar isolating attacks as explained above.
An attacker can isolate a node, and send it blocks with lower difficulty.
To give you an idea of how difficult current times are.
My Apple M1 Max computer does 5.8Mhashes/s.
Thymos in 2010 “generated 5 blocks” on a Pentium processor. (1000 times slower than mine)
Having a Difficulty Adjustment and even more adding these restrictions in order to eliminate these attacks was visionary.
Never mind the fact we never had something like this before.
IF anything this shows us how untrustworthy/ineffective Bitcoin at time management.
Maybe why timechain was replaced with blockchain
As we can see from this comment in the Bitcoin Client released by Satoshi, was hoping add NTP.
NTP – is a networking protocol for clock synchronization between computer system.
Finally, I want to leave you with this Bitcointalk post from 2010 made by satoshi, when a 1.34x increase in difficulty from 1 to 1.82 was quite a braggadocious achievement.