Rishabh's Blog

What is SegWit?

What is SegWit?

Rishabh's photo
Rishabh
·

11 min read

It stands for Segregated Witnesses. It was the biggest Bitcoin protocol update till date(before taproot of course or maybe it was bigger than taproot).

Benefits:

  • It got rid of transaction malleability. SegWit solved this problem by moving the "witness" data of a transaction, which includes the signature, to a new part of a Bitcoin block.
  • It also paved the way for the Lightning Network and other layer two protocols
  • The block size limit was replaced with a four million weight unit limit, which introduce a new way to "count" transaction data. This means that users with SegWit-supporting wallets will pay lower transaction fee.
  • SegWit also made it easier to deploy further upgrades to the Bitcoin protocol. One of these upcoming upgrades could be Schnorr signatures, a new signature algorithm that would further increase programmability and flexibility of Bitcoin protocol

Within Bitcoin's technical community, SegWit was not controversial.

However, there was controversy outside Bitcoin's technical community. The only valid point of controversy which had some validity is that it would have been "cleaner", code-wise, to deploy the upgrade as a hard-fork instead of a soft-fork, as this would leave less technical doubt in the protocol. But hard forks would have had their own problems which would have been much greater.

SegWit was activated in August 2017 even though the code was ready by mid 2016 because some significant Bitcoin miners refused to activate the protocol.

The motivations of these miners are still speculated about, but it seems that they either used SegWit as a bargaining chip or they "blocked" the upgrade because it was incompatible with a mining optimization("AsicBoost") that they were secretly using — or both.

We can use SegWit by using a wallet that has integrated SegWit. This wallet should generate SegWit addresses for us. When we make payments using a SegWit enabled wallet, the fee we pay is less than what we would have payed if we hadn't used SegWit.

There are two types of Segwit addresses.

  • P2SH type, address starts with "3". All addreses starting with 3 might not be SegWit addresses.
  • bech32 address, starts with "bc1" and is always a SegWit address(cheapest)

Note: Addresses starting with "1" are never SegWit addresses

Why doesn't everyone use SegWit?

There are probably two reasons not to use SegWit:

  • Implementing SegWit requires an upgrade and some people are just slow to do it. For large companies it may take significant time and effort.
  • The second reason would be "Political": They aren't upgrading to SegWit as sort of protest. They would have preferred different scaling solution.

It is worth noting that even if everyone doesn't upgrade to SegWit, those who do upgrade enjoy the benefits regardless. While the overall fee level might be slightly lower for SegWit users if everyone else were to use SegWit as well, the added benefit of complete migration is small. Plus, if fewer people use SegWit, Bitcoin blocks are smaller which has benefits as well.

Technicalities

  • SegWit was proposed and implemented as a BIP-9 soft-fok that was activated on Bitcoin's mainnet on August 1st, 2017.
  • In cryptographic terms "witness" is used to describe a solution to a cryptographic puzzle. In bitcoin terms, the witness satisfies a cryptographic condition placed on an UTXO.
  • A digital signature is one type of witness, but a witness is more broadly any solution that can satisfy the conditions imposed on an UTXO and unlock that UTXO for spending. The term "witness" is a more general term for an "unlocking script" or "scriptSig".
  • Before segwit's introduction, every input in a transaction was followed by the witness data unlocked in it. The witness data was embedded in the transaction as part of each input. The term segregated witness or segwit for short, simply means separating the signature or unlocking scriptof a specific output.

Associated BIPs:

  • BIP-141 The main definition of Segregated Witness
  • BIP-143 Transaction Signature Verification for Version 0 Witness Program
  • BIP-144 Peer Services- New network messages and serialization formats
  • BIP-145 getblocktemplate Updates for Segregated Witness (for mining)
  • BIP-173 Base32 address format for native v0-16 witness outputs

Transaction Malleability

The transaction hash used as identifier no longer includes the witness data. Since the witness data is the only part of the transaction that can be modified by a third party, removing it also removes the opportunity for transaction malleability attacks.(See BIP62 and BIP66)

Script Versioning

Every locking script is preceded by a script version number, similar to how transactions and blocks have version numbers. The addition of a script version number allows the scripting language to be upgraded in a backward-compatible way to introduce new script operands, syntax or semantics.

Network and Storage Scaling

The witness data is often a big contributor to the total size of a transaction. More complex scripts such as those used for multisig or payment channels are very large. In some cases these scripts account for the majority of the data in the transaction.

After SegWit, nodes can prune the witness data after validating the signatures, or ignore it altogether when doing simplified payment verification. The witness data need not be transmitted to all nodes or stored on disk by all nodes.

Signature Verification Optimisation

SegWit upgrades the signature functions (CHECKSIG, CHECKMULTISIG, etc) to reduce the algorithm's computational complexity. It reduced the complexity from O(n^2) to O(n).

Offline Signing Improvement

SegWit incorporates the value referenced by each input in the hash that is signed. Previously, an offline signing device such as a hardware wallet, would have to verify the amount of each input before signing a transaction. This was usually accomplished by streaming a large amount of data about the previous transactions referenced as inputs. Since the amount is now a part of the commitment hash, an offline device does not need the previous transactions. If the amounts do not match, the signature will be invalid.

How SegWit Works

  • A transaction can spend Segregated Witness outputs or traditional outputs or both. Therefore we should refer to specific transaction output as "Segregated Witness Outputs"
  • When a transaction spends an UTXOs it must provide a witness In a traditional UTXO, the locking script requires that the witness data be provided inline in the input part of the transaction that spends the UTXO, however specifies a locking script that can be satisfied with witness data outside of the input.

SegWit Output and Transaction Examples

Example of P2PKH output script

DUP HASH160 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 EQUALVERIFY CHECKSIG

With Segregated Witness, Alice would create.a Pay-to-Witness-Public-Key-Hash(P2WPKH) script, which looks like:

0 ab68025513c3dbd2f7b92a94e0581f5d50f654e7

"0" in the P2WPKH script signifies the version number(witness version)

The second part is the witness program. The 20-byte witness program is simply the hash of the public key; as in P2PKH script.

Normal P2PKH output being spent:

[...]
“Vin” : [
"txid": "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2",
"vout": 0,
          "scriptSig": “<Bob’s scriptSig>”,
]
[...]

In SegWit Output there us no sriptSIg and it includes the signature in the witness data in the second part of the transaction.

[...]
“Vin” : [
"txid": "0627052b6f28912f2703066a912ea577f2ce4da4caa5a5fbd8a57286c345c2f2",
"vout": 0,
          "scriptSig": “”,
]
[...]
“witness”: “<Bob’s witness data>”
[...]

Wallet construction of P2WPKH

It is extremely important to note that P2WPKH should only be created by the payee and not converted by the sender from a known public key, P2PKH script, or address. The receiver has no way of knowing if the sender's wallet is SegWit enabled or not. Also, P2WPKH must be constructed from the hash of compressed public key.

P2WSH(Pay to witness script hash)

Example of P2SH output:

HASH160 54c557e07dde5bb6cb791c7a540e0a4796f5e97e EQUAL

Above is a 2-of-5 multisignature redeem script.

Decoded when spending:

[...]
“Vin” : [
"txid": "abcdef12345...",
"vout": 0,
          "scriptSig": “<SigA> <SigB> <2 PubA PubB PubC PubD PubE 5 CHECKMULTISIG>”,
]

Above is normal P2SH. Let's see how it will work with the witness i.e. Pay to Witness Script Hash.

Example of P2WSH output script:

0 a9b7b38d972cabc7961dbfbcb841ad4508d133c47ba87457b4a0e8aae86dbb89

It is a lot simpler and omits the various script operands that we see in P2SH scripts.

Two values get pushed on the stack, a witness version and a 32-byte SHA256 encrypted redeem script.

We essentially put everything in the witness instead of sigScript when redeeming an output

Decoded transaction showing a P2WSH output being spent with separate witness data

[...]
“Vin” : [
"txid": "abcdef12345...",
"vout": 0,
          "scriptSig": “”,
]
[...]
“witness”: “<SigA> <SigB> <2 PubA PubB PubC PubD PubE 5 CHECKMULTISIG>”
[...]

Difference between P2WPKH and P2WSH:

  • The pub key hash in P2WPKH is 20 bytes
  • The script hash in P2WSH is 32 bytes

This allows the wallet to differentiate between two types of witness programs.

For P2WPKH and P2WSH payment types, both the sender and the recipient wallets need to be upgraded to be able to use segwit. Furthermore, the sender’s wallet needs to know that the recipient’s wallet is segwit-aware.

Embedding Segregated Witness inside P2SH

Let's assume Alice's wallet is not SegWIt upgraded but Bob's wallet is. Bob would likely want to use segwit to reduce transaction fees, taking advantage of the discount that applies to witness data.

Bob's wallet can construct a P2SH address that contains a segwit script inside it. Alice's wallet sees this as a "normal" P2SH address anf can make payments without knowing about SegWit.

Both forms of witness scripts, P2WPKH and P2WSH, can be embedded in a P2SH address. The first is noted as P2SH(P2WPKH) and the second is noted as P2SH(P2WSH).

Pay-to-Witness-Public-Key-Hash inside Pay-to-Script-Hash

The first form of witness script we will examine is P2SH(P2WPKH). This is a Pay-to-Witness-Public-Key-Hash witness program, embedded inside a Pay-to-Script-Hash script, so that it can be used by a wallet that is not aware of segwit.

Bob’s wallet constructs a P2WPKH witness program with Bob’s public key. This witness program is then hashed and the resulting hash is encoded as a P2SH script. The P2SH script is converted to a bitcoin address, one that starts with a "3," as we saw in the Pay-to-Script-Hash (P2SH) section.

Bob’s wallet starts with the P2WPKH witness program we saw earlier:

Bob’s P2WPKH witness program

0 ab68025513c3dbd2f7b92a94e0581f5d50f654e7

The P2WPKH witness program consists of the witness version and Bob’s 20-byte public key hash.

Bob’s wallet then hashes the preceding witness program, first with SHA256, then with RIPEMD160, producing another 20-byte hash.

Let’s use bx on the command-line to replicate that:

HASH160 of the P2WPKH witness program

echo \
'0 [ab68025513c3dbd2f7b92a94e0581f5d50f654e7]'\
 | bx script-encode | bx sha256 | bx ripemd160
3e0547268b3b19288b3adef9719ec8659f4b2b0b

Next, the redeem script hash is converted to a bitcoin address. Let’s use bx on the command-line again:

P2SH address

echo \
'3e0547268b3b19288b3adef9719ec8659f4b2b0b' \
| bx address-encode -v 5
37Lx99uaGn5avKBxiW26HjedQE3LrDCZru

Now, Bob can display this address for customers to pay for their coffee. Alice’s wallet can make a payment to 37Lx99uaGn5avKBxiW26HjedQE3LrDCZru, just as it would to any other bitcoin address.

To pay Bob, Alice’s wallet would lock the output with a P2SH script:

HASH160 3e0547268b3b19288b3adef9719ec8659f4b2b0b EQUAL

Even though Alice’s wallet has no support for segwit, the payment it creates can be spent by Bob with a segwit transaction.

BIP-173

The native SegWit address format is defined in BIP-173

BIP-173 only encodes witness scripts. It is not compatible with non-segwit P2PKH or P2SH scripts. BIP-173 addresses are also called bech32 addresses.

BIP-173 addresses use 32 lower-case-only alphanumeric character set, carefully selected to reduce errors from misreading or mistyping. By choosing a lower-case-only character set, bech32 is easier to read, speak, and 45% more efficient to encode in QR codes.

Example of bech32 addresses:

Mainnet P2WPKH

bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4

Testnet P2WPKH

tb1qw508d6qejxtdg4y5r3zarvary0c5xw7kxpjzsx

Mainnet P2WSH

bc1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3qccfmv3

Testnet P2WSH

tb1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3q0sl5k7

Deconstruction

  • There's a human readable part 'bc' or 'tb' identifying mainnet or testnet.
  • The separator '1', which is not the part of the 32-character encoding set and can only appear in this position as a separator
  • A minimum of 6 alphanumeric characters, the checksum encoded witness script.

Transaction identifiers

  • SegWit introduces a wtxid in addition to the already present txid
  • The txid in SegWit is the double SHA-256 hash of the transaction where the scriptSig is empty. So the txid is immutable by the third party.
  • However the wtxid which is like an "extended"ID, in that the hash also incorporates witness data may be malleable, it should be considered malleable until the transaction is confirmed.

SegWit's New Signing Algorithm

  • Signatures in bitcoin transactions are applied on a commitment hash, which is calculated from the transaction data, locking specific parts of the data indicating the signer's commitment to those values. For example, in a SIGHASH_ALL type signature, the commitment hash includes all inputs and outputs.
  • The way commitment hash was calculated introduced the possibility that a node verifying the signature can be forced to perform a significant number of hash computations. Specifically, the hash operations increase in O(n^2) with respect to the number of signature operations in the transaction.
  • SegWit addressed this problem by changing the way commitment hash is calculated
  • For SegWit version 0 witness programs, signature verification occurs using an improved commitment hash algorithm as specified in BIP-143.
  • The number of hash operations become O(n) to the number of signature operations reducing the opportunity to create denial-of-service attacks with overly complex transactions.
  • The commitment hash now also includes the value of each input as part of the commitment. This means that a signer can commit to a specific input value without needing to "fetch" and check previous transaction referenced by the input.
  • In the case of offline devices, such as hardware wallet, removing the need to stream previous transaction for validation.
  • A hardware wallet can accept the input value "as stated" by an untrusted host. Since the signature is invalid if that input value is not correct, the hardware wallet doesn't need to validate the value before signing the inputs
BitcoinSecurityscalability