By now, you’ve probably seen hundreds of articles about blockchain technology, each attempting to describe what it is, and how it’s the next big revolution to hit the tech world. At the end of the article, many people are still perplexed because the authors brush over a lot of complicated concepts without explaining them.
In this guide, we’ll be giving you a ground-up explanation of what blockchains are, how they work, and the key cryptographic concepts behind them. It’s time to get past the hype and buzzwords, and understand what’s really going on at a technical level.
Don’t worry, because we will take things slowly and examine each element in detail, giving you analogies that help to visualize what’s actually happening.
A simple analogy for blockchains
Imagine a primitive village, where they don’t have money in our traditional sense. Instead, they engrave the details of each transaction onto a stone block, then cement it in place in the center of the village.
David swaps fifteen chickens for one of Sarah’s pigs. They engrave this information onto a block, then cement it in the town square. Now, anyone can see that David is the new owner of Sarah’s pigs, while Sarah is now the owner of the fifteen chickens. Since the information is public, there can be no disputes over who legally controls what.
The next day, Jessica trades Mark 100 kilograms of corn for a canoe. This is also engraved on a block, which is then cemented on top of the old block. Since everyone in the town will now be able to publicly verify that Jessica no longer owns the 100 kilograms of corn, she can’t try to sell it again if Mark goes away for a couple of days.
In the coming days, more and more transactions take place, and more blocks of stone with the transaction details engraved on them get cemented in place. Over time, the transaction stones start to form a tower.
All of the details are publicly available to everyone, and the people cannot change or take back the earlier transactions, because a bunch of blocks are cemented on top of them.
This village’s financial system may not be the easiest to use, but it gives everyone in the village a way to keep track of their transactions. It is a public ledger that keeps permanent records, which can’t be altered. One of the most important aspects is that it is decentralized. There is no central bank or government that is responsible for the transactions. It’s all done by the community.
There are a number of differences between blockchains and the above analogy, but it’s still a good starting point to get your head around what blockchains are and why they are useful.
One of the key contrasts is that blockchains aren’t on display in public, instead, anyone who wants to can store a copy of a blockchain on their computer. Blockchains use cryptography, computers and electricity to build the blocks, rather than stone and cement.
The most important aspects of blockchains are that they cannot be changed, aren’t controlled by any single entity, and everyone can view the transactions. These properties are why people believe that the technology has the potential to be used in a vast range of applications.
The history of blockchains
In the late 1990s and early 2000s, there were a series of developments toward digital currencies based on various cryptographic concepts. One of the earliest blockchain-like initiatives was Nick Szabo’s 1998 mechanism called bit gold. Although it was never actualized, it involved a series of cryptographic puzzles, where each solution would be added to the next puzzle, forming a chain.
It wasn’t until 2008 that the idea of blockchains was fully developed, when someone going under the pseudonym of Satoshi Nakamoto published the paper, Bitcoin: A Peer-to-Peer Electronic Cash System.
This person built on previous work in the field, including Hal Finney’s reusable proof-of-work system, to form the bitcoin digital currency, as well as the underlying concept of blockchains. These blockchains have since gone on to be applied in a number of different ways, both as digital currencies and as solutions to other problems.
The bitcoin network was launched in early 2009 and was originally only used by a small group of cryptographers and hobbyists. It wasn’t until bitcoin was adopted by darknet marketplaces such as Silk Road that blockchains began to see their widespread, practical adoption.
As bitcoin gained popularity, a number of spin-off cryptocurrencies, known as altcoins such as Litecoin and Peercoin were developed. These further spread the adoption and use of blockchain technology.
Ethereum launched in 2015 as a distributed computing platform that allowed its users to develop apps and enact smart contracts between parties. Around this time, interest in blockchain technology from the public, major companies and governments grew, in both financial and other use cases. This saw a surge of new activity, with blockchains being proposed as solutions to a range of different problems.
The uses of blockchains
It’s been more than 10 years since the first blockchain was launched, with intensive hype and investment for the past five or so years. Despite the flurry of activity, at this stage there have been relatively few successful real-world implementations of blockchain technology.
While cryptocurrencies have seen their values spike and plummet, they still see comparatively few transactions for everyday use. The number of businesses which accept them is limited, while the transaction costs for bitcoin become too high when the currency is frequently used. On top of this, the bitcoin network can’t handle anywhere near the volume of transactions as an alternative like the traditional Visa system.
While there are cryptocurrencies that seem more promising than bitcoin, these are accepted in even fewer places. At this stage, it seems like the main uses for cryptocurrencies are as speculative investments or to buy illicit products from darknet marketplaces.
Decentralized apps & smart contracts
After bitcoin, the most renowned blockchain-based project is Ethereum, which provides a platform for developing decentralized apps and smart contracts. Despite the excessive hype, having a market cap of $14 billion (at the time of writing), and more than 2,300 decentralized apps, it has very little to show for it at this point in time.
One of the Ethereum network’s most successful projects so far is probably Cryptokitties, a video game where users can breed cats. MakerDAO, the app that currently has the most daily active participants, had less than 1,000 users in the past 24 hours (at the time of writing). Considering that Ethereum has been labeled a “Financial Tech Revolution”, among its many praises, its current applications seem quite underwhelming.
Other blockchain-based initiatives
Over the last several years, there have been countless blockchain-based startup companies as well as many initiatives backed by our biggest tech companies and financial institutions.
Numerous pilot programs and experiments aim to adapt the technology for use in supply-chain management, financial transactions, smart contracts, decentralized storage and more.
These include IBM Food Trust and Walmart’s foray into using the technology to manage food supply logistics, banks such as UBS adapting blockchains for financial settlements, and the Australian Stock Exchange (ASX) aiming to adopt distributed ledger technology.
Despite this, it’s hard to name a successful and widely used product or service that has emerged from all of the investment and effort. This isn’t to say that blockchain technology won’t have any future uses, just that it is yet to be as fruitful as many may have hoped. At this stage, it’s hard to know whether or not some of these projects will be successful in the coming years.
The core cryptographic concepts behind blockchains
Whether or not blockchains currently see a lot of real-world usage, they are still interesting applications of cryptography.
These days, countless blockchains each have their own unique variations. Since it isn’t possible to cover each of their individual aspects, we will focus on the core concepts and how they relate to cryptography. We will mainly be focusing on bitcoin, not because it is the best blockchain, but because it is the first one, which all of the others are based upon.
What is cryptography?
Before we dive in too deep, it’s important to cover the basics. Cryptography is the study and practice of keeping secret information away from adversaries. In the early days, it was done simply, using techniques such as changing each letter in a word to the letter that follows it in the alphabet. Under this type of scheme:
If your recipient knows how to convert the coded message back to its original form and your adversary doesn’t, then you can assume that it is a safe way to communicate.
Over time, people have gotten much better at cracking codes. Technological advances also improved our code-breaking abilities significantly. In order to keep our information secure in the present day, we now have to use codes that are much more complex.
Bitcoin: The first blockchain
Now that you know the basics about the study of cryptography, it’s time to start looking at the underlying structure of bitcoin, the original blockchain. Bitcoin was initially proposed as a cryptography-based currency that could avoid the downsides of having a financial system controlled by central institutions.
At the core of bitcoin is the idea of transferring value through a chain of digital signatures, which are similar to handwritten signatures. This idea in itself wasn’t revolutionary, but it’s important to understand how it works in order to see the bigger picture. Let’s use an example with handwritten signatures to explain how this process can work:
Sarah didn’t have any money, so she asked Ann for $5. Ann said okay, but only if Sarah gave her a massage. Sarah couldn’t do it right at that moment, but Ann is a stickler for rules and enforcement, so she drew up a quick contract.
I, Sarah, owe the bearer of this paper one 10-minute massage.
Ann made Sarah sign it so that the contract was legitimate.
Later on, Ann decided that she didn’t want a massage, and offered to sell the contract to Jason for $5. Jason decided to buy it because he really wanted a massage. Ann then signed it as proof that she was giving it to another party.
While Jason is now the owner of the contract, there is a problem. How can he know whether or not Ann had already redeemed the massage? Maybe Sarah doesn’t owe anyone a massage anymore, and the contract is worthless.
We’ll get to the answer to this problem later on, in the How can blockchains prevent double-spending? section. For now, let’s talk about digital signatures and hashing, two of the most important concepts that form the foundations of blockchains.
Before we can explain digital signatures, we have to do a bit of backtracking and talk about some security basics. When we transmit valuable data online, there are four important properties which we often need:
- Confidentiality – The ability to keep data hidden from unauthorized parties.
- Authentication – This property involves being able to verify that the other party is really who they say they are, and not some impostor or spy.
- Integrity – If data retains its integrity, it means that it hasn’t been altered or tampered with by anyone else.
- Non-repudiation – This property essentially means that the individual or entity who was responsible for an action cannot claim that they weren’t involved. In everyday life, we use our handwritten signatures as a form of non-repudiation. It’s hard for you to deny that you agreed to a contract when your signature has been used to sign it.
Without each of these properties, how could we be confident that important data really represents what it is supposed to, and that our enemies haven’t accessed or changed it?
Normally, we use encryption algorithms such as AES to take care of confidentiality. For the other three properties, we turn to digital signatures.
There are two major types of encryption:
- Symmetric-key encryption – In symmetric-key encryption, the same key is used to both encrypt and decrypt data. This is an efficient method that is used everywhere in information security, from encrypting your hard drive, to securing your connection to a HTTPS website. The most commonly used symmetric-key algorithm is AES.
- Public-key (asymmetric) encryption – Public-key cryptography uses separate keys for the encryption and decryption processes. These are the public key, which is shared openly, and the private key, which must be kept secret. It relies on some interesting mathematical properties, and enables two parties who have never met before to securely exchange information. It is relatively inefficient, so in practice, public-key cryptography is only used to encrypt the symmetric key, which in turn is used to encrypt data.
Digital signatures are much like normal signatures. We sign a receipt to verify that the information on it is correct and retains its integrity. It’s very hard to repudiate our handwritten signatures, because they are so hard to copy. Since we have our signatures on our bank and ID cards, they also serve as a form of authentication. Anyone can check whether a signature matches the government-issued identity.
Digital signatures rely on public-key encryption. If Alice wants to prove that a piece of data is authentic, retains its integrity and she does not want to be able to repudiate it, she can send a digital signature alongside the data.
To create a digital signature, Alice first takes the data and puts it through a hashing algorithm to form a unique string of numbers (this is explained fully in the Hashing section). These numbers are then digitally signed using the ECDSA algorithm and her private key.
Essentially, the hash and Alice’s private key are combined using a complex mathematical formula. The result is the digital signature, which can be verified with Alice’s public key to prove that she is the real owner of her matching private key, and not an impostor.
Digital signatures allow individuals to prove their ownership of the private key without having to reveal it to the other party. For a deeper dive into how this process works, see our comprehensive guide on digital signatures.
Once Alice creates her digital signature, she then sends it to her recipient, Bob, alongside the data. When Bob receives the data, he can verify its authenticity, check whether it retains its integrity and see whether it is non-repudiable, all by using Alice’s public key.
Alice will most likely have shared her public key with Bob ahead of time, otherwise Bob will be able to find it on a key server (this is a server where many people host their public keys, so that others can find them and contact them in a secure manner).
Bob takes the digital signature and Alice’s public key and computes them together using the reverse of the algorithm that Alice used.
Due to the unique mathematical properties of this calculation, the result will be the same as the hash of Alice’s data from before she digitally signed it with her private key.
Bob then runs the message that he received through the same hash function that Alice used. If this message has not been altered since Alice signed it, then the hash function will give Bob the same result that he got from the computation he performed with Alice’s public key.
If the two values are different, it means the data has been altered, that it was not signed by Alice’s real private key, or there was some other problem. For the sake of our example, let’s say that the two values matched, and the data is in fact legitimate.
In bitcoin and other blockchains, digital signatures are mainly used in the transaction process as a way for someone to prove their ownership, without having to reveal their private key.
Hashing is the process of sending data through a hash function to produce a specific, essentially unique hash of a fixed length. In blockchain applications, we use cryptographic hash functions such as SHA-256.
Cryptographic hash functions have several important characteristics which make them useful:
- They are deterministic – a given input will always have the same output.
- Each output is essentially unique. The chances of two separate inputs having the same output are so low that we don’t really worry about it.
- It is infeasible to figure out the original input from the output (under current techniques and technology).
- Hashes can be computed quickly.
- A slight change in the input results in a significantly different output.
As an example, if we put “Let’s eat dinner” through an online hash function, it gives us:
Every time we put it through, it will give us the same result. But if we change even one character, it returns a value that is completely different. “Let’s eat dinnet” gives us:
So, we have this mathematical function with a range of interesting properties, but how is it useful in blockchain applications?
The properties of hashes allow us to:
- Prove that we possess certain information, without having to reveal that information.
- Prevent transactions from being altered by adversaries.
- Verify the confirmation of transactions without having full knowledge of a block.
- Reduce the bandwidth of transactions.
- Make cryptographic puzzles, which are part of the mining process.
These various features of hashes are used in four major areas of the bitcoin system:
- When a transaction is being made, data from previous transactions is hashed and included in the present transaction.
- When a new transaction is made, the data is also hashed to form a transaction ID (txid), which is an identifier that can be used to locate the transaction details on the blockchain.
- A hash of the public key is used as the address where users can send funds. This makes the addresses shorter and more convenient, as well as providing some security benefits.
- As part of bitcoin’s proof-of-work system (this is discussed later in the Proof-of-work section).
Bitcoin’s basic transaction process
Now that we have explained a couple of the major cryptographic techniques behind the bitcoin blockchain, we can take a look at how these are used in a transaction.
The first thing that you need to be aware of is that bitcoin transactions don’t happen in an intuitive way. All of the bitcoin that someone owns aren’t all jumbled together, and they can’t just be scooped out in the exact amount that is needed for a transaction (plus the transaction fees).
Instead, the total balance is kept separately in allotments according to how it was received. Let’s say that Alice has a total balance of 12 bitcoins, which she received over three separate transactions. Her bitcoins will be stored in the separate amounts that she received them in from the previous transactions.
Let’s say that her balance is made up of one previous transaction of three bitcoins, one previous transaction of four bitcoins, and one previous transaction of five bitcoins. This makes a total of 12 bitcoins. Each of these amounts are the outputs from the previous transactions, and they are now under Alice’s control.
Now, let’s say that Alice wants to make a transaction of ten bitcoins to buy a car from Bob. To cover the total costs, she would need to use the bitcoins from each of the three previous transactions. These outputs from past transactions would now become the inputs for the new transaction.
It may seem strange, but since the previous transactions don’t make up exactly 10 bitcoins, Alice can’t just send 10 bitcoins across and leave two in her wallet. Since the only way to make up the 10 bitcoins is to combine all three past transactions as inputs, she would have to send the entirety of the three allotments, totaling 12 bitcoins, to cover the total value of the transaction.
Fortunately, this doesn’t mean that Alice loses the extra two bitcoins. They are processed as part of the transaction, but they are returned to her as change (minus the transaction fee).
The transaction process for buying a 10 bitcoin car
3 (from one of Alice’s previous transactions) > 10 (to Bob)
4 (from one of Alice’s previous transactions) > 2 (back to Alice as change)
5 (from one of Alice’s previous transactions) >
Normally, the fee would also be taken out prior to returning the change to the sender. Since the fee amount would be negligible in comparison to the transaction amounts, we have left the fee out to keep the numbers tidy.
Looking deeper into the transaction
For the above transaction, each of the inputs would have had their previous transaction data hashed, and it would then have been included in the current transaction. Additionally, Alice has to prove that she has ownership of the three separate inputs (which are outputs from previous transactions–we know, it’s confusing!).
Alice does this using signature script, which is an unlocking script. This script is made up of two aspects, Alice’s public key and her digital signature. The public key indicates the address of the outputs from the previous transactions (which she wants to use as inputs for the new transaction), while her digital signature shows that she is the true owner.
As we discussed in the Digital signature section above, her signature proves that she is the owner, because the digital signature could only have been made using her private key. Alice’s ownership is verified with her public key using a public-key script.
This diagram shows how transactions form a chain. In the second transaction, Owner 2 combines their public key with the data from the previous block. Owner 2 also creates a digital signature with their private key to prove their ownership of the coin. This is verified with Owner 2’s public key. Bitcoin transaction visual by Inkscape licensed under CC0
How can blockchains prevent double-spending?
By now, you hopefully have a reasonable idea about the underlying cryptographic processes that bitcoin and other blockchains use in their transactions. This brings us back to where we left off in our earlier example: How can Jason know whether or not the massage has already been redeemed? We refer to this as the double-spending problem.
On any decentralized, pseudonymous network, it’s expected that some people will cheat to try and enrich themselves. Bitcoin and other blockchains solve this problem with a peer-based verification process called mining.
To explain how this works, let’s stretch our earlier analogy a little bit further:
The best way to visualize the blockchain mining process is if, whenever a transaction is made, a copy of the contract is sent to everyone within the friendship group.
After a number of transactions have been made, each person would combine the transaction details into one folder. Everyone in the friendship group would then take the result from the previous folder of transactions, combine it with the current transaction details, and then try to solve a complex mathematical problem using these inputs.
The first person to get find the solution would then broadcast it to all of the other members of the friendship group, who are able to quickly verify whether they have included the correct transactions, and whether they have the right answer or not.
If the answer is correct, the person who succeeded first receives a reward. This reward is what incentivizes everyone to validate the transactions. If anyone tries to cheat, the rest of the group will find out, which means that cheaters have no chance of claiming the reward and that the effort would be wasted.
Once a person has successfully completed the mathematical problem and claimed their reward, the whole group begins collecting new transactions in another folder. Once they have enough, they combine them with the result from the previous folder and compete to solve a new mathematical problem in the hope of winning the next reward.
The result of the previous folder is included in the new one, and a chain of results is formed, which allows people to check the transaction histories and verify that everything along the chain is legitimate.
Since everyone keeps a copy that includes the transaction history, and the only effective financial incentive is to honestly contribute to the validation process, this prevents double-spending from occurring.
This whole process may seem inefficient, but thankfully everyone who makes a blockchain transaction doesn’t have to do this. The task is left to miners and it’s automated, so it doesn’t involve anywhere near as much work as our example does.
The above analogy is imperfect, because it’s a simplification of a relatively complex process. The main aim is to give you a visual idea of what is really going on. We will discuss how things actually work on a more technical level in the following sections.
Nodes & miners
A node stores a copy of the blockchain, while a miner creates and validates the blocks. Full nodes store the entire history of blockchain transactions, while miners are only concerned with the previous block and the current one they are working on.
In addition to storing the blockchain data, nodes serve as network relays, helping to distribute information to both users and miners. Nodes also verify the blocks that miners generate by making sure that hashes match the transaction data.
In the early days of bitcoin, there was no separation between nodes and miners. The terms were used interchangeably to refer to the entities that competed to validate the transactions in a block, and also stored the blockchain that was used to verify past transactions.
These days, it’s possible to host a node without actually mining. A full node can be used both as a wallet, and to verify the chain of transactions, because it contains a complete copy of the blockchain. In the same vein, miners don’t technically have to host a node, although in reality many do.
The first step towards preventing double-spending is to widely publish a record of previous transactions. If everyone has a copy of the previous transaction records, they know whether certain coins have already been spent.
In the bitcoin protocol, blocks of transaction data are hashed, then the hash is spread throughout the network. This hash acts as a timestamp, proving that the data must have existed at the time that the hash was created–otherwise the hash could not exist.
Each new timestamp is a hash that combines the current block’s transaction data and the timestamp of the previous block. This creates a chain of timestamps, with future ones solidifying those timestamps that came before them.
One of the main aims of a blockchain is to create a decentralized system that can verify itself without the need for third parties. This is generally achieved via a peer-to-peer verification process, where the network offers financial incentives for honestly validating transaction data. Many blockchains refer to this process as mining.
In the bitcoin protocol, every time a transaction is made, the details are sent through a relay of nodes until every node on the network receives the data.
The miners then collect each of these transactions and form them into a block. Each miner then tries to solve the cryptographic puzzle for the block. When a miner succeeds, it sends the block to all of the nodes on the network.
The nodes will only accept the block if all of the transactions within it are verified and haven’t already been spent. When nodes accept a block, they take its hash and distribute it to miners, who then integrate it into the next block of transactions that they are trying to solve.
If two separate miners solve a block at the same time, the other miners will take the data from whichever block they received first, and incorporate it into the next block they are working on. They will also save the data from the second block, just in case they need it later on.
The entire network will be working on either one block or the other until the next block is solved. At this point, those that were working on the other block will abandon it. This is because miners will always accept the longest chain as the correct one. They focus their work towards extending the longest chain, because this is the most likely way for them to end up with the reward.
The bitcoin protocol uses a concept known as proof-of-work to validate its transactions. It’s based on Adam Back’s earlier Hashcash scheme. Other blockchains use proof-of-stake, proof-of-storage or proof-of-space systems, but we won’t go into the latter two in this article.
In order to add a timestamp to the network, a miner must be the first to complete a cryptographic puzzle, then spread the result to the nodes on the network, which verify the answer. The cryptographic puzzle requires a significant amount of computational resources, and miners complete it in the hope of solving the block and receiving the reward. The reward is currently set at 12.5 bitcoins.
If a miner creates a block that does not match the results of the rest of the network, the block will be left behind, and the resources that they expended will have been wasted. Excluding exceptional circumstances (such as a 51% attack), it is more profitable for a miner to act honestly, rather than attempt to disrupt the network or post fraudulent results.
This proof-of-work mechanism is what keeps the network honest. If someone wanted to alter or tamper with a block, they would have to completely redo the work of solving the block. The further back a block is on a chain, the more difficult it is to tamper with. This is because all of the blocks that come after it would also need to be altered.
This proof-of-work system is based on the SHA-256 algorithm. It’s suitable for the system’s needs because it is relatively difficult to compute the solution, but easy to verify it.
Proof-of-work algorithms require significant processing power, which makes them expensive in terms of infrastructure and energy costs.
Lightweight alternatives such as proof-of-stake have emerged to make the verification process more efficient. Proof-of-stake blockchain protocols have varying techniques, but they generally involve choosing the creator of the next block based on a combination of randomness and coin age or wealth.
The amount of coins that a user has, or alternatively, how long the coins have been held, act as the user’s stake. The stake ensures that the user is actually committed to the overall health of the system.
While these factors are important in the delegation of the next block, they are combined with randomness to prevent the system from being centralized by the richest or oldest users.
Selecting the next block through proof-of-stake systems ensures transactions are validated correctly, but in a much more efficient manner than the computations involved in proof-of-work schemes.
Ethereum is currently moving towards a proof-of-stake algorithm to increase its efficiency, while PeerCoin and NXT have already implemented proof-of-stake systems.
The only way to solve the SHA-256 cryptographic puzzles and win the reward is through brute force. This involves randomly guessing numbers until a miner comes up with the right answer.
To complete the puzzles, miners take the current block’s transaction data and the hash of the previous block as part of their input. They then need to guess a separate input, known as a nonce (an arbitrary number), so that when all of these inputs are put through the hash function, the resulting value begins with a set number of zeros. A solution could look something like this:
As the bitcoin network becomes more powerful, the difficulty of the puzzles is increased exponentially by requiring solutions to include a greater number of zeros. This makes it much harder and more time consuming to find a correct answer.
It can be hard to visualize how this process works.These cryptographic puzzles may seem complicated, but the proof-of-work system is important for maintaining the integrity of blockchains. If validating transactions didn’t require a significant expenditure of computing power, it would be much easier for attackers to tamper with the system.
Blockchains: Rapidly emerging technology
Throughout this article, we have mainly talked about how these processes work in the context of the bitcoin protocol. This is simply because bitcoin was the first functional blockchain, and most others are based heavily on its design.
These days, there are thousands of different cryptocurrencies and blockchains, and it would have been impractical to cover the technical distinctions between each one. Despite the differences that exist between them, the entire blockchain world is based on cryptographic concepts such as public-key cryptography, digital signatures and hashing.
The varying blockchains just have slightly different structures and apply these concepts in their own individual ways. These different implementations have their own unique benefits, which give blockchains the potential to be used in a wide variety of situations.