Decentralized storage will bring together the best features of blockchain technology, with attributes that meet the practical demands of storing high volumes of data
The total market for cloud storage is large and continues to grow. Amazon Web Services, which includes Amazon’s cloud storage offerings, earned $4.1 billion in revenue in Q2 of 2017, implying a $16B annualized rate for both compute and storage offerings. Market research groups have estimated the future size of the cloud storage market: $74.94 billion by 2021 and $92.49 billion by 2022.
Speed matters. End users want fast, seamless access to content. If your site is slow, they are just going to go to someone who can deliver. Just how quick do you need to be?
A study from 2009 found that 47% of users expect a page to load in 2 seconds. If it didn’t load in under 3 seconds, 40% were out of there. The study was released over eight years ago, and today, we expect a lot more, we expect things on-demand.
Customers won’t wait around for your site to load. They are not going to sit patiently as your images load or video buffers. Almost instant access to a global marketplace means millions of options they could use instead.
It isn’t just users. Google uses site speed on both desktop and mobile sites as part of its SEO rankings. That’s why Content Distribution/Delivery Networks (CDNs) are becoming an integral part of smart business setup.
More than just your website speed, user expectations are spread across the spectrum. We want faster transactions, easier access, more intuitive interfaces and guaranteed privacy.
Storage infrastructure has transformed to generally being very centralized. Rather than businesses and corporations hosting their own servers, many have migrated to the cloud. This turn toward centralization lets companies push workloads to larger, public clouds in order to utilize the storage infrastructure of large companies, like Amazon Web Services or Microsoft Azure.
Because of the centralization currently inherent in cloud computing, resources could be shared and the end customer could have much lower costs and higher efficiency and uptime. With this, cloud computing has massively grown, with Gartner claiming that the “cloud shift” will affect more than $1 trillion in IT spending by 2020.
No one is doubting the benefits of cloud computing. However, data breaches or service interruptions aren’t unheard of, or even uncommon. Whether it’s user error or issues with the hosts themselves, data breaches in 2017 were a major story.
Centralized servers containing valuable or sensitive information are targets for hackers, as we saw in the iCloud hack of 2014 and the Dropbox hack of 2012. These systems are only as secure as the measures put in place by the companies running them. Even the world’s best and most funded tech teams cannot protect themselves: see Equifax, Verizon, Target, etc. The fact that each of these companies actually takes custody of customer data means that there is always risk of theft.
Decentralized storage will bring together the best features of blockchain technology, with attributes that meet the practical demands of storing high volumes of data. As the name suggests, decentralized storage works by distributing the data across a network of nodes, in a similar way to the distributed ledger technology characteristic of blockchain.
Right now, single system and even cloud-based databases are highly centralized, which makes them a beacon for hackers looking to attack. They also have obvious points of failure should a controlling company’s system be affected – for example, as a result of a power outage. In contrast, decentralized storage doesn’t encounter these problems because it utilizes geographically distributed nodes, either regionally or globally.
Any attack or outage at a single point will not have a devastating effect because other nodes in other locations will continue to function. The distributed nature of these nodes also offers the advantages of making decentralized storage highly scalable, as customers can easily access a marketplace of storage vendors, and high performing, as the power of the network provides better uptime.
While decentralized storage displays some of the key characteristics of the blockchain, it also requires us to rethink about how data is stored “on the blockchain.” As blockchain has become flooded with transactions, it too has had to seek out solutions to the problem of scalability. The concept of storing large amounts of data on the blockchain is simply not plausible.
What is Blockchain?
Blockchain is a technology that enables so-called “peer to peer” transactions. With this type of transaction, every participant in the network can transact directly with every other network participant without involving a third-party intermediary. Essentially, a blockchain is a digital contract permitting an individual party to conduct and bill a transaction (e.g. a sale of oil) directly (peer-to-peer) with another party. The peer-to-peer concept means that all transactions are stored on a network of computers consisting of the computers of the provider and customer participating in a transaction, as well as of the computers of many other network participants.
Traditional intermediaries are no longer required under this model, as the other participants in the network act as witnesses to each transaction carried out between a provider and a customer, and as such can afterwards also provide confirmation of the details of a transaction, because all relevant information is distributed to the network and stored locally on the computers of all participants.
Today the cloud plays a central role in storing, processing, and distributing data. Despite contributing to the rapid development of various applications, including the IoT, the current centralized storage architecture has led into a myriad of isolated data silos and is preventing the full potential of holistic data-driven analytics for IoT data. While blockchain is on the rise, it’s hardly the only technology that’s straining existing storage systems. Artificial Intelligence (AI), and particularly the Internet of Things (IoT), are also challenging the current boundaries of storage.
It’s estimated that there will be over 20 billion connected devices by 2020, all of which will generate and then require management, storage, and retrieval of enormous amounts of data. Connected devices, combined with consumer personalization apps and the increasing need to share data across business lines, are all playing their part in increasing demand for storage. Businesses wanting to launch new, data-driven applications face a mountain of time, effort and coordination to provision new databases today.
This drive towards a richer, more data-centric (and data-heavy) way of working is taking place against a global backdrop of major data breaches from centralized data centers. It’s a worrying combination: commercial dependency on data leading to extraordinarily large volumes of it being stored in vulnerable centralized databases, creating risk at a scale seldom seen before.
The advent of decentralized applications built on blockchain technology also creates new challenges, as they will exchange massive volumes of data that need to be stored and managed. Blockchains like Ethereum are not designed for data storage and management, and using them to do so would consume too much space and too much time.
How does distributed cloud storage work?
All users in distributed cloud storage are connected over a peer-to-peer network. This network is more secure, up to 10x faster, and 50 percent less expensive than the traditional datacenter-based cloud storage solutions. Thus, distributed cloud storage enables users to store data in a secure and decentralized manner. This is done by using blockchain features such as transaction ledgers, cryptographic hash functions, and public/private key encryption.
Data centers are the hub of cloud storage capabilities for cloud giants like AWS, Microsoft Azure, and Dropbox. But, these data centers come with a high price tag for cloud developers, providers, and users. Moreover, they come with an even higher cost of data failures and security breaches. From networking equipment and physical servers to other infrastructure demands like electricity, cloud service providers are spending billions of dollars every quarter just to maintain or grow their service offerings.
The blockchain is revolutionizing cloud storage by putting the user back in control over their data and devices. The decentralized aspect of blockchain means that there are no central servers to be compromised, and because of the use of client-side encryption, only the end users have complete access to their un-encrypted files and encryption keys.
CIOs want a Verifiable data supply chain
If you ask CIOs what they need to move their mission critical processes to the cloud then you will hear terms like “accountability, reliability, compliance, security, verifiability, auditability, acceptance of liability” etc. in other words they demand that there is a secure supply chain and that every step in that supply chain can be verified in real-time and when things go wrong it is possible to figure out what went wrong and that there is someone who can be held accountable.
Today not a single cloud vendor can say this. It also shows the opportunity; if such a platform could be built and the concerns of Enterprise CIOs could be satisfied then the entire global enterprise IT budget would be up for grabs.
Blockchain-based systems challenge AWS and Dropbox:
The emerging blockchain-based distributed storage market could challenge traditional cloud storage services, such as Amazon AWS and Dropbox, for a cut of the cloud storage market.
“Distributed compute and storage models are still in their infancy, but I do believe that there is an enormous market for this technology,” said Paul Brody, Ernst & Young’s (EY) Global Innovation Leader for Blockchain Technology.
Advantages of a Decentralized Cloud
So what are the main advantages of decentralized cloud service vs. private centralized cloud services offered by Amazon, IBM, Dropbox, etc.?
- More security & privacy. Decentralized data is also more difficult to attack than centralized data. On a decentralized network, files are broken apart and spread across multiple nodes (with sharding). The files are encrypted with a private key which makes it impossible for the node (participant in the network) to look at your file. Moreover, due to sharding, the files are just a fraction of their original self which render the reading of their content impossible.
- File loss prevention through redundancy in data (extra copies stored in case of error in storing or transmission of data).
- Cost reduction due to more efficiency. Blockchain storage costs can reduce the price of cloud computing between 50% -100%. (Storj VS Amazon S3 vs Microsoft Azure, via Storj Website)
- Increased speed due to servers proximity, scaling effects and fragmentation of data (smaller fragments can be saved and retrieved concurrently).
Blockchain Companies offering decentralised Storage Solutions
Sia also uses a blockchain-managed P2P network to store data on hosts with excess storage capacity; it replicates each file to 50 other hosts for redundancy and reliability. Sia is running more than 1,000 hosts in 50 countries, and while its beginnings may have been with consumer users, it’s setting its sights on the enterprise market, targeting CIOs and CTOs at enterprises, universities and SMBs for cold storage. It claims its service is 10 times more affordable, more secure and reliable than traditional cloud services. Sia is open-source, the company welcomes those to contribute to the project. The company has a burgeoning community of developers and fans; the company’s Slack channel has over 4,000 members, and its Twitter has over 9,000 followers.
Storj is a decentralized cloud storage protocol built on Ethereum. The project is interesting for a number of reasons, most notably its focus on user experience. Storj uses blockchain to track digital “farmers” who, similar to Bitcoin miners, have signed on to allow an application to share excess network and storage capacity on their computers or servers. The blockchain electronic distributed ledger is also used to pay farmers in cryptocurrency – digital tokens whose worth has grown 240 times since launching in 2014.
SAFE Network is one of the most ambitious projects in the space. It attempts not just to create a decentralized storage network but also to build an entirely new infrastructure layer for the internet that provides secure and decentralized storage, application hosting, website hosting, messaging, and more. Because SAFE Network is attempting to accomplish such a huge task, the project entails a high level of technical risk.
Swarm is a file storage protocol that is built into the Ethereum Web3 stack. Swarm functions much like IPFS/Filecoin but has native integration with Ethereum. Some high-level differences between the two protocols have been outlined by the Swarm team. For all but the most technical of individuals, these differences are largely irrelevant. The Filecoin/IPFS project is further along and will likely launch before Swarm. Swarm’s advantage is its inherent integration with Ethereum, which is building a large and loyal following of developers. Swarm provides the storage layer that integrates with Ethereum’s computation layer and Whisper’s secure messaging layer.
Filecoin, raised $52 million in a pre-initial coin offering (ICO) sale and $205 million in the actual ICO – an all-time record, and is building an end-to-end encrypted decentralized storage network and file hosting platform. The company hopes those with large amounts of unused storage will rent their space in exchange for its tokens.
Bluzelle is a data storage platform designed for the decentralized internet. The solution promises to provide high reliability, enterprise-grade scalability, and high performance speeds. Bluzelle enables users to rent out their computer storage space to earn a token while dApp developers pay with a token to have their data stored and managed in the most efficient way.
FortKnoxster is building an end-to-end encrypted inbox, chat, decentralized storage, calling, video conferencing, voice messages etc. – all within one web and mobile platform. In the system, all files and data is encrypted on all devices. FortKnoxster is a cyber-security company that specializes in the development of secure and encrypted communication solutions. As of now, the company developed a E-a-a-S (Encryption-as-a-Service) platform for the B2C market. The FortKnoxster communication platform has been uniquely designed and enabled various features. Users are fully able to carry out all their needs regarding communication and data-storage using the product.
Genaro Network is the first Turing-complete public chain with a decentralized storage network, providing blockchain developers a one-stop solution to deploy smart contracts and store data simultaneously. Genaro provides everyone with a trustworthy internet and a sharing community. As the creator of the blockchain 3.0 concept, Genaro aims to contribute to blockchain infrastructure technology development. Through the Genaro Hub and Accelerator, the company aim’s to foster thousands of DAPPS, to move applications from “Cloud” to “Blockchain” and thereby create a global blockchain ecosystem.
In less than 10 years, Blockchain has emerged from a small presence (Bitcoin) to one of the most talked about technological innovations. In the Financial Services industry, many companies using Blockchain have seen increased security, reduced cost, decreased transaction time, and increased transparency all while eliminating the need for a trusted third party. As the technology matures and the benefits are being realized, this technology is poised for rapid entry into the storage industry.
Decentralized storage projects could benefit individuals and businesses by protecting and securing their personal information at a much lower cost. Companies such as Siacoin and Storj can easily disrupt the space if they establish partnerships with the industry giants. If they manage to do so, Google and Apple could start using their services as back-end platform for storage in order to reduce costs and offer their customers more reliable and consistent uptime.
In the coming years, the entire cloud computing industry will be disrupted by blockchain technology. All parties (clients, industry leaders and blockchain companies) have the potential to benefit from this kind of industry-wide change. However, centralized cloud providers like Dropbox and Amazon, could suffer, see their margins decrease, and even become outdated, overpriced and obsolete if they do not find ways to integrate and evolve with the advances that blockchain technology is bringing to the table.
- Exploring the disruptive potential of decentralized storage and peer to peer transactions in the energy industry (pdf) PWC
- Towards Blockchain-based Auditable Storage and Sharing of IoT Data (pdf)
- MetaDisk A Blockchain-Based Decentralized File Storage Application (2014) (pdf)
- Filecoin: A Decentralized Storage Network (pdf)
- Storj: A Peer-to-Peer Cloud Storage Network (pdf)
- Meta-Key: A Secure Data-Sharing Protocol under Blockchain-Based Decentralised Storage Architecture (pdf)
- Sia: Simple Decentralized Storage (pdf)
- Decentralized Cloud Storage: Putting Data in the Cloud without Losing Control (pdf)
- Distributed resource sharing using the blockchain technology Ethereum (pdf)
- Distributed Storage Meets Secret Sharing on the Blockchain (pdf)
- Blockchain-based Database to Ensure Data Integrity in Cloud Computing Environments (pdf)