Decentralization Storage: The Evolution from Concept Hype to Practical Implementation

Storage used to be one of the hottest tracks in the blockchain industry. Filecoin, as the leading project of the last bull market, once had a market capitalization exceeding $10 billion. Arweave, with permanent storage as its selling point, reached a maximum market capitalization of $3.5 billion. However, as the availability of cold data storage came under scrutiny, the development prospects of Decentralization storage were overshadowed. Until the emergence of Walrus reignited attention, and the Shelby project launched by Aptos and Jump Crypto took hot data storage to new heights. This article will analyze the development trajectories of Filecoin, Arweave, Walrus, and Shelby to explore the evolution of Decentralization storage and attempt to answer the question: How much longer until Decentralization storage is truly widespread?

Filecoin: A Name for Storage, A Reality for Mining

Filecoin is one of the earlier blockchain projects that has emerged, and its development direction revolves around Decentralization. Filecoin combines storage with Decentralization, raising trust issues concerning centralized data storage service providers. However, the sacrifices made to achieve Decentralization have also become pain points that later projects like Arweave and Walrus attempt to address. To understand why Filecoin is essentially a mining coin, it is necessary to understand the objective limitations of its underlying technology, IPFS, in handling hot data.

IPFS: Decentralization architecture transmission bottleneck

IPFS(, the InterPlanetary File System, was introduced around 2015, aiming to revolutionize the traditional HTTP protocol through content addressing. However, the biggest drawback of IPFS is its extremely slow retrieval speed. In today's world where traditional data services can achieve millisecond-level responses, retrieving a file via IPFS still takes over ten seconds, which severely limits its practical application and promotion.

The underlying P2P protocol of IPFS is primarily suitable for "cold data", which refers to static content that does not change often, such as videos, images, and documents. However, when it comes to handling dynamic web pages, online games, or artificial intelligence applications, the P2P protocol does not have a significant advantage over traditional CDNs for "hot data".

Although IPFS itself is not a blockchain, its directed acyclic graph )DAG( design is highly compatible with many public chains and Web3 protocols, making it a natural fit as a foundational building framework for blockchains. Therefore, even lacking practical value, it is sufficient as a foundational framework to carry blockchain narratives. Early projects only needed a runnable framework to set their grand blueprint in motion, but as Filecoin developed, the limitations brought by IPFS began to hinder its progress.

) Logic of mining coins under the storage coat

IPFS was initially envisioned for users to become part of the storage network while storing data. However, in the absence of economic incentives, it is difficult for users to actively participate in this system, let alone become active storage nodes. This means that most people will only upload files to IPFS without contributing their own storage space or storing files for others. It is against this backdrop that Filecoin emerged.

The token economic model of Filecoin mainly includes three roles: users pay fees to store data; storage miners receive token rewards for storing user data; retrieval miners provide data when users need it and receive rewards.

This model has potential cheating space. Storage miners may fill garbage data after providing storage space to gain rewards. Since this garbage data will not be retrieved, losing it will not trigger a penalty mechanism. This allows storage miners to delete garbage data and repeat this process. Filecoin's replication proof consensus can only ensure that user data has not been privately deleted, but it cannot prevent miners from filling garbage data.

The operation of Filecoin largely relies on miners' continuous investment in the token economy, rather than on the actual demand for distributed storage from end users. Although the project is still iterating, at this stage, the ecological construction of Filecoin aligns more with the "mining token logic" rather than the definition of "application-driven" storage projects.

Arweave: The Gains and Losses of Long-Termism

If Filecoin's goal is to build an incentivized and verifiable decentralized "data cloud" shell, then Arweave is moving in the opposite direction in the storage field: providing permanent storage capabilities for data. Arweave does not attempt to create a distributed computing platform; its entire system revolves around a core assumption - important data should be stored once and preserved forever in the network. This extreme long-termism makes Arweave fundamentally different from Filecoin in terms of mechanism, incentive models, hardware requirements, and narrative perspectives.

Arweave focuses on Bitcoin as a learning object, attempting to continuously optimize its permanent storage network over long periods measured in years. Arweave is not concerned with marketing, nor does it care about competitors or market trends. It simply focuses on iterating the network architecture, indifferent to whether anyone is paying attention, because this is the essence of the Arweave development team: long-termism. Thanks to long-termism, Arweave was popular during the last bull market; and because of long-termism, even if it falls into a trough, Arweave may still survive several rounds of bull and bear markets. However, does decentralized storage still have a place for Arweave in the future? The existence value of permanent storage can only be proven by time.

From version 1.5 to the recent version 2.9, although Arweave has lost market attention, it has been committed to enabling a broader range of miners to participate in the network at minimal cost and incentivizing miners to maximize data storage, continuously enhancing the overall robustness of the network. Arweave is well aware that it does not align with market preferences, thus adopting a conservative approach, not embracing the miner community, and the ecological development has completely stagnated, upgrading the mainnet at minimal cost while continuously lowering the hardware threshold without compromising network security.

1.5-2.9 Upgrade Path Review

The Arweave version 1.5 exposed a vulnerability that allowed miners to rely on GPU stacking instead of actual storage to optimize block production chances. To curb this trend, version 1.7 introduced the RandomX algorithm, restricting the use of specialized computing power and requiring general-purpose CPUs to participate in mining, thereby weakening the centralization of computing power.

Version 2.0 adopts SPoA, converting data proof into a concise path of Merkle tree structure, and introduces format 2 transactions to reduce synchronization burden. This architecture alleviates network bandwidth pressure and significantly enhances node collaboration capabilities. However, some miners can still evade the responsibility of holding real data through centralized high-speed storage pool strategies.

To correct this deviation, version 2.4 launched the SPoRA mechanism, introducing global indexing and slow hash random access, requiring miners to genuinely hold data blocks to participate in effective block production, thereby weakening the effect of hash power stacking. As a result, miners began to focus on storage access speeds, driving the application of SSDs and high-speed read-write devices. Version 2.6 introduced hash chain control of block production rhythm, balancing the marginal benefits of high-performance devices and providing fair participation space for small and medium miners.

Subsequent versions further strengthen network collaboration capabilities and storage diversity: 2.7 adds collaborative mining and pool mechanisms to enhance the competitiveness of small miners; 2.8 introduces a composite packaging mechanism that allows large-capacity low-speed devices to participate flexibly; 2.9 introduces a new packaging process in replica_2_9 format, significantly improving efficiency and reducing computational dependencies, completing the closed loop of data-driven mining models.

Overall, Arweave's upgrade path clearly demonstrates its storage-oriented long-term strategy: while continuously resisting the trend of computing power centralization, it steadily lowers the participation threshold to ensure the possibility of the protocol's long-term operation.

Walrus: Hype or Innovation in Hot Data Storage?

The design philosophy of Walrus is completely different from that of Filecoin and Arweave. Filecoin's starting point is to create a decentralized and verifiable storage system, at the cost of cold data storage; Arweave's goal is to build an on-chain library of Alexandria that can permanently store data, at the cost of limited application scenarios; while the core of Walrus is to optimize the cost efficiency of hot data storage.

Magic Modified Error-Correcting Code: Cost Innovation or Old Wine in New Bottles?

In terms of storage cost design, Walrus believes that the storage expenses of Filecoin and Arweave are unreasonable. The latter two adopt a fully replicated architecture, with the main advantage being that each node holds a complete copy, which provides strong fault tolerance and independence among nodes. This architecture ensures that even if some nodes go offline, the network still maintains data availability. However, this also means that the system requires multiple copies for redundancy to maintain robustness, thus driving up storage costs. Especially in Arweave's design, the consensus mechanism itself encourages node redundancy in storage to enhance data security. In contrast, Filecoin is more flexible in cost control, but at the expense of potentially higher data loss risks for some low-cost storage. Walrus attempts to find a balance between the two, with its mechanism enhancing availability through structured redundancy while controlling replication costs, thus establishing a new compromise path between data availability and cost efficiency.

The Redstuff created by Walrus is a key technology for reducing node redundancy, derived from Reed-Solomon ### RS ( coding. RS coding is a traditional erasure code algorithm that can double the dataset by adding redundant fragments for reconstructing the original data. From CD-ROMs to satellite communications to QR codes, it is widely used in everyday life.

Erasure codes allow users to take a data block ) like 1MB (, and then "amplify" it to 2MB, where the additional 1MB is special erasure code data. If any byte in the block is lost, users can easily recover those bytes through the code. Even up to 1MB of data loss, the entire block can still be recovered. The same technology allows computers to read all the data from damaged CD-ROMs.

Currently, the most commonly used is RS coding. The implementation method starts with k information blocks, constructs the related polynomial, and evaluates it at different x coordinates to obtain the encoded blocks. Using RS erasure codes, the probability of randomly sampling lost large blocks of data is very small.

For example: Divide a file into 6 data blocks and 4 parity blocks, totaling 10 pieces. As long as any 6 pieces are retained, the original data can be fully restored.

Advantages: Strong fault tolerance, widely used in CD/DVD, fault-tolerant hard disk arrays ) RAID (, and cloud storage systems ) such as Azure Storage, Facebook F4(.

Disadvantages: decoding calculations are complex and the overhead is high; it is not suitable for scenarios with frequent data changes. Therefore, it is usually used for data recovery and scheduling in off-chain centralized environments.

Under the decentralized architecture, Storj and Sia have adjusted traditional RS coding to meet the practical needs of distributed networks. Walrus has also proposed its own variant - the RedStuff coding algorithm, to achieve lower cost and more flexible redundancy storage mechanisms.

What is the biggest feature of Redstuff? By improving the erasure coding algorithm, Walrus can quickly and robustly encode unstructured data blocks into smaller shards, which are distributed and stored in a storage node network. Even if up to two-thirds of the shards are lost, the original data block can be quickly reconstructed using partial shards. This becomes possible while maintaining a replication factor of only 4 to 5 times.

Therefore, it is reasonable to define Walrus as a lightweight redundancy and recovery protocol redesigned around the Decentralization scenario. Compared to traditional erasure codes ) such as Reed-Solomon (, RedStuff no longer pursues strict mathematical consistency, but instead makes realistic trade-offs regarding data distribution, storage verification, and computational costs. This model abandons the immediate decoding mechanism required by centralized scheduling, opting instead to verify whether nodes hold specific data copies through on-chain Proof, thus adapting to a more dynamic and marginalized network structure.

The core design of RedStuff is to split data into two categories: primary slices and secondary slices. Primary slices are used to restore the original data, and their generation and distribution are subject to strict constraints, with a recovery threshold of f+1 and requiring 2f+1 signatures as availability endorsement. Secondary slices are generated through simple operations such as XOR combinations, serving to provide elastic fault tolerance and enhance the overall robustness of the system. This structure essentially reduces the requirement for data consistency - allowing different nodes to temporarily store different versions of data, emphasizing a practical path of 'eventual consistency'. Although similar to the relaxed requirements for retroactive blocks in systems like Arweave, it reduces the network's