What are the bottlenecks and thresholds for DApp development? dfuse gave his own thinking and improvement

The Ethereum DApp development environment is not yet mature, which limits the further development of DApps. Blockchain API company dfuse analyzed the current development thresholds and limitations of DApps, and proposed solutions and optimization ideas for dfuse.

Original title: "What are the main problems of developing dapp on Ethereum"

 Author: dfuse

Decentralized applications (dapp) are widely considered to bring disruptive innovations to fields such as banking (DeFi) and gaming. However, even the most innovative solutions will not be recognized if they fail to meet consumer expectations.

Consumers need a smooth and mature user experience, and achieving this goal is another major challenge for Ethereum dapp developers.

This article will outline a typical dapp architecture and point out some inherent limitations of today's standard Ethereum stack. It is these limitations that make it difficult for developers to create a convincing user experience.

Next, we will introduce some innovations in the Ethereum infrastructure field that can help developers overcome these challenges, such as dfuse. dfuse is proud to provide blockchain developers with a better development experience and accelerate the mainstream adoption of Ethereum applications.

Classic Ethereum dapp architecture Generally speaking, a dapp on Ethereum contains three main parts (experienced Ethereum developers can skip to the next section):

* Smart contracts, usually written in Solidity, are built using frameworks such as Truffle Suite and deployed on the Ethereum blockchain.

* Front-end code, written in Java.

* Back-end-generally use standard Ethereum blockchain nodes. The communication between the front end and the back end generally uses the JSON-RPC or GraphQL API provided by the node. There are also various libraries that facilitate communication between the front end and Eth nodes, the most popular of which are web3.js and ethers.js. There are also web3 libraries in many other languages ​​(Java, Python, Rust...).

Self-built back-end nodes In the early days of Ethereum, developers had to operate their own Ethereum nodes. After the dapp is released, they must also operate production-level nodes (or node clusters). The task of operating a blockchain node is heavy and will also burden the efficiency of developers.

The above-mentioned challenge of node service (NaaS) providers has led to the rise of "node service" platforms such as Infura, and relatively new companies such as Nodesmith, Quiknode, Blockdaemon, Ethernode, Chainstack, Alchemy, CloudFlare, etc.

These platforms provide developers with cloud-based Ethereum nodes, which saves developers the effort of operating nodes. Solutions for development and production. These platforms can share the system management of the basic operating system and node software itself for developers, such as patches and updates.

The inherent limitations of the Ethereum node Even if the node service can successfully replace the developer as a system administrator, it cannot help the developer achieve the user experience to build a better dapp. This is because of the architecture of the node service and the Ethereum The inherent limitations of the JSON-RPC and GraphQL interfaces supported by the node.

The main limitations include:

The observed state information is inconsistent. In order to expand beyond the capacity of a single node and provide higher reliability, the nodes as the service platform provide access to the node pool through the load balancer.

Since these nodes all operate autonomously as peer nodes in the Ethereum network, when information is spreading through the network at a certain moment, different nodes may be at different block heights, or even at different forks. . This means that the dapp may receive inconsistent information about the state of the blockchain, because the results obtained by its request are provided by different nodes behind the load balancer.

The node service platform usually tries to solve this problem through session stickiness on the load balancer. It always tries to send the specified front-end query to the same back-end node, but this method will fail in many cases:

* When the request generated by the front-end is more than the processing capacity of a single back-end node;

* When a network problem causes the front end and the back end to disconnect, they must be reconnected;

* Multiple node service platforms will route different types of front-end requests (for example, sending transactions or search chain history) to different back-end node groups optimized for the query type. Then, because the front-end often visits multiple back-end nodes, and the blockchain state obtained by these back-end nodes is inconsistent with each other, it is difficult for dapps to handle chain reorganization. When tracing back to the chain history, the dapp may suddenly find that the parent block it is looking for does not exist (because it is now interacting with another node on a different fork). Then dapp developers have to write code to solve this problem (the method is usually by repeatedly reconnecting until it finds a node). This adds unnecessary complexity to the dapp and may cause discrepancies in the information presented to the user.

Searching for information on the blockchain is slow and has limitations. The ability of dapps to search for transactions or history on the chain is limited, because standard Ethereum nodes are not suitable for supporting precise searches or performing screening monitoring of real-time data. To operate in a high-performance manner, we need to do a large number of indexes on millions of blocks and transactions, but:

* Ethereum nodes only index certain fields in the log issued by transaction execution (fields to be indexed must be marked by the developer when deploying the contract)

* Ethereum nodes do not index the data of internal transactions (occurring when a smart contract calls a method of another contract)

* Developers are unwilling to add additional index fields, because the cost of each transaction for each additional index field will increase relatively, which will bring additional costs to contract users

* Ethereum nodes use Bloom filters to perform searches, so it is always a fuzzy search and will produce false positive matches. Exact matching requires additional processing by the front-end. The front-end must obtain the entire block or transaction of fuzzy matching and retrieve it again to find an exact matching result. This not only requires the effort of the developer, but also wastes the bandwidth between the front end and the node

* The available search syntax is very limited-only basic selection and simple substitution are supported

* The speed of obtaining search results is very slow-it may take several hours to perform searches in a large range of blocks

* JSON-RPC is a waste of bandwidth-the returned data far exceeds what you really need. GraphQL interface uses less bandwidth, but does not provide streaming transmission function (the front end must be polled for updates). Lack of atomicity. In most modern environments, such as relational databases, transactions are generally atomic operations, but in Ethereum (or other Blockchain) is not. Each transaction undergoes a series of state transitions, and various problems or failures may be encountered in the process. The dapp must call multiple APIs to query many different data sources (blocks, mempool, network status) in order to track the life cycle of the transaction until it is completed.

Again, this burden falls on the front-end code, repeating polling to figure out what is happening, and users of dapp will experience delays and need to refresh because the dapp performs all these extra work.

Nodes are passive. Ethereum nodes are passive, which means they cannot generate events or callbacks and call Webhooks. All operations must be initiated by the front end, and the front end must also poll the nodes to obtain updated information. The event stream reading function of the Ethereum node is too limited to meet the needs of most dapps, and is only available in the JSON-RPC interface, not on the GraphQL interface (see here).

Rethinking the dapp infrastructure through dfuse dfuse provides a higher-level blockchain API platform. Compared with the native API provided by blockchain nodes, they can do more work more easily. dfuse is designed to give dapp developers the functions they need to build modern blockchain applications through a fast and smooth interface, thereby providing an excellent user experience.

dfuse aims to solve all the above limitations and break the limitations of traditional Ethereum nodes.

Consistent view dfuse is an integrated hyperscale data platform, not a collection of multiple Ethereum nodes on a load balancer. The dfuse platform provides chain state information on all connections and at all points in time. The dfuse platform either sees a block (detecting chain forks and reorganizations at the same time) or does not report the block at all (when the block undergoes rapid reorganization and is not spread far).

In this way, the dapp will never face an inconsistent view of the chain state, and can focus on its main function instead of busy verifying the details of the blockchain.

The high-speed, fine-grained search dfuse enables dapp developers to search the history of the blockchain with extremely fine granularity, extraordinary speed and efficiency, and real-time filtering and streaming through GraphQL, gRPC and Websocket interfaces Read.

* dfuse fully indexes all Log fields-all data sent in the Log for each transaction is directly applicable to high-precision search.

* dfuse fully indexes all internal transactions (sender, receiver, value, method, input parameters), thus fully tracking the operation of the contract in the entire call tree structure

* Indexing will not bring any additional gas costs to your users-dfuse indexing is an integrated function of the dfuse platform and will not increase the resource cost of contract execution

* The search found the exact match result, not the fuzzy search result. No need to write additional front-end code to repeatedly check search results, and no need to waste bandwidth to obtain unwanted data in batches

* dfuse provides a structured query language, similar to the query language of Kibana or GitHub, with complete boolean operations and the ability to directly drill into the specific transaction or command you are looking for

* dfuse provides excellent performance-it can search the entire chain history according to the specified search criteria in less than a second to find a set of exactly matching items

* dfuse can provide a simple response through GraphQL without sacrificing the streaming function, and has the best of both worlds-our GraphQL interface provides a complete real-time filter search, which can effectively provide users with dynamic updates

* Regardless of the traffic on the Ethereum network, the performance of dfuse is consistent with atomic operations. dfuse provides a streaming read endpoint that understands all the complex states that the transaction may enter and informs you when it meets finality . There is no need to laboriously repeat polling or check multiple data sources to track the status of the transaction. You only need to push the transaction up and stay connected to receive real-time status updates, which can also provide your users with the real-time status of the transaction.

The active backend dfuse platform provides you with an active backend that can initiate events. For example, dfuse can call the lambda function (or cloud function) of your choice according to the precise criteria you specify (through the above search and other functions). This allows dapp to implement an asynchronous architecture, and data updates can be smoothly and real-time released to users through multiple communication channels.