Deep Analysis of VM: What virtual machines are used by Ethereum and EOS?
What is a virtual machine? Why is it so important in the world of blockchain?
Hashgard Labs, a research institute of Hashgard, and BKFUND Research Institute jointly conducted an in-depth analysis of virtual machines and wrote a research report about it.
(To read the first report, please refer to Weibo account @Hashgard)
This time, we once again interpreted the virtual machine and selected four representative projects to conduct research and analysis on their virtual machines.
1. EVM
A representative project of Blockchain 2.0 — Ethereum (Etherum’s Virtual Machine) Ethereum virtual machine builds a stack-based virtual running environment, defines a set of environments that are isolated from the node’s own environment, shields the underlying differences of each node, and achieves the same result (determination) of different nodes executing contracts. It is a 256-bit virtual machine with a “quasi-turing complete”.
Another feature of the EVM as a virtual machine is that, because it provides a lower-level common infrastructure, based on this virtual machine, you can design a high-level programming language that conforms to the virtual machine or blockchain system, such as Ethereum. Solidity language of Ethereum, Nxt Javascript and APIs in the corresponding environment. Correspondingly, if the language design is unreasonable, it will bring corresponding defects.
Based on the various characteristics, the virtual machine mode is still a relatively safe technical route in the smart contract technology, and is currently adopted by most blockchain systems including Ethereum.
EVM shortcomings
However, in terms of specific implementation, EVM currently generally believes that there are some design shortcomings, including the following:
1) 256-bit word length. When using the lengths of 8, 16, 32, 64 supported by the current mainstream CPU, the CPU can directly provide native support, and the program efficiency is high. However, the EVM uses a word length of 256 bits, and the processor requires additional operations to process normally, so the computational efficiency is low. At the same time, compared with the mainstream word length, 256 bits will also have a certain storage waste, which will affect the consumption of gas. At present, some tests show that when the word length is changed to 64-bit or 128-bit, the running efficiency of the contract code and the storage cost (indicated by the gas value) are improved.
2) Floating point numbers are not supported and standard libraries are missing. This issue does not support floating point. And as mentioned above, the Solidity language, which is often used in conjunction with EVM, is also more prominent because of the lack of a native standard library. The emergence of OpenZeppelin has eased this problem. The writing framework of ERC20 and ERC721 makes development easy. But there are always risks associated with relying on third-party code bases. For example, there are many ERC20 Certified Contracts that use OpenZeppelin some time ago because of the return value of the function, which can lead to unpredictable transfer results.
3) The contract code cannot be modified and upgraded. Since the EVM does not use the standard von Neumann structure, the program code is stored in a separate virtual ROM, rather than the location of the code area in the general computer memory. In theory, the contract can be upgraded by redeploying the contract. Therefore, when there are bugs and the like in the contract, it cannot be remedied through patching.
EVM is not Turing complete
The Turing machine is a mathematical model proposed by Aaron Turing in 1936. The Turing machine consists of an infinitely long strip of paper, a character table, a read/write head, a status register, and a finite set of instructions. At the beginning of the operation, the Turing machine read/write head starts from a certain position according to the configuration at the moment (current position and current grid content, etc.), and the operation is performed step by step with the instruction set until the state becomes stopped, and the operation ends. In the field of computing, all the problems we study are computational problems, and with Turing completeness, any computational problem can be solved. The essence of a programming language or virtual machine is a Turing machine. A programming language or virtual machine is Turing-complete. It means that it can do all the things that Turing can do, that is, solving all the computational problems.
In the design of the Ethereum virtual machine, since the calculation of the instruction is constrained by gas, this limits the number of calculations that can be completed. This is also a common cause of Turing incompleteness, because the bounded, recursive, or computational bounds cause the program to terminate, so the programs that can run on the EVM are subject to many restrictions, and the EVM is not Turing-complete.
Irrational design of EVM
1. Smart contract design
Lack of standard library support: EVM lacks complete standard library support, and even the most basic string type support. It is very hard in EVM. For example, string splicing, cutting, searching, all need to be implemented by developers themselves. The consequence is that developers need to pay attention to more fragmented details of their own business, instead of focusing on their own business development. At the same time, the self-implemented class library may be too high in time and space complexity, consume a lot of unnecessary gas, or the developer borrows relevant class library code from the open source project, but it will introduce more security issues, aggravating The complexity of contract code auditing is not worth it.
Difficult to debug and test: EVM is difficult to debug and test. In addition to throwing OutOfGas exceptions, EVM does not return any information to the developer. It is impossible to print logs, make breakpoints, and debug single step. Although the event mechanism can partially improve the problem, design of the event mechanism itself determines that it is not an elegant and easy debugging tool.
Floating point numbers are not supported: EVM does not support floating point numbers. Ethereum uses Wei as the smallest unit. It only has integers and does not support other granularity measurement. This design avoids the precision problem caused by the introduction of floating point numbers, but in actual development, developers In order to represent an eth variable, there will be a lot of zeros after the variable, which makes the code maintenance extremely complicated. At the same time, it is undeniable that floating-point numbers still have great value in certain scenarios, and they cannot be abandoned directly.
Contracts cannot be upgraded: EVM does not support contract upgrades. Contract upgrades are a strong requirement in smart contract development. They are also issues that every contract developer must consider. Contract upgrades can implement security patches for existing contracts and extend the features of existing contracts. EVM does not support upgrades at all, and developers can only solve this problem by issuing new contracts, which is time consuming and laborious.
2. Smart contract security
Overflow attack, EVM’s safeMath library is not used by default. For example, when the developer calculates the solidity uint256, if the final result is greater than the maximum value of uint256, the overflow will become a small number, thus creating an overflow vulnerability. . Related tokens such as BEC and SMT have suffered from overflow attacks, which have brought extremely serious consequences. The BEC overflow vulnerability is as follows:
Re-entry attacks, a major feature of solidity is that you can call other external contracts, but when you send eth to an external address or call an external contract, you need to submit an external call. If the external address is a malicious contract, the attacker can add malicious code to the Fallback function. When the transfer occurs, the Fallback function is called to execute the malicious code. The malicious code executes the vulnerable function of the calling contract, causing the transfer to be resubmitted. The most serious re-entry attacks occurred in the early days of Ethereum, known as the DAO vulnerability. The following contract fragment specifically addresses the reentry attack:
Unexpected function execution, EVM does not strictly check function calls, and if the contract address is controllable as an incoming parameter, it may cause unexpected behavior. Details are as follows:
EVM has many problems at the design and security level. Although the EVM team has developed a new contract development language such as Vyper, it has been in the experimental stage and cannot be actually used.
EVM+WebAssembly：
The Ethereum virtual machine based on WebAssembly does not have much improvement in mathematical calculations. Therefore, if only the operation of transferring funds such as ERC20 is implemented, it may be far better than other technical transformations, such as consensus mechanism and fragmentation. More obvious
However, when implementing more complex contract logic such as sorting, WebAssembly will show a strong performance advantage, which can provide better technical support for blockchain business applications in complex scenarios.
EVM evolution
Because of the above shortcomings, after ETF EVM, many blockchain projects implement virtual machines from a variety of technical directions. We can further divide into the following categories according to the characteristics of the technical implementation. It should be noted that at present, many public chains only publish their development plans, and there is no actual virtual machine program code that can be run. The actual implementation may change with the progress of the project.
1. Improve EVM
Despite the above shortcomings of EVM, many blockchain systems are still designed and implemented using EVM improvements. In this way, the function of the original Ethereum can be reused to reduce the development workload; on the other hand, the existing Ethereum ecology can be utilized for rapid development.
Although some public-chain platforms such as EOS have surpassed the Ethereum in terms of user activity, the long-term ecological environment of Ethereum is an important factor for a new-born public chain to consider in future development and operation. factor. Like the Hyperledger, which is the main alliance chain platform, the Burrow project is built with the Ethereum smart contract blockchain node with authorization management, and is compatible with the Ether Equity Solidity development ecosystem with Cosmos’s Tendermint consensus engine.
In this environment, many public chains are designed to implement virtual machines, and they are implemented in an improved way based on EVM.
For example, the CKB of Layer 1 in the Nervos public-chain platform uses RISC-V as the instruction set to implement CKB-EVM. Through the RISC-V toolchain, CKB-EVM can support any programming language for smart contract development.
Aion’s FastVM is designed with a 128-bit word length and uses the LLVM JIT as the running engine at the bottom to improve storage efficiency and speed. Aion’s virtual machines also plan to support multiple word lengths from 128 to 64 and 32 in the future to accommodate more application scenarios and business needs.
It can also be optimized from the running process of the virtual machine. Since Ethereum requires that each full node run the same virtual machine calculation and check the results before updating the ledger, this approach obviously affects performance. Some blockchain platforms have improved this. For example, Zilliqa uses data flow graphs to calculate slices to improve concurrency.
2. Compatible with traditional instruction sets
Some blockchain systems implement the contract engine idea to be compatible with traditional machine instruction set techniques.
One of these is to use traditional virtual machine technology directly as a contract engine, but some modifications are needed to technically solve deterministic problems. For example, R3 Corda uses the JVM as its smart contract execution environment, ie the transaction types in Corda are defined and executed using the JVM’s bytecode. However, since there are still some non-deterministic operations in the traditional JVM, such as random numbers, garbage collection in Java, etc., Corda is currently trying to develop a deterministic JVM sandbox environment (DJVM for short). To detect if the code developed by the user is non-deterministic.
The idea of this approach is that the JVM has some advantages over the EVM implementation of the smart contract engine:
1) The JVM has undergone large-scale, long-term industrial inspections. As a bottom layer, it is more secure and robust than EVM.
2) Using the Java language and the existing development ecosystem, developers can get started faster
3) Some language features, including support for floating point numbers, will also have advantages over Solidity.
The disadvantage is that the JVM has many functions that are independent of the blockchain requirements, and it is not easy to increase the gas mechanism in the pass-through economy to achieve termination.
In addition to traditional virtual machine technology, some use emulators to simulate the traditional instruction runtime environment, support multi-language development and allow inventory code to migrate to the blockchain development environment as low-cost as possible. For example, Qtum’s x86 virtual machine is designed to be compatible with the traditional x86 server instruction set, which allows the industry to accumulate a large amount of code for a long time in a smart contract-based blockchain execution environment. Provide multi-language support.
3. wasm method
WebAssembly technology is increasingly being used to implement smart contract engines.
WebAssembly is a cross-platform binary instruction format that can be used in stack-based virtual machines. It is hoped that both client and server programs can be deployed on the web. Due to its excellent performance and wide compatibility with multi-language support, it has been promoted in web development and supported by modern browsers such as Chrome, Edge, and Firefox, and is considered to be likely to replace existing JavaScript. Many large applications are also trying to use WebAssembly, such as Windows 2000 running in a browser. Many people regard 2018 as the first year of WebAssembly technology.
In the blockchain field, there are also some platforms that have begun to use WebAssembly technology, which is also a popular trend in the recent smart contract implementation technology. For example, EOSIO has adopted WebAssembly to implement its virtual machine WAVM.
Compared with the EVM of Ethereum, the virtual machine using WebAssembly technology has the following advantages:
1) Can support C, C++, Rust and other multi-language development, reducing the entry barrier and learning cost of developers
2) The result of compiling into wasm is transparent to the CPU architecture. Although it cannot run on the CPU, it can be deployed and run in mainstream modern browsers.
3) WebAssembly because the build method is as close as possible to the machine code, the compilation volume is small, the loading speed is fast, and the performance is relatively high.
However, WebAssembly also has some shortcomings to be improved. For example, as an underlying technology, WebAssembly is not mature enough. For example, there are some compatibility issues between new and old browsers; various aspects of ecological tools have yet to be developed.
However, Ethereum is already considering implementing EVM in WebAssembly, ie eWASM. In the current Parity Ethereum node using Rust, the WebAssembly method can be used to implement its virtual machine.
2. NEO VM
Neo smart contract types include: verification contracts, function contracts, and application contracts.
Smart contract trigger
NEO’s smart contracts have two triggers:
1. Authentication of the contract user: The smart contract acts as a contract account and triggers the smart contract when the user uses an asset in the contract account.
2. Manually send a transaction to call a smart contract: The user sends a transaction (Invocation Transaction) to trigger the execution of a smart contract.
A contract can be triggered by both of the above methods. Since the contract triggered by the authentication is the authentication process of the UTXO model, it is executed before the transaction is written to the block. If the contract returns false or an exception occurs, the transaction will not be written to the block.
The contract triggered by the transaction call is called when the transaction is written to the block. At this time, regardless of the application contract return and whether it fails, the transaction has already occurred and cannot affect the UTXO asset status in the transaction.
The figure above is the system architecture diagram of NEO virtual machine (NeoVM). This picture describes the working principle of smart contract. This picture includes three parts:
1. Smart Contract Compiler (Compiler)
2. Virtual Machine Execution and Calculation Engine (AVM)
3. Interoperable layer of the virtual machine (interop Service)
The deployment in the dotted box is the core of the virtual machine. NeoVM consists of three important components: the execution engine, the stack, and the interoperability services.
Execution engine
The green on the left is the virtual machine execution engine (equivalent to the CPU), which can execute common instructions, such as flow control, stack operations, bit operations, arithmetic operations, logic operations, cryptography methods, etc., and can also be called through system calls. The interoperable service layer interacts.
Computation stack
The middle gray part is the virtual machine’s computing stack (equivalent to memory). Today, there are two implementations of virtual machines, stack-based and register-based. These two implementations have their own advantages and disadvantages, and they all have iconic features. product. Stack-based virtual machines, including JVM, CPython, and .Net CLR. Register-based, there are Dalvik and Lua5.0. A stack-based virtual machine has a concept of a compute stack, and the virtual machine interacts directly with the Evaluation Stack when performing real operations.
Since the execution defaults to fetching data from the operand stack, there is no need to specify an operand. For example, x86 assembles “ADD EAX, EBX”, you need to specify where the operation needs to take the operand, and where the result is stored. However, the instructions of the stack-based virtual machine need not be specified. For example, the addition operation is a simple “Add”, because the default operand is stored on the operand stack, and two data are popped directly from the operand stack. Addition, the result of the operation is stored at the top of the stack by default.
Interoperable service layer
The blue part on the right is the virtual machine’s interoperable service layer (equivalent to peripherals). Currently, the interoperable service layer provides APIs for smart contractes to access blockchain data. These APIs provide access to block information, transaction information, contract information, asset information, and more.
In addition to this, the interoperable service layer provides a persistent storage area for each contract. Each smart contract of NEO is optionally enabled with a private storage area. The storage area is in the form of key-value. The NEO smart contract is determined by the callee of the contract to persist the context of the storage area instead of the caller. To decide. Of course, after the caller needs to pass his storage context to the callee (ie, complete the authorization), the callee can perform read and write operations.
Charging mode
NEO Smart Contracts are subject to a certain fee for deployment or execution, divided into deployment and execution costs.
The deployment cost means that the developer will pay a certain fee to the blockchain system when deploying a smart contract to the blockchain. According to the required functions of the contract, the system will charge 100~1000 GAS and use it as system revenue.
Execution costs mean that each execution of a smart contract will pay a certain execution fee to the NEO system.
3. EOS VM
The EOS VM adopts the WASM (WebAssembly)-LLVM (Low Level Virtual Machine) architecture. The bottom layer of the EOS virtual machine is composed of LLVM and WebAssembly. They are all ready-made, LLVM has long been used for object-C, and webAssembly is used. The implementation of the front end of the web page. Wasm is currently mainly used in web applications, and EOS uses it as the final format of smart contracts. The mechanism is not the same as the current browser running and calling.
EOS Smart Contract Summary:
The smart contract acts as an application registered in the EOS blockchain and runs on the EOS node. Smart contracts define related interfaces that contain actions, data structures, and related parameters. The smart contract implements these interfaces, is compiled into a binary file of the wasm suffix, and executed. For the blockchain, the final store is a smart contract transaction.
A smart contract consists of an action and a type definition: Action: defines and implements the behavior and functionality of a smart contract. Type definition: defines the content and data structure required by the contract.
The client sends a message to trigger the execution of the action, and the final storage form of the smart contract is transaction. A transaction can contain multiple actions, including at least one action.
An example of a smart contract definition:
#define EOSIO_API( TYPE, MEMBERS ) \ BOOST_PP_SEQ_FOR_EACH( EOSIO_API_CALL, TYPE, MEMBERS )
Expand the included macros:
#define EOSIO_API_CALL( r, OP, elem ) \ case ::eosio::string_to_name( BOOST_PP_STRINGIZE(elem) ): \ eosio::execute_action( &thiscontract, &OP::elem ); \ break;
This completes the implementation of the call from the contract to the underlying route.
Contract compilation process and virtual machine:
The intelligent contract compilation process is as follows:
1. Use clang to target wasm32 and generate intermediate file bc
2. Generate a link.bc file by using the LLVM-Link link to generate a bc file and a standard library bc file.
3. Use LLVM’s llc to generate s assembly files assembly.s
4. Apply the eosio-s2wasm tool to convert the s file to a wast file
5. Apply the eosio-wast2wasm tool to convert the wast file to the final wast file
Why not use clan instead of wasm, I think it may be due to the fact that wasm represents the future trend, and there are more explanation containers for wasm support in the future, which is convenient for accessing multiple virtual machines.
WASM virtual machine:
In EOS, modules such as WASM that use existing functions are centralized in the library directory, which can be compiled separately and then inherited.
The WASM virtual machine is mainly located under the wasm-jit under the libraries.
1. libraries/chain: mainly defines virtual machine related interfaces
2. libraries/wasm-jit: Mainly the implementation of smart contracts. Also in the contracts directory, there is ABI’s auxiliary code.
The source code of the wasm-jit virtual machine is: ~/eos/libraries/wasm-jit/Source, the main folder: Emscripten, IR, Runtime, Programs, WASM, WAST
1. IR: IR is the first step of high-level language to machine language, followed by WAST parsing, WASM serialization processing, — — based on Runtime processing.
2. Emscripten: a tool for converting high-level languages to LLVM bytecode
3. Runtime execution engine: includes execution bytecode (machine code), external call, translate bytecode to IR, generate machine code, etc.
4. WASM: WASM 的 LICENSE
5. WAST: Parse parsing function modules Other:
6. Logging: Log Management
7. Platforms: Corresponding to different platforms
Wasm-jit is an independent virtual machine in EOS. The reason why WASM is used is because it uses intermediate language to translate high-level language programs into IR (intermediate Representation) intermediate language and then compile into machine language. WAVM can convert between was and wasm, wast is text, using near-machine translation syntax, wasm is binary code
Assemble in.wast out.wasm
Disassemble in.wasm out.wast
The source code of the wasm-jit virtual machine is: ~/eos/libraries/wasm-jit/Source, the directory structure is as follows:
main folders：Emscripten, IR，Runtime, Programs,WASM, WAST
IR: IR is the first step in high-level language to machine language, followed by WAST parsing, WASM serialization, and Runtime processing.
Emscripten: a tool for converting high-level languages to LLVM bytecode
Runtime execution engine: includes execution bytecode (machine code), external call, translate bytecode to IR, generate machine code, etc.
WASM: WASM 的 LICENSE
WAST: Parse parsing function module
Other: Logging: Log Management Platforms: Corresponding to different platforms
Irrational design of EOS
There is a separate set of intelligent contract engine based on WebAssembly, but the current EOS contract development has the following obvious problems:
1. The account system is not friendly: it is difficult to create an account. After creating an account, the contract can be issued. EOS needs to use an existing account to create a new account. Looking for a friend or third party with an EOS account is not a problem for anyone. It’s easy. To create an account, you need to purchase RAM, which means you need to spend money to build an account. If you find a third party to help create an account, there is a financial risk. After creating an account, you need to mortgage EOS in exchange for CPU usage time and net bandwidth to operate on the EOS network. These operations are too cumbersome for developers.
2. RAM price: RAM is expensive, the contract operation must use RAM, EOS opens the RAM market for trading memory, although RAM can be bought and sold, but there are still many people speculation, resulting in RAM expensive.
3. Development is difficult: using C++ as the contract development language greatly improves the contract development threshold. C++ itself is extremely complicated, and it also needs to call EOS.IO C++ API to complete smart contract development, for developers. The ability is extremely high.
Therefore, in the face of various enumerated problems, EOS smart contract development is not very attractive to developers, and may even be the reason for developers to give up EOS.
4. IOST VM
Starting point: Based on V8’s excellent performance on NodeJs and Chrome
IOST V8VM architecture and design
The core of the V8VM architecture is VMManger, which has the following three functions:
1. VM entry, request to receive other modules externally, including RPC request, block verification, Tx verification, etc., pre-processed, formatted and handed over to VMWorker for execution.
2. Manage the VMWorker life cycle, flexibly set the number of workers according to the current system load, and implement worker reuse. At the same time, the JavaScript code hot start and the hotspot Sandbox snapshot persistence function are implemented inside the worker, which reduces the frequent creation of virtual machines and frequently loads the same. The high load and memory soaring problems caused by the code reduce the system consumption and greatly improve the system throughput, making the IOST V8VM more comfortable in dealing with the typical massive user contract of fomo3D.
3. Manage the interaction with the State database to ensure the atomicity of each IOST transaction, and to be able to roll back the entire transaction in the event of a contract execution error or insufficient gas. At the same time in the State database, it also implements a two-level memory cache, and finally will be flushed to RocksDB.
Sandbox core design
As the carrier of the final execution of JavaScript smart contract, Sandbox completes the call to V8VM and the next package of Chrome V8, which is mainly divided into Compile stage and Execute stage:
COMPILE command
Mainly for contract development and winding, there are two main functions:
1. Contract Pack, packaged smart contract, based on webpack implementation, will package all the JavaScript code under the current contract project, and automatically complete the dependency installation, making IOST V8VM develop large contract projects. At the same time, IOST V8VM and Node.js module system are fully compatible, and can seamlessly use methods such as require, module.exports and exports to give contract developers a native JavaScript development experience.
2. Contract Snapshot, with the v8 snapshot snapshot technology, complete the compilation of JavaScript code, the compiled code enhances the efficiency of Chrome V8 to create isolate and contexts, the real execution only needs to deserialize the snapshot to complete the execution, greatly Improves the loading speed and execution speed of JavaScript.
EXECUTE
Mainly for the real implementation of the chain contract, there are two main functions:
1. LoadVM, complete VM initialization, including generating Chrome V8 objects, setting system execution parameters, importing related JavaScript class libraries, etc., to complete all preparations before smart contract execution. Some JavaScript class libraries are as follows:
2. Execute, the final implementation of the JavaScript smart contract, IOST V8VM will open a separate thread execution contract, and monitor the current execution state, when an exception occurs, the use of resources exceeds the limit, the execution time exceeds the maximum limit, will call Terminate to end the current contract execution, Returns an abnormal result.
