What is a virtual machine? Why is it so important in the world of blockchain?
Hashgard Labs, a research institute of Hashgard, and BKFUND Research Institute jointly conducted an in-depth analysis of virtual machines and wrote a research report about it.
1. Smart Contract
To put it simply, smart contract is a self-validating computer trading agreement or program that does not require intermediation and can execute automatically contract terms.
Smart contracts in blockchains can facilitate transfers and describe game rules.
· Smart contract, in a narrow sense, is a computer program running on a distributed ledger with preset rules, state and conditional response, which can encapsulate, verify, and execute distributed node complex behaviors, complete information exchange, value transfer and asset management.
· In a broader sense, smart contract is a self-validating computer trading agreement that does not require intermediation and can execute automatically contract terms.
According to its design purpose, it can be divided into:
· Smart legal contract designed to be an alternative and supplement to the law
· Smart software contract designed as a functional software
· Smart alternative contracts designed to introduce new contractual relationships (such as smart contracts that stipulate machine-to-machine business practices in the Internet of Things).
2. Technical Path:
Blockchain smart contracts should meet the design requirements and implementation ideas.
· Smart contracts should be deterministic and adopt deterministic algorithms and data sources when designing.
· Smart contracts can be terminated, and can be implemented through limited commands, gas mode, resource control, and access restrictions.
At present, there are three technical ways to realize blockchain smart contracts: script mode, containerized mode, and virtual machine mode.
2.1 Script mode
Script method is a traditional way to implement programmable logic, which was firstly adopted in the BTC system. BTC’s UTXO model uses a Forth-like signature scripting system to verify the legitimacy of the transaction. Transaction generally includes an input script and an output script, which are used to unlock the output of the previous transaction and set the unlock condition of the transaction amount.
BTC script uses an reverse Polish notation in the form of a stack structure. User needs to provide matching signature, public key to the script as an execution input in order to unlock the transaction. Therefore, another feature of the script mode is that it works well with the UTXO mode.
But the shortcomings of this mode are also obvious, mainly in the scalability of the function. Therefore, in terms of functional completeness and programming extensibility, many people think that the programming tasks that Bitcoin’s script mode can achieve is very limited. Therefore, from a narrow point of view, it is considered that the script mode can only realize simple programmable features, not smart contract system in the usual sense.
The current script mode that can achieve programmable features is considered as a blockchain version 1.0 system, which can be divided into the following three categories:
1) BTC and related forks and competition token
The token that uses script mode is bitcoin and some of the early tokens of the UTXO model, including BTC’s forked Token, competitive Token, such as Litecoin, Bitcoin Cash and so on.
2) Anonymous Token
Token that boasts anonymous and privacy protection also uses the scripting system. Due to the hidden nature of privacy protection and transaction traceability, many of these tokens use an abstract and simplified scripting system, including Monero, Dash, and so on.
3) DAG Blockchain
There are other blockchain systems that use a scripted way to run contracts based on Directed Acyclic Graph structures (DAGs). Due to the inherent characteristics of the DAG system, including the uncontrollable transaction duration and the absence of a global sorting mechanism, it is difficult to implement a Turing-complete intelligent contract system on the DAG distributed ledger system. Therefore, in order to improve TPS and simplify transaction processing, some DAG-based distributed ledger systems will also use scripting mode to build their own programmable systems.
2.2 Containerization
Containerization is a new type of virtualization technology that has emerged in recent years and is different from virtual machines. In contrast to the virtual machine’s added layer of intermediate environments in the user program and the underlying environment, container technology only needs to package the dependencies required by the application to run independently, without the need for an additional virtual operating system environment.
The containerized approach, compared to the virtual machine approach, is relatively more independent and flexible and can deploy more resources.
Although containerization technology is easier and more flexible from the perspective of overall system architecture, some more “heavy” system factors need to be considered if thinking about one single app, because files and systems can be accessed in container environment. This puts more demands on the operating environment of the blockchain, so blockchain that adopts this method is relatively rare.
A typical blockchain project that uses containerization is the Hyperledger Fabric. Its smart contract works by deploying a chain of code on the node, and all related nodes start a chain code process that runs independently in the Docker container. The chain code completes the interaction with the node through the external gRPC interface in the container.
At present, Fabric still manages and maintains chain code in a manual and base way. Because it is in the environment of the alliance chain, all nodes that are allowed to join the network can use the system resources, that is, the access restriction method.
This is more obvious in maintaining chain code life cycle: in the Fabric 1.3 version, only four commands, package, install, instantiate, upgrade are provided. A command to stop and start the chain code without actually deleting the chain code may be provided in the future.
Therefore, if you want to construct a smart contract environment in a containerized way in a public-chain environment, you need to adopt more methods such as resource control to adapt to the transformation.
2.3 Virtual Machine Mode
The most common way to implement smart contracting is the virtual machine approach. It provides a program with a completely transparent execution environment.
A typical application of this idea can be traced back to Java’s JVM virtual machine in traditional IT technology. The goal is to implement the “write once, run everywhere” feature, rather than having the program developer write different versions of the program for each different server. This feature is exactly what is needed for distributed deployment and operation of smart contracts: the difference between the block execution node’s own execution environment operates consistently on all nodes, to achieve the deterministic characteristics.
Currently, blockchain virtual machines mainly have six directions:
1. Neo VM (AVM, the compiler core is based on .NET CLR MSIL), currently supports the .NET series of languages (C#, VB.NET, F#), Java series languages (Java, Kotlin), Python.
2. WebAssembly (Wasm), currently supports C/C++/RUST/Go.
3. Script language virtual machine (Chrome V8 engine, Lua virtual machine, etc.), js is currently supported.
4. RISC-V (an open-source CPU instruction set architecture), currently supports Solidity.
5. Create a new virtual machine and language, similar to EVM. EVM & solidity (ethereum),Move VM & move (libra).
6. Support multiple virtual machines and multiple languages. The ontology supports neo VM & Wasm, the Thunderbolt supports EVM & Wasm, and the core chain supports neo VM & EVM.
Virtual machine VS container
2.3.1 Summary of Virtual Machine Basics
Core of the virtual machine: memory management, assembly loading, security, exception handling, and thread synchronization. The virtual machine is mainly divided into two types of virtual machines based on Stack and Register.
(1) Interpreter
Interpret the execution source language (php, postscretp, javascript and so on).
Pros and cons
The interpreter starts and executes in a faster way, and you don’t have to wait for the entire compilation process to complete before you can run the code. Translate from the first line and you can continue to execute in order. The interpreter seems to be more suitable for JavaScript. For a web developer, it’s important to be able to execute code quickly and see the results. That’s why the original browsers all use the JavaScript interpreter.
However, when running the same code more than once, the disadvantages of the interpreter are revealed. For example, if you execute a loop, the interpreter will have to translate it again and again. This is an inefficient performance.
(2) Compiler
The source code is compiled into the target code as a whole, and the compiler is not required to execute, and it runs directly on the platform that supports the target code, so that the execution efficiency is much faster than the interpretation execution.
Pros and cons
The problem with the compiler is the opposite of the one with interpreter. The compiler needs to take some time to compile the entire source code and then generate the file to execute on the machine. For codes with loops, it executes very fast because it doesn’t need to be repeated to translate every loop.
Another difference is that the compiler can use more time to optimize the code to make the code be executed faster. The interpreter does this step at runtime, which determines that it is not possible to optimize for a lot of time during translation.
(3) GCC
GNU (Gnu’s Not Unix) Compiler Collection (GCC), a set of programming language compilers, a free software released under the GPL and LGPL licenses, is also a key part of the GNU project and the main part of the GNU toolchain. It is the first compiler choice for cross-platform software. GCC uses the same front-end program on all platforms, producing the same mediation code, so this mediation code is compiled with GCC on various other platforms, and there is a great chance of getting the correct output program.
Supported languages: C, C++, Fortran, Pascal, Objective-C, Java, Ada, Go, etc.
Main processor structures supported: ARM, x86, x86–64, MIPS, PowerPC, etc.
(4) Clang
It is a compiler front end for C, C++, Objective-C, and Objective-C++ programming languages. It uses the underlying virtual machine (LLVM) as its back end. Its goal is to provide a replacement for the GNU Compiler Suite (GCC).
(5) JIT
JIT (Just-in-time compiler): combining the advantages of the interpreter and compiler. In order to solve the inefficiency of the interpreter, browsers introduced the compiler, making it a mixed mode.
Different browsers implement this function differently, but the basic idea is the same. Add a monitor (also called an analyzer) to the JavaScript engine. The monitor monitors information such as the operation of the code, records frequency of the code has been run and how it is run.
JIT is a means of making JavaScript run faster, by optimizing hot code (code that can be executed multiple times) by monitoring the running state of the code. In this way, the performance of JavaScript applications can be increased by multiple times.
In order to make execution faster, JIT adds a lot of extra overhead, including:
· expense of optimization and de-optimization overhead;
· expense of monitor recording the information;
· The memory-to-memory overhead of recovering information when deoptimization occurs;
· Memory overhead for baseline and optimized versions.
There is still a lot of room for improvement, which is eliminating overhead. One of the things that WebAssembly has to do is to further improve performance by eliminating overhead.
(6) JVM
A JVM (Java Virtual Machine)-dependent language (Java, Scala, Groovy, Kotlin, etc.) is compiled into a bytecode, which is a class file, after a JIT (just in time) compilation. Resolved by different JVMs on different operating systems, and finally converted to machine code of different platforms, and finally executed.
(7) NET CLR
NET CLR (Common Language Runtime) An operating environment that can be used by a variety of programming languages (C++/CLI, C#, Visual Basic, F#, Iron Python, Iron Ruby, Intermediate Language (IL), etc.) The main execution engine of the .NET Framework.
(8) GraalVM
Supports non-hosted languages (JavaScript, Ruby, R, Python, LLVM binaries) and JVM-based host languages
(9) LLVM
LLVM (Low Level Virtual Machine) is a reusable compilation tool chain that provides an intermediate language between high-level programming languages and machine languages. LLVM itself can be used as a back end for multiple languages, providing optimization independent of the language itself and code generation for multiple CPUs. (LLVM was developed by UIUC, and the meaning of LLVM (low level virtual machine) has been exceeded.)
LLVM is for the front end of different languages, corresponding to machine code of different platforms.
The compilation process of LLVM is as follows: the source code is compiled into a file of LLVM intermediate format, and then LLVM Linker is used to link and optimize, and the obtained LLVM code is finally translated into machine code of a specific platform. In addition, LLVM supports JIT and will be in the code generation process. Insert some lightweight operation instructions to run the mobile phone information, such as identifying the hot region, and the collected information can support offline optimizer, implement profile-driven optimization strategies, and adjust native code to adapt to specific architectures.
LLVM IR(Intermediate representative)：
It can be seen from the above that the LLVM compiler first translates the source language into an “intermediate language”, and different languages have different IRs, and then translate them into a compiled language of the target platform through the backend program. LLVM IR provides three formats, namely: the IR format in memory, the binary format stored on disk, and the text format stored on disk. In addition to this, there are some file formats related to IR, which are listed below:
· End of bc, LLVM IR file and binary, can be executed by the lli command.
· Ll end, LLVM IR file and text format, can be executed by lli
· End of s, local assembly file
· out suffix and local executable file. The following image shows the conversion of the centralized file:
LLVM’s front-end compiler can use a variety of parsing compilers, usually more than clang, but in EOS is replaced by csm clang.
LLVM JIT（just-in-time) :
JIT writes the part of the original compiler to generate the machine code directly into the current memory. Through the conversion of the function pointer, the corresponding machine code is found and executed. It is often used to deal with memory management, symbol redirection, and processing external symbols.
(10) Wasm
WebAssembly is a new bytecode format, in addition to JavaScript (JS was introduced in 1995, 2008 “performance war” started. Many browsers introduce the Just-in-time (JIT) compiler. In addition to JIT-based mode, JS code runs 10 times faster, another programming language that can be executed in a browser.
Its abbreviation is “.wasm”, and .wasm is a file name suffix, a new underlying secure binary syntax. It can be abstractly understood as the machine language of a conceptual machine, mapping directly to machine code more than JavaScript code, and it also represents a concept of “how to execute code more efficiently on general-purpose hardware.” So it is not directly mapped to machine code for a specific hardware.
Once the browser has downloaded WebAssembly, it can be quickly converted to machine assembly code. It is defined as “streamlined, short-loading format and execution model” and is designed as a Web multi-programming language target file format. This means that the performance of the browser side will be greatly improved, it also enables us to implement a collection of the underlying building blocks. WebAssembly-enabled browsers recognize text in binary format, and have the ability to compile binary packages that are much smaller than JS text, decoding much faster than JS.
Wasm allows users to write in their own familiar language (currently supporting C/C++/Rust) and then run on the browser in the virtual machine engine. It supports sandbox mode, which is to write the wasm module in high-level language and then load it as a library function in JS.
WebAssembly uses a stack-based virtual machine, but it doesn’t mean it works on real physical machines. When the browser translates WebAssembly to machine code, the browser uses registers, and the WebAssembly code does not specify which registers to use. This has the advantage of giving the browser maximum freedom to let it do its best.
WASM allows languages such as C/C++ to be written in WEB. WASM is a bytecode format, which is the underlying binary syntax, load time period and high-speed execution. It is the target file format designed for WEB multi-language programming.
WebAssembly provides two formats: the readable text format wast and the binary format wasm, through the tool wast2wasm to complete the format conversion from wast to wasm. Similarly, wasm2wast implements inverse conversion.
Part of the WebAssembly module
Must be part of:
Type: The function declaration of the function defined in the module and the function declaration of all imported functions.
Function: gives an index to each function in the module.
Code: The actual function body of each function in the module.
Optional parts:
Export: Makes functions, memory, tables, global variables, etc. visible to other WebAssembly or JavaScript, allowing dynamic linking of separately compiled components, the WebAssembly version of .dll.
Import: Allows you to import specified functions, memory, tables, or global variables from other WebAssembly or JavaScript.
Start: A function that can be run automatically when the WebAssembly module is loaded (similar to the main function).
Global: Declares the global variables of the module. Memory: Defines the memory used by the module.
Table: Makes it possible to map to values outside of the WebAssembly module, such as objects mapped to JavaScript. This is useful when calling indirect functions.
Data: Initialize imported or local memory.
Element: Initializes an imported or local table.
Wasm VS JS
· The current WebAssembly can only use numbers (integer or float) as arguments or return values
· For any other complex type, such as string, you must use the memory operations of WebAssembly. If you’re using JavaScript a lot and you’re not familiar with working directly with memory, think back to the C, C++, and Rust languages, which are all manually manipulated memory.
The memory operations of WebAssembly are very similar to the memory operations of these languages. To do this, it uses a data structure called ArrayBuffer in JavaScript. ArrayBuffer is a byte array, so its index is equivalent to the memory address. If you want to pass a string between JavaScript and WebAssembly, you can use ArrayBuffer to write it to memory. At this point, the ArrayBuffer index is an integer and you can pass it to the WebAssembly function. At this point, the index of the first character can be used as a pointer.
Wasm VS asm.js
Asm.js is a strict subset of JavaScript that can be used as a low-level, efficient compiler target language. Asm.js provides an abstract implementation similar to a C/C++ virtual machine, including a large binary heap, integer and floating point operations, high-order function definitions, function pointers, etc. that can be efficiently loaded and stored.
The idea of asm.js is to write JavaScript code using the methods it specifies, and the engine that supports asm.js turns the code into a very efficient machine code. If you compile C++ code into asm.js, you will get a huge performance boost on the browser side.
Web Assembly is a lot more radical than asm.js. Web Assembly even with Js. This kind of thing is too lazy to do it, isn’t it AOT? Can I give the bytecode directly? (It is ater changed to AST tree). For browsers that don’t support Web Assembly, there will be a piece of Javascript that re-translates the Web Assembly into Javascript. This technique is called polyfill, a technique that is commonly used when HTML5 comes out. The reason for using AST is because AST is easier to compress than bytecode and easier to translate. Javascript is compiled to AST first, and then to Bytecode. AST is one level higher than Bytecode.
Compared to asm.js, it reduces the amount of code by about 25%. WebAssembly loads 20 times faster than asm.js, mainly because the JavaScript engine deciphers the binary format faster than parsing the asm.js code. Much faster. Streamlined code, better performance, fewer bugs.
Technical Advantages of ATM
· High performance: WASM uses binary encoding and has excellent performance during program execution;
· Low storage cost: binary-encoded text takes up less storage space than text format;
· Multi-language support: Users can write smart contracts in multiple languages such as C/C++/RUST/Go and compile them into WASM formatted bytecodes;
Reasons why Wasm performs faster than JS:
· During the file fetching phase, WebAssembly is faster than JavaScript to fetch files. Even if JavaScript is compressed, the WebAssembly file is smaller than JavaScript;
· In the parsing phase, the decoding time of WebAssembly is shorter than the parsing time of JavaScript;
· In the compilation and optimization phase, WebAssembly is more advantageous because the code of WebAssembly is closer to machine code, and JavaScript is first code optimized through the server.
· In the re-optimization phase, WebAssembly does not re-optimize. The optimization hypothesis of the JS engine may result in the “abandon optimization code <-> re-optimization” phenomenon.
· In the implementation phase, WebAssembly is faster because developers don’t need to know too much compiler skills, which is needed in JavaScript. WebAssembly code is also better suited to generate more efficient instructions for machine execution.
· In the garbage collection phase, WebAssembly garbage collection is manually controlled and is more efficient than automatic recycling.
Combination with blockchain
Web-side DApp
Web-side compiled languages are mostly interpretive in the JS era. Although they are easy to understand by users, they are not efficient, especially JS. Node.js offers a framework to write local and server-side applications, but it is too inefficient for cryptographic computing and image processing. HTML5 solves a lot of browser problems in features and performance standards, but H5 still uses JS as the main language and does not solve the essential problem. Wasm came into being when many web developments were unable to unify their own compiling languages.
In many projects on the Web platform, support for native new features requires a Web browser or Runtime to provide complex, standardized APIs, but Java APIs tend to be slow. With WebAssembly, these standard APIs are simpler and can be opearted at the bottom. For example, for a facial recognition Web project, we can implement the data flow by a simple Java API, and the face recognition native SDK does the work done by WebAssembly.
So for the blockchain DApp, its meaning is very clear:
1. Allow developers to develop in other languages and then load them on JS.
2. Improve program performance and allow the development of large blockchain DApps.
This is why ETH, EOS and other projects want to use the wasm technology.
Since the EVM needs to be pre-compiled and gas needs to be a cost, the cost to program on the EVM is actually very high. At the same time, it doesn’t solve the problem of bloated EVM. Last, Solidity is harder to learn than its language bacis C.
Wasm is memory-safe and platform-independent structure, and can be effectively mapped to all types of CPU architectures. Its instruction set is efficient and maintains sufficient portability. In addition, the Wasm instruction set can be easily deterministic by removing floating point instructions, which will make it suitable for replacing the EVM language.
At the same time, Wasm can achieve untrusted programming without consuming more memory. Accurate calculations can be performed by stack analysis and metrology on Wasm.
Currently ETH intends to use DApps for smart contracts based on eth-WASM, while EOS, Polkadot and Ontology are for virtual machines.