The Dao Soft-Fork attempt was difficult. Not only has it turned out that we underestimated the side effects on the consensual protocol (i.e. back vulnerability), but we also managed to introduce a data race in the precipitated implementation which was a time bomb. It was not ideal, and even if it was avoided in the last case, the deadline for a fork quickly approached seemed strangely dark to say the least. We needed a new strategy …
The springboard towards this was an idea borrowed from Google (graciousness of Nick Johnson): write a detail autopsy of the event, aimed at assessing the deep causes of the problem, focusing only on technical aspects and appropriate measures to prevent recurrence.
Scale and persist on technical solutions; Blame people don’t do it. ~ Nick
From the post-mortem, an interesting discovery from the point of view of this blog article was made. The code with flexible fork inside (Go-Stage) (solid of all angles: a) It was carefully covered by unit tests with a test / code 3: 1 ratio; b) it was fully examined by six developers of the Foundation; And c) It was even tested manually live on a private network … However, a deadly data race remained, which could have potentially caused a severe disruption of the network.
It turned out that the fault could only occur in a network made up of several nodes, several minors and several blocks struck simultaneously. Even if all these scenarios were held true, there was only a slight chance for the bug to surface. Unit tests cannot catch it, code examiners can catch it or not, and manual tests would catch it would not be likely. Our conclusion was that the development teams needed more tools to carry out reproducible tests that would cover the complex interaction of several nodes in a simultaneous network scenario. Without such a tool, the manual verification of the various cases of the edge is heavy; And without making these checks permanently as part of the development workflow, rare errors will become impossible to discover over time.
And so, hive was born …
What is hive?
Ethereum grew up to the point where test tests have become a huge burden. The unit tests are very good for verifying various implementation quirks, but validating that a customer complies with a basic quality, or validate that customers can play well in a multi-client environment, is almost simple.
Hive is supposed to serve as an easily expandable test harness where anyone can add tests (be these simple validations or network simulations) any Language programming with which they are comfortable, and hive should simultaneously be able to execute these tests against all potential customers. As such, the harness is intended to carry out black box tests where no specific details specific to the customer can be tested and / or inspected, but rather the accent would be placed on membership of official specifications or behaviors in different circumstances.
More importantly, Hive was designed from zero to run as part of the workflow below all customers!
How does the hive work?
The body and soul of Hive are (Docker) (each customer implementation is a docker image; each validation sequence is a docker image; and each network simulation is a docker image. Hive itself is a Docker All Enpelobing image. It is a very powerful abstraction …
From Ethereum customers are Docker images in Hive, customer developers can assemble the best possible environment so that their customers can execute (dependence, tools and configuration). Hive will run as many instances as necessary, all operating in their own Linux systems.
Likewise, as test suites The validation of Ethereum customers is Docker images, the author of the tests can use any programming environment he knows the most. Hive will guarantee that a customer runs when the tester starts, who can then validate if the private customer complies with desired behavior.
Lately, Network simulations Once again are defined by Docker images, but compared to simple tests, the simulators run not only of the code against an executed customer, but can really start and terminate customers at will. These customers run in the same virtual network and can freely (or as dictated by the simulator container) connect to each other, forming a private Ethereum network on demand.
How did the hive help the fork?
The hive is neither a replacement of unit tests nor for a thorough revision. All current used practices are essential for obtaining a specific implementation of any functionality. Hive can provide validation beyond what is possible from the point of view of an average developer: perform in-depth tests which may require complex execution environments; And check the networking cases that can take hours to install.
In the case of Dao Hard-Fork, beyond all consensus and unit tests, we had to ensure the most important that nodes properly share in two subsets in networking: a support and an opposite to the fork. This was essential because it is impossible to predict which negative effects performing two competing chains in a network could have, in particular from the point of view of the minority.
As such, we have implemented three specific network simulations in Hive:
-
The first To verify that minors who execute complete ETHAsh DAGS generate additional fields of data from correct blocks for pro-foour bite and without grinding, even when trying to browse naively.
-
The second To verify that a network composed of fork nodes / minors and without mixed fork is divided properly in two when the fork block arrives, maintaining the split later.
-
The third To verify that given an already forked network, the joint nodes can synchronize, rapid synchronization and light synchronization with the chain of their choice.
The interesting question is however: Hive has Hive really caught errors, or did she simply act as an additional confirmation that everything is fine? And the answer is, both. Catpred hive Three bugs not linked to the fork In Geth, but also strongly helped Geth’s development in Geth’s range by continuously providing feedback on how modifications have affected network behavior.
There were criticisms of the Go-Ethereum team for having taken their time on the implementation of the hard fork. I hope people will now see what we were doing, while simultaneously implementing the fork itself. Overall, I believe hive He turned out to play a fairly important role in the cleanliness of this transition.
What is Hive’s future?
The features of the Ethereum GitHub organization (4 test tools already) (with at least one EVM kitchen reference tool in an external repository. They are not used to their extent. They have a ton of dependencies, generate a ton of waste and are very complicated to use.
With Hive, we aim to aggregate all the different tests dispersed under a Validator of the universal client This has minimum dependencies, can be extended by anyone and can operate in the Daily CI workflow of customer developers.
We are delighted with anyone to make contributions to the project, whether the addition of new customers to validate, validators to test or simulators to find interesting networking problems. In the meantime, we will try to polish the hive itself, adding a support for racing references as well as mixed simulations.
With a little or a work, we may even have support for the execution of Hive in the Cloud, which allows him to execute network simulations on a much more interesting scale.
