The purpose of developing software is to get it to perform as intended. Multiple test iterations are required to achieve that, and the time between iterations is of critical importance when trying to complete the task at hand as soon as possible.
At Origin, we use unit & fork tests for development. Running fork tests can be tediously slow, but we’ve managed to reduce the execution time of fork tests (in some cases) by a factor of 4. Through this article I’ll explain how you can improve the speed of your fork tests, so that you can get your code working as intended in no time.
Typically, whether you use a fork test or unit test depends on the architecture of the protocol being developed. Some teams only use unit tests and some both unit and fork tests. At Origin, our contracts interact with several 3rd party protocols (Curve, Uniswap, Balancer, Convex…) and mistakes in integration can cause catastrophic loss of funds.
Using unit tests, we mock the 3rd party protocol’s ABI and test that our contracts programmatically correctly integrate. But to test a 3rd party protocol’s behavior with all the intricacies and various (legitimate or manipulated) states the protocol can be in is where fork tests shine. And for some engineers on our team, test-driven development using fork tests is the primary development mode. Reducing the “code change” to “test execution” time is essential.
At Origin, we are using Hardhat for our Solidity development stack. It’s important that we understand the various steps a fork test run consists of in order to reduce development cycle time. For simplicity, we shall be running only 1 fork test at a time - which is usually the approach when working on a smart contract.
The slowest approach is to run the fork tests with node forking from the latest block. This way, each time tests are run a different block height is chosen, and hardhat isn’t able to do any caching by reading storage slots from the main-net provider. This is reflected in the slow 101 second execution speed.
This particular fork test sends funds to one of our liquidity mining strategies that deploys assets to a Balancer pool. That causes a great deal of storage slot reads and is the main cause of the slow “test” part of the execution (orange part of the bar above).
The obvious low-hanging fruit is to fix the block height so that Hardhat can save a great deal of time by utilizing a local cache of storage slot reads. The compile time isn’t affected by this optimization, but all other steps are. Greatly reducing the cycle time to 27 seconds (this applies to all non first runs of the test, where the hardhat cache is already warmed up).
The above approach improves the situation significantly, but it is still far from the speed of unit tests and can still seem frustratingly slow at times. We have only changed one contract. Why do we need to re-run the same deployments and post-deployment when ideally just the byte-code of 1 (or more) solidity contracts needs changing? Given that, of course, we are not changing storage slot layout or state between compilations. Enter hot deploys.
Hot-deploy deploys the contract that is being altered and fetches its byte-code. Using hardhat test suite snapshots, it finds the previous version of that contract in a state before the tests were run and replaces its byte-code with the freshly compiled one. On top of that, it runs the test fixtures and the test itself, skipping all the (post) deployment steps.
The cycle time is now reduced to just 7 seconds. This change in the development cycle has helped Origin Protocol greatly improve the productivity and speed at which we can develop features.
Unfortunately, the hot deploys cannot be turned on as a switch (at least not yet) and require some configuration. The way we have implemented the solution at Origin is as follows:
An important requirement is that a standalone hardhat node is running separately from the fork test environment. It might not be immediately obvious, but fork test run will create another node runtime and use the other standalone running node as a provider. The standalone node will run all (post) deployments, and the fork test node will compile the changes, run test fixtures replacing any contract byte-code required, and finally run the test.
It’s good to keep in mind changes to the deployment files, contract storage slots, and their state set up. You should consider this running test fixtures require a restart of the standalone node.
Sometimes we can be tempted to use unit tests for parts of the platform that integrate with 3rd party protocols just to be able to develop faster… at the cost of additional security a fork test can provide. For developers at Origin, hot deploys are an important step to bridge that cycle time gap and try to have our cake and eat it too. I hope others try this, and I hope you can benefit from this approach to reduce your development cycle times.
