Forge std-test Usage Tutorial

Test cases must be prepended with prove_ and the testing contract must inherit from Test from Forge's standard test library. First, import Test: import {Test} from "forge-std/Test.sol"; and then inherit from it via ... is Test. This allows hevm to discover the test cases it needs to run. Like so:

pragma solidity ^0.8.19;
import {Test} from "forge-std/Test.sol";
contract BadVaultTest is Test {
  function prove_mytest() {
  // environment setup, preconditions
  // call(s) to test
  // postcondition checks
  }
}

Once you have written such a test case, you need to compile with forge build --ast (see forge documentation for more details) and then:

$ hevm test
Checking 1 function(s) in contract src/badvault-test.sol:BadVault
[RUNNING] prove_mytest(uint256)
   [PASS] prove_mytest(uint256)

Here, hevm discovered the test case, and automatically checked it for violations.

Setting Up Tests

Tests usually need to set up the environment in a particular way, such as contract address, storage, etc. This can be done via Cheat Codes that can change the address of the caller, set block number, etc. See Cheat Codes below for a range of cheat codes supported. Cheat Codes are a standard method used by other tools, such as Foundry, so you should be able to re-use your existing setup. An example setup could be:

pragma solidity ^0.8.19;
import {Test} from "forge-std/Test.sol";
contract BadVaultTest is Test {
    MyVault vault;

    function setUp() public {
        // Set up environment
        vault = new BadVault();

        address user1 = address(1);
        vm.deal(user1, 1 ether);
        vm.prank(user1);
        vault.deposit{value: 1 ether}();

        address user2 = address(2);
        vm.deal(user2, 1 ether);
        vm.prank(user2);
        vault.deposit{value: 1 ether}();

        address attacker = address(42);
        vm.prank(attacker);
        // call(s) to test
        // postcondition checks
    }
}

The postconditions should check the state of the contract after the call(s) are complete. In particular, it should check that the changes that the function applied did not break any of the (invariants)[https://en.wikipedia.org/wiki/Invariant_(mathematics)] of the contract, such as total number of tokens.

You can read more about testing and cheat codes in the (Foundry Book)[https://book.getfoundry.sh/forge/cheatcodes] and you can see the hevm-supported cheat codes below.

Understanding Counterexamples

When hevm discovers a failure, it prints an example call how to trigger the failure. Let's see the following simple solidity code:

pragma solidity ^0.8.19;
import {Test} from "forge-std/Test.sol";
contract MyContract is Test {
  mapping (address => uint) balances;
  function prove_single_fail(address recv, uint amt) public {
    require(balances[recv] < 100);
    if (balances[recv] + amt > 100) { revert(); }
    balances[recv] += amt;
    assert(balances[recv] < 100);
  }
}

When compiling our foundry project, we must either always pass the --ast flag to forge build, or, much better, set the ast = true flag in the foundry.toml file:

ast = true

In case neither --ast was passed, nor ast = true was set in the foundry.toml file, we will get an error such as:

Error: unable to parse Foundry project JSON: [...]/out/Base.sol/CommonBase.json Contract: "CommonBase"

In these cases, issue forge clean and run forge build --ast again.

Once the project has been correctly built, we can run hevm test, and get:

$ hevm test
Checking 1 function(s) in contract src/contract-fail.sol:MyContract
[RUNNING] prove_single_fail(address,uint256)
   [FAIL] prove_single_fail(address,uint256)
   Counterexample:
     result:   Revert: 0x4e487b710000000000000000000000000000000000000000000000000000000000000001
     calldata: prove_single_fail(0x0000000000000000000000000000000000000000,100)

Here, hevm provided us with a calldata, where the receiver happens to be the zero address, and the value sent is exactly 100. This indeed is the boundary condition where the function call fails. The function should have had a >= rather than a > in the if. Notice that in this case, while hevm filled in the address to give a complete call, the address itself is irrelevant, although this is not explicitly mentioned.

Starting State is Always Concrete

In test mode, hevm runs with the starting state set to concrete values. This means that with the solidity-generated default constructor of contracts, state variables will be zero, and arrays and mappings will be empty. If you need a different starting state, such as e.g. tokens already distributed to some addresses, you can set that up in the setup phase of your test. This can be done via the beforeTestSetup function, as documented in the Foundry Book.

In case you need a symbolic starting state, see the Symbolic execution tutorial. Note that if all you need is a symbolic calldata, then you don't need to run hevm in symbolic mode, you can simply run hevm test and hevm will provide you with a symbolic calldata.

Test Cases that Must Always Revert

Hevm assumes that a test case should not always revert. If you have such a test case, hevm will warn you and return a FAIL. For example this toy contract:

pragma solidity ^0.8.19;
import {Test} from "forge-std/Test.sol";
contract MyContract is Test {
  uint256 cntr;
  function prove_allrevert(uint256 val) public {
      if(val < 0) {
          unchecked { cntr = val; }
          revert();
      } else revert();
  }
}

When compiled with forge and then ran under hevm with hevm test, hevm returns:

Checking 1 function(s) in contract src/contract-allrevert.sol:MyContract
[RUNNING] prove_allrevert(uint256)
   [FAIL] prove_allrevert(uint256)
   Reason:
     No reachable assertion violations, but all branches reverted
     Prefix this testname with `proveFail` if this is expected

This is sometimes undesirable. In these cases, prefix your contract with proveFail_ instead of prove_:

pragma solidity ^0.8.19;
import {Test} from "forge-std/Test.sol";
contract MyContract is Test {
  uint256 cntr;
  function proveFail_allrevert_expected(uint256 val) public {
      if(val < 0) {
          unchecked {
            cntr = val;
            cntr += 1;
          }
          revert();
      }
      else revert();
  }
}

When this is compiled with forge and then checked with hevm, it leads to:

Checking 1 function(s) in contract src/contract-allrevert-expected.sol:MyContract
[RUNNING] proveFail_allrevert_expected(uint256)
   [PASS] proveFail_allrevert_expected(uint256)

Which is now the expected outcome.

Supported Cheat Codes

Since hevm is an EVM implementation mainly dedicated to testing and exploration, it features a set of "cheat codes" which can manipulate the environment in which the execution is run. These can be accessed by calling into a contract (typically called Vm) at address 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D, which happens to be keccak("hevm cheat code"), implementing the following methods: