⚠️ UNDER HEAVY DEVELOPMENT ⚠️

This documentation is actively being developed and may change frequently.

Introduction

Devnet SDK

The Devnet SDK is a comprehensive toolkit designed to standardize and simplify interactions with Optimism devnets. It provides a robust set of tools and interfaces for deploying, managing, and testing Optimism networks in development environments.

Core Components

1. Devnet Descriptors

The descriptors package defines a standard interface for describing and interacting with devnets. It provides:

  • Structured representation of devnet environments including L1 and L2 chains
  • Service discovery and endpoint management
  • Wallet and address management
  • Standardized configuration for chain components (nodes, services, endpoints)

2. Shell Integration

The shell package provides a preconfigured shell environment for interacting with devnets. For example, you can quickly:

  • Launch a shell with all environment variables set and run commands like cast balance <address> that automatically use the correct RPC endpoints
  • Access chain-specific configuration like JWT secrets and contract addresses

This makes it easy to interact with your devnet without manually configuring tools or managing connection details.

3. System Interface

The system package provides a devnet-agnostic programmatic interface, constructed for example from the descriptors above, for interacting with Optimism networks. Key features include:

  • Unified interface for L1 and L2 chain interactions
  • Transaction management and processing
  • Wallet management and operations
  • Contract interaction capabilities
  • Interoperability features between different L2 chains

Core interfaces include:

  • System: Represents a complete Optimism system with L1 and L2 chains
  • Chain: Provides access to chain-specific operations
  • Wallet: Manages accounts, transactions, and signing operations
  • Transaction: Handles transaction creation and processing

4. Testing Framework

The testing package provides a comprehensive framework for testing devnet deployments:

  • Standardized testing utilities
  • System test capabilities
  • Integration test helpers
  • Test fixtures and utilities

Devnet Descriptors

The devnet descriptor is a standardized format that describes the complete topology and configuration of an Optimism devnet deployment. This standard serves as a bridge between different devnet implementations and the higher-level tooling provided by the devnet-sdk.

Universal Descriptor Format

Both kurtosis-devnet and netchef emit the same descriptor format, despite being completely different devnet implementations:

  • kurtosis-devnet: Uses Kurtosis to orchestrate containerized devnet deployments
  • netchef: Provides a lightweight, local devnet deployment

This standardization enables a powerful ecosystem where tools can be built independently of the underlying devnet implementation.

Descriptor Structure

A devnet descriptor provides a complete view of a running devnet:

{
  "l1": {
    "name": "l1",
    "id": "900",
    "services": {
      "geth": {
        "name": "geth",
        "endpoints": {
          "rpc": {
            "host": "localhost",
            "port": 8545
          }
        }
      }
    },
    "nodes": [...],
    "addresses": {
      "deployer": "0x...",
      "admin": "0x..."
    },
    "wallets": {
      "admin": {
        "address": "0x...",
        "private_key": "0x..."
      }
    }
  },
  "l2": [
    {
      "name": "op-sepolia",
      "id": "11155420",
      "services": {...},
      "nodes": [...],
      "addresses": {...},
      "wallets": {...}
    }
  ],
  "features": ["eip1559", "shanghai"]
}

Enabling Devnet-Agnostic Tooling

The power of the descriptor format lies in its ability to make any compliant devnet implementation immediately accessible to the entire devnet-sdk toolset:

  1. Universal Interface: Any devnet that can emit this descriptor format can be managed through devnet-sdk's tools
  2. Automatic Integration: New devnet implementations only need to implement the descriptor format to gain access to:
    • System interface for chain interaction
    • Testing framework
    • Shell integration tools
    • Wallet management
    • Transaction processing

Benefits

This standardization provides several key advantages:

  • Portability: Tools built against the devnet-sdk work with any compliant devnet implementation
  • Consistency: Developers get the same experience regardless of the underlying devnet
  • Extensibility: New devnet implementations can focus on deployment mechanics while leveraging existing tooling
  • Interoperability: Tools can be built that work across different devnet implementations

Implementation Requirements

To make a devnet implementation compatible with devnet-sdk, it needs to:

  1. Provide a mechanism to output the descriptor (typically as JSON)
  2. Ensure all required services and endpoints are properly described

Once these requirements are met, the devnet automatically gains access to the full suite of devnet-sdk capabilities.

Status

The descriptor format is currently in active development, particularly regarding endpoint specifications:

Endpoint Requirements

  • Current State: The format does not strictly specify which endpoints must be included in a compliant descriptor
  • Minimum Known Requirements:
    • RPC endpoints are essential for basic chain interaction
    • Other endpoints may be optional depending on use case

Implementation Notes

  • kurtosis-devnet currently outputs all service endpoints by default, including many that may not be necessary for testing
  • Other devnet implementations can be more selective about which endpoints they expose
  • Different testing scenarios may require different sets of endpoints

Future Development

We plan to develop more specific recommendations for:

  • Required vs optional endpoints
  • Standard endpoint naming conventions
  • Service-specific endpoint requirements
  • Best practices for endpoint exposure

Until these specifications are finalized, devnet implementations should:

  1. Always include RPC endpoints
  2. Document which additional endpoints they expose
  3. Consider their specific use cases when deciding which endpoints to include

Example Usage

Here's how a tool might use the descriptor to interact with any compliant devnet:

// Load descriptor from any compliant devnet
descriptor, err := descriptors.Load("devnet.json")
if err != nil {
    log.Fatal(err)
}

// Use the descriptor with devnet-sdk tools
system, err := system.FromDescriptor(descriptor)
if err != nil {
    log.Fatal(err)
}

// Now you can use all devnet-sdk features
l1 := system.L1()
l2 := system.L2(descriptor.L2[0].ID)

This standardization enables a rich ecosystem of tools that work consistently across different devnet implementations, making development and testing more efficient and reliable.

Shell Integration

The devnet-sdk provides powerful shell integration capabilities that allow developers to "enter" a devnet environment, making interactions with the network more intuitive and streamlined.

Devnet Shell Environment

Using a devnet's descriptor, we can create a shell environment that is automatically configured with all the necessary context to interact with the devnet:

# Enter a shell configured for your devnet
devnet-sdk shell --descriptor path/to/devnet.json

Automatic Configuration

When you enter a devnet shell, the environment is automatically configured with:

  • Environment variables for RPC endpoints
  • JWT authentication tokens where required
  • Named wallet addresses
  • Chain IDs
  • Other devnet-specific configuration

Simplified Tool Usage

This automatic configuration enables seamless use of Ethereum development tools without explicit endpoint configuration:

# Without devnet shell
cast balance 0x123... --rpc-url http://localhost:8545 --jwt-secret /path/to/jwt

# With devnet shell
cast balance 0x123...  # RPC and JWT automatically configured

Supported Tools

The shell environment enhances the experience with various Ethereum development tools:

  • cast: For sending transactions and querying state

Environment Variables

The shell automatically sets up standard Ethereum environment variables based on the descriptor:

# Chain enpointpoit
export ETH_RPC_URL=...
export ETH_JWT_SECRET=...

Usage Examples

# Enter devnet shell
go run devnet-sdk/shell/cmd/enter/main.go --descriptor devnet.json --chain ...

# Now you can use tools directly
cast block latest

# Exit the shell
exit

Benefits

  • Simplified Workflow: No need to manually configure RPC endpoints or authentication
  • Consistent Environment: Same configuration across all tools and commands
  • Reduced Error Risk: Eliminates misconfigurations and copy-paste errors
  • Context Awareness: Shell knows about all chains and services in your devnet

Implementation Details

The shell integration:

  1. Reads the descriptor file
  2. Sets up environment variables based on the descriptor content
  3. Creates a new shell session with the configured environment
  4. Maintains the environment until you exit the shell

This feature makes it significantly easier to work with devnets by removing the need to manually manage connection details and authentication tokens.

System Interfaces

The devnet-sdk provides a set of Go interfaces that abstract away the specifics of devnet deployments, enabling automation solutions to work consistently across different deployment types and implementations.

Core Philosophy

While the Descriptor interfaces provide a common way to describe actual devnet deployments (like production-like or Kurtosis-based deployments), the System interfaces operate at a higher level of abstraction. They are designed to support both real deployments and lightweight testing environments.

The key principles are:

  • Deployment-Agnostic Automation: Code written against these interfaces works with any implementation - from full deployments described by Descriptors to in-memory stacks or completely fake environments
  • Flexible Testing Options: Enables testing against:
    • Complete devnet deployments
    • Partial mock implementations
    • Fully simulated environments
  • One-Way Abstraction: While Descriptors can be converted into System interfaces, System interfaces can represent additional constructs beyond what Descriptors describe
  • Implementation Freedom: New deployment types or testing environments can be added without modifying existing automation code

Interface Purity

A critical design principle of these interfaces is their purity. This means that interfaces:

  1. Only Reference Other Pure Interfaces: Each interface method can only return or accept:

    • Other pure interfaces from this package
    • Simple data objects that can be fully instantiated
    • Standard Go types and primitives

  2. Avoid Backend-Specific Types: The interfaces never expose types that would create dependencies on specific implementations:

    // BAD: Creates dependency on specific client implementation
func (c Chain) GetNodeClient() *specific.NodeClient

// GOOD: Returns pure interface that can be implemented by any backend
func (c Chain) Client() (ChainClient, error)

  3. Use Generic Data Types: When complex data structures are needed, they are defined as pure data objects:

    // Pure data type that any implementation can create
type TransactionData interface {
    From() common.Address
    To() *common.Address
    Value() *big.Int
    Data() []byte
}

Why Purity Matters

Interface purity is crucial because it:

  • Preserves implementation freedom
  • Prevents accidental coupling to specific backends
  • Enables creation of new implementations without constraints
  • Allows mixing different implementation types (e.g., partial fakes)

Example: Maintaining Purity

// IMPURE: Forces dependency on eth client
type Chain interface {
    GetEthClient() *ethclient.Client  // 👎 Locks us to specific client
}

// PURE: Allows any implementation
type Chain interface {
    Client() (ChainClient, error)     // 👍 Implementation-agnostic
}

type ChainClient interface {
    BlockNumber(ctx context.Context) (uint64, error)
    // ... other methods
}

Interface Hierarchy

System

The top-level interface representing a complete Optimism deployment:

type System interface {
    // Unique identifier for this system
    Identifier() string
    
    // Access to L1 chain
    L1() Chain
    
    // Access to L2 chain(s)
    L2(chainID uint64) Chain
}

Chain

Represents an individual chain (L1 or L2) within the system:

type Chain interface {
    // Chain identification
    RPCURL() string
    ID() types.ChainID
    
    // Core functionality
    Client() (*ethclient.Client, error)
    Wallets(ctx context.Context) ([]Wallet, error)
    ContractsRegistry() interfaces.ContractsRegistry
    
    // Chain capabilities
    SupportsEIP(ctx context.Context, eip uint64) bool
    
    // Transaction management
    GasPrice(ctx context.Context) (*big.Int, error)
    GasLimit(ctx context.Context, tx TransactionData) (uint64, error)
    PendingNonceAt(ctx context.Context, address common.Address) (uint64, error)
}

Wallet

Manages accounts and transaction signing:

type Wallet interface {
    // Account management
    PrivateKey() types.Key
    Address() types.Address
    Balance() types.Balance
    Nonce() uint64
    
    // Transaction operations
    Sign(tx Transaction) (Transaction, error)
    Send(ctx context.Context, tx Transaction) error
    
    // Convenience methods
    SendETH(to types.Address, amount types.Balance) types.WriteInvocation[any]
    Transactor() *bind.TransactOpts
}

Implementation Types

The interfaces can be implemented in various ways to suit different needs:

1. Real Deployments

  • Kurtosis-based: Full containerized deployment
  • Netchef: Remote devnet deployment

2. Testing Implementations

  • In-memory: Fast, lightweight implementation for unit tests
  • Mocks: Controlled behavior for specific test scenarios
  • Recording: Record and replay real interactions

3. Specialized Implementations

  • Partial: Combining pieces from fake and real deployments
  • Filtered: Limited functionality for specific use cases
  • Instrumented: Added logging/metrics for debugging

Usage Examples

Writing Tests

The System interfaces are primarily used through our testing framework. See the Testing Framework documentation for detailed examples and best practices.

Creating a Mock Implementation

type MockSystem struct {
    l1 *MockChain
    l2Map map[uint64]*MockChain
}

func NewMockSystem() *MockSystem {
    return &MockSystem{
        l1: NewMockChain(),
        l2Map: make(map[uint64]*MockChain),
    }
}

// Implement System interface...

Benefits

  • Abstraction: Automation code is isolated from deployment details
  • Flexibility: Easy to add new deployment types
  • Testability: Support for various testing approaches
  • Consistency: Same interface across all implementations
  • Extensibility: Can add specialized implementations for specific needs

Best Practices

  1. Write Against Interfaces: Never depend on specific implementations
  2. Use Context: For proper cancellation and timeouts
  3. Handle Errors: All operations can fail
  4. Test Multiple Implementations: Ensure code works across different types
  5. Consider Performance: Choose appropriate implementation for use case

The System interfaces provide a powerful abstraction layer that enables writing robust, deployment-agnostic automation code while supporting a wide range of implementation types for different use cases.

Testing Framework

The devnet-sdk provides a comprehensive testing framework designed to make testing against Optimism devnets both powerful and developer-friendly.

Testing Philosophy

Our testing approach is built on several key principles:

1. Native Go Tests

Tests are written as standard Go tests, providing:

  • Full IDE integration
  • Native debugging capabilities
  • Familiar testing patterns
  • Integration with standard Go tooling
func TestSystemWrapETH(t *testing.T) {
    // Standard Go test function
    systest.SystemTest(t, wrapETHScenario(...))
}

2. Safe Test Execution

Tests are designed to be safe and self-aware:

  • Tests verify their prerequisites before execution
  • Tests skip gracefully when prerequisites aren't met
  • Clear distinction between precondition failures and test failures
// Test will skip if the system doesn't support required features
walletGetter, fundsValidator := validators.AcquireL2WalletWithFunds(
    chainIdx,
    types.NewBalance(big.NewInt(1.0 * constants.ETH)),
)

3. Testable Scenarios

Test scenarios themselves are designed to be testable:

  • Scenarios work against any compliant System implementation
  • Mocks and fakes can be used for scenario validation
  • Clear separation between test logic and system interaction

4. Framework Integration

The systest package provides integration helpers that:

  • Handle system acquisition and setup
  • Manage test context and cleanup
  • Provide precondition validation
  • Enable consistent test patterns

Example Test

Here's a complete example showing these principles in action:

import (
    "math/big"
    
    "github.com/ethereum-optimism/optimism/devnet-sdk/contracts/constants"
    "github.com/ethereum-optimism/optimism/devnet-sdk/system"
    "github.com/ethereum-optimism/optimism/devnet-sdk/testing/systest"
    "github.com/ethereum-optimism/optimism/devnet-sdk/testing/testlib/validators"
    "github.com/ethereum-optimism/optimism/devnet-sdk/types"
    "github.com/ethereum-optimism/optimism/op-service/testlog"
    "github.com/ethereum/go-ethereum/log"
    "github.com/stretchr/testify/require"
)

// Define test scenario as a function that works with any System implementation
func wrapETHScenario(chainIdx uint64, walletGetter validators.WalletGetter) systest.SystemTestFunc {
    return func(t systest.T, sys system.System) {
        ctx := t.Context()

        logger := testlog.Logger(t, log.LevelInfo)
        logger := logger.With("test", "WrapETH", "devnet", sys.Identifier())

        // Get the L2 chain we want to test with
        chain := sys.L2(chainIdx)
        logger = logger.With("chain", chain.ID())
        
        // Get a funded wallet for testing
        user := walletGetter(ctx)
        
        // Access contract registry
        wethAddr := constants.SuperchainWETH
        weth, err := chain.ContractsRegistry().SuperchainWETH(wethAddr)
        require.NoError(t, err)
        
        // Test logic using pure interfaces
        funds := types.NewBalance(big.NewInt(0.5 * constants.ETH))
        initialBalance, err := weth.BalanceOf(user.Address()).Call(ctx)
        require.NoError(t, err)
        
        require.NoError(t, user.SendETH(wethAddr, funds).Send(ctx).Wait())
        
        finalBalance, err := weth.BalanceOf(user.Address()).Call(ctx)
        require.NoError(t, err)
        
        require.Equal(t, initialBalance.Add(funds), finalBalance)
    }
}

func TestSystemWrapETH(t *testing.T) {
    chainIdx := uint64(0) // First L2 chain
    
    // Setup wallet with required funds - this acts as a precondition
    walletGetter, fundsValidator := validators.AcquireL2WalletWithFunds(
        chainIdx,
        types.NewBalance(big.NewInt(1.0 * constants.ETH)),
    )
    
    // Run the test with system management handled by framework
    systest.SystemTest(t,
        wrapETHScenario(chainIdx, walletGetter),
        fundsValidator,
    )
}

Framework Components

1. Test Context Management

The framework provides context management through systest.T:

  • Proper test timeouts
  • Cleanup handling
  • Resource management
  • Logging context

2. Precondition Validators

Validators ensure test prerequisites are met:

// Validator ensures required funds are available
fundsValidator := validators.AcquireL2WalletWithFunds(...)

3. System Acquisition

The framework handles system creation and setup:

systest.SystemTest(t, func(t systest.T, sys system.System) {
    // System is ready to use
})

4. Resource Management

Resources are properly managed:

  • Automatic cleanup
  • Proper error handling
  • Context cancellation

Best Practices

  1. Use Scenarios: Write reusable test scenarios that work with any System implementation
  2. Validate Prerequisites: Always check test prerequisites using validators
  3. Handle Resources: Use the framework's resource management
  4. Use Pure Interfaces: Write tests against the interfaces, not specific implementations
  5. Proper Logging: Use structured logging with test context
  6. Clear Setup: Keep test setup clear and explicit
  7. Error Handling: Always handle errors and provide clear failure messages

Introduction

The devnet-sdk DSL is a high level test library, specifically designed for end to end / acceptance testing of the OP Stack. It aims to make the development and maintenance of whole system tests faster and easier.

The high level API helps make the actual test read in a more declarative style and separate the technical details of how an action is actually performed. The intended result is that tests express the requirements, while the DSL provides the technical details of how those requirements are met. This ensures that as the technical details change, the DSL can be updated rather than requiring that each test be updated individual - significantly reducing the maintenance cost for a large test suite. Similarly, if there is flakiness in tests, it can often be solved by improving the DSL to properly wait for pre or post conditions or automatically perform required setup steps and that fix is automatically applied everywhere, including tests added in the future.

Guiding Principles

These guiding principles allow the test suite to evolve and grow over time in a way that ensures the tests are maintainable and continue to be easy to write. With multiple different teams contributing to tests, over a long time period, shared principles are required to avoid many divergent approaches and frameworks emerging which increase the cognitive load for developers writing tests and increase the maintenance costs for existing tests.

Keep It Simple

Avoid attempting to make the DSL read like plain English. This is a domain-specific language and the domain experts are actually the test developers, not non-technical users. Each statement should clearly describe what it is trying to do, but does not need to read like an English sentence.

Bias very strongly towards making the tests simpler, even if the DSL implementation then needs to be more complex. Complexity in tests will be duplicated for each test case whereas complexity in the DSL is more centralised and is encapsulated so it is much less likely to be a distraction.

Consistency

The "language" of the DSL emerges by being consistent in the structures and naming used for things. Take the time to refactor things to ensure that the same name is used consistently for a concept right across the DSL.

Bias towards following established patterns rather than doing something new. While introducing a new pattern might make things cleaner in a particular test, it introduces additional cognitive load for people to understand when working with the tests. It is usually (but not always) better to preserve consistency than to have a marginally nicer solution for one specific scenario.

The style guide defines a set of common patterns and guidelines that should be followed.

DSL Style Guide

This style guide outlines common patterns and anti-patterns used by the testing DSL. Following this guide not only improves consistency, it helps keep the separation of requirements (in test files) from implementation details (in DSL implementation), which in turn ensures tests are maintainable even as the number of tests keeps increasing over time.

Entry Points

What are the key entry points for the system? Nodes/services, users, contracts??

Action Methods

Methods that perform actions will typically have three steps:

  1. Check (and if needed, wait) for any required preconditions
  2. Perform the action, allowing components to fully process the effects of it
  3. Assert that the action completed. These are intended to be a sanity check to ensure tests fail fast if something doesn't work as expected. Options may be provided to perform more detailed or specific assertions

Verification Methods

Verification methods in the DSL provide additional assertions about the state of the system, beyond the minimal assertions performed by action methods.

Verification methods should include any required waiting or retrying.

Verification methods should generally only be used in tests to assert the specific behaviours the test is covering. Avoid adding additional verification steps in a test to assert that setup actions were performed correctly - such assertions should be built into the action methods. While sanity checking setup can be useful, adding additional verification method calls into tests makes it harder to see what the test is actually intending to cover and increases the number of places that need to be updated if the behaviour being verified changes in the future.

Avoid Getter Methods

The DSL generally avoids exposing methods that return data from the system state. Instead verification methods are exposed which combine the fetching and assertion of the data. This allows the DSL to handle any waiting or retrying that may be necessary (or become necessary). This avoids a common source of flakiness where tests assume an asynchronous operation will have completed instead of explicitly waiting for the expected final state.

// Avoid: the balance of an account is data from the system which changes over time
block := node.GetBalance(user)

// Good: use a verification method
node.VerifyBalance(user, 10 * constants.Eth)

// Better? Select the entry point to be as declarative as possible
user.VerifyBalacne(10 * constants.Eth) // implementation could verify balance on all nodes automatically

Note however that this doesn't mean that DSL methods never return anything. While returning raw data is avoided, returning objects that represent something in the system is ok. e.g.

claim := game.RootClaim()

// Waits for op-challenger to counter the root claim and returns a value representing that counter claim
// which can expose further verification or action methods.
counter := claim.VerifyCountered()
counter.VerifyClaimant(honestChallenger)
counter.Attack()

Method Arguments

Required inputs to methods are specified as normal parameters, so type checking enforces their presence.

Optional inputs to methods are specified by a config struct and accept a vararg of functions that can update that struct. This is roughly inline with the typical opts pattern in Golang but with significantly reduced boilerplate code since so many methods wil define their own config. With* methods are only provided for the most common optional args and tests will normally supply a custom function that sets all the optional values they need at once.

Logging

Include logging to indicate what the test is doing within the DSL methods.

Methods that wait should log what they are waiting for and the current state of the system on each poll cycle.

No Sleeps

Neither tests nor DSL code should use hard coded sleeps. CI systems tend to be under heavy and unpredictable load so short sleep times lead to flaky tests when the system is slower than expected. Long sleeps waste time, causing test runs to be too slow. By using a waiter pattern, a long timeout can be applied to avoid flakiness, while allowing the test to progress quickly once the condition is met.

// Avoid: arbitrary delays
node.DoSomething()
time.Sleep(2 * time.Minute)
node.VerifyResult()

// Good: build wait/retry loops into the testlib method
node.DoSomething()
node.VerifyResult() // Automatically waits

Test Smells

"Smells" are patterns that indicate there may be a problem. They aren't hard rules, but indicate that something may not be right and the developer should take a little time to consider if there are better alternatives.

Comment and Code Block

Where possible, test code should be self-explanatory with testlib method calls that are high level enough to not need comments explaining what they do in the test. When comments are required to explain simple setup, it's an indication that the testlib method is either poorly named or that a higher level method should be introduced.

// Smelly: Test code is far too low level and needs to be explained with a comment
// Deploy test contract
storeProgram := program.New().Sstore(0, 0xbeef).Bytes()
walletv2, err := system.NewWalletV2FromWalletAndChain(ctx, wallet, l2Chain)
require.NoError(t, err)
storeAddr, err := DeployProgram(ctx, walletv2, storeProgram)
require.NoError(t, err)
code, err := l2Client.CodeAt(ctx, storeAddr, nil)
require.NoError(t, err)
require.NotEmpty(t, code, "Store contract not deployed")
require.Equal(t, code, storeProgram, "Store contract code incorrect")

// Good: Introduce a testlib method to encapsulate the detail and keep the test high level
contract := contracts.SStoreContract.Deploy(l2Node, 0xbeef)

However, not all comments are bad:

// Good: Explain the calculations behind specific numbers
// operatorFeeCharged = gasUsed * operatorFeeScalar == 1000 * 5 == 5000
tx.VerifyOperatorFeeCharged(5000)