Release: sc - 0.42.0, codec 0.18.0, vm 0.4.0, scenario-format 0.20.0, sdk 0.2.0

Required rustc version: rustc 1.71.0-nightly or higher.

Multi-endpoints in multi-contracts

Until this release, it was possible to build contract variants with only part of the endpoints and with different configurations, but it was not possible to have different versions of the same endpoint deployed in different variants.

#[init]
fn default_init(&self, sample_value: BigUint) {
    self.sample_value().set(sample_value);
}

#[init]
#[label("alt-impl")]
fn alternative_init(&self) -> &'static str {
    "alternative init"
}

For instance, in this example the alt-impl version offers a different implementation of the sample_value getter. This is important, because it opens the door for versioning functionality in contracts using the multi-contract build system.

A special case of this is versioning the contract constructor. This can be used in having contract upgrade code built from the same contract source. In this example we have 2 variants of the contract built, each with its own constructor.

‍

One could, for instance, be the one we use when deploying a fresh contract, while the other for upgrades.

Major architectural redesign of the debugger

The debugger is composed of two parts: an environment for smart contract execution, as well as a replica of the main components of the Go VM. Previously they were not clearly delimited, moreoever several components of the Rust VM were depending on the smart contract framework.

To clear up the architecture, we decided to imitate the Go system in Rust, and reuse the same boundaries that we have between the Go VM and the executor (Wasmer). This boundary is made up of a number of interfaces:

• Executor and Instance, which allow the VM to call contracts when needed;
• VMHooks, which allows the contracts to call functionality from the VM.

On the VM side, the connection is established through the VMHooksDispatcher object and the VMHooksHandler interface. Most of the logic concerning SC inputs, outputs and managed types was moved here.

On the smart contract side, a new VMHooksApi is utilized for connecting with the VM, via this VMHooks interface. The VMHooksApi is available in various flavors or backends to cater to different requirements:

1. The old DebugApi is now exclusively used at runtime, specifically on the VM context stack.
2. We are introducing the new StaticApi, which provides support for managed types within a regular context, eliminating the need for initialization. This is particularly useful for setting up tests and interactors.
3. We provide the additional SingleTxApi , particularly beneficial for unit tests. This flavor not only supports managed types but also offers a basic context for transaction inputs, results, storage, and block information.

While performing these architectural changes, we also removed most of the legacy functionalities from the smart contract APIs. None of them had been in use for more than a year.

This streamlining enhances the efficiency and maintainability of the system while providing a robust foundation for future developments and improvements.

System SC mock

A system smart contract has been mocked for use in integration tests. The contract now supports issuing various types of tokens, including fungible, semi-fungible (SFT), and non-fungible (NFT) tokens. Additionally, roles can be set for these tokens. While some methods have been implemented, there are still methods to be developed in the future.

Integration of blackbox and whitebox testing into one unified framework

We use the term “blackbox” to refer to any test that only works with the public interface of a contract, i.e. its public endpoints. It imitates execution on a real blockchain, where contracts are compiled and there is no direct access to private code. In contract, “whitebox” testing refers to any system that has access to private methods and fields in contracts. This is a less realistic setup, but it allows developers to write unit and semi-integration tests.

The old Rust testing framework had a whitebox mindset, whereas everything built on the MultiversX Scenario model is blackbox by construction, including the new Rust testing framework.

We would like developers to move over to the new framework, since it is more reliable and better featured. For most tests, we believe blackbox testing is the way to go and enough. We did, however, want to provide a whitebox option for developers migrating from the old framework, or looking to write unit tests. The most important part is that we managed to unify these functionalities under one overarching framework.

In the code snippet below, you can observe the similarity between the two approaches:

 
let mut world = world();
let adder_code = world.code_expression(ADDER_PATH_EXPR);

// Blackbox
world
    .set_state_step(
        SetStateStep::new()
            .put_account("address:owner", Account::new().nonce(1))
            .new_address("address:owner", 1, "sc:adder"),
    )
    .sc_deploy(
        ScDeployStep::new()
            .from("address:owner")
            .code(adder_code)
            .argument("5")
            .expect(TxExpect::ok().no_result()),
    )
    
// Whitebox
let adder_whitebox = WhiteboxContract::new("sc:adder", adder::contract_obj);
world
    .set_state_step(
        SetStateStep::new()
            .put_account("address:owner", Account::new().nonce(1))
            .new_address("address:owner", 1, "sc:adder"),
    )
    .whitebox_deploy(
        &adder_whitebox,
        ScDeployStep::new().from("address:owner").code(adder_code),
        |sc| {
            sc.init(5u32.into());
        },
    )
	

To ensure the compatibility and reliability of the new whitebox framework, we extensively tested it by injecting it into the implementation of the old testing framework and running the existing tests.

The old Rust testing framework is deprecated starting with this release. Its usage is still allowed, but developers will receive deprecation warnings. Continued support is not guaranteed in the long run.

Interactors can now export a trace of their execution, thus producing integration tests.

Interactors now boast the exceptional capability to export a detailed trace of their execution, offering a streamlined approach to creating integration tests.

Consider the following example, which demonstrates the initialization of the interactor and interaction with the real blockchain:

 
// Define the path for the scenario trace file
const INTERACTOR_SCENARIO_TRACE_PATH: &str = "interactor_trace.scen.json";

// Initialize the interactor with the specified gateway configuration and tracer
let mut interactor = Interactor::new(config.gateway())
    .await
    .with_tracer(INTERACTOR_SCENARIO_TRACE_PATH)
    .await;
	

Additionally, our integrated tool effortlessly retrieves the initial states of the involved accounts directly from the blockchain:

 
async fn set_state(&mut self) {
    // Retrieve account details as a scenario to set the state
    let scenario_raw = retrieve_account_as_scenario_set_state(
        Config::load_config().gateway().to_string(),
        bech32::encode(&self.wallet_address),
        true,
    )
    .await;

    let scenario = Scenario::interpret_from(scenario_raw, &InterpreterContext::default());

    // Run pre and post-scenario runners
    self.interactor.pre_runners.run_scenario(&scenario);
    self.interactor.post_runners.run_scenario(&scenario);
}
	

To provide the necessary configurations for the state and overall system, you can refer to the state.toml and config.toml files, respectively.

Example deploy:

 
async fn deploy(&mut self) {
    // Set the required state
    self.set_state().await;

    // Deploy the smart contract using the interactor
    self.interactor
        .sc_deploy_use_result(
            ScDeployStep::new()
                .call(self.state.default_adder().init(BigUint::from(0u64)))
                .from(&self.wallet_address)
                .code(&self.adder_code),
            |new_address, tr| {
                tr.result.unwrap_or_else(|err| {
                    panic!(
                        "deploy failed: status: {}, message: {}",
                        err.status, err.message
                    )
                });

                let new_address_bech32 = bech32::encode(&new_address);
                println!("new address: {new_address_bech32}");

                let new_address_expr = format!("bech32:{new_address_bech32}");
                self.state.set_adder_address(&new_address_expr);
            },
        )
        .await;
}
	

Once executed, this code will produce a trace in the specified format, which can be directly used as a scenario for later testing.

 
{
    "steps": [
        {
            "step": "setState",
            "accounts": {
                "0xe32afedc904fe1939746ad973beb383563cf63642ba669b3040f9b9428a5ed60": {
                    "nonce": "481",
                    "balance": "106274669842530000003",
                    "esdt": {
                        "str:CAN-14dc0a": "1000",
                        "str:CAN-2abf4b": "1000",
                        "str:CAN-6d39e6": "1000",
                        "str:CAN-ac1592": "1000"
                    },
                    "username": ""
                }
            }
        },
        {
            "step": "setState",
            "newAddresses": [
                {
                    "creatorAddress": "0xe32afedc904fe1939746ad973beb383563cf63642ba669b3040f9b9428a5ed60",
                    "creatorNonce": "481",
                    "newAddress": "0x0000000000000000050028600ceb73ac22ec0b6f257aff7bed74dffa3ebfed60"
                }
            ]
        },
        {
            "step": "scDeploy",
            "id": "",
            "tx": {
                "from": "0xe32afedc904fe1939746ad973beb383563cf63642ba669b3040f9b9428a5ed60",
                "contractCode": "file:../output/adder.wasm",
                "arguments": [
                    "0x00"
                ],
                "gasLimit": "70,000,000",
                "gasPrice": ""
            },
            "expect": {
                "status": "0"
            }
        }
    ]
}
	

Interactors can now execute several steps (calls, deploys) in parallel

Interactors have gained the ability to execute multiple steps (calls, deploys) simultaneously.

This is achieved by calling the multi_sc_exec function, which offers the convenience of parallel execution. All that's needed is a list of steps, comprising either deployment or calls. Below, you'll find some usage examples:

Example:

 
// Deployments
self.interactor
    .multi_sc_exec(StepBuffer::from_sc_deploy_vec(&mut steps))
    .await;

// Calls
self.interactor
    .multi_sc_exec(StepBuffer::from_sc_call_vec(&mut steps))
    .await;
	

‍

With this new capability, executing tasks efficiently and concurrently becomes a seamless process.

Redesign of the wrappers around the Rust and Go JSON scenario executors

Our goal for the testing framework is to for developers to “write once, run everywhere”. For this reason, it is important to have a common API facing the developers, but with switchable backends.

The ScenarioRunner interface serves as an essential abstraction between the test API and the various backends used for running tests.

The available backends are as follows:

1. DebuggerBackend: This backend efficiently coordinates the execution of scenario tests, leveraging the Rust implementation of the VM and enabling direct contract execution.
2. ScenarioWorld: Acting as a convenient facade for contract tests, this backend encapsulates all the necessary context required to execute scenarios involving contracts. Presently, most operations are delegated to the blockchain mock object directly, with plans for future refactoring and decomposition into smaller components.

3. ScenarioRunnerList: This backend aggregates multiple scenario runners into one entity and executes them in a sequential order. It even includes an empty object that can act as a placeholder, allowing the provision of a ScenarioRunner that performs no actions when needed.

4. ScenarioTraceFile and ScenarioTrace: These backends handle the loading and writing of scenario traces, making it convenient to manage and store test scenario information.

5. ScenarioVMRunner: As an implementation of the StepRunner interface, this backend wraps calls to the blockchain mock, offering a streamlined approach to execute and monitor steps efficiently.

By utilizing the ScenarioRunner interface in conjunction with these diverse backends, the testing process becomes flexible, maintainable, and adaptable to various test scenarios.

Redesigned syntax of both the testing and the interactor (snippets) frameworks

Although the codebases remain separate, with the latter implemented in async Rust, both share the same method names and arguments, leveraging the scenario infrastructure.

Example:

 
// Tests
pub fn sc_call_get_result(
    &mut self,
    mut step: TypedScCall,
)

// Interactors
pub async fn sc_call_get_result(
    &mut self,
    mut step: TypedScCall,
)
	

‍

We have introduced new methods that empower you to effortlessly chain scenario steps, ensuring efficient result processing throughout the process.

Example:

 
pub fn sc_call_use_result(
    &mut self,
    step: TypedScCall,
    use_result: F,
) -> &mut Self
where
    OriginalResult: TopEncodeMulti,
    RequestedResult: CodecFrom,
    F: FnOnce(TypedResponse)
	

‍

In order to make the code more concise and readable, several defaults have been introduced in the syntax. For instance:

• Code metadata defaults to "all" ;
• By default the transaction is checked to be successful (Ok), but the results are not checked;
• Gas limit is set to 5,000,000 by default.

The old testing framework has been deprecated.

All contract interactors and tests have been updated to use the new syntax. Additionally, the snippets generator has been upgraded to produce code that adheres to the new syntax.

These advancements ensure a streamlined and unified experience across both codebases, making it easier to work with the scenario infrastructure. This results in more efficient development and testing processes, ultimately enhancing the overall performance and reliability of the system.

We plan to actually create a unified API for both at some point in the future, but we are waiting for the Rust compiler to stabilize async traits first.