[Draft] EIP-X: Off-Chain Data Write Protocol

[NameSys]

Hello ENS & ETH Developers

We at NameSys have been working on an updated version of CCIP-Write protocol, which is intended to expand the stagnant EIP-5559 standard to meet the security-oriented needs of a wide array of centralised and decentralised storages. Please take a look at the draft below and offer your valuable feedback

GitHub: Draft

---

Off-Chain Data Write Protocol

Update to Cross-Chain Write Deferral Protocol (EIP-5559) incorporating secure write deferrals to centralised databases and decentralised & mutable storages

Abstract

The following proposal is a generalised revision to EIP-5559: Off-Chain Write Deferral Protocol, targeting a wider set of storage types and introducing security measures to consider for secure off-chain write deferral and retrieval. While EIP-5559 is limited to referring write operations to L2 EVM chains and centralised databases, methods in this document enable secure write deferral to generic decentralised storages - mutable or immutable - such as IPFS, Arweave, Swarm etc. This draft alongside EIP-3668 is a significant step toward a complete and secure infrastructure for off-chain data retrieval and write deferral.

Motivation

EIP-3668, or ‘CCIP-Read’ in short, has been key to retrieving off-chain data for a variety of contracts on Ethereum blockchain, ranging from price feeds for DeFi contracts, to more recently records for ENS users. The latter case is more interesting since it dedicatedly uses off-chain storage to bypass the usually high gas fees associated with on-chain storage; this aspect has a plethora of use cases well beyond ENS records and a potential for significant impact on universal affordability and accessibility of Ethereum.

Off-chain data retrieval through EIP-3668 is a relatively simpler task since it assumes that all relevant data originating from off-chain storages is translated by CCIP-Read-compliant HTTP gateways; this includes L2 chains, centralised databases or decentralised storages. However, each service leveraging CCIP-Read must handle two main tasks externally:

• Writing this data securely to these storage types on their own, and

• Incorporating reasonable security measures in their CCIP-Read compatible contracts for verifying this data before performing on-chain read or write operations.

Writing to a variety of centralised and decentralised storages is a broader objective compared to CCIP-Read largely due to two reasons:

1. Each storage provider typically has its own architecture that the write operation must comply with, i.e. each have their own specific requirements when it comes to writing data to them, and

2. Each storage must incorporate some form of security measures during write operations so that off-chain data’s integrity can be verified by CCIP-Read contracts during data retrieval stage.

EIP-5559 was the first step toward such a tolerant ‘CCIP-Write’ protocol which outlined how write deferrals could be made to L2 and centralised databases. This proposal extends the previous attempt by including secure write deferrals to decentralised storages, while also updating previous specifications with securer alternatives for writing to centralised databases.

Curious Case of Decentralised Storages

Decentralised storages powered by cryptographic protocols are unique in their diversity of architectures compared to centralised databases or L2 chains, both of which have canonical architectures in place. For instance, write calls to L2 chains can be generalised through the use of ChainID since the callData remains the same; write deferral in this case is as simple as routing the call to another contract on an L2 chain. There is no need to incorporate any additional security requirement(s) since the L2 chain ensures data integrity locally, while the global integrity can be proven by employing a state verifier scheme (e.g. EVM-Gateway) during CCIP-Read calls.

Decentralised storages on the other hand, do not typically have EVM-like environments and may have their own unique content addressing requirements. For example, IPFS, Arweave, Swarm etc all have unique content identification schemes as well as their own specific fine-tunings and/or choices of cryptographic primitives, besides supporting their own cryptographically secured namespaces. This significant and diverse deviation from EVM-like architecture results in an equally diverse set of requirements during both the write deferral operation as well as the subsequent state verifying stage. The resolution of this precise issue is detailed in the following text in an attempt towards a global CCIP-Write specification.

In contrast to EIP-5559, this proposal allows for multiple storage handlers to be nested asynchronously in arbitrary order allowing for maximal interdependence. This feature of interdependence is necessary for highly optimised protocols which employ a mix of two or more storage types at their core. For instance, a service may choose to index cheaply on an L2 while storing the data off-chain entirely; stack-enabled interdependent handlers can achieve such functionality trivially.

Specification

Overview

The following specification revolves around the structure and description of an arbitrary off-chain storage handler tasked with the responsibility of writing to an arbitrary storage. Similar to CCIP-Read, CCIP-Write protocol outlined herein comprises of 2 molecular parts: initial reversion with error event StorageHandledBy__(), which signals the deferral of write operation to the off-chain handler using events, and callback() function, which handles operations following the return of initial write deferral. __ in StorageHandledBy__ is reserved for two uppercased characters encoding the type of data handle, for example, __ = L2 for L2, DB for Database, IP for IPFS, BZ for Swarm, AR for Arweave etc; this 2-character identification scheme can accommodate 1000+ different storage types.

Following EIP-5559, a CCIP-Write deferral call to an arbitrary function setValue(bytes32 key, bytes32 value) can be described in pseudo-code as follows:

// Define Revert Event
error StorageHandledBy__(...)

// Generic function in a contract
function setValueWithConfig(
    bytes32 key, 
    bytes32 value,
    config config, // Metadata pertaining to storage handler __
) external {
    // Defer another write call to storage handler
    revert StorageHandledBy__(
        this.callback.selector, 
        config config,
        ...
    )
}

// Callback receiving status of write call
function callback(bytes response, ...) external view {
    return
} 

Interdependence

The condition of interdependence on storage handlers requires that each handler must have a global config interface in the input argument as well as the return statement. This requires that StorageHandledBy__() must be of the form

• error StorageHandledBy__(bytes config, ...), in addition to
• function callback(bytes config, ...) returns (bytes memory newConfig, ...),

where config and newConfig are responsible for the interdependent behaviour. We’ll specify the optimal typing and encoding for these two interfaces in the next section, although both payloads must have the exact same encoding and may include not only data but also metadata governing the behaviour of subsequent asynchronous calls to nested handlers.

In pseudo-code, interdependent and nested CCIP-Write deferral looks like:

// Define Revert Events for storages X1 and X2
error StorageHandledByX1(
    address sender, 
    config config, // Metadata pertaining to storage handler X1
    bytes callData, 
    bytes4 callback, 
    bytes extraData,
)
error StorageHandledByX2( 
    address sender, 
    config config, // Metadata pertaining to storage handler X2
    bytes callData, 
    bytes4 callback, 
    bytes extraData,
)

// Generic function in a contract
function setValue(
    bytes32 key, 
    bytes32 value,
    config config,
) external {
    // Defer write call to X1 handler
    revert StorageHandledByX1(
        address(this),
        config config,
        abi.encodePacked(value),
        this.callback.selector,
        extraData,
        ...
    )
}

// Callback receiving response from X1
function callback(
    bytes response, 
    config config, 
    bytes extraData
) external view {
    (config config2, bytes puke, bytes extraData2) = calculateOutputForX1(response, config, extraData)
    // Defer another write call to X2 handler
    revert StorageHandledByX2(
        address(this),
        config config2,
        abi.encode(puke),
        this.callback2.selector,
        extraData2,
        ...
    ) || return (config2, puke, ...)
} 

// Callback receiving response from X2
function callback2(
    bytes response2, 
    config config2, 
    bytes extraData2
) external view {
    (config config3, bytes puke2, bytes extraData2) = calculateOutputForX2(response2, config2, extraData2)
    // Defer another write call to X3 handler
    revert StorageHandledByX3(
        address(this),
        config config3,
        abi.encode(puke2),
        this.callback3.selector,
        extraData3,
        ...
    ) || return (config3, puke2, ...)
} 

// Callback receiving response from X3
function callback3(...) external view {
    ...
    return
}

Config Interface

Config interface is dedicated to handling all the metadata that different storages require to defer the calls successfully. Keeping in mind the needs of a broad set of storages, config interface consists of four arrays:

1. coordinates: Coordinates refer to the string-formatted pointers to the target storage. For example, for writing to an L2, its ChainID is sufficient information. For writing to a database or a decentralised storage, the handler’s HTTP URL is sufficient information.

2. authorities: Authorities refer to the addresses of authorities securing the off-chain data. For an L2, the contract address is the authority. For a database or decentralised storage, the data should ideally be signed by an Ethereum private key, which is also the authority in this case. If the authority is stored on-chain by some contracts, then this value is not needed.

3. approvals: Approvals refer to the signatures signed by the corresponding authorities, irrespective of whether they are stored on-chain or not.

4. accessories: Accessories refer to the metadata required to update the off-chain storages, if the storage is wrapped in a namespace. For example, for IPFS wrapped in IPNS, signature from the IPNS private key along with the last sequence number is the required accessory. For databases, login credentials are the accessories although clients must be careful not to pass raw login credentials and use encryption strategies.

// Type of config
type config = [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] 
// Data inside config
bytes[] config = [
        coordinates, // List of string-formatted coordinates, e.g. ChainID for L2, URL for off-chain storage etc; can never be empty
        authorities || [], // List of addresses of authorities; can be empty for unsafe record storage in off-chain databases or decentralised storages, or for on-chain signers
        approvals || [], // List of bytes-like signatures/approvals; can be empty for unsafe record storage in off-chain databases or decentralised storages
        accessories || [] // List of bytes-like access signatures for accessories; usually empty except for decentralised storages wrapped in namespaces
    ]

L2 Handler

L2 handler only requires the list of ChainID values and the corresponding contract addresses.

revert StorageHandledByL2(
    address sender,
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        chains, // List of string-formatted ChainID values
        contracts, // List of contracts on L2
        [], // MUST be empty for L2 storage handler
        [] // MUST be empty for L2 storage handler
    ],
    bytes callData,
    bytes4 this.callback.selector,
    bytes extraData
)

function callback(...) external view {
    ...
    return
}

EXAMPLE

function setValueWithConfig(
    bytes32 key, 
    bytes32 value,
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        chains, 
        contracts,
        [],
        []
    ],
) external {
    revert StorageHandledByL2(
        msg.sender,
        [
            chains,
            contracts,
            [],
            []
        ],
        abi.encodePacked(value),
        this.callback.selector,
        extraData
    )
}

function callback(
    bytes response,
    config config,
    bytes extraData
) external view {
    bytes newConfig = calculateOutputForL2(response, config, extraData)
    return (
        newConfig,
        response == true
    )
}

CALL an L2

setValueWithConfig(
    "avatar", 
    "https://namesys.xyz/logo.png",
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        [
            "11", // ChainID for chain 1
            "25" // ChainID for chain 2
        ], 
        [
            "0xc0ffee254729296a45a3885639AC7E10F9d54979", // Contract address on chain 1
            "0x999999cf1046e68e36E1aA2E0E07105eDDD1f08E" // Contract address on chain 2
        ],
        [], // MUST be empty for L2
        [] // MUST be empty for L2
    ]
)

Database Handler

In the minimal version, a database handler only requires the list of URLs (urls) responsible for the write operations. However, it is strongly advised that all clients employ some sort of verifiable signature scheme and sign the off-chain data; these signatures can be verified during CCIP-Read calls and will prevent possible unauthorised alterations to the data. In such a scenario, the list of signatures (approvals) are needed at the very least; if the signing authority is not stored on-chain, then the address of the authority (authorities) must also be attached.

revert StorageHandledByDB(
    address msg.sender,
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        urls, // List of URLs handling writing to databases
        signers || [], // List of addresses signing the calldata
        approvals || [], // List of signatures approving the signers
        [] // MUST be empty for centralised databases
    ],
    bytes callData,
    bytes4 this.callback.selector,
    bytes extraData
)

function callback(...) external view {
    ...
    return
}

EXAMPLE

function setValueWithConfig(
    bytes32 key, 
    bytes32 value,
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        urls,
        signers,
        approvals,
        []
    ],
) external {
    revert StorageHandledByDB(
        msg.sender,
        abi.encodePacked(value),
        [
            urls, 
            signers,
            approvals,
            []
        ],
        this.callback.selector,
        extraData
    )
}

function callback(
    bytes response,
    config config,
    bytes extraData
) external view {
    bytes newConfig = calculateOutputForDB(response, config, extraData)
    return (
        newConfig,
        response == true
    )
}

CALL a DB

setValueWithConfig(
    "avatar", 
    "https://namesys.xyz/logo.png",
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        [
            "https://db.namesys.xyz", // Database 1   
            "https://db.notapi.dev", // Database 2
        ], 
        [
            "0xc0ffee254729296a45a3885639AC7E10F9d54979", // Ethereum Signer 1
            "0x75b6B7CEE3719850d344f65b24Db4B7433Ca6ee4" // Ethereum Signer 2
        ],
        [
            "0xa6f5e0d78f51c6a80db0ade26cd8bb490e59fc4f24e38845a6d7718246f139d8712be7a3421004a3b12def473d5b9b0d83a0899fb736200a915a1648229cf5e21b", // Approval Signature 1
            "0x8d591768f97f950d1c2cb8a51e4f8718cd154d07e0b60ec955202ac478c45b6f3b745ee136276cbfc4a7c1d7c7cdd0a8e8921395b60c556f0c4857ead0447e351c" // Approval Signature 2
        ],
        [] // MUST be empty for centralised databases
    ]
)

Decentralised Storage Handler

Decentralised storage handlers are the most advanced case and require an equivalent config to database handlers. In addition, such storages are usually immutable at core (e.g. IPFS and Arweave) and therefore employ cryptographic namespaces for static data retrieval. Such namespaces typically have their own access keypairs and their own choices of base-encodings as well as elliptic curves. In order to write to such storages wrapped in namespaces, signature and other relevant metadata (accessories) must be included in the config.

revert StorageHandledByXY(
    address msg.sender,
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        urls, // List of URLs handling write operations to off-chain storages
        signers || [], // List of addresses signing the calldata
        approvals || [], // List of signatures approving the signers
        accessories || [] // List of access signatures for native namespaces
    ],
    bytes callData,
    bytes4 this.callback.selector,
    bytes extraData
)

function callback(...) external view {
    ...
    return
}

EXAMPLE

function setValueWithConfig(
    bytes32 key, 
    bytes32 value,
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        urls,
        signers,
        approvals,
        accessories
    ],
) external {
    revert StorageHandledByXY(
        msg.sender,
        abi.encodePacked(value),
        [
            urls, 
            signers
            approvals,
            accessories
        ],
        this.callback.selector,
        extraData
    )
}

function callback(
    bytes response,
    config config,
    bytes extraData
) external view {
    bytes newConfig = calculateOutputForXY(response, config, extraData)
    return (
        newConfig,
        response == true
    )
}

CALL IPNS and ArNS

setValueWithConfig(
    "avatar", 
    "https://namesys.xyz/logo.png",
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        [
            "https://ipns.namesys.xyz", // IPFS-NS Write Gateway    
            "https://arweave.notapi.dev", // Arweave-NS Write Gateway
        ], 
        [
            "0xc0ffee254729296a45a3885639AC7E10F9d54979", // Ethereum Signer for IPFS
            "0x1CFe432f336cdCAA3836f75A303459E61077068C" // Ethereum Signer for Arweave
        ],
        [
            "0xa6f5e0d78f51c6a80db0ade26cd8bb490e59fc4f24e38845a6d7718246f139d8712be7a3421004a3b12def473d5b9b0d83a0899fb736200a915a1648229cf5e21b", // Approval Signature for IPFS
            "0xf4e42fa7d1125fc149f29ed437e8cbbdac7e31bb493299e03df4d8cfd069c9a96bb12f6186e79ed6bc6e740086a67c0da022ffcd84ef50abf6c0e4f83d53a62d1c" // Approval Signature for Arweave
        ],
        [
            abi.encodePacked(
                "0xa74f6d477c01189834a56b52c8189d6fb228d40e17ef0b255b36848f1432f0bc35b1cf4a2f5390a8aef6c72665b752907be6a979a3ff180d9c13c7983df5d9c2", // Hex-encoded IPNS signature over ed25519 curve
                bytes32(1) // Index or sequence or version number required by IPNS signature payloads; give bytes(0) for empty value
            ), // Requires casting to bytes-like payload by gateway for IPFS-NS
            abi.encodePacked(
                "0x8a055b79515356324f68c18071b22085607d4f37577d53fe5c5c2b0ec9769ef1e70a5bc53f9fe901051e493a216a02ae7952a62488e26fa9547e504af01ef25cd904d853ea409fdf23bec0929caae4926d5e8e5353b4663880a", // Hex-encoded Arweave signature over ed25519 curve
                bytes32(0) // Not required for Arweave
            ) // Requires casting back to base64 by gateway for Arweave
        ]
    ]
)

Nested Handlers

EXAMPLE

function setValueWithConfig(
    bytes32 key, 
    bytes32 value,
    [
        string[], 
        address[], 
        bytes[], 
        bytes[]
    ] [
        urls,
        signers,
        approvals,
        []
    ],
) external {
    // 1st deferral
    revert StorageHandledByDB(
        msg.sender,
        abi.encodePacked(value),
        [
            urls, 
            signers,
            approvals,
            []
        ],
        this.callbackDB.selector,
        extraData
    )
}

// Get response after 1st deferral and post-process
function callbackDB(
    bytes response,
    config config,
    bytes extraData
) external view {
    // Calculate output and access signatures for XY's namespaces
    (
        bytes newExtraData, 
        bytes accessories, 
        config config
    ) = calculateOutputForDB(response, config, extraData)
    // 2nd deferral
    revert StorageHandledByXY(
        msg.sender,
        abi.encodePacked(value),
        [
            config.urls, 
            config.signers
            config.approvals,
            config.accessories,
        ],
        this.callbackXY.selector,
        newExtraData
    )
}

// Get response after 2nd deferral and post-process
function callbackXY(
    bytes response,
    config config,
    bytes extraData
) external view {
    // Calculate final output
    bytes newConfig = calculateOutputForXY(response, config, extraData)
    // Final return
    return (
        newConfig,
        response == true
    )
}

Events

1. A public library must be maintained where each new storage handler supported by a native Protocol Improvement Proposal must register their StorageHandledBy__() identifier. This library could exist on-chain or off-chain; in the end such a list of StorageHandledBy__() identifiers must be the accepted standard for CCIP-Write infrastructure providers (similar to multiformats & multicodec table). If the 2-character space runs out, it can be extended to 3 or more characters without any fear of identifier collisions.

2. Each StorageHandledBy__() provider should be supported with detailed docs of their infrastructure along with a Protocol Improvement Proposal.

[nick.eth]

First of all, thank you for working on standardizing this. It’s an important component that hasn’t had enough attention so far.

I’m no longer a huge fan of the revert-based approach; I think it will likely introduce additional complexity in handling write operations, with apps potentially having to make ‘fake’ eth_call requests to resolvers just in order to find out where the request should actually be sent. I’d personally prefer to see something based around a standard metadata API that can be called to fetch write metadata for a resolver instance.

@NameSys

__ in StorageHandledBy__ is reserved for two uppercased characters encoding the type of data handle, for example, __ = L2 for L2, DB for Database, IP for IPFS, BZ for Swarm, AR for Arweave etc; this 2-character identification scheme can accommodate 1000+ different storage types.

Given that clients would have to support each of these reverts separately, is there any reason to hardcode a 2 character limit? It may be simpler to just standardize each one based on the name that makes the most sense.

Also, if you’re defining a different revert type for each platform, it doesn’t seem like you need a generic interface like ‘config’ - each revert type can have platform-specific arguments instead, which will be clearer and easier to implement.

@NameSys

2529×1576 230 KB

I absolutely love this diagram.

@NameSys

The condition of interdependence on storage handlers requires that each handler must have a global argument in input as well as the return statement. This requires that StorageHandledBy__() must be of the form

• error StorageHandledBy__(bytes input, ...), in addition to
• function callback(bytes input, ...) returns (bytes memory output, ...),

where input and output are the aforementioned arguments responsible for interdependent behaviour.

I’m having trouble understanding what this is meant to convey. Can you rephrase it somehow?

I’m also not really seeing the need for a callback, or chaining multiple write calls. Can you elaborate on the intended use-cases here?

I’m also having trouble understanding the end-to-end flow here. A concrete example of the process to configure and use, say, delegation to an L2, would be useful.

[NameSys]

@nick.eth

Given that clients would have to support each of these reverts separately, is there any reason to hardcode a 2 character limit? It may be simpler to just standardize each one based on the name that makes the most sense.

That is correct. No particular reasons for 2-character limit other than having a short code to remember for your favourite storage. Can be adjusted according to popular demand.

@nick.eth

Also, if you’re defining a different revert type for each platform, it doesn’t seem like you need a generic interface like ‘config’ - each revert type can have platform-specific arguments instead, which will be clearer and easier to implement.

That is true although if multiple storage handlers need to be nested, second handler should be able to trust the digest from the first handler’s callback. On second thoughts, you may be right; callback() can be coded to regenerate config from response…

@nick.eth

I’m having trouble understanding what this is meant to convey. Can you rephrase it somehow?

This has just been updated. input, output was simply config, aka a global interface that two or more handlers could blindly trust.

@nick.eth

I’m also not really seeing the need for a callback, or chaining multiple write calls. Can you elaborate on the intended use-cases here?

This is for the use-case where a service intends to use two handlers for their stack. See below:

@NameSys

I wrote my comment before reading this part and we have ended up saying the same thing. We briefly entertained precisely this (= Index-on-L2 + Store Off-Chain) for NameSys v1 but gave up in interest of simplicity at that point; it is still in our To-Do list for NameSys v2 a little further down the line.

For example, a service that wants to serve the data on some storage but index on L2, it may be necessary to await the storage transaction (such as IPFS/Arweave hash) and then index its transaction hash or some other parameter (e.g. IPFS/Arweave hash) on L2. Nesting was constructed keeping such a multi-pronged service in mind.

@nick.eth

I’m no longer a huge fan of the revert-based approach; I think it will likely introduce additional complexity in handling write operations, with apps potentially having to make ‘fake’ eth_call requests to resolvers just in order to find out where the request should actually be sent. I’d personally prefer to see something based around a standard metadata API that can be called to fetch write metadata for a resolver instance.

I’ll ponder on this more.

[nick.eth]

@NameSys

For example, a service that wants to serve the data on some storage but index on L2, it may be necessary to await the storage transaction (such as IPFS/Arweave hash) and then index its transaction hash or some other parameter (e.g. IPFS/Arweave hash) on L2. Nesting was constructed keeping such a multi-pronged service in mind.

Couldn’t this be provided more simply by having a gateway service perform both operations? I’m concerned about making this standard more involved than it needs to be.

@NameSys

This has just been updated. input, output was simply config, aka a global interface that two or more handlers could blindly trust.

Okay; I’m still not quite sure what’s going on here, though. An end-to-end example would be really helpful.

[NameSys]

@nick.eth

Okay; I’m still not quite sure what’s going on here, though. An end-to-end example would be really helpful.

Yes, I’ll make a self-contained example demonstrating this. Perhaps it might make more sense at that stage.

[NameSys]

@nick.eth

Okay; I’m still not quite sure what’s going on here, though. An end-to-end example would be really helpful.

Here goes a more detailed example:

CALL L2 nested in IPNS

In this explicit example, let’s consider a scenario where a service intends to publish the value of a key off-chain with setValueWithConfig() to an IPNS container. IPNS updates are notoriously frivolous in their uptake rate at long TTLs and can return stale results for a while in some cases even after an update. Due to this reason, the service wants to additionally index (at least) the sequence numbers of IPNS updates on an L2 contract (but may also choose to store the IPFS hash), from where CCIP-Read can force-match the expected latest version with the resolved IPNS records before resolving any request. Such a process according to the draft goes as follows:

1. User makes a request to set value of a key with setValueWithConfig(), and attaches the config with legitimate values of gateway urls, off-chain signers as authorities, approvals for off-chain signers, and accessories required to update their IPNS container (IPNS signature + sequence number).

   config config = [
           ["https://ipns.namesys.xyz"], // Gateway URL
           ["0xc0ffee254729296a45a3885639AC7E10F9d54979"], // Signer address
           ["0xa6f5e0d78f51c6a80db0ade26cd8bb490e59fc4f24e38845a6d7718246f139d8712be7a3421004a3b12def473d5b9b0d83a0899fb736200a915a1648229cf5e21b"], // Ethereum signature by signer
           [
               abi.encodePacked(
                   "0xa74f6d477c01189834a56b52c8189d6fb228d40e17ef0b255b36848f1432f0bc35b1cf4a2f5390a8aef6c72665b752907be6a979a3ff180d9c13c7983df5d9c2", // Hex-encoded IPNS signature over ed25519 curve
                   bytes32(1) // Sequence required by IPNS signature payloads
               )
           ]
       ]
   

2. setValueWithConfig() defers the storage to StorageHandledByIP().

   function setValueWithConfig(
       bytes32 key, 
       bytes32 value,
       config config
   ) external {
       // 1st deferral
       revert StorageHandledByIP(
           msg.sender,
           abi.encodePacked(key, value),
           config config,
           this.callbackIP.selector,
           extraData
       )
   }
   

3. callbackIP() receives the response from first deferral which contains the new sequence of the update (and possibly IPFS hash), along with updated newConfig and extraData. Since the next step is L2, newConfig should be returned by the gateway containing the relevant information; in this case the relevant information is ChainID and contract address of the target L2. With newConfig in place, callbackIP() makes 2nd deferral to L2.

   // Get response after 1st deferral and post-process
   function callbackIP(
       bytes response,
       config newConfig,
       bytes extraData
   ) external view {
       // 2nd deferral
       revert StorageHandledByL2(
           msg.sender,
           abi.encodePacked(response), // Sequence number (+ IPFS hash) to update on 
           [
               ["11"], // Expects list of ChainID values
               ["0xc0ffee254729296a45a3885639AC7E10F9d54979"], // Expected list of addresses
               [], // MUST be empty for L2
               [] // MUST be empty for L2
           ],
           this.callbackL2.selector,
           newExtraData || extraData // Calculate newExtraData if necessary
       )
   }
   

4. callbackL2() receives the response of second deferral and post-processes the results accordingly. For instance, if second L2 deferral has failed for some reasons, callbackL2() may choose to undo the first deferral with another (third) deferral. If the second deferral has succeeded, it may choose to emit a custom event.

   // Get response after 2nd deferral and post-process accordingly
   function callbackL2(
       bytes response,
       config newConfig,
       bytes extraData
   ) external view {
       // Post-process response, newConfig and extraData if required
       doStuff(response, newConfig, extraData)
       return
   }
   

Nesting, Complexity & Fidelity

Nesting adds significant complexity to the protocol in the sense that the first gateway itself could have internally carried out the tasks performed by second deferral. This is a valid point and there is no correct answer, and there are several pros and cons to either approach. For instance,

1. Fidelity: Without nesting, one can imagine that future services may combine two or more services A, B & C in different orders internally and each ordered set then requires a new gateway along its arbitrary construction and standardisation (not necessarily a bad thing). With nesting, ‘atomic’ handlers can be standardised once upon ‘registration’ and then all future services can play with the resulting fidelity without a need for a new gateway standard each time.

2. Transparency: Standard ‘atomic’ handlers will generally be more transparent to the end user when compared to complex gateways with several stacked storages A, B, C etc under the hood (if they are not standardised/documented).

3. Complexity: Nesting is complex and adds burden on the protocol, and may require CCIP-Write service providers to support an array of third-party non-EVM services that some handlers may require.

In the end, it is for the developer community to balance their need for fidelity with covariant complexity. I am personally on the fence on this one. While I like the theoretical fidelity, it may not be as feasible in practise.

[clowes.eth]

@NameSys

Here goes a more detailed example:

This is a super useful example. Appreciative of that.
Out of interest, who is it posting as ‘Namesys’?

You point to some of the potential downsides but I am curious as to your insights as well as Nicks regarding clients implementing these standard…

@nick.eth I’ve been reading through [Draft] EIP-X: Off-Chain Data Write Protocol and the linked ERC-5559: Cross Chain Write Deferral Protocol.

I note it states “The user will have to choose an RPC of their choice to send the transaction to for the corresponding chainId”. This proposal also requires clients to either POST to RESTful APIs or submit transactions to L2s, the latter requiring access to a potentially unlimited number of other chains. Was this not one of the crux issues with proposed extensions/additions to the 3668 standard for reads?

@NameSys

balance their need for fidelity with covariant complexity

I think I agree with this but could you state it in language appropriate for a 5 year old please? XD

[nick.eth]

@NameSys

User makes a request to set value of a key with setValueWithConfig(), and attaches the config with legitimate values of gateway urls, off-chain signers as authorities, approvals for off-chain signers, and accessories required to update their IPNS container (IPNS signature + sequence number).

How does the user determine that they need to make this call, and determine the parameters required, given their intent to “update name x”?

@NameSys

callbackIP() receives the response from first deferral which contains the new sequence of the update (and possibly IPFS hash), along with updated newConfig and extraData.

I feel like something is missing between steps 2 and 3. What does the user do, and where do the values for newConfig and extraData come from?

@NameSys

Since the next step is L2, newConfig should be returned by the gateway

What gateway? Is the IPFS write handled by a gateway of some kind rather than directly by the client? How is this gateway discovered, and what is its protocol?

Your step-by-step seems to leave out a number of important steps, which is making it difficult to understand.

[NameSys]

Okay, here goes another attempt in even more detail with ENS as an example:

CALL L2 nested in IPNS

Let’s consider a scenario where SameSys service intends to publish the address 0xADDR of a node off-chain (to an IPNS container) with setAddrWithConfig() on the ENS App. SameSys wants to additionally index the sequence numbers of IPNS updates on an L2 contract along with the IPFS hash, from where CCIP-Read can force-match the expected latest version with the resolved IPNS records before resolving any request. Such a process according to the draft goes as follows:

@nick.eth

How does the user determine that they need to make this call, and determine the parameters required, given their intent to “update name x”?

PRE-FLIGHT: SameSys-enabled clients, e.g. ENS App, could get the config interface from the contract hosting setAddrWithConfig() aka the Resolver. In this case, precise config to pass to setAddrWithConfig() could be stored in the Resolver in a pure public variable. For the sake of SameSys, there should ideally be an ENSIP-42 which details the precise config interface as well as the steps to generate the required values. Ideally, such steps should come in a self-contained public codebase (e.g. samesys.js) which ENS App can easily implement according to ENSIP-42. Instead of storing the config in the Resolver, the ENSIP could be another source of fetching the required config along with the steps to the generate the said config. This is all assuming that SameSys has its own registered StorageHandledBySS() identifier in ENSIP-42.

There could also exist a more generic scenario where SameSys/ENS App uses StorageHandledByIPFS() – a more generic handler writing to a public IPFS service – which should be defined in some other EIP-69420. In the end, a source – on- or off-chain – must exist for each StorageHandledBy__() identifier that can tell clients what config to generate.

1. User makes a request to set 0xADDR of a node with setAddrWithConfig(), and attaches the config – that it fetched during pre-flight – with legitimate values of gateway urls, off-chain signers as authorities, approvals for off-chain signers, and accessories required to update their IPNS container (IPNS signature + sequence number).

   config config = [
           ["https://ipns.namesys.xyz"], // Gateway URL
           ["0xc0ffee254729296a45a3885639AC7E10F9d54979"], // Signer address
           ["0xa6f5e0d78f51c6a80db0ade26cd8bb490e59fc4f24e38845a6d7718246f139d8712be7a3421004a3b12def473d5b9b0d83a0899fb736200a915a1648229cf5e21b"], // Ethereum signature by signer
           [
               abi.encodePacked(
                   "0xa74f6d477c01189834a56b52c8189d6fb228d40e17ef0b255b36848f1432f0bc35b1cf4a2f5390a8aef6c72665b752907be6a979a3ff180d9c13c7983df5d9c2", // Hex-encoded IPNS signature over ed25519 curve
                   bytes32(1) // Sequence required by IPNS signature payloads
               )
           ]
       ]
   

2. setAddrWithConfig() defers the storage to StorageHandledByIP().

   function setAddrWithConfig(
       bytes32 node, 
       address addr,
       config config
   ) external {
       // 1st deferral
       revert StorageHandledByIP(
           msg.sender,
           abi.encodePacked(node, addr),
           config config,
           this.callbackIP.selector,
           extraData
       )
   }
   

   @nick.eth

   I feel like something is missing between steps 2 and 3. What does the user do, and where do the values for newConfig and extraData come from?

3. callbackIP() receives the response from first deferral which contains the new sequence of the update (and possibly IPFS hash), along with updated newConfig and extraData. Since the next step is L2, newConfig should be passed by the 1st gateway as an argument to callbackIP() along with extraData containing the relevant information; in this case the relevant information is ChainID and contract address of the target L2. With newConfig in place, callbackIP() makes 2nd deferral to L2.

   // Get response after 1st deferral and post-process
   function callbackIP(
       bytes response,
       config newConfig,
       bytes extraData
   ) external view {
       // 2nd deferral
       revert StorageHandledByL2(
           msg.sender,
           abi.encodePacked(response), // Sequence number (+ IPFS hash) to update on 
           [
               ["11"], // Expects list of ChainID values
               ["0xc0ffee254729296a45a3885639AC7E10F9d54979"], // Expected list of addresses
               [], // MUST be empty for L2
               [] // MUST be empty for L2
           ],
           this.callbackL2.selector,
           newExtraData || extraData // Calculate newExtraData if necessary
       )
   }
   

4. callbackL2() receives the response of second deferral and post-processes the results accordingly. For instance, if second L2 deferral has failed for some reasons, callbackL2() may choose to undo the first deferral with another (third) deferral. If the second deferral has succeeded, it may choose to emit a custom event.

   // Get response after 2nd deferral and post-process accordingly
   function callbackL2(
       bytes response,
       config newConfig,
       bytes extraData
   ) external view {
       // Post-process response, newConfig and extraData if required
       doStuff(response, newConfig, extraData)
       return
   }
   

---

@nick.eth

What gateway? Is the IPFS write handled by a gateway of some kind rather than directly by the client? How is this gateway discovered, and what is its protocol?

In this example, IPFS & IPNS write is handled by https://ipfs.samesys.xyz which is stored in the Resolver as part of config declaration and this gateway’s coordinates should be pre-fetched before calling setAddrWithConfig() by the client. It’s properties should exist in public domain via ENSIP-42. Workings of this gateway all are internal and the user doesn’t need to do anything after making the initial call to set the off-chain address, other than sending the L2 transaction invoked in second deferral.

I did miss out some keys parts in hindsight. I hope this makes the idea more clear

[NameSys]

@clowes.eth

This is a super useful example. Appreciative of that.
Out of interest, who is it posting as ‘Namesys’?

sshmatrix.eth sharing collective works of NameSys team.

@clowes.eth

I think I agree with this but could you state it in language appropriate for a 5 year old please? XD

What I meant was that community may choose to accept a slightly complex EIP now which includes nesting, because it will reduce long-term workload required to stack multiple storages without the need to document each and every gateway. Or, the community may choose the luxury of a simpler EIP now which might eventually lead to tons of different gateways each with their own diverging standard. Complexity = Freedom = Fidelity

In this case, Complexity covaries with Fidelity. Sometimes, Complexity contravaries with Fidelity but this is not that case.

[nick.eth]

@NameSys

PRE-FLIGHT: SameSys-enabled clients, e.g. ENS App, could get the config interface from the contract hosting setAddrWithConfig() aka the Resolver. In this case, precise config to pass to setAddrWithConfig() could be stored in the Resolver in a pure public variable. For the sake of SameSys, there should ideally be an ENSIP-42 which details the precise config interface as well as the steps to generate the required values. Ideally, such steps should come in a self-contained public codebase (e.g. samesys.js) which ENS App can easily implement according to ENSIP-42. Instead of storing the config in the Resolver, the ENSIP could be another source of fetching the required config along with the steps to the generate the said config. This is all assuming that SameSys has its own registered StorageHandledBySS() identifier in ENSIP-42.

Okay, I think I may have misunderstood the scope of this proposed standard. I thought discovery was part of the intended mechanism here; if clients have to have explicit support for each method of storing data, that’s clearly not the case. It’s not entirely clear to me what purpose the standard serves in this event - if I have to write my app specifically to support “SameSys”, not just to support writing to IPFS and/or an L2, then what do I gain by standardizing these other components?

@NameSys

User makes a request to set 0xADDR of a node with setAddrWithConfig(), and attaches the config – that it fetched during pre-flight – with legitimate values of gateway urls, off-chain signers as authorities, approvals for off-chain signers, and accessories required to update their IPNS container (IPNS signature + sequence number).

Knowing the config presupposes you already know what method is going to be ultimately used to write the data, right? Given that, what purpose is served by calling the contract here? You already know it will revert with basically the same data you just passed in. Why not just go set the data directly, using the config you already fetched?

[quote=“NameSys, post:9, topic:18685”]
callbackIP() receives the response from first deferral which contains the new sequence of the update (and possibly IPFS hash), along with updated newConfig and extraData.[/quote]

Okay, but when and how does the “first deferral” happen? Does this involve calling a gateway in similar fashion to CCIP-Read? Where is the API for this specified?

@NameSys

Since the next step is L2, newConfig should be passed by the 1st gateway as an argument to callbackIP() along with extraData containing the relevant information; in this case the relevant information is ChainID and contract address of the target L2. With newConfig in place, callbackIP() makes 2nd deferral to L2.

So the gateway is calling callbackIP? Or the client is, after receiving a response from the gateway?

You say “since the next step is L2” - how does the client know that, and how does it affect what it does? If the client makes the callback, isn’t the nature of the second deferral opaque to it until it catches the StorageHandledByL2?

@NameSys

In this example, IPFS & IPNS write is handled by https://ipfs.samesys.xyz which is stored in the Resolver as part of config declaration and this gateway’s coordinates should be pre-fetched before calling setAddrWithConfig() by the client. It’s properties should exist in public domain via ENSIP-42. Workings of this gateway all are internal and the user doesn’t need to do anything after making the initial call to set the off-chain address, other than sending the L2 transaction invoked in second deferral.

I’m more confused than ever. Which initial call are you referring to? Is this an as-yet-unmentioned call to the gateway using some API akin to CCIP-Read? It’s still not clear to me what the API of this gateway looks like, and how it functions.

I appreciate you trying to explain this, but you’re continuing to leave a lot out, favoring prose over simple descriptions of control flow. As a result, I’m no closer to understanding what’s going on here than I was before.

I also feel like - based on what I do understand so far - that this protocol is unnecessarily complex; I don’t see what it adds over EIP-5559, especially since it leaves the discovery component unspecified.

[NameSys]

@nick.eth

Okay, but when and how does the “first deferral” happen? Does this involve calling a gateway in similar fashion to CCIP-Read?

Correct! It involves calling a gateway in the exact same fashion as CCIP-Read does and it is this gateway which must tell the client to call callback() with response, config for next step and extradata. This gateway decides autonomously what gets written to L2 after the second deferral.

@nick.eth

Knowing the config presupposes you already know what method is going to be ultimately used to write the data, right? Given that, what purpose is served by calling the contract here? You already know it will revert with basically the same data you just passed in. Why not just go set the data directly, using the config you already fetched?

That is absolutely correct and totally an option. But imagine intergrating such a service in ENS App. ENS App might want a standard CCIP-Write interface which not only supports SameSys individually but also other more generic storage handlers, e.g. IPFS or L2. If SameSys can be achieved by adding two pre-existing IPFS + L2 handlers, then yay for SameSys. If it does more complicated stuff, then probably nay for SameSys. SameSys is best served by standardising at least the standard parts of storage handling and beg the apps to integrate the little astandard part manually that remains.

@nick.eth

if I have to write my app specifically to support “SameSys”, not just to support writing to IPFS and/or an L2, then what do I gain by standardizing these other components?

If SameSys service C is such that it is not equal to the sum of generic A and B storages, then it is up to each individual app to accept or reject SameSys. But SameSys as described in this example is generic and literally the sum of IPNS and L2. If ENS App supports IPNS and L2 individually, then it already supports SameSys as described in this example. If SameSys is unique and astandard (e.g. Umbra Cash), it may require coding your app to support until it is a standard on its own (this is true for NameSys Pro client).

@nick.eth

So the gateway is calling callbackIP? Or the client is, after receiving a response from the gateway?

The latter, the client after receiving response from the gateway invokes callback().

@nick.eth

You say “since the next step is L2” - how does the client know that, and how does it affect what it does? If the client makes the callback, isn’t the nature of the second deferral opaque to it until it catches the StorageHandledByL2?

Because the gateway will tell the client in its response what to expect next. Once the gateway is done writing to IPFS and updating IPNS, it will tell the client what to expect since it is passing the config for upcoming step when invoking callback. The user doesn’t know this since it is the gateway which decides what to do next according to the service provider’s need (in this case indexing).

@nick.eth

Which initial call

Sorry for confusing language, initial call simply meant setAddrWithConfig(). I have been on mobile devices all day.

Just zooming out a bit:

This proposal is nothing more than CCIP-Read with some extra features such as storage-specific revert, callback, nesting and ability to invoke eth_sign and eth_call. Whatever this does can be done by hacking CCIP-Read to at least perform basic off-chain writes (except any L2 stuff).

On the question of discovery, some part of the config must be discovered somehow by the client from the contract. Write operations require access keys to wherever you are writing. It is fundamentally different than read which doesn’t require any discovery and everything is piped through a gateway.

[nick.eth]

@NameSys

Correct! It involves calling a gateway in the exact same fashion as CCIP-Read does and it is this gateway which must tell the client to call callback() with response, config for next step and extradata. This gateway decides autonomously what gets written to L2 after the second deferral.

Okay; this needs to be documented somewhere, with its API, and described in the example.

@NameSys

That is absolutely correct and totally an option. But imagine intergrating such a service in ENS App. ENS App might want a standard CCIP-Write interface which not only supports SameSys individually but also other more generic storage handlers, e.g. IPFS or L2. If SameSys can be achieved by adding two pre-existing IPFS + L2 handlers, then yay for SameSys. If it does more complicated stuff, then probably nay for SameSys. SameSys is best served by standardising at least the standard parts of storage handling and beg the apps to integrate the little astandard part manually that remains.

I don’t really understand what you’re getting at here. I understand apps needing to support each write method (IPFS, HTTPS API, L2, etc) independently. I don’t understand what “SameSys” is beyond composing IPFS + L2, and why that would require specific support from clients, rather than being a gimme if they implement those two protocols.

I’m also not sure why the config is something the client has to come up with out-of-band. Why can’t the first revert include whatever config is required, just like in 5559?

@NameSys

If SameSys service C is such that it is not equal to the sum of generic A and B storages, then yes. But SameSys as described in this example is generic and literally the sum of IPNS and L2. If ENS App supports IPNS and L2 individually, then it already supports SameSys as described in this example. If SameSys is unique and astandard (e.g. Umbra Cash), it may require coding your app to support until it is a standard on its own (this is true for NameSys Pro client).

Okay; perhaps you can give an example that’s internally consistent instead, then?

@NameSys

The latter, the client after receiving response from the gateway invokes callback().

This needs to be spelled out somewhere.

@NameSys

Because the gateway will tell the client in its response what to expect next. Once the gateway is done writing to IPFS and updating IPNS, it will tell the client what to expect since it is passing the config for upcoming step when invoking callback. The user doesn’t know this since it is the gateway which decides what to do next according to the service provider’s need (in this case indexing).

Why can’t the gateway simply do both the IPFS-update and L2-update steps itself, removing the need for the client to do them separately?

[NameSys]

@nick.eth

I don’t really understand what you’re getting at here. I understand apps needing to support each write method (IPFS, HTTPS API, L2, etc) independently. I don’t understand what “SameSys” is beyond composing IPFS + L2, and why that would require specific support from clients, rather than being a gimme if they implement those two protocols.

Because NameSys has the keyless feature which requires one additional signature for keygen when using IPFS as storage. This is not needed by IPFS or IPNS at their core, so it is an additional requirement by NameSys. Umbra Cash is a good direct comparison. Similar to how Umbra Cash requires extra meta transactions to make things private, NameSys also needs extra signatures for keyless interfacing. BUT when using server storage, NameSys is simply DB storage. Just one handler. Things get complicated when IPNS gets involved in keyless fashion. Innovative methods usually have this problem of being astandard perhaps.

@nick.eth

Why can’t the gateway simply do both the IPFS-update and L2-update steps itself, removing the need for the client to do them separately?

Yes, of course. This is what I was mentioning before in one of the comments. It is possible to choose a simpler standard and have one gateway do all the work instead of lots of back and forth between client and several gateways. It is up to you + others to decide and let us know which way to take the draft. It is possible that I am overthinking this and making it more complex than it needs to be.

[nick.eth]

@NameSys

Because NameSys has the keyless feature which requires one additional signature for keygen when using IPFS as storage. This is not needed by IPFS or IPNS at their core, so it is an additional requirement by NameSys. Umbra Cash is a good direct comparison. Similar to how Umbra Cash requires extra meta transactions to make things private, NameSys also needs extra signatures for keyless interfacing. BUT when using server storage, NameSys is simply DB storage. Just one handler. Things get complicated when IPNS gets involved in keyless fashion. Innovative methods usually have this problem of being astandard perhaps.

Okay; how does this look in practice? What information does the user need to supply? And couldn’t this be specified in a more generic way, as an API that can be used for multiple targets?

[NameSys]

@nick.eth

Okay; how does this look in practice? What information does the user need to supply?

The information that user supplies to this extra signature payload is similar to the messageData() in 5559 except for one-two more parameters that 5559 lacks. For 2FA security on keygen, user must be able to provide a secret IPNS key identifier. This secret along with CAIP10 and CAIP02 identifiers form the payload of message to sign. The signed deterministic signature is then fed to HMAC-based Key Derivation Function which finally generates your IPNS keys. This extra simple but necessary flow completes the accessories array in config for NameSys service. For any app to support NameSys with IPNS storage, they will need to support this extra flow to complete the config (hopefully easily with namesys.js). Only way for NameSys to convince apps to do this is through a standard. If NameSys only uses plain database instead of IPFS, then it is very easy and only one universal DB handler is needed.

@nick.eth

And couldn’t this be specified in a more generic way, as an API that can be used for multiple targets?

That is the goal in the end. Other than the IPNS keygen signature, other signatures are quite standard and can be moulded into a general API. In that sense, IPFS (without IPNS!) and DB are the same, and a single API will work.

[NameSys]

Let me try to describe it in plain speak again, only words no code:

Reading is a very simple operation which requires no knowledge of the off-chain storage type, has no accessibility requirements and data can be blindly read by CCIP-Read via a public/custom HTTP gateway. Writing is a more complex task depending on the target storage type; it may be relatively simple for L2/DB where only a signature needs to be attached but it becomes increasingly complex when namespaces like IPNS and ArNS get involved. In these cases, additional information needs to be generated, prompted or fetched in pre-flight to be able to access the IPNS or ArNS containers.

This draft imagines CCIP-Write to be very similar to CCIP-Read with OffchainLookup() revert replaced by StorageHandledBy__() and a global config. The need for global config is unavoidable when writing to any kind of non-standard namespaced storage. When writing to an L2 or DB, config is not necessary since in both cases, the ‘transaction’ payloads are similar to eth_call and eth_sign and therefore natively supported by all clients. For any other complex storage type that needs more than ethereum wallet signature, current ethereum client infra is insufficient and additional pre-flight discovery needs to be made; this is unavoidable.

How is this draft different from EIP-5559?

• It requires config in pre-flight for non-L2/DB storages, which is not supported by EIP-5559
• It requires a better messageData() payload than EIP-5559, which can meet the requirements of secure non-L2/DB storages
• EIP-5559 was written by coinbase for coinbase and didn’t have larger array of decentralised storages in mind
• Nesting (optional)

---

PS.

There is a way to ‘avoid’ client-side pre-flight fetching or prompting or arbitrary code by delegating it all to a standard 0th deferral which is reserved for pre-flight revert GetConfigForStorageHandler__(). This revert would need to be documented for each handler and the gateway responsible for telling the client to invoke callback0() must pass necessary triggers to client in callback0().

For example,

Deferral 0:

1. User calls setAddr(node, addr) in the Resolver contract without any config. The Resolver contract might know some (or all) information about where this node stores its data and reverts with this information via GetConfigForStorageHandler__(node).

   An example of this is the Recordhash or Ownerhash IPNS CID values which are stored on-chain in the Resolver for each node for advanced IPNS storage users of NameSys service.

2. The gateway handling the GetConfigForStorageHandler__(node) revert processes the node, gathers all the necessary triggers that are mandatory from the client side, and tells the client to process those triggers.

   An example of these triggers is the NameSys keygen workflow that I described before. This flow is responsible for generating the IPNS signatures needed to make the update on IPFS network.

3. Client processes the triggers, which fills the config object internally and invokes callback0() which then finally begins the 1st deferral to actually write the data using the config that it just obtained from the client.

   An example of this would be NameSys gateway telling ENS App that the storage will need a signature for keygen with some parameters in the next step (= 1st deferral), and one of these parameters is a secret password that the ENS App should request from the user with a prompt. After, a) getting this password, b) signing the deterministic message containing this password, c) using this message to generate IPNS keypair, and d) using the IPNS private key to sign new record update, the ENS App has the complete config with correct accessories required to make the 1st deferral to store the data.

Of course, all this comes with the complexity that the CCIP-Write protocol is now made up of at least two internal nested deferrals, and the first (0th) deferral (aka triggers) needs to be well-documented for each storage handler. The upside is that the user now only makes generic setAddr(node, addr) call similar to CCIP-Read, without any ...WithConfig(config). The client still needs to process all the triggers as it would have done anyway but at least these triggers and their processing will be well-documented along with their storage handlers.

[nick.eth]

@NameSys

This draft imagines CCIP-Write to be very similar to CCIP-Read with OffchainLookup() revert replaced by StorageHandledBy__() and a global config. The need for global config is unavoidable when writing to any kind of non-standard namespaced storage. When writing to an L2 or DB, config is not necessary since in both cases, the ‘transaction’ payloads are similar to eth_call and eth_sign and therefore natively supported by all clients. For any other complex storage type that needs more than ethereum wallet signature, current ethereum client infra is insufficient and additional pre-flight discovery needs to be made; this is unavoidable.

Thank you for the detailed motivation, that helps. I’m still not understanding how the client is expected to know the config, though, or why it can’t be supplied by the contract as part of the StorageHandledBy__ revert. If the client knows enough about the storage method being used before they make the call, why do we need this protocol at all?

@NameSys

• User calls setAddr(node, addr) in the Resolver contract without any config. The Resolver contract might know some (or all) information about where this node stores its data and reverts with this information via GetConfigForStorageHandler__(node).

An example of this is the Recordhash or Ownerhash IPNS CID values which are stored on-chain in the Resolver for each node for advanced IPNS storage users of NameSys service.

• The gateway handling the GetConfigForStorageHandler__(node) revert processes the node, gathers all the necessary triggers that are mandatory from the client side, and tells the client to process those triggers.

An example of these triggers is the NameSys keygen workflow that I described before. This flow is responsible for generating the IPNS signatures needed to make the update on IPFS network.

• Client processes the triggers, which fills the config object internally and invokes callback0() which then finally begins the 1st deferral to actually write the data using the config that it just obtained from the client.

An example of this would be NameSys gateway telling ENS App that the storage will need a signature for keygen with some parameters in the next step (= 1st deferral), and one of these parameters is a secret password that the ENS App should request from the user with a prompt. After, a) getting this password, b) signing the deterministic message containing this password, c) using this message to generate IPNS keypair, and d) using the IPNS private key to sign new record update, the ENS App has the complete config with correct accessories required to make the 1st deferral to store the data.

This seems like a more viable approach, especially if we can minimize the need for bespoke configuration data - for example, if the contract can supply it all, or simply prompt the client to request a signature from the user. Beyond that, a standardized interface for specifying UI components for a storage handler might work.

But I still think the overall protocol is unnecessarily complex; much like 5559, the first revert should contain enough information that you can get any additional config from the user and then make the offchain call directly, without requiring a second call, and without requiring a subsequent callback to the contract with the result of that call. I’m not really seeing the use-cases for these extra wrinkles that couldn’t be more efficiently served by a gateway.

[NameSys]

@nick.eth

But I still think the overall protocol is unnecessarily complex; much like 5559, the first revert should contain enough information that you can get any additional config from the user and then make the offchain call directly, without requiring a second call, and without requiring a subsequent callback to the contract with the result of that call.

That is a fair point. Client can call the gateway directly with the processed triggers and there is no need for second deferral. I will amend the draft to reflect your comments.

[garypalmerjr]

@NameSys

sshmatrix.eth

undefined on ENS

https://twitter.com/gsissh_matrix
…your ENS-listed Twitter account-link is broken, fyi.

I found the link via your GitHub:
https://twitter.com/sshmatrix_