Standardization web3 messaging

With Heiko’s post dm3-decentralized-messaging-for-web3 from August 22, we started the discussion on web3 messaging and interoperability of web3 messaging protocols, services, and applications in the community.

Since then,

  • The first iteration of the dm3-specification draft was published and we received a lot of feedback.
  • Discussions with a lot of different teams have been held to evaluate requirements and standardized approaches that will enable the web3 world to do much better than the web2 world - being interoperable and giving the decision which app to use and how to manage personal communication data to the user - without building new separated data silos.
  • We incorporated the feedback.

Now the new version (0.3) of the dm3 message transport protocol is published. Thanks to all for contributing to it with their feedback.

The current specification draft can be found at https://dm3.readthedocs.io/en/doc-latest/specification/message-transport/mtp.html.

The following major changes have been included:

  • HASH: we replaced all Keccak hashes with SHA256 hashes
    As SHA-256 is more common (Keccak is more Ethereum-focused), it might be easier for non-Ethereum-based applications to adopt it.
  • MESSAGE TYPES: There is a variety of types possible that might or might not be supported by a messenger. We set most of the types optional, which means dm3 compatible apps can decide whether to support it or not. The user’s settings (readable from mutableProfileExtensionUrl) can now return a list of not supported message types. A sending client needs to check the receiver’s settings and must not send a message type not supported by the receiver.
    Another service message type needed to be added (RESEND_REQUEST). With this service message and a hash referencing the message, the sender can ask to resend a historical or lost message.
  • MESSAGE ENCODING: The encoding of the message text must be plain text (UTF-8), optionally flavored with Markdown highlightings. If other application-specific encodings are to be used, they are provided as an attachment. The plain text representation must be available, too. A client able to interpret the encoded attachment may show this instead of the message text.
  • DELIVERY SERVICE PROPERTIES: The delivery service got an additional API function: dm3_getDeliveryServiceProperties. It returns a list of properties relevant to the delivery service.
    messageTTL: This is an optional parameter. It defines the minimum time an undelivered message will be stored by the delivery service. This parameter might be important for a delivery service to manage resources (otherwise, a delivery service will store all messages until they are picked up.) With this property set, the delivery service can purge its storage, if necessary, in a coordinated way.
    sizeLimit: The delivery service can define the maximum size of a received message to be accepted. As a delivery service can be a gateway to another protocol or app, it might be important to have a harder restriction
  • DELIVERY SERVICE DESCRIPTION: added an appendix to explain delivery service models in detail. In particular, delivery services can act as a gateway to other services/protocols to receive a dm3 message and include it in the ecosystem.

Next steps:

We are again asking for feedback and trying to have in-detail discussions with you. We especially want to open a community discussion on possible integration scenarios of dm3 in applications, protocols, or services. The goal is to have a variety of messaging applications in the web3 space but those solutions are able to interop.

Also, in the next iteration, we want to discuss in more detail the general registry based on ENS, as this is one of the crucial parts of interoperability between different applications and protocols → a standardized way to get information from the receiver (like public keys and how to deliver messages). Using ENS for this registry is a no-brainer. By using CCIP (Cross Chain Interoperability Protocol, based on EIP3668) the integration of Layer-2 registries, registries on sidechains or other chains, and even centralized services is possible as well.

The result of the next iteration should be a proposal of a basic protocol that fits well for a big variety of protocols, services, and dApps as a way to accomplish interoperability without giving up on security or features relevant to that ecosystem.

Steffen Kux
dm3.network

6 Likes

This is looking really good! Is there anywhere to leave granular feedback on the spec on a paragraph-by-paragraph basis?

1 Like

2.2.1.1. Step 1: Preparation of the Message and Envelope

Get dm3 profile

  1. Read the eth.dm3.profile text record of the receiver’s ENS name.

If you are starting from an ENS name (“I want to send a message to nick.eth”) then cool. What if the sender app only has an ETH account to work with? (“I want to send a message to 0xb8c2C29ee19D8307cb7255e1Cd9CbDE883A267d5”)

In that case, would it be correct to first resolve the Primary Name / Reverse Record for the ETH account? So something like:

Get dm3 profile

  1. The sender app may accept either an ENS name or an ETH account as the receiver. If the receiver is an ENS name, move to Step 2. If the receiver is an ETH account, resolve the ENS Primary Name for the account, and verify that it forward-resolves to the account address. If no ENS Primary Name is found, or it does not resolve to the receiver’s ETH account, the message cannot be delivered. It has to stay with the sender until the potential receiver sets an ENS primary name and publishes their profile.

  2. Read the eth.dm3.profile text record of the receiver’s ENS name.

=========

One other question I had is, how exactly does a recipient pick up a message from a delivery service?

If the delivery service integrates with some other service/protocol to auto-deliver to the recipient then all is well. But what about when the delivery service is not acting as such a gateway?

In the API I see dm3_submitMessage and dm3_getDeliveryServiceProperties, but what would the receiver’s app call to:

  • Check whether there are any new messages received (in the case where the delivery service does not send notifications, or the notification was dropped/lost/missed)
  • Retrieve messages from the delivery service
    • One at a time? Bulk retrieve? Retrieve a specific message by hash or timestamp?
2 Likes

Yes, you can download the pdf-version (left-bottom corner): https://dm3.readthedocs.io/_/downloads/en/doc-latest/pdf/ and add comments there and send it directly to me.

Alternatively, you can add comments on GitHub directly: GitHub - corpus-ventures/dm3-spec: White paper, protocol spec, ..., e.g., as pull request.

If you feel more comfortable adding comments to a gdoc, I could provide it, too.

Let me give you some answers to your questions:

If the user has no ENS name to register his/her profile, (s)he needs to register one to publish the contact information (public keys, delivery service). The best approach would be to register a name as a subdomain based on the address, like .dm3.eth - in your case 0xb8c2C29ee19D8307cb7255e1Cd9CbDE883A267d5.dm3.eth. We plan to do this as layer2 connected and verifiable using CCIP. Other domains can offer this in the same way as dm3.

To your proposal for the “Get dm3 profile”-text:
This is exactly the way a client app should work. If as receiver an address is given, the ENS name should be resolved. The approach above is an additional way.
We will add an explanation as you proposed in the spec to make it clear.

This will be specified in the protocol extension: Message Access Specification
As a gateway to another service/protocol doesn’t need to implement this, it is not part of the base protocol (the dm3 Message Transport Protocol). This specification is in work but has not yet been published as we focus at the moment on that part of the protocol which is needed for interoperability.

A few thoughts on the current spec:

  • The registry spec says that text records must contain either the DM3 JSON, or a URL to it. Why not specify that data: URIs must be supported, and thus remove the need for discriminating between the two formats? In the case of data URIs, the requirement to include a hash can be dropped.
  • The registry spec doesn’t specify how public keys should be encoded (eg, key component ordering and representation, ASCII encoding, etc).
  • Listing not supported message types is an odd way to do it, and leads to the assumption that any new message types are supported by default. It probably makes sense to list supported message types intead.
  • You probably want to specify a format for third-party message types, such as using reverse ENS names in the same fashion as you do for text record names.
  • You may want to specify that clients should respect the ENS TTL when looking up user or delivery service profiles; with gateways likely to be sending a lot of messages this could improve latency significantly, especially for delivery service lookups.
  • The choice of UTF-16 for message bodies is rather unusual. Why not UTF-8?
  • You should probably specify a standardised means to add extra metadata to the message and envelope objects.
  • The envelope metadata should contain information on the encryption and signing algorithms used. There’s only one option for now, but that will not be the case indefinitely.
  • For attachments you could also specify that they are URIs, with data as a supported scheme.
  • There’s only one ‘m’ in ‘incoming’.
4 Likes

Hi Nick, thank you very much for your feedback. Please find below some quick answers:

This is a good idea. We will change this.

You are right. We will add this in more detail to the spec.

We discussed this a lot. Actually, there is no perfect solution. If you opt-in (list supported messages), the user (or his/her client) has always to provide this information otherwise nothing would be supported. While this mutableProfileExtension was meant to be optional only, a positive list would not work that well. We are right now in discussions to this topic and will propose a better solution soon.

Did I miss this at one place? Yes, we already changed it to UFT-8.

We will add this as optional field.

Yes, we already have this information included (for the encryption). The receiver’s client could ask for another encryption as default. If the sender supports this, it is fine. We will check, if some info is missing in the data structures.
We will add this for signatures as well.

Good point. We will change this as well.

As it depends on the implementation of a client or delivery service, if such information is cached, we will add the notice that in case of caching, the ENS TTL needs to be respected.

Can you give more info on this topic? I don’t really understand what is your intension for this.

1 Like