ENS Normalization with {FE0F} inclusion - ENS Fee and more

[kraft]

There are several emojis that are unqualified, not even minimally qualified, and still are used as the main normalised version. About the snowflake emoji too, my point was not which one is being normalised, ofc I know metamask, Open Sea and all others follow the one that ENS validates as the normalised version.

consider this eg, you’re not even using the minimally qualified one, you’re using the marked - unqualified version of the emoji. That does not make sense.

Raffy told me that EF0F stripping was brought into the pic because the previously registered ones already didn’t use it while registering, like this above eye-cloud one, so you guys just stripped it off.

To address your point of being fair, I don’t think it’s such, those who already own these - for eg the eye cloud emoji, is using the unicode marked invalid emoji and is up for sale for 1000eth. Heart on fire emoji had the biggest sale too, while its seq too is marked unqualified by Unicode. DAO is already being fair to them by letting it be.

What’s unfair to me is the cost factor that you’re not addressing, these additional 500+ emojis would be several times less costlier to register if it were with EF0F (the marked qualified version of emoji)

here’s an eg of some new ones -

[serenae]

The “cost factor” is not an issue. The FE0F stripped is something that has always been done, so everyone has always played by the same rules. Some emoji sequences have fewer characters than others, that’s life.

I think stripping FE0Fs is still the correct course, and in that example, it still presents the same way in browsers anyway:

If someone went directly to the contracts and registered some other variation with FE0F, then that was their mistake.

It seems that you are hinging a lot of your argument on the idea that unqualified sequences are “unicode marked invalid”. You’re using the word “unqualified” as if it’s supposed to be “invalid”, but that is not the case. It’s just a technical term for a form of the emoji sequence. From the Unicode spec:

ED-17a. qualified emoji character — An emoji character in a string that (a) has default emoji presentation or (b) is the first character in an emoji modifier sequence or (c) is not a default emoji presentation character, but is the first character in an emoji presentation sequence.

ED-18. fully-qualified emoji — A qualified emoji character, or an emoji sequence in which each emoji character is qualified.

ED-18a. minimally-qualified emoji — An emoji sequence in which the first character is qualified but the sequence is not fully qualified.

ED-19. unqualified emoji — An emoji that is neither fully-qualified nor minimally qualified.

Later in the spec, Unicode also notes this:

• minimally-qualified or unqualified emoji ZWJ sequences may be handled in the same way as their fully-qualified forms; the choice is up to the implementation.

The point is that with normalization, all different forms should normalize to the same thing. So all four forms you have listed there will normalize to 👁‍🗨.eth. In my opinion the most sensible and least error-prone way to do that is to be consistent across the board and strip FE0Fs in normalization.

Again, it doesn’t change any utility for the user. Anyone can use any of those forms with FE0F. Any frontend can display whichever form they want. It doesn’t matter, because the normalization library guarantees that all of those forms will point to the same registered ENS name.

[kraft]

The cost factor is an issue though, what would cost $160 or $5 would not cost $640, I don’t understand though where did the stripping FE0F thing came from, why was it suggested in the first place when it was fine even before.

Your point of someone using the smart contract to register being a mistake sounds wrong too, why impose one particular frontend on the masses when you should keep it as such that people should be able to register the right normalised sequence.

I’m aware of the these ED points from the Unicode spec, but that’s also my point however it’s marked, why choose the version that is the costliest to register. I also didn’t intend to make it sound invalid that’s why I used marked, I’m aware they all work the same.

Choosing non EF0F version are not error prone either, I’ve already mentioned some examples above that show that using EF0F was the right move and would have easily avoided wrongly normalised registrations. Even in that case the utility won’t change btw.

I still don’t understand though, the cost multiples 5x when or in some cases 64x, that’s not like a 10-20% increase for so many emojis. Also I’m not iterating something new, as suggested in my original post, it had been talked about previously as well that retaining EF0F in ZWJ in the newer versions seems to be correct move.

@serenae

The FE0F stripped is something that has always been done, so everyone has always played by the same rules. Some emoji sequences have fewer characters than others, that’s life.

That’s not life, it’s not like it cannot be corrected in the newer version if it’s always been this way in the current version, knowing well that if corrected the users won’t have to bear additional cost which is multiple times of it. Earlier today Raffy was giving me an eg of a=A in normalization so {no FE0F}= with FE0F, the argument doesn’t stand cost wise though.

[kraft]

also point me how it sounds unfair to incl EF0F in newer version. You pointing out that they registered via the contract without our frontend is wrong is equivalent to me saying that them register 👁‍🗨.eth with 3 char seq was wrong. We can keep going in this loop, while I don’t see how dec cost for newer emojis harms the previous ones. They too registered an arbitrary version.

[serenae]

@kraft

I’ve already mentioned some examples above that show that using EF0F was the right move and would have easily avoided wrongly normalised registrations.

You have decided that it’s “wrongly normalized”. I disagree. If this is the crux of the argument then I suppose we’ll just have to agree to disagree.

[kraft]

I’ve already pointed out the examples of snowflake, poop emoji, genie emoji, and so on, they don’t look the same without the EF0F. Also you saying that browsers anyway show the same thing doesn’t make sense. Why must we care which medium shows what when we know removing EF0F does create some difference to a selected few.

The right move would have been to strip off the already registered non EF0F ones since not incl EF0F in some - like snowflake, makes it a wrong decision. But ofc we shouldn’t break previously registered emojis.

All we can do is correct it here on and reduce the cost.

And yes, wrongly normalised is the right word to describe that else we won’t have the snowflake issue.

some discussions around this -

ENS Name Normalization

Edit I guess the goal was to have a framework that does it the correct way, and then try to relax it so it fits as many of the registered names as possible. I thought the keycaps could be fixed using my method, but since some are already registered, it has to use the unqualified form. Both the * and # keycaps can use the FE0F. Edit 2: Let me explain that a bit more and summarize: If we use UTS-51 + IDNA2008, my suggestion would be anytime ENS wants to enable new emoji (ie. Unicode updates…

FE0F Emoji crashes ens.domain address

Quoting user “Gee” in the Discord unicode(dot)org/Public/emoji/14.0/emoji-test.txt According to this, FE0F should be preserved to be “fully qualified”. Current implementations SEEM to be as follows: - Obviously, the contract takes anything you send it. - Etherscan and OpenSea will not strip FE0F when resolving/displaying! (But OpenSea for some reason DOES parse it down in SOME contexts, leading to keycap emojis being listed as length 2) - MetaMask and ens.domains does not preserve it. It come…

[kraft]

you saved already registered emoji domains like👁‍🗨.eth, .eth and .eth by sabotaging future registrars and the existing who all would have to pay nearly 5x or 64x just because the DAO decided to move ahead with a version that also unicode marks as unqualified. Whether qualified or unqualified they all look the same so it didn’t make sense to make this move in the first place. I’ve also demonstrated examples of issues it created in a selected few like the snowflake emoji.

[kraft]

Lastly the DAO issued refunds previously to users who registered domains from the frontend and were wrong. Why not do that for non EF0F ones like 👁‍🗨.eth, .eth and .eth why not do that, instead of inc the price 5x or 64x for the future registrars.

Assuming ENS lives on, this cost would only compound.

[serenae]

Your recent comments haven’t really brought anything new, I’ve already addressed all those points (and misunderstandings) in my previous comments, so I’ll leave it at that.

[kraft]

You’ve been addressing a portion of my responses. Haven’t found a response to these inefficiencies -

• cost is 5x or 64x due to this

• why choose a version of emoji seq that is costier, why not refund the already registered ones and correct this issue. The issue is there and I’ve pointed that with a single snowflake eg.

• why not improve that for the upcoming 578 emojis when you know that saves cost for the users.

• why is the DAO defending 👁‍🗨.eth, .eth and .eth this version of the normalisation and ditching others. When with EF0F version would not have any issues in the first place

• and lastly what happens if 5 years down the line EF0F becomes somewhat important like it is for a selected few to distinguish (eg - snowflake versions), we’re not talking a few months but years. If EF0F become relevant for many future emojis, would the DAO ditch all the current version of non EF0F domains.

You’re only assuming that EF0F may not be useful.

What would the DAO do if they do become relevant? what’s the next course of action if your assumption turns out wrong in the future 5 or 10years down the line

[raffy]

You are correct that my original idea was to grandfather all the old/unqualified emoji (without FE0F) and normalize future emoji to their fully-qualified forms. There was an implementation that worked like this.

However, every unqualified emoji sequence is 1:1 with the corresponding fully-qualified form. There are no ambiguities. Since registrations so far have removed FE0F, I see no problem with continuing this trend. I don’t foresee Unicode releasing a compound emoji where parsing varies with the presence of FE0F. See: Emoji Variation Selector Notes.

Even if future emoji were normalized with FE0F, old emoji (depending on platform) would still require some form of beautification to standardize their presentation. The same applies to keycaps, which are the most popular emoji type and the most mangled by the loss of FE0F. The use of beautification makes the lack of FE0F irrelevant for all emoji.

For reference, IDNA 2003 (the only UTS-46 flavor that supports emoji) strips FE0F. Most web browsers use IDNA 2003. EIP-137/ENSIP-1/ens-normalize.js use IDNA 2003.

---

The only benefit of allowing FE0F is to improve the appearance of raw normalized names that contain future emoji (without beautification) and reduce registration costs for the few who registered the shortest variants (since this introduces an additional superfluous invisible character.)

[kraft]

@raffy

I don’t foresee Unicode releasing a compound emoji where parsing varies with the presence of FE0F. See: Emoji Variation Selector Notes.

Quoting from the same unicode source you’ve put above, they have put it as an option for implementations and do mark atleast some old ones as defective. We still could have chosen the more cost friendly path right? nothing stops us from doing that.

@raffy

There are no ambiguities.

I don’t think this part too is correct, there are no more interpretations after normalization because you stopped it at the normalization stage itself. As shown above not having FE0F has created issues with emoji versions of snowflake and a few more. Why are we assuming that things would remain as you think they would. The scenario would be different had ENS been part of the consortium itself, but rn we rely on any info released from them.
a clear example of frowning face emoji.

@raffy

For reference, IDNA 2003 (the only UTS-46 flavor that supports emoji) strips FE0F. Most web browsers use IDNA 2003. EIP-137/ENSIP-1/ens-normalize.js use IDNA 2003.

are you sure this would always remain the case? we’re talking about ENS in general, a change rn would be in effect for decades if not corrected in the starting years itself. What happens if the assumptions are wrong - would the DAO strip old non EF0F ones and legitimize with EF0F ones in case something obvious comes out in support of EF0F?

@raffy

reduce registration costs for the few who registered the shortest variants (since this introduces an additional superfluous invisible character.)

Again as I pointed, this number may be small in registrations but cost wise they could become 5x to 64x which is not a mere 10-20% jump. Furthermore, who are we to punish specifically these selected users by charging hefty. The DAO didn’t (or couldn’t) do anything about future registrations of black bird emoji and many more like that. Then why only hold these selected EF0F ones accountable when unicode suggests there being nothing wrong in using EF0F and you have the option to reduce cost without creating any new issues.

To me it seems like we rely a lot upon speculative texts in the unicode doc which made us strip off EF0F from thousands of emojis - we’re not sure of whether EF0F becomes relevant or not in the future - say 5years down the line. Unicode may not be aware of the effect this creates to us, emojis are scarce and hold value in them.

also the number of combinations that could be made with some emojis like this is not small and thus removal of EF0F is not really a very small selected group of set.

In my current understanding we’re not ready for changes where EF0F becomes necessary, whether from unicode’s end or if some browser starts to use them.

more so, we’d have a few more emojis to register had it not been for EF0F stripping, very few but still.

@raffy

There was an implementation that worked like this.

My suggestion would be bring it into effect atleast for the newer 578 ones and also the DAO to already register them asap to auction them later after these are officially released. I don’t see it as a lot of additional work.

I also suggest the DAO to become a member of the Unicode like how other important orgs are, in ENS case emojis matter a lot and are of great importance. Instead of assuming something, you’d have rights to vote within the Unicode system.

(for some reason I seem to have been rate limited due to the number of replies therefore late response)

[kraft]

@nick.eth would love to know your thoughts on the above highlighted points

[nick.eth]

I defer to Raffy’s expertise here, and very much prefer anything that minimises the number of existing domains that are affected.

[kraft]

my suggestion doesn’t affect existing domains but rather reduces registration cost of newer 578 more emojis (and 100s of combinations). The idea sparked from one of Raffy’s comment itself. I don’t see a reason why we shouldn’t do it.

[nick.eth]

I don’t think changing how names are normalised for pricing reasons is appropriate. We should choose the best normalisation independently.

[kraft]

@kraft

Again as I pointed, this number may be small in registrations but cost wise they could become 5x to 64x which is not a mere 10-20% jump

regarding pricing, the change though is not small to think of it as inconsiderable. $640 for what could be $120 or $5, applicable to combinations as well.

More so, I have also talked about this, asked these questions -

@kraft

are you sure this would always remain the case? we’re talking about ENS in general, a change rn would be in effect for decades if not corrected in the starting years itself. What happens if the assumptions are wrong - would the DAO strip old non EF0F ones and legitimize with EF0F ones in case something obvious comes out in support of EF0F?

@kraft

also the number of combinations that could be made with some emojis like this is not small and thus removal of EF0F is not really a very small selected group of set.

Adopting EF0F versions for the newer ones don’t make new issues other than to label them seperately as Raffy has pointed previously.

[nick.eth]

@kraft

regarding pricing, the change though is not small to think of it as inconsiderable. $640 for what could be $120 or $5, applicable to combinations as well.

Can you give an example of emoji names that would cost $640/year, that you think should cost $5/year?

[kraft]

2 examples from the new 548 ones

26F9 FE0F 200D 2B05 FE0F

stripping off FE0F makes them both 3 characters long instead of 5 characters. This makes their individual registration cost $640 instead of $5 each.

This is just one, there are 100 of examples where it should cost $5 but due to stripping it’d cost $160 or $640

[nick.eth]

@kraft

individual registration cost $640 instead of $5 each.

This is just one, there are 100 of examples where it should cost $5 but due to stripping it’d cost $160 or $640

These appear to the user as a single character. It seems totally reasonable to me that they should have a registration cost on par with a 3 character alphanumeric domain.