I have been working on a reputation system for DAO delegates (eventually other types of contributors) and am requesting feedback from the community to make the algorithm better to reflect the true contribution of delegates.
The delegates are listed in order of their Karma score. Karma score is a quantified score representing the contribution of a DAO member. Given a Signal S and weightage W, karma score is the sum of the product of each signal and its weightage.
Signal (S) can be on-chain or off-chain voting percentage, total votes delegated, forum activity, badges accrued and so on.
Weightage (W) is the multiplier you apply to each signal to indicate the importance of that signal and normalize it.
Karma Score = Sum(S1*W1, …, Sn*Wn)
ENS Karma Score Calculation
Delegate does not have an ENS domain: (-20 points)
Has link to delegate pitch: 10 points
Total proposals initiated in the forum: (10 * total) points
Total proposals discussed in the forum: (2 * total) points
Total posts created: (1*total) points
Total likes received on posts in the forum: (0.5*total) points
Total posts read: (0.25*total) points
Snapshot voting percentage: (1*pct)
POAPs collected: (0.1*total) points
More than median votes delegated: 20 points
Experience for being in other DAOs
<3 DAOs: (count*15) points
3 - 6 DAOs: (count * 10) points
7 - 10 DAOs: (count * 5) points
User Type
Individual: 10 points
Company: 0 points
Non-profit: 50 points
Benefits of this system
Tokens holders can easily decide who to delegate their tokens to
Make changes in working groups based on activity
Get proactive in retaining active contributors
Discover new and upcoming contributors and recruit them into working groups
DAO contributors can show their reputation and find opportunities in other DAOs
Request for feedback
I believe for this reputation score to be accepted by everyone, it should be community driven and so I am requesting feedback from the ENS community to improve this algorithm. Do all the signals and the weights being assigned make sense, should we not consider some of those, should more signals be added (ex: discord)
What relevance does other DAO experience serve for the ENS DAO?
What relevance is User Type?
More than median votes delegated: 20 points makes this system favors those who already have had votes delegated to them. For your system to work, don’t you want people to even question their current choices for someone who will better represent their views by showing ENS activity and commitment, or are we to be stuck with a popularity contest?
Shouldn’t people who have more than one domain name and subdomain names get some credit, as they are making use of the ENS system; and thus, they are showing their commitment to ENS.
I will remove weightage of POAPs. I thought of giving credit to people who have been in the crypto space and collected POAPs.
I had UserType points based on some feedback from discussion in Discord. Some people felt it is good to encourage individuals to be delegates than companies. But I can remove this as well.
I just added you to the list, you are at no. 3! We are changing the logic on how we find delegates (currently we pull the top 1000 delegates by votes). It should solve the problem of missing delegates.
Just that they know how DAOs work and can bring in their expertise
Will remove it, have gotten that feedback from many, it was just that in Discord many people felt individuals should be encouraged to become delegates vs company
I only gave 20 points for that exact reason. It was giving some credit to people who have garnered votes. You can see your own ranking. You are No. 3 with barely any votes delegated to you. So, these 20 points are not adding a whole lot weightage to overall ranking.
I am totally fine adding this if the community feels strongly about it.
Really appreciate all the questions and feedback. None of these are set in stone and I am more than happy to change any of these metrics and weightages depending on what the community feels strongly.
Yea definitely think UserType doesn’t make sense, also don’t see why a “company” would be 0, but a non-profit is better than an individual. None of that seems to make sense to me hehe. The whole reason crypto works is because somewhere in the stack, there is an economic inventive to act in a positive way. A profit-generating group that builds on ENS is one of the most effective groups you want to have a voice in such a protocol!
On #1, is that taking into account DNS-based domains? (ie. non .eth ENS names) It looks like the app doesn’t take “metaphor.xyz”, an ENS delegate with a valid ENS name.
Is the current Karma score realtime? Doesn’t seem like it. In fact, seems quite off for me, probably because it is not matching my username here on forum with my .eth name where the votes are delegated.
It’s not realtime, we update it on a weekly basis. We are working on increasing the frequency. Yeah, there is no place I can look up the ETH + discourse username. This week we are adding a feature to let delegates update username so the score reflects accurately.
It might disadvantage people who don’t have all of their domains on their delegate wallet. It could also be gamed by minting multiple subdomains. I’m not sure if this is the best metric to gauge DAO involvement.
It measures vested interest in the protocol, but it doesn’t fully demonstrate DAO involvement which is what we should be looking for when choosing a delegate… I imagine that this tool is most useful for helping DAO members chose a delegate
I love this project and have been following it for a while. In my mind, one of the most beneficial use cases is providing data for choosing delegates and voting on stewards.
The more variables added to the Karma calculation the more difficult it might become to ensure the scores are equitable.
Because of this, my suggestions weight actual ENS DAO involvement (voting/forum) and are focused more on what specifically our DAO may find valuable. For example: I suggest to remove POAP from the calculation.
Suggestions
Parameter
Suggestion
1. Delegate does not have an ENS domain.
Change. For the ENS DAO, don’t populate delegates without domains or subdomains resolving to them.
2. Has link to delegate pitch.
Remove. The delegate pitch links pulled from metadata can be inaccurate. Mine for example is a shortlink that doesn’t work. A table field or user profile providing the link would be helpful.
3. Total proposals initiated in the forum.
Debate. Important, but favors stewards, existing team members or those already established in roles making proposals.
4. Total proposals discussed in the forum.
Keep.
5. Total posts created.
Keep
6. Total likes received on posts in the forum.
Keep.
7. Total posts read.
Debate. This could be gamed, but the multiplier is low.
8. Snapshot voting percentage.
Change. Increase weight and make on a rolling timeframe. Example: last 10 proposals or last 6mo. This is the key importance of a good delegate, future proofs so new members in a year+ aren’t permanently disadvantaged
9. POAPs collected.
Remove. We currently don’t measure ENS invovement this way. Could be good for other DAOs.
10. More than median votes delegated.
Debate. Undecided if this favors already established delegates and disfavors new involvement.
11. Experience for being in other DAOs.
Change or Remove. Karma score generated in other DAOs is more meaningful than mere involvement. Could add a multiplier based on their Karma in other DAOs.
12. User Type.
Change. The other parameters measure involvement well enough that this shouldn’t be a factor. It would be useful to highlight user-type without affecting score. Highlighting whether a delegate/user is a DAO is important too. Individual, Multi-Party, Company.
You have some valid points. But having a vested interest and investing in ENS shows commitment to ENS. It need not be given great weight but it shouldn’t be ignored. Also, I have one domain name that is not in my delegate wallet (I lost track of the wallet’s recovery phrase and will have to wait until 2025 to get in back ).
I’d be in favor of capping the amount of names and subdomains owned to a reasonable number, say 5-10? People who have multiple names would benefit, but it would limit potential disadvantage to those with other wallets not associated with their delegation address.
While I am not a beneficiary of this, these members of the community have taken on important responsibilities for the community and should have their contributions recognized.
I agree. I’m still thinking on this… Since it’s only 10pts per proposal, it’s not that big of a deal.
I wonder if a rolling timeframe of 1 year on some of these would be useful to account for recency. I’m imagining a future scenario where users would be permanently disadvantaged because they weren’t early adopters. Similarly, DAO Members who are no longer active might stay in the top ranking despite being MIA.
This promotes active involvement. If you’re no longer active, your karma, ranking, and visibility on the leaderboard would decrease.
Already on it! Hope to have this ready in next week or so. The default leaderboard can be last 30d, 90d, season/year. Can change the filter to “all time”.
I made a number of updates based on all the feedback. I removed a number of metrics like POAPs, experience in other DAOs, user type which everyone felt was not useful and changed the weights a bit based on feedback. This is what we have now:
Vote participation: (3 x vote participation percentage) points
Total proposals initiated in the forum: (10 * total) points
Total proposals discussed in the forum: (2 * total) points
Total posts created: (1*total) points
Total likes received on posts in the forum: (0.5*total) points
).