-
I am not a MetaGov steward this term
-
It’s not the unratified external dashboard, it’s the tiebreaker rule that was used in previous elections and has been written extensively, versus your new proposed method to resolve ties by looking at individual matches.
2 Likes
It was clearly an error of the Meta-Governance Working Group to leverage Snapshot to display results of a voting strategy that Snapshot was not compatible with. We should own that mistake and be transparent about it.
When we used this voting strategy in the SPP2 selection last year, we were explicitly clear about Snapshot’s inability to display the results accurately, and we probably assumed that same clarity existed in all the participants’ understanding of this election. This particular edge case of the tie handling is exceedingly rare, although we have mentioned it plainly most times that we’ve discussed this method.
@netto.eth , Could we list here in this thread the places where we announced what the voting strategy would be for tgis election, so that we can at least establish the rules were articulated clearly before the vote, even if they weren’t included in the proposal text? Also, please correct anything that I’m stating here that is inaccurate.
I hear you, Spike, but any decision to change the tiebreak criteria now would be a subjective decision. As mentioned, we have a reference on the tiebreak that was defined by Avsa when he first proposed to use this voting mechanism (here, just search for “tiebreaker” and you’ll find it), and it was simply not fully implemented by snapshot.
I’m sure that if the situation were the opposite, you’d appreciate sticking to what was previously stated rather than taking a subjective decision. We’re improving the tooling gradually, experimenting with different primitives, and some hiccups will happen. I’m sorry it happened to you. Thanks for engaging thoughtfully and for nominating.
Sure! The initial post ifself outlies the copeland as the voting method.
So I’ve been thinking about this whole situation, and let’s step back for a second here and abstract away from this particular election.
In my original pitch, I mentioned that one of major points of my agenda is to apply best practices and standards utilized by public companies in order to ensure transparency and that everything is done by the book. The regulatory landscape is shifting dramatically now, and supposedly pivot towards a different governance model in the form of Foundation might address those new challenges.
Looking at the Ooki DAO precedent. The US government no longer treats unwrapped DAOs as immune decentralized software protocols, they treat them as general partnerships where everyone who participated is liable.
If you look at actual CFTC filing you can see that they operate on pretty strict standards. Regulators specifically target the idea that DAOs can do whatever they want based on informal community consensus. In their own words from official press release:
By transferring control to a DAO, bZeroX’s founders touted to bZeroX community members the operations would be enforcement-proof—allowing the Ooki DAO to violate the CEA and CFTC regulations with impunity
Now, just hypothetically, if any kind of external regulator approaches ENS today assessing whether we are doing things the “right way”, what kind of conclusions can they draw from this election?
They will see an organization that claims to be a decentralized protocol governed by onchain voting. But in reality, when a definitive 2 to 1 (928k to 516k) token majority voters literary expressed onchain, the stewards use an uncodified “average support” metric, which was not included in the proposal and not included in the calculation mechanism of snapshot, to manually override the outcome.
I appreciate that fact that this metric was used in the past and considered to be sort of informal public knowledge, but regulators don’t care about informal knowledge. To an auditor, relying on unwritten rules and offchain dashboards to overturn a 64% token-weighted majority looks exactly like a centrally managed general partnership where a few insiders dictate the outcomes. It completely destroys the legal defense of decentralization.
I’ve been advocating for clear policies, management of conflict of interests and doing things by the book for years now. If this broad shift toward Foundation model would happen, to allow for better defense against regulatory bodies and general better interaction with web2 world, then starting this transition with this kind of precedent doesn’t seem like the best idea. It’s like handling the regulators the exact evidence they need to apply similar arguments to what happened in Ooki case.
I was concerned about this exact scenario, because leading up to last year’s SPP there was ambiguity about how the results would be interpreted. Noticing that gap, my team volunteered to write the reference implementation of the algorithm for calculating results, and then coordinated with Blockful and Agora to make sure everyone got the same output.
During this week’s metagov call, I asked if this time we would be relying on Snapshot’s implementation of Copeland scoring and if we would go with the results Snapshot displays. The answer was yes, we will follow whatever Snapshot says.
So I think it is incorrect to now say the results as displayed are wrong and we have to interpret them differently.
Since this ambiguity was not addressed before the start of voting, I think the only solution is to have a tie breaker vote between the two candidates.
3 Likes
@jkm.eth Thank you for your input, integrity, being transparent in this complicated matter and providing additional valuable insight. I think this is a very important discussion which touches upon key fundamental principles governing interactions within the DAO.
If the agreement on the metagov call was explicitly to “follow whatever Snapshot says”, then applying some additional approach retroactively to determine the winner goes directly against that agreement.
However in my view rerun is not necessary here, it introduces unnecessary friction and governance fatigue on delegates who already spent time thinking through their choices and casting their ballots. The entire point of ranked choice voting and the Copeland method is that the head-to-head matchups are already built into the ballots themselves.
Moreover we don’t need a second round to know how the DAO feels about this specific choice, because the second round already happened inside the data. When you isolate the ballots between myself and Abdullah, the voters literary expressed a definitive 2 to 1 (928k to 516k) majority onchain.
Forcing a rerun just because the visualization tools are ambiguous tells the delegates that their original preference didn’t matter. The current results are entirely sufficient to determine the direct will of the token holders, and we should respect the recorded ledger instead of forcing everyone through another round of voting.
1 Like
To be extremely objective here: when talking about a tie-breaking method, those who want Spike to win will choose to focus on an aspect that leads to that conclusion (the head-to-head result of Spike vs Abdullah). Those who want Abdullah to win will choose to focus on a different aspect (our tie-breaker method from SPP2). Therefore I don’t think either of these work as irrefutable evidence pointing to who should win, because the losing side will simply choose to believe their detail is the more important one.
Those who don’t have a strong opinion about who should win would most likely look at the process: who is the winner based on the stated rules? If the stated rules don’t cover this edge case, or there was widespread misunderstanding about how the rules should have been interpreted, then the only way to find out how the community wants to resolve the situation is to have them vote on it.
My personal opinion is that according to the stated rules Spike should be the winner because that is what Snapshot displayed. However those rules were reinforced verbally on a call that many did not attend, so if there is a strong argument to be made that the official/written rules were ambiguous, a tie breaker vote is the only truly objective way to find a winner without the losing side feeling like the vote was not conducted fairly.
@jkm.eth I really appreciate that candid and objective assessment. It takes a lot of integrity to step back, look at the situation objectively, and state it so clearly, and I want to thank you for that.
You hit the nail on the head with your conclusion:
But I want to elaborate on this - I think everyone understands at this point that the fact that I was placed third in the UI was just a simple indexing glitch, I would’ve been ranked fourth if my name was entered into a proposal in a different order. But the binding “governance” agreement in this situation is the Snapshot itself. When we isolate that interface glitch and actually pull the raw data from the vote - which is essentially what Snapshot displayed - then this must be the honored result.
The current data is entirely sufficient to determine the direct will of the token holders. I ask the Meta-Governance working group to simply honor the stated rules and the recorded ledger.
Strong agree.
If we publicly stated that we would follow the results as displayed by Snapshot and then only bothered to check if the algo was correct after the fact, this is a flat out coordination failure.
Agree with @jkm.eth that a full revote is not needed and a tie-breaker heads up vote is the correct way to legitimise the third seat.
Given the current climate, this coordination failure does nothing to aid the cause to demonstrate ENS DAO/Metagov as a functional body and we should address this quickly and seriously.
I’m not sure I agree in this case. For instance, it does state in the announcement at the top of the thread that it will be Copeland scoring. We all understand and know this voting strategy from the many discussions about its nuances, especially when we initially used it in the SPP 2 vote.
I believe Lighthouse even graciously worked diligently to support this exact scoring style.
Many of us, @SpikeWatanabe.eth included, participated in that SPP2 vote, and the many calls leading up to it, and the many conversations about how this voting strategy works.
IMO, the grounds for a revote would be some sort of a technical glitch that caused confusion in the voting booth, a loss of data, or something that throws trhe results into question. None of that’s the issue here.
We know what the voting method and scoring strategy was for this vote; it was announced before the vote. While it’s disappointing that we hit this edge case where the UI is not displaying correctly, it doesn’t throw into question the results themselves, and since shielded voting was used, the results (or how they were displayed during the vote) didn’t impact the voting process or the integrity of the data.
2 Likes
Thanks @5pence.eth, I totally agree with you if it is understood that we were using the rules from SPP2 (and maybe it was, see below).
Copeland’s method itself does not define a standard tie-breaking method, so simply agreeing that we will use Copeland’s isn’t enough to avoid this ambiguity.
For SPP2, we created our own, unique solution for tie-breaking. When I heard Snapshot was going to do a Copeland integration, I didn’t automatically assume they would use the same tie-breaking method that we invented. (Was this discussed somewhere?)
If it was clear (to the community) that Snapshot was going to use our algorithm, and it turns out they implemented it incorrectly, then I agree we should just run the data through the correct algorithm to find the results.
But if the understanding was that Snapshot would make their own choice on how to handle tie-breaking and we would defer to them, then it isn’t really our place to claim they did it wrong.
I would like to highlight that I have no bias towards either candidate, I am just trying to explore how the community can come to a conclusion that everyone would accept. No one can argue with the results of a tie-breaker vote, so that would be the easiest (laziest?) path forward. But if there is alignment behind “Copeland’s method” having always meant our specific SPP2 variant, then that is fine as well.
1 Like
Fair points, and I’d have to do some digging to understand exactly what Snapshot is using for their algorithm.
I think the key here is that because we were using shielded voting, UI representations in no way impacted the integrity of the voting. The voters made their ordered selections free from any bias of current standings or UI representations.
So the only question becomes: Are we counting the votes consistently with the stated intentions of the election beforehand? If the intention was to use the same ordered voting and scoring algorithm as we did for SPP2 (which, it seems clear to me, was the stated intention), I think we can apply the exact same scoring methodology and algorithm, which we litigated extensively last year, to the results here.
So if the voting method was clearly stated beforehand, and the votes themselves were free from any bias because of the use of shielded voting, I can’t articulate any reason why we would need to re-vote.
3 Likes
Thanks @5pence.eth. Agree with stated intentions nuance.
If the intention was to use the Copeland SPP2 algo variant and was stated clearly as such, I would also agree that a re-vote is not needed and the results should meet the interpretation of what was stated.
Respectfully @5pence.eth I want to address the voting SPP2 point, because I think it touches on a really important reality of how delegates operate.
I did participate in SPP2 vote. But I think we are mixing up here “participating as a voter” with “ratifying a custom voting algorithm (which is very niche and applies to highly specific cases) as permanent DAO law”.
When SPP2 happened, delegates were evaluating a massive multi million budget distribution across multiple service providers which in itself is a massive work. Like most delegates, I was trying to make my best judgment with respect to actual service providers proposals and was casting my vote via interface provided.
It’s no secret that delegates are drowning in proposals, forum posts and flow of information as it is. As such we have to rely on the working groups to build UIs that reflect the rules. There is just not enough bandwidth to audit every specific calculation which was discussed all over the place - in threads on calls and so on. Ideally you would attend every single call and read every single bullet point ever posted on forum, which is physically not possible .
To imply that just because I clicked vote on the SPP2, I somehow implicitly agreed that this highly specific tie-breaker variant would automatically apply to every future Copeland election - even when this proposal just plainly said “Copeland” - is a stretch.
If the intention was to use a custom, proprietary SPP2 variant instead of standard Snapshot mechanics, it should have been explicitly stated like so, linked in the proposal and so on, so that voters would know what would be happening under the hood. It was not the case in this instance.
We can’t govern a DAO on the assumption that delegates perfectly memorized every tiny bit of math and logic which was employed in previous votes. We have to govern based on what is actually written down in the current proposal, and what is recorded on the ledger. And on the ledger, the data shows a definitive margin.
My point in mentioning SPP2 was to highlight that this isn’t a new voting strategy for us. It’s one that we’ve extensively discussed, litigated, and explained in multi-million dollar votes. We explained that as the Copeland method then, and it’s fair to believe that’s exactly what we meant when we said this election would be the Copeland method as well.
They don’t have to, and werent asked to. The delegates were simply asked to rank their choices in linear preference. Because the votes were shielded during the voting process, neither the UI nor details about the voting strategy played any part in that delegate’s linear ordering.
As long as the votes were counted using the strategy that was announced before the election began, and that strategy had been extensively discussed and litigated in other ENS DAO votes, the integrity of the outcome is preserved.
The question that @jkm.eth made in the metagov call was:
“Will this vote use the Snapshot UI?”"
The answer is yes. The assumption was that Snapshot implemented the Copeland method exactly like specified in " RFP for improving Ranked Choice Voting support in Snapshot". I’ll put the screenshot here, of the specific section that mentions the tiebreaker.
It’s simply wrong to say there is no reference or evidence. It’s not a subjective decision; it’s clearly stated.
There are 2 mistakes here:
- Metagov didn’t verify the algorithm against the Snapshot implementation. It’s impossible to foresee and think about all the variables and nuances of these processes. I fully own this mistake.
- Snapshot didn’t implement the tiebreaker part of the algorithm. There is no tiebreaker currently.
My position as MetaGov steward (Term 6)
If there was no evidence, I’d support this going to a token holder vote, because then we would be doing a subjective decision. Where possible, we should not take any subjective decisions. As shown above, there is clear evidence on rules that should be used on Copeland, and it’s simply misleading to say that it’s not the case.
Just one observation: the tie breaker solution you quoted only works with a “none below” option, as we had for SPP2. It won’t work with the steward election, because all candidates appear on all ballots. Easy to adjust, but it’s not clear what Snapshot would have chosen to implement without further guidance from us.
It is obvious what the intent was (to have Snapshot implement the same algo we used for SPP2) but it looks like the final implementation details weren’t ever provided by Snapshot.
2 Likes
@jkm.eth makes a really brilliant point here that I think completely closes this loop. I want to thank James for stepping in to clarify this, because as I mentioned before, I am not an expert on voting theory and pretty much doing the research on the go on the mechanics behind those approaches.
So they way I see the situation is as follows: given that Stewards and broad team of contributors who actually built the Copeland voting architecture for SPP2, I think we should defer to their conceptual vision of how the mechanism should’ve functioned.
To quote this as well:
@netto.eth I appreciate you stepping up and owning the coordination mistake regarding Snapshot’s implementation, these things happen and as a DAO we learn from them. But looking at the point @jkm.eth just pointed out, it becomes clear why that SPP2 Average Support formula was completely incompatible with this steward election.
So the way I understand the situation is that SPP2 vote was a highly specific, custom-built mechanism designed by the DAO for distributing a massive budget. In that scenario, the “none below” option was required to act as a cutoff, once below cutoff they were not included in average support calculation at all. These were considered “unranked” and received zero votes from that voter.
Without that “none below” cut off line in current election, that specific Average Support formula becomes distorted and stops measuring what it was intended to measure originally. My understanding (correct me if I’m wrong) if we apply Average support now then it artificially inflates the score of whoever happened to be least liked, voters could’ve chosen to put them below cutoff line, but they didn’t have the option.
The election process itself, with the Copeland mechanism applied, was essentially valid. There is nothing wrong with the shielded data we collected. It just that applying retroactively custom SPP2 Average Support metric here is conceptually incorrect.
But I think important point here is that I’m not inventing a wheel here
If we look at how this is handled in real life in the majority of similar scenarios where competitors are matched up against everyone else - pretty much any sport if you will - the universal, standard way to break a tie in points is to look at the direct head to head record between the tied competitors.
We don’t need to invent a new custom tie-breaker or force delegates into a rerun, because that head to head matchup already happened inside this exact election. And in that direct matchup, the raw data shows the delegates preferred me by a definitive margin of 928k to 516k.
If we invalidate a perfectly legitimate election outcome and force the delegates into a rerun just to cover up an administrative and conceptual error, that is setting a bad precedent. The most logical path forward that protects the integrity of the voters effort and time spent deciding on candidates is to simply honor that definitive head to head ledger data, exactly as it is done in almost every similar real world scenario.
During this last week, MetaGov stewards from term 6 discussed the situation, and our consensus is that we should stick to what we have as a reference for the Copeland implementation (that was the expectation for what Snapshot has) and avoid making subjective decisions.
Reference: “RFP for improving Ranked Choice Voting support in Snapshot”. Note that the post is not about the mechanics for the SPP2 vote, it’s about improving ranked-choice voting.
Another key aspect is that the incomplete implementation of the algorithm didn’t affected they way in which votes were casted:
The first post (#16) on the election results remains valid.
1 Like
Some additional background info:
During the run-up to SPP2, we wrote a reference implementation of the scoring algorithm so that everyone can independently verify the results, since Snapshot did not provide that functionality at the time. That code can be found here: GitHub - 0xLighthouse/snapshot-copeland-results · GitHub
Although the work was inspired by the unique needs of the SPP2 vote, we designed the code to also work for any regular Snapshot vote using Copeland’s method.
These details might not have been explicitly stated in the run up to the steward election, but the assumption by many was that this reference code is what would be powering Snapshot’s system. Netto’s post #16 highlights the discrepancy between this reference code and Snapshot’s UI.