Auto-Redistrict

"One vote, one bit."

News

2017.01.17: (Blog) Summarizing the National effect of Gerrymandering: REDMAP worked

Earlier I’ve discussed how to measure gerrymandering using a seats-votes curve. By definition, gerrymandering gives one party or group an advantage over another, given the same percentage of the popular vote.

In an ideal world, both sides would gerrymander not at all. We don't have that. Second best option is both sides would gerrymander about the same, and the two would mostly cancel each other out, leaving us with a fairly neutral net effect (except for the un-cancelled effect of taking power away from the voters). We don't have that either.

In the world we live in, both sides do it, but one side does it a lot more than the other. Leaving us not only with less competitive elections, but with results that are systematically out-of-sync with the will of the people.

Aggregating the by-district stats collected from auto-redistrict for every state: (2010 by-district aggregated)

And plugging that into the online seats-votes curve calculator: (calculator)

We get the seats-votes curve:


Which tells us immediately that the entire nation has been strongly gerrymandered in favor of Republicans.

If you look at the red curve’s y coordinate at x = 0.5, you can see that if the overall national vote is tied 50-50, Republicans will win about 57% of the seats. While this 7% may not sound like much, translated into congressional seats, that’s 30 seats. 30 people elected that should not have been elected. 30 entire congressional districts with the wrong representative. Now consider the number of people who voted in the 2016 election: about 128 million. Multiply that by 0.07, and that’s over 9 million voters disenfranchised.

By contrast, this is what the national seats-votes curve would look like if all of the congressional districts were designed by Auto-Redistrict:


Near perfect symmetry. At a 50-50 tie, the seats are 50-50, no seats misallocated, no voters disenfranchised.

The choice seems pretty obvious to me… if you like democracy.


2016.06.03: (Blog) A New Way to Measure Partisan Gerrymandering: Mean-Centered Weighted Average Vote Packing

Firstly, what is gerrymandering?

Picture for a moment 3 districts, each of equal population, each with an exact tie:

  • District 1, dem votes: 1,000
  • District 1, rep votes: 1,000
  • District 2, dem votes: 1,000
  • District 2, rep votes: 1,000
  • District 3, dem votes: 1,000
  • District 3, rep votes: 1,000
Final Outcome: 3 tied seats. This is perfectly non-gerrymandered.

To summarize this in a simple illustration, the districts would look like this:


A perfectly non-gerrymandered set of districts.

Now to gerrymander it by the smallest amount possible, we'd move an equal number of democratic voters and republican voters. We'll take 1 dem voter from district 1, turning it into a republican district, and throw them into district 3. To keep the population balanced we need to take a voter from district 3 and add a voter to district 1. So we just move a republican voter the other way. So now it's:

  • District 1, dem votes: 1,000-1 = 999 ( to district 3)
  • District 1, rep votes: 1,000+1 = 1,001 (from district 3)
  • District 2, dem votes: 1,000
  • District 2, rep votes: 1,000
  • District 3, dem votes: 1,000+1 = 1,001 (from district 1)
  • District 3, rep votes: 1,000-1 = 999 ( to district 1)
Final outcome: 1 dem, 1 rep, 1 tie. Still not gerrymandered.

To gerrymander we need to do one more swap, this time between district 2 and 3.

  • District 1, dem votes: 1,000-1 = 999 ( to district 3)
  • District 1, rep votes: 1,000+1 = 1,001 (from district 3)
  • District 2, dem votes: 1,000-1 = 999 ( to district 3)
  • District 2, rep votes: 1,000+1 = 1,001 (from district 3)
  • District 3, dem votes: 1,000+1+1 = 1,002 (from district 1 & 2)
  • District 3, rep votes: 1,000-1-1 = 998 ( to district 1 & 2)
Final outcome: 1 dem, 2 rep. Now it's gerrymandered.

You see what we did there - we moved 2 democrats into the same district, while moving 2 republicans into different districts. That is how gerrymandering is done. To wrap it all up in a simple illustration:


A republican gerrymander.
Despite having an equal number of votes, Republicans' get 2 seats to Democrats' 1.

So how can we measure this?

Well you'll see that there are 4 republican voters whose votes were 2 votes over what's needed to win, and 4 democratic votes whose votes where 4 votes over what's needed to win. On average, each excess republican voter was in a district 2 votes over that needed to win, whereas each excess democratic voter was in a district 4 votes over that needed to win.

To put this more precisely: weighted average excess votes per excess voter = total over all districts of (excess voters * excess votes) / total over all districts of (excess voters)

For democrats, we have (4*4)/(4) = 4.
For republicans, we have (2*2+2*2)/(2+2) = 2.
Take the simple difference of this, and you get a measure of partisan gerrymandering. 4-2 = 2.

Now to double-check this, let's do another round of gerrymandering, same way:

  • District 1, dem votes: 1,000-2 = 998 ( to district 3)
  • District 1, rep votes: 1,000+2 = 1,002 (from district 3)
  • District 2, dem votes: 1,000-2 = 998 ( to district 3)
  • District 2, rep votes: 1,000+2 = 1,002 (from district 3)
  • District 3, dem votes: 1,000+2+2 = 1,004 (from district 1 & 2)
  • District 3, rep votes: 1,000-2-2 = 996 ( to district 1 & 2)
Final outcome: 1 dem, 2 rep.

For democrats, we have (8*8)/(8) = 8.
For republicans, we have (4*4+4*4)/(4+4) = 4.
Taking the simple difference we have 8-4 = 4.

As expected, twice the gerrymandering led to exactly double the gerrymandering score.

In a nutshell, the proposed partisan gerrymandering score is: the arithmetic difference of the mean-centered weighted average excess votes per excess voter, by party.

For the coders out there, here's the algorithm for computing gerrymandering, in Java:

public double get_partisan_gerrymandering(double[] vote_surpluses) {
	double mean = 0;
	for( int i = 0; i < vote_surpluses.length; i++) {
		mean += vote_surpluses[i];
	}
	mean /= (double)vote_surpluses.length;

	double demsq = 0;
	double repsq = 0;
	double dem = 0;
	double rep = 0;
	for( int i = 0; i < vote_surpluses.length; i++) {
		double d = vote_surpluses[i]-mean;
		if( d < 0) {
			d = -d;
			demsq += d*d;
			dem += d;
		} else {
			repsq += d*d;
			rep += d;
		}
		
		return demsq/dem - repsq/rep; //+ = repub gerrymander, - = dem gerrymander, value = difference in avg. vote packing
	}
}

2016.05.25: (Blog) The Interactive National Map of Gerrymandering

So I've been pretty quiet for a long time. There's a reason for that: I've been busy. With something big... I present to you, a veritable tour-de-force of Auto-Redistrict's power, The Interactive National Map of Gerrymandering.

I've been working on this as a project in cooperation with FairVote.org. (Very friendly people, by the way.) Suffice it to say I, uhh... went a little beyond the scope of the project. What can I say, I'm an over-acheiver. :)

Before giving a brief introduction to the map, let me take an aside to recommend FairVote.org. There are a couple of different proposals for ending gerrymandering, of varying sophisication. But there is one end-all-be-all solution that, ironically, has nothing at all to do with redistricting. The solution is called "mutli-member districts". I've touched on it briefly a few times in prior blogs, below. This is where FairVote.org comes in. They are a non-profit advocacy organization for multi-member districts. I could go on for a long time about how this simple solution of multi-member districts is as magical as it is simple. Indeed, I have before. But this time I will let the results speak for themselves. They say a picture is worth a thousand words, well in that case, here are a few hundred pictures..

Now for my brief introduction. Well, I'm more of a systematizer than a writer, so let me introduce by listing off a few great things about this map:
  • It's all 50 states! Like i said, tour-de-force.
  • It compares a number of different methods! I have the actual 2010 districts, the actual 2000 districts, hypothetical optimized multi-member districts, hypothetical optimally compact single-member districts, and more to come.
  • You can see tons of different stats. From racial composition to partisan gerrymandering, per district or per capita. Just browse through the list.
  • You can zoom-in on a single state, and see a side-by-side comparison of methods.
  • And below that you can see detailed statistics in table form.
  • Both the national map and the state comparison is fully embeddable in any web page.
  • Pie charts!
  • And just to make it feel a little nicer, I added a cross-fade effect.

(None of these maps should be considered "proposals", so to speak; They are intended as hypothetical examples.)
All maps and statistics were created with Auto-Redistrict. (Free and Open Source Software)
Method used:
Vote count used:
Show statistics: unit:
 Embed code
 
Votes Seats Packing / cracking Legend Population Seats Legend
Dem(packed)
Dem(cracked)
Rep(cracked)
Rep(packed)
White
Black
Hispanic
Indian
Asian

Click on a state to see detailed statistics

Alaska Alabama Arkansas Arizona California Colorado Connecticut Delaware Florida Georgia Hawaii Iowa Idaho Illinois Indiana Kansas Kentucky Louisiana Massachusetts Maryland Maine Michigan Minnesota Missouri Mississippi Montana North Carolina North Dakota Nebraska New Hampshire New Jersey New Mexico Nevada New York Ohio Oklahoma Oregon Pennsylvania Rhode Island South Carolina South Dakota Tennessee Texas Utah Virginia Vermont Washington Wisconsin West Virginia Wyoming



2016.05.16: (Blog) How I discovered a new method of counting ranked choice ballots

While in the process of generating multi-member redistricting maps for FairVote.org, we were discussing how the winners are calculated. This is when I first learned of the Droop quota, and how basically all STV vote counting methods use it. My first impression was "that's wrong". Where N is the total number of voters, each candidate represents 1/N voters. So quota should obviously be 1/N.

Then I found there was a nothing thing wrong with the ranked choice was traditionally counted. When no one meets quota, they eliminate the candidate that got the fewest votes and look at the next highest choice for those ballots. Okay, that's fine -- that makes sense -- but then after a candidate meets quota, they never reintroduce those candidates! That elimination should have only been temporary -- only for determining the winner for that seat. You should only eliminate a ballot permanently when it's used to meet quota. Otherwise those voters just aren't being represented at all. They might as well not show up to the polls because you just threw away their ballot.

But now I was starting to see why they were using an improper quota of 1/(N+1) (called the "Droop" quota): because the premature elimination of the ballots was screwing things up, especially for minority candidates. So instead of realizing what was causing this (the premature elimination of ballots), they artificially lowered the bar. They added a second error to try to treat a symptom of the first, thereby compounding the error. I'm reminded of the nursery rhyme "There was an old lady who swallowed a fly..."

Now it made sense. They were using the wrong quota because they were prematurely eliminating ballots. The correct solution was of course to not prematurely eliminate ballots (and to use the correct quota of 1/N). Somehow this had gone unnoticed for centuries. The old lady had gone on swallowing things and the fly was still wiggling and jiggling inside her.

The first iteration

So, equipped now with a proper solution, I did as all good programmers do: I wrote a program. This accomplishes three things:

  1. It helps me make sure I've got every nuance and corner case and all that taken care of; make sure I really have every part in the machine figured out. (And if I do, it demonstrates that.)
  2. It allows me to quickly and accurately run tests to verify and confirm the method.
  3. It gives me a working automated counter to demonstrate the method's superiority. (Or discover its inferiority, be that the case.)

Though I had first to figure out one more thing: how to deal with only partially ranked ballots – that is, what to do when a ballot runs out of choices? It seemed to me the correct way to deal with it was to say that they rank all remaining choices equally – the remaining choices are tied for last. But this seemed like it would make for some complex code, so I looked for a way to simplify it. Sure enough, it turns out that simply reducing the quota as if the ballot was never cast is mathematically equivalent to saying they ranked the remaining candidates as tied for last. So there was my 3rd change:

  • Re-calculate the quota just before testing for it, instead of just once at the beginning. This compensates for ballots that have run out of choices.

Now that I thought I had all the pieces, I wrote it up...

The second iteration

Well turns out I didn't have every nuance figured out. It occurred to me that if I tried (temporarily!) eliminating candidate A, then tried eliminating candidate A and B, well wouldn't it be better to try eliminating just candidate B, before trying both A and B together? The standard method was just to add one more candidate to what you’re eliminating, each time, thus skipping eliminating just B alone. That didn’t seem right. More precisely I knew it was wrong – I just didn’t know what the right way was.

So, I just went with the traditional way for now, see where that gets me. I figured I could revisit the problem later, and be better equipped when I did.

All the results of my tests looked pretty good, except for one. And I knew almost right away what the problem was. Because the elimination order was wrong. It was going from A to A,B -- skipping B. it was time to revisit the elimination order.

Thanks to some conversation on google election science forum, I decided to go all the way with this - to order every single possible combination of candidates. The key was switching the sort criteria from how many ballots were affected, to what I called (for lack of a better term) how many "choice demotions" were produced. Going from 1st choice to 2nd was 1 “choice demotion”, whereas going from 1st to 3rd was 2. Now I had a way score every possible combination of candidates, and thus a way to sort them. Notwithstanding ties (which are unavoidable), I now had a complete ordering.

So I tried that, worked beautifully. Turns out it also solves another problem of traditional STV methods, that they are not strictly monotonic. Which is why I had that result I didn’t like. This creates a strictly monotonic solution.

And now I have my final algorithm. The 4 major changes from traditional STV are:

  • Uses the correct quota of 1/N (don’t swallow the spider)
  • Only permanently eliminate ballots if they are used to make quota (don’t swallow the fly)
  • Recalculate quota right before checking (cause ballots could have run out of choices)
  • Temporarily eliminate sets of candidates in order of fewest "rank demotions" to most, ordering all possible combinations.

The method

For the sake of precision, let me lay it all out (for those who want _provably exact_ precision, here's a counting program I wrote in java: http://autoredistrict.org/ranked_choice_counter.zip ):

STEP 0: Make a copy of the ballots - these will be referred to as "original" cop, even though they'll be altered (reweighted) due to surplus vote transfers. Determining the next winner:
  1. STEP 1: determine quota
    1. Count how many ballots are left (using the weighted value of the ballot), divide by number of seats left to fill. This is the quota.
  2. STEP 2: order all combinations of candidates
    1. Compute all combinations of candidates (e.g. {a},{b},{c},{a,b},{b,c},{a,c},{a,b,c})
    2. For each such combination, count how many "choice demotions" would happen if those candidates are removed, in order for every voter to have 1st choice candidate, or run out of choices. For instance, if a person's choices are {a,b,c,d}, and a,b & d are removed, then the result is {c}, and it counts as 2 demotions. (Since what is now the 1st choice used to be the 3rd).
    3. Sort all combinations by how many choice demotions they produce, least to most. (Ties are fine.) Add the "empty set" (no candidates) to the beginning.
  3. STEP 3: (loop): make a local copy, and try removing the next combination. (first time through, this is removing no combinations)
    1. Make a local copy of the ballots. For the remainder we will be operating on this local copy.
    2. Take the next set of candidates in the sorted demotion list (if this is the first time, this will be an empty set).
    3. Remove those candidates from the ballot, and promote all the choices so it doesn't skip any ranks. if a candidate is out of choices, temporarily remove the ballot
  4. STEP 4: check for quota or 1 candidate left (repeat from step 3 if not reached)
    1. Check if any candidate has reached or exceeded quota (using their weighted value -- always use the weighted value).
    2. If one has, the one with the most votes is elected.
    3. If none have reached quota, and only 1 candidate remains, that one is elected.
    4. If none have reached quota, and there is more than 1 candidate, revert to the local copy you made in step 3, and repeat, using the next candidate combination in the sorted list.
  5. STEP 5: winner selected, transfer surplus votes proportionally (on _original_ copy).
    1. The candidate with the most first place votes is elected.
    2. Re-weight the ballots that were used to make quota - that have that person listed as first in the local copy. Their new weight is whatever their previous weight was, times 1-(quota/(the sum of their current weights)). This re-weighting applies to the _original_ ballots, not our local copy.
    3. Remove the elected candidate from all ballots, and shift the ranks up to make them sequential. If any ballot runs out of choices, remove it.
  6. STEP 6: repeat as necessary
    1. Repeat from the step 1 for the next seat

Testing and valitation

I’ve ran every interesting scenario on this I can find, (13 scenarios in all), and the results all look ideal. Better than any other STV counting method.

I’ve also looked up criteria for voting systems, as far as I can tell it passes all of them. The same cannot be said for other STV counting methods. For example, excepting CPO-STV, other methods are non-monotonic (due to not having a complete ordering for transferring votes when quota isn’t met).

I’ve also looked at tactical voting resistance. It is immune to tactical voting. In other words, honest voting is always the optimal strategy.

Comparing to other methods

I was asked to compare this to Meek’s STV and CPO-STV. Short version is it has about as many advantages over Meek’s as it has over any other STV method, and over CPO-STV it has the advantage of being able to unambiguously transfer surplus votes, thus making sure each person’s vote is counted exactly once.

Advantages over Meek’s STV:

  • Does not prematurely eliminate ballots, thus passing the one person one vote test.
  • Uses the correct quota of remaining ballots/remaining seats, thus being more fair to minority candidates
  • Fully orders potential vote transfers, thus is monotonic.
  • Is immune to tactical voting, whereas Meek’s, being non-monotonic, is not.

Comparison to CPO-STV:

  • Matches CPO-STV results on all scenarios tested.
  • This method provides a clear way to proportionally transfer surplus votes. Presumably with surplus votes transferred, this method would be more fair than CPO-STV in some edge cases.

2016.02.19: (Blog) The North Carolina Compact Re-Gerrymander: a Real-World Example of how Compactness rules do NOT Prevent Gerrymandering

There's a common misconception that "gerrymandering" is equivalent to making districts that are not compact. This is not true. "Gerrymandering" is drawing maps to give one party or ethnicity an advantage over another, irrespective of how citizens actually vote. This can be done without having to make districts that aren't compact. (Conversely, one can make districts that aren't compact without giving one party or ethnicity an advantage over another.)

In an earlier blog, I briefly touched on this theoretically, showing a simple picture of compact, but gerrymandered districts, along side the opposite: non-compact, but non-gerrymandered districts.



But I'm sure questions might still linger in the reader's mind. Questions like:

  • Is it really that easy to do this in real life -- to create a compact gerrymander?
  • Would politicians really attempt this?

Well thanks -- or no thanks, rather -- to every Republican in the North Carolina Senate, we can now answer both questions soundly in the affirmative. Enter exhibit A:



While being far more compact than the previous gerrymander, this proposal -- approved down party lines by Republicans in the North Carolina Senate -- would maintain the 10-3 Republican super-majority made possible by the original non-compact gerrymander, despite the state's population being almost evenly split between Democrats and Republicans.

One might ask: "Yeah, but was this deliberate or an accident? Where they aware of this?" And the answer to that is: Keenly! And they made no effort to hide it:

"Our intent is to use the political data we have to our partisan advantage, I acknowledge freely that this would be a political gerrymander which is not against the law." -- Rep. David Lewis (R-Harnett), speaking to the Joint Select Committee on Congressional Redistricting

For comparison, I used Auto-Redistrict to generate a fair map, using the presidential election vote counts from the '08 election. You can view it, along with detailed statistics and the per-precinct district assignments, at the automatically generated datasheet. The resulting seat balance is 8-5, and the resulting seats/votes curve is almost perfectly symmetric:



Let this be an example to discourage states from adopting compactness rules and thinking that will put an end to gerrymandering. Not in the slightest! To put an end to gerrymandering, you need to use criteria that explicitly measure gerrymandering the same way someone who is intent on doing it would measure it: based on the expected partisan breakdown of the resulting districts.


2016.01.24: (Blog) What is the best way to measure partisan gerrymandering?

There have been a number of different methods proposed to measure partisan gerrymandering, but which is the best one? Which is the most reliable? Sounds like a difficult question, even a subjective one, but if we take care to approach it systematically, we might very well reach an objective answer.

I have scoured the internet for every method of measuring partisan gerrymandering ever proposed, and even come up with my own. There are suprisingly few:

So, five. That is certainly a few. I have taken it upon myself to analyze all of these proposed measures to find what is the best one to use for Auto-Redistrict. This was all done in my head, however, and I've never -- until now -- taken the time to explain my reasoning. This is still a bit over-summarized, but I hope it is complete enough that you can see how my conclusion was inevitable.


STEP 1: Establishing criteria

In order to figure out what the best way to measure gerrymandering is, we first need to establish some objective criteria to use to judge potential measures by. This may seem a bit "meta", but bear with me. We can start with the obvious, and then move on to properties that would be nice to have.

Now the most obvious criteria of all is we want it to detect gerrymanders when there are gerrymanders, and detect that there aren't when there aren't. To make this a little more concrete, there are 4 possibilities:

Gerrymander detectedGerrymander not detected
Is a gerrymander true positive false negative
Isn't a gerrymander false positive true negative

Notice there are two red squares -- most people don't realize this, but there are two qualitiatively different ways of making an error. These are called "type 1" and "type 2" errors. When we detect a gerrymander when in fact there is no gerrymander, that's called a "false positive" or "type 1" error. When we fail to detect a gerrymander when in fact there is one, that's called a "false negative" or "type 2" error. We want, of course, always the green, and never the red. We want to always detect a gerrymander when there is one, and never count something as a gerrymander when it is not. A measure that fails either of these tests is just no good. The mere fact that we can tell that there was an error implies that there is a less-error prone method available. (If nothing else, the very method we used to identify that there was an error!) So here is our first criteria:

  • If the measure can be shown to produce either type 1 or type 2 errors, it's no good.
This is a pass-or-fail criteria. If it fails this test, we can throw it out without considering anything else.

Now the second most obvious criteria is that the result of our measure shouldn't change when things that aren't relevant to what we're interested in changes. For instance, if another Star Wars movie is released, that shouldn't make our measure suddenly say there's a gerrymander when before it did not, or vice versa. This would suggest that there's something very fundamentally flawed with our measure. Now that one was fairly obvious, but there are some less obvious independence requirements. For instance, let's say we want to base a temperature scale off the freezing and boiling points of water. Say, the freezing point is zero degrees and the boiling point is a hundred. Pretty straightforward, you say, nothing wrong with that. That's called Celsius. Ah, but wait - the freezing and boiling point of water changes depending on the ambient pressure. So we have to add in the wording "...at the average pressure of the atmosphere at ground level" to make it so that our measurement of temperature isn't dependant upon the ambient pressure.

We want something similiar for our measurement of gerrymandering. It shouldn't be dependent on things like population, average voter turnout, etc. And it definitely shouldn't go from gerrymandered for Dems to gerrymandered for Repubs just because one party was more popular in one election year. That tells us that we either had a false positive or false negative before, or that we do now. If the result changed, then either it was wrong before or it is now. So that's another pass-or-fail rule:

  • If a uniform vote swing changes the sign of the measure, then we know it has type 1 or type 2 errors, so it's no good.

More generally, but not absolutely crucial, is we want a measure of gerrymandering that is generally unaffected by vote swings, at least statistically speaking; at least in the aggregate. We'd also like it to be relatively unaffected by non-uniform vote swings -- by one district swinging one way, while another swings another. We know that this can't be an absolute disconnect, because gerrymandering does depend on the partisanness of districts. So we can't make either of these absolute pass-or-fail criteria. A gerrymander is by definition reliant on partisanness of districts. But still, it would be nice if our measure was at least fairly statistically independent from changes in voter sentiment.

  • independent of uniform partisan swing (not required)
  • some statistical independence from non-uniform partisan swing (not required)

Another property that we would like, but is not strictly neccessary, is "continuosness" or "smoothness". If our measure has gaps in it - areas where it suddenly jumps from one value to another, this strikes us as something we wouldn't intuitively expect, and makes it more difficult to work with. Again, not a pass-or-fail criteria here. A discontinuous measures is better than none, but if two measures are otherwise equal in all other aspects, we'd prefer the continuous one.


A discontinuous function - we'd rather avoid this, but it's not the end of the world.

  • continuous/smooth (all else being equal...)

STEP 2: Evaluating proposed measures based on the criteria

Compactness

While compactness (or lack thereof) is often touted as a clear sign of gerrymandering, it is in fact merely a potential side-effect. A non-compact district is not necessarily one that isn't gerrymandered, nor is a compact one neccessarily not gerrymandered. Indeed, it doesn't really make sense to say "district x is gerrymandered", as gerrymandering neccessarily involves trading voters among multiple districts to alter statewide results. That is, it isn't districts that are gerrymandered, per se, but whole states. So we know right away that this measure fails, since it only considers a single district, when we know that you need to consider at least 2 districts.

Shown below is an example of a false negative and a false positive, if one were to use compactness as a measure of gerrymandering. The color represents the partisan-ness (more blue = more dem, more red = more repub). The left isn't gerrymandered, the right is. But the compactness measure would detect precisely the opposite - it would fail to detect the gerrymander on the right side, while falsely reporting a gerrymander on the left side.

An excellent real-world example of this is Republicans in North Carolina Senate's attempt to re-gerrymander their state, after being asked by the court to draw new maps. This was widely reported in the news as a re-gerrymander, for example in this article: "Everything you need to know about North Carolina's new Gerrymander" and this article: "Expert: New map is just the latest byproduct of NC's "disastrous and disorganized" system". The old maps were gerrymandered to give Republicans a 10-3 super majority, even though population is about evenly split. The new maps are gerrymandered to give Republicans a 10-3 super majority, even though population is about evenly split. The only difference is the new maps have more compact districts.

Compactness passes all the other tests, though. But not for good reason. It is independant of turnout, swing, etc., precisely because it is independent of the vote count altogether. The number of peas in a pod is fairly independent of the ambient air pressure, but that doesn't mean it's a good indication of temperature. Likewise, compactness is fairly independent of vote swings, but that doesn't mean it's a good indication of gerrymandering.


Efficiency gap

Because the "efficiency gap" measure falsely assumes that first-pass-the-post voting systems produce a diagonal seats-votes curve, it will inevitably produce both false negatives AND and false positives, and will always flip the sign of the detected partisan gerrymander when there's a sufficient uniform partisan swing.


Since it fails the "changes sign on uniform partisan swing" test, we know that it also fails the stronger restrictions "independent of uniform partisan swing" and "independent of non-uniform partisan swing". Also, since as soon as an election goes from 49-51 to 51-49, that election goes from 100% wasted dem votes, 0% wasted repub votes, to 0% wasted dem votes, 100% wasted repub votes, we know that the "efficiency gap" is discontinuous, so it fails that test, too.

Grofman/King asymmetry

Short answer is Grofman/King asymmetry passes all tests, except for smoothness, since it measures number of seats.

Best/McDonald/Wang median minus mean

Short answer is Best/McDonald/Wang median minus mean passes all tests.

Baas asymmetry

Short answer is Baas asymmetry passes all tests.

Now let's summarize this all in a table:

compactness efficiency gap Grofman/King asymmetry Best/McDonald/Wang median minus mean Baas asymmetry
PASS OR FAIL CRITERIA
false positives yes yes no no no
false negatives yes yes no no no
changes sign on uniform partisan swing no yes no no no
INDEPENDENCE CRITERIA
independent of uniform partisan swing yes no yes yes yes
independent of non-uniform partisan swing yes no yes yes yes
DESIRABLE PROPERTIES
continuous / smooth yes no no yes yes

We see right away that the only ones that don't pass all the pass-or-fail criteria are compactness and efficiency gap. So those get ruled out right away. The remaining three all meet all of the criteria. So at this point we're left with a three way tie. Next, we'll look at how to break that tie.


STEP 3: Finding the winner

The "seats-votes" curve below displays all the relevant info about the partisan-ness of the districts in one clear graphic. The x-axis is % votes democrat, and the y-axes is the % of the seats dems would get at that vote % in the given districting scheme. Since this graphic shows all of the information, any measure of partisan gerrymandering can be displayed on it. This makes it a very useful tool for seeing how the different measures relate. Below is a seats-votes curve for the Wisconsin 2012 assembly election. (Note, each election has its own seats-votes curve, and you can average multiple elections together to get a mean statistical seats-votes curve over a multi-year span.) On it you can see four different measures:

  • The Grofman/King asymmetry, which is the number of excess seats at x = 0.5. (y(0.5)-0.5). This successfully detects the partisan gerrymander.
  • The Best/McDonald/Wang median minus mean, which is the vote % in the median district (y = 0.5), minus the vote average (which had been non-dimensionalized to 0.5). (x(0.5)-0.5). This successfully detects the partisan gerrymander.
  • The "efficiency gap", which is the distance of the y coordinate (seats) from the diagonal (x=y) at a certain vote count (x). This changes dramatically depending on the x-coordinate choosen. Depending on the x-coordinate choosen, it will sometimes say there is a gerrymander, and sometimes say there isn't, irrespective of whether there is or not.
  • The Baas asymmetry, which is the total darkened area - the total (unsigned) displacement of the seats votes curve from perfect symmetry. This successfully detects the partisan gerrymander.


You can see right away that minimizing the Baas asymmetry will also minimize both the Grofman/King asymmetry (which is just the vertical displacement at x=0.5) and the Best/McDonald/Wang median minus mean (which is just the horizontal displacement at y=0.5). I'm thus calling it "at least as good as" these two other measures, since optimizing it also optimizes both of them. However, the reverse is not true - optimizing e.g. Grofman/King asymmetry will not fully optimize Baas asymmetry, since Grofman/King asymmetry only minimizes the displacement at x=0.5. Likewise for Best/McDonald/Wang median minus mean.

We can say, also, that Grofman/King asymmetry is at least as good as Best/McDonald/Wang median minus mean, and vice-versa, since optimizing either one is simply moving the center of the curve so that it intersects x=0.5,y=0.5. This covers all combinations, so we can now put all of these relationships in a (completed!) table:


Grofman/King asymmetry Best/McDonald/Wang median minus mean Baas asymmetry
Grofman/King asymmetry x at least as good as not at least as good as
Best/McDonald/Wang median minus mean at least as good as x not at least as good as
Baas asymmetry at least as good as at least as good as x

We can conclude from this table that optimizing Grofman/King asymmetry or Best/McDonald/Wang median minus mean will also optimize the other, but optimizing Baas asymmetry is the only measure that will optimize all three. Consequently, Baas asymmetry is in an important way pre-eminent, because if we had to chooose just one measure to optimize to minimize partisan gerrymandering, it would be the only choice that optimizes all measures that meet the minimum requirements.

So we come at last to our one true winner:

  • Baas asymmetry (Hey, that's me!)
(Cheers and applause!)


2016.01.24: (Blog) The Mathematics of Gerrymandering: Why is it Possible?

If we want to measure gerrymandering, we must first understand how it's done. In most states, there is a single winner in each election: whoever has the most votes. This is called a "first-past-the-post" voting system, or "winner-take-all", or "single-winner plurality", more formally. And it has a major weakness. This weakness has four very undesirable consequences:

  1. the results are not proportional,
  2. a large fraction of the votes are wasted, leading to low voter participation
  3. it forces a two-party system, and
  4. it allows for gerrymandering.
What is this weakness? Well, let me show you how this system works out in practice, and the weakness will become clear.

Think of the voters as a bag of jelly beans. (Nooo! Don't eat them!) Dems are blue, Repubs are red. Now an electoral district contains a subset of these jelly beans. So in our analogy here, we'd grab a handful of beans out of the bag and that represents our district. Since this is a single-winner plurality system, whatever color we have the most of is the winner. Now let's say we only pick 3 beans. Now there are 8 combinations (where r=red and b=blue): rrr,rrb,rbr,rbb,brr,brb,bbr,bbb. But we only care about how _many_ of each color there are, so we can group them accordingly:

countcombos
0 redbbb
1 redrbb,brb,bbr
2 redbrr,rbr,rrb
3 redrrr

Now the chance of getting bbb is just the chance of picking a blue the first time, times the chance of picking a blue the second time, etc. The chance of getting bbr, is the chance of picking blue the first two times, and red the second. Since the order of multiplication doesn't matter (the communitive property), this number is the same for bbr,brb,and rbb. It only matters how many reds and how many blues. So let's fill in our chart a little more - let's say there's a 60% chance of red and 40% chance of blue. Then we have:

countcombos# combosprob. per combototal prob
0 redbbb10.4*0.4*0.41*0.4*0.4*0.4=0.08
1 redrbb,brb,bbr30.6*0.4*0.43*0.6*0.4*0.4=0.288
2 redbrr,rbr,rrb30.4*0.6*0.63*0.4*0.6*0.6=0.432
3 redrrr10.6*0.6*0.61*0.6*0.6*0.6=0.216

Now red gets elected if there are at least 2 red beans. So we simply add up the probability of there being 2 red with the probability of there being 3 red, and we get 0.432+0.216=0.648. So red has a 64.8% chance of getting elected. You see what happened there? 60% of the beans were red, but that led to a 64.8% chance of picking mostly red beans. The more beans we pick the more the odds are going to amplify like that. Here is our first problem: single-winner plurality systems naturally do NOT result in proportional representation. They instead follow what's called a "cumulative normal distribution" function, which looks like this:

And that leads directly to our second problem: wasted votes. Once one party has a slight majority, total victory is all but guaranteed. Anything short of that, defeat is almost certain. So if you're a voter in a district where one party has a sizeable lead in the polls -- and by "sizeable" we're talking like 5% here -- it's very unlikely your vote is going to make a difference. And that's regardless of whether you'd be voting for the losing party or the winning party. Overtime people start to realize this, and people start developing the attitude: "there's no point in me voting, the outcome is already guaranteed in my district: party [x] will win." And for the most part, really, they're right. And so what you get is a drop in voter participation, that just gets worse and worse over the years. And before you know it, you have dismal voter turnout rates. Which is what we have. Compared to any other country, our voter turnout rates are dismal. And a huge part of the reason for that, is that in a first-past-the-post voting system, not turning out to vote is often a fairly rational choice -- since your vote would just be wasted. First-past-the-post voting systems lead to a high number of wasted votes, which leads to learned helplessness, which leads to complacency.

Now let's throw in a third party, say green. Let's say that our bag is 45% red, 45% blue, 10% green. Let's say we picked 9 jelly beans. You see green's problem here: in order to get elected they need more green beans than either of the two other colors. We'd need at least 4 green beans in our handful. The likelihood of this is very slim. MUCH lower than 10%. Red and blue still have about a 50% chance of having a plurality, while green, still pretty much 0. The further green falls short of being tied with red and blue, the chances of it getting a plurality in our handful drops exponentially. And we've come to our third problem with single-winner plurality systems: it makes third parties practically impossible.

And last, but certainly not least, how it enables gerrymandering... Recall in the first problem, red could have either 2 OR 3 jelly beans and would still win exactly 1 election. Furthermore, due to the amplifying effect - red only need win a slight majority in order to have a high likelihood of securing that election. A 60% lead translates to a 64.8% chance. And that was only from picking 3 beans. In an actual election, there are hundreds of thousands of voters -- we are picking hundreds of thousands of jelly beans -- so the effect is much more extreme. These 2 properties of single-winner plurality systems,

  • that whether a candidate gets 51% of the vote or 99% percent of the vote, the result is the same,
  • and that any slight advantage is amplified exponentially
means that if you can move red beans around between handfuls -- if you can move partisan voters between districts -- you can control how many elections have a safe majority for one party -- how many handfuls have more red beans than blue. That is, these two properties, taken together, make it possible to gerrymander.

In conclusion, and to reiterate, it is the "first-past-the-post" or "single-winner plurality" voting system that causes a host of problems for democracy, including:

  1. the results are not proportional,
  2. a large fraction of the votes are wasted, leading to low voter participation
  3. it forces a two-party system, and
  4. it allows for gerrymandering.
A simple, and fairly obvious fix for ALL of these problems is to change our method of assigning winners to a multi-member proportional system.


2016.01.21: (Blog) The Gold Standard of Election Fairness: Bayesian Regret

The problem of redistricting falls under the branch of mathematics known as "Decision Theory". According to Wikipedia, Decision Theory "is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision."

So what is the "optimal decision"? Well, to put it in a phrase, the optimal decision is the decision that minimizes "Bayesian Regret".

What is "Bayesian Regret"?

Put simply, the "Bayesian Regret" of a redistricting plan is the expected avoidable human unhappiness caused by using it.

To unpack this: Let's say each citizen assigns each candidate a "utility" value. Now the best possible outcome is of course when the candidates that get elected are precisely those candidates that maximize this "utility" value, over the sum of all the citizens' ratings. But in a real voting system, that's not what's going to happen. The actual results will fall somewhat short of this ideal. The amount that the actual results fall short of this ideal maximum utility choice is called the "Bayesian Regret".

If a rule increases Bayesian Regret, you know right away that it's a bad rule and you shouldn't use it. For example, so-called "communities of interest" increase Bayesian Regret, so you know right away that they're bad and you shouldn't use them.

  • Any rule that increases Bayesian Regret is a rule fundamentally opposed to the will of the people.
  • Any rule that decreases it leads to an outcome that voters like more overall, by definition.

So you see it's the apex criteria. It tells you quantitatively how bad the final outcome was compared to what voters really wanted; compared to the best possible outcome, as judged by their own assessments. Bayesian Regret is the gold standard for election fairness. From it, everything else follows.

Bayesian regret is well-vetted and has been for centuries.

Bayesian Regret is not a new concept. It dates back to the earliest days of statistics (1800s) and it has been used in at least a dozen papers on non-voting-related subjects. The only thing "new" here is applying this well known concept to voting methods. And that idea is not entirely new, either. It's been discussed, among other places, in papers by Samuel Merrill (Princeton University Press (1988)) and Robert Bordley (American Political Science Review, 77 (1983)).

Mathematicians everywhere recognize the specialness of Bayesian Regret; There's no dispute about it -- it's king.

Bayesian Regret is the mother of all Auto-Redistrict fairness rules.

"Bayesian Regret" is the arcstone of the program's criteria. It is THE litmus test; the Omega. Every fairness criteria available in Auto-Redistrict asymptotically decreases Bayesian Regret. None of them increase it. This guarantees that any and all optimizations done by Auto-Redistrict will increase average voter happiness.

For more information...

  • For a broader overview of Bayesian Regret and its use in decision theory, I recommend starting with Wikipedia's article on the "Loss function".
  • If you want a book about the subject and how it relates to elections, I recommend "Gaming the Vote: Why Elections Aren't Fair (and What We Can Do About It)" by William Poundstone.
  • For more information about Bayesian Regret and its application to voting systems, I highly recommend you check out RangeVoting's introduction.


2016.01.18 (MLK Day): (Blog) Enumerating the Nuances

So I decided to try to enumerate all the technical nuances of redistricting reform, and try to organize it into categories and a 1-dimensional ordering that flows well. Call it my natural compulsion to "systematize". A tell-tale trait of a software developer.

The list was longer than I anticipated. But I think from it you can get a good sense of the issues. And a good sense of the great value of systemizing. It allows you to identify how the different issues relate, so that you can solve them one at a time in a way that guarantees eventually solving the whole thing. And you can identify where the critical points are. The "pivot points", or "centers of gravity", to borrow a phrase from Clausewitz. See if you can find them.

Anyways, without further ado... forgive the lack of explanation for the items, it's meant to be an organized list - a map. As such it more resembles notes or a table of contents than anything else:


PART I: THE PROBLEM

category: the two types of gerrymandering: 
choosing the voters, and choosing the candidates.
* vote packing
** partisan - reducing competitiveness of districts that favor a party
** racial - reducing competitiveness of districts where certain racial demographics are
** "communities of interest" - like racial vote-packing, but can be used for any voting group.
* candidate packing
** hijacking - forcing candidates to run against each other or not run against each other
** kidnapping - forcing a candidate to a place where they'd lose or win
** "communities of interest" - choosing a candidate by packing the voters together.


category: other effects of gerrymandering (besides controlling legislation):
* gerrymandering also reduces voter turnout, since people realize their vote doesn't count.
** this effect tends to exacerbate the partisan advantage gained by gerrymandering


category: how is gerrymandering possible?
* single-winner system enables gerrymandering
   since a 49-51 vote has the same result as a 1-99 vote, votes can be shifted around
   to control the number of a certain result
* single-winner system forces two-party system 
   (since likelihood of getting elected drops exponentially with %vote)
* congress, votes party line
   thus, all that matters is which party has the majority
   that determines what bills get passed and what bills don't
* since congress votes party lines, and can gerrymander, 
   map drawers can control what legislation gets passed by gerrymandering
   (by partisan and racial vote packing)
   FOR THE NEXT 10 YEARS!
* multi-member solves both problems ( FairVote.org )
** prevents gerrymandering
   because now a 49-51 vote doesn't have the same result as a 1-99 vote, 
*** note it also makes candiate packing far less effective
   (since a candidate will automatically have more competitors and a larger voter base)
** allows 3 or 4 party system
   since now a candidate only needs 10% of the vote (1/2 of 1/5th)
   AND that % is drawn from a larger voter base. (since the district is bigger)
*** enabling a third party reduces effectiveness of party-line voting.


category: descriptive vs representattive, and the real impact of "communities of interest"
descriptive remedies:
** communities of interest
** VRA
*** loved by the gop - because it helps them (by vote-packing dems)
** congress, votes party line, so the only thing that matters is which party gets the majority.
  vote packing, regardless of what you call it (communities of interest, etc.),
  reduces the representation those voters prefered party gets in congress.
  thus reducing the likilihood that their prefered policies get enacted.
  in other words, "communities of interest" always _disfavors_ those communities' political interests.
** communities of intereest and VRA "descriptive" interpretation give legal blank check to gerrymander.
   to quote Christopher Ingraham from the Washington Post:
   
"It's a point well-taken. But "community of interest" is an incredibly squishy term. You can define it pretty much however you want. 
As I wrote in 2014, if you're a politician in search of a figleaf justification for putting voters from disparate corners of the state 
into the same congressional district, you can always find one. Communities of interest are a great ideal, but in practice they're so fuzzy 
that they open the door to all manner of redistricting shenanigans, as we've seen."

"The main obstacles to automated redistricting are legal. For starters, the Voting Rights Act mandates that in some states, 
race needs to be a factor in redistricting to ensure that minority voters are represented in Congress. 
Again: a nice idea. But there's a tradeoff: packing all your minority voters into one district diminishes their clout everywhere else. 
We've seen this in the real world in Florida: the 5th District was originally drawn as a majority-minority district by Democrats. 
But Republicans saw fit to keep it that way in subsequent years, because it gave black voters less power in the surrounding districts."
 -- from "This is actually what America would look like without gerrymandering", by Christopher Ingraham, Washington Post 
   ( https://www.washingtonpost.com/news/wonk/wp/2016/01/13/this-is-actually-what-america-would-look-like-without-gerrymandering/ )
   

PART II: THE SOLUTION

category: "Is there an identifiable goal that can be stated in language the average voter will understand?": 
* role of competiveness / voting power
* restating as a communication problem: the goal is to maximize bandwidth.
** restating it that way gives you a very definite solution:
** maximize entropy of the composition of congress
** maximize mutual information between composition of congress and the votes of the citizen.
* this transaltes to:
** maximize overall competitevness of elections (= maximum entropy)
** make each person's vote count the same (due to the gibbs inequality)
*** making each person's vote count the same means making each voting group's elections be equally competitive
* this can be restated as preventing vote packing.
** how to measure and prevent vote packing:
*** maximal competitiveness
*** racial equality
*** partisan equality
** obstacles to preventing vote packing:
*** current software lacks analysis of vote packing (excepting auto-redistrict)
*** the compactness rule disfavors clustered demographics - tends to vote-pack them.
*** the are no laws against it, in fact there are laws FOR it!!:
**** the descriptive interpretation of the VRA legitimizes racial vote-packing.
**** "communities of interest" legitimizes vote-packing across the board
***** and tricks those communities into thinking it's good for them.


category: geometric approach doesn't work
* compactness is a common _symptom_ of gerrymandering, but... 
** but compactness doesn't neccessarily imply the absence of gerrymandering, 
** nor does non-compactness imply the presence of gerrymandering.
* compactness rule (and minimal splits rule) is biased against clustered demographcis
** most notably dems
** and minorities
* can be counteracted by fairness rules


category: fairness rules, how to do them
* first you need the election data, at voting tabulation district resolution. (the highest that's available)
* use only the _empirical_ data; the actual vote counts
* remember to "impute" the uncontested elections (see page 27 & 28 of "PARTISAN GERRYMANDERING AND THE EFFICIENCY GAP")
* all rules must increase the bandwidth (amount of information communicated) from voters to composition of congress.
* rule 1: maximize overall competiveness - this means that more information will be transfereed to the result
** particularly you minimize the sum total of the victory margins
* rule 2: minimize racial/demographic vote packing (per gibbs inequality)
** particularly you do a by-district weighted sum of # of wasted votes, weighted by % population,
    divide that by # of total votes, weighted by % population.  That's your wasted vote rate.
    to get how unequal it is among demographics, you:
    1. normalize it to 1 by dividing by the average wasted vote rate
    2. take the statistical variance of that among different ethnicities.
    the rule should be to _minimize_ this variance.
* rule 3: minimize partisan vote packing (per gibbs inequality)
** single winner elections: maximize seats-votes curve symmetry (total displacement from symmetry)
** multi winner elections: maximize seats-votes curve diagonalness (total displacement from diagonal)


category: need for transparency and citizen vigiliance-
* GIGO (garbage in, garbage out) - programs can be tricked, etc., by feeding it false data.
   deception can bypass everything.  (sun tzu: "war is all about deception")
** data must be public and geocoded in simple format, like GIS format. Wisconsin GAB is a great example.
** public must have free and open source tools for analyzing the data, and the math and computer skills to do it.
* code must be open-source
** _and_ people must actually review that code.
*** means you need good compsci, math, and logic public education.
** also must be open-documentation; methodology must be documented and that documentation public.


category: a few different models, and their shortcomings:
* the california model - lacks any rules.  you fix gerrymandering by controling the rules, not the people.
al it does is remove the fence from the henhouse, which doesnt prevent the fox from getting in.
* ohio model - nice that it's rule-based, but disfavors demographics that cluster.
* independant commision - prolly better, but like california model, it's still anarchy.  need to define the rules.


category: on legislation (putting it all together):
* need to see the data (some legislation prohibits its use)
* some legislation neglects that districts can't be drawn in isolation - moving a line moves two bordies, 
and moves people to one district as much as it takes from the other.
* some proposed legislation contradicts itself, e.g.:
   "cant be drawn to favor or disfavor a party", 
   yet allows for drawing communities of interest, 
   which disfavor that community's prefered party.
* legislation really needs to be rule-based, deciding the people doesn't make as much of a difference.
** rule category: objective rules about the statistical consequences of the map
*** example rule: outlaw vote-packing (both partisan and racial), as determined _empirically_, compared to less packed maps.
*** example rule: maximize overall competitiveness
** rule category: transparency
*** example rule: publicly publish election data at tabulation district level gis file, along w/census and demographics.
*** example rule: publicly publish statistics on how much vote packing the maps do, by party and by race.
** rule category: election system / voting method
*** example rule: multi-member districts


PART III: other types of voting rights abridgment.

category: sly tricks that have the impact of gerrymandering (abridging voting rights);
(disproprtionally inconveniencing voters)
* id's (..and then closing dmv's in certain areas)
* lack of a voting holiday (disproportionally effects low-income voters)
* driving distance
* insufficient voting equipment, staff, etc. in certain areas
* early voting
* poll hours



2016.01.13: (Blog) On Racial Vote Packing and the Voting Rights Act

District lines sometimes minimize the influence of minority voters by disproportionately consolidating them within single districts. This is called vote packing. Packing minority votes together abridges their right to vote. It makes their votes count less. The Voting Rights Act (VRA) expressly prohibits abridging minority rights to vote under color of law. Ironically, people have tried to use the VRA to justify racial vote packing. They abridge minority voting rights under color of precisely that law that says you shouldn't abridge minority voting rights under color of law.

In fact, vote packing doubly disenfranchises. Firstly because it reduces the impact of the person's vote by putting them in a district where the outcome is already predetermined. But secondly, because it spreads that group of voters out among fewer districts, thus reducing the number of representatives they have representing them. It diminshes not only their ability to make their voice heard at the voting booth, but also their ability to make their voice heard when congress votes. Packing a voting group's votes always disfavors their preferred political party, potentially preventing them from getting a majority in congress, when they otherwise would. And in the highly polarized political climate that we are in, where nearly every vote is "down the aisle", that determines everything.

I ran the numbers for the Wisconsin 2012 assembly elections, and in those elections, black voters' votes counted a little less than half as much as everyone else's. More precisely, they had twice the rate of wasted votes per vote than the average voter. It's not an exact linear relationship there - to go from that to saying their votes count half as much. Arguably it's actually worse than that because as elections get less competitive they get exponentially harder to flip. And as it turns out, more than half of the votes they cast were votes that were not needed to win the election - that is, more than half of their votes were wasted. In contrast, white voters had less than a quarter of their votes wasted. This disparity is, of course, obscene. It's reminiscent of the days before the civil war, when blacks were only counted as three-fifths of a person. And this is the kind of thing that people have been trying to use the VRA to justify.

Ethnicity Population Vote dilution % Wasted votes Victory margins Votes
AMINDIAN 68,593 0.802077340 0.209887744 6,937.152939065 33,051.729537655
ASIAN 143,931 1.250616449 0.327261539 23,522.090742586 71,875.512175392
BLACK 380,660 2.184373010 0.571607125 98,877.584928540 172,981.722117862
HISPANIC 336,056 1.265430649 0.331138122 47,708.575761542 144,074.549628278
OTHER 5,249 1.237628361 0.323862814 841.499817613 2,598.321823849
OTHERMLT 11,447 1.492101313 0.390453342 2,135.224081484 5,468.576782877
PISLAND 2,639 0.994068811 0.260128107 339.005569841 1,303.225454070
WHITE 4,738,411 0.891605773 0.233315560 555,098.866159329 2,379,176.362480017
TOTAL 5,686,986 1 0.261680181 2,810,530.000000000 735,460.000000000

How much a vote counts is determined precisely by how competitive the election is. Each person's vote should count about the same, regardless of their ethnicity. The principle of "one person, one vote" requires therefore that no ethnicity is packed into disproportionately uncompetitive elections. The voting rights of minorities shall not be abridged.


2016.01.07: Fixing Wisconsin's Heavily Gerrymandered Districts

Wisconsin Public Radio

Research shows that Wisconsin is among the most gerrymandered states in the country. Both Democrats and Republicans are guilty of redrawing lines to their own political advantage. Our guests will talk about a pending lawsuit that challenges the redistricting process and highlight some possible solutions.

listen - download


2016.01.07: Can New Approach End Gerrymandering?

by Bruce Thompson

The challenge to Wisconsin’s partisan legislative districts recently survived its first hurdle. A three-judge federal panel rejected motions from Wisconsin Attorney General Brad Schimel to throw out the lawsuit on several grounds, including lack of standing on the part of the plaintiffs and the assertion that the US Supreme Court has already rejected their claims.

While the decision itself was largely about process, the three federal judges proved remarkably clear eyed. Schimel argued that Democrats fared poorly because they cluster in cities:

Democratic supporters are concentrated in specific locations such that they naturally are “packed” into districts drawn using ordinary districting principles.

The judges did not buy this argument, instead describing how Republicans set out to gerrymander the legislative districts.... Read the full article at UrbanMilwaukee.com


2015.12.28: (Blog) The Three Ways to Redistrict, and their Results

While every state has their own minor variations in redistricting procedure, they all fall into one of three basic categories:

  • Anarchy (not rule-based)
  • Naive (rule-based, no data)
  • Informed (rule-based, data)

Anarchy (not rule-based) -- In this procedure, no rules are enforced except for the constitutionally mandated equal population. I was going to say that contiguity was also a rule enforced here -- that districts aren't broken up into pieces -- but Maryland serves as a rather egregious counter-example to that. So, only equal population. Basically the people drawing the maps can draw them however they like. If you don't like the map, the only recourse you have is through the court system, and that'll cost the taxpayers literally millions of dollars (they pay for both plaintiff and defendant). Historicaly, most such challenges have failed.

Naive (rule-based, no data) -- In this procedure, rules are used to construct a map, and the rules pretty much uniquely determine the map. So this eliminates the possibility of deliberate human manipulation, except insofar as the choice of rules is deliberate and done by humans. However, no data about voting is used, only geometric data (compactness, county splits, and contiguity), so the method is essentially blind. It is tempting to say that it perfoms as good as randomly picking a map out of a hat, but that would be incorrect. Rules of compactnes, fewest county splits, and contiguity, are neither random nor neutral. Depending on the political geography, these rules may either result in a more fair or less fair map than a random map. To find out which, we'd have to use the voting data. But this method expressly prohibits use of the voting data.

Informed (rule-based, data) -- In this procedure, rules are used to construct a map, and the rules pretty much uniquely determine the map. So this eliminates the possibility of deliberate human manipulation, except insofar as the choice of rules is deliberate and done by humans. However, unlike the naive approach, election data is used to satisfy rules about competitiveness, partisan symmetry, and racial equality. This approach is thus guaranteed to score better in those categories than the two other approaches.

So how often is each method used? Well, surprisingly enough, 49 out of 50 states use the anarchy method. The one exception is Iowa, which uses the naive method. Not a single state uses the informed method.

Now let's take a look at the results of each method. Below is a comparison of the results of each method, used to redistrict the Wisconsin Assembly, using (or not using, as the case may be) 2012 election data. The results of the anarchy method, used by 49 out of 50 states, are, as one might expect, horrendous. The naive method is considerably better, but it turns out much worse than a random map on partisan symmetry, competitiveness, and racial equality. Of the three approaches, the informed approach is, unsurprisingly, the best, and by no small measure.

Anarchy method (current)
(vtd data)
Naive method
(vtd data)
Informed method
(vtd data)
Random map
0.8077487948.0090579719.004492363176.847200658Population imbalance (%)
0.2135151304970.4161503240.330797411770.0158291520260Compactness (isoperimetric quotient)
0.0456560620.028350840.0050253150.002204086Partisan asymmetry
735,460673,03583,326174,308Competitiveness (victory margin)
0.1711289850.1674850710.0380823430.021844718Racial vote dilution


Population imbalance % is calculated as the population of the highest population district, divided by that of the lowest population district, times 100.

Compactness is calculated as the isoperimetric quotient - area divided by perimeter squared, times 4 pi. Borders on state edges were counted half as much, since they can't be moved.

Partisan asymmetry is the total deviation of the seats-votes curve from symmetry. Symmetry being where if party A gets y% of the seats when they get x% of the votes, party B should also get y% of the seats when they get x% of the votes. The seats-votes curve is computed by applying a uniform partisan swing from 0% to 100% at 0.25% intervals. The deviation from symmetry is added up over all vote %'s from 0 to 100. The result ranges from 0 to 1, with 0 being perfect symmetry, and 1 being one party always getting all of the seats.

Competitiveness (victory margin) is the sum total of the victory margin (votes won by) over all districts.

Racial vote dilution is calculated as the variance of the victory margin per vote, among the different ethnicities. For example, If district A has a vote gap of 10,000, 50,000 total votes, and 20% are ethnicity 1, then 10,000*0.2 is added to their total vote gap, and 50,000*0.2 is added to their total vote count. If district B has a vote gap of 20,000, 60,000 total votes, and 10% are ethnicity 1, then 20,000*0.1 is added to their total vote gap, and 60,000*0.1 is added to their total vote count. All districts are totaled this way for every ethnicity. Then their total vote gap is divided by their total vote count. The statistical variance of this figure about the population mean represents the amount of racial vote dilution; the amount that some races votes are diluted more than others.





2015.12.26: (Blog) My recommended changes to Wisconsin Assembly Bill 328

My good friend and representative, Jonathan Brostoff, forwarded me a link to a redistricting Bill ( https://docs.legis.wisconsin.gov/2015/proposals/ab328) that they are pushing for in the Wisconsin Assembly. Having read through that, I have objections to some of the items. I know the intentions are good, but there are some problems in the bill that I believe stem from some common misconceptions about how the math of redistricting and voting theory works out. So I sent him a list of recommended changes to the bill to remedy these unintentional issues. I decided to post those recommendations here, since I think these misconceptions are common across the country, and it would be good to clear them up publicly, for the benefit of both legislatures and voting citizens.

Recommended amendments to Bill 328 (related to legislative and congressional redistricting):

The current bill, due to its false assumption that political geography is inherently neutral, would un-intentially make into law systemic partisan gerrymandering. For a detailed analysis demonstrating the alleged effect, refer to the paper by Jowei Chen and Jonathan Rodden, "Unintentional Gerrymandering: Political Geography and Electoral Bias in Legislatures", available at: http://www-personal.umich.edu/~jowei/florida.pdf Below are a number of recommended amendments ( 3 additions, 2 modifications, and 1 deletion) to remedy this situation.

Rule additions:

  • Maximize the overall competitiveness:

    Minimize the total vote gap, where the vote gap of a district is calculated as the difference in votes received for the two leading candidates, and the total vote gap is counted as the sum of that over all districts.

    For uncontested elections, the presidential vote should be taken, party for party (where available), and the total vote for each party over all _contested_ elections should be matched to that of the elections in question. Then this ratio, party for party, should be taken and applied to the presidential vote counts in the uncontested districts, thus creating substitute vote counts that are matched to the partisan swings of the elections in question.

    Reasoning:
    a) the value of a person's vote is determined by their ability to elect a candidate of their choosing. The less competitive an election is the less ability a voter has to elect a candidate of their choosing. Uncompetitive elections amount to the map-drawer deciding the elections, competitive elections amount to the voters deciding the elections.
    b) incumbents should not have an innate advantage. Uncompetitive elections give incumbents an innate statistical advantage.
    For instance, if the chances of party A winning vs. party B winning is 30-70,
    There's a 30% chance it's party A, and from that it's a 30% chance they will get re-elected, so that's 0.3*0.3 = 0.09.
    There's a 70% chance it's party B, and from that it's a 70% chance they will get re-elected, so that's 0.7*0.7 = 0.49.
    0.09+0.49 = 0.58, so that's a 58% chance that the incumbent would get re-elected. Contrast with a competitive election which would be simply 50%.

  • Minimize partisan gerrymandering:

    For single-winner elections, minimize partisan asymmetry. Where partisan asymmetry is measured by the total deviation of the seats-votes curve from partisan symmetry, over all partisan skews from 0% to 100%. (each vote % counted equally) "Partisan symmetry" meaning that if x% votes for candidate A results in y% of the seats for candidate A, then x% votes for candidate B results in y% of the seats for candidate B.

    For multi-winner elections (of which there should be no less than 3 members per district), minimize the total deviation of the seats-votes curve from the diagonal. (diagonal meaning % seats = % votes)

    For uncontested elections, the presidential vote should be taken, party for party (where available), and the total vote for each party over all _contested_ elections should be matched to that of the elections in question. Then this ratio, party for party, should be taken and applied to the presidential vote counts in the uncontested districts, thus creating substitute vote counts that are matched to the partisan swings of the elections in question.

    Reasoning:
    The ability of a voter to elect a "candidate of their choosing", and for that power to be balanced among all citizens, regardless of political orientation -- in other words, to guarantee that no citizen's freedom of speech is unduly abridged based on their political leaning -- requires that each citizen, regardless of where they lie on the political spectrum, has the same ability to translate their votes into seats.

  • Where possible, voting tabulation districts should not cross natural borders, such as rivers

    Reasoning:
    This is to prevent voters access to the polls from being disproportionately impeded based on political geography.

Rule modifications:

  • Compactness measure:

    The compactness of a district should be counted as the iso-perimetric quotient, that is, area divided by perimeter squared, divided by the ratio of a circle's area to its perimeter. Though lengths on the border of the state should be counted half as much, since these cannot be moved. The total compactness over all districts should be counted as the reciprocal of the sum of the reciprocals of the compactness of the individual districts.

    Reasoning:
    The proposed measure of weighted distance of population from the centroid counts shapes that mix high population areas with low ones as less compact than ones that don't mix, and this:
    a) is not relevant to geometric compactness,
    b) causes the measure to count some districts that are actually more compact than another to be less compact than it,
    c) has the effect of packing votes based on population density, which
    d) disproportionately packs democratic voters, thus causing a systemic partisan gerrymander.

  • Split minimization:

    Remove splitting the highest population county first, and splitting the fewest number of counties. As a substitue one could just use having the fewest total splits. This would be interchangeable with compactness, while not impeding fulfillment of the other criteria nearly as much.

    Reasoning:
    a) This rule makes assumptions about the political geography that are not tested and indeed are unlikely.
    b) Furthermore it impedes fulfillment of both the maximize competitiveness rule and the minimize partisan gerrymandering rule.

Rule deletions:

  • Delete standard #6 relating to use of voting power data:

    Standard #6 must be stricken because it contradicts itself, thus invalidating itself. In order to ensure that districts are non-partisan, you need to use previous election results.
    Indeed, that is how you would determine whether a map unduly favors or dilutes.
    Previous election results _must_ be used in order to ensure fair representation and VRA compliance.

    Reasoning:
    This rule asks for maps that neither favors a party or voting bloc, nor disproportionately dilutes the votes of one. Yet it simultaneously prohibits use of precisely the data that one needs to make such maps. It is not possible to deliberately create a redistricting proposal that meets both of these parts simultaneously. Thus, the only available courses of action are:
    • that at least one of the two parts must be stricken,
    • or the whole rule is stricken,
    • or a redistricting plan is proposed that violates at least one of the parts of the standard,
    • or no redistricting plans are proposed.

    Option 4 is not acceptable, and option 3 is in practice equivalent to option 1 or 2. So in practice the only available courses of action are option 1 or 2.
    Since the first amendment requires that a person's voice should not be unduly abridged on the basis of race, creed, color, etc., at minimum, the part that prohibits use of data needed to guarantee the first amendment must be stricken.


EDIT: To demonstrate the results of these amendments, I went ahead and used auto-redistrict to construct a map based on the original bill, and a map based on said bill amended as proposed. Below are the results. One caveat, though: the "Bill 328 original" map used iso-perimetric quotient instead of the original proposed measure of compactness. The actual compactness score of Bill 328 original would be worse than shown. Notice a dramatic improvement in partisan symmetry, and nearly 100,000 fewer disenfranchised voters.

Current map (2010)Bill 328 originalBill 328 with changesMeasurement
0.213515130497147320.4161503240.33079741177Compactness (isoperimetric quotient)
0.8077487948.0090579719.004492363Population imbalance (%)
0.0456560620.028350840.005025315Seats / vote asymmetry
735,460673,03583,326Competitiveness (victory margin)
0.1711289850.1674850710.038082343Racial vote dilution


I should add, then, another suggested rule to improve VRA compliance:
  • Racial vote dilution minimization:

    Each ethnicity should have about the same voting power per person, measured by the average competitiveness of elections they are in, weighed by the fraction of the population they represent in each district.
    For example,
    If district A has a vote gap of 10,000, 50,000 total votes, and 20% are ethnicity 1, then 10,000*0.2 is added to their total vote gap, and 50,000*0.2 is added to their total vote count.
    If district B has a vote gap of 20,000, 60,000 total votes, and 10% are ethnicity 1, then 20,000*0.1 is added to their total vote gap, and 60,000*0.1 is added to their total vote count.
    All districts are totaled this way for every ethnicity. Then their total vote gap is divided by their total vote count. The statistical variance of this figure about the population mean represents the amount of racial vote dilution; the amount that some races votes are diluted more than others.

    Reasoning:
    To comply with the VRA, to give each voting block the same power to "elect a candidate of their choosing", and to protect against racial vote packing, whether deliberate or geographic.


2015.12.10: Can Auto Redistricting End Gerrymandering?

by Laura Thompson

Can gerrymandered by biased humans by replaced by the objective commands of the computer? Kevin Baas, a City of Milwaukee programmer, believes it’s possible, and has devised an intriguing solution he has already submitted to the Florida legislature to solve a dispute over redistricting.

The problem he sought to solve is suggested by results of the 2012 Wisconsin state assembly elections: Democratic candidates received 53.5 percent of the statewide vote and yet won only 39 percent of the seats. This came in the wake of the 2011 redistricting effort, which was undertaken by members of the GOP-controlled legislature. Opponents have called this one of the most gerrymandered states in the U.S. and have argued that Republican legislators used the redistricting process as a chance to enhance their majority — both by “packing” Democratic voters into fewer districts, and also by “cracking” remaining Democratic areas into small pockets of Republican-dominated districts... Read the full article at UrbanMilwaukee.com

2015.09.03: First Official Release of Auto-Redistrict

Auto-Redistrict is ready for public consumption! The latest stable release can be downloaded from github at https://github.com/happyjack27/autoredistrict/releases .

I get asked on occasion what the intended audience is. The answer is everyone. Ordinary citizens (e.g. you), lawyers, congresspeople, independant commissions, reporters, etc... The more people to review how districts are drawn to make sure they're fair, the better. As they say in the coding world, "given enough eyeballs, all bugs are shallow".


The release contains a plethora of features, and has been thoroughly tested on a variety of states and elections. Remember, Auto-Redistrict is free and open-source. So use and redistribute as you please.

Any questions or concerns about the software or how it works, don't hesitate to contact me by email at happyjack27 [at] gmail [dot] com. You can also just peruse the actual source code, published online at https://github.com/happyjack27/autoredistrict. A snapshot of the source code is included in every release.

If you want to contribute to the project, just fork it and send me a merge request. (Fair warning: I will review all of your code thoroughly!)