The Intellectual Wilderness There is nothing more useless than doing efficiently that which should not be done at all.

2021.03.11 16:31

Social Trust: Why crypto, blockchain and smart contracts will not change the world

Lately I have been involved in a number of discussions about distributed systems, delegated authority structures, cryptocurrency and smart contracts. The hope of many is that cryptocurrency will avert inflationary trouble entirely and help the public hold government in check. There is also hope that smart contracts will magically turn Africa into a paragon of high trust economics and a trade powerhouse of its own accord.

I decided to record my thoughts on this here:

The TL;DW of it is that while these are really great things to hope for, and not impossible goals, the idea that crypto technologies are going to manifest profound levels of social change on their own is misguided. The social and political problems much of the world is facing today are due to an erosion of trust and flagrant, repeated violation of trust and faith from every actor at every angle. In short, the West is moving from a high-trust social model to a low-trust social model and Africa is and has ever been suffering from low-trust social models. Crypto isn’t how that is solved.

2020.11.13 12:37

Comments on Dr. Shiva’s Election System Analysis

Filed under: Computing,Politics / Geopolitics,Society — Tags: , , , , — zxq9 @ 12:37

UPDATE 2020-11-20: Scroll to the bottom for a follow-up.


Dr. Shiva performed a data analysis on the automated voting system results in the 2020 U.S. election and made a video presentation of it just a few days ago. I was asked to give my thoughts on it yesterday, so I watched it this morning and wrote some comments as I went along. The video link is below and my timestamped comments follow.

Votes are stored as “decimal fractions”? This would mean they are stored not as integer values, but as floating point values (unless there is a fixed-point library involved, but that seems extremely unlikely). This is insane. A vote can’t increment the tally by a fraction unless we’re trying to go back to the 3/5 rule or some other nonsense. Right off the bat this is pretty ridiculous.

“Provided SYSTEM cannot change the OUTPUT”.
A bit of elaboration… Of course the system changes the output (it generates the output), but given that the input has two parts, the ballot and the current count, vote(Ballot, Count) -> NewCount. the output value NewCount must be reproducible every time in a consistent way given the same initial inputs. Because we have an iterative counting system, this means the counts should be replayable transactions. The system must be able to fast-forward or rewind to any point in the sequence of transactions. As vote counting is distributed, this also means that each instance of a vote counter must record the sequence in which it counted its votes so that the aggregated total can be broken down and each part replayed, and that the aggregator also log and be capable of replaying each instance of aggregation of a new total update. His statement is correct, but it is useful to break down why it is both correct and also somewhat of an oversimplification.

Ballots are converted to ballot images (a digital representation of the ballot generated by the scanning process). The ballot images are being saved in some places and destroyed in others. This could be OK if the physical ballots are available for replay/recount and there was never any need to conduct a forensic analysis of the operation of the system, but this is not the case and there appears to be no standard for what to do about them. Deleting the ballot images prevents any possibility of comparing the counted ballot images and the original ballots, and this prevents a full inspection the system.

“Weighted votes?” There must be some explanation for this. Weighted voting is insane. It is supposed to be 1 vote 1 increment. What would be a legitimate explanation for having a weight system? This may be the reason why floats (of all insanity) are being used to store the vote tally.

The system DB is written in Access? Like MS Access? Is this accurate? Holy shit… No way that can be accurate. And yes, the numbers are indeed stored as floats (a double, specifically)



WAAAAT? This would mean that higher % Republican straight party vote correlates to lower individual candidate vote support. That also means he gets much higher individual vote support in Democrat straight party vote districts. This is extraordinarily unlikely given that he has 98% support across Republicans.

The phrase “taken from Trump and given to Biden”. I’m not seeing the basis for this yet. It is possible, however extremely unlikely, that voters actually voted this way. So I don’t agree with the language used in the discussion as a first-time watcher who does not know what else is coming up later in the talk, but the graph is clearly too structured to be true without some sound explanation of mechanism (and to be fair, the explanation that voters are more likely to dislike Trump the more likely they are to like Republicans is almost impossible to believe, but I’m keeping an open mind for the moment…).

Ah. They mention the same thing: It is “too structured” and “too perfect”. Because it is.

The idea that Trump did overwhelmingly better in Democrat districts than Republican districts is pretty hard to accept. The differences shown here are big enough for there to have some general revolt in the Republican base, which would have been very noticeable. This election Trump won the support of a large percentage of the original “Never Trumper” faction vote, a lot of the support from which manifested after the riots started. These vote tallies are grading on a curve really hard.

“You could even make the argument that even if you want to believe that Republicans hated Trump so so much, the larger that the population size was, they still wouldn’t be able to hate him in such a straight line.”

Wow. That looks a lot more like natural data. Though there is one really straight line hidden (downward) in there.

OK. Here is where he postulates the existence of an algorithm. He’s hinted about it up to this point, but this is where he comes out and says that it appears there is an algorithm that is applied to districts with a high percentage of Republican votes. That does appear to be the case from what is presented so far, though I would like to see some comparisons with districts of various alignment that were counted by hand and did not use these machines. The difference should be painfully obvious (in the same way the chart at 38:10 was striking).

The “Weighted Race” is a feature of the system that is documented? WTF? That’s insane. And yes, this looks very much like a weighted redistribution. The claim here is that all of the major vote counting system vendors have implemented this feature. Was that in the original contract? What was the motivation for its inclusion? Who came up with this nonsense? There must be some explanation for it to exist at all.

This is the core question: “Is it possible that this voting pattern is normal?” If it is then we should see it recur the same way with hand-counted votes. The slope, though, really does not make sense from the explanation that “Trump pissed off some % of Republicans” — the line would be flat, not a slope, and definitely not such a steep slope.

This is a pretty crazy story. Bad stuff.

YES. The idea of an auditable system is very important. That gets back to my original comments at the top about being able to replay each vote as an individual transaction, and each aggregate rollup event as an individual transaction, in addition to being able to independently verify that the ballot images match the paper ballots and that an alternative method of counting matches the results given by the automatic system over any random sample of ballots (hand counting results should match automated counts among random samples).

“Put a bunch of CPAs on this and they’ll tell you what the problems are.” Yes.


I was discussing with a mathematician about the best way to verify the above claims independently. His suggestion was that making an attempt to debunk the claims would provide the fastest path to verification: if we couldn’t debunk them then the claims are probably strong.

To debunk the claims (or alternately, verify them) would require performing an analysis on data from all districts in all counties in the state so we could compare histrionic data from each for anomalies. The reason a histrionic analysis is the most interesting has to do with the way a “weighted” vote algorithm would look over time compared to a natural count. To me the slopes that Dr. Shiva shows are less compelling (though interesting) than the histrionic plots that inspired him to perform an anlysis at all.

These two screenshots are interesting:

I don’t like that the second one is named only “Other Counties”. I have seen similar curves in other data, and it is indeed how weighted systems tend to plot unless the ballots are sorted before being counted. That is to say, a time sequence plot is interesting, a sorted count plot is not. There is not enough information here to know which one we are looking at though this is extremely likely to be a time series. We would need extremely complete data to evaluate this, though.

As for the slope that the next 40 minutes or so of Dr. Shiva’s video focuses on, that data is indeed quite interesting, but the average of the offset turns out to be more compelling than the fact that there is a slope at all.

In fact, another math channel on YouTube, “Stand-up Maths”, has performed a comparative analysis on the Biden data and found the same slope though its magnitude is different (and he never commented on that, nor did he comment on the time plot, which I thought was particularly weird to leave out since it is the original information that prompted the analysis in the first place).

It is an interesting video (though I don’t particularly care for the cheap ad hominem shots occasionally taken) and that channel has a ton of great math videos that have nothing whatsoever to do with politics (yay!) so go watch it at the link above if you are interested.

Unfortunately collecting data with the granularity necessary to perform the kind of analysis we want turns out to be possible, but quite time consuming. At the state level only aggregates are available, and those can’t tell us very much. Each county has the kind of data necessary to recreate the scatter plots (which turn out to be a bit less interesting than Dr. Shiva’s video make them out to be), but to the graphs of how the count tallies evolved over time is held by each district (and probably each county, but not available on the websites for each). The data is available publicly, but it is just very hard to get a hold of due to the sheer number of districts and counties in the state.


The details of the way the voting machine system is designed and implemented are clearly suspect. Verifying these details should be a priority, as should switching to an auditable and publicly visible system (hopefully something open source that lives on a public repo). The basic principles of its design and requirements seem antithetical to the way voting is supposed to work in the U.S.

As for the data analysis, there may be something weird in the data itself, but verifying it takes a lot of time just due to the level of detail required in the data (aggregates are useless to investigate the issues we really want to see), and whatever weirdness may exist doesn’t seem to have much to do with the bulk of both videos: the slopes shown.

2020.11.7 14:54

Out of the frying pan and…

Filed under: Politics / Geopolitics,Society — zxq9 @ 14:54

2020.09.3 15:35

Smart VS Wise

Filed under: Society — Tags: , , , , , , — zxq9 @ 15:35

A smart youngster and a crotchety old curmudgeon tended a farm together to feed their village. The crotchety old curmudgeon would only plant about half of the farm at a time, just enough to feed the village with a bit of margin for storage and enough left over to sell a bit to the caravan that would come by once a year.

The smart youngster wanted money. He was annoyed at the refusal of the crotchety old curmudgeon to maximize the area usage every season, and couldn’t understand why he wouldn’t allow them to try for three or four growing cycles across the year instead of just one or two.

One day the crotchety old curmudgeon finally died and the youngster got his chance. He planted everything he could everywhere he could, and he went even further: he put three seeds in each hole!

“Why not try for three times the crop from each unit?” he told himself, confident his smart plan would bring a tremendous bounty the likes of which the village had never seen.

The rains came and went, and the seeds started to sprout! But even though the whole farm sprouted at once, all that rose from the ground were stunted, sad, pale little shoots. No seeding occurred, no flowering, and no fruit or grain could be had from those stunted, sad, pale little shoots.

Half the village died that year and the smart youngster lost weight and fell deathly ill. He survived along with about half of the village, but just barely. Thankful to be alive but highly skeptical of smart, untested plans, he became well known for being quite a crotchety old curmudgeon.

2020.05.22 10:20

Economy: How to Spark the American Recovery

Sparking an economic boom in the United States is quite easy. American economic booms are based on two things:

  • Opportunity
  • Excitement

The necessary talent, geography, resources and hustle for success are basic to the American composition — by far the greatest strategic assets of the nation.

The necessary ingredient to for opportunity is merely the permission to try new things. This means one thing: Deregulation.

Deregulate! Opportunity is inherently exciting to the most aggressive Americans. Be explicit and loud about removing regulations. Remove from their minds the idea that “I have a good idea, but it’s probably too expensive to even hire a lawyer to find out if it’s legal to try. Or if I even start my business will probably just get banned and be left in the hole.” As a trivial example, stop issuing citations for kids trying to run lemonade stands (that’s a real thing — I’m not joking). It is an off-the-wall example and sometimes people might laugh about it, but it plants a poisonous seed in the minds of adults who become more fearful about even attempting to run even a simple business.

Deregulate deeply! Most of what passes for labor and safety law are anti-competitive barriers masquerading as public service. It is telling that most business regulations are and were lobbied for by the big, established players in the industries they regulate. Of course they aren’t looking to make their own business more difficult, they are instead looking to make the cost of entering into competition so high that small, aggressive players won’t even go to the trouble.

Deregulate SMBs! Let the SMB boom naturally come back. SMBs are the fastest to start up, fastest to pivot, fastest to hire, and lowest systemic risk to “the system” (no fears of “too big to fail”), and by far the most willing to innovate. This is even true in Africa — an entire continent that has hamstrung its SMBs and startups as an outcome of decades of chasing globalist Marxist rhetoric (it is easy to push such rhetoric there because it masquerades as “anti-colonialist”). No place can survive African style SMB regulations.

Deregulation, not bailouts, is what will allow Americans to save themselves from prior mistakes. Nothing else will. You have to spark activity or else the numbers on the dollar bills won’t mean anything to begin with. If the meaning of money changes (and there is a strong risk this may happen) no bailout will matter because the price of any random product could easily become “none to be had at any price“.

2020.04.30 22:38

Weirdly suppressed COVID-19 brief

Filed under: Society — Tags: , , , , — zxq9 @ 22:38

A few days ago a few doctors from California gave a brief on their analysis of both comparisons of COVID-19 historical data to prior predictions that prompted the “lockdown” trend, and side effects within the healthcare industry overall (focusing on the United States, specifically).

This brief has been uploaded and removed from YouTube and other video sharing sites a number of times, but I think it is an important part of the discussion and find censorship of it to be absolutely ridiculous.

A torrent file for the video is here: Dr_Erickson_COVID-19_Brief.torrent

The torrent’s magnet link:

2020.04.7 21:07

Emotional Investment and Statistics: A Thought Experiment

Consider how the public discourse over COVID-19 tends to evolve.

Let’s imagine we were to set up an online debate someplace known for civility and good faith arguments (you know, like Facebook or Twitter) between the “It’s all Trump’s fault and we are all going to die if we don’t mothball the world for a year” people and the “It’s just the flu, brah” people. What kind of statistics would we see thrown around to support their arguments (whenever numbers are involved at all, that is)? What sort of mortality rate would both sides argue?

The “mothball world” people will claim the worst possible estimates are true: “It’s like over 10% some days in Italy!”. The “just the flu” people will claim the best possible estimates are true: “Dude, it’s like less than 0.1% in Germany and New York and Iceland some days”.

Both sides will have extremely different talking points and both sides will be fully invested in believing their own version of reality so strongly that they will always find a way to dismiss the other side’s numbers, sources and ultimately arguments.

The thing we are certain to not see is a discussion between the two sides about establishing rules for what statistics and sources are permissible, what the rules of cause-of-death accounting should be, and how to reasonably interpret the vast difference between rates derived from transmission projections VS confirmed cases.

This is basically the debate that is raging on social media right now and it isn’t going anywhere or providing any new insights.

Now let’s imagine that we set up an online betting site where people are both making bets on what the general mortality rate estimate will be by the end of the year. On our imaginary betting site we can open the books for additional odds on what the mortality rate estimate will be by the end of the year within various age, gender, and medical category demographics and locales.

The community of people focused around this kind of betting site would be driven by a very different agenda, and the very most important thing they would all be seeking to establish is truth in reporting and rules for how statistics are kept. They must feel absolutely certain they aren’t throwing their money at phantoms and false data lest they be tossing their money straight down the drain. Their investment is in themselves, not any particular social or political “side”.

Considering how shocking the N-order effects of the ongoing economic shutdowns and social lockdowns are likely to be for generations of people whose hardest problem in life to date has been deciding between Android and iPhone, it is quite remarkable that the actual debate we see raging online looks like the first instead of the second example.

2020.04.5 14:44

For Data Reporting, Keep Reliable URLs

Filed under: Science & Tech,Society — Tags: , , , , , , , , — zxq9 @ 14:44

It would be really nice if governments that are trying to be transparent by reporting stats publicly on a website would stop changing their entire URL scheme every week. It is hard to track things down and even the search engines aren’t keeping up very well.

2020.02.3 15:19

X-Y Problems

Filed under: Computing,Science & Tech,Society — Tags: , , , , , — zxq9 @ 15:19

People obsess about their X-Y problems to the point of ignoring accepted wisdom, plugging their ears to the deafening silence of the solution’s instructive whisper, picking themselves up as hard as they can by their own knees and wondering why they can’t fly.

They then run off and formalize their wrong solution as a PR into a core project.

If core maintainers aren’t mindful they’ll incorporate these disturbances into a previously still space, and if they are indelicate they will piss off the misguided (but industrious) boob who made the PR who is already by this point fanatically dedicated to his wrong solution and the idea that nobody “gets it” but him.

Ah, another day at the Bazaar.

2020.01.29 10:20

People who can do, people who can’t…

Filed under: Computing,Society — Tags: , , , — zxq9 @ 10:20

I get these weird solicitations. “Want to write for the ____ code blog?” and so on. I’m sure that would probably be a good career move if I was looking to get hired away by someone else (the main problem there being that I can’t relocate — hahaa!). But what kills me is that people have so much time to write things other than code.

A: “Want to write [prose] for ____?”
B: “No thanks, I’m too busy writing code for [system].”

That’s the exchange I would expect is most common — except that it is 2020 and it seems that over the last several years people write more lines about writing lines than they write actual code these days (and I don’t mean they are making an effort at impeccable documentation).

« Newer PostsOlder Posts »

Powered by WordPress