On the idea that the polls got the election wrong

[Ivytalk]

It was the economy

It sure wasn't God!

[houndawg]

Since I did this today I'll post it. I looked at where the popular vote count was around mid day. It was at Hillary Clinton 48.2%, Donald Trump 46.1% . I did a simulation where I selected 10 independent random samples of 1000 each from a population where 48.2% voted one way and 46.1% voted the other way.

I got poll results ranging from Trump having the edge by 3 percentage points (48.3% to 45.3%) to Clinton winning by 10.1 percentage points (52.0% to 41.9%). And that is an idealized situation. That's a situation in which pollsters would be able to identify everyone who would actually vote then have every single one of the ones they selected for the sample cooperative respond.

Most people just do not understand what the situation is with survey sampling. They don't know how to interpret sampling error involved even with a perfect survey.

The fact that the polls as interpreted by the RealClearPolitics people were 38 of 39 in States where they actually indicated a favorite is absolutely incredible. People ought to be lauding them for what a good job they did instead of expressing their ignorance by saying "they got it wrong."

A famous statistician once said: Statistics are like bikinis, what they reveal is very interesting, what they don't reveal is vital.

[kalm]

Since I did this today I'll post it. I looked at where the popular vote count was around mid day. It was at Hillary Clinton 48.2%, Donald Trump 46.1% . I did a simulation where I selected 10 independent random samples of 1000 each from a population where 48.2% voted one way and 46.1% voted the other way.

I got poll results ranging from Trump having the edge by 3 percentage points (48.3% to 45.3%) to Clinton winning by 10.1 percentage points (52.0% to 41.9%). And that is an idealized situation. That's a situation in which pollsters would be able to identify everyone who would actually vote then have every single one of the ones they selected for the sample cooperative respond.

Most people just do not understand what the situation is with survey sampling. They don't know how to interpret sampling error involved even with a perfect survey.

The fact that the polls as interpreted by the RealClearPolitics people were 38 of 39 in States where they actually indicated a favorite is absolutely incredible. People ought to be lauding them for what a good job they did instead of expressing their ignorance by saying "they got it wrong."

A famous statistician once said: Statistics are like bikinis, what they reveal is very interesting, what they don't reveal is vital.

That's a great quote!

[houndawg]

A famous statistician once said: Statistics are like bikinis, what they reveal is very interesting, what they don't reveal is vital.

That's a great quote!

It was either Juran or Deming, I've forgotten which.

His example was the guy that goes to see his doctor after getting sick on three successive night from drinking whisky and tonic one night, vodka and tonic the second night, and gin and tonic on the third night. The doctor immediately saw the common thread and advised the man to avoid tonic.

That's JSO style statistication. It was OK for a while but he needs a new act.

[CAA Flagship]

I found out that smoking is the leading cause of statistics.
-Matthew Gray Gubler

[CAA Flagship]

Facts are stubborn, but statistics are more pliable.
-Mark Twain

If your experiment needs statistics, you ought to have done a better experiment.
-Ernest Rutherford

Definition of Statistics: The science of producing unreliable facts from reliable figures.
-Evan Esar

Statistics: the mathematical theory of ignorance.
-Morris Kline

[Ivytalk]

Facts are stubborn, but statistics are more pliable.
-Mark Twain

If your experiment needs statistics, you ought to have done a better experiment.
-Ernest Rutherford

Definition of Statistics: The science of producing unreliable facts from reliable figures.
-Evan Esar

Statistics: the mathematical theory of ignorance.
-Morris Kline

Quotes about statistics are worse than statistics.
-- Ivytalk

[kalm]

Facts are stubborn, but statistics are more pliable.
-Mark Twain

If your experiment needs statistics, you ought to have done a better experiment.
-Ernest Rutherford

Definition of Statistics: The science of producing unreliable facts from reliable figures.
-Evan Esar

Statistics: the mathematical theory of ignorance.
-Morris Kline

Quotes about statistics are worse than statistics.
-- Ivytalk

1 in 4 eyetalians are as literate as the average Louisinanan.

[YoUDeeMan]

I am wondering if it's gotten to the point where you guys REALLY don't understand the nature of survey sampling or if you're just jerking chains. But I'll try again.

I found something rare in one story on a national popular vote poll. It's a reference to confidence. It's the last ABC News Washington Post tracking poll (https://www.washingtonpost.com/news/the ... 71de967b5d).

The poll estimated Clinton to be 4 percentage points ahead. But the author of the article wrote this:

Clinton’s edge in the Post-ABC poll does not reach statistical significance

People see a poll like that ABC News Washington Post tracking poll, with Clinton at an estimated 47 percent and Trump at an estimated 43 percent, and they think that means the poll says Clinton is ahead. But it does not. That's why I grit my teeth every time I see pundits during election season hanging on every poll percentage point and saying stuff like "candidate X's lead grew since this poll was taken last month" when the change was something like 48% to 44% vs. 49% t 43%.

Random sampling error such as that illustrated by the simulation I described a few posts ago involves a pure theoretical situation. It is like if the pollster could 100% accurately identify everyone who is going to vote, be 100% sure of being able to contact each one of those people if they are selected for the sample, and be 100% sure that they will answer the questions when they are contacted. And, yes, 100% sure they are going to answer the questions honestly. Obviously that doesn't happen.

For the body of polling to enable the RealClearPolitics people to have the "favorite" win in 38 of the 39 jurisdictions they felt they could designate...with the only "miss" being a jurisdiction in the least confident "lean" category...is remarkable. And to say "the polls got it wrong" under such circumstances is just ridiculous.




Yet, as interpreted by the RealClearPolitics people.

So, when a poll shows a YUGE lead...we should believe it. And, amazingly, the lead proves to be true. Shocking...very shocking (at least for you). You'd have thought Trump was going to carry California.



And if a poll show a relatively small lead, 4-6%, it could be....wait for it...WRONG.


So, the pools, showing Hillary (not Trump) leading in states she lost...were wrong.


Thanks, again, for confirming what we already knew.

[JohnStOnge]

Some of you are hopeless. But if there are any of you on here who want to really understand correct interpretation of statistical surveys I did an analysis of the "toss up" state polls for illustration.

The first thing you need to understand is that you can't interpret poll results as single "point estimates" or numbers. You have to interpret them in terms of ranges of possible outcomes. Usually this is defined by a 95 percent confidence interval. 95 percent confidence intervals can be developed from the results of groups of polls. The group of polls is correct as long as the actual number is within the range of the 95 percent confidence interval. ALSO...because its a 95 percent confidence interval...you would expect that the truth will be outside of that 95 percent interval about 1 in each 20 times. With those things in mind look at this table of 95 percent intervals vs. actual results for the groups of "toss up" state polls that I calculated:



The "Trump Limit" is the best Trump is estimated to do on one edge of the 95 percent interval and the "Clinton" limit is the best Clinton is estimated to do at the other edge. If the "Result" is between those two extremes, the poll is correct. And in this case the poll was correct in each of 14 of the 15 cases. The result for Pennsylvania was just outside of the 95 percent interval. But one "miss" like that is within the range of what one would expect given that we expect about 1 in every 20 poll estimates to be outside of the 95 percent interval. The 95 percent interval for what you expect if you run 15 polls is for 0 through 2 of the actual results to be outside of the range of what you'd expect. Getting one like that is no big deal. Not an indication of a problem with the polling at all.

So there is absolutely no indication in the results of those 15 "toss up" state polls that "the polls got it wrong." Hopefully at least some of you will get it. But I understand there there are those among you who just don't WANT to get it. Maybe some day I'll do all 54 jurisdictions. But if i do it's going to paint the same picture. The actual result is going to be within the 95 percent interval in all but a handful of cases the number of cases where the actual results are outside of the 95 percent intervals is going to be within the range of what one would reasonably expect given that there's a 0.05 probability that that's going to happen in each case.

Note: The Realclearpolitics people apparently used a more conservative approach than I did. Based on the way I calculated 95 percent intervals I would not have called Arizona, Georgia, or Virginia "toss ups." As you can see, the entire range of the 95 percent intervals for Arizona and Georgia had Trump winning those States (he did) and the entire range of the 95 percent interval for Virginia had Clinton winning (she did).

[JohnStOnge]

Nobody's reading this thread anymore but I'll go ahead and say that one place where the pollsters ARE screwing up is in reporting their "margins of error." In statistical survey that pretty much means 95 percent confidence interval. And there's no way those are really 95% confidence intervals because there are so many sources of error other than random sampling error in what they're doing.

You can kind of get an idea as to what the true margins of error are by looking at 538's poll ratings at https://projects.fivethirtyeight.com/pollster-ratings/. The "simple average error" number is basically an indication of what half the 50% confidence interval for difference between R and D looks like for each poll. For instance the ABC News Washington Post poll has a 3.0 percentage point "simple average error." That means history tells us that poll will be within +/- 3.0 percentage points 50% of the time. Half the 95% interval would be about 2.9 times what half the 50% interval is. That's 8.7 percentage points.

Basically, that means that history tells us that on average we can't be confident the the candidate with the higher number in that poll is really ahead unless they are at least something like 8.7 percentage points ahead in the poll. And the Washington Post poll has one of the smaller "simple average error" numbers.

They really need to do a better job of conveying the uncertainty associated with the results.

[YoUDeeMan]

Nobody's reading this thread anymore but I'll go ahead and say that one place where the pollsters ARE screwing up is in reporting their "margins of error."

Error.

Funny...that means they were...WRONG!

The margin for being incorrect.

The margin for being...WRONG.

The polls showed Clinton ahead...and guess what?

They were...anyone want to guess...WRONG.

Thanks, again, for clarifying that the polls were WRONG.

[Skjellyfetti]

The polls showed Clinton ahead...and guess what?

They were...anyone want to guess...WRONG.

Well, the vast majority of the polls were national polls...

And, they pretty much nailed it.

[BDKJMU]

The polls showed Clinton ahead...and guess what?

They were...anyone want to guess...WRONG.

Well, the vast majority of the polls were national polls...

And, they pretty much nailed it.

WRONG

The majority of polls were state polls. Ex. The last week of RCP, POTUS polls. Including all 2 way and 4 way, I'm counting:
Mon, 11/7: 20 national/24 state
Sun, 11/6: 7 national/10 state
Sat, 11/5: 6 national/6 state
Fri, 11/4: 11 national/23 state
Thur, 11/3: 12 national/18 state
Wed, 11/2: 8 national/22 state
Tues, 11/1: 6 national/13 state

Total RCP week before the election: 70 national/116 state.

[YoUDeeMan]

Keep hammering the deniers with FACTS.

The polls were wrong.

The pundits were wrong.

The New York Times had Clinton as an 80-some% favorite...because they believed the polls that, not-so-coincidentally, were...wait for it...WRONG.

The pools didn't say Trump was in the lead...they said Clinton was in the lead.

And...they were WRONG.













































































JSO: but, we admit the pools can be wrong...so we insert something called a margin of error to hide the fact that we can be WRONG.

































I'm going to start a poll called the JSO poll. It will say every candidate is winning with a margin of ERROR up to 100%. Then, like some useless JD Powers award, it will then, for 6 months AFTER the elections, give a prediction as to who won (at least for those 6 months).

TRUMP WINS...polls LOSE.

[JohnStOnge]

JSO: but, we admit the pools can be wrong...so we insert something called a margin of error to hide the fact that we can be WRONG.:

You know, I guess you just genuinely don't understand. I think that's common. I tried but one of the things that's always perplexed me is that it's just so hard to get people to understand what "error' means in statistical survey. And you clearly don't.

But, you know, you SHOULD be able to understand that for whatever reason the body of polling was such that RealClearPolitics staff tabbed 14 jurisdictions as "toss ups." You SHOULD be able to understand that they said those were too close to call.

And you SHOULD be able to understand that the favorite won in 38 of 39 jurisdictions wherein the RealClearPolitics staff judged that the body of polling indicated a favorite. Further, you SHOULD be able to understand that the "lean" category was that associated with the least certainty and that the ONE case in which the favorite by the polls didn't win was in the "lean" category.

And actually I think you do really understand those things. It's just that for some reason it's important to you to believe the polls were wrong when they weren't.

[YoUDeeMan]

JSO: but, we admit the pools can be wrong...so we insert something called a margin of error to hide the fact that we can be WRONG.:

You know, I guess you just genuinely don't understand. I think that's common. I tried but one of the things that's always perplexed me is that it's just so hard to get people to understand what "error' means in statistical survey. And you clearly don't.

But, you know, you SHOULD be able to understand that for whatever reason the body of polling was such that RealClearPolitics staff tabbed 14 jurisdictions as "toss ups." You SHOULD be able to understand that they said those were too close to call.

And you SHOULD be able to understand that the favorite won in 38 of 39 jurisdictions wherein the RealClearPolitics staff judged that the body of polling indicated a favorite. Further, you SHOULD be able to understand that the "lean" category was that associated with the least certainty and that the ONE case in which the favorite by the polls didn't win was in the "lean" category.

And actually I think you do really understand those things. It's just that for some reason it's important to you to believe the polls were wrong when they weren't.

Holy cow...who knew that RealClearPolitics did all of their own polling?

[BDKJMU]

If a poll had candidate A ahead of candidate B by 3 points, and the margin of error was 4 points, if candidate B won, the poll was still WRONG.

And even if every poll that was wrong was within the margin of error, they are still WRONG.

Not that hard of a concept to understand..

[Ibanez]

JSO: but, we admit the pools can be wrong...so we insert something called a margin of error to hide the fact that we can be WRONG.:

You know, I guess you just genuinely don't understand. I think that's common. I tried but one of the things that's always perplexed me is that it's just so hard to get people to understand what "error' means in statistical survey. And you clearly don't.

But, you know, you SHOULD be able to understand that for whatever reason the body of polling was such that RealClearPolitics staff tabbed 14 jurisdictions as "toss ups." You SHOULD be able to understand that they said those were too close to call.

And you SHOULD be able to understand that the favorite won in 38 of 39 jurisdictions wherein the RealClearPolitics staff judged that the body of polling indicated a favorite. Further, you SHOULD be able to understand that the "lean" category was that associated with the least certainty and that the ONE case in which the favorite by the polls didn't win was in the "lean" category.

And actually I think you do really understand those things. It's just that for some reason it's important to you to believe the polls were wrong when they weren't.

Thanks for telling us what 'error' means. Error pretty much means one thing: mistake. The catcher for the Yankees misses a pop up (mistake). I spill coffee on you (mistake). The difference between estimated and true value (mistake).

[YoUDeeMan]

I spill coffee on you (mistake).

Are you sure it would be a mistake?

[Ibanez]

I spill coffee on you (mistake).

Are you sure it would be a mistake?

For the sake of argument....yes.

[JohnStOnge]

Keep hammering the deniers with FACTS.

The polls were wrong.

The pundits were wrong.

The New York Times had Clinton as an 80-some% favorite...because they believed the polls that, not-so-coincidentally, were...wait for it...WRONG.

The pools didn't say Trump was in the lead...they said Clinton was in the lead.

And...they were WRONG.

The fact is that the RCP people were correct on 38 of 39 jurisidictions in which they declared a favorite. And the fact is that the one case in which the favorite did not win was assigned the lowest level of certainty (lean).

I'm going to start a poll called the JSO poll. It will say every candidate is winning with a margin of ERROR up to 100%. Then, like some useless JD Powers award, it will then, for 6 months AFTER the elections, give a prediction as to who won (at least for those 6 months).

TRUMP WINS...polls LOSE.

I don't know what to do about you guys not understanding survey sampling. A "margin of error" is unavoidable. Doesn't matter if you're talking about how many people favor a candidate or how many trees per acre there are in a certain area.

As I've said I think the pollsters need to rethink the way in which they report "margin of error" for individual polls. The page at https://ropercenter.cornell.edu/support ... vey-error/ provides some detail on that.

But the bottom line is that anyone who really understands survey sampling would not have said that the polling just before the election provided sufficient evidence to say that one candidate or the other was going to win.

[cx500d]

Keep hammering the deniers with FACTS.

The polls were wrong.

The pundits were wrong.

The New York Times had Clinton as an 80-some% favorite...because they believed the polls that, not-so-coincidentally, were...wait for it...WRONG.

The pools didn't say Trump was in the lead...they said Clinton was in the lead.

And...they were WRONG.

The fact is that the RCP people were correct on 38 of 39 jurisidictions in which they declared a favorite. And the fact is that the one case in which the favorite did not win was assigned the lowest level of certainty (lean).

I'm going to start a poll called the JSO poll. It will say every candidate is winning with a margin of ERROR up to 100%. Then, like some useless JD Powers award, it will then, for 6 months AFTER the elections, give a prediction as to who won (at least for those 6 months).

TRUMP WINS...polls LOSE.

I don't know what to do about you guys not understanding survey sampling. A "margin of error" is unavoidable. Doesn't matter if you're talking about how many people favor a candidate or how many trees per acre there are in a certain area.

As I've said I think the pollsters need to rethink the way in which they report "margin of error" for individual polls. The page at https://ropercenter.cornell.edu/support ... vey-error/ provides some detail on that.

But the bottom line is that anyone who really understands survey sampling would not have said that the polling just before the election provided sufficient evidence to say that one candidate or the other was going to win.

La times poll


Sent from my iPhone using Tapatalk

[Skjellyfetti]

La times poll

That was a NATIONAL poll that had Trump winning by 3 points NATIONALLY.

It wasn't predicting the electoral college. It was predicting the national popular vote. Clinton won the popular vote by 2 points. So, LA Times poll was off by 5 points.

[CAA Flagship]

La times poll

That was a NATIONAL poll that had Trump winning by 3 points NATIONALLY.

It wasn't predicting the electoral college. It was predicting the national popular vote. Clinton won the popular vote by 2 points. So, LA Times poll was off by 5 points.

So, rounded, it would be 6.93 points. This makes sense now.