Traffic Is Fake, Audience Numbers Are Garbage, And Nobody Knows How Many People See Anything

from the stabs-in-the-dark dept

How many living, breathing human beings really read Techdirt? The truth — the most basic, rarely-spoken truth — is that we have no earthly idea. With very few exceptions, no media property big or small, new or old, online or off, can truly tell you how big its audience is. They may have never thought about it that way — after all, we all get as close as we can to what we think is a reasonably accurate estimation, though we have no way of confirming that — but all these numbers are actually good for (maybe) is relative comparisons. What does it really mean when someone says "a million people" saw something? Or ten or a hundred million? I don't know, and neither do you. (Netflix might, but we'll get to that later.)

Where should we start? How about this: internet traffic is half-fake and everyone's known it for years, but there's no incentive to actually acknowledge it. The situation is technically improving: 2015 was hailed (quietly, among people who aren't in charge of selling advertising) as a banner year because humans took back the majority with a stunning 51.5% share of online traffic, so hurray for that I guess. All the analytics suites, the ad networks and the tracking pixels can try as they might to filter the rest out, and there's plenty of advice on the endless Sisyphean task of helping them do so, but considering at least half of all that bot traffic comes from bots that fall into the "malicious" or at least "unauthorized" category, and thus have every incentive to subvert the mostly-voluntary systems that are our first line of defence against bots... Well, good luck. We already know that Alexa rankings are garbage, but what does this say about even the internal numbers that sites use to sell ad space? Could they even be off by a factor of 10? I don't know, and neither do you. Hell, we don't even know how accurate the 51.5% figure is — it could be way off... in either direction.

Okay, so what about TV ratings? Well, there's a reason they've been made fun of on the shows themselves for as long as our culture has been able to handle "meta" jokes without getting a headache. Nielsen ratings in their classic form are built on monitoring such a tiny sample of households that the whole country's viewing profile can probably be swayed because someone forgot to turn off the TV before going on vacation. They sucked before DVRs and digital distribution began transforming the single household television into a quaint anachronism, and now it's just chaos. Nielsen was slow to catch up with DVRs, and now the TV industry juggles scattered measurements including three or seven days of viewing beyond live air, and constantly complains that the ratings are off — specifically, that they're too low. And they might be right, in the sense that they are too low by comparison to the garbage ratings from the pre-digital age that everyone eventually embraced as a standard for relative rankings. How big are these audiences really, in terms of real living breathing human beings? I don't know, and neither do you.

YouTube view counts? Subject to all the same fake internet traffic problems, plus the fact that there's an opaque system for supposedly ignoring too-short incomplete views according to the genre and nature of the video, but good luck finding out how accurate that is. Channel operators know their length-of-view statistics, but you don't see them bandying them about much. Plus, how often have you heard public view counts casually referred to as the number of "people" who watched something, even though (especially when it comes to short-and-cute viral animal hits and their ilk) the bulk of them probably come from obsessive re-watching? Yeah.

So what about Facebook stats? Everything from impressions to simultaneous live video viewers is padded out by the most transient of idly-scrolling-through-the-newsfeed interactions. Twitter followings and tweet stats? Dig into the bowels of any list of followers, or any trending link, and see how much of it is mindless bots. Print readerships? Don't even get me started. Did you know it's common practice for newspapers to calculate their readership by applying a multiplier to their actual circulation, to account for an imaginary surplus of "readers per copy"? Yes, that soggy "local" paper that's been sitting out in the rain on your porch for two days, and that only exists to give them an excuse to deliver flyers to your door, is not only being counted — it's probably being counted five times. So are all the free/cheap copies that big national papers give to hotels. Oh, and when these companies distribute multiple publications in different channels — with newspapers, magazines and paywalled websites all being given away with each other as free cross-subscriptions, in order to pad out all three subscriber numbers — they add them all up and then try to determine the actual number of individual people they are reaching. How? By applying an opaque "deduplication" formula. I once pressed a newspaper's stats person about what this formula could possibly entail, but details were not forthcoming — because I suspect they just knock off 20% and call it a day, despite the fact that the magazine is distributed inside the newspaper whose audience they are supposedly "deduplicating" it from, and half the website subscriptions were free add-ons with print delivery. That's awfully generous when the truth is they don't know, and neither do I, and neither do you.

So who does know how big of an audience they really have? Well, maybe Netflix, Amazon and other digital subscription services. Their paywalls insulate them from the bulk of random bot traffic, and their proprietary ecosystems give them the ability to closely monitor all activity. Netflix, of course, is famously secretive about viewer numbers and insists on the inaccuracy of those who claim to have worked them out. The most common assumption is that they do this to avoid giving content creators too much leverage, and because the data can be seen as a valuable commodity — but I propose another reason: Netflix's likely-more-accurate statistics, if made public, would have zero context in the topsy-turvy world of nonsense TV ratings. They would probably look exceptionally low, giving the legacy bosses who would like nothing more than to downplay the importance of digital distribution (and there are as many of those as there are record execs who can't spell mp3) a chance to project whatever narrative they wanted onto the numbers.

So why does any of this matter? Because advertising is a multibillion dollar industry, and whenever an industry is worth that much, you have to ask: is that because there are billions of dollars of worthwhile transactions happening, or because every bloodsucker in a ten-industry radius wanted in on the action? So, so much of the advertising industry is pure waste. How much exactly is as impossible to determine as the audience sizes themselves. This is hardly a new idea (in fact it's a century-old quote) but it's probably more true now than ever, despite the fact that in theory technology could have delivered us from uncertainty.

Finally, what can be done about this? There's no simple answer, and maybe no answer at all. Here at Techdirt, we've been working to come up with good advertising solutions by focusing almost entirely on what we know our community likes and might be interested in (as in, our real community of people who talk in our comments and we can say, with confidence, exist) and paying less attention to raw numbers — both a luxury and a necessity for a smaller publication, depending on how you look at it. That's not always easy though, as we face an advertising industry ruled by metrics, where there are often ten spreadsheet-wielding interns between us and someone who might actually care about our creativity. In our experiments with more traditional algorithmic display advertising to monetize the raw traffic numbers we do have, we keep running up against what appears to be a universal truth: the bulk of the global internet ad ecosystem runs on trash. Gigantic prestigious online media brands can sell display campaigns straight to the same people who buy Superbowl ads — everyone else receives a hundred pitches a week from new ad networks that claim to deliver great, relevant content but in fact litter your site with ads for fad diets and ambulance-chasers (at best). And this lowest-common-denominator filler appears to be the only reliably successful form of internet advertising! At least, it never goes away when the good stuff does, and the proud quality networks eventually embrace their roles as crap-peddlers. "Good" internet advertising is a rickety ship navigating an endless roiling ocean of spam, clickbait and outright fraud — but it couldn't float at all without it.

I realize I've painted a grim picture, but these are (more or less) the facts. I'm surely wrong in some of my guesses, but like everything discussed here, nobody knows how wrong or in which direction. We'll never even really know how many people read this — we'll just have a vague estimate that can be compared to other posts on Techdirt. But for now that's the reality, so maybe more people should stop worrying about the supposed size of their audience, and focus on making the content they want to make.

Filed Under: audience, internet, measurements, statistics, traffic

When You Crack Open The Surveillance Door, The Food Police Will Want Your Metadata

from the food-for-thought dept

As you may recall, late last year Australia put into effect a wonderfully ambitious data retention law that required ISPs in the country to do... well... something involving data retention. The problems began immediately, with ISPs unsure of exactly when they were supposed to start collecting all of this data, as the law allowed for some to petition to delay implementing the data collection, but the government hadn't bothered to get back to many of them. Never mind what would happen once this same inept government actually received the mountains of data it had requested.

But those concerns are all about the practical utility of such a law, not the larger concerns over whether this kind of data collection ought to be happening to begin with. To see an example of why a free people shouldn't allow the government to crack open this door, however, one needs only look again at the law in Australia. What was supposed to be collection chiefly to combat major criminal actions is now a collection that even the food police are trying to get in on. And, yes, I really do mean the food police.

If you are in the business of selling lamb chops, make sure you are weighing them properly: the National Measurement Institute wants warrantless access to Australians’ metadata to help them hunt down supermarkets skimping on portions. The NMI is one of 61 agencies that has applied to the attorney general, George Brandis, to be classed as a “criminal law-enforcement agency” in order to gain warrantless access to telecommunications data.

As part of the government’s assurances that there would be sufficient privacy safeguards, it reduced the number of agencies that could access the data. But agencies could reapply, with the permission of the attorney general, if they were involved in enforcing “serious contraventions” of criminal laws.

Now, this is still in the application phase, so the NMI, a government group tasked with investigating retailers to make sure they're packaging their goods properly, doesn't yet have access to Australian's metadata, but its petition did have to be approved by the attorney general as being relevant to even get that far. This is what happens when you crack open the door: the bulk of government will try to force its way in. Keep in mind that the efficacy of bulk collection practices for combating even terrorism and serious criminal actions is debatable, but now we're dealing with the food and retail packaging police wanting in on the action? This really should tell you everything you need to know.

Greens senator Scott Ludlam said: “The only saving grace the government was able to claim when they passed it was that they were narrowing the range of agencies that could access the data. On the face of it that was true, and obviously that’s just been blown to pieces.”

Just because slippery slope arguments are almost always lame, that doesn't mean they don't occasionally apply. It seems on bulk surveillance, they certainly do.

Filed Under: australia, data retention, food, lamb chops, measurements, metadata, surveillance

Should We Be Measuring Happiness As An Economic Measure?

from the this-makes-me-sad dept

A lot of people have finally realized that traditional economic measures have all sorts of problems. Things like GDP mismeasure a ton of things, and by presenting an aggregate set of data, often obscure lots of issues. Also, things like GDP don't handle disruption very well. I've discussed in the past how you could argue that, purely on a GDP basis, something like Craigslist has been horrible. It effectively undercut newspaper classifieds, which was a multi-billion dollar business, and turned it into a much smaller business. If you measured such things purely by GDP, you'd say that it was bad. But, of course, Craigslist also created tons of value, enabling people to make transactions that couldn't have been made before, while also allowing other transactions to be made more efficiently and with less friction. Much of that will never show up in GDP, even if, intrinsically, most people recognize that something like Craigslist provided a lot more value to the world than it took away.

In trying to deal with that, we've started to see new forms of economic measurements pop up. One popular one is "happiness." There's even been some talk about using "Gross National Happiness" as a key economic measure. There's a great book from a couple of years ago by Nobel-prize winning economist Joseph Stiglitz, with Amartya Sen and Jean-Paul Fitoussi, called Mismeasuring Our Lives: Why GDP Doesn't Add Up. It was actually the result of a request from then French President Nicolas Sarkozy to explore how useful (or not) GDP was, including looking into alternate measurements, such as this idea of Gross National Happiness. If you haven't read the book, I highly recommend it.

Recently, the folks at Planet Money also did a report on the growing interest in measuring happiness, particularly as an official stat for American economic health. There appears to be growing interest in establishing a happiness index for the US, not unlike the unemployment index. Of course, you can think of the immediate problem. Just how do you measure happiness:

But once you get into the details, there's a lot of debate over the happiness data. One big divide: Should you ask people how they're feeling right now, or how they feel about their life in general?

You get different answers depending on what you ask. Which one is more important is a squishy, philosophical question.

The difference between asking about "right now" or "their life in general" can be massive. It shows up clearly in the data about how happy parents are vs. non-parents. There are tons of studies that suggest parents are miserable compared to non-parents. But nearly all of those studies are based on questions about "how happy are you now" type questions. Not surprisingly, the parent changing a diaper is probably going to report slightly less current happiness compared to the non-parent who's out at the bar with some friends, for example. But... it's not that simple. When other studies are done that ask parents and non-parents about how happy their overall lives are or how fulfilled their lives are, parents frequently report much higher feelings of fulfillment/happiness on a grand scale, while non-parents often report more regret. In other words: time frame makes a huge difference.

Of course, as the Planet Money report points out, just because something is difficult to measure, or involves highly subjective concepts, doesn't mean it can't be done. For example, unemployment data. You might think that this involves a nice, simple objective question, but when you look at the details, it's actually pretty subjective as well.

In the U.S, in order to be counted as unemployed, you have to be out of a job and looking for work. But what counts as looking for work? Checking Craigslist? Sending out three resumes a week? Five?

"It's actually kind of a hard question," says Justin Wolfers, an economist at the University of Michigan. "It's very subjective."

Yet every month, a single unemployment number is released.

So, you could see why a "Happiness Index" might be a compelling bit of economic data -- especially if you believe (as I do) that GDP is misleading. After all, if people are happier, isn't that a pretty important thing? Well, yes and no. Even as I find the topic interesting, I also worry a lot about the embrace of "Happiness" as an economic measure beyond the reasons laid out in the Planet Money report. Yes, it's difficult to calculate, but perhaps you can get past that so long as the calculation is done the same way over time. The real problem, for me, is that when you choose to make something a key economic number like that, you are guaranteed to start optimizing for it. That's what happens when you create metrics. Whether they're important or not, whether they're accurate or not, once you have a number, you naturally try to optimize for it.

It shouldn't be difficult, then, to quickly come up with scenarios for why a National Happiness Index could create significant problems as people optimize for it. First off, you encourage the kinds of short-term rewards that lead people to say they're happier, even if that creates massive costs down the road. Want to see governments leverage the present and put the costs on the future? Start using a happiness index. Second, if the focus is on maximizing present-day happiness, then you just focus on drugging the population. Yes, that's an extreme example, but hopefully it gets the point across. In economics, you need to measure the costs and benefits to things. You can "maximize happiness" in all sorts of ways if you ignore the costs to it. Put happy drugs in the water, and let everyone be thrilled. The Happiness index fails to take into account all of the consequences of doing something like that.

So while it's encouraging to see more of an exploration into alternative metrics, and getting beyond some of the older metrics that clearly "mismeasure" important aspects of our lives, we need to be careful to not just leap to the "next great thing" without realizing that it, too, likely has downsides.

Filed Under: economics, gdp, gross national happiness, happiness, happiness index, measurements, metrics

DailyDirt: If I Could Catch Time In A Bottle...

from the urls-we-dig-up dept

The scientific method relies on independent verification of measurements and results, but sometimes it's not easy to replicate experiments or measure things at the leading edge of science. Scam artists are often identified when they explain that their results are so far advanced that no one else can replicate them. Real scientists, though, don't buy that. Here are a few scientific discoveries that still need a bit more verification.

• Neutrinos traveling faster than the speed of light don't seem to exist, and the CERN team that first said they saw some FTL neutrinos, now admit they made a couple mistakes. the CERN team, after reporting unusual data, determined definitively that it was caused by a mistake (and what that mistake was) with input from the broader scientific community. As some expected, there were systematic errors that produced a timing error of just 60 nanoseconds. [url]
• CERN scientists have trapped anti-hydrogen for a little over 15 minutes -- a new record that could allow them to determine more about how antimatter behaves. Does anti-hydrogen rise, fall or do nothing in a gravitational field? [url]
• The Large Hadron Collider is still hunting down the elusive Higgs boson which might not exist at all. This sub-atomic particle has been nicknamed the "God Particle" -- but it looks like it's just a matter of time before people either find it or start re-writing modern physics. [url]
• To discover more interesting science-related stuff, check out what's currently floating around the StumbleUpon universe. [url]  

By the way, StumbleUpon can recommend some good Techdirt articles, too.

Filed Under: anti-hydrogen, antimatter, ftl, gravity, higgs boson, lhc, measurements, neutrinos, particles, physics, science
Companies: cern

DailyDirt: Everyday Scientific Mysteries

from the urls-we-dig-up dept

There are all kinds of scientific complexities around us. Just try to seriously answer all the questions that a curious five-year-old can come up with, and you'll get a feeling for how little we actually know about the world around us. Thankfully, we have some modern scientific equipment to help sort out these minor mysteries. Here are just a few answers for some common phenomena.

• The physics of dried out coffee rings has been clarified. Yay! Now there's at least some reasonable explanation for why coffee drops dry out into dark-edged rings. [url]
• Maybe Southwest Airlines' method of boarding isn't optimal, but compared to some Monte Carlo simulations, it's not too bad. Did it really take an astrophysicist to prove that boarding by blocks of rows isn't the most efficient way to pile into a plane? [url]
• If you've ever wondered what the difference between the green-yellow and yellow-green crayons was, here is your answer. This is what you get when you stick 24 Crayola crayons in a spectrophotometer. [url]
• To discover more interesting science-related stuff, check out what's currently floating around the StumbleUpon universe. [url]  

By the way, StumbleUpon can recommend some good Techdirt articles, too.

Filed Under: boarding, coffee, crayons, measurements, monte carlo
Companies: southwest