Click Fraud Detection and Prevention

Google States Undetected Click Fraud Rate Less Than 0.02%

originally posted 3/4/2007 By: - TrafficSentry.com

Last week, Google finally announced it’s own internal estimates of the scope of the click fraud problem to it’s AdWords advertisers. Much focus has been directed on the 0.02% number Google defines as “reactively detected as invalid”. This number is overshadowing the larger “less than 10%” overall number that Google claims includes the clicks they proactively detect and filter out before billing the advertiser.

Let’s look at the 0.02% figure first.

Closer examination of exactly what Google said brings us to the conclusion that the statement is cleverly worded in such a way that it means absolutely nothing without other variables that they are not disclosing.

I think that many people (including some in the press) interpreted the statement as “0.02% of total clicks are both fraudulent and not being detected and discounted..”. What was actually stated is that the number of billed clicks reported to Google by advertisers and then found (by Google) to be invalid, is 0.02%.

This number represents the clicks that made it through all phases of filtering by Google, were billed to the advertiser, reported by the advertiser, and then finally found to be invalid by Google examiners upon investigation after the advertiser has complained. That is not hard to believe at all, because if Google did not detect a click as invalid the first time, the chances are that they will not consider it invalid upon a complaint either.

So to get to a number as low as 0.02% should be relatively easy. First, we all know that the vast majority of advertisers are not taking the time to file complaints, for a number of reasons:

• The advertiser is unaware of the problem.
• The advertiser assumes Google’s fraud prevention has filtered all bad clicks.
• The advertiser is unaware of the claim procedure.
• The advertiser assumes that their request will be denied, as is often true.
• The advertiser lacks the evidence to prove their claim.
• The volume/reward is too low to be worth the time.

Next, out of the small number of clicks that ARE submitted by advertisers for further review, the vast majority of clicks claimed are rejected by Google. This is a number that they do not seem to be disclosing and without it, you cannot complete a calculation of their 0.02% claim.

At this point, we have already eliminated most of the clicks from the total; those that are not reported, those that have already been filtered out, and those which are reported but rejected (whether correctly or not).

Finally, and perhaps the largest unknown variable, are all of the clicks that are both unreported AND undetected by Google.

This is the obvious flaw in Google’s statement that everyone seems be overlooking. They are not stating anything with regards to the total overall click fraud rate, including both detected and undetected. Of course they can’t report fraudulent clicks they don’t know about, and I don’t think anyone thinks their filters are perfect. All of the stated numbers represent invalid clicks that they eventually detected.  Any invalid clicks that slip through completely (and oh yes there are many) would come out of the 90% that they claim are “valid”, and that however large the number of such clicks, it would not affect the 0.02% figure, based on the way that it was worded.

Perhaps this number of unaccounted for clicks makes up the mysterious and significant gap between Google’s estimates of the scope of the invalid click problem, and the 17-30% estimates consistently reported by third-party click tracking services.

After two years of debate (harsh at times) over the accuracy of third-party click fraudreports and click tracking methodologies, Google seems to have made an abrupt reversal in its position on the subject, as some feel is evidenced by the announcement last week that Google would begin “cooperating” with click fraud firm Click Forensics, its primary rival in the click fraud arena.

But are some in the industry reading too much into this announcement? To what extent will this change in policy help online advertisers in their ongoing struggle with losses from invalid clicks?

It would be difficult to imagine Google and Click Forensics actually working together after two years of publicly slamming each other. With the vast majority of its revenue connected to per-click advertising, Google has a lot to lose if 16-30% of clicks were fraudulent (as reported by Click Forensics) and those clicks were successfully eliminated. Google’s own publicly stated estimate of the click-fraud rate is a much lower 0.02%.

It’s hard to believe that these companies ever could be “best-buddies” with their inherent conflicts in objectives. In fact, reports that the two companies are “working together” appear to overstate the relationship, which (as announced) is simply an agreement to accept automated electronic click complaint reports. That in itself is a significant development for advertisers, as until now, it has seemed as though Google has made the process of reporting invalid clicks unnecessarily difficult and time-consuming for customers, and often with a less-than-favorable outcome. Whether intentional or not, this has resulted in a large number of fraudulent and unwanted clicks to go unreported, as it is simply not cost-effective for most businesses to pursue a refund.

Allowing automated reports and standardizing the reporting format will significantly streamline this process for advertisers, which will undoubtedly result in a sharp increase in the number of reports and volume of questionable clicks being reported. This will not only affect Google’s top line directly (as a result of more click refunds) but may significantly increase the staff requirements in the AdWords Click Quality division.

You might think “Wouldn’t automation decrease the need for human case reviewers?”. Well first of all, the increased volume of click quality complaints will come largely an addition to current reports received through existing channels, as the automation feature only applies to advertisers using Click Forensics’s FACTr service. Secondly, you would be mistaken if you assumed that Google would just be accepting all these reports as “accurate and correct” and automatically spewing out refunds. No way! They have only agreed to “receive” the reports in the electronic format, and have made no statements to retract previous statements that Click Forensics’s tracking methodologies are “inept” and “fundamentally flawed”.

This brings up the second problem. Although, yes, Google will now accept the reports electronically, they are not saying that they will agreewith the reports. The reality is that rejection rate of claims will likely increase, and while they will be refunding a greater number of billed clicks than in the past, they will be refunding a lower number, as a percentage of clicks that are reported.

In my experience with reporting invalid clicks to Google, the process has been grossly overcomplicated and time-consuming, with several layers of case reps having widely varying degrees of training/knowlege. The point here is that this entire process occurs after the report is filed. There is no mention of any changes to the review procedures nor approval policies.

So while we do have the ability to zip a report over to them with a few clicks, which is a much appreciated improvement, what we don’t have here is Google finally admitting that Click Forensics’ (and other click tracking services’) numbers are accurate and that they will cooperate on refunds based on that data. That will never happen. They would in essence be admitting that over 20% of their 2007 revenue was based on false clicks.

One of the biggest problems now with all search advertising networks is that they will not provide advertisers with all the data that they have collected on the clicks they are billing them for (the data they use internally when reviewing complaints but for some reason will not make available in client reports). This include line item data (date, time, IP, etc..) for each click. They have this information, why not make it available?

Yes, it may now be easier to report our data to Google, but it will not be possible to reconcile our data what we are being billed for until they make that information available to us.

For more information, articles, and whitepapers on click fraud prevention please visit http://www.trafficsentry.com