arrow-downarrow-leftarrow-rightarrow-upbiocircleclosedownloadfacebookgplus instagram linkedinmailmenuphoneplaysearchsharespinnertwitteryoutube

IAB and ABCE Announce Global Spiders & Bots Filtering List

Combines IABs Ad List with ABCEs Content Traffic List for First Time

New York, NY (February 7, 2006)The Interactive Advertising Bureau (IAB) and ABC ELECTRONIC (ABCE), today announced the launch of an expanded global Spiders & Bots List. This new list will include the two different types of non-human activity by combining the IAB comprehensive list of advertising related spiders and bots with ABC ELECTRONIC’s extensive list of site-traffic associated with robotic activity.

During audit processes, one of the key ingredients of accurate advertising impression counts is determining the impact that non-human activity may have on the ad-serving process. Previously, it is suspected that this contributes to up to 40 percent of ad-impression counts. The new global list will help to provide the most effective method of filtering out all non-human activity affecting both ad impression counts and site traffic counts – enabling site owners to present transparent, accurate and robust auditable data relating to counts of variable user interaction.

This filtering is essential for obtaining accurate numbers for delivered ad-impressions and, now, for general content traffic, says Greg Stuart, IABs President & CEO. By working jointly with ABCE, we will be able to expand the watch list to include Spiders & Bots more widely used in Europe and together we will be able to offer broader protection to thousands of web sites.

The IAB/ABCE International Spiders & Bots List will be distributed by the IAB to North American headquartered subscribers, regardless of IAB membership, and by the ABCE to EU-headquartered companies.

ABC ELECTRONIC is delighted to be working together with the IAB to create a single list that is recognized globally. The joint watch list will support those companies that have not deployed other tools to help them differentiate real traffic from that generated by machines, says Richard Foan, Managing Director of ABC ELECTRONIC. According to The New York Times, just 17 percent of the 250,000 or so companies with at least $1 million in annual revenues and a web site now use analytical tools and services for those sites. The remaining 83 percent have been deterred by the cost of such tools, among other things.”

This list is a crucial tool for sites to use when filtering their logs. Websites need to understand how a user interacts with the site and marketers; advertisers and media agencies need to obtain accurate data in-order to establish how many real users will actually see their ad, says Jeremy Fain, Director of Industry Products, IAB. Now that we have merged with the ABCE list, North American companies will also have access to a more comprehensive, up-to-date list of email and mobile browsers and user agents.”

In January 2005, the IAB contracted with ImServices to keep an industry-wide list of ad-related spiders and robots for use in internal and external auditing of ad impression reporting. An IAB Policy Board consisting of members with experience in ad operations and online ad-related auditing had updated this list once a quarter with the help of ImServices. The new merged list will now be updated monthly through consultation with JICWEBS in the UK and the IAB Policy Board in the US.

Current US-based subscribers of the watch list include: 24/7 Real Media, AOL, Atlas DMT, CNET Networks, Walt Disney Internet Group, DoubleClick, Falk eSolutions, MSN,, and Yahoo.

For more info or to subscribe to the list click here.


ABC ELECTRONIC ( was established in 1996 as the industry owned, tri-partite, not for profit organization, which works on behalf of advertisers, media buyers and media owners to provide third party independent verification and certification for data related to electronic media (Internet, Email, SMS, Interactive TV, WAP, PDA and Streaming audio and video) in the UK and Ireland. ABC ELECTRONIC delivers global measurement standards for local markets through its work as Chair of the IFABC web standards group.

About the IAB:

Founded in 1996, the Interactive Advertising Bureau ( represents over 225 leading interactive companies that actively engage in, and support the sale of interactive advertising. IAB members are responsible for selling over 86% of online advertising in the United States. On behalf of its members, the IAB evaluates and recommends standards and practices; fields interactive effectiveness research, and educates the advertising industry about interactive advertising.

About IAB Europe:

IAB Europe ( is a federation of 15 national IAB associations and think-tanks set up to help marketers get the most from the new digital channels. Supported by every major media group, agency, portal, technologist and service provider, the IABs across Europe embraces more than two thousand companies, helping unlock the potential of online marketing.

IAB Europe coordinates activities in the region and handles all pan-European issues of research, marketing, lobbying, standards and best practice. Its mission is to grow the interactive advertising markets among pan-European clients, and to nurture the development of national markets and their IABs.

About ImServices:

ImServices Group Ltd. (ImS) is a leading provider of independent consulting and verification services, including process reviews, activity and privacy audits, for Web sites, third-party ad servers, email delivery systems and other interactive media companies. ImS also works with clients to review a wide range of other types of electronic transactions including e-commerce, wireless activity on mobile devices, etc. For more information, visit

IAB Media Contact:
Lynn Trono

Get the latest from IAB in your inbox

Be the first to know. Sign up to receive news about the IAB programs, standards, events, classes, and more!