|The Axandra newsletter archive - 5 October 2004|
|Welcome to the latest issue of the Search Engine Facts newsletter.
This week, we're trying to find out if Google has sent out spiders to find spammers.
In the news: Yahoo releases new search tools, MSN Search launches a new beta version, there's a new article about Google's web browser plans and more.
Table of contents:
We hope that you enjoy this newsletter and that it helps you to get more out of your web site. Please pass this newsletter on to your friends.
|1. Facts of the week: Google's new web page spider|
|Search engines use automated software programs that crawl the web. These programs called "crawlers" or "spiders" go from link to link and store the text and the keywords from the pages in a database. "Googlebot" is the name of Google's spider software.
Many webmasters have noticed that there are now two different Google spiders that index their web pages. At least one of them is performing a complete site scan:
The normal Google spider: 188.8.131.52 - "GET /robots.txt HTTP/1.0" 404 1227 "-" "Googlebot/2.1 (+http://www.google.com/bot.html)"
The additional Google spider: 184.108.40.206 - "GET / HTTP/1.1" 200 38358 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
What is the difference between these two Google spiders?
The new Google spider uses a slightly different user agent: "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)".
This means that Googlebot now also accepts the HTTP 1.1 protocol. The new spider might be able to understand more content formats, including compressed HTML.
Why does Google do this?
The second theory is that Google's extensive crawling might be a panic reaction because the index needs to be rebuilt from the ground up in a short time period. The reason for this might be that the old index contains too many spam pages.
What does this mean to your web site?
It's likely that the new spider announces a major Google update. We'll have to see what this means in detail.
|2. Search engine news of the week|
"My Yahoo! Search addresses a whole host of everyday search problems. If you've ever experienced the tedium of searching again for something you found before but can't quite remember, if you've ever been annoyed by search results you don't like but show up again and again, if you ever wanted to share something you found in search but were forced to cut-and-paste... give My Yahoo! Search a shot."
"What do we have under the hood? Our new algorithmic search engine with a few more features and a bigger index than last time you visited this site. We used your comments to make it better so give it a spin."
"Microsoft's Internet division has invited dozens of Webloggers, researchers and others to its Redmond, Wash., campus next week for an event called 'Search Champs.' They are scheduled to hear about and test upcoming features for MSN Search and to meet researchers from Microsoft Research, according to several people invited to participate."
"The Alexa Web Information Service (AWIS) provides developers with programmatic access to the information Alexa Internet collects from its Web Crawl, which currently encompasses more than 100 terabytes of data from over 4 billion Web pages. "
Editor's note: Alexa.com and Amazon's other search engine A9.com currently use Google's index. This beta service might be a first indicator that Amazon will be switching to its own search technology.
|3. Articles of the week|
"One Microsoft technologist said web search technology will be a big money maker for Google and others in the short term but he predicted the party won't last long."
"The move would be key in helping the outfit expand beyond search, tie its many offerings together, and hold off Microsoft."
"All five gave Google high ratings, even though it was a far more expensive stock than at the IPO just six weeks earlier, having risen almost 40 percent. Why any investor took these opinions seriously is a mystery to mebut the stock soared around 8 percent. It's as if people have forgotten the whole unseemly spectacle of analysts' hyping stocks their firms helped underwrite. Does anything ever change? Maybe not."
"Based on Google's share price of around $130, Salary.com senior vice president Bill Coleman estimated that around 60 percent of the 1,900 Google employees with the company at the end of March each now hold stock options worth at least $1 million."
"A new aptitude test Google is circulating purports to find the best and brightest. But is it really a brilliant ad campaign?"
|4. Recommended resources|
Another 5 star rating for IBP
"IBP is the multi award winning web site promotion tool that helps you to get more revenue with high search engine rankings in Google, Yahoo and all other major search engines.
The suite of 10 professional website promotion tools includes a search engine submission tool, a search engine ranking checker, a top 10 optimizer and a keyword generator. IBP creates professional reports in PDF, HTML and Microsoft Word format."
How does your web site compare to your competitors? Does your web site have more incoming links? How much traffic do your competitors get?
The new version 3.0 of the freeware tool Link Popularity Check helps you to get the answers quickly and easily. Click here to download your free copy.
|5. Previous articles|