• Welcome to the new NAXJA Forum! If your password does not work, please use "Forgot your password?" link on the log-in page. Please feel free to reach out to [email protected] if we can provide any assistance.

someone must have said somthing....

Okie Terry said:
Ohhhhhh.

Wha?

Spidered???

I spent most of today (and last weekend too) laying on my belly, digging red clay with crude OG tools, and setting concrete blocks below grade. Throughout the project I've been snaked, spidered, hammered, and the bees got after me one time.

I hope Yahoo's got tougher spiders than I do.

Woody
 
From the URL in the post I made:
Why are you crawling my site? Yahoo! Slurp is Yahoo!'s web-indexing robot. The Yahoo! Slurp crawler collects documents from the Web to build a searchable index for search services using the Yahoo! search engine. These documents are discovered and crawled because other web pages contain links directing to these documents. As part of the crawling effort, the Yahoo! Slurp crawler will take robots.txt standards into account to ensure we do not crawl and index those pages that you would not like to have returned via Yahoo! Search Technology. If a page is disallowed to be crawled by robots.txt standards, it is neither considered for inclusion nor placed in the search engine's database.


woody said:
Spidered???

I spent most of today (and last weekend too) laying on my belly, digging red clay with crude OG tools, and setting concrete blocks below grade. Throughout the project I've been snaked, spidered, hammered, and the bees got after me one time.

I hope Yahoo's got tougher spiders than I do.

Woody
 
Scott Mac. said:
So........that's a good thing?
Always. We are constantly being "spidered", "crawled" etc.... It is a very good thing.
Glenn
 
And I thought he was talking about screaming Yahoo! rock crawling at Poison Spider...goes back to putting white-out on the computer screen to fix the typo's...
 
Back
Top