 |
Recent Spiders & Robots Articles  |
|
|
: Home
:
Spiders & Robots
|
|
|
 |
Search Engine Robots - How They Work, What They Do (Part I)
by Daria Goetsch 4/23/04
Automated search engine robots, sometimes called "spiders" or "crawlers", are the seekers of web pages. How do they work? What is it they really do? Why are they important?.... more...
|
|
Every Search Engine Robot Needs Validation
by Daria Goetsch 4/23/04
Your website is ready. Your content is in place, you have optimized your pages. What is the last thing you should do before uploading your hard work? Validate. It is surprising how many people do not validate the source code of their web pages before putting them online.... more...
|
|
Working with robots.txt file
by Pannu Jagdeep.S. 4/23/04
Learn all about working with robots.txt file. A useful guide that talks about what robots.txt file is, its advantages & disadvantages, how to optimize & use robots.txt file to define the content you want excluded from indexing, thus saving the crawler's indexing time… more...
|
|
How Deep into Your Site are the Spiders Crawling?
by Robin Nobles 4/17/04
Have you ever wondered how far down into your site the search engine spiders are going? Are the pay inclusion spiders coming at regular intervals like they’re supposed to? more...
|
|
Software to help you create a robots.txt file
by Sumantra Roy 2/27/04
Describes a software which will automate the task of creating a robots.txt file. more...
|
|
Creating a robots.txt file
by Sumantra Roy 2/27/04
How you can create a robots.txt file in order to prevent your site from being penalized for spamming by the search engines more...
|
|
Dissecting Teoma, Part 3 of 3
by Dan Thies 2/14/04
The series concludes with some important tips for achieving top rankings on Teoma/AskJeeves search. more...
|
|
Dissecting Teoma, Part 2 of 3
by Dan Thies 2/14/04
In part 2 of this series, we examine what makes Teoma's technology different from other search engines. more...
|
|
Dissecting Teoma - Part 1 of 3
by Dan Thies 2/14/04
Is the new Teoma 2.0 search the best on the web? Maybe not, but it's getting better, and growing in popularity. Part 1 of this series explains how Teoma works, and how to get your site listed. more...
|
|
Search Engine Spiders Like to Read
by Andy Beal 2/10/04
While a good domain name and targeted META tags can help improve search engine rankings, they alone will not help a Web site if it does not contain relevant text on the page being optimized. more...
|
|
|
|
