who or what is this?

  • Thread starter Thread starter coolbal
  • Start date Start date
  • Replies Replies 9
  • Views Views 2,673
coolbal

coolbal

Regulars
Messages
541
Location
Chennai
ISP
Airtel Xstream Fiber
i just looked at the list of currently active members and i find these unusal things or peopleYahoo! Slurp Spider ,Google spider etcso what is this are they trackes from google or yahoo?did they come here to promote our site in their search lists?
 
search engine spiders. for indexing the forum...
 
so many spiders? wudnt they chew up bandwidth?
 
spiders are important mate. they dont chew much bandwidth. have enough bandwidth.
 
pretty true. i have not marketed this forum personally. all the traffic it has is due to search engines.



a good example of how good google is with indexing of sites. incidentally techwhack is banned in msn live search engine.
 

no no i know spiders are good for promotion. but yesterday i saw way too many yahoo spiders.
 
there is nothing like too many search engine spiders... 🙂 as long as you dont run out of SQL connections... and we are no longer. for now atleast.
 
i think using robots.txt file u can restrict these spiders to some level. u may not need this now but may be in future. now that google indexing is much better so u can expect more and more hits as time passes
 
most spiders are smart enough to slow down indexing if they find out that the server is not able to cope up with the traffic. google provides an option to slow down the speed in google webmaster panel.yahoo follows robots.txt. currently i have nothing to worry about the search engine spider load.
 

Top