Recently we've had a problem with high trafic load on our servers for FeminaPortal - portal and forums for women, girls and ladies. It is well optimized and so far there were not any issues with it.
We determined that the problem was caused by Google search engine crawlers that performed many requests and that had impact our online classified site. It looks like we were being rapidly crawled by GoogleBot.
Microsoft Bing search engine configures crawl reates and frequency determined by your server's load.
In some cases the website could be overhelmed by too many requests in short period of time and you need to reduce the crawl rate. Great number of requests too quickly could make your site slow.
If you do not have issues with server load or your site's bandwidth we recommend that Bing choose the optimal crawl rate.
Google has advanced algorithms for choosing when to crawl a website. The Google search engine tries to crawl pages from a site without impact on the site's performance.
In some cases Google can affect your site crawling too meny pages in a very short period of time. In that cases you would like to tell Google to reduce crawl process because too many requests could overhelm your site, especially if you have busy site or it is hosted on shared host.
Domain names are consist of parts separated with dots. They are:
Top-Level Domain name or TLD
Second-Level Domain name or SLD
TLD name has the last position in domain. For example, in feminaportal.com, .com is top-level domain name and feminaportal is second-level domain name. What makes your domain name unique and recognizable is SLD; in our example it is feminaportal.
When talking to clients about marketing their business online I am asked "How do you get to Number One on Google?". While this article is not specially about how to become Number One - I will explain what the process is and what's involved.