Welcome to the Small Business Ideas Forum! We are a community of over 100,000 small business folks with over 163,000 posts for you to browse. We pride ourselves on being the friendliest forum you will find and we'd love to have you as a member of our community. Please take a moment and register for a free account. If you need any help, please contact Chris Logan.

Small Business Ideas Forum

Small Business Ideas Forum

A friendly place to share small business ideas and knowledge, ask questions, find help and encourage others that are involved in the small business industry. Topics include small business marketing, generating revenue and small business computing.

Go Back   Small Business Ideas Forum > Small Business Marketing - Online > Search Engine Optimization
Register Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 30th July 2007, 01:21 PM   #1
Dale King
VIP Contributor
 
Dale King's Avatar
 

Join Date: Nov 2006
Location: Guilderland, NY
Posts: 797
Exclamation Will A Sitemap Get Google To Crawl My Site Faster?

I'm often asked, "why doesn't Google crawl my site faster?"

While there can many reasons for this including server speed, in my experience, the main reason is this:

If Google doesn’t find a good number of relevant, quality links pointing to your site, it doesn’t consider your site to be very important. In other words, the more relevant, quality links your website has, the more value Google places on your site - hence, the more your site is crawled.

Here's how having a sitemap can help: When the Googlebot gets ready to crawl your site, the first thing it does is look for a sitemap. If it does not find a sitemap, it will proceed to crawl your site manually and determine for itself your site's most relevant pages. So the main advantage for having a sitemap is speed.

By speed I mean, Google will take much less time to crawl your site, crawling will occur more frequently, and the Googlebot will automatically know what pages on your site are the most relevant.

There's another reason why you should consider adding a sitemap to your website - increased PageRank. Let me explain:

In addition to external links, Google's PageRank system also takes into account internal links. The more pages your website has, the higher your PageRank. There is a caveat to all of this, however:

In order to reap the benefits of having lots of pages, it is advisable your pages contain original content and not simply be duplicates of each other. Otherwise, your website may actually be penalized and your PageRank may actually decrease.

So do you have a sitemap?

Have you realized the advantages I mentioned?

Dale King


Last edited by Dale King; 30th July 2007 at 06:49 PM.
Dale King is offline   Reply With Quote
Register or log in to remove this ad.
Old 31st July 2007, 06:54 AM   #2
kanasia
Member
 

Join Date: Jul 2007
Posts: 12
Default

I don't have any idea on how to create sitemap. Anybody can provide tutorial for it?

kanasia is offline   Reply With Quote
Old 31st July 2007, 09:53 AM   #3
Trainer Bubble
Member
 
Trainer Bubble's Avatar
 

Join Date: Aug 2006
Posts: 40
Default

There are many Google Site Map Generators. Here is a link to one that came to the top when I searched google http://www.xml-sitemaps.com/ The process for creating it is simple and well explained on the site.

__________________
For all your free business training resources, tools and course materials visit www.trainerbubble.com
Trainer Bubble is offline   Reply With Quote
Old 31st July 2007, 05:58 PM   #4
Old Welsh Guy
Moderator
 
Old Welsh Guy's Avatar
 

Join Date: Jun 2004
Location: Wales
Posts: 700
Default

If your site is currently well spidered then avoid a google sitemap like the plague IMO. I have seen sites slaughtered by poor use of sitemaps. Most generators treat all pages with a 0.5 score and this is not good.

OK so adding a sitemap might get your pages spidered but in a short period of time unless you have adequate inbound links to push your overall pagerank up, your pages will get moved to the Supplemental index.

__________________
Old Bald & Stupid, but more than compensated for by being born Welsh.
Google Expert blog - Internet Marketing Consultancy
Old Welsh Guy is offline   Reply With Quote
Old 1st August 2007, 06:37 AM   #5
copywriter
Moderator
 
copywriter's Avatar
 

Join Date: Jul 2004
Location: South Carolina, USA
Posts: 629

Default

Quote:
the more value Google places on your site - hence, the more your site is crawled.
I'm sorry, but this is just not true. Anytime something changes on your site, the Googlebot will crawl. It will also crawl for a dozen other reasons that have nothing to do with the value of your site or whether or not you have a site map.

Quote:
Here's how having a sitemap can help: When the Googlebot gets ready to crawl your site, the first thing it does is look for a sitemap.
Uh... no, not really. It starts with whatever link pointed it to your site and weaves its way through as much of your site as it can going from page link to page link. What a site map does is make sure every page has an accessible link pointing to it.

Quote:
If it does not find a sitemap, it will proceed to crawl your site manually and determine for itself your site's most relevant pages.
Manually? It's a robot. The whole process is automated from start to finish.

Quote:
...crawling will occur more frequently, and the Googlebot will automatically know what pages on your site are the most relevant.
Crawling won't occur more frequently unless something on your site changes.

I'm not following what you're saying about Google knowing which pages are "relevant."

__________________
Read Karon's Copywriting Blog or learn online & SEO Copywriting with the Step-by-Step Copywriting Course
copywriter is offline   Reply With Quote
Old 1st August 2007, 12:48 PM   #6
torka
Moderator
 
torka's Avatar
 

Join Date: Mar 2006
Location: Triangle area, NC, USA, North America, Earth (usually)
Posts: 4,101

Search Engine Guide Blogger

Default

Somehow, years ago, people go the idea that it's somehow "better" to be spidered more often. The fact is, unless you have time-sensitive content and/or content that changes frequently, then it doesn't really matter how often you get spidered.

Frequency of spidering has nothing to do with how well you rank. Googlebot could visit one of your pages every 10 seconds, but the frequency of those visits won't make that page rank any higher (or lower) than the one it visits every two or three months.

The ranking algorithm takes into consideration a lot of factors (anywhere from 100 to over 200, depending on who you ask), but I'm pretty sure "how often the page is accessed by Googlebot" is not one of those criteria.

A sitemap might be useful if there's some technical impediment that's keeping the spiders from identifying links on your pages, or from retrieving the pages to which those links point. But only if you can't for whatever reason fix the technical problem. The best solution in that case is always to fix the problem; the sitemap is simply a "last resort" in those sorts of cases.

If you have deeply-buried interior pages of your site that aren't getting indexed, this is an indication the SEs don't think those pages are "important" enough to spend resources retrieving. Simply getting those pages spidered through a sitemap isn't going to help them rank well, though.

If you have interior pages that are already being spidered, but they don't rank well, a sitemap isn't going to help that much there, either. One of the XML sitemaps won't do anything for the page's inbound link weight. A "regular" sitemap (the kind that real people sometimes use) does count as one more link pointing to those pages, but it's only one link -- and one from a page that typically has a lot of outbound links on it. So the "weight" it adds to your interior pages will likely be slight. Not usually enough to make a noticeable difference in the rankings.

The solution to both problems is to get better links pointing at those "buried" pages. It may mean fixing your site navigation to bring your pages closer to the "top." It may mean spending some time/energy on building a few deep links. If you can do both, even better.

BTW, relevance has nothing to do with spidering or indexing. Relevance is a search-term-specific ranking concept. The spiders are there simply to retrieve the code of the page and return it to the datacenter.

Googlebot makes no judgement or valuation of relevance. That's done at a much later part of the process (i.e. in the seconds between when you enter your search query and the SERP displays on your screen) by the ranking algorithm.

--Torka

__________________
Diane Aull - NineYards.com: Helping Businesses Do Business Online
Whether you think you can, or that you can't, you are usually right.
torka is offline   Reply With Quote
Old 6th August 2007, 01:09 AM   #7
WebDrops
Member
 

Join Date: Aug 2007
Posts: 42
Default

Quote:
Originally Posted by Dale King View Post
I'm often asked, "why doesn't Google crawl my site faster?"

While there can many reasons for this including server speed, in my experience, the main reason is this:

If Google doesn’t find a good number of relevant, quality links pointing to your site, it doesn’t consider your site to be very important. In other words, the more relevant, quality links your website has, the more value Google places on your site - hence, the more your site is crawled.

Here's how having a sitemap can help: When the Googlebot gets ready to crawl your site, the first thing it does is look for a sitemap. If it does not find a sitemap, it will proceed to crawl your site manually and determine for itself your site's most relevant pages. So the main advantage for having a sitemap is speed.

By speed I mean, Google will take much less time to crawl your site, crawling will occur more frequently, and the Googlebot will automatically know what pages on your site are the most relevant.

There's another reason why you should consider adding a sitemap to your website - increased PageRank. Let me explain:

In addition to external links, Google's PageRank system also takes into account internal links. The more pages your website has, the higher your PageRank. There is a caveat to all of this, however:

In order to reap the benefits of having lots of pages, it is advisable your pages contain original content and not simply be duplicates of each other. Otherwise, your website may actually be penalized and your PageRank may actually decrease.

So do you have a sitemap?

Have you realized the advantages I mentioned?

Dale King
thanks Dale King... this is a great piece of advice... exactly what i was looking for ... thx

WebDrops is offline   Reply With Quote
Reply   

Bookmarks




Thread Tools

Get Updates
RSS Feeds:
RSS Feed for Search Engine Optimization RSS for this Category Only: Search Engine Optimization

RSS Feed for Small Business Ideas Forum RSS for Entire Forum
Forum Rules


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


Small Business Ideas Forum


 
At Your Business - Small Business Directory
Free Business Forms - Prewritten Documents
 
Search Engine Guide
Small business guide to search marketing

 
Small Business Brief
Fetching the Best Small Business Info


Used Values - Free Links
Blue Book





All times are GMT -5. The time now is 11:12 AM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright 2004 - 2018 - Privacy