Webmaster Tools
Free WebMaster Tool to help beginners and pros alike get their site to the top!
Welcome to our Webmaster Tool, a free utility to build sitemaps and help improve your ranking on search engines. Perhaps you are new to website design or perhaps an expert looking for that top spot, either way, this web tool is for you!
The webmaster tool allows you to see your website the way the search engines do and has the following features:
- Build sitemaps for any search engine no matter how large your site is.
- Build Yahoo Sitemaps, Google sitemaps, XML sitemaps and more.
- Build HTML Sitemaps for human visitors and smaller search engines.
- Find broken or missing links and what pages are linking to them.
- Check the load time for each of your urls.
- View server headers, check for proper redirects and more.
- View and analyze page content for proper formatting
- View keywords and other information used on pages.
If you want to jump right into the program and skip the documentation, then click on our Webtool graphic to your left.
Also, see this 60 second Web Master Tool Video.
Web Tool
For those of you that want to read up a little before using web master tool, then continue reading.
This webmaster tool has a number of windows, such as Settings, Crawler, Sitemap, URL check and System. All items in each window include a question mark (?) that provides basic information. Below, I'll cover the basics.
XML Sitemap Generator
The webmaster tool has many sections: For detailed information on how to create a HTML, Google or XML sitemap, then visit our page dedicated to Building Sitemaps. The page you are on (this page) provides an information about the webmaster tool as a whole.
* In order to build XML Sitemaps, you'll need Sun's Java Runtime which is free and located here: http://www.java.com/en/download/manual.jsp
To get started building your Free XML sitemap, simply enter your website address in the 'URL' field located under the 'Settings' tab. You can also enter a project name if you would like to save the project for later. The start the process, simply click on the green arrow located at the bottom of the webmaster tool.
This tool will spider an entire site and look for broken links. If broken links are found, it will report exactly why that link is broken, the files linking to the bad link and other important information.
To start the link checker you will first need to visit the 'Settings Tab' and enter the URL of the site you would like to validate.
If there are web pages that you would like to exclude from the link checking process, then simply enter them in the exclude field, one per line. Everything you enter is case sensitive. A few examples would be:
*/images/* - excludes all images from validator
*.pdf - excludes all lower case pdf files from the link checker
*.PDF - excludes all upper case PDF files from the link checker
You can tell the link checker to check only links that start from a directory, such as yoursite.com/store/storefront1/
To do that, simply enter the full address in the URL bar, such as http://www.yoursite.com/store/storefront1/ and then select 'Load from directory' in the rules section.
The link checker will not check any links or (links in directories) that are excluded in the robots.txt file, unless you deselect 'Respect Robots.txt file'. The same applies to the meta name and no follow.
A cool feature of the link checker is the ability to change the user agent. Why would you do this? Say that you track your visitors and page views and do not want to skew the numbers, you can enter your own custom user agent (just click on it and start typing) and then tell your website tracking software to ignore traffic from that user agent.
I've had an ISP actually block the Googlebot! I was not ranking well or getting indexed because Google could not see my pages. Turns out that the ISP thought they were doing me a favor by preventing the server from crashing due to the number of requests Google was making - doesn't say much about their servers does it :)
By setting the useragent to appear as Google bot, I was able to spider my own site and verify response times.
Link Checker
The crawler is pretty self explanatory and allows you to slow the link checking process or speed it up. To slow the link checker down, you would increate the request delay and thread count, to speed the link checking process, you would decrease connect time out, read time out, and thread count.
Sitemap
The Sitemap tab (I thought about making naming it Links) is where you will find information about links, such as load times, ping times, broken links, error messages, page titles and much more.
All the headings, such as Nr, Location and Title can all be expanded. If you click on the Title of a column, it will sort all the web pages based on that that column. Click the column again to inverse the sort.
For SEO research, you'll want to review the Titles - It is never a good thing to have messed up or duplicate titles.
If errors are found, the type of error will be reported and in the In-L column, you'll see a plus next to a number. Click on the plus and you will see all the pages that link to (reference) this bad page.
URL Check
This is an important tool to finding out why a page does not load or why Yahoo, MSN or Google is not including it in their index.
To start, enter the full address of the page or site you would like to check. For example, http://www.badspiderbites.com
In the 'request properties' box, you can enter values to send to the server such as the user agent or referer. There is a user agent setting in under the sitemap tab, but it's not the same as this url check setting.
To enter a user agent and referrer when validating a page, simply enter:
User-Agent=X
Referer=X
Where X equals the value you want. Here is an example:
Referer: http://www.badspiderbites.com
User-Agent: Firefox/11.2Beta
If you ran the URL check on your site with these settings, your log files would show that the request was made by the Firefox browser and the visitor was referred by the site www.badspiderbites.com
To see the content of a webpage or site, simply check the 'to content view' before you click the 'Start' button.
So, why would you want to set these values anyway? Unfair Competition is an excellent example! Let me explain:
Unfair Competition
You work hard to create a site with great content hoping that you'll rank high but for some reason, there is a site that seems to outrank you no matter what you do. In fact, the site that is holding you from the top position has junk and in no way should be doing as well as it is.
What they are likely doing is called 'Cloaking' and not only is it wrong, but if you discover it and report it to the search engine, they will be removed from the index (read, you move up a position in ranking!).
To find out if a site is cloaking, simply set the name of the search engine bot to that of your search engines robot and if you see different content than what appears under the default value, chances are the site is cloaking.
Another hint that a site may be cloaking is that they have their page set to not be cached by the search engines. Different content plus no-cache are strong indications that you are dealing with unfair competition.
Server Headers
When you check a site or webpage and press start, you'll see the server headers. This the information the server (website you are checking) sends to the browser. In most cases, you want the first line under Header Fields to report 200 OK.
Advanced: If you had a webpage and are redirecting it to your new webpage, you'll probably want it to be a permanent redirect (301) and not a temporary one (302).
A common problem with low page rank or no indexing is that a website referrers to their main page but that main page is a redirect to another page, usually deep inside their directory structure.
View the Code - document properties
When you click the 'to content view' option, then click start and then click the 'Content' Tab (next to 'Request'), you'll see the content of the webpage (code).
If you then click 'Parse document info', you'll see a document tree and document info. The document tree is more advanced and can help webmasters discover missing head tags, body tags and more.
The Document Info will show you the title, number of links, meta tags and a listing of all the links found on that page.
System Information
The last option is the 'System Tab' shows you how much memory Java has available for use. If you are checking links or building a XML Sitemap for a large site, you'll want to allocate more memory. Allocating more memory is as simple as issuing a command - see this 60 second video on increasing Java memory for more information
To increase the memory available to Java, simply add the parameter –XmxNNNm, where NNN is ½ of your total conventional memory in megabytes. On Windows, this is done through Control Panel -> Java -> Java tab -> Java Applet Runtime Settings -> View.
For example, say that you are running the link checker on website containing 500,000 pages, simply type "-Xmx512m" in "Java Runtime Parameters" field (provided you have at least a total memory of 1GB - on average, you can go up to half of your computer memory).
Cool Webmaster Tool
Like this webmaster tool? Please let others know by displaying this icon.
Simply copy and paste the code snippet below onto your webpage:
or for the icon below
Use the following code snippet:
and of course, you can also buy me a cup of Chai :)