# To block the entire site, use a forward slash.
# Disallow: /
# To block a directory and everything in it, follow the directory name with a forward slash.
# Disallow: /junk-directory/ 
# To block a page, list the page.
# Disallow: /private_file.html
# To remove a specific image from Google image search, add the following:
# User-agent: Googlebot-Image
# Disallow: /images/dogs.jpg 
# To remove all images on your site from Google image search:
# User-agent: Googlebot-Image
# Disallow: / 
# To block files of a specific file type (for example, .gif), use the following:
# User-agent: Googlebot
# Disallow: /*.gif$
# To prevent pages on your site from being crawled, while still displaying AdSense ads on those pages, disallow all bots other than 
# Mediapartners-Google. This keeps the pages from appearing in search results, but allows the Mediapartners-Google robot to analyze the pages to # determine the ads to show. The Mediapartners-Google robot doesn't share pages with the other Google user-agents. For example:
# User-agent: *
# Disallow: 
# 
# User-agent: Mediapartners-Google
# Allow: