Sometimes you just want to take pages out of the Google index, or out of any search engine’s listings. You may have pages on a site that reveal personal or private information, like social security numbers, and you want them taken out of a search engine’s cache or index. You may be displaying duplicate or copyrighted content and need to take it off the site in response to a legal request or court order. As always, it is much easier to remove URLs from sites that you own, since removing information (defamatory or duplicate content) on other sites is a more complex process.
The first step in URL removal usually involves taking down the physical page, or moving it to a location where a search engine can’t read it. You can also use a robots.txt file exclusion to tell the search engine not to read the page or the directory that the page is stored in. Note that taking the page down does not instantly take it out of the search engines, since they tend to crawl each page every couple of weeks under most circumstances. The search engine will still have a cached version of your page available until it determines the page is no longer there. Sometimes, you may even take a page down and see it cached for months in search engine results, so it is best to 301 redirect the old page address to the homepage if you are not using a robots.txt exclusion.
 If you have gotten someone else to remove sensitive content, but it is still cached in Google, you can use the Public URL Removal Tool to request that Google take that information out of its index.
Google has a four part series (see parts 1, 2, 3, and 4 ) on URL removal, and it is important to note that Google’s standard for removing a URL is based on very important reasons for taking a URL out of Google’s index. Normally this is done as a last resort or as due diligence when you have also done as much as you can on your own end to take content down. For example, if there is a page that has sensitive/personal content on it that may breach privacy rules, and you have taken the file off your site but it is still cached in Google, then you would put in a request. Google’s articles also make it abundantly clear that there are situations where you do not want to remove URLs if you want your site to get found.
What if someone is copying your content, and you want the search engines to remove their pages? Under the Digital Millennium Copyright Act, you can compel Google to remove pages from its index if they contain your content. There are forms to fill out, and the other site may appeal your request. As a side note, Google also forwards takedown requests to third party websites so you should not put anything in a DMCA request that is not 100% true and that you wouldn’t mind seeing in a newspaper.
Removing pages from search engines should only be done when it is absolutely necessary. Aside from the reasons mentioned above, page removal can poke holes in your site’s on-page link profile, and make your site appear incomplete to search engines. Unless people are likely to look at the cached version of a page you’ve taken down, redirecting the offending page (if you can’t just get rid of the content) may be a better solution. Even if you do make a request to remove a page or pages, the search engine may choose not to comply.
Tags: remove URL from Google
