Batoi Research Group 
There are many scenarios when at times you may want to stop search engines from crawling your WordPress website or listing it in search results. For example, you are developing your website at a development location and do not want search engines to index it. WordPress comes with a built-in feature that allows you to instruct search engines not to index your site. In this article, we will guide you how to stop search engines from crawling a WordPress website.
Go to 'Settings > Reading' and check the box next to Search Engine Visibility as shown in Figure 1.
Note: When this box is checked, WordPress adds some lines to your website's header and modifies your website's robots.txt file which asks the robots (web crawlers) not to index your pages. However, it is totally up to search engines to accept this request or ignore it. Even though most search engines respect this, some page or random image from your website may get indexed.
