To prevent your site from being indexed or crawled by search engines do the following:
- At the WordPress level run this: Go to <site>/wp-admin/options-privacy.php.
- From the WordPress Dashboard: Settings > Reading > Check: Search Engine Visibility (Discourage search engines from indexing this site).
- Add a Robots.txt file to the root of your site containing the following 2 lines:
- User-Agent: *
- Disallow: /