TopCrewSystemsBot

TopCrewSystemsBot is the crawler Top Crew Systems uses. A crawler is a program that visits web pages and reads them. In your server logs it shows up as:

What it is

Top Crew Systems reads a business's own public website, only when that business is our client and asked us to, to build and keep their new site accurate.

What it reads

Public pages only, at a slow rate. It does not fill in forms and it does not log in.

How to stop it

Add these two lines to the robots.txt file on your site:

Contact

Questions about the crawler? Email [email protected].