The Construction Industrry guidance during this pandemic states: "Stop all non-essential visitors coming to or entering site". Are auditors classed as "non-essential?"

Depends upon perception and ultimately the purpose of the audit.

Also are auditors "vistors"? Typically they are representatives of a company with direct site involvement.

If it is a financial auditor verifying the disposition of company assetts then this is very much date specific, and in the current business climate very essential

If it is a management systems auditor then their employers would be seeking to control risk as much as the site so would likely be capable of some time delay

If it is an H&S Audit by the client/contractor/sub-contractor then it is an inspection of standards applicable upon the date - whilst it may be advisable to delay you can only put off such an audit for so long (at one employment our employees were typically on site for less than five days at any one job)