The decision to wear or not to wear high visibility is based on your risk assessment and whether other controls implemented adequately control the risk of traffic and pedestrians interfacing. The HSE state:

It is important to have a safe system of work for traffic and pedestrian management. Every workplace should be organised so that pedestrians and vehicles can circulate safely. Workplace traffic routes should be suitable for the people and vehicles using them. Where vehicles and pedestrians use the same traffic route, there should be adequate separation between them.

You can achieve separation by keeping pedestrians and vehicles well clear of one another, ideally using completely different routes. If routes must be close to each other (e.g. side by side), physical barriers to prevent pedestrians or vehicles straying into each other’s areas should be put in place where this is reasonably practicable. Control measures could also include:

pedestrian- or vehicle-only areas; separate areas for car parking away from lorry and lift truck operation; clear pedestrian walkways; barriers and kerbs; crossing points; separate pedestrian doors at vehicle access/egress into buildings; traffic light controls; increased lighting, and hence visibility; ‘no pedestrian access’ areas during loading and unloading operations; ‘no unauthorised access’ areas (visitors and contractors); clear control of drivers once they have left vehicles (walkways, safe zones etc.) and wearing of high-visibility clothing.