Sounds like you are working with an umbrella agency.

If this worker is truly self employed why are you paying wages to anyone else other than the worker?

You may wish to read up on IR35.

Either way your contract = your problems

K. 

PPE regs 2022 tried to clarify who the "employer" is and therefore who providesPPE. Have you read Personal protective equipment (PPE) at work regulations from 6 April 2022 (hse.gov.uk) to decide if these contractors are limb a or limb b?

In general H&S terms as an inspector we always used to look at who had the major control of the work activity when deciding who should take responsibility.  Remember Reg 3 of management regs - coordination and co-operation where more than 1 employer occupies a site. Used to be a realy good ACOP but went under the "red tape" attack on H&S a few years ago.

Its possible to specify specific training needed as part of a contract to bring these people onto site, but its lickly you are always going to have to do site specific induction/training as you will be generating the Hazards/Risks.

Dont forget the general principle of HASAW etc 1974 - to keep people safe from work activities -and do whats reasonably practicable. Sometimes i think we get two hung up looking for a specific regulation that tells us exactly what to do.