As with any role and organisation, working for HSE has its pros and cons. On the plus side, you generally get good training, including a good grasp of OS&H law, plus wide experience of many different types of organisations, processes and situations - good and bad. Also, the support from its specialists was and very probably remains good. However, though I was a general inspector with HSE for over 10 years, I left in 1987, so various aspects of HSE will inevitably have changed since then. I joined with minimal experience of the world of work, having only done temporary jobs during college vacations and after leaving college. By contrast, people with wider experience of organisations and work activities may be of more use as recruits to HSE.
By its nature, HSE is a notably bureaucratic organisation. Also, inspectors and specialists can be liable to transfer anywhere in the UK - which can cause related disruption and strains, especially for those with families. However, transfers are probably very limited these days in view of the recent major cut in HSE's budget. Another effect of the cut is that HSE is closing some of its regional offices. As a result, inspectors will almost certainly find their travelling increases considerably and have even less face-to-face contact with colleagues. In my experience HSE had a wide spectrum of people, including some who were notably down to earth, dedicated and helpful, plus a few who seemed to enjoy being officious and exerting their authority.
My experience, skills and knowledge gained with HSE have been very useful, and probably enhanced my chance of getting another job when I decided that I didn't want to work for HSE for the rest of my working life. (As nobody tried to persuade me to stay, I was probably mediocre as an inspector!) One skill which I had tended to take for granted was interviewing and taking effective statements from witnesses. Therefore, if HSE advertises professional vacancies which might be of interest to some forum users, consider getting more information and then weigh up the pros and cons of such jobs when deciding whether or not to apply.