We plan on introducing a system for Assessing the quality of a process safety programs. This means that it is important to distinguish between Audits and Assessments. Doing so helps us create process safety "wisdom", as discussed in Safety Moment #24: Where Then Shall (Process Safety) Wisdom Be Found?
We offer an ebook to do with the Audit/Assessment distinction. Some further thoughts on the topic are provided in this Safety Moment.
Evaluations of risk management programs can take one of two forms. The first is a formal audit in which a management program is measured against an external standard such as a regulation, an engineering standard or corporate benchmark. An audit compares “what is” with “what should be”. It is fundamentally a pass/fail test; it is legalistic in nature.
Audits are a necessary component of any management system. By comparing actual performance with written standards audits deficiencies and gaps are identified. For example, paragraph 1910.119(f)(1) of the OSHA (Occupational Safety & Health Administration) Process Safety Standard states that:
The employer shall develop and implement written operating procedures that provide clear instructions for safely conducting <an> . . . Initial startup;
An auditor who is examining a facility’s operating procedures will check that they do in fact contain instructions to do with initial startups. If they do, then the audit requirement has been met. If the auditor finds that the requirements have not been met then he or she generates a finding or gap. It is then the responsibility of the facility management to determine the cause of the problem and then to correct it. It is not the auditor’s job to provide suggestions or guidance as to how the gap might be closed.
(The formal nature of audits does not mean that they cannot lead to performance improvements or that the audit is just a paper exercise. Indeed, the fear looking bad or of being penalized is a strong motivator that can generate major performance improvements.)
If the auditor is asked to provide guidance — as is often the case — then he or she has stopped being a formal auditor and has become an advisor or consultant. There is nothing wrong with this shift in the task description but it is important to understand that the shift has taken place. An auditor’s opinions and insights are often extremely valuable, but they are not a part of the formal auditing process.
The second type of program evaluation is less formal — it is a Review or Assessment in which the reviewer provides an opinion as to the quality of the process safety program. In the case of initial startup procedures, for example, a reviewer will provide an opinion as to whether or not those procedures will actually help ensure that the facility starts safely and according to plan. He or she will develop that opinion by asking questions to do with the level of detail, the writing style and the clarity of the instructions.
(The terms Verification and Validation are sometimes used to make the distinction between audits and assessments. Verification is concerned with ensuring that a facility meets the letter of the law or regulation; validation determines whether it is meeting the spirit of the same law.)
An auditor must be strictly objective and show (a) that he or she has been measuring performance against an external standard, and (b) that all findings are fully documented. A reviewer, on the other hand, is allowed to be subjective and is encouraged to offer opinions based on general experience.
Much of the literature to do with auditing stresses the “team relationship” that should exist between the auditor and the personnel of the facility being audited. According to this view both auditor and persons being audited work together to develop ever higher levels of excellence. This rather rosy picture belies that the fact that an auditor aims to find gaps and deficiencies. Indeed, given that the identification of gaps or deficiencies may result in penalties being applied or careers being held back, it would be naïve to believe that the auditor and the facility management are always “on the same team”. A reviewer, however, is certainly on the same team as his or her client.
Process Safety Wisdom
Safety Moment #24 introduced the Venn Diagram shown below. It shows process safety as being made up of (1) formal knowledge and education, (2) management systems, and (3) practical experience. Of these three components it is suggested that it is the third — practical experience — tends to be the weakest and which therefore provides the best opportunity for improving process safety performance.
But, just an audit is the starting point when developing management systems, so it is necessary to conduct an assessment in order to identify issues to do with the integration of practical experience into the program.
In Safety Moment # 26: Once upon a time . . . we noted that people communicate best through story-telling. So — as the opportunity permits — we will provide a brief story that will help illustrate an aspect of process safety management. This week’s story is to do with a missed educational opportunity.
An oil refinery underwent an unscheduled shutdown. The problems that had lead to the shutdown were corrected and the refinery was ready to be re-started. The refinery superintendent — who knew the facility extremely well — had recently retired for health reasons. However management asked him to come back and take charge of the re-start because they knew that his experience would help them get up to speed in as short a time as possible. He did come back and spent most of two days and nights in the control room directing the startup. His knowledge and experience was so great that he was able to do all of this from memory — he did not need operating procedures or checklists. And he successfully brought the refinery back on line in a short amount of time.
But management missed an opportunity. If they had made a video of him standing there barking out orders they would have had a wonderful training guide for people who were new to the facility. But they did not make that video — the opportunity was lost.