Audits and Assessments

Audits and assessments in the process and energy industries


$31.50 USD

Add to Cart


All management programs need process safety audits on a regular basis — there are always gaps between what “should be” and “what is”. Process risk and safety management programs should also be evaluated for effectiveness. It is possible for a system to meet audit requirements but still not be all that effective. For example, the program will call for Operating Procedures to be written. An auditor will look for a set of Operating Procedures and, if they exist, will “check the Operating Procedures box”. However the audit does not tell management other crucial information such as whether the procedures are too long or too short, whether they are readable and whether they are usable in operating conditions (such as when it is raining).

This ebook discusses both the audit and review of assessment processes.

An audit has been defined as follows (CCPS 2011),

A systematic, independent review to verify conformance with established guidelines or standards. It employs a well-defined review process to ensure consistency, and to allow the auditor to reach defensible conclusions.

In other words, an audit compares “what is” with “what should be”. A management program is measured against an external standard such as a regulation or corporate benchmark. It is fundamentally a pass/fail test. For example, paragraph 1910.119(a)(1) of the OSHA (Occupational Safety & Health Administration) Process Safety Standard states that:

the employer shall complete a compilation of written process safety information . . .
This information shall consist of at least the following:

(i) Toxicity information;
(ii) Permissible exposure limits;
(iii) Physical data;
(iv) Reactivity data:
(v) Corrosivity data;
(vi) Thermal and chemical stability data; and
(vii) Hazardous effects of inadvertent mixing of different materials that could foreseeably occur.

An auditor who is examining a facility’s process safety information against the OSHA standard will check that the information specified is written down and made available to those who need it. If those requirements are met then the audit requirement has been met. If they do not then the auditor has identified a deficiency or gap. It then becomes someone else’s responsibility to turn the findings into recommendations or action items. An auditor’s job is to objectively uncover deviations from the standards — no more, no less. The auditor is interested primarily in the letter of the law. Therefore, with regard to the safety information example just provided it is not the auditor’s job to assess the quality of the information or the manner in which it is communicated.

The findings of a formal audit can lead to major improvements in the design and implementation of management systems; the fear of looking bad or of penalties is a strong motivator. (Concern to do with penalties was in evidence in the early 1990s. Although the value of process safety management principles had long been recognized, it took a regulation from OSHA to force companies to complete their process safety work.) 

The audit team should strive to make general conclusions from the detailed observations. For example, if the auditors find that some pressure vessels have not been inspected according to schedule they should use those findings to thoroughly examine the management program for equipment inspection and follow-up. Examples of situations where audit findings lead to the identification of system or institutional failures are provided by Hazzan et al. (Hazzan 2011).

In addition to audits, as discussed above, company management may elect to have outsiders conduct a review or assessment. This type of evaluation is less formal. A reviewer provides an opinion as to the quality of the risk management program. In the case of the initial startup procedures just discussed a reviewer will provide an opinion as to whether or not those procedures will actually help ensure that the facility starts safely and according to plan. He or she will develop that opinion by asking questions to do with the level of detail, the writing style and the clarity of the instructions. Based on the answers to those questions, he will provide an opinion as to the effectiveness of the initial start-up procedures. (The terms Verification and Validation are sometimes used to make the same distinction between audits and assessments. Verification is concerned with ensuring that a facility meets the letter of the law or regulation; validation determines whether it is meeting the spirit of the same law.) Hence it is quite possible for a facility to satisfy audit requirements but to receive a low review evaluation.

It is therefore important to keep maintain a clear distinction between the roles of auditor and reviewer. For example, an auditor may find that a facility has a set of operating procedures, as required by regulation. A reviewer, however, could state that, in his opinion, the procedures are not well effective and should be rewritten.

On the other hand, an auditor may report that many of the elements of the standard to do with operating procedures have not been completed and so the audit requirements have not been met. It is natural at this point for the facility management to ask the auditor to offer an opinion as to the causes of these failings. If the auditor does offer an opinion then he or she has moved into a reviewer role. An auditor’s opinions and insights may be extremely valuable, but they are not a part of the formal auditing process.

In the same vein, much of the literature to do with auditing stresses the “team relationship” that should exist between the auditor and the personnel of the facility being audited. According to this view both auditor and persons being audited work together to develop ever higher levels of excellence. This rather rosy picture belies that the fact that a formal audit is a structured process in which a facility’s performance is measured against some predefined standard. The auditor helps only by identifying gaps. How management elects to close those gaps is not the auditor’s concern. Indeed, given that the identification of gaps or deficiencies may result in penalties being applied or careers being held back, it would be naïve to believe that the auditor and the facility management are always “on the same team”. Audits inevitably are adversarial, at least to a degree. A reviewer, however, is on the same team as his or her client. He or she is not measuring performance against an external standard, but trying to find ways of improving performance.

Two dilemmas:

  1. The company may have an excellent safety program but have snags regarding the formal side of the program.
  2. The company may do well on the audit but actually have a poor program.

This ebook describes the related topic of maintenance, where the term maintenance refers to repairing equipment that has failed, either totally or partially, and preventive actions taken prior to equipment failure.

Maintenance activities exclude upgrades and changes to the system — they are to do only with “replacements in kind”. If new types of equipment or instrumentation are to be installed then these activities should be handled through the facility’s Management of Change (MOC) program. Even larger changes constitute stand-alone projects in their own right.

Maintenance work frequently involves potential contact with sources of high energy such as electricity, process fluids at high temperature and pressure or toxic and flammable materials. Because of their importance to all aspects of safe operations energy control procedures are discussed in the next ebook.

Table of Contents

Formal Audits
   Reasons for Audits 
      Accident Follow-Up
      Regulatory / Standards Compliance
      Stakeholder Outreach
      Voluntary Check
      Insurance and Business Security
   Audit Standards
      Reporting Requirements
      Industry Standards
      Internal Standards 
   Audit Frequency
   Audit Personnel
      Outside Auditors 
      Internal Auditors
      Team Composition
   Auditor Attributes 
      Audit Service Providers
      Interview Skills
      Technical Knowledge
      Writing Skills
   The Host Company
      First Impressions
   Planning the Audit
      Determine the Audit Standard
      One-Point Contact
      Pre-Audit Activities
   Audit Forms
   Conducting the Audit
      Auditor Preparation
      Kick Off Meeting
      Plant Tour
      Information Collection
      Role of Personnel
      On-Site Inspection
      Close-Out Meeting
      Draft Report
      Report Distribution
      Letter of Certification
      Audit Verification
      Positive Findings
      Report Retention
   Follow Up
   Unannounced Audit
The SEMS Audit Rule 
   Audit Requirements 
   Independent Third Party Auditors (I3Ps) 
   I3P Qualifications
National Emphasis Program (NEP)
Reviews and Expert Assessments
   Review Issues
      Management Systems Effectiveness
      Workforce Involvement 
      Real World Usefulness 
      “Learned to Live with It” Problems
      Lessons Learned
   Reviewer Attributes
Management Elements Assessment
   Level 1 — Risk Management 
   Level 2 — Management Element Spreadsheet
   Level 3 — Detailed Questions
   Scoring Template
   Benefits of the Elements Assessment Approach
      Independent of Events
      Handling Abstraction 
      Smoothing of Results

Copyright © Ian Sutton. 2018. All Rights Reserved.