Pipeline Podcast Network

The Source of Shared Knowledge for Pipeline Professionals

Episode 8: Control Room Team Training with Ross Adams

WIN A PRIZE PACK FROM PIPELINERS PODCAST  

Pipeliners Podcast

Our Host and Guest

Description

In the first Pipeliners Podcast episode of 2018, host Russel Treat welcomes Ross Adams back to the podcast to discuss the timely topic of control room team training.

This episode goes into detail about the new frequently asked questions (FAQs) published by PHMSA at the start of this year. What do the FAQs mean for implementing the latest roles and responsibilities? What do they mean for the team training requirements for the pipeline control room?

Also included in this episode is background on what originally led to these addendums to the Control Room Management Rule and why the FAQs are an important resource for control room managers.

Control Room Team Training: Show Notes, Links, and Insider Terms

Control Room Team Training: Full Episode Transcript

Russel Treat:  Hello, Ross, welcome back to the Pipeliners Podcast. You are our first returning guest, so congratulations.

Ross Adams:  I appreciate that. It’s great to be back. I’m a little surprised but I’m definitely honored.

Russel:  The reason I asked you on is last week — last Tuesday in fact, on the 16th of January — there was a set of frequently asked questions (FAQs) that were published by PHMSA, and these were to clarify implementation of some new PHMSA requirements in the Control Room Management Rule, as it was modified right at a year ago.

Probably the best place to start here is to ask you to talk to us a little bit about the advisory bulletin that came out in 2014 that led to these rule changes so that we have a little background on what’s driving all this.

Ross:  Certainly. As you said, the Department of Transportation, the DOT, issued an advisory bulletin — as part of PHMSA, the Pipeline and Hazardous Materials Safety Administration — to operators that detailed a lot of the findings, or at least the initial findings and lessons learned around the well-known Enbridge Marshall incident up in Michigan in 2010.

A lot of that advisory bulletin circles around Integrity Management programs. It also speaks a lot to control room operations, specifically, the training and understanding of roles and responsibilities for control room staff.

Russel:  Exactly. For the purposes of this conversation, we want to focus on the control center operations modifications and what came up. One of the things I know about the Marshall Incident is that there were a lot of different roles in the control center involved around responding to the incident.

Frankly, I would tell you that at least in my experience, I don’t think that’s unique. It’s very common that leak alarm response involves some technical response to determine if the alarm’s actually a leak and some physical response to mitigate a leak should it be occurring.

I think one of the things that occurred in the Enbridge case is they were having difficulty diagnosing the leak alarm and determining if it was a real leak. Basically, what happened is a small problem became a big problem.

Ross:  That’s right. I think some of the ways that that happened — at least in terms of from an organizational standpoint, and you touched on it — isn’t necessarily specific to Enbridge.

I don’t want to speak too broadly, but it could be somewhat of a pipeline industry type pattern where companies pop up and they grow fast and they hire fast. Their initial staff may be trained appropriately, given roles and responsibilities in how to operate with other teams within their organization. Over time, you have attrition and you hire new staff, or you have new asset growth which means new personnel.

Sometimes those personnel aren’t integrated into that original training and so there’s discrepancies. There’s miscommunication. There’s opportunities for improvement in terms of teamwork and lack of clarification of structure and who’s able to make what decisions and when. That was certainly the case in terms of the Enbridge incident and what came out of that advisory bulletin.

Russel:  That kind of leads us, Ross, into what we want to talk about in a little bit more detail, and that is these roles and responsibility modifications and the control room team training requirements that are coming out of this.

I actually have a copy of the advisory bulletin. You got that into my hands, I thank you very much. For the listeners, we will link that up in the show notes, so that you can find a copy of this advisory notice if you would like.

One of the things it mentions is it says the leak detection process was prone to misinterpretation, and that the analysts and operators were not adequately trained in how to recognize or address leaks, particularly during startup and shutdown.

Being a bit of a leak detection guy, what’s true in leak detection is mostly detection systems run optimally when the pipeline’s flowing steady state, meaning a constant pressure, a constant flow rate.

They don’t run as well during transients, things like shutdown, startup or so forth, and consequently, it’s much more difficult to identify whether or not a leak alarm is valid in the case of transient operations. The last thing it says here in the document is Enbridge failed to train control center staff in team performance.

I think this goes to your point you were making a little earlier is that I know, at least in the work that we’ve done, that when we’ve done roles and responsibilities work, there’s tended to be a focus on the controller and not as much a focus on others that might direct the controller in case of emergency or abnormal operations.

As we all know from playing the (telephone) game in kindergarten where you pass a sentence by whispering it to a number of different people, clear, concise, well-understood communications is difficult enough under the best of circumstances. Any kind of situation that adds any ambiguity or complexity, or stress, causes the ability to do that communication effectively to get more difficult. Really, I think that’s the underlying root of what’s coming out of this recommendation.

Ross:  Yeah. Who’d have thought the telephone game, it’d be circling back around all these years later [laughs] to be pertinent to Federal rule, but that’s certainly been the case.

Russel:  What were the changes that came out in the rule in January of 2017?

Ross:  There’s two elements to the changes that came about in January of 2017, one specific to roles and responsibilities, and the other’s specific to the team training we’ve been talking about, both of which are in the 192.631 rule pertaining to gas transmission, as well as the 195.446 rule pertaining to liquids.

It’s important to know that — depending on what kind of assets you have — chances are this rule will apply to you. The roles and responsibilities rule comes about in 192.631, Section B, Subsection 5. I’ll read that for our listeners real quick.

Roles and Responsibilities Section B says: “Each operator must define the roles and responsibilities of a controller during normal, abnormal, and emergency operating conditions to provide for a controller’s prompt and appropriate response to operating conditions, an operator must define each of the following.”

This is where the new subsection comes out. “The roles, responsibilities, and qualifications of others with the authority to direct or supersede the specific technical actions of a controller.”

Russel:  Let’s talk about that a little bit. I find, at least in my experience, it’s very common that for emergency operations, control of the incident or situation will shift from the controller to a operations manager or executive in the field, and that can be quite complex.

Ross:  I think that’s right. It presents a lot of questions to a controller where the controller is oftentimes comfortable in a normal operating condition, because that’s, after all, the operating condition that is normal.

Not to repeat myself, but to go outside of that means that there are shifts to roles and responsibilities, and it also means that there’s probably going to be personnel involved in either an abnormal or emergency operating condition that aren’t normally involved in the day-to-day tasks of the control room.

Russel:  Yeah, and I think that’s a key point. This comment of “the authority to direct the controller or supersede specific technical actions of a controller,” we’re going to unpack that a little bit more, I think, as we talk, in a bit more detail.

Why don’t you kind of go through for us the team training addition to Section H?

Ross:  Again, Section H — and this is for both the 192 and the 195 rule — what it stipulates is that “Each operator must establish a controller training program and review the training program content to identify potential improvements at least once each calendar year, not to exceed intervals of 15 months. An operator’s program must provide for training each controller to carry out the roles and responsibilities defined by the operator. In addition, the training program must include each of the following elements…”

The new element that’s been included is in (h)(6), and it says that “control room team training and exercises that include both controllers and other individuals defined by the operator who would reasonably be expected to operationally collaborate with controllers and control room personnel during normal, abnormal, or emergency situations. Operators must comply with the team turning requirements under this paragraph by no later than January 23, 2018.”

Russel:  I think it’s important to note, too, that there is no date on the (b)(5). It’s basically in effect immediately upon publication of the rule, but kind of a moot point because January 23rd, as we record this podcast, is tomorrow. If you don’t have this in place, you’re behind. [laughter]

I think the thing that we want to talk about now is, what is team training, and why is it important? I know you’ve done a little research about some other industries that are doing team training. What have you found there?

Ross:  It’s certainly evident, functional, and beneficial for other industries. Oftentimes, team training is actually referred to as either team resource management or crew resource management, so either TRM or CRM.

If you Google that, there’s all kinds of things that start to come up. Kind of in the history of all of that, especially in, for example, the aircraft industry, they were finding that pilots and co-pilots who didn’t normally work together were having issues communicating, or they were taking for granted communication.

Important information that should have been communicated from co-pilot to pilot or vice versa wasn’t being communicated, and it was leading to a lot of incidents. The FAA adopted rules and requirements surrounding team training which have been prevalent in their industry for a while. It’s in railroad, it’s in maritime, and now it’s in the pipeline industry.

Russel:  In my experience, having spent about five years in the Air Force, is it’s all through what’s done in the military in all kinds of exercises. When you’re doing exercises in the military, a lot of that is about communications, and making sure that communications is timely, accurate, and properly understood.

Maybe we should segue a little bit and talk about the frequently asked questions that were published last week. I personally find this information kind of compelling. I don’t know that we need to talk about all the FAQs. Again, we’ll put these out on the show notes and link them up so that any of the listeners that are interested can find them and print them out.

One of the questions is, “Who are the individuals described as ‘others with authority to direct or supersede?'”

At least in my experience, that tends to be operations executives. Generally, they’re individuals with a great deal of experience, and with probably experience in all kinds of operations, not just from the control center, but also in the field, and also emergency operations.

One of the things it’s saying here is that those individuals need to be qualified and authorized to direct or supersede the technical actions of a controller. That raises the question, what does it mean to be qualified and authorized? I’ll direct that to you. What does that mean?

Ross:  That’s a good question, and it’s an important question. The FAQs speak to that to a certain degree. “Qualified” and “authorized” are buzzwords that have kind of been part of these rules and requirements for some time around training.

In terms of this rule, it’s interesting, because I think you pointed out that chances are, these are going to be operations personnel outside of your typical control room staff, which may include executives.

To be qualified, it requires that you would participate in operator qualification training, commonly known as OQ training, in terms of the OQ requirements. The FAQs point out that those other individuals don’t necessarily have to be trained as a controller for the OQ requirements, but they do need to be trained as an “other” who has the authority to make decisions that could impact the control room personnel and the controller’s decision-making, and the actions that are taken by the controller in any of the normal, abnormal, or emergency operating conditions.

To be authorized, I would say, is probably that your company, through policy, has stipulated that this job title with this qualification is allowed to perform these certain tasks in these certain conditions.

Russel:  Right, perform these certain tasks under these certain conditions, like under a pipeline emergency or that sort of thing.

I think the challenge here is one, making sure those people are OQ’d, and number two, making sure the policies reflect reality. What often happens in a real emergency response is that you do what you need to do as best as you are able to do to respond to the emergency, which may not rigorously follow policy. That’s kind of the rub here is what I think that we’re being asked to move towards in the industry is actually to the team training to figure out, what are the authorities that need to be authorized, and what is necessary to properly qualify?

Ross:  That’s right. It’s so important to be very clear in terms of policies and procedures in that regard, but also in the training.

One of the things that really struck me doing the research and reading up on team training and the NTSB reports coming out of the Enbridge incident, is this phenomenon where the NTSB noticed that people who would otherwise have the knowledge to make the decision and probably — given the roles and responsibilities hierarchy — should have been making decisions, ceded their authority to someone else who wasn’t otherwise qualified or authorized. That led to decisions being made that were detrimental to that situation.

Russel:  I think the other thing that’s important to underscore, particularly as it relates to the pipeline control room, is very rarely does the control room actually cause an issue. But, they are always involved in responding to an issue, so the effectiveness of a pipeline control operation — one way to measure that is how quickly do they identify an abnormal or emergency operation, and what do they do to mitigate it, and how effective is that mitigation?

Certainly, there’s a lot that’s talked about in a lot more detail in the NTSB report, and again, we’ll link that up.

One of the frequently asked questions that was published was, “What qualifications are required for an individual to the authorized to direct or supersede?” It refers to OQ requirements, but not necessarily as broadly qualified as a controller.

One of the things you can do in your OQ program for these authorized other people is to qualify them within the role that they’re operating, within the responsibility that they’re operating, not necessarily as broadly as pipeline control. That, of course, raises the question, “What is that distinction?”

I think the other thing we probably ought to talk about a little bit, Ross, is the B.07 response. B.07, that frequently asked question is, “What are the responsibilities of individuals who have the authority to direct or supersede?”

That needs to be delineated in the roles and responsibility, but ultimately what happens is that individual is now taking responsibility for the operational actions, and the controller no longer is responsible.

One of the premises, in my interpretation, throughout the rule, is one and only one person can have operational control at any given time. It further elaborates that the procedures should account for the occasion when such authority is evoked, and the controller disagrees. I think that’s very pertinent and very material, particularly if you understand the fullness of what was going on during the Marshall Incident, that that’s a really key thing.

But how does that actually work from a policy perspective?

Ross:  That’s a great question. [laughs] What I think of is the importance of stop work protocol in industry, and how that’s shaped and worked to create an environment where any and every employee can speak up when he or she feels that operating conditions may be unsafe.

I think this is somewhat in the same vein. It’s saying that there are individuals who have authority to intercede in control room operations, but that doesn’t detract from the fact that a controller still has knowledge, experience, and expertise to speak to a particular issue.

Russel:  Right. As you’re thinking about your procedures for how you’re going to implement this stuff — and the reason I address this is if you’re going to do team training, you need to do it within the context of procedures. What I think comes up is first, under what circumstances does another individual have the authority to direct the controller? Secondly, what happens when the controller disagrees?

Maybe another way to ask the question is, what happens when I get a leak alarm, either through direct detection, or maybe through aerial inspection, or maybe through a call in from somebody who’s a third party and see something on the right of way. What happens? How long do I have to determine whether or not it’s a real leak, and then, once that time has expired, what do I need to do?

These decisions need to be made at a policy and procedural level, and they’re potentially different for different operations, given their specific risk, the product they’re flowing. What’s the implications for taking any particular actions? That all has to kind of be a prerequisite, if you will, to doing this team training.

Lastly, I think it’s probably a good idea to talk a little bit about the team training and what it says the training ought to look like.

Ross:  Just to put a cap on what you were saying, it strikes me, if you’re an operator who’s got a control room management plan, that plan should detail roles and responsibilities. It may be advisable to take a closer look at that section, given these new FAQs.

To answer your question concerning the team training, and how it should be conducted, I’m looking at FAQ H.07, which may come as a surprise to some folks, just because what it’s saying is that they requirements for this particular type of training are going to be a little bit more intensive than what you may be used to in terms of OQ training.

What I mean by that is that a lot of OQ training can be performed as part of a computer-based training, or on-the-job training. What PHMSA is saying is that those types of training delivery — in this context of team training — aren’t sufficient.

And that what they want is for you to perform exercises that include all of the different individuals who may be involved in particular scenarios concerning the control room, including those others that we’ve talked about from the updates to Section B.

And they need to walk through not only lessons learned from your experiences as an operator, but also from industry experience. It strikes me, the Enbridge Incident’s probably a good place to start.

But by no means is computer-based training or on-the-job training outdated or not at least helpful in delivering some of this information, but it’s very important to PHMSA to integrate those exercises and those tabletop drills into this team training program that you’re building.

Russel:  Right. What would be your interpretation of how frequently they need to run one of these exercises?

Ross:  The FAQs, in my reading, do not specify specifically how often those training courses need to take place, but I think from an organizational standpoint, it makes sense to do them on the same schedule as the rest of your required training, which is annually, not to exceed 15 months.

Russel:  That makes sense to me as well. When you look at all of the training requirements that are in the rule, the training piece to all this can get fairly big.

Ross:  I think, kind of putting it on that schedule, knowing that you have to do analysis of the effectiveness of this team training program, and you’ve got to get feedback from the participants, but also that you’ve got to manage the documentation and the performance of these training exercises. Having them all on the same schedule, in my mind, makes them much more manageable.

Russel:  Yeah. I think that’s right. The next FAQ talks to this a little bit.

Again, it doesn’t give you any specific time frames on how frequently you need to, do this, but it does say that it should be taken into account if you have newly hired employment, new or changing job assignments, or other reasons, and certainly, other reasons would be, “Am I adding new pipeline?”

Ross:  That’s right. I think also, you’re always going to be examining your training program, and you may be adding new coursework or new subjects. Industry may push new subjects.

It does specify that folks who have taken team training originally wouldn’t necessarily have to repeat it if you were to make those additions. But I’m looking at the list of all of the exercises that PHMSA’s recommending, and the list is somewhat exhaustive. It strikes me that some of them are soft skills, but some of them have a lot to do with the roles and responsibilities that we talked about earlier.

Here’s a little bit of the list that’s recommended in Section H.06 of the FAQs, and it says, “Important skills include, but are not limited to, teamwork, communication, situational awareness, decision making, leadership, professionalism, understanding roles and responsibilities including how company leadership and executive management are involved in operational decisions.”

The code word there is “others.” … “Recognition and appropriate responses to emergencies, resolution of data discrepancies, error diagnostics, error management relative to procedures, and problem-solving.”

Again, you kind of see that this is a very broad list, and it’s going to require some work by the operators to really get wrapped around what this would look like for them. It’ll look a little different for everybody, but it’s going to be a challenge, I think.

Russel:  Yeah. Again, like a lot of these things, it probably improves how we operate. It probably increases pipeline safety, but it is going to be some work to get it in place.

I think the other thing… again, I’m looking at these FAQs. The other one is, “Does every team training exercise have to have a controller?”

Ross:  The answer is yes, and I think it’s interesting that they state that only one fully qualified controller must participate. [laughs]

I don’t know if that guy draws a short straw or what, but he would be responsible, in my mind, for collecting the information of the lessons learned out of that exercise, and bringing them back to the control room to share with his peers.

I’d say that the more people that are involved, within reason and with respect to your organizational structure, the more awareness they’ll have of the requirements and of the expectations and of how to execute all of the lessons that are coming out of these training courses.

Russel:  Right. Is there anything you think we ought to address as we kind of wrap up here?

Ross:  Just as we wrap up, I think one of the key focal points of the FAQs is an area that we talked about, actually, in my last podcast appearance, which is documentation. We talked in that episode about how much of a challenge documentation can be for operators.

As you may imagine, as there are new requirements and new expectations for operators in terms of growing roles and responsibilities, and growing a new team training program, there’s also going to be a lot of expectations surrounding documentation and record keeping.

J.05 addresses that in quite a bit of detail, which is helpful. Clearly, it establishes the need to document and retain records, especially the qualifications from the OQ training for each of the people who have been authorized, as well as the time frame or the necessary renewal date for those qualifications.

It goes on to talk about the fact that that information has to be passed down to the controllers, so it’d be helpful to put together a list that states who’s qualified, what are they qualified for, how long that qualification lasts.

So that if an incident does occur and someone — one of those others, if you will — is working to intervene in control room operations, the controller is aware of that person’s qualification and authority, whether it’s active or not, and whether they’re up to date on their training to determine whether it’s safe to proceed given the expectations surrounding the roles and responsibilities in that situation.

Russel:  I think the word in that FAQ that leaps out at me is “unambiguously.” What it says specifically is so that “every controller unambiguously knows which individuals are authorized.” You’re elaborating on that a little bit about what would be required so that it’s unambiguous, right?

Ross:  That’s right. “Unambiguous” was too big a word for me, so I had to use a lot of words to describe what you did in a very short period of time but that’s exactly right.

Russel:  Obviously, there’s more information in the FAQs than what we’ve talked about on the podcast. Ross, I appreciate you coming back and providing that information. I’m sure people will find it helpful.

We will get all of this identified and linked up, and we’ll get that on the show notes page. For those listeners that want to do the detailed analysis of all of this, they’ll have a simple place to go find it all.

With that, Ross, thank you very much for being a guest. We look forward to having you back the next time we get some changes in all the PHMSA rules.

Ross:  Thanks so much. Hopefully for the operators’ sake, it’ll be a little while longer before that happens, but it’s always a pleasure.

Russel:  All right, Ross, thanks a bunch. We look forward to having you back.

[show outro]

Transcription by CastingWords

Categories: Podcast, Control Room Management, Pipeline Safety, Pipeliners Podcast

More Episodes