This week’s Pipeliners Podcast episode features one of our original guests, Will Gage, returning to the podcast to discuss Management of Change (MOC) in the pipeline control room.
In this episode, you will learn about why MOC is different in the control room compared to other groups, the people that should be involved with MOC in the control room, and the importance of teamwork when supporting MOC.
You will also learn about the value of automation in the control room, the challenges of ensuring that automation works correctly in the control room, and the importance of having a management system in the control room.
Pipeline Control Room Management of Change: Show Notes, Links, and Insider Terms
- Will Gage is the Director, SCADA at Targa Resources. Connect with Will on LinkedIn.
- Listen to Will’s previous appearances on the podcast discussing cybersecurity, 5G technology, IT, OT, and more topics.
- Management of Change (MOC) focuses on planning the processes in a system, the type and quality of changes required, the risks associated with the plan for change, and the people that will be impacted by the change so that it is clearly communicated to stakeholders.
- SCADA (Supervisory Control and Data Acquisition) is a system of software and technology that allows pipeliners to control processes locally or at remote location.
- AOC (Abnormal Operating Condition) is defined by the 49 CFR Subpart 195.503 and 192.803 as a condition identified by a pipeline operator that may indicate a malfunction of a component or deviation from normal operations that may indicate a condition exceeding design limits or result in a hazard(s) to persons, property, or the environment.
- Charles Alday is a previous Pipeliners Podcast guest and has discussed workload analysis and team training on the podcast.
- HMI (Human Machine Interface) is the user interface that connects an operator to the controller in pipeline operations.
- PPE (Personal Protective Equipment) ensures the safety of an individual working in the field or at a site. PPE is especially critical when working in dangerous or hazardous work conditions.
- PLCs (Programmable Logic Controllers) are programmable devices placed in the field that take action when certain conditions are met in a pipeline program.
- PHMSA (Pipeline and Hazardous Materials Safety Administration) ensures the safe transportation of energy and hazardous materials.
- The CRM Rule (Control Room Management Rule) captures the regulatory and safety requirements prescribed by PHMSA to support specific aspects of control room management, including Shift Handover (SHO), Hours of Service, and other requirements.
- Bellingham Advisory Bulletin is an advisory bulletin released by PHMSA that covers pilot-operated pressure relief valves installed in hazardous liquid pipelines. The bulletin provides pipeline operators guidance on whether their inspection and test procedures are adequate to determine if these valves function properly.
- Nick (Nicholas) Guinn is a previous Pipeliners Podcast guest and has discussed MOC and the logistics of compressors on the podcast.
Pipeline Control Room Management of Change: Full Episode Transcript
Russel Treat: Welcome to the Pipeliners Podcast, episode 130, sponsored by Energy Worldnet, a worldwide service provider to the oil and gas industry, making the world safer by providing pipeline operators and contractors innovative solutions for operator qualification, safety training, content authoring, and guidance as pipelines operate in compliance with PHMSA, OSHA, and other regulatory requirements. To learn more about Energy Worldnet, visit energyworldnet.com.
[background music]
Announcer: The Pipeliners Podcast, where professionals, Bubba geeks, and industry insiders share their knowledge and experience about technology, projects, and pipeline operations. Now your host, Russel Treat.
Russel: Thanks for listening to the Pipeliners Podcast. I appreciate you taking the time, and to show that appreciation, we give away a customized YETI tumbler to one listener each episode. This week our winner is Mark Graves with Phillips 66. Congrats, Mark, your YETI is on its way. To learn how you can win this prize, stick around ‘til the end of the episode.
This week, Will Gage returns to the Pipeliners Podcast. We’re going to talk about management of change, specifically management of change in the control room.
Will, welcome back to the Pipeliners Podcast.
Will Gage: Thanks, Russel, for having me back. It’s always great.
Russel: In the midst of this recording, you and I are probably both sequestered at our houses due to the coronavirus. How are you holding up being at home with the kids?
Will: I’ll tell you, I’m very isolated trying to keep myself away from everything and everyone. I don’t know. I literally am trying to keep myself away from everything and everybody with the concern of getting work done and everything. I think everybody’s okay. I don’t know. [laughs]
Russel: We’re doing good here. In fact, you’re probably going to hear the doorbell ring in a second because it looks like an Amazon Prime guy just ran up to the house, and he’s running back to his truck. I guess he’s running behind. I think those guys are busy right now.
Will: Yeah, I would imagine so. Who would have ever thought that we would be in a pandemic and we would be seeing Amazon as critical, essential services?
Russel: We’re getting way off the topic that I asked you to come on and talk about, but I do think that on the back side of all this, we’re going to have a very different attitude about how we work.
I think a lot of people are going to figure out there’s a way to work that’s unique and different, and allows them to be more connected to their family and their community versus spending two or four hours a day in a car commuting.
Likewise, I think you’re going to find a lot of the service providers, and schools, and all that kind…I think it’s going to have a big transformational impact, because a lot of people are going to get exposed to things that they wouldn’t have otherwise been exposed to because of this.
Will: Oh, yeah.
Russel: It will be a long time before we know how all this shakes out.
Will: That’s true, and that’s also not a bad segue into what we are going to talk about, if you think about it.
Russel: That’s true, so let’s dive in. I asked you to come on and talk about management of change and in specific management of change in the control center. Let’s start out, we’ll just talk about what is management of change?
Will: Management of change can be broken down into the simplistic form of managing the outcome of change. I know earlier in some of the earlier podcasts we talk about risk management. That’s really just managing outcome. The same is true with management of change.
You want to make sure that you have your change documented that you’re proposing. You want to make sure that all parties are involved. You have to run it like a project, essentially. Then through that process you’re working keeping everybody informed and involved.
When changes have been completed, that the right parties understand and know what those changes are, how it affects their operation, or data, or whatever it may be that process, and then that everybody’s signed off and good.
Russel: A big part of management of change is effective teamwork and effective communications.
Will: Correct.
Russel: The teamwork side of that would be making sure that all the people that need to provide input or have knowledge about the change have an opportunity to participate. The other part of it is trying to get that — from a communication — is for all the people that need to know, they know that this is a change we’re contemplating, versus this is a change we’re making, versus here comes the change.
Will: Yes.
Russel: Those are very different conversations.
Will: Absolutely, they are, and both are critical.
Russel: Yeah, exactly. The kinds of things that we would change would be I think the thing that people in our business, in the pipeline world, I think the thing that tends to happen is people hear management of change, and they hear big project.
I’m going to make a major change, I’m going to change out my SCADA system, going from Vendor A to Vendor B. Or I’m changing out my communications, I’m moving off of satellite and on to field data radio. Or I’m changing my pipeline system by taking a line offline, or I’m adding a new interconnect and it’s going to change operations.
That’s what we tend to think about when we think about management of change. Would you agree with that?
Will: Yes. I would agree that there’s a vast number of folks that would depict management of change to be that.
Russel: What is it really? [laughs] Right?
Will: Yeah, exactly. It’s everything, I mean honestly. It’s always like looking at different paperclips. They’re all paperclips, they’re just different sizes. Management of change, no matter how big the project, no matter what it is, you really should be, when dealing with a SCADA system with data that helps a control room or even the organization as a whole.
It really should be managed and managed well so that way from start to finish, you have completed the necessary work that everybody was involved, and everybody knows the outcome has been completed, so that way you’ve got the tied bow on the MOC.
Russel: Yeah, you’re basically wrapping it up.
Will: The tied bow.
Russel: If we talk about MOC, and we talk about MOC in the control center, in your opinion, how is that unique or different?
Will: Let’s look at the SCADA side, and we’ll compare that to the control room, because a lot of folks — I wouldn’t say a lot of SCADA people — only see the SCADA piece of it. We’re being required to do all this documentation and do all these steps in order to get our work done.
There’s quite a few people that would probably agree with me in that conversation, but to just say, “Okay, yes, I’ve done all these things and I’ve checked it out, and now I’ve filled out my paperwork, the display is done, and I’ve signed off, and everybody’s good,” that’s only a small fraction of it.
The differentiation between the SCADA and the ops control room would be that the control room is looking at it from how do I see the visibility that I need? How do I make sure that the data that’s informing me and providing me the feedback to maintain my situational awareness of that system, that pipe, that station, whatever it may be, how do I know that it’s the best?
How do I know that there aren’t changes being made without me knowing? How do I understand the AOC, right? The abnormal operating condition. How do I understand really what’s taking place out there, because I don’t have eyes everywhere, I can only make a decision based upon what’s in front of me?
Russel: If I were going to put this Will, in my own words, I would say a couple things. One, it’s how do I understand and believe what I’m looking at?
Will: Exactly.
Russel: Charles Alday and I did a podcast on this a while back, but he talks about Plato’s thing where you’ve got somebody and they’re standing, and they’ve got a fire behind them. They’re looking at a wall, so they’re seeing a reflection of the light on the wall in between them and the fire are dancers.
They see these shadows on the wall dancing, but they don’t really have any idea of what’s really occurring. In a control room it’s similar, I see these shadows on the wall, basically my HMI screens, but it’s not a full picture of what’s going on.
There’s always conversation about what’s needed in the control room versus what’s not needed in the control room to do their job, and yet we expect them to situational awareness which goes beyond just what they’re seeing on their screen.
Kind of tie this back towards what you were saying earlier, from a SCADA perspective is you’ve got the number, but from a control room perspective it is, yes, but can I believe that number and what does that number mean in the context of everything else I’m looking at. That reality, how does that impact MOC?
Will: Oh, heavens.
Russel: [laughs]
Will: I know, right? It’s just so many things that you could go from there. The reality of it is how do you make someone comfortable with the work you’ve done?
Russel: That’s very well said.
Will: I would love to point this back to look at all the healthcare workers right now. They are in PPE, you know the personal protective equipment, that we’re familiar with in our industry but a little different for them. How do they know that that equipment is good to protect them for helping us? It’s that cyclical of exactly where we’re at.
We do work for the control room, and the control room is doing work for the company, and the company is helping us. It’s just that cycle of how do we make sure that someone can trust the work we did every single day. That’s important.
Russel: What I would say is, again, talking about MOC in the control room, there are multiple people that are involved. Typically, there’s going to be controllers, there’s going to be the SCADA group, there might be field automation, there might be communications, there might be the marketing folks if they’re looking at data that’s coming off of SCADA related to scheduling.
They all have their own unique needs and context, and they all need to have confidence that this change is going to allow them to either do their job better or at the worst, continue to do the job the way they’re doing it. That requires a lot of communications, even for something that’s relatively small.
Will: Absolutely.
Russel: When we don’t work a process what we end up with unintended consequences.
Will: That’s right, and I would even mention standardization. Where if you don’t have a standard, you’re also creating the potential for a different outcome.
Russel: Yeah. You’re creating confusion and you’re creating inefficiencies, and there’s always the pressure to get the job done. Without a standard, without an MOC, without a process, there’s a temptation to get the job done, put in place a short cut, and cause difficulties for somebody that’s unintended.
Will: Correct.
Russel: One of the things I think about when I think about MOC in the control room, is that in general the things we’re changing are small. It’s as simple as I’m adding a screen because I’m adding a new site, I’m changing out a piece of equipment at a field site for maintenance or repair, and I’ve got to make sure that all the numbers are correct when I’m done with that.
It could be as simple as a controller found something on a screen and they wanted a label change.
Will: Yes.
Russel: It could be something that simple, that trivial, and yet all of that stuff has to go through a process. In fact, there’s regulatory requirements around all this that you’ve got to keep all the controllers informed.
When I say all the controllers, it’s not just the guy who was on shift when the change was made, it’s everybody that’s going to be controlling that operating area after that change is made, until that change is understood by the entirety of the control room team, or the console team, the people responsible for that operating area which could take weeks given the way a 24/7 shift rotation works.
Will: Correct. Absolutely, that’s right. In some instances, it could take multiple weeks depending upon how large the control room is, how often they sit on that board, and vice versa.
Russel: There’s some unique issues around the communication challenge because of the 24/7 operation, and the nature of shift work and shift schedules.
Will: Yeah.
Russel: Yeah, it adds some real complexity around communications.
Will: It does, indeed.
Russel: What do you think the challenges are? If I’m ACME Pipeline Company, and I’ve got a control center, and I want some kind of system for managing change that’s occurring in my control center, what am I going to come up against?
Will: That’s a good one. I think you’re going to come up against some folks that see it their way. What I mean by that is you’re going to have folks that want to build something. You’re going to try to create even a manual process.
Again, coming back to automating — I spoke about it a little bit earlier — with the fact that any time where you can take the chance for putting responsibility on a human to communicate or fill out the proper information, you can risk missing a step. You can risk quite a few things, and so automation is key.
Russel: I’m going to play the devil’s advocate a little bit here, because I’m going to offer up in this day and age there are a multitude of systems out there for doing task management, work management, and change management. There’s a multitude, there’s a bunch.
Will: Oh, yes. Absolutely.
Russel: Getting to automation is easy, but as you well know, it’s not just about automation, it’s about automating the right thing the right way.
Will: Yeah, and you’re right. It’s making sure that the process that you currently do is set up the same way if you’re going to adopt a solution, a system to do this. One of the biggest things that you’ll find is people go try to build something without a process.
Russel: Oh, yeah.
Will: Then all they do is spin the wheel, and they don’t actually get what they need in order to have a good adoption, or great adoption.
Russel: Right. A lot of times what will happen, too, is it ends up being very myopic. In other words, I’m a gas controller and I’m going to build a change management system, versus I’m a SCADA engineer and I’m going to build a change management system, versus I’m a field technician and I’m going to build a change management system, because all of those guys have different needs.
Will: Yes, and I would say at that point where you’ve got SCADA in the control room coming together to look at their needs and bring that together into one system that allows both parties to work thoughtfully through a shared process so that way the information, the data, all of the documentation, everything is there.
It’s simple to communicate those changes have somebody read them in the control room, sign off on them, and really have that flow. Yes, you could build something, I guess they’re pre-canned out there, but you’re right. It’s taking and making these changes, a rinse and repeat, so that way you get good at it.
The control room says I can trust this work, because I can see where here the change starts, here’s where everything is happening, and here’s the end where I’m looking at it, and I trust that process because here’s all the documentation, here’s what I see, and SCADA feeling like they have some ownership in that change of not owning the change per se, but owning their work.
Work speaks for what you’re doing. The quality of work speaks for the level of trust that’s going to be placed on you. I know quite a few people, a lot of controllers that will always call that favorite SCADA guy and ask, “Hey, can you do this change?” And they’ll make a change.
Again, that comes back to are they doing that, because one, they’re really good at what they do? Or two, are they doing it just because they want something done? That’s coming right back to that little MOC, well if you get a phone call, there needs to be a ticket, it needs to go through the MOC.
It needs to be approved, and everybody needs to understand what it was that was done, because the next time someone looks at it, they’ll go, “Whoa, why is that different?” Then that will mess up an entire process of their operations procedures.
Russel: Yeah, that’s right. That’s exactly right. I want to ask another question. We talked a little bit about automation and what the value of that is, I talked a little bit about the challenge of getting automation right, which means you really need to define what you’re trying to accomplish before you set about trying to automate it.
The other thing I think is important in this conversation, is most companies already have something in place to do work management or to do management of change. Those systems are generally around field maintenance activities, maintaining and supporting equipment in the field, or they’re around large project work.
They’re not generally around small incremental change as a normal course of doing business. Would you agree with that?
Will: Absolutely, depending upon the philosophy of the control room and the SCADA department, I would absolutely agree with that and I would say that most are that way.
Russel: You said the philosophy of the control room, what do you mean by that?
Will: What I mean by that is you’re going to have some control room managers that are going to say, “Oh, well if you make that small little change there, I won’t require an MOC.” You might have — if an IP address is changed on a PLC, say we’re making some changes to the subnet — they don’t want to see an MOC because nothing has changed for them.
Then you have a group where they’re like, “No, I really want to understand that. I want to make sure that when you’re done.” I go in and look and say, “Yeah, that looks normal to me like I would normally see if I was going to trend something,” or it makes sense for the process that is already in place.
There are two different groups and you’re going to see back to that is a big MOC, is it a little MOC?
Russel: Whether it’s big or little may depend on what group you’re in.
Will: Absolutely, because that’s all in the eyes of either A, who’s working on it, or B, who’s getting it.
Russel: Right. The other thing too that’s unique about the control room is one of the things they need is they need a management system to know where things are that they’re working, because we’ve talked about things like, simpler things like making a change on a SCADA screen.
There’s more complex things which are I’ve got this alarm and it’s becoming a bad actor, and I don’t understand why, and I need to do some kind of root cause analysis and run that to ground.
Once I get it run to ground, I need to do some after-action to figure out what are the changes that need to be made in terms of the system, policy, procedures, maintenance, any of that kind of stuff. Same kind of thing with emergencies, or near misses, or those kinds of things as well. Where do I actually need a deep dive?
Will: That’s a great point of what gives you an easy access to see what’s occurring, and I think that also lends into the future of management of change, and how that kind of plays in with work orders for say, a device. How do I know what work is being done and was that through an MOC? How does that affect me?
Absolutely, there’s a vast array of areas that I think are currently not thought about all the time when we’re talking about an MOC, but I think especially with a remote workforce as we are right now, it’s vastly becoming more important for everybody to understand what’s occurring, when, and how.
Russel: Yeah, because I can’t just, even in a small operation, put a white board up and list things on a white board and have everybody know what’s going on when I start working remotely. It’s just a whole dadgum different dynamic.
Will: You walk in, and you put a little K-cup in the machine, and you go, “Oh hey, by the way, got a call we’re going to do this today, just letting you know now.” That does not happen. It’s nice that you caught them and talked to them, but where’s the MOC? Where’s the communication? That kind of thing.
Russel: You’re absolutely right, Will. We are, in our operating company, looking at what’s required for this, trying to design requirements and get clear. This conversation’s actually quite helpful to me personally, and to our company generally.
I’m wondering if you were going to do this, in the state of the art software development, and you know this well given what you do, a lot of people times people ask well what’s the minimum viable solution? What do you think is the minimum viable solution here? What’s the key feature set that’s required to actually add value?
Will: I would say it is being able at any given time to go pull up any work order or MOC that is currently being done on the console or group that I have responsibility for at any given time, anything. That would be, I think, monumental because you almost could have the day start out with an operator, they’re going through their shift change over, they’re doing their document.
They’re signing off on things, they’re making sure they understand any AOCs and pieces that were there, but is there an opportunity for them to go and say, “Oh, here’s scheduled work today.” Or, “Here’s this, I’ve got to be thinking about these things.”
I think that right there is huge at any given time to be able to do that, especially at the start of that shift, because it gives them the opportunity to start thinking in their process, their head, downstream pieces that they need to understand.
They could help keep things running better if there was already planned work and they understand I know that they’re going to do work here, I’m going to try to prepare for that now so that way I’m ready and we keep things moving forward instead of waiting. That would be a huge focus.
Russel: That’s more of a management information system, much more so than a workflow system.
Will: True. I would say that that’s just one piece of it. The workflow piece is still back in the back end.
Russel: I actually like that idea, because I think a lot of the workflow stuff people already have in place. They’re using something like a Maximo, or something, that they put all these kind of tasks in, but they don’t really have a management view for that.
They don’t have a quick, easy way for the controller or the control room manager to open a work order when that needs to occur, open a task, or whatever. If there was a way to open it, and track it, particularly in the control room when it’s done by others. That’s kind of my two cents.
Will: I absolutely agree, because if you think about it, speaking from prior knowledge, PHMSA wants to see work order management, integrated for closed loop AOC. They want to know, “Okay. This was initiated by this person, this is why, here was the work that was done, here’s who completed it, and here it was signed off and checked.”
Russel: Yup. Yes, I think you’re absolutely right.
Will: I definitely see that definitely being the case, because I would say that from a perspective SCADA, that’s just one area where you could go back to that advisory bulletin that came out in, I think, 2003 or something.
Russel: Yeah, the Bellingham Advisory Bulletin, exactly.
Will: Yeah, so then you downshift a little bit into well what’s that equipment out there. It’s not just SCADA, it’s the equipment also. There’s equipment out there that we don’t always show in SCADA, but it’s there, and that could also adversely affect operations.
Russel: That’s one of the things, that starts getting into the conversation about what is a controller’s job, really?
Will: Yeah, oh, yeah.
Russel: Versus what is it given the way the regulators would like to see it done versus what the operating philosophy actually is in the pipeline operator.
Will: PHMSA, document everything, see everything, know everything.
Russel: Right, which is not necessarily realistic, right?
Will: That’s right.
Russel: Either in terms of what’s possible or commercially viable.
Will: Exactly.
Russel: Course the flip side of that is the conversation about we don’t actually control in the control room. We just monitor.
Will: Oh.
Russel: We’re not going to unpack that, but my point…
Will: No, please don’t. [laughs]
Russel: …my point just being that somewhere between those two extremes is where most people are operating.
Will: Yes, absolutely without a shadow of a doubt.
Russel: This is really helpful for me, Will. I’ve done a couple of other episodes on this. Probably the best one was with Nick Guinn back on Episode 43 when I talked about how do you figure out what the right level of MOC’s required. This conversation’s more what are the specific things that are needed in the control room.
I think the point you’re making is really you’d like somebody who’s responsible for an area of operations to know what’s occurred in their area of operations that would impact them.
Will: That’s correct.
Russel: All the folks that are qualified on that area of operation ought to know that. How are you going to do that? Which is primarily a communications thing. There’s some other work order management stuff, too, but if you could just get there, that’d be…
Will: That’s the piece of it depending upon the tool that you currently have. You’re already potentially sending what issues may have happened over the past day.
Then if those reports are going out that also include the work that’s being done to facilitate resolving any issues, from not just a control room perspective, we focus there a lot, but from an operation respective, the whole company wins in that situation. Ultimately, that’s what we want.
We don’t want to build a multitude of things. We really want to have something that’s functional for the control room but can also provide extreme value for the rest of the company in how they operate and interface with that control room.
Russel: I absolutely agree with that. It’s really more about communications than it is about tools. It’d be very hard, because there’s so many tools out there that do this kind of thing, it’d be very hard to…Anyways, it’s an interesting challenge for sure.
Will: It is. That’s the nice thing is there are those multitude of solutions and how do you take and roll those up. Luckily, it’s only a technology issue, not necessarily a, how do I say it, it’s not necessarily a domain issue.
Russel: Right. Yeah, exactly. Hey, listen, man, thanks for coming back on the podcast. Great to have you. As always, I learn as much as, I’ll say it this way, I learn a lot every time we talk because it certainly gets my creative juices going. I very much appreciate you being a perennial guest.
Will: Yeah. Thanks. No worries. It’s always great to be on. I know we have heard a lot of people, especially even in the past couple weeks with everything getting a little crazy, that they’re starting to listen more.
I’m glad that this is here for us as an industry. I really do think that there’s just an amazing amount of content still just barely even being tapped.
Russel: Oh, boy. This is episode 120 something I think as we’re recording it. I would have thought that by now I would have covered the — I haven’t even covered all the domains yet — much less the detail around all the domains. It’s crazy. It’s absolutely crazy.
Will: It is.
Russel: We’ll keep doing it. Another 20, 30 years, I might have it reasonably addressed.
[laughter]
Will: We can only hope.
Russel: There you go. All right, Will. Take care, man.
Will: Yeah, thank you.
Russel: I hope you enjoyed this week’s episode of the Pipeliners Podcast and our conversation with Will Gage. Just a reminder before you go, you should register to win our customized Pipeliners Podcast Yeti tumbler. Simply visit pipelinepodcastnetwork.com/win to enter yourself in the drawing.
If you would like to support this podcast, please leave us a review. You can do that on your smart device using Apple Podcast, Google Play, or whatever device app you use. You can find instructions at pipelinepodcastnetwork.com.
[background music]
Russel: If you have ideas, questions, or topics you’d be interested in, please let me know either on the Contact Us page at pipelinepodcastnetwork.com or reach out to me on LinkedIn. Thanks for listening. I’ll talk to you next week.
Transcription by CastingWords