The Pipeliners Podcast welcomes back Jeremy Coleman of Northwest Natural to discuss the challenges of Alarm Management, dating back to the original conceptualization getting the process going.

Jeremy and host Russel Treat walk through the timeline of advancing from the concept of Alarm Management to achieving compliance to optimizing Alarm Management for pipeline operations excellence.

Included in the discussion are the real-time tools used by controllers in the control room and the post-analytic tools used by management to understand how to modify alarms. Listen for a full discussion of the value provided to each stakeholder in pipeline operations.

Plus, listen all the way through for a special challenge from Russel and Jeremy to add to the conversation!

Challenge of Alarm Management: Show Notes, Links, and Insider Terms

• Jeremy Coleman is the Gas Control Manager at Northwest Natural in Oregon. Connect with Jeremy on LinkedIn.
  • Listen to Jeremy’s first Pipeliners Podcast appearance in Episode #5 discussing High-Performance HMI.
• Alarm Management is the process of managing the alarming system in a pipeline operation by documenting the alarm rationalization process, assisting controller alarm response, and generating alarm reports that comply with the CRM Rule for control room management.
  • Alarm Rationalization is a component of the Alarm Management process of analyzing configured alarms to determine causes and consequences so that alarm priorities can be determined to adhere to API 1167. Additionally, this information is documented and made available to the controller to improve responses to uncommon alarm conditions.
    • An Alarm Response Sheet indicates to a pipeline controller what the alarm is, what could cause the alarm, how to verify the cause, and what actions should be taken given the cause.
  • Asset Management is understanding the condition of pipelines in the field and taking appropriate actions to maintain the integrity of the pipelines.
• The CRM Rule (Control Room Management Rule as defined by 49 CFR Parts 192 and 195) introduced by PHMSA provides regulations and guidelines for control room managers to safely operate a pipeline. PHMSA’s pipeline safety regulations prescribe safety requirements for controllers, control rooms, and SCADA systems used to remotely monitor and control pipeline operations.
• Gas Certification Institute (GCI) teaches the fundamentals of measurement in-person in Houston, Texas. One of the certification courses is the Fundamentals of SCADA.
• SCADA (Supervisory Control and Data Acquisition) is a system of software and technology that allows pipeliners to control processes locally or at remote location. SCADA breaks down into two key functions: supervisory control and data acquisition. Included is managing the field, communication, and control room technology components that send and receive valuable data, allowing users to respond to the data.
• MAOP (Maximum Allowable Operating Pressure) was included in a bulletin issued by PHSMA informing owners and operators of gas transmission pipelines that if the pipeline pressure exceeds MAOP plus the build-up allowed for operation of pressure-limiting or control devices, the owner or operator must report the exceedance to PHMSA on or before the fifth day following the date on which the exceedance occurs. If the pipeline is subject to the regulatory authority of one of PHMSA’s State Pipeline Safety Partners, the exceedance must also be reported to the applicable state agency.
• Telemetry is an automated communications process. During this process, measurements and other data are collected at remote locations and transmitted to receiving equipment for monitoring and data analysis.
• Fatigue Management is the pipeline control room process of managing controller workload, adhering to PHMSA guidelines for Hours of Service, and being able to prove the necessity for any deviations from your internal Control Room Management Plan (CRMP).
• HMI (Human Machine Interface) is the user interface that connects an operator to the controller in pipeline operations. High-performance HMI is the next level of taking available data and presenting it as information that is helpful to the controller to understand the present and future activity in the pipeline.
• EnerSys Corp. (the company owned by Russel Treat) offers the Control Room Management (CRM) Suite of software products to support Alarm Management and other critical pipeline control room functions.
• GIS (Geographic Information System) is a method of capturing the earth’s geographical profile to produce maps, capture data, and analyze geographical shifts that occur over time.

Challenge of Alarm Management: Full Episode Transcript

Russel Treat:  Welcome to the Pipeliners Podcast, episode 51.

[background music]

Announcer:  The Pipeliners Podcast, where professionals, Bubba geeks, and industry insiders share their knowledge and experience about technology, projects, and pipeline operations. Now, your host, Russel Treat.

Russel:  Thanks for listening to the Pipeliners Podcast. I appreciate you taking the time. To show that appreciation, we’re giving away a customized YETI tumbler to one listener each episode. This week, our winner is Andrew McVey with ONEOK. Congratulations, Andrew, your YETI is on its way.

To learn how you can win this signature prize pack, stick around to the end of the episode. This week on the Pipeliners Podcast, Jeremy Coleman with Northwest Natural comes back to join us. You may recall, you long-time listeners, that about a year ago, Jeremy came on to talk to us about his experience with high-performance HMI.

Now we’re bringing him back to talk about his experience with Alarm Management. Jeremy, welcome back to the Pipeliners Podcast.

Jeremy Coleman:  Thanks, Russel.

Russel:  You realize it’s been almost a year since you were on here last time.

Jeremy:  I do not. It really feels like it’s only been a few months, but time flies.

Russel:  [laughs] No kidding. No kidding. I asked you on today. I wanted to talk about the challenge of Alarm Management. In particular, I wanted to ask you to come on because we did a project together sometime back, really right after the Control Room Management Rule got written.

I thought it might be interesting for listeners to hear from some folks that have worked through this and get some of our thoughts. Maybe a good way to start is a little bit of a review of what we did back in the 2011-2012 time frame around Alarm Management. Do you recall what that process was like and what we did?

Jeremy:  Very much so. I remember being thrown into this whole control room management and trying to get my brain wrapped around all of it. A big part of it, obviously, was alarm management and just trying to figure out what was being asked for, what the value of that was going to be to us, and trying to just say, “Okay, what is it that we need to do here?”

In fact, I recall we left Alarm Management ‘til towards the end of our creation of our program, just to really, fully understand what we were after. I think it wasn’t until after I met you and then we talked about a few things. That’s when we started off on our journey together for the Alarm Management.

Russel:  What do you think was the most challenging part of getting an alarm management process going?

Jeremy:  Once again, understanding what was being asked for and not just putting something out there that would check the boxes in an audit but something that would actually help the control room in their day-to-day, that was probably the hardest part.

Then, once we got through that very quickly, the second-hardest part was sitting down with the right people and finding the time to actually go through and start the Alarm Rationalization process. That’s where you guys came in and really helped us out.

Russel:  I think that’s exactly right. I’ll share a story with you. I don’t even know if I’ve told you this, but one of the things we do at our affiliate Gas Certification is we teach the Fundamentals of SCADA.

When we first put that class together, we asked a gentleman by the name of Doug Rothenberg, who’s an Alarm Management guru, to come in and teach the Alarm Management piece. I went to his class three times before it finally snapped and I went, “Oh, that’s what Alarm Management is.” I fundamentally think that’s our biggest challenge, just as an industry. This idea about Alarm Management is really something fairly radically different than what most people think of when you talk about alarming.

Jeremy:  You are absolutely right. I love that you brought this up. I skipped over something when you asked me about this in the beginning. One of the first things that we did to try to get our brains around Control Room Management was there was a pretty easy line item, which was get an Alarm Philosophy put together.

We had Mr. Rothenberg come in and help us with that. My mind was so blown by the time he left that I think we shelved the [laughs] Alarm Management process and waited until the end. I say that laughing now because he knows so much about it. I couldn’t even keep up with him at the time. It took a while just to figure out where we were at.

Russel:  I would absolutely agree with that. Doug is a brilliant guy. In fact, I need to reach out to Doug and see if we can get him to come on and talk about this subject. I think that’d be awesome.

Jeremy:  Absolutely.

Russel:  I think really the biggest thing that I was hung up with is, I had these preconceived notions about what alarming is. Even though Doug was walking through, “This is how we define an alarm,” and all that type of thing, I just was not getting it. I’ll be very honest. I was not getting it.

It took me a while, and then when I finally got it, it was like a big, “Aha!” It’s one of those things like when you get slapped up the head with an idea, and all of a sudden it makes sense, and I’m like, “Oh, okay.”

I think you’re exactly right. I think the second biggest challenge is getting the right people in a room, for the right amount of time, with the right focus, to actually do something meaningful around rationalization.

Jeremy:  Correct.

Russel:  How do you define Alarm Rationalization?

Jeremy:  Oh boy. How do you define it?

[laughter]

Russel:  Oh boy. [laughs]

Jeremy:  I think I’m going to really simplify this, and if Mr. Rothenberg ever listens to this, he’s probably going to roll his eyes. First and foremost, I really take it to heart that alarms require action.

The first thing we had to do was really look at our operations and try to figure out what should be alarmed and what shouldn’t.

When we have worked on that, we set up the process of how we’re going to rationalize, and you guys helped us quite a bit with that, but then really it was trusting that process and sticking to that process all the way through to the end. I don’t know if I’m answering that question that well.

Russel:  I think you’re right on it. You said an alarm is something that requires action. I actually add a little bit more to that. I would say that an alarm is something that requires actions, or something bad will happen in a time frame, right?

Jeremy:  Yeah.

Russel:  That means if I don’t take action, then we’re going to have a bad outcome, and the minute I get the alarm, that’s telling me the clock’s ticking. That’s a different way of thinking about alarm.

In the industry, we tend to use this word alarm very generically to mean a whole lot of different things. That, I think, is the first thing.

The second thing is this idea, and I got this from Doug, it’s part of his teaching about permission to operate. Permission to operate, easily understood just meaning that I have the permission to operate as long as I understand what’s going on.

Alarming, in particular, alarm rationalization, is a mechanism to support an operator or a controller’s permission to operate.

Jeremy:  Correct.

Russel:  Basically, what you’re trying to do is give him a checklist. The biggest thing for me that trips me up in a rationalization, even to this day, is you want to start with, “What are the abnormal operating conditions?” not with, “What are the alarms?”

That trips me up. I don’t know about you, but maybe you could talk to that a little bit as well.

Jeremy:  There’s lots of discussion in the industry, especially back when we were going through all this about alerts. Some people like to use alerts and some people don’t, and what is an alert versus an alarm, and all that.

In trying to define these alarms and trying to figure out what the abnormal operating conditions would be, we really came to realize that alerts would be very valuable for us. Like up here in the winter time when we have high flows, we are going to have lower pressures on our system. We know that.

Previously, we would have a low alarm and a low, low alarm. We would basically ignore the low alarm in the winter time because we know that’s where it’s going to be.

Conversely, in the summer time, we have much higher pressures when demand is down, so we would have high alarms and high, high alarms. We’d ignore the high alarms because that’s what it’s going to do in the summer time.

In thinking about our abnormal operating conditions and moving away from that a little bit, we realize that we actually have these extended normal operating areas when it is cold out, which would be that previously low alarm, and when it’s hot out, which would be that previously high alarm.

We actually called those “alerts,” just to call it out, say, “Hey, just to see the direction you’re headed.”

It might seem like a small change, but it actually was a pretty fundamental change for us to think about these extended operating areas, and then to go on from that, what are our abnormal operating conditions from there?

In a gas system, if you have too much gas in the pipeline, we have reliefs that are going to take care of that. [laughs] What’s funny is, as an operator, as a controller, I want higher pressure in my gas line.

I certainly don’t want to go over MAOP, but having a little bit higher pressure is usually not a bad thing.

Russel:  Exactly.

Jeremy:  Talking about that and understanding, I am more afraid of very low pressures in my main, of losing customers, especially at certain times of year.

Russel:  Jeremy, we should probably clarify for the listeners that you’re operating a pipeline control center for a gas utility.

Jeremy:  That’s correct.

Russel:  That helps with the context you’re talking about because in a gas utility, the primary mission — second only to safet — is deliverability. Particularly in the winter, that’s about keeping people’s heaters on, so it’s an important thing. High pressure gives you the ability to keep the heaters on.

Jeremy:  Absolutely.

Russel:  I think you make a really interesting point, too, about this idea of — I’m going say it a little differently — but a seasonal, normal operating range?

Jeremy:  Yes.

Russel:  That being something that can evolve versus something that is very clearly a safety. I think that’s the other part of the complexity in the rule is, they talk about safety-related alarms, which requires you to define what safety related is.

Jeremy:  Correct. It does.

Russel:  Of course, there’s some definition in the rule about that, but there’s certainly a lot of leeway for the operators to do what makes sense.

Jeremy:  Absolutely, there is. Safety-related, we really went round and round on that, trying to define it early on and staying up on it.

Russel:  Exactly. The idea of what is safety-related, and should I rationalize…We said an alarm is, I must take action or something bad happens in a time frame. That bad thing might not be safety related. It might be purely commercial.

Jeremy:  True.

Russel:  I still want to rationalize those things, and I still want to treat them like a safety-related alarm. I think that’s one of the other things that, again, I just think the industry is having some difficulty really getting its brain around, what’s the absolute best way to do this?

Jeremy:  Yeah. I feel like just using the term safety-related kind of threw everybody into a tizzy, and made things a little bit harder to understand. I know I’ve taken — and we’ve rationalized quite a few alarms that have nothing to do with safety, actually. As you said, they’re commercial. They’re financial.

I don’t like to particularly alarm flows, but I do have some flows that I’ve alarmed just because we have the capabilities of over-spinning meters, causing equipment damage. We want to make sure that we can see these happen, or the potential for them before they actually do happen.

I have a couple of customers that I alarm just because they can impact my system. Of course, I’m going to alarm them, but there’s no safety-related reason for that, and then some other odds and ends that we alarm just because we want action to be taken even though it’s not safety-related.

Russel:  I think that’s a very important point because now you’re starting to talk about what’s the value of an Alarm Management program, versus purely the compliance need of an Alarm Management program.

Jeremy:  Yeah, the care and feeding of a proper Alarm Management program, and I can spend a lot of time talking about that.

Russel:  Actually, that’s exactly where I wanted to go in this conversation because I think the things we’ve been talking about are, I don’t know if the solutions are universally understood or agreed to.

I think the industry, particularly the utility industry, is really working on this, but the idea of care and feeding of an Alarm Management program, what does that take, in your opinion?

Jeremy:  In my opinion, it’s harder than setting it up, initially. Setting it up, one was obviously coming to understanding of what it all means, bringing in help, bringing in Mr. Rothenberg, bringing you in, having to set it up. It was something that we were able to accommodate time-wise.

Financially, we got this thing set up and then everybody goes away, and you guys aren’t here anymore. We are here working in the day-to-day. Functionally, our system is slowly changing over time, so we need to be aware that if set points need to change for an alarm, that there is a process that we have to follow.

We have the care and feeding of the application that helps us rationalize, that maintains all that information, and that provides alarm reports for us. There is the getting ahead of, let’s say, a new site that’s going to have new SCADA taggers coming in. They’ll all need to be rationalized prior to being put into the SCADA system and being implemented in operations.

That’s probably the hardest part, is getting ahead of those kinds of things.

Russel:  It’s not just the site you’re adding. It’s the impact to the system that the site you’re adding is going to have.

Jeremy:  Exactly.

Russel:  Understanding where are the boundaries of that in all the things you got to touch.

Jeremy:  That, frankly, is probably one of the hardest things is getting ahead of that, finding the time, finding the right people to get ahead of that, and have it all done. Our focus, especially on a new site, obviously is getting the pipe in the ground, getting it functional, having a good design, getting it installed.

In the past, installing the telemetry to get that data back into the control room has really been a secondary thing, so we’ve pushed really hard to get that into the initial installation process.

We’ve done a good job of that, but there’s the backside of that, which is, now all that data is coming in, and what are you going to do with it? It’s still really hard to get ahead of that.

Russel:  I want to talk a little bit about the tools. You’ve talked about, obviously there are some clear requirements in the Control Room Management Rule. I’ve got to do a monthly analysis of my bad actors, and be taking actions to improve things, and then I’ve got to do an annual review of my program.

Part of that is supported by alarm management tools, and there’s a lot of vendors that do that. I think one of the things that we found is that a lot of these tools really weren’t designed for pipeline. They were designed for process systems, refineries, petrochemical, all that kind of stuff.

Alarming, and Alarm Management, and those kind of facilities is actually a quite different thing than alarming and Alarm Management in pipeline facilities.

Jeremy:  I would imagine so. I have a real hard time with the requirements in the rule, how they’re written, and what the reports are that they call for. When I look at those reports, those are much more administrative in my mind. They’re not helping directly with the day-to-day, with what the controllers are using.

What I mean by that is, the controller that’s operating the system today isn’t running those reports. It is somebody else that’s running them, and looking back, and making those changes that, slowly over time, are going to have a big impact, but they’re not really helping with the day-to-day operations while you’re in the moment. Those are different kinds of reports that aren’t called out for.

You brought up technology. Using applications, as you said, not really designed for what we need, makes this incredibly hard.

Russel:  It adds administrative overhead.

Jeremy:  Absolutely.

Russel:  It adds technical and administrative overhead, and there’s two parts of that. One is, whatever tool you create or procure has to be kept current.

The other part of that is, this is an analytical task. It is not an operating task. Historically, in SCADA organizations, we don’t see analytical tasks. We see project tasks and administrative tasks, but we don’t see analytical tasks. Where does that land? Where in the organization does that land? Where do you find that kind of expertise?

Jeremy:  From my standpoint, it stays in gas control. It’s a corporate problem that remains in gas control, so I’m farming controllers out to be the analysts to run these reports to, one, keep us in compliance, but two, try to move us forward as well.

That becomes problematic for many reasons, one of which is controllers, they already have a job. They are controllers.

You have to be mindful of Fatigue Management, and then all the other requirements of Control Room Management, as well as recognizing that I’m not going to consistently have the same one or two people reviewing these reports, going through the same motions.

It’s going to be changed and moved around amongst controllers, and that becomes problematic as well.

Russel:  I agree with all of that. I think there’s one other aspect of it that took me a while to realize. It’s one thing to run these reports. It’s another thing to analyze them. It’s quite a different thing to really understand what are the appropriate actions to improve the effectiveness of the alarm program.

That could be things like communications, field automation, instrumentation, alarm setting. It could be getting rid of some alarms. It could be doing some programmatic alarms, meaning, saying, “If this happens, and this happens, and it happens in this sequence, then this is the alarm you generate.” That kind of thing.

That’s actually a pretty broad skill set to do that.

Jeremy:  Absolutely.

Russel:  It doesn’t really exist in the control room.

Jeremy:  It doesn’t exist wholly, but…

Russel:  Exactly. Wholly is a very important distinction to make.

Jeremy:  Yeah, because the controllers understand the alarm process. They understand the hows and whys. They typically understand, if not a hundred percent of how to fix it, they know a fair amount of how to fix it.

We still don’t have that, and I don’t want to say buy-in, because that’s not the right term. We still don’t have the assistance from other departments that could come in and really complete this because everybody is so busy.

One of the joys of being a public utility is, everybody is wearing six hats and it’s really hard to carve out the appropriate amount of time with the appropriate amount of people consistently to stay on top of this.

Russel:  I think that’s so true. Of course, we work not just with public utilities. We work with midstream companies and other kinds of large-scale pipelines, but we see this very consistently.

It seems like the Alarm Management — program management — is probably one of the most challenging aspects of the rule that to have well-implemented — when I say “well-implemented, everybody is doing something,” but really getting to the point where getting beyond doing something that’s a compliance activity and we’re actually adding value and improving the way that the system is running, that the program is providing value to the overall operation.

That’s a challenging place to get to. It’s because of the complexity of understanding what this is, and then the number of organizations that are possibly impacted by what needs to occur.

Jeremy, if you were king of the world, how would you fix this problem?

Jeremy:  How would I fix this problem? There are several fixes. It’s not just Alarm Management. It’s the whole Control Room Management, but I’ll stick to [Alarm Management}. Treat it like the program that it is. It’s not just a one-time set-up and walk away.

Have oversight. Have the proper care and feeding that it needs, so, humans. You got to have consistent humans. Involve the right ones.

Secondly, we desperately need technology changes. I believe that what we’re using today is not enough. We need more. We need something different. We need to come together as an industry and demand change, and we’re not doing that. I’m starting to climb up on my soapbox right now, but…

Russel:  I want to unpack that a little bit. Obviously, I’m a technologist and I’m very interested in that. I tend to agree with you, and you know this story.

I don’t know that the listeners will, but when we worked with you, we used a third-party tool to do the Alarm Management, and then we worked with a number of customers with that third-party tool, and we came to a conclusion that in order to do something for pipelining, we needed to build something specific for pipelining.

Just to talk about a couple of things that will clarify this, you really need very good copy and paste capability, and that sounds easy, but it’s not. The nature of pipelining is, we have a lot of things that are replicated. A regulator station, and a gas utility is a regulator station, and they’re going to be 85, 95 percent the same everywhere you have a regulator station.

That means if I add a new one, I ought to be able to take another regulator station and copy from and paste to the new one. When I do that, I get all the tag names and all the rationalization and everything, all the alarm settings, at a starting point where I can review it. I don’t have to build it.

That’s an example — and you don’t find those kind of tools typically in Alarm Management tools; we’ve built some of that stuff into what we do — but I’m interested what other things you might see like that, that are capabilities that are missing?

Jeremy:  For the first part is, for instance, my SCADA system has alarm functionality to it, and zero alarm reporting capabilities. As we were unpacking the Alarm Management part of CRM and realizing that we needed these reports, the first part was, how do we get these reports?

That’s why we ended up coming to you and getting this third-party application to take care of this, so that’s one minor aggravation is that I don’t have those reports in my system.

The second aggravation to go onto that and to kind of tack onto what I was saying earlier is, those heat of the moment — I’m going to call them an operational report — is when I am seeing, even maybe before I get my alarm.

Of course, now I’m dropping back into our HMI discussion from a year ago. It’s let’s start running some quick reports on that one tag. When was the last time it was an alarm? How many times have been alarm in the last six months? What are some associated tags that I can look at to help me try to diagnose what might be becoming a problem?

That doesn’t exist in my SCADA system. If I want that, I have to start customizing, which we all know is bad. It’s just not there, and my provider is not giving that to us, so where am I going to get that?

Russel:  That is such an excellent question. When we look at this particular challenge, our product, which we call alarm managers…part of what we call CRM Suite, and it’s designed to work with any SCADA system. The challenge every single time is, not every SCADA system does this stuff the same way.

Jeremy:  Correct.

Russel:  The other challenge is, some of the functionality needs to be in the SCADA system — has to be. The alarm, the activation, the acknowledgment, the clearing — all of that has to happen in the SCADA system, and so then you start getting this conversation about, what is the real-time aspect of effective Alarm Management versus what SCADA system ought to be doing?

It really adds some complexity to this whole conversation. For a vendor like ourselves, it’s more complex because every operator in every SCADA system is a little different.

I wonder. Boy, you’re causing my wheels to turn. I’m making notes, man. That’s good stuff.

Jeremy:  That’s really good.

Russel:  [laughs] You do that to me all the time, Jeremy.

Jeremy:  I know.

Russel:  This is not anything new.

Jeremy:  I haven’t even started. I’m trying to limit myself here.

Russel:  I get what you’re getting at. I’ll tell you one of the things that I think would be helpful. One of the things we do is, all these alarm…We call the alarm response sheets. When you’re done with rationalization, that’s the output, I get an alarm response sheet.

It tells me what the alarm is, what could cause it, how do I verify the cause, what actions do I take given the cause, that kind of thing. That’s standard rationalization. All our measurement tools do that.

One of the things we do is, we keep it in a database and we’ll hook it up to the tag in the SCADA system. In some platforms, you can actually pull that up to the operator when that alarm occurs, which is really resourceful.

To me, you really need to go another level, particularly when you start talking about things that require multiple people to do the alarm response, or where I want to turn that alarm response into an alarm response procedure and I want to track how people are working the procedure.

Certainly, in leak alarm response, this is a big deal where I’ve got a controller, and maybe a leak analyst, and maybe a supervisor that are collaborating to do the alarm response.

You start thinking about, “Well, okay, but that’s not a SCADA functionality, at least not historically,” Where does that live, and how does that tie to SCADA? What part of that is SCADA, and what part of that is the alarm, too? It’s a challenging question.

Jeremy:  It really is, and I’ve been working hard on this. You said so many things right there. My mind was just spinning. The Alarm Response Sheets are incredibly important, and we have those.

We created those with your help, but one of the problem is — I’m going to put my HMI hat on — is, how do you get that information in a usable format in front of the controller or operator, right when they need it?

I’ve said this before. As a controller, I want the information, I want how I want it, when I want it, where I want it, right?

Russel:  Exactly.

Jeremy:  It’s got to be right here, and incredibly usable.

Russel:  Only at that moment.

Jeremy:  Only at that moment. It might be just something I need to take a quick look at and say, “Okay, I don’t need you anymore; send it away,” and go on to the next thing.

You dovetailed into a couple of other things, which is, I might need access not only to this alarm reporting structure that I have, but if it’s a regulator, cut and paste, that we’ve put in, when was the last time that regulator was serviced? Now, I’m getting into a different kind of application: Asset Management.

Russel:  Alarm Management and Asset Management, when done well, they’re first cousins.

Jeremy:  Absolutely, but they sure as heck don’t live in the same house often, I would guess.

[laughter]

Russel:  That is really good. I like it. You’re right. You’re exactly right. It’s two different departments in a company.

Jeremy:  Right, but to jump on this, now you’re starting workflows. Now, I recognize a problem, right?

Russel:  Mm-hmm.

Jeremy:  Now I want to get other people involved, outside of my direct group. I want to involve engineering, I want to involve in tech, maybe I need to involve some telecommunications people. How do I start that workflow?

SCADA is not made for this kind of thing. I’ve said this to you many times — you’re going to recognize it — in the past, at my work, we think of the SCADA system as the toolbox, and what I’ve come to realize is that the SCADA system isn’t the toolbox.

The SCADA system is a very important tool. It’s the socket set. It’s the wrenches. It’s incredibly important, but it’s not actually the toolbox itself.

We actually need to create the toolbox where I can get these workflows involved, where I can get the ability to run reports on-demand, where I can go look at all of these other applications that are completely disparate to the SCADA system now.

I need access to those, and I know security people right now are probably rolling their eyes, “Well, how are we going to do this?” The fact is, if we want to improve and move from the position that we’ve been in for a very long time, this is the direction we need to go. This is where the industry needs to go.

Russel:  Absolutely. I think it’s very well said. It’s a point I make is that, SCADA is a tool. It is not a toolbox, and when you start trying to do things using SCADA that SCADA was not intended to do…I’ll use a few examples and this might ruffle some feathers.

GIS is a mapping tool. SCADA is not a mapping tool. In fact, you don’t really even want maps in — there’s a number of valid reasons why you don’t want maps in SCADA.

You need things like subway maps that give you an idea of where things are in relationship to one another and how they interrelate. That’s not like a geographic representation of a map, but there are times when I’m in the control room, I need an actual map.

I want to ask this question. You’ve talked about the tools that are the real-time tools that support the controller doing their job. I think there’s also some management tools that support the after-analysis process, tracking that, “Hey, we want to make this modification to an alarm, and is that getting done?”

I would call that the kind of the post-analytic workflow, and that workflow is typically going to cut across — could potentially cut across — multiple organizations within a company. Which do you think would provide the most value, the real-time workflow or the post-analytic workflow?

Jeremy:  I believe right now the most value is the heat of the moment workflow, because I believe that we really…

Russel:  [sighs]

Jeremy:  I know. I’m sorry. I broke your heart.

Russel:  I was hoping you’d give me the other answer. [laughs]

Jeremy:  To be fair, I think they need to come together. I think they are opposite sides of the same coin. You can’t have one without the other.

You certainly need that oversight, but if it’s not getting done in real time, the oversight isn’t going to help as much, and it might just be I’m approaching this problem differently than you. I don’t think there’s a right answer.

Russel:  No. Actually, I’ll break it down why I have the response I have.

Jeremy:  Sure.

Russel:  The after-analytic workflow, from a pure software development perspective, is easier to get your mind around because that’s really a management system.

I have something I need to get done. I need to open a work order, or I need to send a communication to somebody and interact with the things that they use to do their job, and then I need to follow up and make sure those loops are closed. That’s fairly straightforward. The difficulty is in the details, but that’s fairly straightforward.

When you start talking about the real-time aspect of it, that starts getting — to my mind — it’s just the problem is more complex, because I don’t know where the SCADA system ends and the Alarm Management system begins.

From an analytical standpoint, that’s pretty simple. You take the alarm activations, the alarm events, and you feed that to a tool, and you process that data and build all the reports you need. That’s, to me, fairly straightforward.

The, “what do I need to do in real-time when I get the alarm?” and the kind of things that the controller would be asking, oh man, that starts getting a lot more complex.

Jeremy:  It does.

Russel:  That part is historically done in the SCADA system. That’s why I go, “Oh my gosh. Why did you give me that answer, Jeremy?” On the other hand, to be fair, if I were in your job, I think I’d have the same answer because that’s where the real value for me is, right?

Jeremy:  Right.

Russel:  I want to be more responsive, more accurate, more effective, more timely, better data at my fingertips, well-organized — this is always going to be helpful. I don’t know easily how you get to that.

Jeremy:  I don’t, either. I don’t have an answer. You said it. To shed a little more light on my position as a control room manager, I have plenty of things that I’m not getting into. There’s not enough hours in the day, so every improvement I can make right at the ground level, right at my controllers’ fingertips, is just money in the bank for me.

That’s where I want improvements to be made, and we will get to the things that would help me later, if that makes sense.

Russel:  Makes perfect sense. Makes absolutely perfect sense. I think that’s right on target.

I often try to do three key takeaways. I’m going to try to do three key takeaways off of this.

My first takeaway is that when the Control Room Management Rule came out, everybody was learning, and everybody did something. I don’t know we talked about this, but I would say, to some degree, everybody did something a little different.

Jeremy:  I think so.

Russel:  I think the second takeaway is we’re just now getting to the point as an industry that we’re beginning to understand the problem.

Jeremy:  Yeah. As an industry, I would say we’re not understanding it yet, but more and more people are beginning to, I believe.

Russel:  I think we’re just beginning to, as an industry, understand it. I’m not saying we do understand it. We’re just kind of beginning to.

Jeremy:  Sneaking up on it.

Russel:  Yeah, and I think the other thing that this conversation pointed out, I had a thought about this before, but there’s really two aspects to this. One is the real-time pipeline controller aspect of what is needed for them to be effective, versus the analytic and system management perspective, which is something a little different.

Those are two very unique perspectives, and they’re probably different technical solutions, so yeah, interesting.

I want to make a shout-out to the listeners and challenge the listeners, for those that might have an interest in this subject or might have something they think could be added to the conversation. I’m going to do a shout-out to the listeners and say this: please reach out and contact me.

I’ll give you the details there at the end in a little bit, and maybe either Jeremy and I or someone else, we could add to the conversation. We could kick this ball around a little bit more, because I think there’s a lot of meat on this bone, if you will.

Jeremy:  There absolutely is nowhere to go but up from here, I believe. Looking forward to getting this going.

Russel:  I should say, too, that congratulations because I think you’re right around our first anniversary guest.

Jeremy:  Hey, all right. Big prize, right?

Russel:  Exactly. I’ll could probably send you a cool YETI.

Jeremy:  I would accept that, absolutely. Put your face on the side, that would be fantastic.

Russel:  It has the Pipeliners Podcast logo. All you have to do is go to pipelinepodcastnetwork.com/win, and enter yourself to win.

Jeremy:  All right.

[laughter]

Jeremy:  I love it.

Russel:  Jeremy, this has been fun. It’s a pleasure, as always, and we will definitely have you back.

Jeremy:  Thank you, sir. I really appreciate it.

Russel:  I hope you enjoyed this week’s episode of the Pipeliners Podcast, and our conversation with Jeremy Coleman about Alarm Management. I certainly got a big kick out of it, and certainly learned some things, and got some good notes about where we might want to go with our businesses in the future, so that was awesome. I hope you enjoyed it as well.

Just a reminder before you go, you should register to win our customized Pipeliners Podcast YETI tumbler. Simply visit pipelinepodcastnetwork.com/win to enter yourself in this drawing.

If you’d like to support this podcast, please leave us a review on Apple Podcast, Google Play, Stitcher, or whatever smart device podcast app you use. You can find instructions at pipelinepodcastnetwork.com.

[background music]

Russel:  Finally, if you’ve got ideas, questions or topics you’d be interested in, please let us know on the Contact Us page at pipelinepodcastnetwork.com, or reach out to me on LinkedIn. Thanks again for listening. I’ll talk to you next week.

Transcription by CastingWords