In this episode of the Pipeliners Podcast, host Russel Treat welcomes back Giancarlo Milano of Atmos International to discuss the critical topic of simulators for pipeline training.

The discussion focuses on three types of trainers typically used to support pipeline operations, the methods used to build up competency and measure the competency of a controller or operator, the importance of being able to interact with the SCADA and HMI, and more valuable training-related topics.

Simulators for Pipeline Training: Show Notes, Links, and Insider Terms

• Giancarlo Milano is the Senior Simulation Support Engineer at Atmos International. Connect with Giancarlo on LinkedIn.
  • Listen to Giancarlo’s five-part series on leak detection previously on the Pipeliners Podcast.
  • As part of our ongoing series with Giancarlo, enter to win our book giveaway contest for the “Introduction to Pipeline Leak Detection” by Atmos founders Michael Twomey and Jun Zhang.
• The Real-Time Transient Model (RTTM) for leak detection simulates the behavior of a pipeline using computational algorithms. The model, which is driven by the field instrumentation, monitors discrepancy between the measured and calculated values potential caused by a leak. RTTM uses flow, pressure, temperature, and density among many other variables.
• The types of trainers typically used to simulate pipeline operations include generic trainers, hybrid trainers, and high-fidelity trainers.
  • A generic trainer is an off-the-shelf solution that provides an accurate experience of operating a pipeline with elevation changes, multiple supplies, and deliveries.
  • A hybrid trainer merges different parts and equipment from several pipelines of a company into one pipeline, allowing the trainee to experience the operation and challenges of different pipelines, without the high cost of building and maintaining individual training systems for each pipeline.
  • A high-fidelity trainer replicates a real-time system, allowing the trainee to operate an exact repetition of a real pipeline.
• HMI (Human Machine Interface) is the user interface that connects an operator to the controller in pipeline operations.
• A pipeline system includes various components for the safe transport and delivery of the product in a pipeline.
  • An injecting station is the beginning of the system, where the product being transported is injected into the line.
  • Intermediate block valves provide protection along the pipeline through the ability to block flow in the line for a leak, rupture, or maintenance.
  • An intermediate delivery station or partial delivery station allows the pipeline operator to deliver a portion of the product in the line.
  • A final delivery station is the location where the product is distributed to the end user.
  • The PLC (Programmable Logic Controller) is a computerized system that automates processes that require reliability within a given time period. PLCs are especially useful for pipeliners to automate difficult tasks in the field.
  • DCS (Distributed Control Systems) is a method of controlling processes within the operating location, utilizing a central operator for supervisory control. This increases reliability and security by keeping the central control functions at the operating plant.
• The Advanced Control Room Trainer (ACRT) offered by EnerSys and Atmos International is a training tool that emulates the operator’s system. ACRT trains pipeline controllers to effectively manage adverse and emergency situations in a pipeline control station by simulating events in their system.
• Atmos offers a full suite of pipeline simulation software tools to train operators on pipeline operations. [Watch their YouTube video series]
• API 1175 is a recommended practice published by the American Petroleum Institute addressing how pipeline operators should maintain their leak detection program. The goal of the standard is to have the best leak detection system possible by always looking for continuous improvements to the individual LDS components achieving operational buy-in with the culture, strategies, KPIs, and testing.

Simulators for Pipeline Training: Full Episode Transcript

Russel Treat:  Welcome to the Pipeliners Podcast, episode 32.

[background music]

Announcer:  The Pipeliners Podcast, where professionals, Bubba geeks, and industry insiders share their knowledge and experience about technology, projects, and pipeline operations. And now your host, Russel Treat.

Russel:  Thanks for listening to the Pipeliners Podcast. We appreciate you taking the time and to show that appreciation, we’re giving away a customized YETI tumbler to one listener each episode. This week, our winner is J. Scrivner, with Boardwalk Pipeline. Congratulations, and your YETI is on its way.

This week, we have our perennial guest. I guess you could say, “the guest we have had most frequently on the Pipeliners Podcast.” Giancarlo is coming back, and we’re going to talk about simulators in use for pipeline operations training.

Giancarlo, welcome back to the Pipeliners Podcast. I think you are now officially our most favorite guest. You’ve been here more than any other of our guests. We’re glad to have you back to talk about pipeline simulators and how to use them.

Giancarlo Milano:  Hi, Russel. Thank you for having me. I’ll take that as a compliment, or not. [laughs]

Russel:  It is a compliment.

Giancarlo:  It’s definitely good to back on the series and to talk about a topic that is very close to me when it comes to pipeline hydraulic simulators and trainer systems. Thank you for having me.

Russel:  I want to share with the listeners that that is how you and I originally got introduced. We were doing a project for a customer. This was probably back in 2008 maybe, 2009 timeframe.

Giancarlo:  Around that time, yes.

Russel:  We were building multiple systems for this customer from a SCADA’s standpoint, and you were building simulators so that the operators could get trained. This is a little bit of a trip on the wayback machine.

Giancarlo:  Yes, it is. It was definitely a lot of fun working with your and your team back then to create a trainer/simulator. This particular one was on a gas pipeline system and network and distribution. A lot of fun putting that one together.

Russel:  Let’s dive in. Giancarlo, how would you define a simulator and what it is?

Giancarlo:  In our leak detection series, we talked about RTTM, Real-Time Transient Models, for simulation purposes, and how you could use that for leak detection. When it comes to a trainer system for pipeline operators, we are using the same technology, the RTTM, Real-Time Transient Model, in order to simulate a pipeline.

We’re not doing a leak detection with that model. What we’re doing is simulating every single scenario or operating conditions so that an operator is able to be trained on the pipeline hydraulics, and also how to react to both normal and abnormal operating conditions.

If we just talk about the hydraulic simulator itself, that’s not really a full trainer. A trainer is when you combine that hydraulic simulator with a SCADA or some sort of HMI frontend that is the console where the operator interacts with their pipeline in their day-to-day.

When you combine those two, what you’re doing is you’re virtually simulating or mimicking the operation of the pipeline in the simulator itself. You’re modeling for flows, pressures, and dynamics, based on the different operations of that pipeline.

Then, all of that information is being sent back to the HMI or the SCADA frontend in order for the operator to be presented with those values as if they were coming from the real pipeline system. When you think about it, a trainer system is when you sit an operator in front of a SCADA console, and he’s looking at pressures, flows, valves, and pump statuses.

The operator is operating the pipeline as if it was the real system. In reality, that information is not coming from the pipeline. It’s coming from the simulator itself. That’s the concept of a trainer. It’s when you take a pipeline, you virtualize it as far as doing hydraulic simulation on it, and then you use those results from the simulation to fit the SCADA, so the operator can react accordingly.

Russel:  When we were talking in advance of the episode — talking about how I might put this together — you were talking about types of trainers. Why don’t you walk out that for the listeners? What are the types of trainers?

Giancarlo:  There’s different ways that you can configure a pipeline trainer system or operator training system, as it’s also called in the industry. One of the them, the very first one and the most basic, it’s a generic trainer.

What a generic trainer is, it’s a pipeline that doesn’t exist. It’s one that’s been made up. It contains a special element that mimics what a real pipeline actually does have. We’re talking about an injecting station. We’re talking about intermediate block valves. We’re talking about intermediate delivery stations, as well as final delivery stations.

Within each one of these stations, we are simulating the main components that drive the hydraulics of the pipeline. We will be able to simulate the tanks or the storage where the fluid is stored, the valve manifolds, the booster pumps, or compressors. As well as the injecting compressors or pumps into the pipeline system, or we’re talking about a gas, and product coming from a cavern, we’re talking about regulators to bring the pressure down into the pipeline system.

We are mimicking each station with every key element that is needed to hydraulically drive the pipeline. That would be a generic trainer system. It’s one that we made up. Then on the SCADA frontend, we could use different vendors.

It’s configured to have its SCADA frontend to that particular pipeline. If we have an overview of the whole system, and then it would have individual screens for each one of the stations, as well as an alarm management system.

The next type would be what we call a hybrid trainer. What a hybrid trainer is, is one that would combine different stations or operations from an operator pipeline. Rather than doing one trainer for each pipeline that an operator has, what you would do is that you would combine the key elements of operation within one pipeline system.

Then to that system, you would add a corresponding SCADA HMI in order to connect to that pipeline, and visualize the information. This one is taking it back a little bit further from the generic trainer, the generic trainer being one that we created based on our experience, one that we have made up of the pipeline.

The hybrid trainer is one that includes details of the operator’s pipeline, but not just one pipeline. It would include information from different pipeline systems into one.

Then, the third one that we offer, it’s a high-fidelity trainer system. This is one that would include an exact replica of a pipeline for an operator and its own replica of the SCADA HMI. When the operator looks at the SCADA HMI, he cannot really tell whether he’s looking at the real pipeline system or the trainer.

To him, all the values look the same. He has the same touch and feel. He will be able to train on that environment to the full of its capacity from the point of logging in, to moving between screens, to reacting to normal operating conditions or abnormal operating conditions.

Those are the three type of trainers that we offer.

Russel:  The next question I had in mind to ask is, what’s involved in using a simulator for training. I think your answer here, or your conversation here about types of trainers, leads into that. The first thing is, I got to build out my math model, right?

Giancarlo:  Correct.

Russel:  It all depends on what kind of simulator, what kind of training device am I trying to create? To the extent it’s an exact replica of the SCADA system, that’s by far the most effort to create. Now, I’ve got to really make sure that the behavior in the trainer is going to be as close as possible to the behavior I’d see in real operations.

Giancarlo:  When you think about it, the high-fidelity is the ideal one. It will give the operator the closest feeling to what he sees day-to-day. When he’s sitting in front of that SCADA console, it looks exactly like the real system.

The way that he operates and the way that he moves around should be no different than what this is every day. That’s really going to give him the best training that he can possibly get. The generic trainer is one that is off-shelf.

If we get requested to deliver a generic trainer, that’s something that we could deliver. Within the same week, it is all done. It’s a matter of just sending it to the customer. Then, the hybrid trainer, that’s one that takes a little bit of effort, as far as sitting down with the customers and creating one pipeline that will contain the element for the different pipelines that they have.

It’s a little bit more cost effective to have the generic or the hybrid, but then again, the high-fidelity, that’s going to be the full best trainer that one can possibly has, because it would replicate exactly what the operator sees on this day-to-day.

Russel:  Having been in the Air Force and been around aviation — I’m not a pilot, but was around it a lot — looking at what they had in the way of trainers, it’s really interesting. Modern technology makes trainers way better now than they used to be.

You can get a flight simulator program for your PC now that is as close to flying an aircraft as you can get, as to being in the actual seat. The difference there is, I build a particular aircraft. That aircraft has characteristics, and I build a trainer for that aircraft.

Every pipeline’s unique. That adds some additional complexity to try and achieve that reality.

Giancarlo:  Absolutely. Flight simulators, it’s a perfect example for an analogy between what a pipeline trainer is. If you think about it, the generic trainer will be the same as having just a generic flight simulator, where all you do is take off, and make sure that you move the planner up and forward, and then you land.

If you’re looking at a hybrid or high-fidelity, then you’re looking at how specifically to fly a 747, or an Airbus.

Russel:  Even more to the point, I think, is that trainer, particularly high-fidelity trainers, and particularly in aviation, they are more commonly used to teach abnormal flight than normal flight.

Giancarlo:  Correct.

Russel:  In aviation, if you’re a pilot, particularly if you’re a commercial pilot, you’re required to have a certain number of hours in a simulator to maintain your qualification. In those simulators, they’re throwing you abnormal operating conditions.

The advantage of a high-fidelity trainer is the ability to learn abnormal operations, how to respond to abnormal operations, and do that in a way where you’re not taking risk on the actual live pipeline. That means that in addition to just building out the system to emulate the physical pipeline, and look the same from a SCADA standpoint, I also have to build scenarios.

Giancarlo:  You make an excellent point there. When you’re talking about any of these systems, the scenarios is the most important thing. What is exactly that you want to train the pipeline operator on? It could be that you have a new hire, and you use a trainer simulation just to take them through their normal day-to-day operations, shutdowns, start-ups.

Making sure that they follow the procedure that the company has established. Then, once you have that base, then you would take them to more dedicated scenarios for abnormal conditions. That would include unexpected shutdowns of valves, pumps, or compressors, or maybe a leak at a particular location.

What’s the procedure that they should follow when that leak alarm comes in? A few weeks back, you and I were talking about leak alarm response. Having a pipeline trainer simulator would be a good way to teach or train the operator in how to react to these instances: what to look for, how to maneuver himself from one screen to the other, which will be the same as when the pilot receives some sort of notification or alarm via sound, then what gauge he should be looking at in order to troubleshoot that specification notification.

When it comes to a pipeline hydraulic simulator for training purposes, it’s the exact same thing. How does the operator react to this particular instance? Where should he go, and how should he react? Those are the main elements of the trainer system itself.

Russel:  I think the point we’re both trying to make is that it’s not just the ability to interact with the SCADA screens.

It’s actually the scenario and all of the other, being able to train in the full policy and procedure around a particular kind of scenario, including the alarm response, doing the logbook, and all the other things I need to do in the control room. It becomes a comprehensive thing.

Giancarlo:  Correct.

Russel:  I want to talk a little bit about a project we did. You’re laying out a hierarchy of training that a simulator can support, and I want to try and unpack that a little bit. We did a project for a pipeline operator on the island of Oahu in Hawaii.

A small pipeline, but the thing about a pipeline is the size, even if I’m a short run of pipeline, 20 miles or less, I still have to do all the things a pipeline operator has to do. This particular pipeline operator was actually the power company.

They were taking the operations of the pipeline from a third party and bringing them in internal. They were building up a capability for scratch. Part of our project scope was to build a trainer. We actually built the trainer, and had the trainer running three months before the operator took over control of the pipeline.

One of the things you can do with these trainers is you can actually use them as a way to learn what I actually need to build. [laughs] It’s interesting.

Giancarlo:  That’s the perfect example of a good use of a trainer system. Before you build a pipeline, you can do a hydraulic simulation of how the pipeline would react during some operations.

Then when you take all of that information, and you fit it to SCADA, then you can put that information in front of the operator so he knows what to do and how to react when the actual pipeline comes to life, and he starts to operate in that pipeline for real.

Russel:  There’s a hierarchy, too, in this operating, for real. You talked to that a little bit already, but it starts with normal — normal startup, normal shutdown, normal steady state operations. It starts with that, and then there’s this other idea of normal, not usual — meaning, it’s a normal way to operate the pipeline, but I don’t do it very often.

Giancarlo:  Correct, yes.

Russel:  Trainers can be a way, like for example, if I have some kind of seasonal operation, and I do it once a year — it’s normal, and I do it every year — I might want to build a training scenario about how to do that before I actually do it on the live system as a way to refresh people’s memory.

Giancarlo:  Absolutely. Before that operation comes to life, you can create one scenario for that particular trainer system to cover that specific instance that you don’t see every day. You’re ready for when that happens.

That was a great project, by the way, on how that particular customer used the trainer system, because they were able to train their operators before the pipeline came to life. Not only on the hydraulics on the line, but also to identify themself with the SCADA screens, how to log in.

You think this is silly, but how to log in, how to move from one screen to another, how to visualize the alarms, how to interpret them.

Russel:  Simple little things, like how do I operate a valve, how do I start and stop a pump? All that kind of stuff, being able to do that on a trainer, it’s really, really helpful.

Giancarlo:  Correct. It’s all about giving the operator all the tools that you can to make sure that he feels more confident when he sits in front of that console. If he has more confidence, then he’s going to feel more relaxed when something comes up. Because of that, he’ll be able to react to it a lot better.

Russel:  The companies that do this, they’re constantly looking at what scenarios do we need to add or improve, or any of that kind of stuff. Once I’m doing normal, and normal not usual, then I need to abnormal.

Then, I need to do different kinds of abnormal. Then, I need to do emergency. All of that is valuable.

Giancarlo:  Absolutely, it is. Emergency, abnormal situations, and how to react to emergency situations, that’s one of the key points of the trainer. That’s the end goal. Obviously, all the normal stuff is good, too, but emergency and how to react to those, that’s the most important part.

It takes me back to the first trainer system that I did. When I did that trainer system — this was one system in Canada — 350 kilometers worth of pipe with heavy crudes and diluent. For this particular pipeline, it was an instance where I started building the pipeline before the system came live.

The first thing that I was handed was 500 pages of this control philosophy. Then, emergency shutdown keys, or cause & effect metrics for what to do during an emergency situation if the pressure got too high, or the pressure got too low, a pump trips, and all sort of things.

The cool thing about the trainer system is that all of these can be implemented within the hydraulic simulator. Everything that is done at the PLC level or the SCADA level for control purposes and emergency response could be integrated into the model itself, without having a PLC or a DCS next to it to make all of that control for you. You can have it integrated within the hydraulic simulation. By me doing that, I was able to review all the controls for their pipeline, for their individual elements. Talk about permissions to operate, what does it take for a pump to be started?

You think about it, you can’t just go around and push a start button. You have to make sure that the valve alignment — it’s all open from the source all the way to the discharge. If those valves are not on, then you cannot have that start button enabled, because the operator can just go ahead and push it.

It was so very interesting to go ahead and take all of this control philosophy and implement it within the hydraulic simulator, to the point of view that I knew exactly how the pipeline worked by the end of the project.

In some instances, the operators would come to me, and it’s like, “Hey, how do you do this, or how do you that?” I’m like, “Well, you are the operator.”

Russel:  Giancarlo, I think that’s such a very good point, because my experience has been that I learn a certain amount putting in a leak detection system. I learn more putting in a SCADA system, but when I put in a simulator, I got to learn it all.

I got to get down to the itty-bitty, nitty-gritty details. It’s like I’ve built a whole system. It’s equivalent to building a whole system up from the PLC all the way through the leak detection. Because, in effect, what I’m doing in the hydraulic model is, I’m not only doing everything the PLC does, I’m also — I’m actually taking the equipment, the pumps, the valves. I have to put in their CVs. I got to configure their specific behaviors so that the hydraulic model works correctly. There’s a lot of detail in doing that, but once you get there, boy, it can really add some big value.

We actually, on the project that we did with you, we came up with a standard approach, which we call ACRT, which is advanced control room trainer. Which takes the Atmos SIM and puts our intelligent operator console (IOC) on top of it.

We’ve made that a standard approach for our customers that are interested in a trainer. I think the other thing we ought to talk about is, we’ve talked about, you’ve got to build a model up. You’ve got to get a clear bunch of scenarios.

You need to look at your training hierarchy from normal, through abnormal, into emergency. We’re all doing this for two purposes. One, we operate better, but the other is, there is a regulatory requirement that pipeline controllers be qualified. How does that relate to all this conversation about trainers?

Giancarlo:  It’s all very interesting to put the trainer system together, the hydraulics, the interface with the SCADA. Then, as you mentioned, the end goal here is to train the operators, and give him the tools that he has.

Once the operator has gone through a training session, 15, 30 minutes, a few hours, depending on what area it is that you’re training him on, then you need to analyze how he did and take a look at his performance.

What we have within our tool, the Atmos trainer, is we have an operator qualification module. What that does is that analyzes all the analog alarms that are exceeded during the sessions, and it evaluates whether those alarms should have come in or not during that particular scenario.

It also follows and analyzes digital sequencing. Let’s say, for instance, that a pressure got too high, and something was shut down. Then after that comes in, the procedure will dictate the operator to follow a series of steps.

If the operator goes in, and he follows the steps that he’s supposed to, then everything is great. He’s not penalized, but if he doesn’t, or he deviates from that sequence of the procedure, then he could be penalized on that.

The way that the system works — what the operator qualification module does — is that once you configure all of these digital and sequencing alarms and analog alarms, we run that, based on the results of the particular scenario for the simulation, in order to get a grade of his performance.

Based on that grading for his performance, then we’re able to dictate whether an operator is well-qualified during those situations, or whether he needs more training.

One of my emphases in all the years that I’ve done trainer system is that the end goal is not to fail the operator. It’s to aid the way does his day-to-day, give him all the tools that he can have in order to make his life easier. Making sure that he has the proper training, and he knows how to react during those situations. If he didn’t do so well in a training session, he can then go on his own and practice on that particular scenario to focus on what exactly he did wrong, and what he needs to improve on.

Then, after a certain period of practice, he can come back and go through the goal of the scenario again in order to make sure that he is aware of what to do now. That way, you can keep track of the performance of the operator from the very beginning to the very end.

It’s very important to not just do the training session, but do the analysis of the performance after that’s done.

Russel:  I think there’s also another thing that’s implied here is that there’s a distinction between building up the competency and demonstrating the competency.

Operator qualification, ultimately, what you’re trying to do as a pipeline operator from a regulatory requirement is demonstrate that the particular controller has the specific competencies required for the responsibilities that they’re going to be operating under.

The ability to be able to train and score gives you an indication of, are they moving towards having that competency? This is a big deal, particular for larger operators would multiple consoles. Those controllers need to be qualified for the specific areas of responsibility.

That can be different, depending on: is it a liquid system, is it a gas system, is a gathering, is it a transmission? All of that goes into what’s required to have the competencies to operate that, and be operator qualified for that system. That’s pretty cool.

Giancarlo:  Correct. I was reading an article that the life cycle of these operators being moved from one console to another is about three years. On average, every three years, they are switched from operating one pipeline to operating a different pipeline.

If an operator has a lot of pipeline systems, they’re circulated around. They’re different pipelines within that period of time. Having a trainer system on these pipelines can help the operator learn what it is that they’re getting into when moving from one system to the next, and get ready before they start physically operating that pipeline.

Having a trainer system is a great tool to have for any pipeline operator that wants to make sure that their operators are capable of operating their pipeline with the highest level of confidence.

Russel:  Being a guy who tracks the regulators and tries to stay current, not only on the status of various rulemaking, but what the regulators are thinking, and what they’re talking about, I know that some of the pipeline regulators have had conversations with the aviation regulators about what they require for training and what they require of simulators and trainers.

Trying to think about how those requirements might be brought into the pipeline space, I think ultimately, that’s inevitable. I couldn’t begin to speculate on the timeframe. I can certainly say that building trainers for pipelines is a very different problem than building trainers for aircraft. But, that being said, I do think we’re headed in that direction.

Giancarlo:  Absolutely. It’s so important to make sure that our operators are knowledgeable, and they’re confident in how their pipeline works and what to do during any normal or abnormal operating conditions. I definitely agree with you in the fact that it’ll be heading there some time in the future.

Russel:  Let’s wrap it up. There’s a couple of things I want to mention. I think there’s a couple of resources that you want to make available to the listeners. You mentioned that there is a YouTube video available on pipeline trainers that Atmos has been involved in. Can you tell the listeners about that?

Giancarlo:  Within the last month or two, we put together a five-minute, I think it’s less than five minutes, about a three-to-four-minute video on how a trainer system works.

It goes from the very beginning of how trainer system works with the different parts of it, the integration, and how it could be used for the purpose of training an operator on his day-to-day or abnormal conditions. We are going to put the link to that video on the show notes. Be sure to go there and check it out.

In addition to that, we also have some resources in our website within the simulation portion of our website at atmosi.com. You can go in there, and you can find a brochure and more description on the different offers that we have for pipeline trainer systems.

Please be sure to go to our website to check out those, and also take a look at the video. I think it’s a pretty cool one. I think you’ll definitely enjoy it.

Russel:  Lastly, I want to promote this again because you guys have been so very generous. The Atmos International folks have offered to give away copies of the book, “Introduction to Pipeline Leak Detection.”

If you have an interest in that, we’ll have a link in the show notes and give you a pathway to go get a copy of that, as well. Again, Giancarlo, thank you for being such a big fan and supporter of the Pipeliners Podcast.

I am certain we’ll have you back. I’ve already got some other things that I’d like to get on our calendar to talk about, in particular one being API 1175, which is the leak program management recommended practice. I think that might also be something the listeners would have an interest in. Thanks again, man.

Giancarlo:  Absolutely. Thank you, Russel, for having me. I look forward to being on the show again. As always, it’s been a lot of fun to chat with you. Thank you.

Russel:  Thank you. I hope you enjoyed this week’s episode of the Pipeliners Podcast and our conversation with Giancarlo. Just a reminder before you go, you should enter to win our customized Pipeliners Podcast YETI tumbler. Simply visit pipelinepodcastnetwork.com/win to enter yourself in the drawing.

[background music]

Russel:  If you have ideas, questions, or topics you’d be interested in hearing about on the Pipeliners Podcast, then let us know on the Contact Us page at pipelinepodcastnetwork.com. You can also reach out to me directly on LinkedIn. My profile is Russel Treat. Thanks again for listening. I’ll talk to you next week.

Transcription by CastingWordsx