Direct fatigue monitoring is essential for today’s pipeline control room. But, is it really about fatigue mitigation or maximizing alertness to ensure safe operations?

To help understand the latest technology available for enhancing alertness, Clementine Francois of Phasya joins Russel Treat on the Pipeliners Podcast to discuss this important topic.

Listen for unique insight on her scientific research surrounding fatigue monitoring, the technology developed by Phasya to support automotive and industrial user alertness, the real-world application of their technology, and what this could mean for the pipeline industry going forward.

Direct Fatigue Monitoring: Show Notes, Links, and Insider Terms

• Clementine Francois is the Chief Scientific Officer at Phasya. Connect with Clementine on LinkedIn or via email at c.francois@phasya.com.
• Phasya offers software for monitoring the physiological and cognitive states (e.g. drowsiness, stress, cognitive load, etc.) that influence human performance.
• Fatigue Mitigation, as outlined by PHMSA, requires operators to implement fatigue mitigation methods to reduce the risk associated with controller fatigue that could inhibit a controller’s ability to carry out the roles and responsibilities the operator has defined.
  • The PHMSA CRM Rule (Control Room Management Rule) provides guidance on how operators should support fatigue management in the control room. [Read the CRM Rule FAQs for specific guidance.]
• The Three Mile Island accident occurred in Pennsylvania in March 1979 when radiation leaked into the environment from the nuclear power plant known as TMI-2 (Three Mile Island Nuclear Generating Station). 40 years later, the plant is expected to officially close down in Fall 2019.
• An electroencephalogram is a test or record of brain activity produced by electroencephalography, which is the measurement of electrical activity in different parts of the brain and the recording of such activity as a visual trace.

Direct Fatigue Monitoring: Full Episode Transcript

Russel Treat:  Welcome to the Pipeliners Podcast, episode 80, sponsored by Gas Certification Institute, providing training and standard operating procedures for custody, transfer, and measurement professionals. Find out more about GCI at gascertification.com.

[background music]

Announcer:  The Pipeliners Podcast, where professionals, Bubba geeks, and industry insiders share their knowledge and experience about technology, projects, and pipeline operations. Now, your host, Russel Treat.

Russel:  Thanks for listening to the Pipeliners Podcast. I appreciate you taking the time, and to show that appreciation we give away a customized YETI tumbler to one listener each episode. This week, our winner is Justin Mohn with Energy Transfer. Congratulations, Justin, your YETI is on its way. To learn how you can win this prize pack, stick around until the end of the episode.

This week, we have with us Clementine Francois. She’s going to be talking to us from Belgium. Clementine is a Ph.D. engineer and researcher that has some very interesting technology around fatigue management. Hope you’ll enjoy this conversation.

Clementine, welcome to the Pipeliners Podcast.

Clementine Francois:  Thank you for having me, Russel.

Russel:  For the listeners, Clementine is with us from…I’m sorry, you’re in Denmark? Tell me again where you’re from, where you’re actually sitting today.

Clementine:  I’m sitting in Liège. It’s a city in Belgium. Belgium, it’s in the heart of Europe.

Russel:  Yes, right. Actually, kind of appropriate, seeing how we’re circling the 75th anniversary of D-Day.

Clementine:  Right.

Russel:  That’s a rabbit hole. We won’t walk down that.

Clementine, I’m so glad you took the time to do this. I’m really very interested in what you’re doing. I think, first, I’d like to ask you to tell the listeners a little bit about who you are and your background.

Clementine:  Sure. Actually, I have a master’s degree in biomedical engineering and a Ph.D. in engineering sciences from the University of Liège here in Belgium. I first started my career as a teaching assistant and a researcher in the Department of Electrical Engineering and Computer Science at the University of Liège.

I was on a project where the goal was to design a fatigue monitoring system in the form of a pair of eyeglasses with a built-in camera in order to avoid accidents in transport. My part of the job was actually to develop algorithms, so software solutions, and also to validate the solutions using particular references in the field.

This project was also the topic of my Ph.D. Then, in 2014, we had arrived with the technology mature enough to face the market. And so, we decided to create a new company, a start-up, called Phasya, to commercialize the technology.

Phasya is now a company specialized in the development of software solutions for the monitoring of physiological and cognitive states of individuals; of operators that have an impact on performances and decision-making. I’m now one of the co-founders of Phasya and also the Chief Scientific Officer, so more responsible for the scientific, technical, and innovations aspect at Phasya.

Russel:  That’s a very impressive background. I’ll share with the listeners that one of the reasons I asked you to come on…I actually looked at a technology like this a number of years ago that was eyewear. You guys have taken it beyond that.

I read a stack of research about this stuff — just found it really fascinating — how the eye indicates fatigue and how monitoring the eye can give you an indication of fatigue. Maybe you can talk a little bit about what goes on with the eye and how is it that the eye is an indication of fatigue.

Clementine:  Actually, the eye is really the reflect of what is happening in the brain. It’s really considered as a very reliable indicator of fatigue, and particularly the fatigue state because fatigue is quite a complex phenomenon, it’s difficult to characterize it. There is no unique and standardized reference in order to characterize and monitor fatigue. The eyes are recognized as one of the best one.

Actually, when someone is fatigued, the eye movements are slower. Of course, when we see like a child at the back of the car that is falling asleep, you can see that the eyelids are becoming very dozed, they’re closing very slowly and re-opening very slowly, and so on so.

Russel:  The English phrase for that would be, “The eyelids are becoming heavy.”

Clementine:  Yes, that’s it. That was the term that I was searching for, sorry. There are many, many ocular parameters that are indicators of fatigue. It’s not only about determining if the eye is closed or not for a certain period of time. It’s the combination of several parameters/features that we can extract from eye movements that enable to determine a reliable detection of fatigue.

Russel:  As I recall from the research that I read some time back, there’s a number of things that people will look at — blink rate, the position of the top eyelid across the eye, the dilation of the pupil, and then once the eye closes, how long is the eye closed when you blink, being an indication?

Those are the things that I recall. What’s interesting to me about this is that…and you used the illustration of a baby falling asleep. I think that’s something that pretty much everybody can relate to.

You can look at a kid and you can know that kid is tired, and the kid is fighting to stay awake. You can just see right. To actually take and break that down scientifically, that’s interesting to me. I guess, my question would be, if you break that down, is there a level at which you can see that drowsiness, if you will, with the computer before a human would be able to see it.

Clementine:  That’s actually the goal because the idea is of course to detect early signs of fatigue in order to avoid errors or accidents that might be related to fatigue.

What we have developed is the monitoring of drowsiness from the analysis of eye images. When you have a camera that takes images of the eye at a high speed — this is of course the best way to capture really fast movements of the eye and to be able to analyze a lot of different features.

From the analysis of images of the eye like this, you can extract a lot of different information and even more information than what a human eye is able to do when looking at the child, for example, you know you can go higher in terms of frame rate, in terms of resolution, and so on.

Technology can help in order to detect more early signs of fatigue, to prevent a worker or an individual from falling asleep in a situation that he couldn’t.

Russel:  People who work in control centers will be aware of this. People who drive professionally will be aware of this because it’s part of their training. What a lot of people don’t realize is that when you’re fatigued, in terms of your ability to control your body, it’s very similar to being drunk.

If you think about it, probably most people have driven when they’re tired and have had an experience when they didn’t really realize they weren’t there and they come back to their senses and they realized they’re not actually in the lane they started in. That’s the kind of thing that you’re trying to avoid.

The other thing that people don’t realize about this, and you could probably speak to this better than me, I think that people don’t realize that there’s a sliding scale, if you will, and the more complex the task you’re trying to do, the more fatigue impacts are believed to do the task.

Clementine:  Yes, you’re completely right…I will just come back to your comparison with alcohol because this is completely true.

I read some time ago in the literature that when you are awake for more than 24 hours, it’s really equivalent, in terms of consequences on your behaviors and so on, if you had one gram per liter — sorry, this is European units — but one gram per liter of alcohol in the blood.

I cannot do the conversion for you, but it’s quite a high level of alcohol in the blood, and it’s, of course, in terms of effect, so on the consequences of fatigue. We know that when you are fatigued you have increased attention deficits, you have increased slowed cognitive functions, increased reaction times, and of course, with consecutively impaired performance and that can lead sometimes to errors, or it might also lead to accidents. So it’s really important to be conscious that fatigue can sometimes have dramatic consequences.

Russel:  You may not be aware of this, but if you study incidents in the process industries — plant incidents and refining in petrochemical and nuclear power, and those kinds of things — a disproportionate or a statistically significant portion of the incidents occur between 1:00 and 4:00 in the morning.

You can’t say that’s just because of fatigue, but you can certainly say that fatigue would likely be an impact to the people’s ability to respond to the incident.

Clementine:  Yes, completely. I think that it was one of the factors that also had an impact on the disaster of Three Mile Island. It was a nuclear disaster, or at least an industrial plant disaster in the United States, in 1979. Yes, actually there were some errors, human errors, during the night. So, yes, we could, at some point, relate that to fatigue.

Russel:  It certainly was a contributing factor in Three Mile Island, as it has been in a number of other things. I think there’s a lot of training on it, but in general, I don’t think it’s really very well understood.

Anyway, that’s kind of a good basis to tee-up why this is so important. Obviously, this is a pipeline audience, and those that work 24/7 and in the control room are familiar with the idea of fatigue.

Talk to us about how you guys are applying your technology, and what are the business cases, the actual applications you’re currently working with for doing this fatigue monitoring by using cameras to watch the eye.

Clementine:  There are different business models, if I can call them like this. We have, of course, the operational real-time monitoring of fatigue, so from images of the face or of the eye. It can be with a remote camera that can be placed in front of the operator, or it can be also in a head-mounted device, for example.

This is for real-time fatigue monitoring, and for that, our main market right now is more for the automotive sector. We have contracts with the automotive industry in order to integrate our technology into cars for the monitoring of fatigue, and also the monitoring of distraction for drivers.

We also have a model in which we can perform studies, so to really do the monitoring of fatigue in several operational environments in order to make some prevention plans and to help companies in order to define and create better schedules for the operators. This approach aims to provide some statistics that can be relayed to the management where you can put some prevention plans in place.

We have both sides. I don’t know if I was really clear on that, but either the real-time operational monitoring, so with the continuous monitoring of the state of the individual, or really more as studies to help to put in place some training or some plans, and within the company management plans.

Russel:  In our vernacular, that’d be fatigue mitigation.

Clementine:  Yes, that’s it.

Russel:  I want to talk a little bit more about what you guys are doing with the auto industry to put monitoring technology into cars. Is this something that the manufacturers are looking at as safety equipment, and how would that get applied?

Clementine:  Yes. This is something that is coming, more or less, urgently, at least in Europe, because there is a new regulation on that that will soon be signed by the European Commission that will oblige OEMs and Tier 1s and so on, to equip the cars with systems for the monitoring of fatigue and of distraction, because when we will arrive with more automation in cars, the driver will be also less and less stimulated by the task. The driver will be more induced with fatigue, and so we need to be able to monitor the state of the driver, because the driver will still need to be able to take back the control of the car in case of problems or sometimes in certain zones, in the city center, for example, and so on.

These are preoccupations of the car manufacturers right now, and they are seeing that as a safety option in the car, so maybe as a first step, as more of a feedback to give to the driver, but then with more and more automation, the goal is really to have this feature as a real safety feature.

Russel:  That’s very interesting to me. Just like we’ve been talking about fatigue monitoring, and you mentioned distraction monitoring, so distracted driving. I think you could also throw into there impaired driving, as well.

Clementine:  Right. We are also working on it, so distraction detection, but also other physiological and cognitive states that have an impact on human performance and decision-making.

We are working, notably, on stress monitoring and cognitive load, because, of course, sometimes you need to be at a task for an extended period of time, or you have much information to process at the same time, and you need to react very quickly.

You need to remain concentrated, even under stressful situations and so on. It’s also important to be able to monitor other states that could also have an impact on the performance.

Russel:  Sure. I think it’s pretty clear what kinds of things you could detect with this technology. That, to me, is pretty clear.

When I start thinking about this, and I think about how this might add value in the pipeline world and in the pipeline control center, certainly there would be some value to knowing how one particular shift rotation impacts controllers versus a different shift rotation, and how changes in things like lighting and so forth impact alertness level.

But I think there would be some resistance by the folks who are working the consoles to being monitored, and I’m curious how that comes up, and how y’all address that conversation with this kind of technology.

Clementine:  You’re right. This is an issue for adopting our technology, because there is resistance to being monitored, essentially because we have, of course, more experience in the automotive industry, either on the car or on the commercial vehicles and we encounter that kind of resistance more in the commercial vehicle sector, because professionals are afraid of being monitored. We faced some social barriers about that.

I think that the communication to the workers is really important, because this is really to help them in order to make fewer errors, but also to have a better quality of life, so to stop when they physiologically need to.

It’s really something that is more to improve safety and the well-being of the professionals, of the operators. The goal is not to monitor what they are doing, but it’s more to be able to secure them in some way. I think this is really a matter of communication to the operators, because this is really to improve safety.

Then, of course, there are several options that we could put in place, like for example, having a first study in order to do some statistics and help to improve schedules and so on so that the professionals would be more comfortable with their planning at work.

And then maybe go as a last step with the continuous monitoring of the operators so that they could learn the interests of having the monitoring of their states.

Russel:  Yeah. I think that’s well said. I think one of the other things, too, that intrigued me about what you guys are doing, when I originally talked to you, was not so much the fact that you’re doing monitoring, but how you’re doing the monitoring. What I had looked at before was all eyewear, and you guys are actually doing it without eyewear. You’re just doing it with cameras.

Clementine:  Right.

Russel:  I think that is less intrusive than eyewear, if you will. If you put on a piece of eyewear, you’re aware you’re being monitored, but if there’s a camera there, you can kind of transcend the fact, move past the fact that you’re being monitored.

Clementine:  You’re completely right. Our goal is really to be as transparent as possible for the operators, so having cameras placed remotely would, of course, help with that, and this is what we are doing.

We are also able to monitor other data from the human body, for example, from smart watches, where we can get the heart rate. This would also be in some way transparent for the operator, because this is not something that is really placed in front of him. This is just recording signals from the wrist.

These are different kinds of sensing modalities that we can use. We really want to be flexible and adaptive to the data that is available in the environment, so we don’t want to put a lot of sensors and a lot of equipment on the head of the operator.

The goal is really to take as much as data as possible, of course, in order to have reliable monitoring of fatigue and other states, but also to be as less invasive as possible.

Russel:  Non-intrusive, exactly. I think one of the things that I would love the opportunity to do… In the other research I had read, they used a piece of monitoring equipment to monitor fatigue level. They called it alertness level. I actually like that name better, because it’s really what we’re trying to do is maximize alertness, right?

Clementine:  Sure, yes.

Russel:  We’re not trying to mitigate fatigue. We’re trying to maximize alertness. I think that’s a better way to say it. They would take people and have them drive a car simulator, and they would have them at various levels of fatigue. They’d keep them up at different levels of time, and have them do exercise and various things, and then they would have them drive.

They would measure their alertness level, and they would measure their lane departures, and they would correlate those things. The research was really compelling, because there was a point where if you didn’t have at least this level of alertness, then your risk of lane departure was really high.

This was a big thing in mines, in particular, where they’re moving lots of very large, heavy equipment, because a lane departure can be a catastrophic thing. It can be catastrophic regardless, but it was interesting to me.

I say that to kind of tee this up. What I would love the opportunity to do is to set up a pipeline simulator and some normal scenarios of what a pipeline operator does, and then have them at different levels of known fatigue, and certain levels of sleep deprivation, and then see how they’re able to perform these tasks and correlate that.

I think that’d be really interesting, and I think it’d be really compelling, once you had the data and could do that. I don’t know if there’s an appetite for that in the pipeline community, but certainly I think it’s an interesting notion.

Clementine:  Sure. Actually, I would be very interested to do that kind of study, because we have already performed the same kind of studies for the automotive sector, so more or less the same thing as you were mentioning, with the driving simulator and having the monitoring with the cameras.

We also used other kinds of equipment, like an electroencephalogram. So, electrodes that you put on the scalp in order to record brain activity, because we are also used to analyze signals of brain activity. This is really, of course, the main and the most reliable reference that we can use in order to determine the real state of the person.

I would be very interested in having this kind of study in place in the pipeline industry, and to see the correlates of the task that is, of course, different from driving, and to see how fatigue has an impact on performance, while performing the tasks and so on.

Russel:  I think it’d be interesting, too, because the other thing you’d have to look at is the impact of cognitive load. There’s certain things that a pipeline controller would do that are more routine. They’re more like operating a car.

Then, there’s other things that a pipeline controller would do that are high cognitive. I’ve got to sit down and do a hand calculation, or something like that. Those would be, when you start adding the cognitive piece into it, it gets more complicated.

Clementine:  Sure. We have a limited amount of cognitive resources, so of course, if the task is too demanding, or if it’s for a too long, extended period of time, and so on, this could be also a problem of missing, for example, some alarms and so on. It could really help to monitor, also, cognitive load, I think, for operators of the control rooms in the pipeline industry.

Russel:  It reminds me of, I used to do Crossfit. I don’t know if you’re familiar with Crossfit, but it’s a big deal in the U.S. It’s a kind of workout.

Occasionally, we would do these workouts where at the end of the workout, you had to solve a math problem. I’m a guy who’s pretty good at doing math in my head. Doing math in your head when you’re completely burnt at the end of a workout, that’s a whole different kind of thing. It’s again, I think, an illustration of the principle.

Clementine:  Yes.

Russel:  I think the last question I want to ask you is this. What’s really the goal of all this? What are we trying to accomplish?

Clementine:  The goal is really to be able to improve performance and decision-making in order to avoid errors or potential accidents. This is more related to safety.

Another goal is also to improve the well-being of workers so that they have less stress in their operational field, and so they have a better, also, quality of life. This is really a matter of helping to improve health and safety in general for workers, and of course, to avoid errors and critical, catastrophic accidents.

Russel:  I think the point you make about the quality of life and being able to reduce stress, that is what is really the benefit that would be of interest to people in the control room.

Those kinds of things that improve the quality in a 24/7 operation around lighting, temperature control, workload, and how the workload is scheduled out through the day. By changing the environment, by dealing with ambient sound levels, by changing the way the work is scheduled, you can take a very high-stress job and turn it into a relatively low-stress job, without really changing the job.

To do that, and do it well, you need a basis for that decision-making. I think that’s really what the promise of this is. It creates information that can become a basis for analysis and decision-making.

Clementine:  I completely agree. Yes.

Russel:  Look, thank you so very much for coming on. I know you were a bit nervous about your English. I think your English is awesome. I know it’s not your first language, but I think you did great.

Clementine:  Thanks.

Russel:  I think it was very good. I only added a couple of things trying to translate. I’m so glad to have you on. I certainly hope some people reach out to you, and maybe there’s a project that happens in the pipeline world around this kind of thing. I think that would be awesome.

Clementine:  Of course. I hope so. Thank you very much for having me. I really enjoyed it. Thanks.

Russel:  Clementine, if somebody wanted to reach out to you, what’s the best way to get in touch?

Clementine:  On our website, on Phasya, it’s www.phasya.com, and there is all of the information about Phasya, or of course, I have my email address. It’s c.francois@phasya.com.

Russel:  All right, Clementine. Thank you so much. For the listeners, we’ll link these details up in the show notes.

Clementine:  Thank you.

Russel:  I hope you enjoyed this week’s episode of the Pipeliners Podcast, and our conversation with Clementine Francois.

Just a reminder before you go, you should register to win a customized Pipeliners Podcast YETI tumbler. Simply visit pipelinepodcastnetwork.com/win to enter yourself in the drawing.

If you would like to support this podcast, please leave us a review on iTunes/Apple Podcast, Google Play, Stitcher, or whatever smart device podcast application you happen to use. You can find instructions for this at pipelinepodcastnetwork.com.

[background music]

Russel:  If you have ideas, questions, or topics you’d be interested in, please let us know on the Contact Us page at pipelinepodcastnetwork.com, or reach out to me on LinkedIn. Thanks for listening. I’ll talk to you next week.

Transcription by CastingWords