The COVID-19 outbreak is reshaping lives and neighborhoods. As a company unreservedly committed to our employees, our customers, and our communities, QSI is doing everything in our power to minimize its impact.

Informed by expert advice, we are taking actions that comply or even exceed CDC guidance – all to keep our workforce and surrounding communities safe.

Our social distancing, sanitization measures, and working from home are all steps that will help flatten the curve. So too will aligning with our clients’ individual risk minimization requirements and expectations when on-site.

We realize that manufacturing businesses are experiencing unique challenges. Constant communication is essential to help our customers succeed during this difficult time. As a nimble, adaptive company, we will always find ways to meet your changing needs while putting safety first.

Right now, this means shifting to remote and automated solutions:

Remote access.
Diagnosing and solving issues from afar doesn’t need to be challenging. Our secure remote access capabilities empower us to quickly, safely and efficiently analyze and solve emerging issues to support your team. We can get customers started with hardware that can be connected and configured remotely, allowing us to troubleshoot and provide support while adhering to social distancing guidelines, and even across the country. With remote access, projects don’t need to stop. Even if a site is running on a skeleton crew, or is restricting personnel access for safety reasons, you can still progress by proactively laying the groundwork keep things running smoothly.

Production support. Due to COVID-19 many plants are running with reduced personnel. Some may also face challenges sourcing OEM equipment technicians. As industrial automation experts, we can help fill these gaps experienced by manufacturers nationwide. We are also more than happy to serve customers within driving distance. We stringently adhere to customer’s individual risk minimization guidelines, conducting our business safely and efficiently while on-site.

Ad hoc problem solving. QSI exists to solve problems. We offer tailored solutions to meet business and production challenges as they arise. Many customers currently face unprecedented demand, running overtime as production ramps up. Using our real-time overall equipment effectiveness (OEE) diagnostics, you can minimize waste and maximize yield. We can also help customers future-proof their businesses with solutions that maximize output, minimize human error and automate repetitive tasks. The outcome is better ROI and improved overall worker satisfaction.

We understand that COVID-19 is changing the way the nation lives and works, and we are here to support our customers every step along the way. We are actively reaching out to customers, and invite you to speak to us about solutions to help troubleshoot or maximize output during times of restructure and change.

 

Hi there, welcome back. This is the second part of an interview I did with Atomic Revenue about the difference between Legacy Controls Migration and Controls Modernization. Read the first part, which included references to video games, the growth mindset, and data analytics, here.

Quick summary: Legacy Controls Migration is getting an old system up to the latest and greatest hardware and software, to avoid downtime in case of failure. Controls Modernization is a separate project to take advantage of new functionality for higher efficiency, greater reliability, and better analytics, among other aspects.]

AR: How does that set you up for the future? If you complete a major modernization project now, when might you need another? Is it two years? Five? Or could you say that if you did it right, you won’t need to have that same kind of transformative modernization project for a significant amount of time?

QSI: I would say after modernization they are good for a significant amount of time. A lot of these systems that they’re upgrading are 20, 30, years old. And they’ve worked fine to that point in time. But now as things get more competitive, if they want to stay ahead of the curve, you know, they may not want to wait another 30 years. You may continue to upgrade along the way, which is easier with more scalable control systems. You see that a lot, in these old systems they’ve added devices over time to better control the process. They say “We want to rewrite this whole thing, because along the way we’ve added this, and this, and this conveyor offshoot, and this burner, and it’s made our process better, but it was pieced together.”

Things kind of get disorganized. We call it “spaghetti code”, and it becomes very hard to manage. But the structure of programs now is so much better and so much easier to manage, which is another benefit that you wouldn’t think of as obvious. It’s having more structure to your programs, better organization and your controls are laid out in a much more logical fashion. Documentation is much better, too, so you’re not losing descriptions in your programming, and all that consistency makes your maintenance more effective. Because what was written before, by, say, potentially 10 different operators as they added to the code, now is rewritten with by an experienced engineer with structure and organization in mind. We’ve reimagined the whole system and there is no need to be diving into the programming all the time, because it works.

You know, I’ve described that process where you have something that works, but you need a new function, so you just stick something else on to the side, I’ve called that the Frankenstein method. “We’re just going to add this little thing here, we’re just going to work-around that there, we’re going to add one other feature here,” and in the end it’s almost unrecognizable compared to what you started with. But it does kind of the things that you need it to do, yet you’re fighting with it all the time and it doesn’t really behave. It sort of takes on a mind of its own at times. So you occasionally need to hit the reset.

People who think like that are growth-minded people. We like working with those kind of people. Those are your facilities that are more likely to modernize. The ones that have been driving constant change as new technology advancements come out, and have been saying, “Hey we can improve this?” Or, “This is a process that’s broken.”

Often they’ve done an adequate job as they’ve grown, they’ve improved along the way, and the last piece is for them to take the next step and modernize. They need to rethink it all integrated together in the way that it works now, which is so different than the way that it did 30 years before. Those are the customers that are fun to work with.

The ones that put in a system 30 years ago and it was very limited and they didn’t really look to change it, as new technologies came out, they didn’t add on new functionality, those are the ones that are more likely to be about just mitigating risk. They are more likely to say, “Hey, give me the same thing in a new system, I don’t care to modernize.” So that’s your two different types of mentalities, and typically one of the factors that decides between migration and modernization.

That first mindset is the growth mindset, the second mindset is “I’m just going to do my job, and I don’t want anything to break along the way.”

Yeah, and that’s why I say it’s not completely a cost thing. It’s not just cost, it’s cost plus that growth mindset versus risk mitigation. Those are the two factors, and which is the right approach is going to be different for everyone. Because, if you were satisfied with the system you had, and you just didn’t want it to go down, you really wouldn’t want to spend the money on rewriting, adding bells and whistles that you don’t care to use. It’s the difference between a growth mindset and a “Hey, this works, don’t fix what isn’t broken.”

In terms of helping with planning, obviously small, medium, and large projects are going to have different time frames for each one. What might be some minimum and maximum estimates for those, how long it would take from start to finish?

If you’re doing a simple migration, you have a small system and you just want to get to the latest and greatest hardware or software platform, usually you’re looking at about four to six weeks of engineering time, which might not always be linear. That’s usually we’re handing over the keys of exactly what you had before. There’s no differences, there’s just an upgraded hardware platform on the back end. That’s a small timeline, and we would just do the design, work on the migration, update the code, and then go out and do the testing and training. There’s not usually a whole lot of training on that, so about four to six weeks of work for an engineer and now they don’t have the risk of failure and downtime.

If you’re going into the modernization, usually, you would look at that differently, like I’ve said many times, as if it was a new project. So a small system, for example, we’ve done a project for a sucrose tank system. There’s a three-tank sucrose skid that basically receives sucrose from trucks, supplies its own header, keeps its temperature, maintains pressure, and that’s about it. It lets other systems pull and open a valve to deliver the sucrose to another process. This is a small, enclosed system with a few racks of I/O [Input/output].

You would do a sequence of operations on how it’s supposed to work, you would do a design to incorporate the I/O into the control system, you would write the program, you go onsite and test out the I/O, and then test the functionality. Usually a couple of days of support and training with the operators and you’re out. Smaller system like that, maybe an additional six weeks of engineering time to get that done, once you’ve completed the migration.

For more mid-sized systems, like sand filters, of which we have a case study, that’s a bigger and more complex system. Four tanks, with detailed multi-step sequences, like the backwashing skid, and a lot more equipment, but the control is mostly repeated. The control, the functionality, the visibility is all shown pretty much the same on each tank, which makes it a little simpler. We could modernize a more complicated system like that, in a few months worth of time. About double the time, but potentially completed in the same time for the calendar with two to three engineers.

Then you could get into big, multi-line systems, like the cookers, and the flour mills, those are more in the four to six month range. When it came to the cookers, that was a 7-line production floor with tote dumping, cooking, cooling, oil coating and tank storage. The flour mills project had hundreds of devices integrating into multiple processes, with many different control loops. All these different processes all put together in one line, those more complicated systems you’re looking at more like a four to six month modernization with several engineers on the job.

This has been a very enlightening discussion. What else do you think people should know, to take away from this conversation?

I just want to point out that, in addition to all that we’ve discussed, there’s often a pretty clear case for a positive ROI on modernization projects. You can justify the spend by integrating different data history and visualization packages that prove out better consistency, increased productivity, and reduced maintenance troubleshooting. Sometimes it’s easier to see ROI justification for a migration, you know, improving to the latest and greatest hardware and software, and minimizing the risk of some dollar amount of lost production time. The improvements, though, in the modernization phase, are all about the upside. There’s a lot of potential that may not be quantifiable at the outset, but will be clearly evident after the project is in place and better decisions are being made, better efficiency is demonstrated, and reduced costs are dropping to the bottom line.

Thank you!
Very welcome.

A Controls Modernization Q&A (Part 1 of 2)

Recently I had a talk with one of our business contacts at Atomic Revenue. He was a little confused about the difference between “Legacy Controls Migration” and “Controls Modernization”. QSI has expertise in both, so I decided to have a Q&A session to help him (and anyone reading) understand the differences and what some of the deciding factors might be when choosing which way to go.

AR: Tell me a little bit about the difference between modernization and migration.

QSI: So a migration you would typically call a “legacy controls migration”.

The first step is a migration, as it sounds like, where you’re really replacing old legacy hardware or software, because you’re trying to eliminate a risk factor. You’re moving that to the latest and greatest platform. It’s an upgrade. You’re upgrading to a newer version of hardware or software to mitigate the risk of downtime.

The manufacturers, they don’t support things forever. What you have may be a version of the software that’s just old, or the hardware is “silver series” which means they’re going to stop making it and supporting it. A lot of times that’s when people start looking into a migration. They’ll think, “Okay, I’m not going to be able to buy this, if my PLC-5 goes out, or it’s going to cost an absurd amount of money to replace this old hardware.” That’s usually when you get people wanting to migrate.

A lot of these systems were designed 2 to 3 decades ago. Many technologies & devices available today offer improved functionality, simpler configuration, cleaner programming integration, and more data for the control system.  Many of these devices didn’t exist when the system was originally installed, so you could have designed a great system back then that would be hard to manage and have very basic functionality now. Your productivity and efficiency would have a ceiling, and your ability to scale and upgrade to new devices would be limited. Therefore you migrate, in order to get to the latest and greatest hardware.

Some companies will stop there, because they no longer have the risks. “Modernization”, then, is an extension of a migration. It’s what you would do next.

Modernization is when you would rethink the entire programming and functionality of that system. You could modernize without migration, if, for instance, you put in a system two years ago on a limited budget, maybe you just got some fundamental controls up and running very basic. At that point you might say “I want to start this whole thing over, this is not what we hoped it would be,” and already have had the latest and greatest hardware, it’s just pretty rare. Usually you would see the migration first, and it would lead pretty quickly into a modernization.

At that point you rethink all the programming, you rethink all the overall control functionality of the system, understanding all the new technologies, the new capabilities, the data collection, all the different improvements you could make. You’re essentially starting it over as if it’s a new engineering project. It’s one thing to have 21^st century tools, but it’s another to actually use the full functionality of 21^st century control systems.

An analogy might be to a home computer. If you have a 10 year old computer, you can get new hardware, such as a faster processor with a new operating system.  This would be equivalent to a migration. Rather than installing new versions of your favorite applications, though, you still re-install 10-year old programs.

Exactly right. You aren’t getting all the benefits of a new PC. You wouldn’t upgrade to a faster hard drive with more RAM, and then use photo editing tools from 2007. A good way to put a bow on migration, when you do that, you’re essentially replicating the system they had before. So you’re handing it back over without any functionality upgrades. So the processor is better and has less risk of failure, but you have the exact same system and performance, ignoring the technology gains of the previous decades. Same control, same alarming, same everything you had before.  You’re not including any new technologies, any communication advancements, new programming standards, or improved PID control. You’re not actually using the new tool, you’re just going to the new tool.

If you take your computer analogy, you upgrade it to the 21^st century, then you sit down with your hard wired keyboard and mouse and start working with the 20 year old version of Excel, or maybe playing Oregon Trail [both laughing], you’re doing everything you did 20 years ago.

And you’re not taking advantage of any of the things that are available in the broader market, in the broader world.

Right.

It would be taking a very single-minded view, “This is the thing I’m going to do with my system to protect against failure,” as opposed to a modernization is going to open up a lot of opportunities for you to advance.

Yup. I like the computer analogy. You know, you’re worried the old computer is going to break down, so you get yourself a new computer, and you do the exact same tasks. No better, no worse, just with no risk of it crashing and keeping you from doing your work. So I always look at a migration customers as having a set of risks that they’re trying to avoid.

Modernization is saying, “Hey, if I were to do this whole project over today, how would I do it?” And that’s a completely different mindset. That’s why I say it’s really a continuation of a migration. I like to think of them as a 1-2 combination.

Are there signs that indicate one is preferred? Clearly you would say that if you have an old system, first you’re going to go with a migration and then a modernization, and that’s kind of what you’ve already said.

It’s really cost-sensitive. If you’re asking me, what makes somebody’s decision to do one or the other, the first issue is cost. Let’s take two different perspectives. One has a limited budget and only wants to prevent potential risk of downtime. The other is looking at it from a optimizer or efficient operations point of view and saying, I could potentially get higher productivity out of this system. They’re looking for better efficiency, less waste, better data. There’s a lot more  that you can get out of a system if you were to do the project today versus 20 years ago. And those people have that growth mindset to want to do the modernization versus the, “I’m on a budget, and I just want to mitigate my risk.”

So where do you start with a modernization project? I mean, let’s assume that the migration happened, we’ve done the 1. Now, with the modernization, where do you start?

Great. Now you’re on the latest and greatest hardware platform, and you’re saying, whether this was an old system, or a new system that didn’t get put in right, you would treat it as a brand new project. From a control standpoint, you would get into the design philosophy, your I/O, which devices were used to perform the controls that you want to perform, you would probably even redesign the control cabinet to fit their physical requirements, and you would do the sequence of operations to rewrite the entire application.

You would even sit down with their best operators and make sure you understand the ones that get the best results, versus the ones that get poorer results. You would then take those techniques and mold them into the sequence of operations, and then standardize that for all other operators. That would be your approach.

Are there some individuals who are working in a production facility that can get different results out of the same equipment and the same processes? That sounds like what I heard.

Absolutely. And that’s really what sparks that growth mindset. A manager or efficiency engineer would see something in their data, whatever limited data they have, they would see that they got better results from one person over another. They would ask “What are you doing?” and try to replicate that. Therefore one big driver of modernization is seeing different results from different operators and having a desire to standardize around the way the best operators perform those process.

We can program that all in and automate those best practices. Now you’re at the latest hardware and software platform, and there are more integrations, and there is more technology to get that done. So searching for more consistency in operations is one big driver.

And the other mindset is you’ve just got a plant manager or an engineer, that just knows about controls, and understands there are better ways to do it, so you get a lot of continuous improvement engineers inside a facility, or you know, lean manufacturing, Six Sigma types that will just look at the processes and say, “This is outdated, this could be much better. We’ve collected some data and we have this percent downtime and we know that we should be other.”

That’s the other kind of observation, you’ll have an engineer in the facility that’s looking at the bottlenecks of a line and pushing to change out the system. We see both, and again, for each one you basically approach it like a new project and again it depends on their budget.

If you only want to modernize the control, you may not change out any devices. But if you’re really going to do a modernization, that facility would sit down with a mechanical firm, and they would look at the devices they had on their line, and they would ask, “Are there better devices out there? Are there better valves, are there better conveyors, are there better burners?”

Whatever they need for their processes. Then they would come to us with brand-new device list, and we would plan out the control system, the I/O, the application, then start defining the functionality of the system. So again, even a modernization could have, multiple levels, depending on what they want to do.

Okay. You know, we talked about some of the obvious benefits. Reliability, flexibility, upgraded control systems. What are some of the not-so-obvious benefits? You touched on a couple of them. One of those non-obvious benefits is that you can improve lower-performing operators by giving them the tools to do it in the same way as the higher-performing operators. That would be one. What are some others?

Definitely consistency. Plus being able to automate more processes, being able to replicate and standardize performance of your best operators, things that were obvious to them. They might just have an intuition, you know, “do this first, do this next,” but you need to get that education to everyone.

Think back 20 years and the capabilities of everything were so much lower. Then they would just get their systems running. Even the user interfaces were not even close to the same as today. A lot of time they had manual push-buttons instead of HMI touch-screens. Now, flash-forward 20 years later, it’s so different. Think of the changes in video games; back then it was a very simple input pattern, all you could do was say “up” or “down”, something like that. Now you have all these new technologies that you can integrate in. So you’re rethinking the whole operator interface, which allows you to standardize on your best operators.

Another one is scalability. It kind of goes along the same lines. There are new technologies, new analysis software, even data historian packages that allow you to collect data on how well things are running, all these things integrate much more seamlessly into an upgraded platform.

You could continue to modernize by saying “I want to rewrite because back then, we only had these tools and now we have these tools.” Then you could take it a step further and say, “I also want to replace all these devices, because I know there are much better devices out there and I could get even better control.” So again, there’s different stages of what you can do so scalability is important. When you modernize a system you should be able to integrate new technologies much more seamlessly.

That’s the end of the first half of our conversation. I was really glad we got into an explanation of the different mindsets between simple downside avoidance and the growth mindset . Next time we’ll get into Frankenstein, spaghetti code, and the ROI of each.