A pickle — that’s what your obsolete distributed control system (DCS) has placed you in. The tried-and-trusted system that’s been running your facility for years or even decades is becoming a liability. Perhaps it can’t adequately support digital transformation and Industry 4.0 efforts, or poses risks from cybersecurity vulnerability threats or spare parts shortages. You realize migration is necessary but don’t even know how to start such an important yet complex project. You need a plan and the right crew mix to execute it.
A good migration plan is crucial for success. Just like you wouldn’t build a house without a blueprint, you shouldn’t start a control system migration without a plan. A control system migration is a rare event at a plant. It often is the largest project a running facility will undertake and tends to involve first timers — participants who usually only experience a migration project once in their career. Depending on the project’s complexity, you may need to engage a third-party partner knowledgeable in most control-system and other platforms and third-party interfaces.
Every project executed in a running (brownfield) facility invariably poses risks, as any change or improvement in one area could cause unforeseen problems in other areas or upset production. A DCS can span the breadth of the entire facility; so, modernizing most likely will involve every instrument and control device. Instead of working on upgrading one area, you may need to replace all the controllers in your facility. Often, this requires recreating all the operating screens and rewriting every line of control code. However, in the end, the focus and time spent on the process control system and the upfront planning process provide a rare opportunity not only to significantly improve operations but also to set a new standard.
As a project moves forward, your ability to alter the outcome decreases while the cost to change increases (Figure 1). This especially holds true if you’re making an improvement or fixing a mistake. A favorite quote of mine is “Bad news early = good news.” It sounds counterintuitive at first but identifying and communicating a problem early is always better than doing it later. A good migration plan not only reflects this reality but also shifts key decisions to as early in the timeline as is practical.
Most major chemical companies follow a system of funding toll gates and upfront engineering activities (Figure 2). Such a front-end-loading (FEL) process forces the project team to invest enough thought and engineering before it releases the significant capital required to design and execute the project. This structure and cost control is a good thing. It’s more than just additional approvals and hoops to jump through; it focuses the right amount of quantitative investigation early to foster making good decisions for the project and the business.
The FEL Process
Let’s now look at each step in the process more closely.
FEL0/1 — Justify and weigh the risk. This is where the project is formally justified. Business stakeholders evaluate the risk of doing the project versus doing nothing. It typically involves developing a rough order-of-magnitude (±50%) estimate of the installed cost. The key takeaway from this step is a decision on whether the rough total cost of the project outweighs the cost of doing nothing.
This process can be particularly difficult when doing a control system migration. If the only driving factor is obsolescence, which has no inherent return on investment (ROI), it can be a tough sell to a leadership team that primarily is focused on the bottom line. The key to making that argument is to quantify the risk of not upgrading. For example, you can point out the cost of increased downtime as well as lost opportunity because your company is missing out on modern capabilities that a new system would provide.
FEL2 — Use data to make major decisions. This step is all about the options and leveraging data. It focuses on identifying and formalizing a project’s major decisions. These could involve:
• choosing the control platform to use;
• breaking up the project into multiple phases;
• upgrading or enhancing control strategies; and
• consolidating or improving control rooms.
Figure 1. Changes become harder and costlier, and impact the schedule more as a project progresses.
It is important to do enough engineering to provide the essential data necessary. If you are wrestling with the decision of which control system platform to use, comparing not just total installed cost (TIC), i.e., the cost to put the system in, but also the total cost of ownership (TCO) is a best practice to ensure lifecycle costs are accounted for and clearly communicated to all key stakeholders. The output of an FEL2 typically involves a ±30% estimated TIC. This gives enough quantitative definition to help make key decisions but doesn’t require as much engineering and cost as the more-thorough FEL3.
System complexity, the impact of downtime related to the cutover, or constraints on capital spending or resources can drive the decision about whether to break up the project into multiple phases. The cost, schedule and risk impacts of splitting up the migration are important considerations. It’s always more efficient to complete the entire migration at once. However, if you must split up the project, you should consider factors such as the implications of hidden communications between migrated and un-migrated systems. Also, a phased approach often will require the operators to interact with separate human-machine interface (HMI) systems. Planning is crucial to successfully operate the facility between the phases.
Figure 2. With a solid plan, the project team can focus on the capital funding required for each FEL phase.
The biggest risk and the biggest reward for the project can come from upgrading or improving control strategies. To minimize risk, a common misconception is that the goal should be a “like-for-like” migration. This implies duplicating the old control-system code in the new system. This approach, if strictly followed, poses two distinct problems: You’re losing the advantage of the new system’s capabilities; and there may be hidden differences between the old and new system that can be difficult to root out. This doesn’t mean you should start over from scratch. The old system often has years of operating lessons and experience built into its code. The challenge is examining the operation of the old system and making the most of the new system. The task in the FEL2 is to weigh the rewards of the improvements against the risk of making the improvements.
The FEL2 phase also should address other questions such as whether to consolidate control rooms, and whether to upgrade or replace the uninterruptible power supply, environmental systems, etc. Like the other considerations stated above, this demands enough engineering effort to make the right decision at this stage. Changing direction later can cost substantially more.
FEL3 — Define scope for project execution. Here, the focus is all about defining the scope, costs and schedule to ensure successful project implementation. The main objective is to minimize risk. If a decision was made in the FEL2 to break the project into phases, then a separate FEL3 usually takes place for each phase. At this stage, you don’t want to get into detailed design but you must put forth enough effort to produce a thorough estimate of what will be required. An FEL3 typically targets a ±10% estimate of TIC. The FEL3 involves a variety of crucial tasks:
• Developing a project execution plan, including:
◦ engineering scope of work — detailed tasks that must be done to execute a successful project;
◦ preliminary commissioning plan; and
◦ project schedule;
• Defining responsibilities via an “RACI” matrix:
◦ Responsible — who is responsible for the execution of the task?
◦ Accountable — who is accountable for the tasks and signs off the work?
◦ Consulted — who are the experts who must be consulted?
◦ Informed — who are the people who must be updated on the progress?
• Performing a risk assessment that identifies and ranks the risks to the project and ensures risk mitigation plans are in place;
• Completing DCS equipment architecture, bill of materials and specifications;
• Finalizing the list of inputs/outputs;
• Producing final TIC and TCO estimates; and
• Including optional preliminary detailed design elements like:
◦ piping-and-instrumentation-diagram updates;
◦ high-level functional descriptions;
◦ instrument specifications; and
◦ cable schedules and other installation details.
A successful FEL3 requires participation by the right members of the project execution team. This means involving people who understand and can implement a project as they will be the ones helping to execute the work.
Another term for the definition of work done in an FEL3 is a front-end engineering and design (FEED) study. This study commonly is just a scope-definition activity prior to a fixed-price proposal. In many cases, the two terms — FEL3 and FEED — are interchangeable.
With adequate involvement from the project team in the planning (in particular, the FEL3/FEED), the project is set up for success. You have a good plan, now you must follow it. Strong, proven project-management methodologies (i.e., management of change) will keep the project on track and on budget.
As you move through the FEL phases, don’t neglect any sustaining services necessary. It’s never too early to start planning for any support you might need after the project is complete. Service and support contracts, spare parts and patch management tremendously impact the long-term value derived from a new system. If you wait until the project is underway to consider these plans, it’s often too late. Consideration for long-term support should start in the FEL2 stage, if not sooner.
Getting More From The Plan
All this planning is crucial to help minimize the risk of the project going wrong. In addition, it can reveal opportunities and areas for the project to go right, if you know where to look. Four aspects deserve particular attention:
1. System code. Migrating your existing code from one system to another is a big risk. You have countless hours and intelligence invested in running the facility, all of which is hidden in the current control code. There’s no “easy button” that transfers this code to a new system, despite what some people might say. Extraction tools can get you part of the way there but, to make the most of your new system, you’ll want to take advantage of better ways to do certain tasks. Beware of the phrase “like-for-like,” it seldom is the most useful plan. The best practice is to extract the old code into a readable format that can be evaluated by operations staff and engineers for improvements and then coded into the new system.
2. Control strategy. Upgrading the control strategy can have major impacts on facility operation. Your legacy system likely performed well but was hindered by some constraints. A new modern control system often is capable of more. Taking advantage of improvements (e.g., advanced regulatory control) can bolster the bottom line while new control methods (e.g., state-based control) can improve turnaround time and automate responses to critical situations.
3. Operator experience. Shiny new computers surely will enhance the operator experience. However, implementing high-performance HMI (HP-HMI) graphics can bring more benefits than those originally targeted by the abnormal situation management standard. HP-HMI graphics aren’t just gray-scale backgrounds that call attention to alarms and abnormal situations. A graphics workshop where you plan out the screens and interface with the operators can have huge benefits to the way they operate the facility. In my experience, a graphics workshop can streamline the number of screens by as much as 40% and reduce the cost of the overall migration.
4. Security improvements. Strengthening the security of the control system may have been a driving factor in the project justification. However, even if it wasn’t, the advances in firewalls, threat detection and patch management of a new system can provide secure remote access and other benefits. Improved security also can open new doors to what you can do with the control system. A secure connection to the business network can enable more direct communication from the control system to business systems like production scheduling and maintenance management. Also, secure connections from the outside to the control system allow safe remote visualization and troubleshooting of production issues as soon as they appear.
Master Migration
Your legacy control system has served you well for a long time but eventually will require replacement. This is not an easy task; a good planning process and the right team are the keys to success. Where lack of resource bandwidth is an issue, a third party can help you define a control system plan, scope and budget. Look for a partner that offers sustaining services and won’t just walk away once the project is underway and started up. With a planned approach and the right expert resources in place, you not only can minimize the risks associated with modernizing a control system but also can ensure you’re getting the most benefits and highest ROI from your new fully integrated, highly functioning control system.
SCOTT HAYES is Baton Rouge, La.-based DCSNext Portfolio Manager at Maverick Technologies, a Rockwell Automation company. Email him at [email protected].