The truism given in the headline may seem trite but still bears repeating and reinforcing. It certainly applies in troubleshooting and in plant operations. If you want your measurements to mean something, they must meet two criteria. First, the measurements must relate to something important. Second, the person responsible for the measurements must find it easier to make correct measurements than incorrect ones, and must consider them so important that any temptation to take shortcuts is unthinkable.
[pullquote]
My earliest memory of the importance of that dates from high school. I was reading a book on the steamer wars in New York in the late 1800s and early 1900s. A full chapter was dedicated to the shameful disaster of the steamer “General Slocum.”
Steamers had to carry cork life vests. Regulations in 1904 called for checking these life vests by weighing them. Assuming they were filled with cork, enough weight would ensure a minimum buoyancy. On June 15, 1904, the steamer General Slocum caught fire. Before the day finished 1,021 people had died and others were never found. The official report into the disaster [1] starts with a letter from President Theodore Roosevelt that bluntly states: “The Department of Justice has secured the indictment of the manager and three employees of the Nonpareil Cork Works of Camden, N.J., for putting upon the market compressed cork blocks for use in making life-preservers, each of which blocks contained in its center a piece of bar-iron weighing several ounces. This last offense was of so heinous a character that it is difficult to comment upon it with proper self-restraint.” While the iron bars in the cork weren’t the cause of most deaths, they certainly compromised the buoyancy of the life vests.
Figure 1. A fire on June 15, 1904, on the “General Slocum” cost over 1,000 lives. Source: Government Printing Office.
What was measured was the weight of the life vest. It weighed the correct amount despite containing some iron instead of cork.
Advanced process control (APC) systems can add large benefits for modern plants. They continuously optimize against plant constraints. They even may directly calculate factors such as $/h of enhanced profitability or $/h of loss due to non-optimum operation. Yet, in spite of their potential benefits, some plant operators resist them — often for good reason. The automation industry has a history of over-promising and under-delivering with advanced controllers. Operators may turn off an advanced controller if they see few benefits or if the controller creates problems for them. Nevertheless, even imperfect APC may provide some value.
One way to foster the continuing use of APC is to measure the fraction of the time the software is controlling the process. More than one operating company measures the amount of time advanced controllers run and then pays bonuses to the plant staff based on the hours running.
Of course, if you measure hours, what you get is hours. If the advanced controller had problems that made the operators turn it off, these problems don’t just go away but the time-in-service measurement provides no insight about them. (In their quest to get the bonus, the operators invariably mount ongoing pressure to get the problems solved another way.)
When trying to troubleshoot a process with an advanced controller, you can’t do much without some critical basic information. You should immediately check three things:
• Is the process operating within the correlation range of the controller?
• How much is the controller allowed to vary the controlling variables?
• What are the interaction parameters in the controller?
If the process is operating outside the correlation range or calibrated conditions of the controller, there’s not much you can do until the controller gets updated. This is its own huge topic outside the scope here. A relatively recent CP article “Overcome Fear of Advanced Process Control” [2] describes some issues and warns: “The benefits of APC can degrade over a short period of time if the application isn’t properly monitored and maintained.”
The other two tie back to you get what you measure.
If all the control valves have extremely limited ranges of allowable variation, the plant doesn’t have an advanced controller. If the controller can’t move anything, it isn’t optimizing.
One of the most common uses of APC is to handle systems with a large amount of interaction between parts of the process. Most advanced controllers explicitly deal with this via some type of interaction matrix. Review the matrix when the controller is acting oddly. If the matrix shows extremes in interaction parameters, the controller has lost much of its purpose.
At one extreme, highly correlated parameters are essentially the same as ratio control between variables. At the other extreme, highly non-interacting parameters are like splitting the advanced controller into smaller parts. In some cases, the advanced controller is nothing more than a conventional collection of single-input/single-output control loops hidden inside a mysterious box.
The operators have responded to their problem (APC that doesn’t work) and their incentives (more money if it stays turned on). They continually harangue and press the person setting the tuning until the system doesn’t create problems for them. The right answer is to fix the control system so it makes the operators’ job easier, not harder.
When troubleshooting, always keep in mind incentives and measurements can determine how the plant is run.
REFERENCES
1. “Report of the United States Commission of Investigation Upon the Disaster to the Steamer ‘General Slocum’,” Government Printing Office, Washington, D. C. (1904)
2. Nordh, P., “Overcome Fear of Advanced Control,” Chemical Processing (November 2015)