Hurricane Harvey has beaten up on Texas. Hurricane Irma made its way up the west coast of Florida after a brutal march through the Caribbean with yet more hurricanes setting their sights on the Atlantic and Gulf of Mexico coasts. I would be remiss in not mentioning Hurricane Katia, striking the east coast of Mexico, while a massive earthquake struck off the west coast near the border with Guatemala, and the nearly 100 fires burning all down the west coast of North America from British Columbia to California. Through the use of television and social media, we’ve witnessed every moment from the formation of these storms off the shores of Africa and in the Gulf of Mexico, with their steady marches toward their final destinations. My heart goes out to those in the way of these enormous natural events; I can’t help but to imagine the fatalities that might have happened had there not have been the advanced knowledge afforded by the technological innovations in weather tracking and warning systems. Mankind benefits daily from the advance of technology, but, for those of us serving in the trenches, the advance of technology can sometimes have a negative effect on the control systems we work with.
I was recently working on a project to save my company some money on the use of barcode reading systems. Traditionally, we have bought stand-alone barcode audit systems, and, when a good majority of our packaging machinery was relay logic or early PLC vintage, this made perfect sense. However, over my tenure with the company, we have upgraded more than a few of our machines to a new control system. While there are more advanced PLCs available, we chose to match the controllers in some vertical cartoners that we have bought over the past seven years in order to minimize the impact on our parts inventory, as well as the technical expertise needed to support them. Our reasoning was that the PLCs in question used Ethernet for communications, so we should be able to connect to variable frequency drives (VFDs) and the such with the added bonus of using the same programming software used for many of the early PLCs I mentioned.
The first of our new control systems went through the usual trials and tribulations. New hardware always presents its own share of challenges, and this system was no different. The team elected to stay with a conventional hardwired vs. network architecture for the most part, in the interests of getting one under our belts. We are, after all, a user of packaging equipment and this step into the world of the original equipment manufacturer was done with not a little trepidation by our management team. We managed, however, to get through it relatively unscathed and the result was a significant improvement over the relay-based system we replaced.
We quickly moved into a second rebuild and, emboldened by our first experience, stepped into new territory by utilizing the Ethernet network for more than the PLC-to-HMI connection. We included the main-drive VFD in the communications architecture—we had used Modbus-slave on the first system—and after some initial struggles we managed to produce another successful cartoner rebuild. We added so many new innovations that we gave the system a 1.5 designation, much like a software company would.
It was during the launch of this version 1.5 control system that we decided to look into integrating the barcode reader into our rebuild plans. For those who aren’t familiar with food production, a barcode reader is a critical control point (CCP). In order to protect our consumers, we must take measures to ensure that the right product gets in the right package. The stand-alone barcode systems work as designed, but we were less than happy with some of the inherent deficiencies of these systems. Primarily, since it is a stand-alone system, it doesn’t know when the packaging machine is stopped. If it stops with a package in front of the trigger sensor, an alarm will follow. These alarms have to be reset at the source, usually remote from the operator station of the packager. Another challenge to integrating a stand-alone system into a packager is not all packagers have a normally closed stop circuit. That means that it isn’t inherently fail-safe, a critical challenge point for food-safety protocols. In order to make the barcode reader work with the packager, additional control circuit components needed to be added, depending on what packager the barcode reader was installed on. The third challenge for us was the need to periodically change passwords to ensure that only key personnel could control the ability to turn off the barcode reader to ensure that production wasn’t run without this important CCP in operation. By bringing the function of the barcode reader into our control system for the packaging equipment, we could resolve all of the issues listed above and have the added benefit of displaying the current and taught barcodes on the operator screen, as well as messages related to the function of the barcode system.
We elected to go with an image-based decoder. The trend in the industry has leaned toward cameras for a number of years, and it made sense for us to go in this direction. The added advantage of having setup software that gives a visual interface to the barcode being read is a distinct advantage. All in all, we were very happy with our decisions and eager to get a system ready to go. We had a multi-part plan. The first step was to get a camera running on our 70-cartons-per-minute (cpm) packaging line, then expand to our 110-bottles-per-minute (bpm) bottling line and then produce a stand-alone system as we could still beat the commercial systems by a significant price point. Our PLCs had Ethernet; the barcode reader had Ethernet—it was a perfect match. By doing the integration ourselves, we could save the company a whopping $4,000 over the cost of a stand-alone system.
The manufacturer of our barcode reader has been in the industry for many years, and it has been very good to keep both old and new technology in mind as their product has matured. This particular camera had the ability to communicate with programmable controller communication commands (PCCCs), utilized by older PLCs such as the PLC-5 and SLC 500, as well as the assembly object format of newer PLCs such as the Rockwell Automation’s CompactLogix and ControlLogix platforms. With some time spent figuring out the format of the status and command registers—tech companies like to put out documentation that is written by the person who wrote the software, and it isn’t always clear to the reader what magic is happening inside the box—we managed to create an interface between the reader and the PLC. Our production people were happy with the results, and our quality assurance (QA) people had a field day coming up with new ways to challenge the system to match their lengthy wish list. The results were great, and we even went back to the version 1.0 packaging machine and gave them the same integrated barcode reader.
As with many projects that I have the pleasure of working on, my work is often interrupted by the need to support production in our two facilities. I can often get distracted for weeks or months at a time, and it is hard to stay on point with these new innovations. Such was the case with the barcode reader. Stage 2 of the plan involved installing one on our 110-bpm bottling line. Right in the middle of this adventure, our client decided to come out with a bottle that didn’t have a handle on one side. As a matter of fact, the bottle was perfectly symmetrical, meaning that it could be presented to the barcode reader with the code facing or opposing the reader. The vendor of the camera must have been thinking about such scenarios because this one could actually connect up to four readers, via the communications interface, and any one of the readers seeing a barcode would be the same as the master camera seeing the code. Put a camera on each side of the bottle conveyor, and problem solved. Well, that didn’t quite work out as planned. It seems that 70 cartons per minute isn’t the same situation as 110 bottles per minute when there is only a minimal gap between successive bottles. Our reader could read in time, and our PLC had a similar scan time to the camera decode time, but we kept missing reads. Queue the aforementioned interruption to the thought process, and our company embarked on the largest internal project in our history. The project was a complete success, three high-speed cartoning lines with a completely automated material delivery system. Eighteen months later, we finally had an opportunity to get back to the task of barcode reader systems, but, instead of the 110-bottles-per-minute challenge, we were faced with putting integrated barcode readers on the newly installed packaging lines.
Here is where the mixing of old and new technology comes into focus. During that 18-month span between project advancement opportunities, we came to the realization that our original camera choice, while functional, was hard for our maintenance folks to understand and set up. Our camera vendor, of course, had an answer to that concern. The same camera, but in a big-brother version, uses liquid-lens technology to provide for auto-focus and auto-contrast at the touch of a button. This meant that we weren’t limited to a 4-inch focal point on the camera. Our setup people could get tripped up on this requirement time and time again. The great part was it used the same communications interface as the original camera, so it was a win-win situation. We swapped out the cameras on the original two machines and even made stand-alone versions for three other systems that weren’t scheduled for controls upgrades in the near future. Back to the task at hand—the cameras on our high-speed lines. New camera means better results, right? Not a chance. The same issue reared its ugly head.
Now, I’m the kind of guy that takes great pride in solving problems. I don’t like to let anything beat me. It’s just a camera, right? What can be so hard about this? The vendor says these cameras are capable of more than 400 captures per minute. I’m only looking for about one-third of this, so it can’t be that hard. Back into deep-study mode I go. I start at the beginning of the manual and make my way through it. Multi-function camera. Two interface cables—one for power/signal, one for Ethernet. I picked Ethernet because, well, it’s newer and you can do everything with the Ethernet that you can do with the wired system. In fact, you can even use power over Ethernet (PoE) to power the device. Down through all the settings I go, looking for something to save me more time on the decoding. Smaller field of view, larger field of view, contrast, limit the decoding to a single code type (UPC)—all the stuff that makes the decision come faster. I get 35 ms per decode. That is great as my total exposure time for each carton is 190 ms. Plenty of time to get the decision over to the PLC for processing. Heck, my PLC scan time is also 35 ms, so, worst-case scenario, it will take 70 ms, if I assume that I just missed a decision.
Try it again on the machine. Failure. Try again. Same result. I am faced with every good controls designer’s worse nightmare—I must call the vendor for advice.
The vendor rep is, by admission, familiar with his product but can’t be considered an expert on the PLC programming side. But, hey, it’s just a camera, right? I’ll save you all the long story and report that my helper went down the very same path I went down and, guess what? Same result—it doesn’t work.
We set about talking options. I could always put this camera on the bigger PLC—an automation controller because, to use the application engineer’s words, “Well, they just work with them.” Or I could resort to using the old archaic power/signal connection to send the trigger and get the results—good, no read or mismatch—from the camera. Well, I wasn’t about to spend another $1,200 plus for an automation controller—my boss would kill me—so hardwired would be my next step. My ever-present, right-hand man and electrician quickly hooked up the six wires to some I/O on my PLC, while I exchanged the Ethernet trigger output and three results inputs with the hardwired addresses. Five minutes later, we throw the machine into run mode and hit the start button. Well, what do you know? Successful scans at 150 cartons a minute. Turn up the speed on the machine to 160 cartons per minute. Turn down the speed to 110 cartons a minute. Further reduction to 90 cartons a minute. Not a flinch. High-fives all around. A two-year journey comes to a successful conclusion.
The point of this story is the challenge presented by new technology. Our default response is to favor new technology. “New” means advanced, and we all want to advance. The issue I was dealing with is, essentially, the difference between a programmable logic controller (PLC) and a programmable automation controller (PAC). A PLC has a single processor, through which all functions must pass—program, I/O and communications, all through a single master device. A PAC can have one or more master processors but a separate processor for the communications. The communications module can send packets to and from devices without the need to prompt from the processor. The processor simply pokes the communications module at scheduled times to get the results. The simplified explanation is that events can happen independently of the locked-in method of a traditional PLC:
- read the inputs
- process the code
- write the outputs.
The variable in my original configuration was the Ethernet communications. My PLC uses Ethernet, not even Ethernet I/P, and it accesses the communications chip at a rate that is effectively random. The solution was to rely on the I/O, which is processed each and every program scan. Trigger the reader with a real output, and 35 ms later a result is ready to be read by the PLC using real inputs. The most time between trigger and result is based on the camera processing time plus the PLC scan time, in my case a total of 70 ms.
Sometimes, to get the best results, we need to mix old and new technology—a hard lesson to learn in this age of newer and faster, but a valuable lesson for me. And it is definitely something to consider when designing your next control system.
