Consider an application where five similar, but not identical, sets of parts need to be welded. Without a vision system, five robot stations are required. Each part must be mounted in a costly high-precision fixture manually and then welded by a tool attached to a robot arm that moves with high accuracy and absolute repeatability.
Dan Hebert was senior technical editor for Control, Control Design and Industrial Networking.
When a vision system is paired with a robot, one welding station can weld all five part types, as well as other parts. Each part doesn't need to be manually and precisely mounted, but instead just needs to be securely attached to a servo-driven housing, with the vision system aligning the parts as required.
Vision systems free robots from the narrow restrictions of time and space, providing a host of advantages in a wide variety of manufacturing applications. These advantages enable more widespread use of robots by guiding their actions to closely fit ever-changing and more-demanding production requirements.
See It, Do It
When a robot can't see, the part on which it will perform an action must be precisely placed, requiring a very accurate fixture or positioning system and a high-precision robot. But with vision, close is good enough, as the robot can find its own way.
"Vision systems make it possible to use a less precise and expensive robot for precise placement applications such as wafer alignment, part orientation or part presence," says Bob Fung, vice-president of engineering at Owens Design, Fremont, Calif., an automation services company that creates, designs and builds equipment for the semiconductor, disk drive, solar, flat-panel, LED, fuel cell and mobile electronics markets.
Also read: Case Study: Vision guides high-speed robotic packaging
Figure 1: Vision is becoming a more viable option due to advances in software and hardware along with a corresponding drop in costs, making it more prevalent in applications like this layer de-palletizer.
Source: Intelligrated
Chris White, project manager for automation at CMD, Appleton, Wis., agrees. CMD built an automated packaging system based on a Yamaha robot and vision system that eliminates most of the problems with fixtures, part-positioning, orientation, part defects and product changeovers. "The vision system provides the robot with the position and orientation of the part, allowing the robot to pick and place the product as desired," he explains.
Matt Wicks, vice president of product development at Intelligrated, Mason, Ohio,, a materials handling specialist, says his company uses vision in applications when the material handling operations require more understanding about the environment, material or products. "We recently demonstrated a robotic de-palletizing application (Figure 1) that used 2-D and 3-D vision to determine the location and orientation of cases stacked on a pallet," Wicks says. "Then the robot was commanded to de-palletize the products onto a takeaway conveyor. This type of operation wouldn't be possible without the use of vision-guided robotics."
In some instances, seeing the parts and performing actions could be performed by sensors, but cameras are often a better solution. "As the price of vision systems drops, they're replacing sensors as they provide more functionality and flexibility," Fung adds. "For example, in the past we had to use multiple sensors or move the product past the sensors to determine the quantity and orientation of parts in a carrier. A single camera now can do the same without moving the part, and it can support product changes without changing hardware."
A bakery used a 2-D vision system to guide cake-decorating robots until about a year ago. It upgraded to 3-D to improve flexibility and increase throughput. (See "Robotic Cake Decorator" sidebar)
How Hard Is It?
Adding vision to a robot isn't always easy. "There are challenges with the seamless integration of sophisticated vision, robotics and the accompanying automation systems," Wicks says. "This advanced level of integration requires a collaborative effort on the part of the vision systems providers, robotic arm manufacturers and robotic systems integrators because it requires in-depth knowledge of not only the vision piece, but also how it integrates with the robot and other automation equipment."
Figure 2: A vision system tells the ABB robot where to go to pick up car bodies at Honda's plant in China.
Source: Teledyne
Fung says that most vision systems must be designed specifically to meet the requirements of the tool into which they are to be incorporated. "This increases the overall tool cost as well as the design time," he explains. "There's a lack of generic, industrial-quality vision systems that can be programmed easily to meet a variety of system requirements. Most vision systems are difficult and complex to operate. Much of the system complexity is due to the high image quality required for most applications, as well as challenges involved in accurately measuring parts that don't conform to regular shapes."
Software and lighting pose problems, too. "We've seen advances in both software and lighting for vision systems, but there are many opportunities for further improvement," CMD's White says. "While the software has become increasingly user-friendly, end users still need some proficiency with programming and setting up the cameras when they introduce a new product. Depending on the type of features the customer is looking for, this can be a tedious process of adjusting the camera and lighting to show the intended features. Products have different patterns, colors, sheens and materials that can complicate setup."
Fung agrees and suggests improvements. "Lighting systems must be able to fit into the tool without adding excessively to tool size," he says. "Designers usually have to use very thin, industrial-grade backlighting, which is expensive and further increases overall system cost. Filtering to prevent stray light reflection is also a concern."
Most robot and vision vendors recognize the integration challenges and provide solutions. For example, White explains how Yamaha software helps, saying, "The software program in the Yamaha RCX controller is a modified form of BASIC. The commands are actually quite simple. For instance, a MOVEP command (move to position) tells the robot to move to a given X,Y,Z coordinate. The Yamaha firmware and hardware determine the most efficient path and control all the motors to arrive at the destination as smoothly as possible and within the defined limits."
1. Automates manual steps and processes
2. One camera can replace multiple sensors
3. Speeds production and increases throughput
4. Greatly reduces programming effort to guide robot
5. Increases flexibility of the robot
6. Recipe-driven production reduces changeover times
7. Allows for less precise positioning of parts
8. Less expensive and more precise robots can be used
9. Eliminates expensive fixtures
10. Allows a robot to perform multiple tasks
Steve Zhu, director of business development for Teledyne Dalsa, describes a system the company built at a Honda plant that also uses vendor software to advantage. "Our GEVA1000 vision system and Sherlock software is being used for China's Dongfeng Honda Assembly line, where an ABB robotic gripper is used to grip the car body accurately and reliably (Figure 2). Our GEVA1000 with Sherlock guides the giant ABB robot to locate the car body properly."
Sherlock software tells the robot where to go. "With its search tool, the software can recognize marks and locate the right positions where the grippers should be put on car bodies of different sizes," Zhu says.
HMI software vendors also provide assistance with robots and vision. "The volume of data produced by vision systems operating at high speed can't be handled by most database servers, and specialized data historians may be required just to capture the data stream," says Fabio Terezinho, vice president of consulting services at InduSoft. "The OEM's vision system uses our HMI capabilities to show and ameliorate volumes of complex, high-speed data, especially in a quality assurance (QA) application where parts are inspected by the high-speed vision system at 10 msec intervals."
Some of the integration problems can be eliminated by using a robot with a vision system supplied by the robot manufacturer (Figure 3). "We use Fanuc robots with their integrated iRVision, so we no longer purchase many third-party vision systems, says Tom Spisak, senior controls engineer at integrator Automated Cells & Equipment, Painted Post, N.Y. "Before iRVision, we purchased Cognex cameras and integrated them into the robot cells." Why the change? "Faster integration and easier support from a single manufacturer, which translates into lower costs and increased performance," Spisak says.
Figure 3: An integrated robot/vision system allows an overhead camera on this laser engraver/labeler to find the part for the robot so that it can be picked up and placed in front of the laser engraver.
Source: Automated Cells
If buying an integrated robot and vision system makes everything so easy, you might expect this to be the Next Big Thing in robotics, but there are drawbacks to this approach. (See sidebar, "Integrated Versus Separate," for more on this topic.)
Peering into the Future
Although vision systems are used extensively in various industrial applications, there's room for much more growth if certain improvements can be made. "When it comes to vision system hardware, cost is always an important issue — that is, getting a camera and lens with the resolution and depth-of-field required for the application at an acceptable cost," Fung says. "All too frequently design engineers are forced to compromise between image quality and system cost."
So how about an ultra-cheap vision sensor backed up by gaming software? Intelligrated's Wicks reveals that the vision system his company used in the robot de-palletizing application described above is a Microsoft Kinect — the same $150 sensor used in the Xbox video game. "Low-cost sensors such as the Microsoft Kinect and the recently released Kinect 2.0 provide powerful vision data that can be leveraged by sophisticated software to yield impressive results," Wicks claims.
No one was more surprised than Microsoft when robot experts started adopting the Kinect. An article in Wired magazine reports, "…the company's official response to all this activity has gone from hostility to acceptance to vigorous support… The company is also granting access to the high-powered algorithms that help the machine recognize individual bodies and track motion, unleashing the kind of power that was previously available to only a small group of PhDs."
[sidebar id="6"]
Microsoft might emerge as a major competitor, especially if robot and vision vendors don't improve their software. "Easy-to-use software will bring the barrier down to many users and operators who are interested in using vision for their automation and robots," Zhu says.
Vision and robot systems also need easier communications. But this is happening as various suppliers recognize the need. Marc Wolf, business development manager at B&R Industrial Automation, says Pro Pack Systems in Salinas, Calif., one of its machine OEMs, uses vision to detect position and orientation of incoming product, as well as to sort different products. Pro Pack connects the robot and vision system using Powerlink, and programs it with B&R Automation Studio Software. "Vision over Powerlink enables Pro Pack to use a single network and single PC for the control system logic, vision and motion control," Wolf says.
"Smart machines are the dawn of a new era, wherein integrated vision, sensor and control loops are established to drive repeatable and accurate performance across semi-complex and complex job routines," believes Muthuraman Ramasamy, industry manager at Frost & Sullivan. "There is ongoing activity in universities to enhance the quality of image processing by allowing the robot to understand shapes and geometries in ways that were not possible before."
Industry needs combined with hardware and software advances promise to increase the penetration of vision-guided robots, delivering a host of operational advantages at ever-decreasing costs, with improved performance and ease of use.
