Image processing sensors

Image processing is part of the basic equipment of most optical and multisensor coordinate measuring machines due to its flexible application options and the good visualisation of the object and the measured features. Similar to image generation in visual measurement with measuring microscopes, the measuring object is imaged through a lens onto a matrix camera in the simplified manner shown in Figure 7. The camera electronics convert the optical signals into a digital image, which is used to calculate the measurement points in an evaluation computer with corresponding image processing software. The intensity distribution in the image is analysed. The individual components such as lighting systems, imaging optics, semiconductor camera, signal processing electronics and image processing algorithms have a decisive influence on the performance of image processing sensors[1, 4].

Imaging optics with a telecentric lens lead to the smallest measurement errors. Due to the telecentric lens, the image scale remains almost constant when the object distance changes within the telecentric range (Fig. 9). An aperture ensures that only almost parallel light rays are involved in the image generation for each pixel. This is particularly important for lenses with telecentric lenses for a constant image scale at low magnifications, as these have a large depth of field, which is why they can only be roughly focussed on the lens. Telecentric lenses with fixed magnification achieve the best quality.

Zoom optics offer the greatest flexibility. In conventional zoom optics, the movement of the lens packages is realised by mechanical curved guides (Fig. 10a). The positioning movements of the optical components in the lenses cause losses in accuracy, but these can be reduced by taking suitable measures. The simplest but very time-consuming method consists of repeated qualification after each zooming process. In order to achieve high reproducibility when zooming, motorised linear guides with minimal positioning uncertainty are used (Fig. 10b). The mechanical curved guides are replaced by corresponding characteristic lines in the control software. This allows different working distances to be realised in addition to different magnifications. In practice, magnifications of 0.5 to 10 and working distances in a department of 30 mm to a maximum of 250 mm can be achieved.

By selecting the appropriate magnification, the most favourable compromise between the measuring range of the sensor and the achievable measurement uncertainty can be selected. The working distance can be adapted to the requirements of the measuring object largely independently of this: accurate measurement with a calibration standard working distance with the best image quality or measurement with a large working distance to avoid collisions.

The illumination systems are the basis for every optical measurement and ensure that the features to be measured are displayed with the highest possible contrast. This is most easily achieved on the outer edges of the measuring objects. In this case, transmitted light can be used (Fig. 11a). Flat measuring objects offer ideal conditions. In contrast, with spatially extended edges (prismatic or cylindrical objects), the interaction between the illumination, measuring object and imaging beam path must be taken into greater consideration. The aperture angles (apertures) of the illumination systems and the lenses must be matched to each other, taking into account the application (shape of the measuring object).

In the simplest case, ring-shaped arrangements of light-emitting diodes (LEDs) are used for the darkfield incident light. By switching on different diode groups, the object can be illuminated from different spatial directions and thus optimally adapted to the measurement task (Fig. 11c). With MultiRing® illumination (Fig. 11d), in combination with zoom optics with a variable working distance (see Fig. 10, p. 16 Image processing sensors ), it is also possible to vary the angle to the optical axis over a wide department. In addition, measurements can be taken at a sufficiently large working distance from the objects. Figure 12 shows examples of the effects of different types of lighting. The light sources can be controlled by the operator or – in automatic mode – by the measurement software. In order to be able to measure in a practical manner, i.e. on changing material surfaces such as metal surfaces with different degrees of gloss and differently coloured plastic parts, light control is used. It automatically adjusts the lighting to the values specified by the programme based on the light reflected by the object. A mathematical correction of the lighting characteristics (light intensity in relation to the set value in the user interface) also allows the CNC programmes to be used with different lighting hardware with different lighting characteristics, e.g. on different machines or after repairs.

The image processing algorithms used to analyse the image content and determine the measurement points also have a significant influence on the quality of the measurement results from image processing sensors. Today, evaluation is mainly realised using PC hardware and software. In a first processing step, the image can be improved with image filters (optimising contrast, smoothing surface defects: Fig. 13a, b). In the simplest method for determining the measurement points, the intersections of predefined lines in the image with the visible contours of the object are determined, e.g. by threshold operations (colloquially known as Edge Finder). This is repeated one after the other at many points in a previously defined evaluation area (window). This results in a large number of measurement points, which are summarised into a group by the window. However, a separate one-dimensional evaluation is carried out for each individual point determination. The comprehensive two-dimensional information contained in the image is therefore not taken into account. This is particularly disadvantageous when measuring in incident light. Interfering contours caused by surface structures, chipping and soiling can only be recognised and compensated for to a limited extent.

In contour image processing, the image within an evaluation window is viewed as a planar whole. Contours are extracted from this image using suitable mathematical algorithms (operators). Each pixel of a contour corresponds to a measurement point. The measurement points are strung together like a string of pearls. This makes it possible to recognise and filter out interferences during measurement (contour filters) without changing the form of the contours (Fig. 13c, d). It is important for practical use that several contours can be differentiated within a capture range (Fig. 14c, d). In a further step, modern systems interpolate the coordinates of the measurement points within the pixel grid (subpixeling: Fig. 15) and thus allow higher accuracies[5].

Contours larger than the field of view of the respective lenses can be captured as a whole using automatic contour tracking in conjunction with the CNC axes of the coordinate measuring machine (contour scanning). This scanning method is well suited to checking a small number of relatively large contours, e.g. on punching tools. In this application, both the punches and dies of the cutting tools are captured directly on the cutting edge and can be compared with each other or with the CAD data set.

Another method for capturing larger areas of the workpiece is "Raster Scanning HD". Here, the image processing sensor captures images of the workpiece at high frequency during movement (Fig. 16). These are resampled and overlaid to create an overall image with up to 4000 megapixels (as of 2019). In the "in the image" evaluation, for example, 100 bores can be measured in 3 s (see Sensors and device axes). The accuracy is also increased by measuring large areas with high magnification and averaging over several images, which improves the signal-to-noise ratio. The method can be adapted to the requirements of the measurement task (see Measurement during movement).

Image processing is initially only suitable for the measurement of two-dimensional features. Accordingly, the field of application includes all two-dimensional measuring objects such as flat metal sheets, foils, printed circuit boards, cross sections of aluminum, rubber or plastic profiles, prints, inserts, lead frames and chrome masks. If focus methods are also realised with the same sensor hardware (optics, camera technology, etc.) (see focus variation sensors), the frequently used basic sensor type for multisensor coordinate measuring machines is available. By combining both methods in one sensor hardware, many three-dimensional measuring tasks can be solved. Determining the functional dimensions of plastic parts, such as the distance of latching lugs and the geometry of sealing grooves and connector latches, is one of the main areas of application. Other examples of applications include punched bent parts made of sheet metal, watch components, furniture fittings, nozzles for fuel injection, print heads, tools and turned parts.

A swivelling camera head can be used for flexible three-dimensional measurement with image processing sensors. The standard types of illumination described and an interchangeable interface for the fiber probe (see Measuring tactile-optical sensors) are integrated into this. A tilt and swivel head, which is also used for tactile sensors, allows the sensor to be spatially aligned with the workpiece. A further interface allows different optical or tactile sensors to be operated in automatic alternation (Fig. 17).

Another method for capturing larger departments of the workpiece is "HD raster scanning". Here, the image processing sensor captures images of the workpiece at high frequency during movement (Fig. 16). These are resampled and overlaid to create an overall image with up to 4000 megapixels (as of 2019). In the "in the image" evaluation, for example, 100 bores can be measured in 3 s (see Sensors and device axes, p. 89 ff.). Measuring even large departments with high magnification and averaging over several images, which improves the signal-to-noise ratio, also increases accuracy. The method can be adapted to the requirements of the measuring task (see Measuring during movement, p. 95 f.).

Image processing is initially only suitable for measuring two-dimensional features. Accordingly, the field of application includes all two-dimensional measuring objects such as flat metal sheets, foils, printed circuit boards, cross sections of aluminum, rubber or plastic profiles, prints, inserts, lead frames and chrome masks. If focus methods are also realised with the same sensor hardware (optics, camera technology, etc.) (see focus variation sensors, p. 24 ff.), the frequently used basic sensor type for multisensor coordinate measuring machines is available. By combining both methods in one sensor hardware, many three-dimensional measuring tasks can be solved. Determining the functional dimensions of plastic parts, such as the distance of latching lugs and the geometry of sealing grooves and connector latches, is one of the main areas of application. Other examples of applications include punched bent parts made of sheet metal, watch components, furniture fittings, nozzles for fuel injection, print heads, tools and turned parts.

A swivelling camera head can be used for flexible three-dimensional measurement with image processing sensors. The standard types of illumination described and an interchangeable interface for the fibre probe (see Measuring tactile-optical sensors, p. 45 ff.) are integrated into this. A tilt and swivel head, as used for tactile sensors, allows the sensor to be spatially aligned with the workpiece. A further interface allows various optical or tactile sensors to be operated in automatic alternation (Fig. 17).