Leverage AI effortlessly in industry without specialist knowledge

Artificial intelligence (AI) is an extremely generalist and at the same time successful technology for automating processes. It is capable of pushing the boundaries of what has been feasible to date – automation based on traditional algorithms. Nevertheless, the advantages of AI will only become established in industrial applications when the corresponding AI models can be used easily without the need for specialist AI expertise. This is precisely what prompted Beckhoff to develop the TwinCAT Machine Learning Creator software.

Classic algorithm-based automation relies on fairly rigid constructs – almost like a set of rules. If situation A occurs, respond with B to get the desired result C. In other words, the path from a defined situation to a desired result is preconceived and then implemented. AI-based methods, on the other hand, use examples to automatically learn the path from the situation to the result, so it does not have to be explicitly thought up by humans and transferred into an algorithm.

The industrial landscape boasts numerous potential AI applications, with AI-enabled visual inspection currently at the forefront. These include end-of-line inspections of manufactured products, the sorting of (often natural) products into quality or other property classes, and optical process monitoring and classification. Specific examples of this broad field of application include:

• performing final inspections of a metallic body with regard to its shape and/or surface quality
• sorting natural products such as fruit, wooden surfaces, and wool into different quality classes
• sorting waste for recycling
• monitoring process zones – e.g., during laser welding
• handling visual localization tasks, such as those designed to locate and grasp specific objects

AI-based methods for tackling these tasks exhibit a stand-out property in that, provided it has been properly trained, the learned algorithm is highly resilient to variances in the input data. This means that even within its limitations, a well-trained AI can also effectively handle unfamiliar situations. Whether it’s defects in manufactured products, anomalies in a laser welding process, or wood surfaces that never look quite the same – the algorithm has to be able to contend with all of these unknowns and more.

Given the immense potential at stake, the real challenge facing industrial companies today is the lack of skilled workers who can create AI models on an industrial scale. Data science and machine learning degree programs have now become commonplace at colleges and universities, yet the demand for AI experts far exceeds their availability on the current job market. What’s more, AI experts can only successfully solve automation challenges together with an automation or process expert. This is where Beckhoff comes in: The TwinCAT Machine Learning Creator automates sophisticated AI training processes, which directly empowers automation or process experts to create AI models themselves. This makes the potential of this technology directly applicable to everyone.

The Beckhoff AI ecosystem

Beckhoff offers a complete ecosystem for industrial AI applications with a clear focus on the execution of AI models directly on the industrial controller (in the PLC). Various sensors can be connected to the control system via the EtherCAT fieldbus and corresponding EtherCAT network devices. An extensive system-integrated range of vision hardware is also available, including industrial, robust cameras, industrial-grade lenses, and illumination devices. The sensor information is transmitted to a PC-based controller where it can be processed directly – including with AI. PLC-integrated execution modules for trained AI models are available for precisely this purpose in the form of TwinCAT Machine Learning Server, TwinCAT Vision Neural Networks, and TwinCAT Neural Network Inference Engine. These can access both the computing resources of the CPU and those of an optionally available NVIDIA GPU. The AI execution modules load trained AI models that have been stored in the open standard "ONNX". This gives the user the freedom to train AI models in any training environment and then execute them in the TwinCAT controller. With the C6043 ultra-compact Industrial PC, Beckhoff offers scalable hardware with an integrated embedded GPU from NVIDIA in line with industry standards. This makes the entire Beckhoff ecosystem optimally geared toward the integration of AI models into the machine’s control level.

Control-integrated AI models offer the advantage that their results can be used directly in the control system to perform actions. If, for example, a component is ejected, specially treated in downstream processes, or passed again through the process that has just been completed, the information can be calculated in the machine control system and used immediately. In addition, cost-intensive additional hardware devices with complex interfaces for control, independent maintenance and update schedules, and separate IT security protocols are no longer required.

Automated creation of AI models

In line with the philosophy of open control technology, Beckhoff has designed the existing PLC-integrated execution modules for AI models irrespective of the AI training environment used by supporting the ONNX standard. An ONNX file describes a trained AI model as a sequence of operators with associated parameters. These description files can be loaded with TwinCAT 3 Functions, such as TwinCAT Machine Learning Server, and then executed from the PLC; however, the relevant AI frameworks such as PyTorch or Scikit-learn, which are generally used to train AI models, are aimed at specialist AI experts who prepare training data in the Python programming environment, create AI model structures, and then train AI models.

With TwinCAT Machine Learning Creator, Beckhoff now offers a much simpler approach in the form of a web-based interface that guides you through the steps of data upload, model training, model analysis, and download. Target groups include automation and process experts without a specific background in data science, with the aim here being to standardize the training process for AI models.

The data upload

The entire concept of machine learning revolves around learning by example. With this in mind, it is essential to have a clean, representative data set that can be used to learn the task. This usually requires an annotated data set: in the field of image classification, for example, this means that a certain number of sample images have already been classified by a human. As a result, each image has its own kind of label, which represents the desired result. The relationship between image and label is established by a label file, which, in the simplest case, is a table containing the file name and corresponding label.

The data upload is open and supports various image data formats and label file formats. This means that users are free to choose a labeling tool (if one is required). Work is currently in progress to use TwinCAT Analytics Data Scout as a labeling tool to achieve data integration from the TwinCAT controller to TwinCAT Machine Learning Creator.

The AI model training

The configuration of an AI training session is kept lean, which involves a model name being created and a data set (or several data sets) added to the training process. All other configurations are optional and allow the AI model runtime behavior on the TwinCAT controller to be specified if required. If a Beckhoff TwinCAT hardware platform and the TwinCAT software on which the AI model to be created is to be executed are specified, users can specify a maximum acceptable execution time for the AI model. This information is taken into account during the AI model creation process. If no maximum execution time is specified, optimization is based exclusively on AI model performance (generalization capability).

The model analysis

AI models, especially the deep neural networks (deep learning models) created with TwinCAT Machine Learning Creator, have very good generalization properties. This means that the expected performance of the models is also very good; however, neural networks are "black boxes" whose functionality can only be deciphered through specialized analysis methods, rather than directly. These methods are also referred to as "Explainable AI".

The analysis methods for a trained AI model are diverse. The software automatically separates the uploaded data set into training data, which is used for model training, and test data, which is used for model analysis. The test data includes unknown cases for the AI model, where the result is known via the labels. This makes it possible to calculate statistical values and show how often a model is correct and how often it is not. A confusion matrix, for example, provides detailed information on how "true" labels and "predicted" labels are distributed. Confidence values can also be calculated for each model execution and displayed statistically. It is even possible to generate an attention map for each model execution, which shows which image regions were used for classification when superimposed over the input image.

The Explainable AI methods serve to increase the acceptance of the AI model. After all, only those who gain a more fundamental insight into how the trained model behaves will ultimately be prepared to use the models.

The model download

Once an AI model has been trained and is ready to be integrated into the machine control system, it can be downloaded from the platform as an ONNX file. This means that the AI model is not tied to execution in TwinCAT and can be deployed as often as required on any platform. Furthermore, the complete PLC code for TwinCAT can be downloaded from the platform in PLCopen XML format. This includes the complete process flow from image acquisition and image pre-processing through to AI model execution and post-processing. The transition from the training tool to the TwinCAT PLC is correspondingly seamless.

Another advantage of exporting an ONNX file from the training platform is the potential involvement of AI experts. These experts can use TwinCAT Machine Learning Creator to obtain an initial AI model quickly and in a standardized way. The result in ONNX format can then be imported into individual expert tools for continue working with it – for example, for additional analyses or further refinement of the model.

Features and benefits

The TwinCAT Machine Learning Creator offers the following properties:

• simple model creation via no-code platform in TwinCAT
• latency-optimized AI models for real-time applications
• implementation of open standards, interfaces, and best practices
• provision of trained models in the open ONNX standard format
• transparency in the development, testing, and validation of AI models
• in-house standardization and further development of AI models
• ideal for AI-supported quality assurance in image processing

The automated model creation results in the following application benefits:

• AI potential unlocked for all companies
• greater competitive edge through AI, even for smaller companies
• support in view of the increasing shortage of skilled labor
• minimized workload and error risk for AI experts
• application-specific data remains within the company and is therefore protected
• faster project development and return on investment