22 October 2023
Fast, flexible 3D image processing
The IPA researchers worked on several development focuses as part of the project, during the course of which they expanded their expertise as well as the resulting range of 3D image processing-based products and services. One core aspect was the creation of what is known as semantic digital twinning of retail products. This involves using sensors to capture any product in a scanning station, for example. The product is then modeled three-dimensionally based on a wide range of information such as shape, color, and texture. The modeling process, in which Fraunhofer IPA collaborated with the Kaptura company, also involved machine learning methods, a subfield of AI (Artificial Intelligence).
With the help of the technologies developed, it is possible to model wide product ranges quickly and in 3D with high resolution texture. During the modeling process, semantic information such as logos, nutritional value or quantitative information on products is also recognized and digitally collected. The models generated are the basis, for example, for digital twins of a real store, a web store or automated product analyses, such as those carried out in laboratories.
In addition, a scalable object recognition solution optimized for retail has been created, enabling automation solutions for production and logistics through precise optical product identification. Application examples include robot-based bin picking or bin packing, i.e. the packing of goods of all types and shapes. The latter task, also known as “order picking” or “pick & pack”, in which goods are removed from crates or shelves and neatly packed into a target carton, is becoming increasingly important, especially in these times of booming online retail. The object recognition solution developed to support these processes can be taught fully automatically on digital 3D product models in less than an hour for each new product.
Smart shelving and kiosk systems
Another result from Fraunhofer IPA is a mobile pointer unit that “helps” to move goods. The solution is aimed at skilled workers in logistics and retail and makes it easier for them to stock shelves. Workers wear a small barcode ring scanner on their finger which registers a product or its box to be placed on the shelf. A moving spotlight then illuminates the location where the product needs to be placed based on a digital store twin. The application is particularly useful for small products as well as those that are very similar to one another and saves time (see video).
The Smart Kiosk System is a sales solution that was developed and tested. It uses the micromarket concept in order to branch out from supermarkets to include additional sales channels in the form of canteens and break rooms. For this purpose, weight sensors and cameras are installed in a refrigerator. With the help of object recognition, they make it possible to automatically recognize hands and products as well as remove or deposit products. This is combined with personal identification via a user interface and connection to a billing system (e.g., in this project, to SAP through partner Nagarro). Consequently, product withdrawals can be automatically booked in the virtual shopping cart and later billed digitally. In addition, service personnel receive a message when products need to be replenished, as the kiosk system constantly updates the digital twin on the connected K4R cloud platform. The project work took place in consultation with practice partner Livello.
Integration in the open-source platform
Last but not least, the IPA experts also advised on how the solutions developed as part of the project could best be specified and integrated in the K4R open-source platform. The openly expandable platform provides functions and interfaces for connecting to various logistics systems, sensors, robots or the extensive software modules from the Robot Operating System (ROS) environment, as well as providing the necessary infrastructure for store and product networked digital twins.
Powerful object recognition
Making stationary retail more attractive to customers again with the help of digital solutions – this was the goal of the partners working together on the Knowledge4Retail (K4R) project from January 2020 to December 2022. The group presented the results at the final meeting this spring. The core of the project is a K4R open-source platform available on internet host GitHub, a version control system, which makes the results accessible to retailers. Retail companies can use the platform to connect analog and digital worlds as well as analyze processes more easily and generate purchasing behavior forecasts, for example. In the process, the project partners developed solutions for applications in logistics, store set-up, service robotics and a fully automated kiosk.
 Foto: Fraunhofer IPA/Photograph: Rainer BezObject recognition enables a robot to recognize products on shelves and tidily pack them into boxes with the help of “bin packing” |
The IPA researchers worked on several development focuses as part of the project, during the course of which they expanded their expertise as well as the resulting range of 3D image processing-based products and services. One core aspect was the creation of what is known as semantic digital twinning of retail products. This involves using sensors to capture any product in a scanning station, for example. The product is then modeled three-dimensionally based on a wide range of information such as shape, color, and texture. The modeling process, in which Fraunhofer IPA collaborated with the Kaptura company, also involved machine learning methods, a subfield of AI (Artificial Intelligence).
With the help of the technologies developed, it is possible to model wide product ranges quickly and in 3D with high resolution texture. During the modeling process, semantic information such as logos, nutritional value or quantitative information on products is also recognized and digitally collected. The models generated are the basis, for example, for digital twins of a real store, a web store or automated product analyses, such as those carried out in laboratories.
In addition, a scalable object recognition solution optimized for retail has been created, enabling automation solutions for production and logistics through precise optical product identification. Application examples include robot-based bin picking or bin packing, i.e. the packing of goods of all types and shapes. The latter task, also known as “order picking” or “pick & pack”, in which goods are removed from crates or shelves and neatly packed into a target carton, is becoming increasingly important, especially in these times of booming online retail. The object recognition solution developed to support these processes can be taught fully automatically on digital 3D product models in less than an hour for each new product.
Smart shelving and kiosk systems
Another result from Fraunhofer IPA is a mobile pointer unit that “helps” to move goods. The solution is aimed at skilled workers in logistics and retail and makes it easier for them to stock shelves. Workers wear a small barcode ring scanner on their finger which registers a product or its box to be placed on the shelf. A moving spotlight then illuminates the location where the product needs to be placed based on a digital store twin. The application is particularly useful for small products as well as those that are very similar to one another and saves time (see video).
The Smart Kiosk System is a sales solution that was developed and tested. It uses the micromarket concept in order to branch out from supermarkets to include additional sales channels in the form of canteens and break rooms. For this purpose, weight sensors and cameras are installed in a refrigerator. With the help of object recognition, they make it possible to automatically recognize hands and products as well as remove or deposit products. This is combined with personal identification via a user interface and connection to a billing system (e.g., in this project, to SAP through partner Nagarro). Consequently, product withdrawals can be automatically booked in the virtual shopping cart and later billed digitally. In addition, service personnel receive a message when products need to be replenished, as the kiosk system constantly updates the digital twin on the connected K4R cloud platform. The project work took place in consultation with practice partner Livello.
Integration in the open-source platform
Last but not least, the IPA experts also advised on how the solutions developed as part of the project could best be specified and integrated in the K4R open-source platform. The openly expandable platform provides functions and interfaces for connecting to various logistics systems, sensors, robots or the extensive software modules from the Robot Operating System (ROS) environment, as well as providing the necessary infrastructure for store and product networked digital twins.
More news about "iba":
18 May 2025
iba 2025: Platform for start-ups
At the iba.START UP AREA, founders will showcase pioneering ideas for the baking industry. In Hall 13, visitors will find an exciting forum with space for networking. Young companies present innovative business models of tomorrow live on stage. (more…)1 April 2025
Latest food trends at iba 2025
What are the latest food trends in the baking industry? Visitors will find the answers at iba, the world's leading trade fair for the baking and confectionery industry in May in Düsseldorf. There is also a brand new concept of the iba.FOOD TRENDS AREA, where exhibitors present the latest trends from innovative raw materials and baking ingredients to snack ideas and healthy ingredients. At the iba.STAGE you can listen to informative presentations. Of course, there will also be tastings so that visitors can savour the brand new trends at iba. (more…)17 April 2019
Garibay new EXPO PACK General Director
Jorge Martínez Garibay has been named EXPO PACK General Director at PMMI, The Association for Packaging and Processing Technologies’ Mexico office. Garibay will direct sales, marketing, operations and public relations efforts of EXPO PACK México and EXPO PACK Guadalajara. (more…)4 May 2017
iba and iran food + bev tec to cooperate
GHM Gesellschaft für Handwerksmessen, organiser of the trade fair iba, has established a marketing cooperation with the Heidelberg-based trade fair organiser fairtrade and its trade fair iran food + bev tec. GHM CEO Dieter Dohr expects this cooperation to offer iba exhibitors access to the regions around the Caspian Sea and the Persian Gulf. (more…)10 September 2015
Sweet treats for breakfast, crunchy salad or warm dishes for lunch, coffee, cake, and pastries in the afternoon, and hearty meals in the evening: Bakers and confectioners are the perfect place to go for a bite to eat at any time of the day, and score points thanks to their fresh products, as well as their irresistible aroma which draws in customers. (more…)
| FAIR NAVIGATOR FOOD / BEVERAGE | |
|---|---|
|
| E-PAPER IBA |
 |