Key Technologies for Embodied Intelligence in Humanoid Robots

Introduction

Humanoid robots are expected to become another disruptive product after computers, smartphones and new energy vehicles, and they are the ultimate form of embodied intelligence. Leveraging its full-stack humanoid robotic technologies, UBTECH has developed embodied intelligence technologies, including Large Language Model for General Task Planning, Semantic VSLAM Navigation Technology, and Learning-based Whole-body Motion Control Technology. These advancements endow humanoid robots with smart brains and agile cerebellums, thereby accelerating their industrialization process.

Large Language Model for General Task Planning, Semantic VSLAM Navigation Technology, Learning-based Full-body Motion Control Technology

UBTECH has achieved significant advancements in the key technologies of embodied intelligence for humanoid robots, laying the technical foundation for the commercialization of such robots.

In the aspect of Large Language Model for General Task Planning, UBTECH uses simulation and real-world data to craft datasets for embodied intelligence and trains a multi-modal large language model. This enables sensor data to align with the motion of robots and achieves effective planning. By utilizing the advanced planning model technology for general tasks, robots gains the ability in intention understanding and fine-grained task planning. The innovative applications and the complexity of implementation of these technologies rank among the first tier in the global industry.

In the aspect of Semantic VSLAM Navigation Technology, this achievement designs a tightly coupled method of semantic perception information and traditional VSLAM, and expands the high-dimensional semantic information of the VSLAM maps. In addition, the nearest neighbor point planning of semantic information realizes a coarse to fine two-stage semantic navigation, improving the precision of humanoid robots during terminal operations. Currently, this method is an industry-first technology in the humanoid robotics field.

In the aspect of Learning-based Whole-body Motion Control Technology, in order to meet the motion requirements of humanoid robots for different task scenarios, we have designed an end-to-end learning-based whole-body motion control framework integrating sensing and control. This framework combines imitation learning and reinforcement learning strategies through mixing and cascading. By implementing this framework, the humanoid robot has dexterous manipulation skills in its upper limbs and stable walking abilities in its lower limbs, enhancing humanoid robots’ generalized execution capabilities for complex non-structured tasks.

Develop a Full-Stack Technology Layout for Humanoid Robots, Taking the Lead in Realizing Practical Applications of Humanoid Robots

UBTECH is one of the few companies that owns self-developed full-stack humanoid robotic technologies in the world. UBTECH is also one of the few companies in the world to accomplish mass production of small torque to large torque servo actuators. According to the Frost & Sullivan Report, UBTECH’s Humanoid Robot Walker is China’s first commercialized bipedal life-sized humanoid robot. As of June 20, 2024, this project has secured 670 authorized patents globally. UBTECH also leads and participates in nearly 40 global standards related to intelligent robots.

We are currently leading the deployment of humanoid robots globally, with our focus on three scenarios, which are industrial manufacturing, commercial services, and household companionship. Regarding commercial service scenarios, Walker has provided intelligence services in the China Science and Technology Museum, in the AI Education Base and in NEOM in Saudi Arabia. UBTECH Walker has also participated in performances during the Spring Festival Gala and the Ceremony of Universiade. In addition, UBTECH served as the "Official Exclusive Intelligent Robot Partner" of the China Pavilion at the Expo 2020 in Dubai. Its humanoid robots provided intelligent services to nearly 1.76 million visitors of the China Pavilion in over 182 days, creating the first-ever true long-term commercialized application of humanoid robots in human history. UBTECH has focused on key manufacturing areas, such as the automotive and 3C industries during this year, enhancing the tool operation and task execution capabilities of humanoid robots, and achieving deep integration applications in typical manufacturing scenarios. UBTECH has collaborated with multiple enterprises and achieved in-depth application in typical manufacturing scenarios, aiming to build a humanoid robot application ecosystem.

Develop an Application Paradigm for Humanoid Robots Oriented towards Multi-Task Industrial Scenarios

UBTECH is one of the few companies that owns self-developed full-stack humanoid robotic technologies in the world. UBTECH has collaborated with multiple enterprises, including Dongfeng Liuzhou Motor, Geely Auto, Hongqi Auto, FAW-Volkswagen, Qingdao Branch, Audi FAW, Foxconn, and SF Express, to jointly create demonstrative applications for humanoid robots. Up to now, UBTECH is the only humanoid robot enterprise globally that has announced strategic cooperation with multiple electric vehicle manufacturers. Furthermore, the Walker Series have been introduced into vehicle manufacturing assembly lines as "interns" to assist in car production. In addition, UBTECH has already received over 500 intent orders from electric vehicle manufacturers.

UBTECH has made breakthroughs in the key technologies of embodied intelligence for humanoid robots, providing application demonstrations for the commercialization of humanoid robots. Starting from the manufacturing of new energy vehicles, we will strengthen the stability and reliability of hardware, and then expand to 3-5 dedicated scenario applications to achieve commercialization and reduce costs. Subsequently, we will expand across industries and eventually evolve into a multitask general-purpose industrial humanoid robot.