Collection: Open Data Processing Service (ODPS) - A hyper-scale big data computing platform designed for multiple scenarios

Alibaba Cloud offers an integrated big data computing platform and data warehouse product called ODPS, providing multifunctional, cost-effective, high-performance, stable, secure, open, and easy-to-use solutions for enterprise digital transformation, the company provides a full range of services. 

It is an internationally leading technology completely independently developed

As an integrated big data computing platform and data warehouse product developed by Alibaba Cloud, ODPS provides key technology innovations such as the multi-model unified computing engine, hyper-scale unified resource scheduling engine (Fuxi), and multi-tenant security isolation technology. 

From the aspect of storage technology, ODPS has built-in Pangu and OTS engines, which solve the bottlenecks of metadata hotspots, allowing it to scale to more than 10,000 physical servers and 30,000 nodes in a single cluster.

ODPS utilizes a SQL engine that is compatible with the Hive ecosystem, and is capable of addressing all data warehouse business scenarios in an end-to-end fashion through the use of one syntax. By utilizing a multi-role optimizer and dynamic code generation technology, ODPS is able to achieve over three times the performance of open-source Spark. ODPS has been ranked No. 1 worldwide in terms of performance and cost-effectiveness in the TPCx-BB benchmark test for the past five years. 

The Open Data Processing Service (ODPS) offers various capabilities, including cross-platform unified metadata processing, automatic connection to external data sources, and automatic creation of metadata repositories. ODPS has also adopted the data lakehouse architecture, which sets a new standard for the integration of lake and warehouse data, as well as the integration of multiple Hadoop big data systems into high-value data warehouses. 

ODPS has the ability to manage multiple clusters across multiple domains. With a unified set of accounts, permissions, metadata, and management and control capabilities, ODPS allows users to enable projects on clusters in any region, collect data locally, store or back up data across clusters, and perform computing across clusters, eliminating single-point failures in the central cluster.

Widely used in many industries in the international market

In China, Southeast Asia, and Europe, ODPS has been widely utilized in various industries including life services, Internet finance, transportation, mutual entertainment, e-commerce, games, media, and social networking. Furthermore, the ODPS plays a vital and irreplaceable role in the fields of industrial engineering, digital government, and urban planning.

In Hangzhou, China, the Hangzhou City Brain, which uses ODPS as its core, steers 1,300 intersections with traffic lights and more than 200 traffic police in real time. From 2016 to 2018, Hangzhou dropped 52 places from the list of the most congested cities in China as a result of the Hangzhou City Brain. To make government affairs data accessible, Zhejiang has used the MaxCompute big data processing platform. A unified data warehouse based on the same standards imports more than 7 billion pieces of data on 100 affairs that are most closely related to civilian affairs. Thus, civil servants will need to visit the Governmental Affairs Center at most once or even handle government affairs online.

Achievements of ODPS integrated big data computing platform

A total of 159 patent applications have been filed for the ODPS integrated big data computing platform, many related papers have been published in top international conferences, and 22 software copyrights have been obtained. Furthermore, ODPS has successfully passed a number of big data product capability evaluations conducted by the China Academy of Information and Communication Technology (CAICT).