Impala - Overview

Advertisements

Previous Page
Next Page  

What is Impala?

Impala is a MPP (Massive Parallel Processing) SQL query engine for processing huge volumes of data that is stored in Hadoop cluster. It is an open source software which is written in C++ and Java. It provides high performance and low latency compared to other SQL engines for Hadoop.

In other words, Impala is the highest performing SQL engine (giving RDBMS-like experience) which provides the fastest way to access data that is stored in Hadoop Distributed File System.

Why Impala?

Impala combines the SQL support and multi-user performance of a traditional analytic database with the scalability and flexibility of Apache Hadoop, by utilizing standard components such as HDFS, HBase, Metastore, YARN, and Sentry.

  • With Impala, users can communicate with HDFS or HBase using SQL queries in a faster way compared to other SQL engines like Hive.

  • Impala can read almost all the file formats such as Parquet, Avro, RCFile used by Hadoop.

Impala uses the same metadata, SQL syntax (Hive SQL), ODBC driver, and user interface (Hue Beeswax) as Apache Hive, providing a familiar and unified platform for batch-oriented or real-time queries.

Unlike Apache Hive, Impala is not based on MapReduce algorithms. It implements a distributed architecture based on daemon processes that are responsible for all the aspects of query execution that run on the same machines.

Thus, it reduces the latency of utilizing MapReduce and this makes Impala faster than Apache Hive.

Advantages of Impala

Here is a list of some noted advantages of Cloudera Impala.

  • Using impala, you can process data that is stored in HDFS at lightning-fast speed with traditional SQL knowledge.

  • Since the data processing is carried where the data resides (on Hadoop cluster), data transformation and data movement is not required for data stored on Hadoop, while working with Impala.

  • Using Impala, you can access the data that is stored in HDFS, HBase, and Amazon s3 without the knowledge of Java (MapReduce jobs). You can access them with a basic idea of SQL queries.

  • To write queries in business tools, the data has to be gone through a complicated extract-transform-load (ETL) cycle. But, with Impala, this procedure is shortened. The time-consuming stages of loading & reorganizing is overcome with the new techniques such as exploratory data analysis & data discovery making the process faster.

  • Impala is pioneering the use of the Parquet file format, a columnar storage layout that is optimized for large-scale queries typical in data warehouse scenarios.

Features of Impala

Given below are the features of cloudera Impala −

  • Impala is available freely as open source under the Apache license.

  • Impala supports in-memory data processing, i.e., it accesses/analyzes data that is stored on Hadoop data nodes without data movement.

  • You can access data using Impala using SQL-like queries.

  • Impala provides faster access for the data in HDFS when compared to other SQL engines.

  • Using Impala, you can store data in storage systems like HDFS, Apache HBase, and Amazon s3.

  • You can integrate Impala with business intelligence tools like Tableau, Pentaho, Micro strategy, and Zoom data.

  • Impala supports various file formats such as, LZO, Sequence File, Avro, RCFile, and Parquet.

  • Impala uses metadata, ODBC driver, and SQL syntax from Apache Hive.

Relational Databases and Impala

Impala uses a Query language that is similar to SQL and HiveQL. The following table describes some of the key dfferences between SQL and Impala Query language.

Impala Relational databases
Impala uses an SQL like query language that is similar to HiveQL. Relational databases use SQL language.
In Impala, you cannot update or delete individual records. In relational databases, it is possible to update or delete individual records.
Impala does not support transactions. Relational databases support transactions.
Impala does not support indexing. Relational databases support indexing.
Impala stores and manages large amounts of data (petabytes). Relational databases handle smaller amounts of data (terabytes) when compared to Impala.

Hive, Hbase, and Impala

Though Cloudera Impala uses the same query language, metastore, and the user interface as Hive, it differs with Hive and HBase in certain aspects. The following table presents a comparative analysis among HBase, Hive, and Impala.

HBase Hive Impala
HBase is wide-column store database based on Apache Hadoop. It uses the concepts of BigTable. Hive is a data warehouse software. Using this, we can access and manage large distributed datasets, built on Hadoop. Impala is a tool to manage, analyze data that is stored on Hadoop.
The data model of HBase is wide column store. Hive follows Relational model. Impala follows Relational model.
HBase is developed using Java language. Hive is developed using Java language. Impala is developed using C++.
The data model of HBase is schema-free. The data model of Hive is Schema-based. The data model of Impala is Schema-based.
HBase provides Java, RESTful and, Thrift API’s. Hive provides JDBC, ODBC, Thrift API’s. Impala provides JDBC and ODBC API’s.
Supports programming languages like C, C#, C++, Groovy, Java PHP, Python, and Scala. Supports programming languages like C++, Java, PHP, and Python. Impala supports all languages supporting JDBC/ODBC.
HBase provides support for triggers. Hive does not provide any support for triggers. Impala does not provide any support for triggers.

All these three databases −

  • Are NOSQL databases.

  • Available as open source.

  • Support server-side scripting.

  • Follow ACID properties like Durability and Concurrency.

  • Use sharding for partitioning.

Drawbacks of Impala

Some of the drawbacks of using Impala are as follows −

  • Impala does not provide any support for Serialization and Deserialization.
  • Impala can only read text files, not custom binary files.
  • Whenever new records/files are added to the data directory in HDFS, the table needs to be refreshed.
Advertisements