Apache Pinot

This is the implementation of the Pinot data handler for MindsDB.

Apache Pinot is a real-time distributed OLAP database designed for low-latency query execution even at extremely high throughput. Apache Pinot can ingest directly from streaming sources like Apache Kafka and make events available for querying immediately.

Prerequisites

Before proceeding, ensure the following prerequisites are met:

1. Install MindsDB locally via Docker or use MindsDB Cloud.
2. To connect Apache Pinot to MindsDB, install the required dependencies following this instruction.
3. Install or ensure access to Apache Pinot.

Implementation

This handler was implemented using the pinotdb library, the Python DB-API and SQLAlchemy dialect for Pinot.

The required arguments to establish a connection are as follows:

• host is the host name or IP address of the Apache Pinot cluster.
• broker_port is the port that the Broker of the Apache Pinot cluster is running on.
• controller_port is the port that the Controller of the Apache Pinot cluster is running on.
• path is the query path.

Usage

In order to make use of this handler and connect to the Pinot cluster in MindsDB, the following syntax can be used:

CREATE DATABASE pinot_datasource
WITH
    engine = 'pinot',
    parameters = {
      "host":"localhost",
      "broker_port": 8000,
      "controller_port": 9000,
      "path": "/query/sql",
      "scheme": "http"
    };

You can use this established connection to query your table as follows:

SELECT *
FROM pinot_datasource.example_tbl;