[discuss] Native Graph Query Language Support in Pinot

[ankitsultana]

I wanted to check if there's any interest in the community about adding Native Graph Query Language Support to Pinot (openCypher or GQL).

My read of the Graph space is that companies often have a very limited set of use-cases which can benefit from Graphs, and having a separate Database like Neo4J might be quite an overkill.

Users can technically use solutions like Puppy Graph to avoid maintaining another datastore, but it's not open source.

Having Graph Query Language support in Pinot would have the advantage that users can use the same tables for low-latency SQL and Time Series Monitoring/Alerting use-cases. So even if their graph use-cases fizzle out, they can still use SQL or Time Series QL with their datasets.

However, I don't think we'll be able to do super high QPS or super low latencies. My guess is that 100ms or so latencies with QPS in the order of 100 might be doable, and this might be sufficient for most use-cases.

Technical details aside, are there any companies that use Pinot that are interested in this?

[xiangfu0]

I feel it could be a good feature addition if there are active contributors to maintain and use this feature.

Would love to understand your use case.

[xiangfu0]

Design Doc: openCypher Graph Query Frontend for Pinot

Overview

This design adds an experimental openCypher-subset frontend that transpiles Cypher queries into SQL JOINs and executes them via Pinot's existing multi-stage engine (MSE). No native graph runtime operators are introduced.

Architecture

Cypher Query → CypherParser → CypherIR → CypherSemanticValidator → SqlGenerator → SQL
                                                                                    ↓
                                                              MSE (existing) ← BrokerRequestHandler
                                                                                    ↓
                                                                            Tabular Results

Key decision: Cypher-to-SQL transpilation (not Cypher-to-RelNode). This keeps the graph layer as a pure preprocessor with zero impact on the existing query engine.

Module Layout

pinot-graph/
├── pinot-graph-spi/        # GraphSchemaConfig, PinotGraphConfiguration
└── pinot-graph-planner/    # CypherParser, CypherIR, SqlGenerator, CypherToSqlTranslator

Graph Schema Model

The graph schema maps vertex labels and edge labels to Pinot tables:

{
  "vertexLabels": {
    "User": { "tableName": "users", "primaryKey": "id", "properties": {"name": "user_name"} }
  },
  "edgeLabels": {
    "FOLLOWS": {
      "tableName": "follows", "sourceVertexLabel": "User", "sourceKey": "follower_id",
      "targetVertexLabel": "User", "targetKey": "followee_id"
    }
  }
}

For MVP, the schema is passed inline in the request JSON. Future: stored in ZK with controller CRUD APIs.

Query Translation Example

1-hop:

MATCH (a:User {id: 1})-[:FOLLOWS]->(b:User) WHERE b.age > 25 RETURN b.name ORDER BY b.name LIMIT 100

→

SELECT b.user_name FROM follows AS e
  JOIN users AS a ON e.follower_id = a.id
  JOIN users AS b ON e.followee_id = b.id
  WHERE a.id = 1 AND b.age > 25
  ORDER BY b.user_name ASC LIMIT 100

2-hop (friends of friends):

MATCH (a:User {id: 1})-[:FOLLOWS]->(b:User)-[:FOLLOWS]->(c:User) RETURN DISTINCT c.id LIMIT 100

→

SELECT DISTINCT c.id FROM follows AS e1
  JOIN users AS a ON e1.follower_id = a.id
  JOIN users AS b ON e1.followee_id = b.id
  JOIN follows AS e2 ON e2.follower_id = b.id
  JOIN users AS c ON e2.followee_id = c.id
  WHERE a.id = 1 LIMIT 100

Supported Cypher Subset

Feature	Syntax	SQL Lowering
1-hop & 2-hop MATCH	(a)-[:R]->(b)-[:R]->(c)	Chained JOINs
Inline property filters	{id: 1}	WHERE clause
WHERE clause	=, <>, <, >, <=, >=, AND, OR, NOT	Direct mapping
String predicates	STARTS WITH, ENDS WITH, CONTAINS	LIKE patterns
IN lists	WHERE a.id IN [1, 2, 3]	SQL IN
RETURN with aliases	RETURN b.id AS userId	SELECT ... AS
DISTINCT	RETURN DISTINCT b.id	SELECT DISTINCT
Aggregations	count, sum, avg, min, max	SQL aggregates + GROUP BY
ORDER BY	ORDER BY b.name DESC	SQL ORDER BY
SKIP / LIMIT	SKIP 20 LIMIT 100	OFFSET / LIMIT
Direction	outgoing ->, incoming <-, undirected -	Join key swap

Entry Point

POST /query/graph on the broker, gated behind pinot.graph.enabled=true (default: false).

Distributed Execution

Since generated SQL uses standard JOINs, the MSE handles all distributed execution:

• Hash exchange for join key partitioning across servers
• Multi-stage plan fragmentation
• gRPC mailbox for inter-server data transfer

Explicit Non-Goals (MVP)

• No native graph operators (BFS, shortest path)
• No variable-length paths [*1..N]
• No writes (CREATE, MERGE, DELETE, SET)
• No OPTIONAL MATCH, WITH, UNION, subqueries
• No graph-specific cost model or optimization
• No Bolt protocol (HTTP REST only)
• No ZK-stored graph catalog (inline schema only)

Test Coverage

233 tests: 211 unit (parser/generator/lowering) + 12 handler + 10 E2E integration (2 servers, distributed data)

[hpvd]

would like to point to this somehow related graph issue, opened some time ago:
#14568