HIVE-29579: K8s operator by ayushtkn · Pull Request #6452 · apache/hive

ayushtkn · 2026-04-24T14:33:58Z

What changes were proposed in this pull request?

Add Hive Operators to deploy Hive

Why are the changes needed?

To deploy Hive on K8s

Does this PR introduce any user-facing change?

No

How was this patch tested?

Deployed on Docker Desktop -> Follow Readme -> Build Hive master -> Build hive image
After building the apache/hive-4.3.0-SNPASHOT docker image like

mvn clean install -DskipTests -Pdist -Pkubernetes
cd packaging/src/docker/
./build.sh -hadoop 3.4.1 -tez 0.10.5 -tez-snapshot 1.0.0-SNAPSHOT

Via HELM Chart

Inside

cd packaging/src/kubernetes

Ozone

# ZooKeeper
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install zookeeper bitnami/zookeeper \
  --set replicaCount=1 --set auth.enabled=false \
  --set image.repository=bitnamilegacy/zookeeper \
  --set image.tag=3.9.3-debian-12-r21 \
  --set global.security.allowInsecureImages=true --wait

# PostgreSQL
helm install postgres bitnami/postgresql \
  --set auth.username=hive --set auth.password=hive123 \
  --set auth.database=metastore --wait

# Create the DB password secret
kubectl create secret generic hive-db-secret --from-literal=password=hive123

helm repo add ozone https://apache.github.io/ozone-helm-charts/
helm install ozone ozone/ozone --version 0.2.0 --wait
sleep 50
kubectl exec statefulset/ozone-om -- ozone sh volume create /s3v
kubectl exec statefulset/ozone-om -- ozone sh bucket create /s3v/hive

Deployed Hive

helm install hive ./helm/hive-operator \
  --set cluster.database.type=postgres \
  --set cluster.database.url="jdbc:postgresql://postgres-postgresql:5432/metastore" \
  --set cluster.database.driver="org.postgresql.Driver" \
  --set cluster.database.username=hive \
  --set cluster.database.passwordSecretRef.name=hive-db-secret \
  --set cluster.database.passwordSecretRef.key=password \
  --set cluster.database.driverJarUrl="https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.5/postgresql-42.7.5.jar" \
  --set cluster.zookeeper.quorum="zookeeper:2181" \
  --set cluster.storage.coreSiteOverrides."fs\.defaultFS"="s3a://hive" \
  --set cluster.storage.coreSiteOverrides."fs\.s3a\.endpoint"="http://ozone-s3g-rest:9878" \
  --set-string cluster.storage.coreSiteOverrides."fs\.s3a\.path\.style\.access"=true \
  --set 'cluster.storage.envVars[0].name=HADOOP_OPTIONAL_TOOLS' \
  --set 'cluster.storage.envVars[0].value=hadoop-aws' \
  --set 'cluster.storage.envVars[1].name=AWS_ACCESS_KEY_ID' \
  --set 'cluster.storage.envVars[1].value=ozone' \
  --set 'cluster.storage.envVars[2].name=AWS_SECRET_ACCESS_KEY' \
  --set 'cluster.storage.envVars[2].value=ozone'

Beeline

kubectl exec -it deployment/hive-hiveserver2 -- beeline -u "jdbc:hive2://hive-hiveserver2:10000/"

External S3 Bucket

Postgres & ZK as above, then

helm install hive ./helm/hive-operator \
 --set cluster.database.type=postgres \
 --set cluster.database.url="jdbc:postgresql://postgres-postgresql:5432/metastore" \
 --set cluster.database.driver="org.postgresql.Driver" \
 --set cluster.database.username=hive \
 --set cluster.database.passwordSecretRef.name=hive-db-secret \
 --set cluster.database.passwordSecretRef.key=password \
 --set cluster.database.driverJarUrl="https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.5/postgresql-42.7.5.jar" \
   --set cluster.zookeeper.quorum="zookeeper:2181" \
 --set cluster.storage.coreSiteOverrides."fs\.defaultFS"="s3a://ayush-k8s-bucket" \
 --set 'cluster.storage.envVars[0].name=HADOOP_OPTIONAL_TOOLS' \
 --set 'cluster.storage.envVars[0].value=hadoop-aws' \
 --set 'cluster.storage.envVars[1].name=AWS_ACCESS_KEY_ID' \
 --set 'cluster.storage.envVars[1].valueFrom.secretKeyRef.name=aws-s3-creds' \
 --set 'cluster.storage.envVars[1].valueFrom.secretKeyRef.key=accessKey' \
 --set 'cluster.storage.envVars[2].name=AWS_SECRET_ACCESS_KEY' \
 --set 'cluster.storage.envVars[2].valueFrom.secretKeyRef.name=aws-s3-creds' \
 --set 'cluster.storage.envVars[2].valueFrom.secretKeyRef.key=secretKey'

Output

Google GCS Bucket

Deployed Postgres & ZK like above

kubectl create secret generic gcs-creds  --from-file=key.json=/Users/ayushsaxena/Downloads/ayushs-143910-7697f09f1e41.json


helm install hive ./helm/hive-operator \
  --set cluster.database.type=postgres \
  --set cluster.database.url="jdbc:postgresql://postgres-postgresql:5432/metastore" \
  --set cluster.database.driver="org.postgresql.Driver" \
  --set cluster.database.username=hive \
  --set cluster.database.passwordSecretRef.name=hive-db-secret \
  --set cluster.database.driverJarUrl="https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.5/postgresql-42.7.5.jar" \
  --set 'cluster.storage.coreSiteOverrides.fs\.defaultFS=gs://ayush-gcs-bucket' \
  --set 'cluster.storage.coreSiteOverrides.fs\.gs\.impl=com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem' \
  --set 'cluster.storage.coreSiteOverrides.fs\.gs\.auth\.type=SERVICE_ACCOUNT_JSON_KEYFILE' \
  --set 'cluster.storage.coreSiteOverrides.fs\.gs\.auth\.service\.account\.json\.keyfile=/etc/gcs/key.json' \
  --set-string 'cluster.storage.coreSiteOverrides.fs\.gs\.reported\.permissions=777' \
  --set 'cluster.storage.externalJars[0]=https://repo1.maven.org/maven2/com/google/cloud/bigdataoss/gcs-connector/hadoop3-2.2.25/gcs-connector-hadoop3-2.2.25-shaded.jar' \
  --set 'cluster.storage.volumes[0].name=gcs-key' \
  --set 'cluster.storage.volumes[0].secret.secretName=gcs-creds' \
  --set 'cluster.storage.volumeMounts[0].name=gcs-key' \
  --set 'cluster.storage.volumeMounts[0].mountPath=/etc/gcs' \
  --set 'cluster.storage.volumeMounts[0].readOnly=true'

Output

Via Operator

From Hive root directory

mvn clean package -pl packaging/src/kubernetes -Pkubernetes -DskipTests
cd packaging/src/kubernetes
kubectl apply -f helm/hive-operator/crds/hiveclusters.hive.apache.org-v1.yml

kubectl create namespace hive-operator

kubectl apply -f config/rbac/service-account.yaml
kubectl apply -f config/rbac/cluster-role.yaml
kubectl apply -f config/rbac/cluster-role-binding.yaml

export HIVE_VERSION=4.3.0-SNAPSHOT
envsubst < config/operator/deployment.yaml | kubectl apply -f -

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install zookeeper bitnami/zookeeper \
  --set replicaCount=1 \
  --set auth.enabled=false \
  --set persistence.enabled=false \
  --set image.repository=bitnamilegacy/zookeeper \
  --set image.tag=3.9.3-debian-12-r21 \
  --set global.security.allowInsecureImages=true \
  --wait



helm repo add ozone https://apache.github.io/ozone-helm-charts/
helm install ozone ozone/ozone --version 0.2.0 --wait
sleep 50

kubectl exec statefulset/ozone-om -- ozone sh volume create /s3v
kubectl exec statefulset/ozone-om -- ozone sh bucket create /s3v/hive

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install postgres bitnami/postgresql \
  --set auth.username=hive \
  --set auth.password=hive123 \
  --set auth.database=metastore \
  --wait


kubectl create secret generic hive-db-secret \
  --from-literal=password=hive123


envsubst < config/samples/hivecluster-full-ha.yaml | kubectl apply -f -

Running Pods:

Beeline

kubectl exec -it deployment/hive-hiveserver2 -- beeline -u "jdbc:hive2://hive-hiveserver2:10000/""

HiveServer2 UI:

ayushsaxena@Q3NW54Y0C5 kubernetes % kubectl port-forward svc/hive-hiveserver2 10002:10002
Forwarding from 127.0.0.1:10002 -> 10002
Forwarding from [::1]:10002 -> 10002

Testing with External S3 Bucket

All Steps - the Ozone from above
Changed

Created Key

kubectl create secret generic aws-s3-creds \
  --from-literal=accessKey="<My KEY ID>" \
  --from-literal=secretKey="<MY KEY>"

In hivecluster-full-ha.yaml
changed

  hadoop:
    coreSiteOverrides:
     fs.defaultFS: "s3a://ayush-k8s-bucket"

  # Environment variables injected into all component pods (credentials, JVM options, etc.)
  envVars:
    - name: HADOOP_OPTIONAL_TOOLS
      value: "hadoop-aws"
    - name: AWS_ACCESS_KEY_ID
      valueFrom:
       secretKeyRef:
         name: aws-s3-creds
         key: accessKey
    - name: AWS_SECRET_ACCESS_KEY
      valueFrom:
       secretKeyRef:
         name: aws-s3-creds
         key: secretKey

All same steps as above -Ozone helm lines, Created an Iceberg table & checked the location of the files ingested

Testing with GCS Bucket

Create bucket and get the access key then create secret like

kubectl create secret generic gcs-creds  --from-file=key.json=/Users/ayushsaxena/Downloads/ayushs-143910-7697f09f1e41.json

Changed in yml

  hadoop:
    coreSiteOverrides:
      fs.defaultFS: "gs://ayush-gcs-bucket"
      fs.gs.impl: "com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem"
      fs.gs.auth.type: "SERVICE_ACCOUNT_JSON_KEYFILE"
      fs.gs.auth.service.account.json.keyfile: "/etc/gcs/key.json"
      fs.gs.reported.permissions: "777"

  # GCS connector JAR — downloaded into all pods and added to HADOOP_CLASSPATH
  externalJars:
    - "https://repo1.maven.org/maven2/com/google/cloud/bigdataoss/gcs-connector/hadoop3-2.2.25/gcs-connector-hadoop3-2.2.25-shaded.jar"

  # Mount the GCS service account key secret into all Hive pods
  volumes:
    - name: gcs-key
      secret:
        secretName: gcs-creds
  volumeMounts:
    - name: gcs-key
      mountPath: /etc/gcs
      readOnly: true

Created a table & validated the location

With Oracle DB + S3 Bucket

Same as above.
Deployed Oracle as

kubectl apply -f - <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: oracle
spec:
  replicas: 1
  selector:
    matchLabels:
      app: oracle
  template:
    metadata:
      labels:
        app: oracle
    spec:
      containers:
      - name: oracle
        image: gvenzl/oracle-free:latest
        env:
        - name: ORACLE_PASSWORD
          value: hive123
        - name: APP_USER
          value: hive
        - name: APP_USER_PASSWORD
          value: hive123
        ports:
        - containerPort: 1521
---
apiVersion: v1
kind: Service
metadata:
  name: oracle
spec:
  selector:
    app: oracle
  ports:
  - port: 1521
EOF

Updated the yml as

  metastore:
    database:
      type: oracle
      url: "jdbc:oracle:thin:@oracle:1521/FREEPDB1"
      driver: "oracle.jdbc.OracleDriver"
      username: hive
      passwordSecretRef:
        name: hive-db-secret
        key: password
      driverJarUrl: "https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc11/23.3.0.23.09/ojdbc11-23.3.0.23.09.jar"
    warehouseDir: "/hive/warehouse"
    configOverrides:
      metastore.catalog.servlet.port: "9001"
      metastore.catalog.servlet.auth: "none"

Created a table & verified inside Oracle DB

External HMS

Deployed HMS outside k8s with docker pointing to local Ozone.

Built Hive Metastore 4.3.0-SNAPSHOT HMS Docker Image then

docker compose as

version: '3.9'
services:
  postgres:
    image: postgres
    restart: unless-stopped
    container_name: postgres
    hostname: postgres
    environment:
      POSTGRES_DB: 'metastore_db'
      POSTGRES_USER: 'hive'
      POSTGRES_PASSWORD: 'password'
    ports:
      - '5432:5432'
    volumes:
      - hive-db:/var/lib/postgresql
    networks:
      - hive

  metastore:
    image: apache/hive:standalone-metastore-${HIVE_VERSION}
    depends_on:
      - postgres
    restart: unless-stopped
    container_name: metastore
    hostname: metastore
    environment:
      DEFAULT_FS: "s3a://hive"
      HIVE_WAREHOUSE_PATH: "/warehouse"
      HADOOP_CLASSPATH: "/opt/hadoop/share/hadoop/tools/lib/*:/tmp/ext-jars/*"
      DB_DRIVER: postgres
      SERVICE_NAME: 'metastore'
      SERVICE_OPTS: >
        -Xmx1G
        -Djavax.jdo.option.ConnectionDriverName=org.postgresql.Driver
        -Djavax.jdo.option.ConnectionURL=jdbc:postgresql://postgres:5432/metastore_db
        -Djavax.jdo.option.ConnectionUserName=hive
        -Djavax.jdo.option.ConnectionPassword=password

      S3_ENDPOINT_URL: "http://host.docker.internal:9878"
      AWS_ACCESS_KEY_ID: "ozone"
      AWS_SECRET_ACCESS_KEY: "ozone"
    ports:
        - '9001:9001'
        - '9083:9083'
    volumes:
        - warehouse:/opt/hive/data/warehouse
        - type: bind
          source: ${POSTGRES_LOCAL_PATH}
          target: /opt/hive/lib/postgres.jar
        # Mount local jars to a temporary staging area (Read-Only)
        - ./jars:/tmp/ext-jars:ro
    networks:
      - hive

volumes:
  hive-db:
  warehouse:

networks:
  hive:
    name: hive

Verified the HMS lives alone in Docker env

Modified the metastore column in hivecluster-full-ha.yaml to be as

  metastore:
    enabled: false
    externalUri: "thrift://host.docker.internal:9083"

The externalUri points to our external HMS

Deployed hivecluster-full-ha.yaml No HMS here

zhangbutao · 2026-04-24T15:27:28Z

I can't wait to try out this feature!!!
BTW, besides the LLAP mode, can we also execute normal Tez tasks?

ayushtkn · 2026-05-04T20:43:31Z

BTW, besides the LLAP mode, can we also execute normal Tez tasks?

Thanks @zhangbutao for taking a look! Currently, the operator only supports LLAP mode (along with a Tez Local mode strictly for development and testing). Standard Tez mode isn't on my immediate radar right now; I can look into Tez in a follow-up maybe after them, though LLAP generally makes more sense in cloud environments

zhangbutao · 2026-05-05T13:44:08Z

Standard Tez

@ayushtkn That sounds great. I think LLAP on cloud/k8s is definitely a milestone for Hive — a really nice feature.

However, as far as I know, in the real world, there are still a large number of Hive on standard Tez mode jobs running on YARN. Many users want to migrate these Tez jobs to native Kubernetes(without relying on YARN's scheduling), just like Spark/Flink on k8s. I’m really looking forward to further updates on Tez on k8s as well.

Thanks!

ayushtkn · 2026-05-05T20:48:14Z

Thanks @zhangbutao for sharing the use case. I’ll definitely look into it. Most likely, we’ll have support for both LLAP mode and Tez mode before the release — or at least I’ll try to cover as much as possible. I will create a ticket to track that.

Need to see from Tez side, I think it would require changes there first YARN -> Kubernetes

In the end, it’s up to the users which mode they want to use. Whichever mode they choose, they’re still using Hive, which is a win-win for us :-)

ayushtkn · 2026-05-19T11:53:08Z

I have published the HELM charts in my repo till we find it a home in the ASF ecosystem. This can be used to deploy Hive on Ozone (LLAP) without building anything just copy-paste commands
https://hub.docker.com/r/ayushtkn/hive-operator

In case anyone wants to try, without building the PR & docker images

sonarqubecloud · 2026-05-24T07:31:26Z

Quality Gate passed

Issues
36 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

simhadri-g · 2026-05-24T20:50:19Z

Hi @ayushtkn ,

This is really awesome !

I tried out the hive Kubernetes operator using the helm chart you have published here:https://hub.docker.com/r/ayushtkn/hive-operator

After fixing minor issue on my end of installing helm and enabling Kubernetes in my docker desktop , all the ozone, zookeeper, postgres and hive pods did come up seamlessly. That was awesome!

Once all the pods were up, I was able to launch hive queries on llap demons via beeline .
I even tested it with my s3 buckets and i did not face any major issue. You can find the full terminal logs below.

I do have a few comments in readme and a few comments on improvements but i think we can handle them in a new jira.


PS I:\simhadrig\hive>   helm install hive oci://registry-1.docker.io/ayushtkn/hive-operator   --version 4.3.0-SNAPSHOT   --set operator.image.repository="ayushtkn/hive"   --set operator.image.tag="operator-4.3.0-SNAPSHOT"   --set cluster.image="ayushtkn/hive:4.3.0-SNAPSHOT"   --set cluster.database.type=postgres   --set cluster.database.url="jdbc:postgresql://postgres-postgresql:5432/metastore"   --set cluster.database.driver="org.postgresql.Driver"   --set cluster.database.username=hive   --set cluster.database.passwordSecretRef.name=hive-db-secret   --set cluster.database.passwordSecretRef.key=password   --set cluster.database.driverJarUrl="https://repo1.maven.org/maven2/org/postgresql/postgresql/42.7.5/postgresql-42.7.5.jar"   --set cluster.zookeeper.quorum="zookeeper:2181"   --set cluster.storage.coreSiteOverrides."fs\.defaultFS"="s3a://hive"   --set cluster.storage.coreSiteOverrides."fs\.s3a\.endpoint"="http://ozone-s3g-rest:9878"   --set-string cluster.storage.coreSiteOverrides."fs\.s3a\.path\.style\.access"=true   --set 'cluster.storage.envVars[0].name=HADOOP_OPTIONAL_TOOLS'   --set 'cluster.storage.envVars[0].value=hadoop-aws'   --set 'cluster.storage.envVars[1].name=AWS_ACCESS_KEY_ID'   --set 'cluster.storage.envVars[1].value=ozone'   --set 'cluster.storage.envVars[2].name=AWS_SECRET_ACCESS_KEY'   --set 'cluster.storage.envVars[2].value=ozone'
Pulled: registry-1.docker.io/ayushtkn/hive-operator:4.3.0-SNAPSHOT
Digest: sha256:031447a5bd55a0fefec2d242d6dcb03c4e57e8f3e9a0e937f64f28263b496832
NAME: hive
LAST DEPLOYED: Mon May 25 02:01:30 2026
NAMESPACE: default
STATUS: deployed
REVISION: 1
DESCRIPTION: Install complete
TEST SUITE: None
PS I:\simhadrig\hive> kubectl wait --for=condition=Ready hivecluster/hive --timeout=300s
PS I:\simhadrig\hive> kubectl exec -it deployment/hive-hiveserver2 -- beeline -u "jdbc:hive2://hive-hiveserver2:10000/"
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.25.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://hive-hiveserver2:10000/
Connected to: Apache Hive (version 4.3.0-SNAPSHOT)
Driver: Hive JDBC (version 4.3.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 4.3.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://hive-hiveserver2:10000/> create table hive_example(a string, b int) partitioned by(c int);
>   alter table hive_example add partition(c=1);
>   insert into hive_example partition(c=1) values('a', 1), ('a', 2),('b',3);
>   select count(distinct a) from hive_example;
>   select sum(b) from hive_example;
INFO  : Compiling command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f): create table hive_example(a string, b int) partitioned by(c int)
INFO  : HMS client filtering is enabled.
INFO  : Resolved metastore uris: [thrift://hive-metastore:9083]
INFO  : Trying to connect to metastore with URI (thrift://hive-metastore:9083) in binary transport mode
INFO  : Opened a connection to metastore, URI (thrift://hive-metastore:9083) current connections: 6
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f); Time taken: 8.76 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f): create table hive_example(a string, b int) partitioned by(c int)
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20260524203703_59b12949-84ff-4ee3-86cc-70ad36d3002f); Time taken: 15.567 seconds
No rows affected (26.192 seconds)
INFO  : Compiling command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2): alter table hive_example add partition(c=1)
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2); Time taken: 0.865 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2): alter table hive_example add partition(c=1)
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20260524203730_aca893f3-184b-46db-9625-9ceec743dbe2); Time taken: 4.181 seconds
No rows affected (5.184 seconds)
INFO  : Compiling command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d): insert into hive_example partition(c=1) values('a', 1), ('a', 2),('b',3)
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:col1, type:string, comment:null), FieldSchema(name:col2, type:int, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d); Time taken: 17.279 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d): insert into hive_example partition(c=1) values('a', 1), ('a', 2),('b',3)
INFO  : Query ID = hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d
INFO  : Total jobs = 1
INFO  : Launching Job 1 out of 1
INFO  : Starting task [Stage-1:MAPRED] in serial mode
INFO  : Subscribed to counters: [] for queryId: hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d
INFO  : Tez session hasn't been created yet. Opening session
INFO  : Dag name: insert into hive_exam...... ('a', 2),('b',3) (Stage-1)
INFO  : HS2 Host: [hive-hiveserver2-757b7b5987-257nv], Query ID: [hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d], Dag ID: [dag_1779654818589_0000_1], DAG App ID: [application_1779654818589_0000], DAG App address: [hive-tezam-0.hive-tezam.default.svc.cluster.local]
INFO  : Status: Running (Executing on YARN cluster with App id application_1779654818589_0000)

----------------------------------------------------------------------------------------------
       VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 ..........      llap     SUCCEEDED      1          1        0        0       0       0
Reducer 2 ......      llap     SUCCEEDED      1          1        0        0       0       0
----------------------------------------------------------------------------------------------
VERTICES: 02/02  [==========================>>] 100%  ELAPSED TIME: 35.02 s
----------------------------------------------------------------------------------------------
INFO  : Status: DAG finished successfully in 33.12 seconds
INFO  : DAG ID: dag_1779654818589_0000_1
INFO  :
INFO  : Query Execution Summary
INFO  : ----------------------------------------------------------------------------------------------
INFO  : OPERATION                            DURATION
INFO  : ----------------------------------------------------------------------------------------------
INFO  : Compile Query                          17.28s
INFO  : Prepare Plan                            9.60s
INFO  : Get Query Coordinator (AM)              0.26s
INFO  : Submit Plan                             4.99s
INFO  : Start DAG                               0.65s
INFO  : Run DAG                                33.12s
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : Task Execution Summary
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      DURATION(ms)   CPU_TIME(ms)    GC_TIME(ms)   INPUT_RECORDS   OUTPUT_RECORDS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1          24727.00              0              0               3                1
INFO  :  Reducer 2           2006.00              0              0               1                0
INFO  : ----------------------------------------------------------------------------------------------
INFO  : FileSystem Counters Summary
INFO  :
INFO  : Scheme: S3A
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1              0B             6                  0                12B             4
INFO  :  Reducer 2              0B             6                  0               187B             4
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : Scheme: FILE
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1             64B             0                  0               114B             0
INFO  :  Reducer 2              0B             0                  0                 0B             0
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : org.apache.tez.common.counters.DAGCounter:
INFO  :    NUM_SUCCEEDED_TASKS: 2
INFO  :    TOTAL_LAUNCHED_TASKS: 2
INFO  :    DURATION_SUCCEEDED_TASKS_MILLIS: 26324
INFO  :    RACK_LOCAL_TASKS: 1
INFO  :    AM_CPU_MILLISECONDS: 25650
INFO  :    WALL_CLOCK_MILLIS: 26324
INFO  :    AM_GC_TIME_MILLIS: 511
INFO  :    INITIAL_HELD_CONTAINERS: 0
INFO  :    TOTAL_CONTAINERS_USED: 2
INFO  :    TOTAL_CONTAINER_LAUNCH_COUNT: 2
INFO  :    TOTAL_CONTAINER_RELEASE_COUNT: 2
INFO  :    NODE_USED_COUNT: 1
INFO  :    NODE_TOTAL_COUNT: 2
INFO  : File System Counters:
INFO  :    FILE_BYTES_READ: 64
INFO  :    FILE_BYTES_WRITTEN: 114
INFO  :    FILE_READ_OPS: 0
INFO  :    FILE_LARGE_READ_OPS: 0
INFO  :    FILE_WRITE_OPS: 0
INFO  :    S3A_BYTES_READ: 0
INFO  :    S3A_BYTES_WRITTEN: 199
INFO  :    S3A_READ_OPS: 12
INFO  :    S3A_LARGE_READ_OPS: 0
INFO  :    S3A_WRITE_OPS: 8
INFO  : org.apache.tez.common.counters.TaskCounter:
INFO  :    REDUCE_INPUT_GROUPS: 1
INFO  :    REDUCE_INPUT_RECORDS: 1
INFO  :    COMBINE_INPUT_RECORDS: 0
INFO  :    SPILLED_RECORDS: 2
INFO  :    NUM_SHUFFLED_INPUTS: 1
INFO  :    NUM_SKIPPED_INPUTS: 0
INFO  :    NUM_FAILED_SHUFFLE_INPUTS: 0
INFO  :    MERGED_MAP_OUTPUTS: 1
INFO  :    INPUT_RECORDS_PROCESSED: 4
INFO  :    INPUT_SPLIT_LENGTH_BYTES: 1
INFO  :    OUTPUT_RECORDS: 1
INFO  :    OUTPUT_LARGE_RECORDS: 0
INFO  :    OUTPUT_BYTES: 70
INFO  :    OUTPUT_BYTES_WITH_OVERHEAD: 78
INFO  :    OUTPUT_BYTES_PHYSICAL: 82
INFO  :    ADDITIONAL_SPILLS_BYTES_WRITTEN: 0
INFO  :    ADDITIONAL_SPILLS_BYTES_READ: 82
INFO  :    ADDITIONAL_SPILL_COUNT: 0
INFO  :    SHUFFLE_CHUNK_COUNT: 1
INFO  :    SHUFFLE_BYTES: 82
INFO  :    SHUFFLE_BYTES_DECOMPRESSED: 78
INFO  :    SHUFFLE_BYTES_TO_MEM: 0
INFO  :    SHUFFLE_BYTES_TO_DISK: 0
INFO  :    SHUFFLE_BYTES_DISK_DIRECT: 82
INFO  :    NUM_MEM_TO_DISK_MERGES: 0
INFO  :    NUM_DISK_TO_DISK_MERGES: 0
INFO  :    SHUFFLE_PHASE_TIME: 158
INFO  :    MERGE_PHASE_TIME: 197
INFO  :    FIRST_EVENT_RECEIVED: 87
INFO  :    LAST_EVENT_RECEIVED: 87
INFO  : HIVE:
INFO  :    CREATED_FILES: 2
INFO  :    DESERIALIZE_ERRORS: 0
INFO  :    RECORDS_IN_Map_1: 3
INFO  :    RECORDS_OUT_0: 1
INFO  :    RECORDS_OUT_1_default.hive_example: 3
INFO  :    RECORDS_OUT_INTERMEDIATE_Map_1: 1
INFO  :    RECORDS_OUT_INTERMEDIATE_Reducer_2: 0
INFO  :    RECORDS_OUT_OPERATOR_FS_15: 1
INFO  :    RECORDS_OUT_OPERATOR_FS_4: 3
INFO  :    RECORDS_OUT_OPERATOR_GBY_13: 1
INFO  :    RECORDS_OUT_OPERATOR_GBY_7: 1
INFO  :    RECORDS_OUT_OPERATOR_MAP_0: 0
INFO  :    RECORDS_OUT_OPERATOR_RS_8: 1
INFO  :    RECORDS_OUT_OPERATOR_SEL_1: 1
INFO  :    RECORDS_OUT_OPERATOR_SEL_14: 1
INFO  :    RECORDS_OUT_OPERATOR_SEL_3: 3
INFO  :    RECORDS_OUT_OPERATOR_SEL_6: 3
INFO  :    RECORDS_OUT_OPERATOR_TS_0: 1
INFO  :    RECORDS_OUT_OPERATOR_UDTF_2: 3
INFO  :    TOTAL_TABLE_ROWS_WRITTEN: 3
INFO  : Shuffle Errors:
INFO  :    BAD_ID: 0
INFO  :    CONNECTION: 0
INFO  :    IO_ERROR: 0
INFO  :    WRONG_LENGTH: 0
INFO  :    WRONG_MAP: 0
INFO  :    WRONG_REDUCE: 0
----------------------------------------------------------------------------------------------
       VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
INFO  : Starting task [Stage-2:DEPENDENCY_COLLECTION] in serial mode       0       0       0
INFO  : Starting task [Stage-0:MOVE] in serial mode      1        0        0       0       0
INFO  : Loading data to table default.hive_example partition (c=1) from s3a://hive/hive/warehouse/hive_example/c=1/.hive-staging_hive_2026-05-24_20-37-35_688_3563499212412533844-1/-ext-10000.02 s
----------------------------------------------------------------------------------------------
       VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 ..........      llap     SUCCEEDED      1          1        0        0       0       0
Reducer 2 ......      llap     SUCCEEDED      1          1        0        0       0       0
----------------------------------------------------------------------------------------------
VERTICES: 02/02  [==========================>>] 100%  ELAPSED TIME: 45.67 s
----------------------------------------------------------------------------------------------
INFO  : Partition {c=1} stats: [numFiles=1, numRows=0, totalSize=12, rawDataSize=0, numFilesErasureCoded=0]
INFO  : StatsTask took 2980
INFO  : Completed executing command(queryId=hive_20260524203735_40867723-378d-4da4-bd32-c30c582bcf0d); Time taken: 61.102 seconds
3 rows affected (78.489 seconds)
INFO  : Compiling command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e): select count(distinct a) from hive_example
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e); Time taken: 7.146 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e): select count(distinct a) from hive_example
INFO  : Query ID = hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e
INFO  : Total jobs = 1
INFO  : Launching Job 1 out of 1
INFO  : Starting task [Stage-1:MAPRED] in serial mode
INFO  : Subscribed to counters: [] for queryId: hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e
INFO  : Session is already open
INFO  : Dag name: select count(distinct a) from hive_example (Stage-1)
INFO  : HS2 Host: [hive-hiveserver2-757b7b5987-257nv], Query ID: [hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e], Dag ID: [dag_1779654818589_0000_2], DAG App ID: [application_1779654818589_0000], DAG App address: [hive-tezam-0.hive-tezam.default.svc.cluster.local]
INFO  : Status: Running (Executing on YARN cluster with App id application_1779654818589_0000)

----------------------------------------------------------------------------------------------
       VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 ..........      llap     SUCCEEDED      1          1        0        0       0       0
Reducer 2 ......      llap     SUCCEEDED      1          1        0        0       0       0
Reducer 3 ......      llap     SUCCEEDED      1          1        0        0       0       0
----------------------------------------------------------------------------------------------
VERTICES: 03/03  [==========================>>] 100%  ELAPSED TIME: 10.48 s
----------------------------------------------------------------------------------------------
INFO  : Status: DAG finished successfully in 9.92 seconds
INFO  : DAG ID: dag_1779654818589_0000_2
INFO  :
INFO  : Query Execution Summary
INFO  : ----------------------------------------------------------------------------------------------
INFO  : OPERATION                            DURATION
INFO  : ----------------------------------------------------------------------------------------------
INFO  : Compile Query                           7.15s
INFO  : Prepare Plan                            0.67s
INFO  : Get Query Coordinator (AM)              0.00s
INFO  : Submit Plan                             1.71s
INFO  : Start DAG                               0.57s
INFO  : Run DAG                                 9.96s
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : Task Execution Summary
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      DURATION(ms)   CPU_TIME(ms)    GC_TIME(ms)   INPUT_RECORDS   OUTPUT_RECORDS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1           5866.00              0              0               3                2
INFO  :  Reducer 2            562.00              0              0               2                1
INFO  :  Reducer 3           1007.00              0              0               1                0
INFO  : ----------------------------------------------------------------------------------------------
INFO  : FileSystem Counters Summary
INFO  :
INFO  : Scheme: S3A
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1             12B             1                  0                 0B             0
INFO  :  Reducer 2              0B             0                  0                 0B             0
INFO  :  Reducer 3              0B             6                  0               101B             4
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : Scheme: FILE
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1             64B             0                  0                54B             0
INFO  :  Reducer 2              0B             0                  0                 0B             0
INFO  :  Reducer 3              0B             0                  0                 0B             0
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : org.apache.tez.common.counters.DAGCounter:
INFO  :    NUM_SUCCEEDED_TASKS: 3
INFO  :    TOTAL_LAUNCHED_TASKS: 3
INFO  :    DURATION_SUCCEEDED_TASKS_MILLIS: 7206
INFO  :    DATA_LOCAL_TASKS: 1
INFO  :    AM_CPU_MILLISECONDS: 8120
INFO  :    WALL_CLOCK_MILLIS: 7206
INFO  :    AM_GC_TIME_MILLIS: 0
INFO  :    INITIAL_HELD_CONTAINERS: 0
INFO  :    TOTAL_CONTAINERS_USED: 3
INFO  :    TOTAL_CONTAINER_LAUNCH_COUNT: 3
INFO  :    TOTAL_CONTAINER_RELEASE_COUNT: 3
INFO  :    NODE_USED_COUNT: 1
INFO  :    NODE_TOTAL_COUNT: 2
INFO  : File System Counters:
INFO  :    FILE_BYTES_READ: 64
INFO  :    FILE_BYTES_WRITTEN: 54
INFO  :    FILE_READ_OPS: 0
INFO  :    FILE_LARGE_READ_OPS: 0
INFO  :    FILE_WRITE_OPS: 0
INFO  :    S3A_BYTES_READ: 12
INFO  :    S3A_BYTES_WRITTEN: 101
INFO  :    S3A_READ_OPS: 7
INFO  :    S3A_LARGE_READ_OPS: 0
INFO  :    S3A_WRITE_OPS: 4
INFO  : org.apache.tez.common.counters.TaskCounter:
INFO  :    REDUCE_INPUT_GROUPS: 2
INFO  :    REDUCE_INPUT_RECORDS: 2
INFO  :    COMBINE_INPUT_RECORDS: 0
INFO  :    SPILLED_RECORDS: 4
INFO  :    NUM_SHUFFLED_INPUTS: 1
INFO  :    NUM_SKIPPED_INPUTS: 0
INFO  :    NUM_FAILED_SHUFFLE_INPUTS: 0
INFO  :    MERGED_MAP_OUTPUTS: 1
INFO  :    INPUT_RECORDS_PROCESSED: 2
INFO  :    INPUT_SPLIT_LENGTH_BYTES: 12
INFO  :    OUTPUT_RECORDS: 3
INFO  :    APPROXIMATE_INPUT_RECORDS: 1
INFO  :    OUTPUT_LARGE_RECORDS: 0
INFO  :    OUTPUT_BYTES: 11
INFO  :    OUTPUT_BYTES_WITH_OVERHEAD: 29
INFO  :    OUTPUT_BYTES_PHYSICAL: 61
INFO  :    ADDITIONAL_SPILLS_BYTES_WRITTEN: 0
INFO  :    ADDITIONAL_SPILLS_BYTES_READ: 22
INFO  :    ADDITIONAL_SPILL_COUNT: 0
INFO  :    SHUFFLE_CHUNK_COUNT: 1
INFO  :    SHUFFLE_BYTES: 22
INFO  :    SHUFFLE_BYTES_DECOMPRESSED: 18
INFO  :    SHUFFLE_BYTES_TO_MEM: 0
INFO  :    SHUFFLE_BYTES_TO_DISK: 0
INFO  :    SHUFFLE_BYTES_DISK_DIRECT: 22
INFO  :    NUM_MEM_TO_DISK_MERGES: 0
INFO  :    NUM_DISK_TO_DISK_MERGES: 0
INFO  :    SHUFFLE_PHASE_TIME: 97
INFO  :    MERGE_PHASE_TIME: 76
INFO  :    FIRST_EVENT_RECEIVED: 69
INFO  :    LAST_EVENT_RECEIVED: 69
INFO  :    DATA_BYTES_VIA_EVENT: 15
INFO  : HIVE:
INFO  :    CREATED_FILES: 1
INFO  :    DESERIALIZE_ERRORS: 0
INFO  :    RECORDS_IN_Map_1: 3
INFO  :    RECORDS_OUT_0: 1
INFO  :    RECORDS_OUT_INTERMEDIATE_Map_1: 2
INFO  :    RECORDS_OUT_INTERMEDIATE_Reducer_2: 1
INFO  :    RECORDS_OUT_INTERMEDIATE_Reducer_3: 0
INFO  :    RECORDS_OUT_OPERATOR_FS_19: 1
INFO  :    RECORDS_OUT_OPERATOR_GBY_13: 2
INFO  :    RECORDS_OUT_OPERATOR_GBY_15: 2
INFO  :    RECORDS_OUT_OPERATOR_GBY_16: 1
INFO  :    RECORDS_OUT_OPERATOR_GBY_18: 1
INFO  :    RECORDS_OUT_OPERATOR_MAP_0: 0
INFO  :    RECORDS_OUT_OPERATOR_RS_14: 2
INFO  :    RECORDS_OUT_OPERATOR_RS_17: 1
INFO  :    RECORDS_OUT_OPERATOR_SEL_12: 3
INFO  :    RECORDS_OUT_OPERATOR_TS_0: 3
INFO  : Shuffle Errors:
INFO  :    BAD_ID: 0
INFO  :    CONNECTION: 0
INFO  :    IO_ERROR: 0
INFO  :    WRONG_LENGTH: 0
INFO  :    WRONG_MAP: 0
INFO  :    WRONG_REDUCE: 0
INFO  : org.apache.hadoop.hive.llap.LlapThreadLocalStatistics$LlapExecutorCounters:
INFO  :    EXECUTOR_CPU_NS: 4180180300
INFO  :    EXECUTOR_USER_NS: 3170000000
INFO  : org.apache.hadoop.hive.llap.counters.LlapIOCounters:
INFO  :    CACHE_MISS_BYTES: 12
INFO  :    CONSUMER_TIME_NS: 2885588
INFO  :    DECODE_TIME_NS: 47502348
INFO  :    IO_CPU_NS: 805223800
INFO  :    IO_USER_NS: 740000000
INFO  :    NUM_DECODED_BATCHES: 1
INFO  :    NUM_VECTOR_BATCHES: 1
INFO  :    ROWS_EMITTED: 3
INFO  :    TOTAL_IO_TIME_NS: 1167676778
INFO  : org.apache.hadoop.hive.llap.counters.LlapWmCounters:
----------------------------------------------------------------------------------------------
       VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 ..........      llap     SUCCEEDED      1          1        0        0       0       0
Reducer 2 ......      llap     SUCCEEDED      1          1        0        0       0       0
Reducer 3 ......      llap     SUCCEEDED      1          1        0        0       0       0
----------------------------------------------------------------------------------------------
VERTICES: 03/03  [==========================>>] 100%  ELAPSED TIME: 11.02 s
----------------------------------------------------------------------------------------------
INFO  : Completed executing command(queryId=hive_20260524203854_d1018576-27ac-4a0f-8cbf-01bab307db0e); Time taken: 13.939 seconds
+------+
| _c0  |
+------+
| 2    |
+------+
1 row selected (21.814 seconds)
INFO  : Compiling command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d): select sum(b) from hive_example
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d); Time taken: 4.025 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d): select sum(b) from hive_example
INFO  : Query ID = hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d
INFO  : Total jobs = 1
INFO  : Launching Job 1 out of 1
INFO  : Starting task [Stage-1:MAPRED] in serial mode
INFO  : Subscribed to counters: [] for queryId: hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d
INFO  : Session is already open
INFO  : Dag name: select sum(b) from hive_example (Stage-1)
INFO  : HS2 Host: [hive-hiveserver2-757b7b5987-257nv], Query ID: [hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d], Dag ID: [dag_1779654818589_0000_3], DAG App ID: [application_1779654818589_0000], DAG App address: [hive-tezam-0.hive-tezam.default.svc.cluster.local]
INFO  : Status: Running (Executing on YARN cluster with App id application_1779654818589_0000)

----------------------------------------------------------------------------------------------
       VERTICES      MODE        STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
----------------------------------------------------------------------------------------------
Map 1 ..........      llap     SUCCEEDED      1          1        0        0       0       0
Reducer 2 ......      llap     SUCCEEDED      1          1        0        0       0       0
----------------------------------------------------------------------------------------------
VERTICES: 02/02  [==========================>>] 100%  ELAPSED TIME: 20.78 s
----------------------------------------------------------------------------------------------
INFO  : Status: DAG finished successfully in 20.35 seconds
INFO  : DAG ID: dag_1779654818589_0000_3
INFO  :
INFO  : Query Execution Summary
INFO  : ----------------------------------------------------------------------------------------------
INFO  : OPERATION                            DURATION
INFO  : ----------------------------------------------------------------------------------------------
INFO  : Compile Query                           4.03s
INFO  : Prepare Plan                            0.51s
INFO  : Get Query Coordinator (AM)              0.00s
INFO  : Submit Plan                             0.25s
INFO  : Start DAG                               0.13s
INFO  : Run DAG                                20.35s
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : Task Execution Summary
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      DURATION(ms)   CPU_TIME(ms)    GC_TIME(ms)   INPUT_RECORDS   OUTPUT_RECORDS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1            525.00              0              0               3                1
INFO  :  Reducer 2          18743.00              0              0               1                0
INFO  : ----------------------------------------------------------------------------------------------
INFO  : FileSystem Counters Summary
INFO  :
INFO  : Scheme: S3A
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1             12B             2                  0                 0B             0
INFO  :  Reducer 2              0B             6                  0               101B             4
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : Scheme: FILE
INFO  : ----------------------------------------------------------------------------------------------
INFO  :   VERTICES      BYTES_READ      READ_OPS     LARGE_READ_OPS      BYTES_WRITTEN     WRITE_OPS
INFO  : ----------------------------------------------------------------------------------------------
INFO  :      Map 1              0B             0                  0                 0B             0
INFO  :  Reducer 2              0B             0                  0                 0B             0
INFO  : ----------------------------------------------------------------------------------------------
INFO  :
INFO  : org.apache.tez.common.counters.DAGCounter:
INFO  :    NUM_SUCCEEDED_TASKS: 2
INFO  :    TOTAL_LAUNCHED_TASKS: 2
INFO  :    DURATION_SUCCEEDED_TASKS_MILLIS: 19017
INFO  :    DATA_LOCAL_TASKS: 1
INFO  :    AM_CPU_MILLISECONDS: 9160
INFO  :    WALL_CLOCK_MILLIS: 19017
INFO  :    AM_GC_TIME_MILLIS: 373
INFO  :    INITIAL_HELD_CONTAINERS: 0
INFO  :    TOTAL_CONTAINERS_USED: 2
INFO  :    TOTAL_CONTAINER_LAUNCH_COUNT: 2
INFO  :    TOTAL_CONTAINER_RELEASE_COUNT: 2
INFO  :    NODE_USED_COUNT: 2
INFO  :    NODE_TOTAL_COUNT: 2
INFO  : File System Counters:
INFO  :    FILE_BYTES_READ: 0
INFO  :    FILE_BYTES_WRITTEN: 0
INFO  :    FILE_READ_OPS: 0
INFO  :    FILE_LARGE_READ_OPS: 0
INFO  :    FILE_WRITE_OPS: 0
INFO  :    S3A_BYTES_READ: 12
INFO  :    S3A_BYTES_WRITTEN: 101
INFO  :    S3A_READ_OPS: 8
INFO  :    S3A_LARGE_READ_OPS: 0
INFO  :    S3A_WRITE_OPS: 4
INFO  : org.apache.tez.common.counters.TaskCounter:
INFO  :    SPILLED_RECORDS: 0
INFO  :    NUM_SHUFFLED_INPUTS: 0
INFO  :    NUM_FAILED_SHUFFLE_INPUTS: 0
INFO  :    INPUT_RECORDS_PROCESSED: 2
INFO  :    INPUT_SPLIT_LENGTH_BYTES: 12
INFO  :    OUTPUT_RECORDS: 1
INFO  :    APPROXIMATE_INPUT_RECORDS: 1
INFO  :    OUTPUT_LARGE_RECORDS: 0
INFO  :    OUTPUT_BYTES: 3
INFO  :    OUTPUT_BYTES_WITH_OVERHEAD: 11
INFO  :    OUTPUT_BYTES_PHYSICAL: 39
INFO  :    ADDITIONAL_SPILLS_BYTES_WRITTEN: 0
INFO  :    ADDITIONAL_SPILLS_BYTES_READ: 0
INFO  :    ADDITIONAL_SPILL_COUNT: 0
INFO  :    SHUFFLE_BYTES: 0
INFO  :    SHUFFLE_BYTES_DECOMPRESSED: 0
INFO  :    SHUFFLE_BYTES_TO_MEM: 0
INFO  :    SHUFFLE_BYTES_TO_DISK: 0
INFO  :    SHUFFLE_BYTES_DISK_DIRECT: 0
INFO  :    SHUFFLE_PHASE_TIME: 108
INFO  :    FIRST_EVENT_RECEIVED: 107
INFO  :    LAST_EVENT_RECEIVED: 107
INFO  :    DATA_BYTES_VIA_EVENT: 15
INFO  : HIVE:
INFO  :    CREATED_FILES: 1
INFO  :    DESERIALIZE_ERRORS: 0
INFO  :    RECORDS_IN_Map_1: 3
INFO  :    RECORDS_OUT_0: 1
INFO  :    RECORDS_OUT_INTERMEDIATE_Map_1: 1
INFO  :    RECORDS_OUT_INTERMEDIATE_Reducer_2: 0
INFO  :    RECORDS_OUT_OPERATOR_FS_11: 1
INFO  :    RECORDS_OUT_OPERATOR_GBY_10: 1
INFO  :    RECORDS_OUT_OPERATOR_GBY_8: 1
INFO  :    RECORDS_OUT_OPERATOR_MAP_0: 0
INFO  :    RECORDS_OUT_OPERATOR_RS_9: 1
INFO  :    RECORDS_OUT_OPERATOR_SEL_7: 3
INFO  :    RECORDS_OUT_OPERATOR_TS_0: 3
INFO  : org.apache.hadoop.hive.llap.LlapThreadLocalStatistics$LlapExecutorCounters:
INFO  :    EXECUTOR_CPU_NS: 10931285300
INFO  :    EXECUTOR_USER_NS: 9650000000
INFO  : org.apache.hadoop.hive.llap.counters.LlapIOCounters:
INFO  :    CACHE_MISS_BYTES: 12
INFO  :    CONSUMER_TIME_NS: 31652541
INFO  :    DECODE_TIME_NS: 6697414
INFO  :    IO_CPU_NS: 147236800
INFO  :    IO_USER_NS: 120000000
INFO  :    NUM_DECODED_BATCHES: 1
INFO  :    NUM_VECTOR_BATCHES: 1
INFO  :    ROWS_EMITTED: 3
INFO  :    TOTAL_IO_TIME_NS: 262142261
INFO  : org.apache.hadoop.hive.llap.counters.LlapWmCounters:
INFO  :    GUARANTEED_QUEUED_NS: 0
INFO  :    GUARANTEED_RUNNING_NS: 0
INFO  :    SPECULATIVE_QUEUED_NS: 1262906425
INFO  :    SPECULATIVE_RUNNING_NS: 16623777320
INFO  : org.apache.hadoop.hive.ql.exec.tez.HiveInputCounters:
INFO  :    GROUPED_INPUT_SPLITS_Map_1: 1
INFO  :    INPUT_DIRECTORIES_Map_1: 1
INFO  :    INPUT_FILES_Map_1: 1
INFO  :    RAW_INPUT_SPLITS_Map_1: 1
INFO  : Completed executing command(queryId=hive_20260524203916_aecfe59c-a918-4c32-9ac0-4f39c808e11d); Time taken: 21.61 seconds
+------+
| _c0  |
+------+
| 6    |
+------+
1 row selected (25.961 seconds)
0: jdbc:hive2://hive-hiveserver2:10000/>
org.jline.reader.UserInterruptException
Closing: 0: jdbc:hive2://hive-hiveserver2:10000/
command terminated with exit code 2
PS I:\simhadrig\hive>

asf-ci-hive added the tests pending label Apr 24, 2026

ayushtkn changed the title ~~WIP: K8s operator~~ WIP: HIVE-29579: K8s operator Apr 24, 2026

asf-ci-hive added tests failed and removed tests pending labels Apr 24, 2026

ayushtkn force-pushed the k8sOperator branch from 11ebef6 to 3deaee8 Compare April 24, 2026 14:37

ayushtkn marked this pull request as draft April 24, 2026 14:39

asf-ci-hive added tests pending tests failed and removed tests failed tests pending labels Apr 24, 2026

ayushtkn force-pushed the k8sOperator branch from 3deaee8 to f56cf2d Compare April 24, 2026 19:50

asf-ci-hive added tests pending tests unstable and removed tests failed tests pending labels Apr 25, 2026

ayushtkn force-pushed the k8sOperator branch from f56cf2d to 553a07a Compare April 28, 2026 14:05

ayushtkn marked this pull request as ready for review April 28, 2026 14:05

asf-ci-hive added tests pending tests passed and removed tests unstable tests pending labels Apr 28, 2026

ayushtkn changed the title ~~WIP: HIVE-29579: K8s operator~~ HIVE-29579: K8s operator May 5, 2026

ayushtkn force-pushed the k8sOperator branch from 553a07a to e11017e Compare May 5, 2026 20:47

asf-ci-hive added tests pending and removed tests passed labels May 5, 2026

github-advanced-security AI found potential problems May 5, 2026

View reviewed changes

Comment thread ...tes/src/java/org/apache/hive/kubernetes/operator/dependent/MetastoreDeploymentDependent.java Fixed

ayushtkn force-pushed the k8sOperator branch from bb3714b to a068c02 Compare May 10, 2026 20:28

asf-ci-hive added tests pending and removed tests passed labels May 10, 2026

ayushtkn force-pushed the k8sOperator branch from a068c02 to 4ee4d95 Compare May 10, 2026 20:38

asf-ci-hive added tests failed tests pending tests passed and removed tests pending tests failed labels May 10, 2026

ayushtkn force-pushed the k8sOperator branch from 4ee4d95 to 41780fb Compare May 11, 2026 10:53

asf-ci-hive added tests pending tests passed and removed tests passed tests pending labels May 11, 2026

ayushtkn force-pushed the k8sOperator branch from 41780fb to d68e236 Compare May 13, 2026 09:05

asf-ci-hive added tests pending tests passed and removed tests passed tests pending labels May 13, 2026

ayushtkn force-pushed the k8sOperator branch from d68e236 to d7dc08d Compare May 14, 2026 09:49

asf-ci-hive added tests pending and removed tests passed labels May 14, 2026

ayushtkn added 3 commits May 20, 2026 16:42

Add Hive K8s Operators

4d71208

HELM Charts

b1af901

Upgrade & Refactor

e25466c

simhadri-g approved these changes May 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HIVE-29579: K8s operator#6452

HIVE-29579: K8s operator#6452
ayushtkn wants to merge 3 commits into
apache:masterfrom
ayushtkn:k8sOperator

ayushtkn commented Apr 24, 2026 •

edited

Loading

Uh oh!

zhangbutao commented Apr 24, 2026

Uh oh!

ayushtkn commented May 4, 2026 •

edited

Loading

Uh oh!

zhangbutao commented May 5, 2026

Uh oh!

ayushtkn commented May 5, 2026 •

edited

Loading

Uh oh!

Uh oh!

ayushtkn commented May 19, 2026 •

edited

Loading

Uh oh!

sonarqubecloud Bot commented May 24, 2026

Uh oh!

simhadri-g commented May 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

ayushtkn commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Via HELM Chart

External S3 Bucket

Google GCS Bucket

Via Operator

Testing with External S3 Bucket

Testing with GCS Bucket

With Oracle DB + S3 Bucket

External HMS

Uh oh!

zhangbutao commented Apr 24, 2026

Uh oh!

ayushtkn commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhangbutao commented May 5, 2026

Uh oh!

ayushtkn commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ayushtkn commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sonarqubecloud Bot commented May 24, 2026

Quality Gate passed

Uh oh!

simhadri-g commented May 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ayushtkn commented Apr 24, 2026 •

edited

Loading

ayushtkn commented May 4, 2026 •

edited

Loading

ayushtkn commented May 5, 2026 •

edited

Loading

ayushtkn commented May 19, 2026 •

edited

Loading