Skip to content

feat(services/hdfs-native): service hdfs-native support high availability (HA) cluster#7248

Merged
Xuanwo merged 1 commit intoapache:mainfrom
qingfeng-occ:hdfs-native-ha
Mar 31, 2026
Merged

feat(services/hdfs-native): service hdfs-native support high availability (HA) cluster#7248
Xuanwo merged 1 commit intoapache:mainfrom
qingfeng-occ:hdfs-native-ha

Conversation

@qingfeng-occ
Copy link
Copy Markdown
Contributor

@qingfeng-occ qingfeng-occ commented Mar 18, 2026

Which issue does this PR close?

Closes #7246 .

Rationale for this change

The current service hdfs-native only supports connecting to HDFS clusters with a single active NameNode, such as hdfs://127.0.0.1:9000. If the HDFS cluster is started in High Availability (HA) mode, hdfs-native will fail to connect when the active NameNode failover.

What changes are included in this PR?

HdfsActiveBuilder supports both single active NameNode and High Availability (HA) modes via the name_node parameter.The builder automatically parses these URIs into standard configuration properties required to initialize the hdfs-native client.

  • Standalone Mode: Specify a single URI, e.g., hdfs://127.0.0.1:9000.
  • HA Mode: Specify multiple NameNodes, e.g., hdfs://namenode1:9000,namenode2:9000.

Additionally, any other HDFS configurations(Currently the supported configs refer to https://github.com/Kimahriman/hdfs-native) can be passed via the options parameter.

Are there any user-facing changes?

AI Usage Statement

@qingfeng-occ qingfeng-occ requested a review from Xuanwo as a code owner March 18, 2026 07:50
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Mar 18, 2026
@qingfeng-occ qingfeng-occ marked this pull request as draft March 18, 2026 08:04
@qingfeng-occ qingfeng-occ marked this pull request as ready for review March 18, 2026 08:35
@qingfeng-occ qingfeng-occ force-pushed the hdfs-native-ha branch 3 times, most recently from 0bbd83b to c0b047a Compare March 23, 2026 00:52
Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for working on this! Only a small question.

Comment thread core/services/hdfs-native/src/backend.rs Outdated
…lity (HA) cluster

Signed-off-by: qingfeng <qing.feng@zte.com.cn>
Copy link
Copy Markdown
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Mar 31, 2026
@Xuanwo Xuanwo merged commit 611784f into apache:main Mar 31, 2026
98 checks passed
@qingfeng-occ qingfeng-occ deleted the hdfs-native-ha branch March 31, 2026 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: service hdfs-native support high availability (HA) cluster

2 participants