Repository navigation
docs(mayastor): document dedicating a network interface for storage traffic - #696
Open
krishnaGajabi wants to merge 3 commits into
Open
krishnaGajabi wants to merge 3 commits into
krishnaGajabi wants to merge 3 commits into
Conversation
niladrih
approved these changes
Nov 4, 2026
niladrih
left a comment
niladrih
left a comment
Member
There was a problem hiding this comment.
Should this have examples for IPv6 as well?
|
|
||
| ## Network Isolation | ||
|
|
||
| By default, the NVMe-oF target binds to the IO engine pod's own IP address, which places volume |
Member
There was a problem hiding this comment.
hmm we don't use the cluster pod network, it's always using the host network
By default it uses the K8s node IP
Contributor
Author
There was a problem hiding this comment.
made the changes
Comment on lines
+124
to
+125
| Pointing the target at a dedicated interface - a separate NIC, a VLAN, or a storage-only subnet | ||
| - keeps that traffic on its own path. |
Member
There was a problem hiding this comment.
does this read a bit funny with the newline and - ?
Contributor
Author
There was a problem hiding this comment.
made the changes, PTAL
| will disrupt any volume whose target is hosted on the node you restart. | ||
| ::: | ||
|
|
||
| Because this changes the address that volume targets are published on, the targets hosted on a |
Member
There was a problem hiding this comment.
Actually it's worse than that leading to full rebuilds and potentially a broken application if HA is not enabled
Contributor
Author
There was a problem hiding this comment.
made the changes
Contributor
Author
made the changes |
…raffic The NVMe-oF target binds to the IO engine pod's IP unless io_engine.target.nvmf.iface is set, which leaves volume traffic sharing the cluster pod network with application and control plane traffic. The Performance Tips page covered CPU isolation but said nothing about the network equivalent. Add a Network Isolation section covering the value syntax, why a single Helm value means subnet: generalises across nodes where name: often does not, how to apply it at install time and on an existing deployment, how to verify it from the IO engine logs, and which traffic it does not move. Signed-off-by: krishnaGajabi <gajbikrishna23@gmail.com>
The IO engine DaemonSet is the only one in the chart using updateStrategy: OnDelete, so a plain helm upgrade updates the DaemonSet and leaves the running pods on their old -T= argument. The setting looks like it did nothing until the pods are restarted. Say so, and point at the kubectl plugin, which restarts the IO engines one node at a time and moves volume targets off a node first. Signed-off-by: krishnaGajabi <gajbikrishna23@gmail.com>
The IO engine runs with host networking, so the NVMe-oF target binds to the node's own address; there is no pod network in that path. The same mistake ran through "What This Does Not Move", which listed gRPC, etcd and the REST API together as pod network traffic. The IO engine's own gRPC endpoint, the HA node agent and the CSI node and controller plugins all run with host networking and stay on the node address; only agent-core, api-rest and etcd use the pod network. Restarting an IO engine republishes its targets, which costs more than the earlier wording implied. Initiators connected to those targets lose their path, recovered by the HA node agent when agents.ha.enabled is set and not recovered at all when it is not, and replicas on the restarted node are rebuilt in full rather than resynced. Also record that subnet: accepts IPv6 CIDRs, and that the value selects the interface rather than the address family -- the family published follows the IO engine's gRPC address and falls back to whatever the interface holds. The verification step now points at the log line that confirms the match, rather than only the one that reports the attempt. Signed-off-by: krishnaGajabi <gajbikrishna23@gmail.com>
krishnaGajabi
force-pushed
the
docs/dedicated-storage-interface
branch
from
November 11, 2026 07:33
d0e846d to
63037f5
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Performance Tips page covers CPU isolation but nothing about the network equivalent, even though the setting to do it already exists.
Unless
io_engine.target.nvmf.ifaceis set, the NVMe-oF target binds to the IO engine pod's own IP, so volume traffic shares the cluster pod network with application and control plane traffic. Setting it moves the target onto a chosen interface — and because every listener shares the target address, both client I/O and replication between nodes follow it.Adds a Network Isolation section to
docs/main/.../additional-information/performance-tips.mdcovering:<kind>:<value>syntax, and that an unprefixed value is matched as an interface name (so an unprefixed CIDR matches nothing and the IO engine fails to start)subnet:usually generalises better thanname:— this is a single Helm value rendered into the DaemonSet, so the same value reaches every IO engine pod, and interface names often differ between nodesDocs only. Built locally with
npm run buildand checked the rendered page under themainversion.