1 What: /sys/class/rtrs-client
4 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
5 Description: When a user of RTRS API creates a new session, a directory entry with
6 the name of that session is created under /sys/class/rtrs-client/<session-name>/
8 What: /sys/class/rtrs-client/<session-name>/add_path
11 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
12 Description: RW, adds a new path (connection) to an existing session. Expected format is the
15 <[source addr,]destination addr>
16 *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ]
18 What: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts
21 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
22 Description: Maximum number reconnect attempts the client should make before giving up
23 after connection breaks unexpectedly.
25 What: /sys/class/rtrs-client/<session-name>/mp_policy
28 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
29 Description: Multipath policy specifies which path should be selected on each IO:
32 select path in per CPU round-robin manner.
35 select path with minimum inflights.
38 select path with minimum latency.
40 What: /sys/class/rtrs-client/<session-name>/paths/
43 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
44 Description: Each path belonging to a given session is listed here by its source and
45 destination address. When a new path is added to a session by writing to
46 the "add_path" entry, a directory <src@dst> is created.
48 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state
51 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
52 Description: RO, Contains "connected" if the session is connected to the peer and fully
53 functional. Otherwise the file contains "disconnected"
55 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect
58 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
59 Description: Write "1" to the file in order to reconnect the path.
60 Operation is blocking and returns 0 if reconnect was successful.
62 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect
65 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
66 Description: Write "1" to the file in order to disconnect the path.
67 Operation blocks until RTRS path is disconnected.
69 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path
72 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
73 Description: Write "1" to the file in order to disconnected and remove the path
74 from the session. Operation blocks until the path is disconnected
75 and removed from the session.
77 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name
80 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
81 Description: RO, Contains the name of HCA the connection established on.
83 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port
86 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
87 Description: RO, Contains the port number of active port traffic is going through.
89 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr
92 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
93 Description: RO, Contains the source address of the path
95 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr
98 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
99 Description: RO, Contains the destination address of the path
101 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/cur_latency
104 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
105 Description: RO, Contains the latency time calculated by the heart-beat messages.
106 Whenever the client sends heart-beat message, it checks the time gap
107 between sending the heart-beat message and receiving the ACK.
108 This value can be changed regularly.
110 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all
113 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
114 Description: RW, Read will return usage help, write 0 will clear all the statistics.
116 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration
119 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
120 Description: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's
121 not the case, the processing of an I/O response could be processed on a
122 different CPU than where it was originally submitted. This file shows
123 how many interrupts where generated on a non expected CPU.
126 is the CPU on which the IRQ was expected, but not generated.
128 is the CPU on which the IRQ was generated, but not expected.
130 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects
133 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
134 Description: Contains 2 unsigned int values, the first one records number of successful
135 reconnects in the path lifetime, the second one records number of failed
136 reconnects in the path lifetime.
138 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma
141 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
142 Description: Contains statistics regarding rdma operations and inflight operations.
143 The output consists of 6 values::
145 <read-count> <read-total-size> <write-count> \
146 <write-total-size> <inflights> <failovered>