TrueCloudLab/frostfs-node

Fork 27

Possible problems with XHeader processing #465

New issue

Closed

opened 2023-06-26 11:32:09 +00:00 by fyrchik · 1 comment

fyrchik commented

2023-06-26 11:32:09 +00:00

Owner

We don't set netmap epoch XHeader for auxiliary queries in the object service

// FIXME: (neofs-node#1194) not supported by client

However, it seems we set it impliticly via the x-headers. It is a problem:

The code doesn't do what it is supposed to be doing.
Suddely all our operations become exponential in complexity netmap_depth ^ TTL (first node searches at epochs N..N-D+1, another nodes searches N..N-D+1, N-1..N-D etc.), because XHeaders are being sent unchanged.

API description is vague

// Netmap epoch to use for object placement calculation. The `value` is string

Let's discuss and fix the behaviour.
cc @alexvanin @realloc

We don't set netmap epoch XHeader for auxiliary queries in the object service https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/71a63b8e9c629fb7efb5343e108995296df97ed3/pkg/services/object/internal/client/client.go#L84 However, it seems we set it _impliticly_ via the x-headers. It is a problem: 1. The code doesn't do what it is supposed to be doing. 2. Suddely all our operations become exponential in complexity `netmap_depth ^ TTL` (first node searches at epochs N..N-D+1, another nodes searches N..N-D+1, N-1..N-D etc.), because XHeaders are being sent unchanged. API description is vague https://git.frostfs.info/TrueCloudLab/frostfs-api/src/commit/dbfa9c944bd7486f96daad3090cad6c238e9f833/session/types.proto#L145 Let's discuss and fix the behaviour. cc @alexvanin @realloc

fyrchik added the

labels 2023-06-26 11:32:09 +00:00

fyrchik added this to the v0.38.0 milestone 2023-06-27 06:53:52 +00:00

acid-ant was assigned by fyrchik

2023-06-27 11:13:21 +00:00

fyrchik modified the milestone from v0.38.0 to v0.39.0

2024-02-12 06:28:44 +00:00

acid-ant was unassigned by fyrchik

2024-02-12 06:28:48 +00:00

fyrchik modified the milestone from v0.39.0 to v0.40.0

2024-05-14 14:15:49 +00:00

fyrchik modified the milestone from v0.40.0 to v0.41.0

2024-06-01 09:19:49 +00:00

fyrchik modified the milestone from v0.41.0 to v0.42.0

2024-06-14 07:07:39 +00:00

acid-ant self-assigned this 2024-07-01 06:50:13 +00:00

fyrchik modified the milestone from v0.42.0 to v0.43.0

2024-07-23 06:34:45 +00:00

fyrchik modified the milestone from v0.43.0 to v0.44.0

2024-09-30 11:51:37 +00:00

fyrchik modified the milestone from v0.44.0 to v0.45.0

2024-11-25 10:46:50 +00:00

fyrchik commented

2024-12-25 09:04:23 +00:00

Author

Owner

The problem with request amplification is not here, because it DEPTH header is filtered

 	for i := range xHdrs {
 		switch key := xHdrs[i].GetKey(); key {
 		case session.XHeaderNetmapEpoch:
 			var err error
 			prm.netmapEpoch, err = strconv.ParseUint(xHdrs[i].GetValue(), 10, 64)
 			if err != nil {
 				return nil, err
 			}
 		case session.XHeaderNetmapLookupDepth:
 			var err error
 			prm.netmapLookupDepth, err = strconv.ParseUint(xHdrs[i].GetValue(), 10, 64)
 			if err != nil {
 				return nil, err
 			}
 		default:
 			prm.xhdrs = append(prm.xhdrs, key, xHdrs[i].GetValue())
 		}
 	}

The EPOCH header still needs to be attached, though.

The problem with request amplification is not here, because it DEPTH header is filtered https://git.frostfs.info/TrueCloudLab/frostfs-node/src/commit/2832f4443750fb11ee827be5491f34f749c9122a/pkg/services/object/util/prm.go#L158-L177 The EPOCH header still needs to be attached, though.

fyrchik referenced this issue from a pull request that will close it,

2024-12-25 09:06:05 +00:00

objsvc: Set NETMAP_EPOCH xheader for auxiliary requests #1580

fyrchik referenced this issue from a commit

2024-12-26 09:18:01 +00:00

[#465] objsvc: Set NETMAP_EPOCH xheader for auxiliary requests

fyrchik closed this issue

2024-12-26 09:18:01 +00:00