Number of gRPC PUT operations is always twice the number of s3 requests #102

New issue

Closed

opened 2023-04-28 10:58:29 +00:00 by a.bogatyrev · 3 comments

a.bogatyrev commented

2023-04-28 10:58:29 +00:00

Owner

With a load flow in 1 REP, the number of gRPC PUT operations is always twice the number of s3 requests.

For example:

CID:

container ID: 9qjAg5PMgAFsmzEZA5P5PRR5meFDbCphqBPxDwqhxZuw
owner ID: Nbq31gnvQBSQPHTEwbwuXAJf4ZFStsHbKs
basic ACL: fbfbfff (eacl-public-read-write)
...
placement policy:
REP 1
CBF 1
SELECT 1 FROM *

With a load flow in 1 REP, the number of gRPC PUT operations is always twice the number of s3 requests. For example: ![image](/attachments/f19e3fb1-048a-4718-b0bb-a82f3f7f94f6) ![image](/attachments/b4dee654-0d26-427f-9384-51d331995dc8) CID: ``` container ID: 9qjAg5PMgAFsmzEZA5P5PRR5meFDbCphqBPxDwqhxZuw owner ID: Nbq31gnvQBSQPHTEwbwuXAJf4ZFStsHbKs basic ACL: fbfbfff (eacl-public-read-write) ... placement policy: REP 1 CBF 1 SELECT 1 FROM * ```

image.png

54 KiB

image.png

55 KiB

a.bogatyrev commented

2023-04-28 11:08:00 +00:00

Author

Owner

The same issue is actual and for REP2. For example:

The same issue is actual and for REP2. For example: ![image](/attachments/9a41d42d-6d47-4711-aba5-8fd237206365) ![image](/attachments/d4480902-3fe5-4297-a621-131ae4579fa0)

image.png

112 KiB

image.png

53 KiB

snegurochka added the

bug

label 2023-05-03 17:15:01 +00:00

alexvanin self-assigned this 2023-05-04 13:29:13 +00:00

alexvanin commented

2023-05-04 14:28:24 +00:00

Owner

@a.bogatyrev can you provide exact metric names which are used in those graphs?

alexvanin commented

2023-08-31 10:46:01 +00:00

Owner

With a load flow in 1 REP, the number of gRPC PUT operations is always twice the number of s3 requests.

This happens when you upload object to the node out of placement. Then one S3 requests produced two gRPC requests: one from gateway to node and one redirection from node to another node by placement policy.

Also make sure to filter api labels in frostfs_s3_requests_total, because it might include other operations besides PutObject.

frostfs_s3_requests_total{api="CreateBucket"} 1
frostfs_s3_requests_total{api="PutObject"} 4

/cc @a.bogatyrev

> With a load flow in 1 REP, the number of gRPC PUT operations is always twice the number of s3 requests. This happens when you upload object to the node out of placement. Then one S3 requests produced two gRPC requests: one from gateway to node and one redirection from node to another node by placement policy. Also make sure to filter `api` labels in `frostfs_s3_requests_total`, because it might include other operations besides `PutObject`. ``` frostfs_s3_requests_total{api="CreateBucket"} 1 frostfs_s3_requests_total{api="PutObject"} 4 ``` /cc @a.bogatyrev

alexvanin closed this issue

2023-08-31 10:46:01 +00:00