Commit graph

217 commits

Author SHA1 Message Date
Casey Bodley
ed8944c9cb s3: reenable tests with range requests and checksums
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2025-02-19 15:12:42 -05:00
Casey Bodley
cc56036255 s3: disable multipart/ranged tests with checksum failures
boto update exposed some bugs in our checksum feature around multipart
copy and ranged requests. disabling them with fails_on_rgw until
https://tracker.ceph.com/issues/69936 is resolved

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2025-02-13 11:55:09 -05:00
Casey Bodley
dfb8fc5b16 s3: disable request_checksum_calculation for test_multipart_checksum_upload_fallback
the test sends create_multipart_upload() with a requested checksum
algorithm, and upload_part() requests without. newer boto automatically
adds `x-amz-sdk-checksum-algorithm: CRC32` when nothing is specified,
so we have to explicitly disable that behavior via botocore config

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2025-02-13 11:55:09 -05:00
Casey Bodley
5aff597245 s3: clean up use of tenanted bucket names
the botocore.handlers.validate_bucket_name validation is enabled by
default, but we can disable it with unregister(). this avoids having to
mess with the signature or url

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2025-02-13 11:55:09 -05:00
Casey Bodley
25a502872b s3: sse-kms policy tests use v4 client
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2025-02-13 11:55:09 -05:00
Casey Bodley
e786aa4f42 s3: mark sigv2 tests with fails_on_rgw
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2025-02-13 11:55:08 -05:00
Raja
3456f8b66c
Merge branch 'ceph:master' into wip-raja-fix-multipart-upload-cant-get-obj-tag 2025-01-29 20:44:09 +05:30
Raja
710dd45f31
Merge pull request #610 from ArbitCode/wip-raja-rgw-test-support-64526
rgw test-case to support x-expected-bucket-owner
2025-01-28 22:11:37 +05:30
Raja Sharma
e8bcf49484 test-case to fix obj by multipart upload cant get tag
Signed-off-by: Raja Sharma <raja@ibm.com>
2025-01-21 17:52:28 +05:30
Raja Sharma
8f36772ce0 rgw test-case to support x-expected-bucket-owner
If the bucket is owned by a different account, the request fails with the HTTP status code 403 Forbidden (access denied).
PR: ceph/ceph#61215
Fixes: https://tracker.ceph.com/issues/64526

Signed-off-by: Raja Sharma <raja@ibm.com>
2025-01-17 23:33:34 +05:30
Yuval Lifshitz
75693ccff7 rgw/logging: add cleanup tests
* use flushing api instead of sleep when possible
* test flushing concurrency

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2025-01-15 16:38:15 +00:00
Yuval Lifshitz
87df05aaa4 rgw/logging: test log bucket encryption error
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2025-01-15 16:25:01 +00:00
Yuval Lifshitz
d1d1b4e8a4 rgw/logging: test source and log have different names
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2025-01-15 16:25:01 +00:00
Yuval Lifshitz
30ecc7cad6 rgw/logging: test tenant support
when source and log buckets are on a specific tenant, or
on different tenants

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2025-01-15 16:25:01 +00:00
Matt Benjamin
ff16b9c0c3
Merge pull request #571 from linuxbox2/wip-get-objattrs
test get_object_attributes
2025-01-13 11:09:29 -05:00
Adam Emerson
333a8a69a9
Merge pull request #549 from jzhu116-bloomberg/wip-63995
add test case for delete-marker expiration with 'Days' tag

Reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2025-01-08 16:03:34 -05:00
Matt Benjamin
a30f6e0010 mark attribute tests as failing on dbstore (for now)
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2025-01-07 15:24:56 -05:00
Casey Bodley
8f10b2e532 more tests for GetObjectAttributes
* multipart upload without checksums
* multipart upload with a single part
* pagination of multipart parts
* non-multipart upload with/without checksum
* versioned object, current and non-current
* sse-c encrypted object

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2025-01-07 15:24:46 -05:00
Matt Benjamin
5f94278e22 multipart fallback to create-multipart checksum algorithm
there seem to be workloads which assume checksum algorithm can be
omitted from upload-part

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2025-01-07 15:23:41 -05:00
Matt Benjamin
f4ce789ca1 test get_object_attributes
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2025-01-07 15:23:32 -05:00
Yuval Lifshitz
ae8bebd87f
Merge pull request #606 from yuvalif/wip-yuval-bucket-logging-flush
rgw/logging: tests for flush API

Reviewed-By: Ali Masarwe <ali.masarwa@ibm.com>
2024-12-18 13:44:24 +02:00
Casey Bodley
14768d8554
Merge pull request #601 from pritha-srivastava/wip-rgw-merge-store-tests
rgw/s3-tests: adding missing tests for testing delete operations (related to bucket policy and bucket public access block)
2024-12-17 16:34:03 -05:00
Seena Fallah
3145b1b0c3 HeadBucket: add read-stats query string for usage test
Refs: https://github.com/ceph/ceph/pull/56756

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-12-12 23:59:16 +01:00
Yuval Lifshitz
79169a8b4f rgw/logging: tests for flush API
as well as test for object key filters

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2024-12-11 20:54:21 +00:00
Casey Bodley
74188c64ba
Merge pull request #576 from yuvalif/wip-yuval-bucket-logging
rgw/logging: basic set of tests for bucket logging
2024-12-10 11:07:44 -05:00
Yuval Lifshitz
10d46f4e33 rgw/logging: basic set of tests for bucket logging
Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2024-12-05 20:45:50 +00:00
Casey Bodley
7c53738f0e
Merge pull request #603 from ArbitCode/wip-raja-rgw-test-fix-68211
rgw:modify testcase rgw header when bucket  or user quota is disabled
2024-11-25 10:27:12 -05:00
Raja Sharma
2aea98e604 rgw:modify testcase rgw header when bucket quota or user is disabled
By default quota is disable at user/bucket level.
if quota is disable then these value will not return so need to modify test_head_bucket_usages

'X-RGW-Quota-User-Size'
'X-RGW-Quota-User-Objects'
'X-RGW-Quota-Bucket-Size'
'X-RGW-Quota-Bucket-Objects'
to Fix: https://tracker.ceph.com/issues/68211

Signed-off-by: Raja Sharma raja@ibm.com
2024-11-20 23:09:21 +05:30
Pritha Srivastava
14f5672d09 rgw/s3tests: making the tests generic to check only
for 403 error code (and not specific errors) so that
the tests pass for any order of auth engines.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2024-11-20 09:42:18 +05:30
Pritha Srivastava
9a24fa8aba rgw/s3-tests: adding a test to check put, get, delete and
then get to ensure that public block access is deleted
successfully.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2024-11-12 11:15:48 +05:30
Pritha Srivastava
3fa3367190 rgw/s3-test: adding test to set, get, delete and then get
a bucket policy to ensure that delete correctly removes
the bucket policy

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2024-11-12 11:15:21 +05:30
Soumya Koduri
7362e522cc Updating lifecycle_noncur* tests to include objects with instance empty
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2024-10-10 00:49:03 +05:30
Casey Bodley
08df9352f9
Merge pull request #591 from cbodley/wip-68292
s3: test GetObject with PartNumber and SSE-C encryption
2024-10-04 17:21:04 -04:00
Casey Bodley
acc8ef43c9 s3: test GetObject with PartNumber and SSE-C encryption
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-09-27 15:06:21 -04:00
Casey Bodley
999d39d4db s3: add v2 signature presigned put_object tests
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-09-25 11:29:01 -04:00
Tobias Urdin
ac71900ffb Add v2 signature presigned get_object tests
This adds tests for get_object presigned URLs
using signature v2.

Also code formatting.

Signed-off-by: Tobias Urdin <tobias.urdin@binero.com>
2024-09-25 11:22:37 -04:00
Seena Fallah
ecf7a8a7a9 UploadPartCopy: add test for source bucket with policy
Ref: https://github.com/ceph/ceph/pull/59253

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-08-16 14:30:17 +02:00
Casey Bodley
2e41494293 s3: reenable tenanted bucket policy test
the before-call hook url-encodes the ':' part of tenanted bucket names
to resolve SignatureDoesNotMatch errors

removed the list-v2 version of the test since it isn't relevant to
bucket policy test coverage

add a new test case that creates the bucket under the tenanted user,
then uses the main client to access it

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2024-08-14 08:19:41 -04:00
Casey Bodley
218f90063f
Merge pull request #574 from clwluvw/sse-c-policy
BucketPolicy: add test for sse-c in conditions
2024-07-26 14:09:31 +01:00
Seena Fallah
c9aded48e5 BucketPolicy: decouple encryption tests from invalid algo and unencrypted
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-07-23 20:47:22 +02:00
Seena Fallah
87b496f25f BucketPolicy: add test for sse-c in conditions
Ref. https://github.com/ceph/ceph/pull/58689

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-07-19 23:07:03 +02:00
Seena Fallah
a83396cda7 BlockPublicPolicy: add test when policy has principal
Ref. https://tracker.ceph.com/issues/67048

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-07-19 20:51:10 +02:00
Seena Fallah
93a3b6c704 PolicyStatus: add test for policy with Principal
Ref. https://github.com/ceph/ceph/pull/58686

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-07-19 20:50:26 +02:00
Seena Fallah
474c1404e2 BucketPolicy: donot allow NotPrincipal with Allow Effect
Ref. https://github.com/ceph/ceph/pull/58686

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2024-07-19 20:48:06 +02:00
Casey Bodley
4eda9c0626
Merge pull request #570 from cbodley/wip-66705
test Get/HeadObject with partNumber for single-multipart upload
2024-07-05 15:51:15 +01:00
Matt Benjamin
8277a9fb9a mark two tests that fail on dbstore
also add @pytest.mark.checksum for new checksum
tests

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-07-03 09:53:08 -04:00
Matt Benjamin
c0f0b679db remove duplicate size assigment [rkhudov review]
Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-07-03 09:53:02 -04:00
Matt Benjamin
95df503ced add test_post_object_upload_checksum
this tests a two-megabyte binary upload with validated
(awscli-computed) SHA256 checksum, and also verifies failure when
a bad checksum is provided

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-07-03 09:52:55 -04:00
Matt Benjamin
9577cde013 add test_multipart_checksum_3parts
tests a full multipart upload cycle with 3 unique parts, which
verifies composite checksum computation and the logic to propagate
parts_count to ComleteMultipart

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-07-03 09:52:49 -04:00
Matt Benjamin
a3dbac7115 test_multipart_upload_sha256: work around failures re-trying complete-multipart
As described in https://tracker.ceph.com/issues/65746, retrying complete-multipart
after having attempted to complete the same upload with a bad checksum argument
fails with an internal error.

The status code is 500, but I'm unsure if it can be retried again, or whether
the upload can be aborted later.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2024-07-03 09:52:39 -04:00