Commit graph

121 commits

Author SHA1 Message Date
Casey Bodley
264e371101 test_sse_s3_default_multipart_upload verifies encryption header
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit febbcc12c2)
2023-04-10 22:44:40 -04:00
Mark Kogan
6342b3d258 test object PUT with chunked transfer enconding
Before the RGW fix PR was responding with 411 instead of 200

RGW fix PR: https://github.com/ceph/ceph/pull/50235

Signed-off-by: Mark Kogan <mkogan@redhat.com>
(cherry picked from commit 13a9bfc00a)
2023-04-07 09:20:15 -04:00
Casey Bodley
67e6c5aecc boto3: multi-object-delete tests use client.delete_objects()
when the tests were converted from boto2, they were rewritten as loops
over client.delete_object(). switch back to multi-delete

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 787dc6bd43)
2023-03-31 14:43:34 -04:00
Casey Bodley
a1139f5301 add test_get_object_torrent
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit bb27e04c45)
2023-03-14 13:43:08 -04:00
Soumya Koduri
d6a9168220 lifecycle: Fix test_lifecycle_expiration_header_* testcases
Few checks were incorrectly mapped when switched to 'assert'. This
commit fixes the same.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 29b0e27e49)
2023-03-08 08:58:56 -05:00
Soumya Koduri
02fdf70f0a dbstore: add back missing 'fails_on_dbstore' tag
Mark testcase "test_lifecycle_expiration_header_and_tags_head" as
fails_on_dbstore

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit d976f47d74)
2023-03-06 13:00:00 -05:00
Robin H. Johnson
a8fd8d430e test_post_object_upload_size_rgw_chunk_size_bug: new testcase
`ERR_TOO_SMALL` is wrongly returned if all of the following are true,
- the get_data returns multiple items (chunks)
- the length of the last item is smaller than the POST Policy's min
  value for content-length-range.

The check should be `(ofs < min_len)` instead of `(len < min_len)`

This is further confirmed by the next line of `s->obj_size = ofs`

Move the `int len` scope inside loop to try and prevent the bug in
future.

The bug was refactored in 2016, but was introduced in Oct 2012, when
this functionality was first added to RGW in commit 7bb3504d3f0974e9863f536e9af0ce8889d6888f.

Reference: 933a42f9af/src/rgw/rgw_op.cc (L4474-L4513)
Reference: 7bb3504d3f
Signed-off-by: Robin H. Johnson <rjohnson@digitalocean.com>
(cherry picked from commit 5914eb2005)
2023-02-20 09:30:30 -05:00
Casey Bodley
968d0841da test_s3: remove test_user_policy()
this has been failing consistently in local testing. test_sts.py has
lots of user policy test coverage, so this test case in test_s3.py is
superfluous

Fixes: https://tracker.ceph.com/issues/58365

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 18a41ab63f)
2023-02-09 10:34:45 -05:00
Casey Bodley
bb4158dec6 cleanup duplicate lines
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit d13ed28a5c)
2023-01-25 18:04:15 -05:00
Casey Bodley
b4bb9afb52 nose: remove nose attrs and imports
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4c75fba0de)
2023-01-25 18:03:58 -05:00
Casey Bodley
8186dd7561 pytest: replace nose eq() with assert ==
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f5d0bc9be3)
2023-01-25 18:03:50 -05:00
Casey Bodley
5396b04f1b pytest: replace nose SkipTest with pytest.skip()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 7e7e8d5a42)
2023-01-25 18:03:42 -05:00
Casey Bodley
e8d56603a3 pytest: replace @nose.with_setup with fixtures
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit c80e9d2118)
2023-01-25 18:01:16 -05:00
Casey Bodley
b6dcf111fa pytest: add custom marks for each nose @attr
and register them in pytest.ini

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4864dbc340)
2023-01-25 18:00:37 -05:00
Casey Bodley
60cc478d70 remove tests tagged fails_strict_rfc2616
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 3652cfe2ec)
2023-01-25 18:00:28 -05:00
Casey Bodley
c749452fd0 pytest: add global configfile and autouse teardown fixtures
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 672a123348)
2023-01-25 18:00:17 -05:00
Casey Bodley
02ee15041a add test_object_copy_16m to test refcounting of tail objects
i don't think any of our CopyObj test cases were large enough to have
tail objects, so weren't exercising our tail object ref counting
strategy

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit defb8eb977)
2022-10-06 16:59:30 -04:00
Soumya Koduri
698f19d621 cloud-transition: Adjust lc wait time
Fixes: https://tracker.ceph.com/issues/57401

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 774172ad43)
2022-09-06 22:40:34 +05:30
Soumya Koduri
2227b61eee dbstore: Update the tests marked 'fails_on_dbstore'
Tag/Untag testcases failing on dbstore as per latest run against main

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit a8ee732732)
2022-08-19 10:30:49 +05:30
Igor Fedotov
c70849d527 Test case for locked and "marked-deleted" object version removal.
Reproduces: https://tracker.ceph.com/issues/55766
Signed-off by: Igor Fedotov <igor.fedotov@croit.io>
2022-08-03 11:54:06 -04:00
Shriya Deshmukh
d3e1ccda9c rgw/s3_boto3: Added http status code verification for DeleteBucketTagging
case.

Updated test_set_bucket_tagging test for verifying the http status code
for DeleteBucketTagging case.

Related CEPH PR: https://github.com/ceph/ceph/pull/47262

Signed-off-by: Shriya Deshmukh <shriya.deshmukh@seagate.com>
(cherry picked from commit c8fc8cd7c8)
2022-08-01 14:08:34 -04:00
Casey Bodley
001b7f3f55 remove tests that fail on boto3's parameter validation
Fixes: https://tracker.ceph.com/issues/55193

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-06-06 14:28:00 -04:00
Casey Bodley
9f1b9d02a5 fix GetBucketTagging error code
related to https://tracker.ceph.com/issues/55460

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-19 12:14:13 -04:00
Soumya Koduri
54699553d5 Tag sse* tests with 'fails_on_dbstore' attr
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 33afb4eb88)
2022-05-11 23:37:37 +05:30
Soumya Koduri
358547142a Adjust wait time for cloud-transition test failures
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 25d05a194b)
2022-05-11 14:20:14 +05:30
Priya Sehgal
78ce27ae27 sse s3 jumbo patch - all in one fixes for sse-s3 logic
original tests by Priya Sehgal <priya.sehgal@flipkart.com>:
rgw/s3_boto3: Tests added for SSE-S3 (GET, PUT, HEAD, MPU).

Additions by Casey Bodley <cbodley@redhat.com>:
add 'sse-s3' tag to test cases
sse: add _put_bucket_encryption() helper function
sse: document test cases with default bucket encryption
sse: expects encryption response header on put/get
sse: add 8MB default-encrypted upload
sse: test uploads that request x-amz-server-side-encryption=AES256

Lastly all my changes (Marcus Watts <mwatts@redhat.com>):

remove obsolete test - do it only in boto3 now.

Combine or rename duplicated function names.
Giving more than one test the same name is a Bad Thing(tm).

sse: expand test_bucket_policy_put_obj_enc, and _put_bucket_encryption

test_bucket_policy_put_obj_enc was testing too many things at once.
new tests:

* customer encryption and sse-s3: should fail
* customer encryption and sse-kms: should fail
* deny if not sse-s3: no-enc fails, sse-s3 succeeds.
* deny if not sse-s3: kms fails
  deny if not sse-ksm: no-enc fails, sse-kms succeeds.
  deny if not sse-ksm: s3 fails

_put_bucket_encryption was only testing sse-s3.
* test both these variations: sse-s3 and sse-kms

Note:
* these tests will fail on pre-sse-s3 ceph.

python3: comment out all boto3.set_stream_logger() calls
They made too much output.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit dd7cac25f5)
2022-05-02 17:02:11 -04:00
Soumya Koduri
73f458da05 Tag testcases failing on dbstore with 'fails_on_dbstore' attr
To be able to successfully run s3tests on dbstore backend in teuthology,
mark all the s3-tests currently failing on it with 'fails_on_dbstore' attr

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-04-28 23:32:22 +05:30
Soumya Koduri
2aa4a22c5a lifecycle: Adjust lc wait time
Adjust wait time to fix spurious failures reported -
https://github.com/ceph/ceph/pull/40703#issuecomment-1060811704
http://qa-proxy.ceph.com/teuthology/mbenjamin-2022-02-23_15:42:19-rgw-wip-rgwlc-noreset-distro-basic-smithi/6703349/teuthology.log

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-04-06 01:07:06 +05:30
Pragadeeswaran Sathyanarayanan
e60dda8e1d Improving check_grants reliability
Signed-off-by: Pragadeeswaran Sathyanarayanan <psathyan@redhat.com>
(cherry picked from commit 5f96a32045)
2022-03-03 12:53:42 -05:00
Soumya Koduri
3c52fa8bb9 lifecycle/deletemarker_expiration: Increase timer window
Increase wait time in test_lifecycle_deletemarker_expiration(..)
to avoid any spurious failure.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-08 12:29:33 -05:00
Soumya Koduri
23cd521b5f Enable lifecycle tests
Add an option to configure lc debug interval and adjust lifecycle
tests sleep as per the value set.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-04 11:02:28 -05:00
Soumya Koduri
e5617b783f Add testcases for rgw cloudtransition feature
Feature PR: https://github.com/ceph/ceph/pull/35100

Also ported lc testcases from boto2 to boto3

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-04 11:02:28 -05:00
Or Friedmann
41006d68c2 Test expiration header for lc rules with tags
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
2022-01-17 12:12:18 -05:00
胡玮文
ae981dd3a8 Test list_objects_v2 KeyCount with Delimiter
Test for: https://github.com/ceph/ceph/pull/37396

Signed-off-by: 胡玮文 <huww98@outlook.com>
2022-01-17 12:12:18 -05:00
Casey Bodley
d1a38df407 check Owner/Initiator fields of ListMultipartUploads response
new test case test_list_multipart_upload_owner() uses two different
users to initiate multipart uploads, then tests that
list_multipart_uploads() shows the correct user ids and display names
for each upload

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 490d0a4c4f)
2021-11-18 14:32:20 -05:00
iraj465
e95809024c chore:Removes unused scaffolds
(cherry picked from commit d2a7ed88f1)
2021-09-15 17:00:26 -04:00
iraj465
93a6d89681 rgw/boto3_s3:Adds delete_objects key limit for list-objects-v2
(cherry picked from commit 459e3c870a)
2021-09-15 16:59:52 -04:00
iraj465
907331d90c chore:Bump the list-objects to paginator
(cherry picked from commit 20aa9aa071)
2021-09-15 16:59:42 -04:00
iraj465
6e426d3291 rgw/s3_boto3:Adds new delete_objects tests for checking key delete limit
(cherry picked from commit 64bdc3beec)
2021-09-08 16:00:35 -04:00
Mark Kogan
ec530d049c
Merge pull request #390 from mkogan1/multipart-extra-complete
test extra complete_multipart_upload()
2021-08-23 18:06:58 +03:00
Casey Bodley
f004493dcc object-lock: test changes between retention modes
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8662815ebe)
2021-08-12 13:46:22 -04:00
Rahul Dev Parashar
df426ea041 rgw/s3_boto3: Add tests for bucket encryption APIs
Tests are added for GetBucketEncryption, PutBucketEncryption,
and DeleteBucketEncryption APIs.

Related PR: https://github.com/ceph/ceph/pull/42222

Signed-off-by: Rahul Dev Parashar <rahul.dev@flipkart.com>
(cherry picked from commit 44643af0b0)
2021-08-12 13:45:16 -04:00
iraj465
e22a689a44 rgw/s3_boto3:Adds lifecycle transition test for invalid iso8601 date
(cherry picked from commit ba9525f425)
2021-08-12 13:44:56 -04:00
Pragadeeswaran Sathyanarayanan
0fef1637ae Add support for disabling SSL certificate verification
Signed-off-by: Pragadeeswaran Sathyanarayanan <psathyan@redhat.com>
(cherry picked from commit ea3caaa76b)
2021-08-09 14:04:47 -04:00
Mark Houghton
a81ad3515e Add tests for issue 47586.
(cherry picked from commit 7fe0304e9c)
2021-08-05 11:29:25 -04:00
Danny Abukalam
0d7111ffc2 Add test to check retain date is in iso 8601 format
Signed-off-by: Danny Abukalam <danny@softiron.com>
(cherry picked from commit e229d1aaf6)
2021-07-28 09:38:30 -04:00
Casey Bodley
72957ece35 test that listed buckets have creation time
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 66ced9af1d)
2021-06-28 11:09:59 -04:00
Or Friedmann
8423389033 test multipart upload with bucket policy
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit cfdf914c4b)
2021-06-07 14:05:58 -04:00
Mark Kogan
c4635f9463 test extra complete_multipart_upload()
after the 1st successfull one should also return 200 OK

related tracker issue: https://tracker.ceph.com/issues/50141
related pr: https://github.com/ceph/ceph/pull/40594

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2021-04-25 15:30:03 +03:00
Or Friedmann
e3d31ef6eb Add test for head bucket usage headers
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit ef8f65d917)
2020-12-03 16:04:29 -05:00