Commit graph

100 commits

Author SHA1 Message Date
Casey Bodley
001b7f3f55 remove tests that fail on boto3's parameter validation
Fixes: https://tracker.ceph.com/issues/55193

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-06-06 14:28:00 -04:00
Casey Bodley
9f1b9d02a5 fix GetBucketTagging error code
related to https://tracker.ceph.com/issues/55460

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2022-05-19 12:14:13 -04:00
Soumya Koduri
54699553d5 Tag sse* tests with 'fails_on_dbstore' attr
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 33afb4eb88)
2022-05-11 23:37:37 +05:30
Soumya Koduri
358547142a Adjust wait time for cloud-transition test failures
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 25d05a194b)
2022-05-11 14:20:14 +05:30
Priya Sehgal
78ce27ae27 sse s3 jumbo patch - all in one fixes for sse-s3 logic
original tests by Priya Sehgal <priya.sehgal@flipkart.com>:
rgw/s3_boto3: Tests added for SSE-S3 (GET, PUT, HEAD, MPU).

Additions by Casey Bodley <cbodley@redhat.com>:
add 'sse-s3' tag to test cases
sse: add _put_bucket_encryption() helper function
sse: document test cases with default bucket encryption
sse: expects encryption response header on put/get
sse: add 8MB default-encrypted upload
sse: test uploads that request x-amz-server-side-encryption=AES256

Lastly all my changes (Marcus Watts <mwatts@redhat.com>):

remove obsolete test - do it only in boto3 now.

Combine or rename duplicated function names.
Giving more than one test the same name is a Bad Thing(tm).

sse: expand test_bucket_policy_put_obj_enc, and _put_bucket_encryption

test_bucket_policy_put_obj_enc was testing too many things at once.
new tests:

* customer encryption and sse-s3: should fail
* customer encryption and sse-kms: should fail
* deny if not sse-s3: no-enc fails, sse-s3 succeeds.
* deny if not sse-s3: kms fails
  deny if not sse-ksm: no-enc fails, sse-kms succeeds.
  deny if not sse-ksm: s3 fails

_put_bucket_encryption was only testing sse-s3.
* test both these variations: sse-s3 and sse-kms

Note:
* these tests will fail on pre-sse-s3 ceph.

python3: comment out all boto3.set_stream_logger() calls
They made too much output.

Signed-off-by: Marcus Watts <mwatts@redhat.com>
(cherry picked from commit dd7cac25f5)
2022-05-02 17:02:11 -04:00
Soumya Koduri
73f458da05 Tag testcases failing on dbstore with 'fails_on_dbstore' attr
To be able to successfully run s3tests on dbstore backend in teuthology,
mark all the s3-tests currently failing on it with 'fails_on_dbstore' attr

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-04-28 23:32:22 +05:30
Soumya Koduri
2aa4a22c5a lifecycle: Adjust lc wait time
Adjust wait time to fix spurious failures reported -
https://github.com/ceph/ceph/pull/40703#issuecomment-1060811704
http://qa-proxy.ceph.com/teuthology/mbenjamin-2022-02-23_15:42:19-rgw-wip-rgwlc-noreset-distro-basic-smithi/6703349/teuthology.log

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-04-06 01:07:06 +05:30
Pragadeeswaran Sathyanarayanan
e60dda8e1d Improving check_grants reliability
Signed-off-by: Pragadeeswaran Sathyanarayanan <psathyan@redhat.com>
(cherry picked from commit 5f96a32045)
2022-03-03 12:53:42 -05:00
Soumya Koduri
3c52fa8bb9 lifecycle/deletemarker_expiration: Increase timer window
Increase wait time in test_lifecycle_deletemarker_expiration(..)
to avoid any spurious failure.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-08 12:29:33 -05:00
Soumya Koduri
23cd521b5f Enable lifecycle tests
Add an option to configure lc debug interval and adjust lifecycle
tests sleep as per the value set.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-04 11:02:28 -05:00
Soumya Koduri
e5617b783f Add testcases for rgw cloudtransition feature
Feature PR: https://github.com/ceph/ceph/pull/35100

Also ported lc testcases from boto2 to boto3

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2022-02-04 11:02:28 -05:00
Or Friedmann
41006d68c2 Test expiration header for lc rules with tags
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
2022-01-17 12:12:18 -05:00
胡玮文
ae981dd3a8 Test list_objects_v2 KeyCount with Delimiter
Test for: https://github.com/ceph/ceph/pull/37396

Signed-off-by: 胡玮文 <huww98@outlook.com>
2022-01-17 12:12:18 -05:00
Casey Bodley
d1a38df407 check Owner/Initiator fields of ListMultipartUploads response
new test case test_list_multipart_upload_owner() uses two different
users to initiate multipart uploads, then tests that
list_multipart_uploads() shows the correct user ids and display names
for each upload

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 490d0a4c4f)
2021-11-18 14:32:20 -05:00
iraj465
e95809024c chore:Removes unused scaffolds
(cherry picked from commit d2a7ed88f1)
2021-09-15 17:00:26 -04:00
iraj465
93a6d89681 rgw/boto3_s3:Adds delete_objects key limit for list-objects-v2
(cherry picked from commit 459e3c870a)
2021-09-15 16:59:52 -04:00
iraj465
907331d90c chore:Bump the list-objects to paginator
(cherry picked from commit 20aa9aa071)
2021-09-15 16:59:42 -04:00
iraj465
6e426d3291 rgw/s3_boto3:Adds new delete_objects tests for checking key delete limit
(cherry picked from commit 64bdc3beec)
2021-09-08 16:00:35 -04:00
Mark Kogan
ec530d049c
Merge pull request #390 from mkogan1/multipart-extra-complete
test extra complete_multipart_upload()
2021-08-23 18:06:58 +03:00
Casey Bodley
f004493dcc object-lock: test changes between retention modes
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 8662815ebe)
2021-08-12 13:46:22 -04:00
Rahul Dev Parashar
df426ea041 rgw/s3_boto3: Add tests for bucket encryption APIs
Tests are added for GetBucketEncryption, PutBucketEncryption,
and DeleteBucketEncryption APIs.

Related PR: https://github.com/ceph/ceph/pull/42222

Signed-off-by: Rahul Dev Parashar <rahul.dev@flipkart.com>
(cherry picked from commit 44643af0b0)
2021-08-12 13:45:16 -04:00
iraj465
e22a689a44 rgw/s3_boto3:Adds lifecycle transition test for invalid iso8601 date
(cherry picked from commit ba9525f425)
2021-08-12 13:44:56 -04:00
Pragadeeswaran Sathyanarayanan
0fef1637ae Add support for disabling SSL certificate verification
Signed-off-by: Pragadeeswaran Sathyanarayanan <psathyan@redhat.com>
(cherry picked from commit ea3caaa76b)
2021-08-09 14:04:47 -04:00
Mark Houghton
a81ad3515e Add tests for issue 47586.
(cherry picked from commit 7fe0304e9c)
2021-08-05 11:29:25 -04:00
Danny Abukalam
0d7111ffc2 Add test to check retain date is in iso 8601 format
Signed-off-by: Danny Abukalam <danny@softiron.com>
(cherry picked from commit e229d1aaf6)
2021-07-28 09:38:30 -04:00
Casey Bodley
72957ece35 test that listed buckets have creation time
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 66ced9af1d)
2021-06-28 11:09:59 -04:00
Or Friedmann
8423389033 test multipart upload with bucket policy
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit cfdf914c4b)
2021-06-07 14:05:58 -04:00
Mark Kogan
c4635f9463 test extra complete_multipart_upload()
after the 1st successfull one should also return 200 OK

related tracker issue: https://tracker.ceph.com/issues/50141
related pr: https://github.com/ceph/ceph/pull/40594

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2021-04-25 15:30:03 +03:00
Or Friedmann
e3d31ef6eb Add test for head bucket usage headers
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit ef8f65d917)
2020-12-03 16:04:29 -05:00
Or Friedmann
3fe80dc877 Add test for GetUsage api
Signed-off-by: Or Friedmann <ofriedma@redhat.com>
(cherry picked from commit f4f7812efd)
2020-12-03 16:04:14 -05:00
Matt Benjamin
86bc2a191f Add test for HeadBucket on a non-existent bucket
n.b., RGW does not send a response document for this operation,
which seems consistent with
https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-11-25 00:06:12 -05:00
Casey Bodley
65b067486e test bucket recreation with different acls
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit f6218fa1de)
2020-10-08 13:32:53 -04:00
Matt Benjamin
3dc4ff5da8 add noncurrent version expiration rule w/tag filter
Create 10 object versions (9 noncurrent).  Install a noncurrent
version expiration at 4 days.  Verify that 10 versions exist at
T+20, and only 1 (current) at T+60.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-01 11:30:33 -04:00
Matt Benjamin
c9792cb975 add lifecycle expiration test mixing 2-tag filter w/versioning
By design this test duplicates test_lifecycle_expiration_tags2,
but enables object versioning on the bucket.

The tests install a rule which requires -2- tags to be matched,
and creates 2 objects, one matching only 1 of the required tags,
the other matching both.  Only the 2nd object should expire.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-01 11:30:24 -04:00
Matt Benjamin
b930f194e4 add tests for lifecycle expiration w/1 and 2 object tags
Note that the 1-tag case contains a filter prefix--which exposes
an apparent bug parsing Filter when it contains a Prefix element
and a single Tag element (without And).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-01 11:30:15 -04:00
Matt Benjamin
253b63aa11 fix lifecycle expiration days: 0
In fact test_lifecycle_expiration_days0 is should fail, as 0-day
expiration is permitted for transition rules but not expiration
rules.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-01 11:30:08 -04:00
Matt Benjamin
6bd75be1d6 s/test_set_tagging/test_set_bucket_tagging/;
The test exercises bucket tagging, has nothing to do with object
tagging (clarity).

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-01 11:30:01 -04:00
Matt Benjamin
61804bcf91 fix test_lifecycle_expiration_header_{put,head}
Primarily fixes the expiration header() verifier function
check_lifecycle_expiration_header, but also cleans up
prefix handling in setup_lifecycle_expiration().

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-01 11:29:52 -04:00
Matt Benjamin
ea9f07a2bf fix and remark on test_lifecycle_expiration_days0
1. fix a python3-related KeyError exception

2. note here:  AWS documentation includes examples of "Days 0"
   in use, but boto3 will not accept them--this is why the days0
   test currently sets Days 1

3. delay increased to 30s, to avoid occasional failures due to
   jitter

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-10-01 11:29:45 -04:00
Matt Benjamin
8ca96c4519 fix/restore test_lifecycle_expiration checks
Commit bf956df71e adding
listobvjectsv2 tests inadvertently changed the v1
test_lifecycle_expiration test, which it had copied to
create a v2 version.  Revert this.

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2020-05-26 11:07:53 -04:00
Abhishek Lekshmanan
7b3df700cc fix ignore public acls with py3 compatible code
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 4d675235dd)
2020-03-26 16:28:12 +01:00
Abhishek Lekshmanan
4fc133b1b5 add tests for ignore public acls
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 3b1571ace6)
2020-03-26 16:28:12 +01:00
Abhishek Lekshmanan
0a495efc8c add test for block public policy
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit b4516725f2)
2020-03-26 16:28:12 +01:00
Abhishek Lekshmanan
a48cf75391 use empty bodies for canned acl tests with BlockPublicAccess
This should be a temporary workaround until #42208 is fixed

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit d02c1819f6)
2020-03-26 16:28:12 +01:00
Abhishek Lekshmanan
a20e0d47f2 remove redundant get_client calls
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 4996430709)
2020-03-26 16:28:12 +01:00
Abhishek Lekshmanan
19947bd541 add ability to get svc client for s3config set of apis
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 6d3f574a8e)
2020-03-26 16:26:39 +01:00
Abhishek Lekshmanan
94168194fd add tests for public access configuration
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 1ad38530e0)
2020-03-26 16:26:19 +01:00
Abhishek Lekshmanan
0e3084c995 add a few test cases for public bucket policies
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 3f9d31c6c7)
2020-03-26 16:24:14 +01:00
Abhishek Lekshmanan
1d39198872 boto3: add bucket policy status checks for public ACLs
Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 02b1d50ca7)
2020-03-26 16:23:46 +01:00
Casey Bodley
13452bd25f
Merge pull request #337 from ceph/wip-python-3-port
Wip python 3 port
2020-01-15 08:48:54 -05:00