rclone/backend/s3
Thomas Kriechbaumer 584e705c0c s3: introduce list_chunk option for bucket listing
The S3 ListObject API returns paginated bucket listings, with
"MaxKeys" items for each GET call.

The default value is 1000 entries, but for buckets with millions of
objects it might make sense to request more elements per request, if
the backend supports it. This commit adds a "list_chunk" option for
the user to specify a lower or higher value.

This commit does not add safe guards around this value - if a user
decides to request a too large list, it might result in connection
timeouts (on the server or client).

In AWS S3, there is a fixed limit of 1000, some other services might
have one too.  In Ceph, this can be configured in RadosGW.
2020-01-02 12:15:01 +00:00
..
s3.go s3: introduce list_chunk option for bucket listing 2020-01-02 12:15:01 +00:00
s3_test.go s3: fix SetModTime on GLACIER/ARCHIVE objects and implement set/get tier 2019-09-14 09:18:55 +01:00
v2sign.go s3: fix v2 signer on files with spaces - fixes #2438 2018-10-14 00:10:29 +01:00