frostfs-s3-gw/CHANGELOG.md

Changelog

This document outlines major changes between releases.

Unreleased

0.31.1 - 2024-11-28

Fixed

• Ignore precondition headers with invalid date format (#563)
• MD5 calculation of object-part with SSE-C (#543)

0.31.0 - Rongbuk - 2024-11-20

Fixed

• Docker warnings during image build (#421)
• PartNumberMarker in ListMultipart response (#451)
• PostObject handling (#456)
• Tag logging errors (#452)
• Removing of duplicated parts in tree service during split brain (#448)
• Container resolving (#482)
• FrostFS to S3 error transformation (#488)
• Default bucket routing (#507)
• encoding-type in ListBucketObjectVersions (#404)
• SIGHUP support for tracing.enabled config parameter (#520)
• trace_id parameter in logs (#501)
• Listing marker processing (#539)
• Content-MD5 header check (#540)
• Precondition check (#538)
• Bucket name check during all S3 operations (#556)

Added

• Support for separate container for all CORS settings (#422)
• X-Amz-Force-Delete-Bucket header for forced bucket removal (#31)
• Location support in CompleteMultipart response (#451)
• Tree pool request duration metric (#447)
• Expiration lifecycle configuration support (#42, #412, #459, #460, #516, #536)
• Add support for virtual hosted style addressing (#446, #449, #493)
• Support frostfs.graceful_close_on_switch_timeout (#475)
• Vulnerability report document (#413)
• Support patch object method (#462, #473, #466, #479)
• Enhanced logging and request reproducer (#369)
• Root CA configuration for tracing (#484)
• Log sampling policy configuration (#461)
• sign command to frostfs-s3-authmate (#467)
• Support custom aws credentials (#509)
• Source IP binding configuration for FrostFS requests (#521)
• Tracing attributes (#549)

Changed

• Split FrostFS interface into separate read methods (#427)
• golangci-lint v1.60 support (#474)
• Updated Go version to 1.22 (#470)
• Container removal after failed bucket creation (#434)
• Explicit check for . symbol in bucket name (#506)
• Transaction waiter in contract clients (#522)
• Avoid maintenance mode storage node during object operations (#524)
• Content-Type does not include in Presigned URL of s3-authmate (#505)
• Check owner ID before deleting bucket (#528)
• S3-Authmate now uses APE instead basic-ACL (#553)

Removed

• Reduce using mutex when update app settings (#329)

0.30.8 - 2024-10-18

Fixed

• Error handling for correct connection switch in SDK Pool (#517)

0.30.7 - 2024-10-03

Fixed

• Correct aws-chunk encoding size handling (#511)

0.30.6 - 2024-09-17

Fixed

• Object size of objects upload with aws-chunked encoding (#450)
• Object size of objects upload with negative Content-Length (#486)

0.30.5 - 2024-09-16

Fixed

• Panic catchers for fuzzing tests (#492)

0.30.4 - 2024-09-03

Added

• Fuzzing tests (#480)

0.30.3 - 2024-08-27

Fixed

• Empty listing when multipart upload contains more than 1000 parts (#471)

0.30.2 - 2024-08-20

Fixed

• Error counting in pool component before connection switch (#468)

Added

• Log of endpoint address during tree pool errors (#468)

0.30.1 - 2024-07-25

Fixed

• Redundant system node removal in tree service (#437)

Added

• Log details on SDK Pool health status change (#439)

0.30.0 - Kangshung -2024-07-19

Fixed

• Fix HTTP/2 requests (#341)
• Fix Decoder.CharsetReader is nil (#379)
• Fix flaky ACL encode test (#340)
• Docs grammar (#432)

Added

• Add new reconnect_interval config param for server rebinding (#291)
• Support GetBucketPolicyStatus (#301)
• Support request IP filter with policy (#371, #377)
• Support tag checks in policies (#357, #365, #392, #403, #411)
• Support IAM-MFA checks (#367)
• More docs (#334, #353)
• Add register-user command to authmate (#414)
• User field in request log (#396)
• Erasure coding support in placement policy (#400)
• Improved test coverage (#402)

Changed

• Update dependencies noted by govulncheck (#368)
• Improve test coverage (#380, #387)
• Support updated naming in native policy JSON (#385)
• Improve determining AccessBox latest version (#335)
• Don't set full_control policy for bucket owner (#407)

Removed

• Remove control api (#406)
• Remove notifications (#401)
• Remove layer.Client interface (#410)
• Remove extended ACL related code (#372)

0.29.3 - 2024-07-19

Fixed

• Support tree split environment when multiple nodes may be part of the same sub path (#430)
• Collision of multipart name and system data in the tree (#430)
• Workaround for removal of multiple null versions in unversioned bucket (#430)

0.29.2 - 2024-07-03

Fixed

• Parsing of put-bucket-setting retry configuration (#398)

0.29.1 - 2024-06-20

Fixed

• OPTIONS request processing for object operations (#399)

Added

• Retries of put-bucket-setting operation during container creation (#398)

0.29.0 - Zemu - 2024-05-27

Fixed

• Fix marshaling errors in DeleteObjects method (#222)
• Fix status code in GET/HEAD delete marker (#226)
• Fix NextVersionIDMarker in list-object-versions (#248)
• Fix possibility of panic during SIGHUP (#288)
• Fix flaky TestErrorTimeoutChecking (make test sometimes failed) (#290)
• Fix log-level change on SIGHUP (#313)
• Fix anonymous put request (#311)
• Fix routine leak from nns resolver (#324)
• Fix svace errors (#325, #328)

Added

• Add new frostfs.buffer_max_size_for_put config param and sync TZ hash for PUT operations (#197)
• Add X-Amz-Version-Id header after complete multipart upload (#227)
• Add handling of X-Amz-Copy-Source-Server-Side-Encryption-Customer-* headers during copy (#217)
• Add new logger.destination config param (#236)
• Add X-Amz-Content-Sha256 header validation (#218)
• Support frostfsid contract. See frostfsid config section (#260)
• Support per namespace placement policies configuration (see namespaces.config config param) (#266)
• Support control api to manage policies. See control config section (#258)
• Add namespace label to billing metrics (#271)
• Support policy-engine (#257, #259, #282, #283, #302, #307, #345, #351, #358, #360, #362, #383, #354)
• Support proxy contract (#287)
• Authmate: support custom attributes (#292)
• Add FrostfsID cache (#269)

Changed

• Generalise config param use_default_xmlns_for_complete_multipart to use_default_xmlns so that use default xmlns for all requests (#221)
• Set server IdleTimeout and ReadHeaderTimeout to 30s and allow to configure them (#220)
• Return ETag value in quotes (#219)
• Use tombstone when delete multipart upload (#275)
• Support new parameter cache.accessbox.removing_check_interval (#305)
• Use APE rules instead of eACL in container creation (#306)
• Rework bucket policy with policy-engine (#261)
• Improved object listing speed (#165, #347)
• Logging improvement (#300, #318)

Removed

• Drop sending whitespace characters during complete multipart upload and related config param kludge.complete_multipart_keepalive (#227)
• Unused legacy minio related code (#299)
• Redundant output with journald logging (#298)

0.28.2 - 2024-05-27

Fixed

• anon user in billing metrics (#321)
• Parts are not removed when multipart object removed (#370)

Added

• Put request in duration metrics (#280)

0.28.1 - 2024-01-24

Added

• MD5 hash as ETag and response header (#205)
• Tree pool traversal limit (#262)

Updating from 0.28.0

See new features.md5.enabled and frostfs.tree_pool_max_attempts config parameters.

0.28.0 - Academy of Sciences - 2023-12-07

Fixed

• Handle negative Content-Length on put (#125)
• Use DisableURIPathEscaping to presign urls (#125)
• Use specific s3 errors instead of InternalError where possible (#143)
• grpc schemas in tree configuration (#166)
• Return appropriate 404 code when object missed in storage but there is in gate cache (#158)
• Replace part on re-upload when use multipart upload (#176)
• Fix goroutine leak on put object error (#178)
• Fix parsing signed headers in presigned urls (#182)
• Fix url escaping (#188)
• Use correct keys in list-multipart-uploads response (#185)
• Fix parsing key-marker for object list versions (#237)
• GetSubTree failures (#179)
• Unexpected EOF during multipart download (#210)
• Produce clean version in debian build (#245)

Added

• Add trace_id value into log record when tracing is enabled (#142)
• Add basic error types and exit codes to frostfs-s3-authmate (#152)
• Add a metric with addresses of nodes of the same and highest priority that are currently healthy (#186)
• Support dump metrics descriptions (#80)
• Add copies_numbers section to placement_policy in config file and support vectors of copies numbers (#70, #101)
• Support impersonate bearer token (#81, #105)
• Reload default and custom copies numbers on SIGHUP (#104)
• Tracing support (#84, #140)
• Return bearer token in s3-authmate obtain-secret result (#132)
• Support multiple version credentials using GSet (#135)
• Implement chunk uploading (#106)
• Add new kludge.bypass_content_encoding_check_in_chunks config param (#146)
• Add new frostfs.client_cut config param (#192)
• Add selection of the node of the latest version of the object (#231)
• Soft memory limit with runtime.soft_memory_limit (#196)
• server_health metric for every S3 endpoint status (#199)

Changed

• Update prometheus to v1.15.0 (#94)
• Update go version to go1.19 (#118)
• Remove object from tree and reset its cache on object deletion when it is already removed from storage (#78)
• Finish rebranding (#2)
• Timeout errors has code 504 now (#103)
• Use request scope logger (#111)
• Add s3-authmate update-secret command (#131)
• Use default registerer for app metrics (#155)
• Use chi router instead of archived gorlilla/mux (#149, #174, #188)
• Complete multipart upload doesn't unnecessary copy now. Thus, the total time of multipart upload was reduced by 2 times (#63)
• Use gate key to form object owner (#175)
• Apply placement policies and copies if there is at least one valid value (#168)
• statistic_tx_bytes_total and statistic_rx_bytes_total metric to statistic_bytes_total metric with direction label (#153)
• Refactor of context-stored data receivers (#137)
• Refactor fetch/parse config parameters functions (#117)
• Move all log messages to constants (#96)
• Allow zero value of part-number-marker (#207)
• Clean tag node in the tree service instead of removal (#233)

Removed

• Drop tree.service param (now endpoints from peers section are used) (#133)

0.27.0 - Karpinsky - 2023-07-12

This is a first FrostFS S3 Gateway release named after Karpinsky glacier.

Fixed

• Using multiple servers require only one healthy (#12)
• Renew token before it expires (#20)
• Add generated deb builder files to .gitignore, and fix typo (#28)
• Get empty bucket CORS from frostfs (#36)
• Don't count pool error on client abort (#35)
• Handle request cancelling (#69)
• Clean up List and Name caches when object is missing in Tree service (#57)
• Don't create unnecessary delete-markers (#83)
• Too many pings error (#145)

Added

• Billing metrics (#5, #26, #29)
• Return container name in head-bucket response (#18)
• Multiple configs support (#21)
• Bucket name resolving policy (#25)
• Support string Action and Resource fields in bucketPolicy.Statement (#32)
• Add new kludge.use_default_xmlns_for_complete_multipart config param (#40)
• Return X-Owner-Id in head-bucket response (#79)
• Support multiple tree service endpoints (#74, #110, #114)

Changed

• Repository rebranding (#1)
• Update neo-go to v0.101.0 (#14)
• Update viper to v1.15.0 (#14)
• Update go version to go1.18 (#16)
• Return error on invalid LocationConstraint (#23)
• Limit number of objects to delete at one time (#37)
• CompleteMultipartUpload handler now sends whitespace characters to keep alive client's connection (#60)
• Support new system attributes (#64)
• Abstract network communication in TreeClient (#59, #75)
• Changed values for frostfs_s3_gw_state_health metric (#91)

Older versions

This project is a fork of NeoFS S3 Gateway from version v0.26.0. To see CHANGELOG for older versions, refer to https://github.com/nspcc-dev/neofs-s3-gw/blob/master/CHANGELOG.md.