Commit graph

33 commits

Author SHA1 Message Date
Leonard Lyubich
271451dc32 [#128] downloader: Simplify detecting the Content-Type from payload
Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-04 13:43:34 +03:00
Leonard Lyubich
c482bbd25a [#126] downloader: Replace Read with Iterate on ObjectListReader
Make `Downloader.DownloadZipped` to call `Iterate` method instead of
`Read` one during processing the `ObjectListReader`.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-03 18:25:53 +03:00
Leonard Lyubich
2b7e4a36fb [#126] Upgrade NeoFS SDK Go library
Core changes:
  - `object.ID` moved to new package `oid`;
  - `object.Address` moved to new package `address`;
  - `pool.Object` interface changes.

Signed-off-by: Leonard Lyubich <leonard@nspcc.ru>
2022-03-03 18:25:53 +03:00
Denis Kirillov
39c29c6d6a [#125] Set object ids in a separate function
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-02-15 12:21:42 +03:00
Denis Kirillov
2764fabf04 [#125] Use the same HEAD, GET headers formation
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2022-02-15 12:21:42 +03:00
Angira Kekteeva
9d085740e0 [#123] Add url-encoded queries for prefix values
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-01-27 16:51:11 +03:00
Angira Kekteeva
c11b2332f9 [#123] Add url-encoded queries for attributes
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2022-01-27 16:51:11 +03:00
Angira Kekteeva
1bb6c5f384 [#111] Fix X- headers
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-12-09 16:12:13 +03:00
Angira Kekteeva
d6dd244756 [#111] Fix expiration epoch headers
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-12-09 16:12:13 +03:00
Angira Kekteeva
dbbc9e05cf [#111] Move attributes to a separate file
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-12-09 16:12:13 +03:00
Alex Vanin
79765fb7cc [#106] Update neofs-sdk-go to the latest version
Includes:
 - fix for unavailable endpoints of client pool,
 - session token cache with request retry in case
   of missing session token,
 - migrated neofs client.
Signed-off-by: Alex Vanin <alexey@nspcc.ru>
2021-11-15 14:46:50 +03:00
Denis Kirillov
6265fcb26a [#105] Add newline to errors
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-11-12 15:32:18 +03:00
Denis Kirillov
9b92c0146c [#96] Add zip compression
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-11-01 11:27:22 +03:00
Denis Kirillov
0b364504a7 [#92] Support zip download
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-10-22 18:36:09 +03:00
Denis Kirillov
03a3f986ff [#30] Fix not found responses
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-20 13:40:39 +03:00
Denis Kirillov
87918483c6 [#76] Refactor HeadObject logic
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-15 10:57:22 +03:00
Denis Kirillov
d856fdf4c4 [#76] Added HEAD methods
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-07-14 18:56:27 +03:00
Roman Khimov
dee182a0c1 downloader: fix Last-Modified header
It should never ever be sent in local time, instead of
  Last-Modified: Tue, 29 Jun 2021 12:02:57 MSK
one should receive
  Last-Modified: Tue, 29 Jun 2021 09:02:57 GMT

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-06-29 12:25:14 +03:00
Denis Kirillov
5eed0fd22d [#67] Added streaming to download
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-24 13:06:13 +03:00
Denis Kirillov
9a49e2c85e [#65] Supported object Content-Type
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-21 16:56:51 +03:00
Denis Kirillov
fffb0b2f12 [#63] Using client.Object from sdk
Signed-off-by: Denis Kirillov <denis@nspcc.ru>
2021-06-15 11:28:30 +03:00
Roman Khimov
e177e1c9ae *: use client.Object where appropriate
It's enough to do the job, we don't really need full client.Client interface
here.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-06-11 11:54:30 +03:00
Angira Kekteeva
2f8ab7cb8d [#57] downloader, uploader: Replace deprecated
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-06-04 15:55:56 +03:00
Roman Khimov
e8b94553c3 *: update to use unneofsed sdk-go
Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-31 23:07:01 +03:00
Angira Kekteeva
82b2126bfd [#46] *: Remove moved to sdk packages, refactoring
Removed connections, logger, neofs because they were moved to sdk repo.
Made changes in downloader, uploader, main.go and app.go via
refactoring of neofs.
Replaced dependencies to removed packages by sdk packages.

Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-28 14:45:46 +03:00
Angira Kekteeva
33b1a28bf1 Replace http-gate by http-gw in files
Signed-off-by: Angira Kekteeva <kira@nspcc.ru>
2021-05-18 15:14:36 +03:00
Roman Khimov
df3c87af79 *: fix all comment-related golint warnings
Some of this code is going to be moved to SDK library, so it's important.

Signed-off-by: Roman Khimov <roman@nspcc.ru>
2021-05-13 18:57:42 +03:00
Roman Khimov
293debe6db downloader: ensure downloader/uploader header symmetry
Use X-Attribute-* everywhere.
2021-04-30 14:31:14 +03:00
Roman Khimov
7a46917781 downloader: limit headers sent to the client to some valid subset
fasthttp doesn't do complete filtering/escaping for us, thus filter here.
2021-04-30 14:31:14 +03:00
Roman Khimov
91c894cb8c downloader: set object/container/owner x-attributes after other ones
Avoid name clashes with regular attributes.
2021-04-30 14:31:14 +03:00
Roman Khimov
afbb9d51f1 *: drop github.com/pkg/errors dependency
Use standard error wrapping/unwrapping instead. The conversion is mostly
straightforward, but see grpc/grpc-go#2934 for GRPC `status.FromError`, it
doesn't currently support unwrapping/errors.As(), so we're unwrapping manually
here.
2021-04-30 14:31:14 +03:00
Pavel Korotkov
fad05b76d4 Put artifacts into pool solely
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
2021-04-08 12:39:29 +03:00
Pavel Korotkov
3a5d9fe94c [#19] Extract downloading logic into a separate package
Signed-off-by: Pavel Korotkov <pavel@nspcc.ru>
2021-04-06 12:19:21 +03:00
Renamed from receive.go (Browse further)