Commit graph

55 commits

Author SHA1 Message Date
galsalomon66
a99d90faa3 Revert "Revert "Using get bucket name""
This reverts commit 9aaca64335.

revert to get_bucket_name
2023-06-22 19:40:25 +03:00
Ali Maredia
9aaca64335
Revert "Using get bucket name" 2023-06-21 16:47:56 -04:00
Gal Salomon
fa0ea9afe0
Merge branch 'master' into using_get_bucket_name 2023-06-08 10:19:37 +03:00
galsalomon66
2998ea91eb instead of hard-coded bucket-name using get_bucket_name.merging PR #488 alphaprinz
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2023-06-08 10:07:54 +03:00
galsalomon66
008f5025f7 progress nessage is sent back upon processing the object, the change make sure it stay with the max result
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2023-04-10 12:26:59 +03:00
galsalomon66
bc2a3b0b70 modifying of the run_s3select routine; to handle the different statuses (progress,stats,end)
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2023-04-09 15:02:24 +03:00
galsalomon66
89bbe654ca upon removing the payload tag, the response index should be changed also
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2023-04-05 01:30:00 +03:00
Albin Antony
2c710811fa s3select: json output serialization
Signed-off-by: Albin Antony <aantony@redhat.com>
2023-02-13 14:45:36 +05:30
Amit Prinz Setter
819dd5aa32 Use get_new_bucket_name() in select tests.
Signed-off-by: Amit Prinz Setter <aprinzse@redhat.com>
2023-02-05 17:54:46 +02:00
Casey Bodley
4c75fba0de nose: remove nose attrs and imports
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-01-21 22:40:33 -05:00
Casey Bodley
f5d0bc9be3 pytest: replace nose eq() with assert ==
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-01-21 22:40:33 -05:00
Casey Bodley
4864dbc340 pytest: add custom marks for each nose @attr
and register them in pytest.ini

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-01-21 22:40:33 -05:00
Casey Bodley
672a123348 pytest: add global configfile and autouse teardown fixtures
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2023-01-21 22:40:33 -05:00
galsalomon66
60b26f210e remove comment
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2022-10-18 00:37:29 +03:00
galsalomon66
b200013565 the use of std::to_string change the double presentation(not accuracy)
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2022-09-18 10:46:04 +03:00
galsalomon66
89f97ed35c the upgraded CSV parser has some impact on engine result. (redundant comma, quote behavior)
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2022-09-12 15:33:35 +03:00
gal salomon
a4d282c1db revert the arrow installation(causing failure on some distro"s, such as fedora)
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2022-01-12 19:01:06 +02:00
gal salomon
60593c99dd fix output-serialization tests(upon comparing query results need to remove redundant columns)
skip output-serial test. the results from both queries are not equal, thus it raise an assert. the problem seems to be the formatting before the comparision

remove test_output_serial_expressions until fixing the test

experiment pyarrow for parquet testing, adding arrow/parquet to bootstrap, installing pyarrow,pandas for reading/writing parquet

Signed-off-by: gal salomon <gal.salomon@gmail.com>
2022-01-11 20:42:38 +02:00
gal salomon
6019ec1ef3 merging master tests into parquet branch
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2021-12-28 03:03:54 +02:00
gal salomon
a3b849e4db fix for assert of error messages
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2021-12-28 02:58:58 +02:00
gal salomon
93099c1fb0 remove redundant comma. s3select-engine produced redundant result column before its last fix, s3tests should align with that
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2021-12-28 02:58:58 +02:00
galsalomon66
23be1160f5 adding datatime queries from #395
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-11-25 11:57:52 +02:00
galsalomon66
eef8d0fa67 rollback to use like without escape syntax (explore valgrind issue)
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-11-12 11:39:45 +02:00
galsalomon66
f51101d752 modify the queries syntax to like-escape; the queries semantics is the same; results are the same; it is a part of exploring valgrind issue around like-operator
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-11-11 16:15:49 +02:00
galsalomon66
749e29185b add output-serialization tests; add syntax-error tests; run_s3select_output should merge into run_s3select
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-10-21 03:32:57 +03:00
gal salomon
7c07bad930 remove comments from the like-expressions
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2021-10-17 14:03:39 +03:00
galsalomon66
723853fd18 search for the cause of the valgrind issue; remove the like expression that may cause it
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-07-27 08:26:28 +03:00
Albin Antony
a3447c50df s3select: test progress stats
Signed-off-by: Albin Antony <aantony@redhat.com>
2021-07-07 15:18:29 +05:30
Albin Antony
aaa355f20b s3select: align s3select tests with ceph
Update s3-tests to handle the error-response (return 400, and error-description)

Signed-off-by: Albin Antony <aantony@redhat.com>
2021-06-30 15:13:45 +05:30
gal salomon
1572fbc87b modifying tests to be align with change of s3select compare sign( == -> = )
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2021-05-05 11:35:32 +03:00
Albin Antony
c6a4ab9d12 tests
Signed-off-by: Albin Antony <aantony@redhat.com>
2021-04-08 16:21:47 +05:30
Albin Antony
e7102e8cb0 tests
Signed-off-by: Albin Antony <aantony@redhat.com>
2021-03-30 20:59:31 +05:30
Albin Antony
60dd3444b3 tests
Signed-off-by: Albin Antony <aantony@redhat.com>
2021-03-22 16:24:26 +05:30
Albin Antony
4a86ebbe8b s3select: align s3-tests with new changes in s3select
Fix when then, date functions and NULL, add escape, trim tests

Signed-off-by: Albin Antony <aantony@redhat.com>
2021-03-19 13:10:43 +05:30
galsalomon66
ea7d5fb563 change test_s3select.py permission; add s3select attribute
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2021-01-13 09:24:31 +02:00
galsalomon66
6a63d0cf91 reduce object size for test_like_expression, it may cause timeouts in teuthology runs
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2020-12-15 11:51:19 +02:00
galsalomon66
5d6166bf53 rename charlength and character_length function names
Signed-off-by: galsalomon66 <gal.salomon@gmail.com>
2020-12-05 18:58:45 +02:00
Gal Salomon
26f06011ee
Merge pull request #364 from albin-antony/when-than
s3select tests for coalesce and case
2020-11-12 11:53:12 +02:00
Albin Antony
d7c243ba83 s3select tests for coalesce and case
Signed-off-by: Albin Antony <aantony@redhat.com>
2020-11-12 00:13:03 +05:30
Albin Antony
ae3052fa8a s3select predicate tests
Signed-off-by: Albin Antony <aantony@redhat.com>
2020-10-09 13:15:43 +05:30
gal salomon
350fcbb4ec per each new uploaded file(for test), it got unique name(random), and uploaded file is verified for its content
Signed-off-by: gal salomon <gal.salomon@gmail.com>

Signed-off-by: gal salomon <gal.salomon@gmail.com>
2020-08-12 13:29:30 +03:00
gal salomon
fce9a52ef4 add filter for s3select tests
Signed-off-by: gal salomon <gal.salomon@gmail.com>
2020-07-07 00:10:50 +03:00
gal salomon
72e251ed69 fix comments;remove non-used imports;enable test for projection;using get_client() 2020-05-31 16:09:03 +03:00
gal salomon
fb39ac4829 adding radom-generated tests, one for where clause , the second for prjection. random arithmetical expression is generated and used for building s3-select query; result is compared to python-engine 2020-05-27 01:20:31 +03:00
gal salomon
e006dd4753 python linter; replace assert with assert_equal; add complex query test(sum,count,where); add test-schema ; 2020-05-26 11:36:38 +03:00
gal salomon
1a9d3677f7 using config parameters 2020-05-23 15:10:14 +03:00
gal salomon
5dc8bc75ab add tests to validate csv-header-info functionalities is correct 2020-05-08 16:51:24 +03:00
gal salomon
94b1986228 adding test cases for processing CSV objects with different CSV definitions; validate null,escape-rules and quotes are processed correctly 2020-04-28 19:10:24 +03:00
gal salomon
4c7c279f70 adding utcnow test 2020-04-15 17:20:52 +03:00
gal salomon
5925f0fb3f adding tests for date-time functionalities 2020-04-09 16:43:11 +03:00