Commit graph

78 commits

Author SHA1 Message Date
Wesley Spikes
298bc99c5d Packagify s3tests.common so we can add more magic into it later. 2011-07-15 11:44:05 -07:00
Wesley Spikes
e4effe5479 realistic.names shouldn't return a 0-length filename 2011-07-15 11:08:38 -07:00
Yehuda Sadeh
3303b0f33b fix 100-continue test to support https 2011-07-14 16:26:46 -07:00
Steven Berler
c312832c70 use time.time() instead of time.clock()
We want to measure actual time, not cpu time.
2011-07-14 11:40:26 -07:00
Kyle Marsh
3d13883f62 dhq-qa: remove query auth from generated urls
Object generation for static load testing now sets the ACL on the test
bucket and keys to 'public-read' so we don't need an expiring, temp url.
2011-07-14 11:37:54 -07:00
Steven Berler
7b51f55a1b fix zero devision error in readwrite test
Was dividing by 0 if somehow it managed to upload or download a file
in under .01 seconds.
2011-07-14 11:00:48 -07:00
Steven Berler
91c4f88860 fix random file verifier with small files
The file verifier will now correctly verify small files (smaller than
the md5 digest size), as long as the data that is there matches what
the digest would be.
2011-07-13 15:59:38 -07:00
Tommi Virtanen
91a39caf46 Fix bug where prefix was None.
Accessing a non-constant global from another module is tricky,
as the import binds to the original object. Rebinding the name
in the original module does not change it in importers. Use a
getter as a quick workaround.
2011-07-13 14:49:07 -07:00
Tommi Virtanen
02a4c82dfa Make generate_objects.py and rand_readwrite.py inside the package.
Handle dependencies properly. Now e.g. "./bootstrap &&
./virtualenv/bin/s3tests-test-readwrite" should just work.
2011-07-13 13:52:54 -07:00
Tommi Virtanen
33b25c577e Move setup/teardown into s3tests.functional __init__, to share across modules. 2011-07-11 15:24:39 -07:00
Tommi Virtanen
e9ad6f7ee6 Move test_s3 into s3tests.functional, so it can be split to multiple files nicely. 2011-07-11 15:24:39 -07:00
Stephon Striplin
e253e328a7 remove unused variable 2011-07-11 14:57:46 -07:00
Stephon Striplin
83830e2951 add xml-based acl tests 2011-07-11 14:22:31 -07:00
Stephon Striplin
cd84d65e9c add test_object_acl_* and test_bucket_acl_* tests 2011-07-11 14:22:31 -07:00
Stephon Striplin
f929069189 add several test_object_set_get_metadata_* tests 2011-07-11 14:22:31 -07:00
Stephon Striplin
bb3f214369 add test_object_set_get_metadata_empty 2011-07-11 14:20:14 -07:00
Stephon Striplin
bbbe6c45c3 add test_object_copy_not_owned_bucket 2011-07-11 14:18:46 -07:00
Stephon Striplin
1132698605 add test_bucket_delete_nonempty 2011-07-11 14:18:46 -07:00
Tommi Virtanen
c4f95efc70 Cleanup: remove unneeded parens. 2011-07-11 13:29:36 -07:00
Tommi Virtanen
9600fd10d9 Cleanup: avoid using "file" as identifier. 2011-07-11 13:29:36 -07:00
Tommi Virtanen
c07409ef6e Cleanup: no need to listify sum() input. 2011-07-11 13:29:36 -07:00
Tommi Virtanen
deb898161b Cleanup: don't look before you leap. 2011-07-11 13:29:36 -07:00
Tommi Virtanen
016febc40a Cleanup: new-style raise. 2011-07-11 13:29:36 -07:00
Tommi Virtanen
99dcf267ea Cleanup: non-Python semicolon infection. 2011-07-11 13:29:36 -07:00
Tommi Virtanen
21893e51b3 Cleanup: has_key is deprecated. 2011-07-11 13:29:36 -07:00
Tommi Virtanen
3fe595abd4 Whitespace cleanup. 2011-07-11 13:29:35 -07:00
Tommi Virtanen
e91a3aea66 Pyflakes cleanup. 2011-07-11 13:29:35 -07:00
Kyle Marsh
31d9032ea5 remove generate_objects.conf
generate_objects.conf is obsolete; its functionality is now part of
config.yml in the file_generation.groups section.
2011-07-11 13:11:05 -07:00
Tommi Virtanen
012b94f8ee Typo. 2011-07-11 11:54:04 -07:00
Tommi Virtanen
81be846c3c Do pseudorandom string generation 1 MB at a time.
This gives it a ~10x speedup.
2011-07-11 11:53:49 -07:00
Tommi Virtanen
2edd78ebbc Merge remote branch 'newdream/utilities' 2011-07-11 11:53:31 -07:00
Wesley Spikes
90d0f06522 Removing dead code (generate_objects:connect_s3) 2011-07-08 15:04:22 -07:00
Steven Berler
99ef831a0d dho qa: update rand readwrite for file generator
Now works correctly again after the changes to the random file
generator.  Also now gets the true size of files when generating
using a stddev != 0 (rather than just assuming all files were
the mean size).
2011-07-08 14:52:22 -07:00
Kyle Marsh
e5f9783053 dho-qa: disentangle file generation from uploading
Static load test script now provides separate functions for generating a
list of random-file pointers and uploading those files to an S3 store.  When
run as a script it still does both, but you can call each function
individually from a different script after loading the module.
2011-07-08 14:41:41 -07:00
Steven Berler
88c9b56c8b Revert "atomic read tests"
This reverts commit c75fb48e81.
2011-07-08 14:15:47 -07:00
Steven Berler
a1e5c50dc6 dho qa: rand_readwrite
Adds the rand_readwrite utility.
Updates realistic.py with a file verifier class.
Updates generate_objects.py to allow the filename seed to be set.
2011-07-08 13:14:49 -07:00
Steven Berler
c75fb48e81 atomic read tests
NOT DONE YET
2011-07-08 11:31:05 -07:00
Kyle Marsh
a4e5be5f41 dho-qa: static file generator updated to use common
Updated generate_objects.py to use Wes's common setup and configuration
stuff.  Can still override config on the command line.
2011-07-08 11:27:56 -07:00
Kyle Marsh
951dc0fcdb dho-qa: Add siege config file and document running siege
Adds siege.conf file for siege configuration options
Adds docstring to main function in generate_objects.py describing how to run
the static content load test.
2011-07-08 11:27:56 -07:00
Kyle Marsh
3a3cbb3d25 DHO QA: Random Object Generation Script
Script to generate garbage objects and push them to a bucket.

Script takes a config file on the command line (and some other command line
options using optparse) and generates a bunch of objects in an S3 bucket.
Also prints public URLs to stdout.

Number and sizes of the objects are determined by a yaml config file with each line
looking like this:
- [A, B, C]
A: Number of files in this group
B: Mean size of files in this group (in bytes)
C: Standard deviation (normal distribution) of file sizes in this group

command line options are:
 - S3 access key
 - S3 secret key
 - seed for PRNG
 - output file to write URLs to
 - flag to add md5 checksum to url list
2011-07-08 11:27:56 -07:00
Colin Patrick McCabe
015bc308af add tests for setting wacky metadata
Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
2011-07-07 19:41:03 -07:00
Wesley Spikes
262f1eecd1 Add teardown to common. 2011-07-07 17:34:43 -07:00
Wesley Spikes
3ee10d3dbe test_s3 should reset permissions prior to attempting delete 2011-07-07 17:34:43 -07:00
Wesley Spikes
82693cea22 Adding common, and a sample config.yml
Introduces a new dependancy on PyYAML
2011-07-07 17:34:43 -07:00
Steven Berler
d81e2d40e6 atomic dual-write tests
refactors so the FakeFile and Verifier classes can be used in multiple
tests and adds a helper function to verify data.

adds new tests similar to the previous atomic write tests, but this time
does a second write in the middle of writing (rather than doing a read
in the middle)
2011-07-07 13:35:17 -07:00
Steven Berler
9279998aa6 fix atomic write test
the interrupt() now shouldn't be getting called extra times
after the file has been read completely
2011-07-07 12:11:41 -07:00
Steven Berler
d65a33421a atomic write test
The atomic write test writes a large file of all A's followed by
overwriting the file with B's. The file is verified (to be either
all A's or all B's) after each write and just before the overwrite
is complete.

The test is performed 3 times, with sizes of 1 MB, 4 MB, and 8 MB.
2011-07-06 17:22:10 -07:00
Yehuda Sadeh
d061f44e74 fix a botched cleanup 2011-06-29 14:14:51 -07:00
Yehuda Sadeh
abcb67dec0 test for 100-continue 2011-06-28 12:49:20 -07:00
Yehuda Sadeh
5412934ec7 multipart tests 2011-06-17 15:12:47 -07:00