25a1b96537
Reorganise FsObject and factor more stuff into methods Note that os.Chtimes() isn't setting the same precision (us) as we are reading from os.Lstat() (ns) so the modification times don't roundtrip properly yet.
87 lines
2.7 KiB
Text
87 lines
2.7 KiB
Text
make 100% compatible with swift.py?
|
|
|
|
Make Env vars compatible with st?
|
|
|
|
Get and put the metadata in the libray (x-object-meta-mtime) when getting and putting a file?
|
|
|
|
This also puts meta-mtime
|
|
https://github.com/gholt/swiftly
|
|
|
|
As an integer, but it does parse it as a float
|
|
subargs.append('x-object-meta-mtime:%d' % getmtime(options.input_))
|
|
|
|
Need an iterate all objects routine... Could use a channel
|
|
- could just be a flag to Objects()
|
|
|
|
FIXME progress meter would be nice! Do this by wrapping the Reader with a progress bar
|
|
|
|
Do bandwidth limit by wrapping the Reader too
|
|
Maybe using https://jra-go.googlecode.com/hg/linkio/ which will work for multiple
|
|
uploads or downloads.
|
|
* code.google.com/p/mxk/go1/flowcontrol - only does one flow at once
|
|
Or maybe put into swift library.
|
|
|
|
Could have an integrity check mode where we check the MD5sums of the local vs the remote
|
|
|
|
Some stats would be nice!
|
|
|
|
Windows paths? Do we need to translate / and \?
|
|
|
|
Make swift timeouts be settable with command line parameters
|
|
|
|
Add bandwidth limit?
|
|
|
|
Make a wrapper in connection which
|
|
* measures bandwidth and reports it
|
|
* limits bandwidth using Reader and Writer
|
|
* for a pool of all individual connectinos
|
|
* does timeouts by setting a limit, seeing whether io has happened
|
|
and resetting it if it has
|
|
|
|
Check the locking in swift module!
|
|
|
|
501 not implemented for paths with a ? in them! Probably need to do some escaping...
|
|
hello? sausage: Failed to read info: HTTP Error: 501: 501 Not Implemented
|
|
|
|
Need to make directory objects otherwise can't upload an empty directory
|
|
* Or could upload empty directories only?
|
|
|
|
Make a fs.Errorf and count errors and log them at a different level
|
|
|
|
Seem to be able to read stats to ns precision (using struct stat
|
|
defined in asm-generic/stat.h)
|
|
|
|
https://www.kernel.org/doc/man-pages/online/pages/man2/stat.2.html
|
|
|
|
but only write them to us precision.
|
|
This is confirmed by looking at the source for Chtimes
|
|
|
|
func Chtimes(name string, atime time.Time, mtime time.Time) error {
|
|
var utimes [2]syscall.Timeval
|
|
atime_ns := atime.Unix()*1e9 + int64(atime.Nanosecond())
|
|
mtime_ns := mtime.Unix()*1e9 + int64(mtime.Nanosecond())
|
|
utimes[0] = syscall.NsecToTimeval(atime_ns)
|
|
utimes[1] = syscall.NsecToTimeval(mtime_ns)
|
|
if e := syscall.Utimes(name, utimes[0:]); e != nil {
|
|
return &PathError{"chtimes", name, e}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
And
|
|
|
|
func NsecToTimeval(nsec int64) (tv Timeval) {
|
|
nsec += 999 // round up to microsecond
|
|
tv.Sec = nsec / 1e9
|
|
tv.Usec = nsec % 1e9 / 1e3
|
|
return
|
|
}
|
|
|
|
Seems likely Go should be using
|
|
utimensat() and futimens()
|
|
Instead of utimes() - the syscall is defined already
|
|
SYS_UTIMENSAT
|
|
Also for all the *bsd so likely they could be fixed too
|
|
|
|
Can also fix Futime which isn't using the syscall under linux
|
|
|