Add a mega.nz remote #163

Not supported yet:
  * Hash
  * ModTime
  * Server Side Copy

Otherwise fully functional and passing all the tests.

README.md

@@ -25,6 +25,7 @@ Rclone is a command line program to sync files and directories to and from
   * Google Drive
   * HTTP
   * Hubic
+  * Mega
   * Microsoft Azure Blob Storage
   * Microsoft OneDrive
   * Openstack Swift / Rackspace cloud files / Memset Memstore / OVH / Oracle Cloud Storage

backend/all/all.go

@@ -16,6 +16,7 @@ import (
 	_ "github.com/ncw/rclone/backend/http"
 	_ "github.com/ncw/rclone/backend/hubic"
 	_ "github.com/ncw/rclone/backend/local"
+	_ "github.com/ncw/rclone/backend/mega"
 	_ "github.com/ncw/rclone/backend/onedrive"
 	_ "github.com/ncw/rclone/backend/pcloud"
 	_ "github.com/ncw/rclone/backend/qingstor"

backend/mega/mega_test.go

@@ -0,0 +1,17 @@
+// Test Mega filesystem interface
+package mega_test
+
+import (
+	"testing"
+
+	"github.com/ncw/rclone/backend/mega"
+	"github.com/ncw/rclone/fstest/fstests"
+)
+
+// TestIntegration runs integration tests against the remote
+func TestIntegration(t *testing.T) {
+	fstests.Run(t, &fstests.Opt{
+		RemoteName: "TestMega:",
+		NilObject:  (*mega.Object)(nil),
+	})
+}

bin/make_manual.py

@@ -35,6 +35,7 @@ docs = [
     "drive.md",
     "http.md",
     "hubic.md",
+    "mega.md",
     "azureblob.md",
     "onedrive.md",
     "qingstor.md",

cmd/cmd.go

@@ -82,6 +82,7 @@ from various cloud storage systems and using file transfer services, such as:
   * Google Drive
   * HTTP
   * Hubic
+  * Mega
   * Microsoft Azure Blob Storage
   * Microsoft OneDrive
   * Openstack Swift / Rackspace cloud files / Memset Memstore

docs/content/about.md

@@ -28,6 +28,7 @@ Rclone is a command line program to sync files and directories to and from:
 * {{< provider name="Hubic" home="https://hubic.com/" config="/hubic/" >}}
 * {{< provider name="IBM COS S3" home="http://www.ibm.com/cloud/object-storage" config="/s3/#ibm-cos-s3" >}}
 * {{< provider name="Memset Memstore" home="https://www.memset.com/cloud/storage/" config="/swift/" >}}
+* {{< provider name="Mega" home="https://mega.nz/" config="/mega/" >}}
 * {{< provider name="Microsoft Azure Blob Storage" home="https://azure.microsoft.com/en-us/services/storage/blobs/" config="/azureblob/" >}}
 * {{< provider name="Microsoft OneDrive" home="https://onedrive.live.com/" config="/onedrive/" >}}
 * {{< provider name="Minio" home="https://www.minio.io/" config="/s3/#minio" >}}

docs/content/docs.md

@@ -33,6 +33,7 @@ See the following for detailed instructions for
   * [Google Drive](/drive/)
   * [HTTP](/http/)
   * [Hubic](/hubic/)
+  * [Mega](/mega/)
   * [Microsoft Azure Blob Storage](/azureblob/)
   * [Microsoft OneDrive](/onedrive/)
   * [Openstack Swift / Rackspace Cloudfiles / Memset Memstore](/swift/)

docs/content/mega.md

@@ -0,0 +1,108 @@
+---
+title: "Mega"
+description: "Rclone docs for Mega"
+date: "2018-04-09"
+---
+
+<i class="fa fa-archive"></i> Mega
+-----------------------------------------
+
+[Mega](https://mega.nz/) is a cloud storage and file hosting service
+known for its security feature where all files are encrypted locally
+before they are uploaded. This prevents anyone (including employees of
+Mega) from accessing the files without knowledge of the key used for
+encryption.
+
+This is an rclone backend for Mega which supports the file transfer
+features of Mega using the same client side encryption.
+
+Paths are specified as `remote:path`
+
+Paths may be as deep as required, eg `remote:directory/subdirectory`.
+
+Here is an example of how to make a remote called `remote`.  First run:
+
+     rclone config
+
+This will guide you through an interactive setup process:
+
+```
+No remotes found - make a new one
+n) New remote
+s) Set configuration password
+q) Quit config
+n/s/q> n
+name> remote
+Type of storage to configure.
+Choose a number from below, or type in your own value
+ 1 / Alias for a existing remote
+   \ "alias"
+[snip]
+14 / Mega
+   \ "mega"
+[snip]
+23 / http Connection
+   \ "http"
+Storage> mega
+User name
+user> you@example.com
+Password.
+y) Yes type in my own password
+g) Generate random password
+n) No leave this optional password blank
+y/g/n> y
+Enter the password:
+password:
+Confirm the password:
+password:
+Remote config
+--------------------
+[remote]
+type = mega
+user = you@example.com
+pass = *** ENCRYPTED ***
+--------------------
+y) Yes this is OK
+e) Edit this remote
+d) Delete this remote
+y/e/d> y
+```
+
+Once configured you can then use `rclone` like this,
+
+List directories in top level of your Mega
+
+    rclone lsd remote:
+
+List all the files in your Mega
+
+    rclone ls remote:
+
+To copy a local directory to an Mega directory called backup
+
+    rclone copy /home/source remote:backup
+
+### Modified time and hashes ###
+
+Mega does not support modification times or hashes yet.
+
+### Duplicated files ###
+
+Mega can have two files with exactly the same name and path (unlike a
+normal file system).
+
+Duplicated files cause problems with the syncing and you will see
+messages in the log about duplicates.
+
+Use `rclone dedupe` to fix duplicated files.
+
+### Limitations ###
+
+This backend uses the [go-mega go
+library](https://github.com/t3rm1n4l/go-mega) which is an opensource
+go library implementing the Mega API. There doesn't appear to be any
+documentation for the mega protocol beyond the [mega C++
+SDK](https://github.com/meganz/sdk) source code so there are likely
+quite a few errors still remaining in this library.
+
+Mega allows duplicate files which may confuse rclone.

docs/content/overview.md

@@ -27,6 +27,7 @@ Here is an overview of the major features of each cloud storage system.
 | Google Drive                 | MD5         | Yes     | No               | Yes             | R/W       |
 | HTTP                         | -           | No      | No               | No              | R         |
 | Hubic                        | MD5         | Yes     | No               | No              | R/W       |
+| Mega                         | -           | No      | No               | Yes             | -         |
 | Microsoft Azure Blob Storage | MD5         | Yes     | No               | No              | R/W       |
 | Microsoft OneDrive           | SHA1        | Yes     | Yes              | No              | R         |
 | Openstack Swift              | MD5         | Yes     | No               | No              | R/W       |
@@ -131,6 +132,7 @@ operations more efficient.
 | Google Drive                 | Yes   | Yes  | Yes  | Yes     | Yes     | No    | Yes          | Yes         | Yes |
 | HTTP                         | No    | No   | No   | No      | No      | No    | No           | No [#2178](https://github.com/ncw/rclone/issues/2178) | No  |
 | Hubic                        | Yes † | Yes  | No   | No      | No      | Yes   | Yes          | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
+| Mega                         | Yes   | No   | Yes  | Yes     | No      | No    | No           | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
 | Microsoft Azure Blob Storage | Yes   | Yes  | No   | No      | No      | Yes   | No           | No [#2178](https://github.com/ncw/rclone/issues/2178) | No  |
 | Microsoft OneDrive           | Yes   | Yes  | Yes  | No [#197](https://github.com/ncw/rclone/issues/197) | No [#575](https://github.com/ncw/rclone/issues/575) | No | No | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |
 | Openstack Swift              | Yes † | Yes  | No   | No      | No      | Yes   | Yes          | No [#2178](https://github.com/ncw/rclone/issues/2178) | Yes |

docs/layouts/chrome/navbar.html

@@ -66,6 +66,7 @@
                     <li><a href="/drive/"><i class="fa fa-google"></i> Google Drive</a></li>
                     <li><a href="/http/"><i class="fa fa-globe"></i> HTTP</a></li>
                     <li><a href="/hubic/"><i class="fa fa-space-shuttle"></i> Hubic</a></li>
+                    <li><a href="/mega/"><i class="fa fa-archive"></i> Mega</a></li>
                     <li><a href="/azureblob/"><i class="fa fa-windows"></i> Microsoft Azure Blob Storage</a></li>
                     <li><a href="/onedrive/"><i class="fa fa-windows"></i> Microsoft OneDrive</a></li>
                     <li><a href="/qingstor/"><i class="fa fa-hdd-o"></i> QingStor</a></li>

fstest/test_all/test_all.go

@@ -132,6 +132,11 @@ var (
 			SubDir:   false,
 			FastList: false,
 		},
+		{
+			Name:     "TestMega:",
+			SubDir:   false,
+			FastList: false,
+		},
 	}
 	// Flags
 	maxTries = flag.Int("maxtries", 5, "Number of times to try each test")