This article provides instructions on how to copy DSEFS files across DataStax Enterprise clusters.
- DSE source and target clusters must have nodes running in Analytics mode.
- DSE Analytics nodes have DSEFS enabled and configured.
- DSE clusters need to be running the same major DSE version for compatibility.
- Some features may not be compatible when source and target clusters are on different DSE versions.
- Execute DSEFS commands on the cluster running with the newer DSE version though it is not guaranteed to work with mixed versions.
- If authentication is enabled, both clusters must share the same credentials since it not possible to supply multiple credentials on the command line.
There are cases where a user or application requires the ability to copy files across clusters. Examples are data migration, moving data between production and staging, populating data on a development environment.
In order to provide working examples, the following 2 clusters were created for this article:
CLUSTER 1 - DSEFS source
Datacenter: Analytics ===================== Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns Host ID Rack UN 10.100.1.10 126.79 KiB 128 ? 38d9fe7a-a848-4b77-b440-803550d28db3 rack1
CLUSTER 2 - DSEFS destination
Datacenter: Analytics ===================== Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns Host ID Rack UN 10.200.2.20 126.79 KiB 128 ? 96a457be-d47b-43f8-ace7-76dabcea49d5 rack1
On the source cluster, a test directory was created in DSEFS as follows:
10.100.1.10 $ dse fs "mkdir portfolio" 10.100.1.10 $ dse fs "ls" portfolio
Then one of the files from a local demo directory was copied to DSEFS:
10.100.1.10 $ dse fs "cp file:demos/portfolio_manager/README.md portfolio/" 10.100.1.10 $ dse fs "ls -R portfolio" portfolio: README.md
On the destination cluster, the destination directory was created in DSEFS as follows:
10.200.2.20 $ dse fs "mkdir assets" 10.200.2.20 $ dse fs "ls" assets
The file was copied to the remote cluster as follows:
10.100.1.10 $ dse fs "cp portfolio/* dsefs://10.200.2.20:5598/assets/"
Check that the file exists remotely:
10.100.1.10 $ dse fs "ls -R dsefs://10.200.2.20:5598/assets" /assets: README.md
Check locally on the destination cluster:
10.200.2.20 $ dse fs "ls -R assets/"