CLI Commands
mapfs provides a set of CLI commands for users to interact with the filesystem:
List all available commands
# list all available commands $ mapfs # help message $ mapfs --help $ mapfs -h # version $ mapfs --version $ mapfs -v # help message for a specific command $ mapfs <command> --help # e.g. help message for "mount" command $ mapfs mount # e.g. help message for "umount" command $ mapfs umount # e.g. help message for "add" command $ mapfs add
Start Channel Server
Channel Server is used to fanout/broadcast metadata changes among registered mapfs daemons in a cluster.
Usage:
$ channeld start [-p port] $ channeld stop $ channeld show
Show currently connected clients (mapfs mount daemons doing cache-invalidation metadata exchange through this ChannelServer):
$ channeld show Channel Index IP Joined a1b2c3d4e5f6a7b8 5 10.0.1.23 Yes a1b2c3d4e5f6a7b8 7 10.0.2.45 Yes c9d8e7f6a5b4c3d2 9 10.0.3.12 No
Each row is one connected client: Channel is the cache-invalidation group's fingerprint
(clients mounting the same volume share one, so an invalidation fans out only within that group, not to
every connected client); Index is the client's internal socket index; Joined is
whether it has completed its join handshake yet. The total number of currently connected clients is just
the row count below the header:
$ channeld show | tail -n +2 | wc -l
If nothing is connected, it prints No clients connected instead of the table.
Add/Remove one volume
Adding one volume means creating a mapping between a cloud storage and a local volume.
Removing one volume means deleting the mapping between a cloud storage and a local volume.
Usage:
$ mapfs add <VolumeName> <CloudVendor> [CloudOptions] [CacheOptions]
Up to 253 characters. May contain only lowercase letters, digits, '-' and '.'.
Must start and end with a lowercase letter or digit.
<CloudVendor>: aws | s3, google | gcs, aliyun, wasabi, cloudflare | r2, oracle | oci, s3compatible, azure
CloudOptions (required, depends on <CloudVendor>):
aws | s3 | gcs | google | aliyun | wasabi:
--access-key-id, -i <AccessKeyID>
--secret-access-key, -k <SecretAccessKey>
--bucket-name, -b <BucketName>
--region, -r <Region>
oracle | oci:
--access-key-id, -i <AccessKeyID>
--secret-access-key, -k <SecretAccessKey>
--bucket-name, -b <BucketName>
--region, -r <Region>
--oci-namespace, -n <OCI-Namespace>
cloudflare | r2:
--access-key-id, -i <AccessKeyID>
--secret-access-key, -k <SecretAccessKey>
--bucket-name, -b <BucketName>
--account-id, -u <AccountID>
s3compatible:
--access-key-id, -i <AccessKeyID>
--secret-access-key, -k <SecretAccessKey>
--bucket-name, -b <BucketName>
--endpoint, -e <Endpoint>
azure:
--connection-string, -c <ConnectionString>
--container-name, -t <ContainerName>
CacheOptions (optional):
--cache-dir, -d <CacheDir>
the directory to hold cache data.
default: /var/cache/mapfs/<VolumeName>
e.g., --cache-dir /data/MyVolume, -d /data/MyVolume
--cache-size, -z <CacheSizeInGB>
the maximum GB for the cache.
default: 100, maximum 100000 (100TB)
e.g., --cache-size 10, -z 10
--cache-timeout, -T <CacheTimeoutInSeconds>
the cache expire time in seconds. "0" means never expire.
default: 0
e.g., --cache-timeout 300, -T 300
Example:
$ mapfs add my-volume aws -i <AccessKeyID> -k <SecretAccessKey> -b my-bucket -r us-east-1 $ mapfs add my-volume oracle --access-key-id <AccessKeyID> --secret-access-key <SecretAccessKey> --bucket-name my-bucket --region us-ashburn-1 --oci-namespace mynamespace
List all configured volumes:
$ mapfs list
Remove one volume:
# mapfs remove <VolumeName> # for example $ mapfs remove my-volume
Configure Cache parameters
By default, the cache directory relies on /opt/mapfs/cache directory. And the default data cache size is 100GB.
1, Cache size is in GB.
2, Shrinking cache size is not supported.
3, Changing cache directory is not supported.
Configure cache size or timeout for an existing volume:
# mapfs configure <VolumeName> cache-size=<GB> # mapfs configure <VolumeName> cache-timeout=<seconds> # for example $ mapfs configure s3_volume cache-size=10 $ mapfs configure s3_volume cache-timeout=3600
Mount/Unmount one volume
After mouting one configured volume, users can access the cloud storage through the local mount point.
Note:
1, "iouring" mount option is recommended for extremely high IO performance scenarios with small size read requests.
for example, when the read request size is 4KB or even smaller, users can enable "iouring" to improve IO performance.
2, "mount" and "umount" commands require root privileges.
3, to mount one backend storage to multiple local directories, you have create multiple volumes this is because every volume has dedicated cache directory.
for example, to mount one s3 bucket to /mnt/fuse-a and /mnt/fuse-b, you must create two volumes for the single s3 bucket.
# mount one volume # mapfs mount <VolumeName> <MountPoint> [--channel IP[:port]] [--iouring] # for example $ mapfs mount s3_volume /mnt/s3_volume # sometimes, users may want to enable "fuse over io uring" to improve IOPS, then they can add "iouring" option when mounting the volume $ mapfs mount s3_volume /mnt/s3_volume iouring # after mouting, you can access the cloud storage through the local mount point $ ls /mnt/s3_volume $ cat /mnt/s3_volume/file.txt $ mkdir /mnt/s3_volume/new_folder
Unmounting one volume will disconnect the mapping between the cloud storage and the local mount point, but won't delete any data in cloud storage.
# unmount one volume # sudo mapfs umount <VolumeName>, or # sudo umount <MountPoint> [-f] # for example $ sudo mapfs umount s3_volume $ sudo umount /mnt/s3_volume $ sudo umount /mnt/s3_volume -f
Warm up cache
$ mapfs load <filename>
Get volume runtime status
Displays the IO Status, Cache Hit/Miss Statistics information for all the mounted volumes.
# get runtime status for all mounted volumes $ mapfs status