I am mounting a S3 bucket on a amazon EC2 instance using S3FS. Things have been working great for now over 18 months.
Today I've tried to download a 5GB file using wget directly on the S3 bucket (things I already done before without issue) and I have a No space left on device
error.
user@ec2-instance:$ /home/workspaces/$ wget http://example.org/file.xml.gz
Cannot write to ‘files.xml.gz’ (No space left on device).
user@ec2-instance:$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 8125880 6576804 1113264 86% /
none 4 0 4 0% /sys/fs/cgroup
udev 15400288 12 15400276 1% /dev
tmpfs 3082424 216 3082208 1% /run
none 5120 0 5120 0% /run/lock
none 15412116 0 15412116 0% /run/shm
none 102400 0 102400 0% /run/user
s3fs 274877906944 0 274877906944 0% /home/workspaces
user@ec2-instance:$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/xvda1 524288 119569 404719 23% /
none 3853029 2 3853027 1% /sys/fs/cgroup
udev 3850072 380 3849692 1% /dev
tmpfs 3853029 262 3852767 1% /run
none 3853029 5 3853024 1% /run/lock
none 3853029 1 3853028 1% /run/shm
none 3853029 3 3853026 1% /run/user
s3fs 0 0 0 - /home/workspaces
I don't understand what df -i
return. I suspect that s3fs showing 0 inodes is an issue but I don't know what do do next.
This is not related to the Issue 288 in s3fs: No Space Left On Device for files larger than 5GB since:
- it is now fixed
- I've been able to download file larger than 5GB
Best Answer
According to this How to upload file directly to Amazon S3 from a remote server, you will have to download the file to local server before uploading it to S3 bucket.
Since you have not defined the save path, I afraid you are storing the large file locally at /tmp folder in which you don't have enough space for storing the 5 GB file.
You can further refer to this: How to upload a file to S3 without creating a temporary local file to see whether you can get rid of the temporary file created or expand your EBS volume to hold the large file before uploading to S3 bucket if you don't want to write an upload program.