I have solved my problem. The reason for not being able to access files over 2GB in size is due to my antivirus software (McAfee for Linux - Endpoint Security for Linux Threat Prevention). It was not due to apparmor, funky filesystem options, permissions, disk space or older filesystems.
Here is how I came to my conclusion. The troubleshooting steps might be interesting.
Immediately following a reboot, I can quickly create a 2G file and probe it like so:
root@asci /tmp
# dd if=/dev/zero of=/tmp/file2049.iso bs=1M count=2049
2049+0 records in
2049+0 records out
2148532224 bytes (2.1 GB, 2.0 GiB) copied, 2.05888 s, 1.0 GB/s
root@asci /tmp
# md5sum file2049.iso
4555da35a7064cc499ba1e3f6fa1993a file2049.iso
Within a minute, the md5sum no longer works.
To rule out crontabs, I disabled crontab, and also wrote a 1-liner script to output 0 upon successful read and 1 if unsuccessful. Notice that it breaks at the 40'th second (indicating most likely not a cron job):
# while [ 1 == 1 ]; do echo -n "`date` - ";dd if=asm8.iso bs=1M count=30 2>&1|grep -c "Operation not permi"; sleep 1;date >> /var/log/ps.log; ps -efww >> /var/log/ps.log; done
Fri Jan 19 18:57:28 CET 2018 - 0
Fri Jan 19 18:57:30 CET 2018 - 0
Fri Jan 19 18:57:31 CET 2018 - 0
Fri Jan 19 18:57:32 CET 2018 - 0
Fri Jan 19 18:57:33 CET 2018 - 0
Fri Jan 19 18:57:34 CET 2018 - 0
Fri Jan 19 18:57:35 CET 2018 - 0
Fri Jan 19 18:57:36 CET 2018 - 0
Fri Jan 19 18:57:37 CET 2018 - 0
Fri Jan 19 18:57:38 CET 2018 - 0
Fri Jan 19 18:57:39 CET 2018 - 0
Fri Jan 19 18:57:40 CET 2018 - 1
Fri Jan 19 18:57:41 CET 2018 - 1
Fri Jan 19 18:57:42 CET 2018 - 1
Fri Jan 19 18:57:43 CET 2018 - 1
Fri Jan 19 18:57:44 CET 2018 - 1
I only dd the start of the file, as that's all I need to test it, as it takes 48 seconds to read the entire thing. I logged the ps output to /var/log/ps.log. To do a diff on the ps output by comparing what changed between the 39th second and 40th second, I did this:
# cat ps.log |grep "Fri Jan 19 18:57:39 CET 2018" -A10000 |grep "Fri Jan 19 18:57:40 CET 2018" -B10000 > /tmp/ps39
# cat ps.log |grep "Fri Jan 19 18:57:40 CET 2018" -A10000 |grep "Fri Jan 19 18:57:41 CET 2018" -B10000 > /tmp/ps40
# cd /tmp
Comparing the ps outputs between the 39th and 40th second:
# diff ps39 ps40
Fri Jan 19 18:57:40 CET 2018
266c266
root 2412 1276 97 18:57 ? 00:00:19 /opt/isec/ens/threatprevention/bin/isectpd
275,276c275,278
root 2632 2412 1 18:57 ? 00:00:00 /opt/isec/ens/threatprevention/bin/isectpd
> root 2635 2412 11 18:57 ? 00:00:00 /opt/isec/ens/threatprevention/bin/isectpd
> root 2663 2518 0 18:57 pts/1 00:00:00 ps -efww
So, that's it! Once the isectpd process started up at the 40th second, it disabled access to my 2GB file.
Once I did this:
# systemctl stop isectpd
It started working.
Obviously, this is a bandaid until I find out how to allow > 2GB files from McAfee. If anyone has any experience with this, feel free to chime in.
Cheers.
Best Answer
I had a similar issue trying to make a swap partition on sda2. My solution was to type
This seemed to reset whatever issue I was having. To make sure this worked, use "lsblk" to check the mountpoint.