I am working on a device that uses the Microchip MDDFS library to store data to an SD card. The logger will log data at a maximum rate of 1 entry (56bytes) every minute. The problem is the device may lose power at any time, potentially in the middle of a write sequence. I am wondering what is the best way to protect my data against corruption. I have found that if the file is open when the power is lost, all data that was written to the file after the last file-close is lost. I don't know if the same holds true if power is lost in the middle of the write sequence.
Since the write procedure doesn't happen very frequently I could open the file, write the data, and then close the file, every time data is logged. Would this approach damage the sd card over time?
Another approach could be to keep the file open but after every 10 or 50 writes I could close the file and then re-open it.
I could also buffer data in memory, then flush the data occasionally maybe after a kbyte or so.
The last idea I had was, in my circuit, I could add a large capacitor that would provide power to my pic/sd card long enough after the power is disconnected to quickly close the file. The problem with this approach is that the time it takes to close the file and/or save data is very inconsistent. From my understanding, this time can very depending on the current place in a flash page that the file is in.
Anyways, what would you guys suggest?
Best Answer
A few things can happen when you write data to a file. I'm going to describe the sequence that needs to happen for data to be safe, not necessarily library calls.
When you're writing, and adding on to the end of the file (normal write mode), you read the last block of the file into memory, modify it with your write data, and then write the whole block back to the SD card. If the block is full, then a new block must be found in the File Allocation Table (FAT). After finding a new block, the FAT must be updated, which is a read-modify-write cycle. If we're done with the file, then we need to update the file attributes (such as file length) in the root directory, which causes another read-modify-write cycle.
Minimizing your write time
Make sure that the file is already holding your data when you write a sector. If you start out with a large file and overwrite the data instead of appending the data, the data will be safe as soon as the SD card sector write is finished. You can eliminate one to two read-modify-write cycles that way. My start-up code would write 0's to a file in sector increments until the SD card is full, and then rewind to the beginning of the file.
Make the size of your data entries such that an integer number of entries fit in a sector. I would bump up your entries to 64 bytes. While this is less efficient, it will prevent you from needing to read-modify-write two sectors.
Create a variant of the FSwrite function that allows you to write whole sectors. If you keep the whole sector in SRAM, then your cycle goes from "read-modify-write" to "modify-write"
Keep your PIC and SD power on as long as possible
Big capacitors are good. 470uF should give you more than enough power to finish a write cycle.
Make sure your power source won't suck the power out of your back-up capacitor! Add a diode if necessary.
Know when you're out of power