Electronic – What’s the difference between a generated hex file and a binary file in embedded systems

bootloaderembeddedotastm32

I have an STM32 project (let's call it blinky). This project is developed using Keil IDE. It blinks an LED every 1 second. When I build it, a HEX file is generated. Let's say I want to upgrade the board using OAD (Over the Air Download). The new code allows the LED to blink every 2 seconds. In order to flash it using the OAD, a bootloader should be present. Let's suppose that this bootloader exists.

My questions are:

  • In order to generate a binary file and upgrade the board using OAD, what should I do in my Keil project exactly?
  • Should I just export the project as a binary instead of a hex file?
  • Are there any other configurations to do?

Best Answer

There are MANY different file formats that all accurately represent the term "binary". When you compile a printf("Hello World!\n"); program on your computer that is a binary but the binary contains much more information than just the instructions and data for the program. And depending on the operating system multiple different binary formats are supported. If using the gnu tools you can see from arm-whatever-objcopy --help the list the supported file formats at the end, on mine

arm-none-eabi-objcopy: supported targets: elf32-littlearm elf32-littlearm-fdpic elf32-bigarm elf32-bigarm-fdpic elf32-little elf32-big plugin srec symbolsrec verilog tekhex binary ihex

If I take a simple program for your microcontroller:

.thumb
.globl _start
_start:
    .word 0x20001000
    .word reset
reset: b reset

And build it with the gnu tools (for demonstration purposes will make sense in a second)

arm-none-eabi-as so.s -o so.o
arm-none-eabi-ld -Ttext=0x08000000 so.o -o so.elf
arm-none-eabi-objdump -D so.elf
arm-none-eabi-objdump -D so.elf

so.elf:     file format elf32-littlearm


Disassembly of section .text:

08000000 <_start>:
 8000000:   20001000    andcs   r1, r0, r0
 8000004:   08000008    stmdaeq r0, {r3}

08000008 <reset>:
 8000008:   e7fe        b.n 8000008 <reset>

so we see there are 10 bytes of machine code. The file format was shown as elf32-littlearm. Elf is a very popular file format used in a number of places as it is quite flexible and not tied in any way to an operating system nor target.

In this case the elf file is 66228 bytes of which we now know 10 are all we need in the mcu to run, the rest is debug information, file structure, etc.

hexdump -C so.elf
00000000  7f 45 4c 46 01 01 01 00  00 00 00 00 00 00 00 00  |.ELF............|
00000010  02 00 28 00 01 00 00 00  00 00 00 08 34 00 00 00  |..(.........4...|
00000020  c4 01 01 00 00 02 00 05  34 00 20 00 01 00 28 00  |........4. ...(.|
00000030  06 00 05 00 01 00 00 00  00 00 01 00 00 00 00 08  |................|
00000040  00 00 00 08 0a 00 00 00  0a 00 00 00 05 00 00 00  |................|
00000050  00 00 01 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00010000  00 10 00 20 08 00 00 08  fe e7 41 13 00 00 00 61  |... ......A....a|
00010010  65 61 62 69 00 01 09 00  00 00 06 02 09 01 00 00  |eabi............|
00010020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00010030  00 00 00 00 00 00 00 08  00 00 00 00 03 00 01 00  |................|
00010040  00 00 00 00 00 00 00 00  00 00 00 00 03 00 02 00  |................|
00010050  01 00 00 00 00 00 00 00  00 00 00 00 04 00 f1 ff  |................|
00010060  06 00 00 00 08 00 00 08  00 00 00 00 00 00 01 00  |................|
00010070  0c 00 00 00 00 00 00 08  00 00 00 00 00 00 01 00  |................|
00010080  0f 00 00 00 08 00 00 08  00 00 00 00 00 00 01 00  |................|
00010090  21 00 00 00 0a 00 01 08  00 00 00 00 10 00 01 00  |!...............|
000100a0  12 00 00 00 0a 00 01 08  00 00 00 00 10 00 01 00  |................|
000100b0  20 00 00 00 0a 00 01 08  00 00 00 00 10 00 01 00  | ...............|
000100c0  59 00 00 00 00 00 00 08  00 00 00 00 10 00 01 00  |Y...............|
000100d0  2c 00 00 00 0a 00 01 08  00 00 00 00 10 00 01 00  |,...............|
000100e0  38 00 00 00 0c 00 01 08  00 00 00 00 10 00 01 00  |8...............|
000100f0  40 00 00 00 0a 00 01 08  00 00 00 00 10 00 01 00  |@...............|
00010100  47 00 00 00 0c 00 01 08  00 00 00 00 10 00 01 00  |G...............|
00010110  4c 00 00 00 00 00 08 00  00 00 00 00 10 00 01 00  |L...............|
00010120  53 00 00 00 0a 00 01 08  00 00 00 00 10 00 01 00  |S...............|
00010130  00 73 6f 2e 6f 00 72 65  73 65 74 00 24 64 00 24  |.so.o.reset.$d.$|
00010140  74 00 5f 5f 62 73 73 5f  73 74 61 72 74 5f 5f 00  |t.__bss_start__.|
00010150  5f 5f 62 73 73 5f 65 6e  64 5f 5f 00 5f 5f 62 73  |__bss_end__.__bs|
00010160  73 5f 73 74 61 72 74 00  5f 5f 65 6e 64 5f 5f 00  |s_start.__end__.|
00010170  5f 65 64 61 74 61 00 5f  65 6e 64 00 5f 73 74 61  |_edata._end._sta|
00010180  63 6b 00 5f 5f 64 61 74  61 5f 73 74 61 72 74 00  |ck.__data_start.|
00010190  00 2e 73 79 6d 74 61 62  00 2e 73 74 72 74 61 62  |..symtab..strtab|
000101a0  00 2e 73 68 73 74 72 74  61 62 00 2e 74 65 78 74  |..shstrtab..text|
000101b0  00 2e 41 52 4d 2e 61 74  74 72 69 62 75 74 65 73  |..ARM.attributes|
000101c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000101e0  00 00 00 00 00 00 00 00  00 00 00 00 1b 00 00 00  |................|
000101f0  01 00 00 00 06 00 00 00  00 00 00 08 00 00 01 00  |................|
00010200  0a 00 00 00 00 00 00 00  00 00 00 00 04 00 00 00  |................|
00010210  00 00 00 00 21 00 00 00  03 00 00 70 00 00 00 00  |....!......p....|
00010220  00 00 00 00 0a 00 01 00  14 00 00 00 00 00 00 00  |................|
00010230  00 00 00 00 01 00 00 00  00 00 00 00 01 00 00 00  |................|
00010240  02 00 00 00 00 00 00 00  00 00 00 00 20 00 01 00  |............ ...|
00010250  10 01 00 00 04 00 00 00  07 00 00 00 04 00 00 00  |................|
00010260  10 00 00 00 09 00 00 00  03 00 00 00 00 00 00 00  |................|
00010270  00 00 00 00 30 01 01 00  60 00 00 00 00 00 00 00  |....0...`.......|
00010280  00 00 00 00 01 00 00 00  00 00 00 00 11 00 00 00  |................|
00010290  03 00 00 00 00 00 00 00  00 00 00 00 90 01 01 00  |................|
000102a0  31 00 00 00 00 00 00 00  00 00 00 00 01 00 00 00  |1...............|
000102b0  00 00 00 00                                       |....|
000102b4

Now usually when folks say hex file they mean intel hex, which you can look up the format on wikipedia.

arm-none-eabi-objcopy so.elf -O ihex so.hex
cat so.hex
:020000040800F2
:0A0000000010002008000008FEE7D1
:0400000508000000EF
:00000001FF
hexdump -C so.hex
00000000  3a 30 32 30 30 30 30 30  34 30 38 30 30 46 32 0d  |:020000040800F2.|
00000010  0a 3a 30 41 30 30 30 30  30 30 30 30 31 30 30 30  |.:0A000000001000|
00000020  32 30 30 38 30 30 30 30  30 38 46 45 45 37 44 31  |2008000008FEE7D1|
00000030  0d 0a 3a 30 34 30 30 30  30 30 35 30 38 30 30 30  |..:0400000508000|
00000040  30 30 30 45 46 0d 0a 3a  30 30 30 30 30 30 30 31  |000EF..:00000001|
00000050  46 46 0d 0a                                       |FF..|
00000054

This is an ASCII file format that contains the relevant information the address where the information is to be stored and the bytes themselves with a very small overhead, in this case 84 bytes.

My preference is motorola s-record, I was an intel person back in the day, but like srecord because it is easy to put the full address on the line. Yes this is no accident these file formats go back to the days when intel and motorola were the big dogs and beating each other up and doing things specifically to be incompatible, you have a file format, we we will have a file format. you can look up srecord on wikipedia as well might be named srec or motorola srec or something.

arm-none-eabi-objcopy --srec-forceS3 so.elf -O srec so.srec
cat so.srec
S00A0000736F2E7372656338
S30F080000000010002008000008FEE7C3
S70508000000F2

also an ASCII file easy to carry around and parse just like ihex. you can directly see the 0x08000000 address and the 0x20001000, 0x08000008 address (doh, lol, I put a bug in there) and the machine code 0xe7fe. Can see it in the intel hex as well, just the top of the 0x0800 address is on one line and the offset on another (why would we need a program larger than 64KBytes on an 8086?).

.thumb
.globl _start
_start:
    .word 0x20001000
    .word reset
.thumb_func
reset: b reset

bug fixed

08000000 <_start>:
 8000000:   20001000    andcs   r1, r0, r0
 8000004:   08000009    stmdaeq r0, {r0, r3}

08000008 <reset>:
 8000008:   e7fe        b.n 8000008 <reset>

Now so far all three of those file formats are called binaries in the same way that your hello_world.exe file is called a binary. The contain the "binary" information in this case a vector table and some machine code, they also contain the where I want to load the program information. Which for this program isnt really that interesting, but for programs where say I have some .text that wants to land at 0x00000 and some .data that wants to land at 0x10000000 say 1 byte at 0x00000000 and 1 byte at 0x10000000 I really only need two addresses and two bytes in a file format to store that information.

.text
.byte 0x11
.data
.byte 0x22

arm-none-eabi-as so.s -o so.o
arm-none-eabi-ld -Ttext=0x00000000 -Tdata=0x10000000 so.o -o so.elf
arm-none-eabi-objcopy --srec-forceS3 so.elf -O srec so.srec
cat so.srec 
S00A0000736F2E7372656338
S3060000000011E8
S3061000000022C7
S70500000000FA

now comes the rub, some folks (most of us) re-use the term binary to mean a byte for byte memory image, in this case instead of a 78 byte s-record file it would take a 0x10000001 or 268,435,457 bytes. To hold that file with most of the bytes being fill so the 0x22 lands in the right place relative to 0x11.

With our original program above.

arm-none-eabi-objcopy so.elf -O binary so.bin
hexdump -C so.bin
00000000  00 10 00 20 09 00 00 08  fe e7                    |... ......|
0000000a

its a 12 byte file, only the byte we need and is still confusing as it is not a 0x08000000+12 byte file to be aligned at address zero but the user of this file has to know that it needs to land in the stm32 memory space at 0x08000000 which although we have those values or close to them in the file there is no file structure that allows us to know that you just have to remember and not mess up.

So after all that TL;DR, the point is the term binary comes in many forms, as far as the type of embedded system you are talking about (at this point in time embedded system is such a wide concept and our phones are basically embedded systems where the microcontroller you are talking about is a much smaller system with more restrictions and rules) it is all about the tools you are using to place the file in the flash. The stm32 family in particular us supported by a vast array of tools, Kiel of which is a very small part of that. Kiel has file formats it supports you can look them up, some tools only support the memory image type binary although I find that foolish, some support simple ones like intel hex and/or motorola srecord and that has been a norm for decades. These days if you only support one it would be most likely elf and that would be fine for 99.9% with a bunch of nines after that of us. The memory image style does make some sense when it comes to the what turned into an mbed type approach the nucleo boards from st show up as a virtual usb/thumb drive on your computer and you drag and drop the file onto that, the front end bootloader mcu stops and loads the target mcu's flash. but when using a debugger like openocd or something in a gui you want more than just the memory image for various reasons. so the elf or whatever the native file format of the compiler that is often used in front of that tool makes sense.

For your specific question you need to read the documentation for your tool or just do some experiments to figure out the supported file formats.

I would kinda be surprised of Kiel produced or only output intel hex format files, maybe they call it a .hex but it isnt an intel hex. If you examine it with a text editor does it look like the intel hex files above?

If it is you can use gnu binutils tools with that kiel hex file to for example make a memory image type file:

arm-none-eabi-objcopy -I ihex so.hex -O binary so.bin
hexdump -C so.bin
00000000  00 10 00 20 09 00 00 08  fe e7                    |... ......|
0000000a

but because so much information other than the minimal to load bytes at some address is lost in the intel hex format file you cant convert it back to a mostly usable elf or other such formats. Which is why tools often use the feature rich file formats like elf from which you can work directly or you can convert to a format that retains less information.