Electronic – Execute instructions from RAM in a hobbyist-friendy microcontroller

Maybe i'm missing something here (paragraphs would help) but with the C18 compiler local variables within a function are generally allocated on the software stack so i have no idea what the following means:

but the reality is you have a finite amount of space to allocate all local function vars that exist in an entire project

By moving all your variables to globals within a module you are requiring there is space for all of them at the same time.

buffers and have to access that memory via pointers so I can avoid the semantics of memory banking.

What compiler are you using?

You have two separate issues:

1. Building the code for the external RAM address range.

   This is actually very easy. All you have to do is make a linker file that contains only the address ranges you want the code to occupy. Note that you not only need to reserve a particular program memory address range for these external apps, but also some RAM space. This RAM space, like the program memory addresses, needs to be fixed and known. Simply make only those fixed and known address ranges available for use in the linker file. Don't forget to make them NOT available in the base code linker file.

   After the base code loads a new app into external memory, it has to know how to execute it. The easiest thing is probably to have execution start at the first external RAM location. This means your code will need one CODE section at that absolute start address. This contains a GOTO to the right label in the rest of the code, which will all be relocatable.

2. Linking apps to library routines in the base code.

   There is no immediately simple way to do this with the existing Microchip tools, but it's not that bad either.

   A much bigger issue is how you want to deal with base code changes. The simplistic strategy is to build your base code, run a program over the resulting map file to harvest global addresses, then have it write a import file with EQU statements for all the globally defined symbols. This import file would then be included in all app code. There is nothing to link, since the app source code essentially contains the fixed address references to the base code entry points.

   That is easy to do and will work, but consider what happens when you change the base code. Even a minor bug fix could cause all the addresses to move around, and then all existing app code would be no good and have to be rebuilt. If you never plan to provide base code updates without updating all apps, then maybe you can get away with this, but I think it is a bad idea.

   A better way is to have a defined interface area at a chosen fixed known address in the base code. There would be one GOTO for every subroutine that app code can call. These GOTOs would be placed at fixed known addresses, and the external apps would only call to those locations, which would then jump to wherever the subroutine actually ended up in that build of the base code. This costs 2 program memory words per exported subroutine and two extra cycles at run time, but I think is well worth it.

   To do this right you need to automate the process of generating the GOTOs and the resulting export file that external apps will import to get the subroutine (actually GOTO redirector) addresses. You might be able to do with with some clever usage of MPASM macros, but if I were doing this I would definitely use my preprocessor since it can write to a external file at preprocessing time. You can write a preprocessor macro so that each redirector can be defined by a single line of source code. The macro does all the nasty stuff under the hood, which is to generate the GOTO, the external reference to the actual target routine, and add the appropriate line to the export file with the known constant address of that routine, all with appropriate names. Perhaps the macro just makes a bunch of preprocessor variables with regular names (kindof like a run-time expandable array), and then the export file is written once after all the macro calls. One of the many things my preprocessor can do that MPASM macros can't is to do string manipulation to build new symbol names from other names.

   My preprocessor and a bunch of other related stuff is available for free at www.embedinc.com/pic/dload.htm.