There are many microcontrollers out there (a microprocessor is slightly different), and propably an ARM is the first choice if you want performance. Hook up RAM and processor is conceptually trivial, but the practice can be complicated if you are not expert.
RAM is probably embedded if you buy a microcontroller, so just load the software from the Flash; and wait for using operating systems, first try to play a little bit with the board.
If you don't have a specific requirement for multiprocessor systems (and you should have them in the same chip, I don't think is feasible with microcontrollers) then it makes no sense to get crazy with that kind of stuff. And if you have to learn, I would suggest again of starting with something simple, you can't think to build a computer without even have build a microcontroller board, at least once.
And probably you are expert in programming, but even if you have the PCB made, programming with microcontrollers requires the management of the hardware, and that's also a thing to learn before going further.
What's this patching of that gizmo worth to you? If you know nothing about 8051 it may be a long and difficult task. Disassembling the code will give you a listing of tens of pages where only the instructions are readable, at least for someone who knows 8051. Labels and variables are just hollow numbers, and of course it's not commented either. If the source was C instead of assembler it may even look more abstract.
The datasheet is not a great help re the programming. It mentions ISP (In System Programming) and IAP (In Application Programming), but the ISP procedure seems to include setting SFRs, so it looks like that's actually IAP as well.
ISP means that you have outside access to a microcontroller which is mounted on your product's PCB, to program it. Everything is controlled through external pins.
IAP also means programming the microcontroller inside your product, but here the microcontroller has to execute a software procedure to talk to the external programming equipment and program the Flash.
So the controller seems to need programming code to reprogram the device. It will probably use the UART for this, but the programming facility has to be programmed in the device in the first place, and most low-cost commercial devices are not made to be reprogrammed.
The datasheet doesn't say anything about programming the device on a programmer, but since it's 8051 compatible the programming may be compatible with that as well, and any legacy universal programmer may be able to program it.
You'll have to carefully remove it from the PCB, so that you can resolder it later.
Pault's AVR alternative may be a good idea. I've never used them to replace 8051's, so check the datasheet carefully, pin for pin. Pault mentions the ATMega162, and it seems the ATMega8515 is also 8051 pin-compatible. Note, however, that neither datasheets mentions anything about this.
Best Answer
It would be essentially impossible for you to reprogram the microcontroller in a commercial product like a Canon calculator. First of all, the chip is probably propriety, and not something you could find documentation on. Even if you could, it is unlikely it is able to be reprogrammed within the device -- the chips are likely preprogrammed at the factory before being soldered in place. You would need to buy the necessary hardware to reprogram the chip -- that alone would cost more than your printer.
Then you have the issue of writing new firmware. Without source code, you would have to write the entire firmware from scratch. We're not talking a few days, but several weeks, at the least, of effort. Anyway, without a schematic, and a knowledge of all of the other chips on the board, you would have no idea where to start.
Is it possible? Very unlikely. Is it worth the effort? Absolutely not.