I hope that you are all and your families safe!
Assuming that I have a Hash function that resides in the bootloader section (which acts as a virtual ROM without physical access), while I have the normal software running in the rest of Flash memory starting from address 0x00. Considering serial communication (for simplicity), if I have the microcontroller connected to my laptop while this laptop sends a request over serial every a couple of minutes. This request requires computing the Hash value of the entire memory to check if something is changed or not. Now, in the current design, the request gets received by the normal software which redirects it to the Hash function in the bootloader section. My problem is that I do not want to trust this software. I would assume it contains malware. Is there a way in AVR that this request is handled by hardware first, i.e., interrupts, where the hardware would decide to whom to give this message. In other words, I suppose with each message received that there is an interrupt is fired where this interrupt should fire another interrupt that its interrupt vector routine exists in the bootloader section (considering it as a secure memory area). The task of this routine is to check this request and see the content of the message. If it contains some specific words, it would jump into the first instruction of the Hash function. Otherwise, it would just continue the normal operation of the software. But I think this approach, if it is valid, is still not guaranteed as there is still a possibility for the untrusted software to disable interrupts using cli() function. Am I missing something? Any suggestion? hint? tip?
So, in short, I want a technique to ensure the availability of HASH function residing in the bootloader section in the sense that whenever I invoke it over serial communication, it responds.