My current program has sort of grown, willy-nilly, and without much formal design. It was originally intended to be “just a little upgrade” from an existing program, leveraging everything that was muddled through in that first version.
Well, this second edition has become an almost total re-write. And two core routines, one to read data from a FIFO in a sensor, and the other to write to a uSD memory card, recently went through what was intended to be a “rewrite by design”.
I listed the requirements. I listed the various conditions they needed to respond to. I paid special attention to timing. I drew nice, and, I thought, thorough, flow charts. Only then was code for two functions committed to (virtual) paper. I was elated. All I need to do, now, is some thorough testing to verify that it does what it is supposed to do.
Oh, how naive! The cursory “well, does it crash?” test was passed. No biggie, I thought. Then, I started exercising them with various conditional cases. Ohhh, how sad. I had missed several of these conditional cases, and the algorithms that the functions implemented were really incorrect. For all practical purposes, both now need to be rewritten. But, even now, have I identified ALL of the conditional cases? I am not sure! I think I have, but I am not certain and I fear that I won’t know until testing after the rewrite of the rewrite happens.
I am now starting to believe that for most of us, especially not part of a team in which individuals can cross-check each other, the idea of being able to implement a fully designed process is more of a fantasy than reality. Oh, you can probably do it for a BLINKY program, but for anything with any complexity, I have real doubts. Its not like the program, itself, is a mess. It is driven by an orderly FSM. Events and timing are taken care of with the care of an RTOS. The big problem seems to be that of the fundamental complexity of the thing.
Maybe my mental capacity is too limited, or too aged, or too …. something? I can design electronic circuits in a snap and I have been doing embedded programming, off and on, for 30 years, so I’m not a newbie. But, something here just isn’t coming through. I guess that I just don't know how to manage complexity?
<edit> I guess that one of the conclusions that one can draw from this experience is that a good design requires absolutely perfect knowledge. And, that is something that I am unlikely, ever, to possess. </edit>