Suppose I have a small boost regulator to bump up 2 AA cells to 5V, and I want to extend the battery life by going to 3 AA cells (4.8V with fresh batteries). This is outside the acceptable range of input voltage for such a regulator in the sense that the PWM control loop will be driven to one extreme or the other, right? But is this a bad thing? Even if it's out of regulation, if it just feeds the 4.8V through unregulated, that would be fine. Then as the batteries drain down, the regulator can start regulating normally.
BTW, going to larger-diameter batteries is not an option. Too bad they don't make longer AA cells with more capacity.