AVR Freaks

My bad, seeing the replies made me realize the info I provided is not enough. I'll try to give more at the end but for now I got it working.

Looks interesting never used it before, I'll read up on that.

I might be wrong, but conventional style wasn't giving me the results that I wanted. Hence, trying this weird thing.

My brain fart.

This right here did the trick, now I get it, Yes task_B does not take more than 15*1600 clks to complete, therefore it should be fine to call it in the interrupt itself, sei() enables the interrupts in task_B which I wanted. Back of my mind I knew calling a function inside an interrupt is not a good idea, everywhere I read it's not recommended. But I guess if you do it carefully it should be fine.

Sorry, I didn't mention this, the task_A() function is very very long, you can say by the time task_A()function is completed, task_B() function would have been called many times (around 14, see diagram below). So the ISR, task_B and task A all run parallel, I know it can't run in parallel but you get the idea by interrupting each other it can. So priority wise ISR is the highest, then task_B() and then task_A().

Like you said here is what worked in dummy code (working on actual code right now to see it works),

#include <avr/io.h>
#include <avr/interrupt.h>

uint8_t stepCount = 15;
uint8_t run_A_now = 0;

inline void task_A(void)
{
	if(run_A_now)
	{
		run_A_now = 0;
		__asm__ __volatile__ ("nop");
	}
}

void task_B(void)
{
	__builtin_avr_delay_cycles(2000);	// Simulating some work to do
}

ISR(TIMER1_COMPA_vect)
{
	static uint8_t count = 0;

	__asm__ __volatile__ ("nop");	// Pretending to do some critical work here

	if(++count == 15)
	{
		count = 0;					// On every 15th interrupt, count resets to zero
		if(++stepCount == 16)
		{
			stepCount = 0;
			run_A_now = 1;
		}

		sei();
		task_B();
	}
}

int main(void)
{
    OCR1A = 1600;
    TIMSK1 |= (1 << OCIE1A);
    TCCR1B |= (1 << WGM12) | (1 << CS10);
    sei();

    while (1)
    {
		task_A();
    }
}

To make things more clear, Here is a timing diagram of each function, how much time it would really take, from here you can see why I was trying to do things this way. :- 

Apologies for still not explaining the problem properly. 

Agreed. All this just because the microcontroller is too slow for job or I am a bad coder. Got myself too familiar with the megaAVR that now going to another microcontroller is feeling like a big task. 

I guess it's time for me to read more about the scheduler.

I wish I could do that, but I can't. task_A is very ambiguous and such a hot mess cause it's written by me, can not split into chunks. Well that's a matter for another day.

Yes, exactly, more like :-

ISR:        |-|   |-|   |-|   |-|   |-|   |-|   |-|   |-|   |-|   |-|   |-|   |-|

Task-B:              |--   ---   --|

Task-A:   |-   ---                     ---   ---   ---   ---   ---   ---   --| 

Although thread is already marked solved, here is something I am experimenting with now (it's mostly the same with what clawson mentioned), it also does the job well, let me know is this better or worse (I mean less chance of something going horribly wrong)

#include <avr/io.h>
#include <avr/interrupt.h>

ISR(TIMER1_COMPA_vect)
{
	static uint8_t count = 0;

	// Critical ISR

	if(++count == 15)
	{
		count = 0;
		TIMSK1 |= (1 << OCIE1B);	// Enable run_task_b interrupt
	}
}

ISR(TIMER1_COMPB_vect, ISR_NOBLOCK)
{
	TIMSK1 &= ~(1 << OCIE1B);		// Disable run_task_b interrupt

	// Run task_B here
}

int main(void)
{
	TCCR1B |= (1 << WGM12) | (1 << CS10);
	OCR1A = 1600;
	TIMSK1 |= (1 << OCIE1A);
	sei();

    while (1)
    {
		// Run task_A here
    }
}

I am using TIMER1_COMPB_vect with ISR_NOBLOCK for task_B, on every 15th interrupt the execution from task_A goes to ISR then goes to task_B and returns to task_A (interrupts also works in task_B), on other cases it just goes back and fourth between task_A and ISR. No more calling function in an ISR.

If taskA don't get done before the next taskB and taskB doesn't get done in between time of 15 ISR's, that is definitely a catastrophic failure in my application.

I am fighting with clock cycles since the day one (learned a lot), now I got the hang of it, all tasks finish long before the deadline, So I am 100% sure that will not happen.

I see where this thread will lead to if I answer that. I try to keep the discussion in the thread about the problem mentioned, let's not deviate from that, of course If I tell the dependencies, someone might tell me why are you doing this X through Y when there is Z already available. In other words I have created this pyramid after many iterations of code, so if someone tell me to change something at bottom because that is the main cause of the problem, I won't be able to do that, the pyramid will fall, all the progress will be lost. It's okay if I fail in the end, but I can't go back right now. I already know I am not doing things the traditional way (be it better code, faster chip or multiple IC solution), before I hop on that train, I'll keep digging. With traditional way's (that I am aware of) the application is not possible "yet". Yes you might have a better solution, since you guys are the expert, I am just a newbie. If I fail hard, then I will create the thread about the underlaying workings and ask questions then, it will be more relevant, but until then Imma keep it on my own.

The application is 32 channel software PWM (capable upto 40) with individual channel control from outside, with high refresh rate and high bit depth ( with gamma correction for all channels ), plus still enough headroom to run led animations ( with individual animation speed control ) and decoding of animation data.  (megaAVR) Single Chip solution, baby let's gooooo!!

I don't think that is happening.

There are just two ISR's, one is very critical and the other I am using just as a function (task_B), task_A is running in main while loop. All is good right now, unless I find something else.

#include <avr/io.h>
#include <avr/interrupt.h>

ISR(TIMER1_COMPA_vect)
{
	static uint8_t count = 0;

	// Critical ISR

	if(++count == 15)
	{
		count = 0;
		TIMSK1 |= (1 << OCIE1B);	// Enable run_task_b interrupt
	}
}

ISR(TIMER1_COMPB_vect, ISR_NOBLOCK)
{
	TIMSK1 &= ~(1 << OCIE1B);		// Disable run_task_b interrupt

	// Run task_B here
}

int main(void)
{
	TCCR1B |= (1 << WGM12) | (1 << CS10);
	OCR1A = 1600;
	TIMSK1 |= (1 << OCIE1A);
	sei();

    while (1)
    {
		// Run task_A here
    }
}

No scheduler needed as of now.

Why an extra ISR for task_B you might ask?

Because as you see in clawson's reply (or marked solution), calling task_B function in the critical ISR push's all the registers to stack, because the complier thinks there is a possible of task_B happing in every interrupt. This hinders with main ISR execution time, due to all the pushing registers, what I did I put the task_b into another ISR with which has interrupts enabled by the way (also executes right after the main ISR) so all the registers all still getting pushed to stack for the task_B ISR, not for the main ISR. This insures that the critical ISR's execution and code is neat and clean and quick. So when task_B is not running, the switching time between the critical ISR and task_A is quick.

Yes indeed, there won't be a call to task_B therefore huge register PUSH/POPs will not happen for it but they gotta happen somewhere, wouldn't they take place when there will be a call to ISR(TIMER1_COMPA_vect), in this case ?

__attribute__((always_inline)) static inline void task_B(void)
{
    // Run task_B
}

ISR(TIMER1_COMPA_vect)
{
    static uint8_t count = 0;

    // critical ISR

    if(++count == 15)
    {
        count = 0;			// On every 15th interrupt, count resets to zero
        task_B();
    }
}

there are many instances in code where task_B is not happening and just switching between task_A and ISR.

I'd rather have huge push/pop's happen for task_B but not for the main ISR.

ISR(TIMER1_COMPA_vect)
{
	static uint8_t count = 0;

	// Critical ISR

	if(++count == 15)
	{
		count = 0;
		TIMSK1 |= (1 << OCIE1B);	// Enable run_task_b interrupt
	}
}

ISR(TIMER1_COMPB_vect, ISR_NOBLOCK)
{
	TIMSK1 &= ~(1 << OCIE1B);		// Disable run_task_b interrupt

	// Run task_B here
}

See, this is what the application really needs. Usually people use ISR's for setting a flag and do the work in the main while loop. I am here doing real work in ISR's, it drives the output pins. That's the difference.

Because it's complicated, the main while loop where Task_A is running, it's preparing data for the Task_B, and based on that data, for the next 16 times task_B is called, it's further manipulating the data, making it ready for the next 15 critical ISR's. All works like a clockwork. You might say why use task_B at all, cut the middle man, prepare the data in Task_A feed straight to critical ISR's. That won't be possible in my case, That's too much work for Task_A. You can say I am offloading some of the work of Task_A and turning into Task_B. Attempt to run them somewhat parallel, so the critical ISR can get what it wants in time. 

I will reduce the number of clock cycles going to waste(Idling) and turn them into higher refresh for led's. 

I see based on many people suggestions of RTOS is the way to go, or faster microcontroller for this kind of stuff, but before going that way I want to try this with megaAVR.

Even clawson back in 2008, suggested the idea of two ISR's in this thread.

If I knew that the one interrupt can interrupt itself with the use of sei() in interrupt beginning, or with ISR_NOBLOCK attribute, this thread wouldn't exist, that is what marked solution showed me, but that came with one penalty that was huge push/pop of the interrupt in my case, so I went with two interrupts, that way critical one stays out of huge push/pop's. The critical one can also interrupt the second interrupt. It's a win win.