AVR Freaks

What an interesting project.

I'm guessing that none of the serial comm's channels use either hardware or software flow control?

If any of the channels used flow control then clearly your board doesn't control the delay at the other end, and your board could easily overflow any buffer size...

I'm also guessing that the system is such that you won't have two incoming packets at the same time with the same destination?  If so, then the task is much more complex, as you need to make sure you don't "mix" / interleave data from the two incoming sources into the single outgoing channel's buffer.

I wonder about the need for obtaining / verifying a full packet before sending it on.  If this is necessary then so be it.  If the end receiving device is going to do its own incoming data packet analysis and end of packet determination, then your board in the middle may not need to do so. 

The issue here is that from a data throughput perspective, if you hold a packet until it is full before initiating its transmission then you impeding your overall system's throughput for two reasons.

First you are spending processing time continually testing for a full packet condition.

Next, the USART isn't being allowed to use any of the time while the packet is being collected to send data.  i.e. the USART transmit side has a lot of dead time where it isn't doing any data transmission.  Then, when there is a full packet, it suddenly has to send 200 Bytes, (or whatever), all at once.  The system is essentially "batching" the data transmission, so the USART transmitter is either doing nothing, or it is very busy.  With sixteen USARTs running simultaneously, it might be better to have them all running continuously, instead of potentially have them all get very busy simultaneously.

Next question, is it just the Received data that is ISR driven, or both TxData and RxData.  I would certainly expect the RxData to be interrupt driven, but one might envision a system where the Main Loop, as it looks at the various buffer/USART channels, reads that USART's TxData busy bit and then stuffs the next data Byte to send.  That would (likely) be more efficient from an overall system perspective than using interrupts for the transmission side as well.  Also, given the number of data channels for the Main Loop to Round-Robin process, the Tx Busy bit is likely never to be busy when read.

Two final comments:  Is there enough processor power remaining to monitor a "watermark" for each buffer to see if they ever exceed 80% (or whatever) full?  If so, one can turn on a Red warning LED...  (A project can never have too many LED's!)  The real purpose, however, is not to have more flashing lights, but to be able to verify the buffer safety margin while the system is in real time usage, and perhaps be able to set the buffers to different sizes if it turns out that certain buffers tend to need more space than others.

Finally, it sounds like this might be a "mission critical" application, so evaluate this last comment carefully.  The Xmega's, if one isn't using the EEPROM or the analog modules, (ADC, DAC, AC), can likely run well while being a bit over-clocked.  Atomic Zombie says they run well at 64 MHz (IIRC).  I've run a couple at 48 MHz without any problems, (room temperature).  Getting an extra 16,000,000 clock cycles / second by changing the Xmega clock PLL value might be an option if you need more processing power. 

Remember that the chip's 32 MHz spec is what the manufacturer is willing to guarantee over the full production run, and temperature and voltage range.  There are many, many PC's and digital O'scopes, etc., that run just fine being over-clocked, and validated by the designer/manufacturer, albeit "out of spec".  Engineering is all about tradeoffs, and OC'ing isn't an inherently bad thing to do as long as one goes about it in a reasonable manner.   But it sounds like your system is already up and running well without this being required.

JC

I'm not sure if the MCU @ 32 MHz can handle all those interrupts.

I used to work with network "Terminal Servers."   7 ports at 115200bps full duplex is indeed challenging.   That's 161000+ interrupts/s - less than 200 cycles per interrupt.  Faster CPUs (ARM) might not help much, because interrupt overhead tends to go up as well.  In the terminal server world, the issue was solved with smarter UARTs.  Deep FIFO, Hardware flow control done by hardware, DMA, PPP and SLIP engines IN THE UART, and stuff like that.  The XMEGA UART only has two bytes of FIFO on the receiver, so...

There are a couple of tricks you can look at:

1. don't forget the AVR-GCC "penalty" for calling functions from an ISR.  Arrange your code so that each ISR service routine is one function.
2. Interrupt on TX Complete instead of TX Data Register Empty.   Halves the number of TX interrupts, at a possible cost of small gaps in the TX stream.
3. Or use DMA.  DMA for transmit is relatively easy to implement.  RX is more of a problem, because of the whole "asynchronous"  thing.  If your data doesn't have "end" conditions that the DMA controller can recognize, it may not be worthwhile.
4. If your messages have a recognizable structure, run state machines in the ISR instead of circular buffers with message recognition in the main code.
5. Arrange to pass messages from rx on one uart to tx on another without needing to copy all the data.
6. I put a bipolar LED tweak in my ISR code.  Red when in the ISR, Green when you exit.  A really quick indicator of how bad things were...

FSF AVR GCC 8 is reducing some of the ISR overhead.

I don't think I understand how it could get rid of pushes of the Call-used registers (r18-r27, r30-r31); it's sorta required by the ABI definition.

There are plenty Dual/Quad even Octal UARTS on the 8bit parallel bus. Xmega A1 has 8bit parallel bus so it should be easy.

https://www.nxp.com/products/ana...

Maxim Quad UART, SPI/I2c: MAX14830ETM+

Mouser in stock.

EXAR Dual Uart, SPI/I2C: XR20M1172IG28-F

Mouser in stock

I would recommend the SC16IS750/52/60/62 I2C/SPI uarts family from NXP. They also have 8 GPIOs and are quite cheap in fact.

https://www.nxp.com/docs/en/broc...

Of course you need two dual uarts for your project, but only one XMEGA. (8+2+2 = 12 uarts)

Trust me, dual cpu firmware developing is a nightmare

I have a project which needs 9 uarts. * uarts are plain RS232 full-duplex ports, baud range 9600-38400. One uart is half-suplex RS485 at 38400.

The thing is I also needed a SPI for interfacing a SPI ethernet controller.

At first I made the board with an XMEGA. The RS485 port was the software one, at least it was half-duplex. The problem with the XMEGA was that using all 8 UARTs "consumed" the SPI ports also, so I was forced to bit-banged the SPI, which took quite a good deal of the cpu processing bandwidth (this was for an spi ethernet controller doing tcp/ip !).

The final design is using a STM32F091 chip. 8 UARTs, RS485 still software, but hardware SPI and more bandwidth overall. I have done a "torture" test to see what bandwidth the board supports on all serials, concurently. I remember STM32 was able to deliver about 30-40% more data over the ports.

Octal UARTs are ancient. I wouldn't use them.

And LOL, I just forgot the unavailability of the SPI/I2C ports when using all UARTs, in the same post :)

So maybe a better system would be an XMEGA A1 with an SC16C554/654/754 quad uart over external 8bit parallel bus...

32 cycles, which is more than enough for an efficient interrupt handler.

I don't think so.  My back-of envelope calcs say ~10 cycles for ISR entry and reti (not including any context save.)  Minimum 5 cycles to read/inc/write the buffer pointer, 4 cycles to read/write the uart data.  That'll use at least 3 registers and the PSR, so add 12 cycles for push/pop and 2 more for reading/writing PSR.  That's 10+5+4+12+2 = 33 cycles, without checking for buffer wraps, or UART status, or keeping a byte count, or other things that are typical for a UART ISR.

Fortunately, you miscalculated - 115200bps is 8.6us per BIT, but only 1/10th of that per character.  OTOH, you also forgot "full duplex."

Ah yes, I clearly had not drunk enough coffee before writing that this morning.

But 32 cycles is definitely enough.  Consider this:

	push	r18
	push	r30			; Z
	push	r31

	lds		r18, 0x08A0	; USARTC0_DATA
	in		r30, 0x00	; GPIO0
	in		r31, 0x01	; GPIO1
	st		Z+, r18
	out		0x00, r30	; GPIO0

	pop		r31
	pop		r30
	pop		r18
	reti

21 cycles, plus worst case 8 cycles to enter the interrupt (3 to finish executing current instruction, 2 to save PC, 3 for JMP) which gives us 29 cycles.

I only store the lower byte of the buffer pointer so that I have have a circular 256 byte buffer. New bytes are detected by the main loop simply caching the old pointer value and comparing. Could potentially have issues with buffer overflow.

But this is quite interesting, because it puts the theoretical limit at around 8 x 1Mbaud USARTs at 32MHz. Thus this application should be pretty straight forward, performance wise.

Edit: I wonder if you need to use r18, or if you can ld into r31...

I may said it a bit ambiguous: two distinct microcontrollers (same or not - doesn't matter) each of which having a firmware project, compilation, binary downloading (two dongles?), testing etc.

Not to say the testing when something doesn't work... Is it micro #1? Or micro #2?

The solution is of course a very clear separation of the firmware, ideally developing one of the firmware first, completely, and thorougly tested. Even then, thorougly tested, you are not 100% sure the version is "stable". There are great chances it will change in the developing stage of the firmware #2...

I'd suggest using SPI for inter-MCU communication too. It's fast, robust, easy to use with DMA... EBI will work but is more error prone, and harder to debug.