lwIP Raw TCP Data Transmission

Go To Last Post
3 posts / 0 new
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0



My SAME70Q21-based device uses the Ethernet interface to send data to a CPU situated elsewhere on the same PCB (i.e. a direct Ethernet connection, no switches or external networks involved). I'm using lwIP (Raw mode, i.e. NO_SYS = 1) for this purpose, making use of the 1.4.1 port for SAME70, which is included in the Atmel Studio 7 'THIRDPARTY_LWIP_RAW_BASIC_HTTP_EXAMPLE' example project.


The SAME70 uses lwIP primarily in TX mode, but does RX some data from the CPU from time to time (updating configuration, requesting certain data, etc.). I've tested the RX functionality, and it works reliably without issue. However, I'm having issues with the TX side of my lwIP implementation, which doesn't work as intended, for which I could really use some input.


Rather than posting hundreds of lines of code, I'll try and break down the code and approach function-by-function. Similarly, as the RX functionality works, and I'm able to send some TX data, I'll skip over the init() and accept() functions.


void someFunction(void)
    //Perform pre-processing on some data
    //Package a string to send to the CPU
    //buffer is an unsigned char array
    snprintf(buffer, sizeof(buffer), "%s%s%s%s%s%s%s", "1", char1, char2, char3, char4, char5, "0");
    //Add the data to the TCP queue
    //global_PCB is defined in the header file, and is assigned to the
    //TCP connection struct tcp_pcb in the accept() callback function
    err_t = err;
    err = tcp_write(global_PCB, &buffer, sizeof(buffer), 1);
    //Register the tcp_sent() callback
    tcp_sent(global_PCB, tcpSent);

As shown above, someFunction() performs some pre-processing on some data, and packages it into a char array buffer. The buffer is added to the TCP queue with tcp_write(), and the tcp_sent() callback function is then registered for when data is successfully received by the CPU. As far I can understand, the initial call to send the buffer data on the wire is made here, with any subsequent sending required (e.g. if the buffer data size is larger than the maximum segment size) managed by the tcp_sent() callback.


err_t tcp_sent_callback(void *arg, struct tcp_pcb *tpcb, u16_t len)
    struct tcp_pack *tcpPack;
    tcpPack = (struct tcp_pack*)arg;
    if (tcpPack->p != NULL) {
        tcp_send_data(tpcb, tcpPack);
    } else {
        if (tcpPack->state == STATE_CLOSING) {
            tcp_close_conn(tpcb, tcpPack);
    return ERR_OK;

The tcp_sent_callback() function is called when a packet of data has been received and acknowledged by the CPU. It checks whether the acknowledged pbuf (tcpPackage->p) is empty, and if not, sends the tcp_pack to the tcp_send_data() to send the next/remaining pbuf/data.


void tcp_send_data(struct tcp_pcb *tpcb, struct tcp_pack *tcpPack)
    struct pbuf *ptr;
    err_t wr_err = ERR_OK;
    while ((wr_err == ERR_OK) && (tcpPack->p != NULL) && (tcpPack->p->len <= tcp_sndbuf(tpcb))) {
        ptr = tcpPack->p;
        wr_err = tcp_write(tpcb, ptr->payload, ptr->len, 1);
        if (wr_err == ERR_OK) {
            u16_t plen;
            u8_t freed;
            plen = ptr->len;
            tcpPack->p = ptr->next;
            if (tcpPack->p != NULL) {
            do {
                freed = pbuf_free(ptr);
            } while (freed == 0);
            tcp_recved(tpcb, plen);
        } else if (wr_err == ERR_MEM) {
            tcpPack->p = ptr;

The tcp_send_data() function above is taken from the contrib tcpecho_raw example. Below are the pertinent components of my lwipopts.h file:


#define NO_SYS                    1
#define LWIP_RAW                  0
#define TCP_QUEUE_OOSEQ           1
#define TCP_OVERSIZE              TCP_MSS

#define MEM_SIZE                  (256 * 1024)
#define MEMP_NUM_TCP_PCB          1
#define MEMP_NUM_TCP_SEG          32
#define PBUF_POOL_SIZE            32

#define LWIP_TCP                  1
#define TCP_MSS                   1460
#define TCP_WND                   (4 * TCP_MSS)
#define TCP_SND_BUF               (8 * TCP_MSS)

#define TCP_SND_QUEUELEN          ((4 * (TCP_SND_BUF) + (TCP_MSS - 1)) / (TCP_MSS))

Now, the issue I'm having with the above is as follows: the data is successfully sent for a couple of packets (verified by receiving the expected data at the CPU), but TX always stops within a couple of seconds. Unfortunately I don't have access to the lwIP debugging functionality, as I don't have access to UART on my custom board; only the tracepoint functionality offered by the ATMEL-ICE programmer.


If I set tracepoints on the tcp_write() function call in someFunction(), it does show the err value change from 0 (ERR_OK) to -1 (ERR_MEM). I've tried tweaking a range of lwipopts.h values, but they don't seem to make any difference; the TX data stops being received by the CPU within a couple of seconds of starting the MCU program. As you can see from someFunction(), the payload is 7 chars, totalling <20 bytes. The someFunction() function is called every x milliseconds, so while the volume of tcp_write() calls is quite high, the actual payload/throughput of TCP data is quite low.


Does anyone have any pointers? I'm not 100% comfortable with the lwIP approach and code posted above, so I'm more than happy to alter/change/add/remove as needed, in addition to tweaking the lwipopts.h definitions.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Bit of an update. I've got the lwIP stack setup and running on my SAME70 Xplained board, with a crossover ethernet cable connecting it to a computer. I'm now able to make use of the EDBG functionality on the Xplained board, and monitor various printf and lwIP debug statements. I've built a test routine which sends 20 bytes of data (the same 20 bytes each time, dynamically created each time) every 2ms. On the SAME70 Xplained, I've printed the size of snd_buf() on each iteration, which shows the below very strange result. As you can see, normal operation persists for some time (i.e. data is queued with tcp_write() until the queued data is put on the wire and sent out, at which time the queue is freed and snd_buf() returns to its maximum size again), but there always comes a point where transmission stops (as verified on the receiving computer), data continues to be added to the queue at a constant rate, until the send buffer is full (i.e. snd_buf = 0). At this point, the tcp_write: too much data error is thrown.



There doesn't appear to be any reason/logic as to why transmission stops. lwIP doesn't throw any warnings or errors until snd_buf is exhausted. I've tried turning on all lwIP debugging, trying each of the debugging levels, but nothing coincides with the data transmission stopping. I'm currently thinking there are two likely causes:


  1. The remote computer misses an ACK or doesn't respond with an ACK in time? This shouldn't be the case, as TCP allows for dropped packets, and should be able to handle this situation. I'm going to try and get wireshark up and running today to verify the actual sequence of events.
  2. The SAME70 lwIP 1.4.1 port has a bug. The fact that lwIP isn't reporting any issues, and goes about adding data to the queue (i.e. tcp_write()) without issue, despite no data actually being sent out on the wire suggests that there may be an ethernet issue, but that it resides at a lower level than lwIP?


If anyone can think of anything else to check/consider I'd be all ears :)

Last Edited: Sat. Apr 18, 2020 - 08:07 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Can you get a packet trace (tcpdump) on the wire?