Ok I am aware of the memory fragmentation after using malloc multiple times and freeing multiple times.
Whats the way around it? I am allocating and deallocating 2K ish memory size for each of my tcp sockets tx and rx buffers. So all up 4k memory for each sockets. After sockets are closed I free the memory. But it seems after about doing a few of these my malloc fails and returns with NULL :(
Is this normal?
Why not pre allocate an array of packets and just use an array entry when you need one?
If the packets are variable size make the array entries as large as the largest one you will get and just waste the final few bytes for smaller packets.
You will know at compile time how many such array elements the system can cater for and if you need more you know you need to trade up to a larger device or add RAM (if technically possible).
As you've found fragmentation is, otherwise, quite a problem on micros with only a few K of memory unless all malloc()s and free()s are the same size (in which case it'll just keep reallocating the same blocks). In fact this would be another solution - make all the malloc's as big as the biggest one.
Another idea would be to "clear the heap" from time to time. When you get a lull in activity simply free() everything that's been allocated and the heap will return to its original "clean" condition.
Ah seems I was not freeing the actual tx rx buffers in the sockets! I was freeing other data structures except for the tx rx buffers! grrr... anyways so I can now seem to be able to create sockets indefinitely...
I wonder if I could still come across the fragmentation issue... ?
Are the allocations of different size or not?
BTW when we develop we often do complete system simulation in a PC app with I/O "faked" and you can then implement a malloc()/free() that not only makes the allocation but draws a picture of the heap as dots on a window rectangle (maybe one pixel = 16 bytes or something?). As bytes are allocated paint the equivalent pixels red. When freed return to background colour. You then get a visual representation of how the heap is being used and you can see when it starts to fragment and so on.
I wonder if I could still come across the fragmentation issue... ?
You can often create a worst case on paper. e.g. a combination of some big and many small blocks.
You can devise a worst case that could be allocated in the worst way.
If you can run out of memory, then one day this worst case will actually happen in real life.
Cliff's suggestion of a static array is often a very good solution.
David.
Thanks guys.... but the point is as long as i free ALL allocations then fragmentation is not a problem right?
Lets say I create sockets of size 2K then some with 4K and some with 1K buffers. As long as all sockets get processed and thus removed and freed all buffers then I have no issues with fragmentation right?
but the point is as long as i free ALL allocations then fragmentation is not a problem right?
Quote:
but the point is as long as i free ALL allocations then fragmentation is not a problem right?
Right. You can even push that a bit further: As long as you free in the reverse order of the allocation order you will have no fragmentation.
What model of avr you using (just curious?)
What model of avr you using (just curious?)
(luvocean1 likes to ask his "generic" questions here rather than the AVR32 fora as apparently he gets a better response ;-) The problems of malloc() are likely to be the same everywhere though it obviously gets worse the less RAM you have))
Why should the order matter? Consecutive free blocks are merged, so at the end there is only one big free block no matter of the freeing order.
Better now?
That's true but the implication is that it's the only way to clear fragmentation. The point (I think) was that if you free() everything then it doesn't matter what order you do it in as everything will be merged back into one amorphous great lump.
Quote:
What model of avr you using (just curious?)
Peering into my crystal ball I actually see a UC3(luvocean1 likes to ask his "generic" questions here rather than the AVR32 fora as apparently he gets a better response ;-) The problems of malloc() are likely to be the same everywhere though it obviously gets worse the less RAM you have))
Clawson's witchcraft prediction is correct :P
So as long as ALL the sockets eventually gets closed I shouldnt have any memory issue.
Another question...what about for data structures I define not as malloced pointers but rather global variables? Where do they get stored? not heap?
Another question...what about for data structures I define not as malloced pointers but rather global variables? Where do they get stored? not heap?
The static data sections (.data and .bss). Variables get allocated there by the linker. There's discussion (for the 8 bit AVRs, but the same happens in almost all microcontrollers running C code) here:
http://www.nongnu.org/avr-libc/u...
- S
hhhm..thanks for that.
So I should try to define data structures as global variables where possible...
So I should try to define data structures as global variables where possible...
OK.
Whats a good way to store strings you can print using Usart debug to hyperterminal? I have lots of them...
does having them as global static const keep them only in code and not in RAM? Quite often I just do UsartSendString("hello worl") in my functions or subfunctions....where do these strings get stored?
Whats a good way to store strings you can print using Usart debug to hyperterminal? I have lots of them...
If you do use:
UsartSendString("hello world")
then by default that "hellow world" will be held in flash then as C starts up it will be copied to RAM then a pointer to its location there will be passed into the above call to UsartSendString(). In AVR8 the avr/pgmspace.h support offers
UsartSendString(PSTR("hello world"))
And the UsartSendString() would be written to dereference tha passed pointer as a code, not RAM address. Again I haven't a clue how this is handled in UC3 land.
Under the hood the UC3 is a modified Harvard Architecture. The flash and RAM live in the same address space although at different and non overlapping address ranges obviously. From a programmer's perspective you can't really distinguish it from von Neumann except that the flash is read only. You can put executable code in the RAM and the UC3 will happily execute it from there.
Declaring an initialized variable as const will automatically put it in flash (with the Atmel supplied tools).
const char *hw = "hello world";
const int intArray[] = {1,2,3,4,5};
I'm not sure how string constants are handled in the case you cite above. I'll have to spend a little time with an assembly listing. ;)
edit: after looking through some disassembled code, something like this:
usart_write_line (&AVR32_USART1, "hello world");
causes the string constant to be stored in flash and the address of that string in flash is passed as a parameter to the function.
Thanks Clawson and digitaldan...so if I pass the string inside the UsartSendString(), the string is placed in the flash section. So it shouldnt be affecting my RAM size.
Coming back to the original topic here... I did end up finding a section in the code where I was dereferencing a NULL unallocated memory area...this may have cause follow on issues I guess and then eventually corrupting the heap and causing another malloc to freeze the micro :)
Thanks Clawson and digitaldan...so if I pass the string inside the UsartSendString(), the string is placed in the flash section. So it shouldnt be affecting my RAM size
If you want to know what IS using your RAM then study the map file (though it will only show global/static allocations and not automatics or the heap).
Another question... is it possible to find out how much free memory is available in the heap? Any built in functions in C available for checking free memory?
Well one way is to use a for() loop starting with a stupidly high value and keep trying to malloc(). That is:
char * p;
int i;
for (i=32767; i>0; i--) {
p = (char *)malloc(i);
if (p) break;
}
if (p) {
free(p);
}
At the end of this i says how big the largest malloc() possible is.
lol that is really scary!
Surely there has to be some low level keeping track of the heap? When calling malloc it must check whether the size of free available memory is bigger than the space asked for in malloc....
So free memory checking is inside malloc function... we just need to know what variable it checks may be?
Well for AVR-LibC the source of malloc() is open:
http://svn.savannah.nongnu.org/v...
BTW while the C standard calls for malloc() and free() in stdlib it does not specify a freemem() type call or a specific form of implementation.
From a user perspective the repeated calls to malloc() I showed above really is the way to do it. My method will tell you the largest single block you can allocate. This may not be what you want. Perhaps you plan to make 100's of small malloc()s and there could still be some small blocks available after the large one is allocated so an alternative strategy is to just keep making "small" mallocs of N bytes until you get 0. Then the total is M * N where M is the number you made. Because you don't know how many you are likely to make and you need to free them all after then you probably want to build a linked list of allocations and then walk the list freeing them. This shows one way to do it:
Surely there has to be some low level keeping track of the heap? When calling malloc it must check whether the size of free available memory is bigger than the space asked for in malloc....
The source of avr-gcc malloc, free and the heap data structure is public and browsable. Have you had a look?
The source of avr-gcc malloc, free and the heap data structure is public and browsable. Have you had a look?
hhmmm.. interesting code! Malloc if not very efficient, the code for allocation seems expensive!
clawson... i was reluctant to use that for loop malloc method cos I was hoping to have a periodic task function that monitored the memory usage (free mem) on the cpu every now and then and made sure there arnt any memory leak etc....seems I cant do that then if I have to use those for looped malloc way...
a periodic task function