memory allocation question.

Go To Last Post
14 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm cretin there is no way to achieve this but figured I had better ask. I have a situation where I will be sending data to a computer via USB. I'm using an atmega 328 with a usb driver (v-usb). Because of the nature of my data I'm using a custom size. Much like CAN on a the OBD2 network. I first send a head instructing the length and type of data then send each line sequentially with a line number.

 

(header containing length and other info)

(1, x1,x2,x3,x4,x5,x6,x7)

(2, x8,x9,x10,x11,x12,x13,x14)

and so on.

 

For the report ID I just have a size of 8 and on the device side I assemble the data in to an array. The write way to often for EEprom (I'd burn that out in a matter of day). So I'm forced to use a static data size. My issue is that the atmega does not have much memory and I'm near %100. The data must be read in its full size. So I must read from usb in to an array first.

So my question is can the compiler be told in any way that this data size is dynamic. As far as I know, that's not possible as I need to prepare the memory. The data could stretch to 1024 at times and it must be global.

unsigned char data[1024];

 

The data is read from the main and used in the other cpp files. Just curious if this is the best way to accomplish this

 

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If allocations are dynamic why wouldn't you be using malloc/free?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If it can be 1024, then allocate a static 1024 and be done with it. What do you save by making it smaller? Your program HAS to be written to tolerate at least 1024 in one case, so what would you do differently to effectively use the memory not being used within the smaller cases? 

 

Jim

 

Until Black Lives Matter, we do not have "All Lives Matter"!

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi Jim, I don't pretend to know everything out there but it was my fear that was the case. I was not sure if there was a way to utilize other regions of the chip.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm a little confused about the order of things. What is sending data *to* the atmega, and what is accepting data *from* the atmega?

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

USB sends to the mega328, a controller its getting data from the mega328.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

What "other regions" are there?

 

Here is the scenario that I see:

 

1) You have some version that uses 1024 byte buffer. So, you write your code in such a way to give that much buffer space.

 

2) Now, you have a second version of the same app that only needs a 32 byte buffer.

 

3) What are you going to change in that second version to effectively use the 992 bytes that you don't need for buffer space? Yes, there are situations where you might trade buffer space for something else, bot those are really not very common.

 

4) So, just leave that buffer at 1024 bytes for all versions and don't look back. The only reason to change it is IF you have one of those rare situations where you can grow something else to use up to 992 bytes when the buffer is only 32 bytes. And, remember, this is RAM, not FLASH.

 

Jim

 

 

Until Black Lives Matter, we do not have "All Lives Matter"!

 

 

Last Edited: Sun. May 20, 2018 - 12:11 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thx Jim, that's what I have now and as I said its the only way I see this working.

 

Issues.

 

1) I'm at %100 memory usage and its well know that code does not work well at that level.
2) I certainly can not go further so either a) development ends here or b) I use a smaller size and deal with life.

 

 

solutions

1) Deal with 512 as a max, limit the capabilities of the product.

2) Refine the code looking for better memory management (i.e wasted memory)

Last Edited: Sun. May 20, 2018 - 03:17 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The write way to often for EEprom (I'd burn that out in a matter of day). So

What kind of data are you talking about? And why mention the EEPROM, you are sending to the PC, so storing in EEPROM should have nothing to do with the transfer. 

 

My issue is that the atmega does not have much memory and I'm near %100. The data must be read in its full size. So I must read from usb in to an array first.

WHAT???!?

I don't thing you have a clear picture of what you are doing, or maybe need to state it once again.

 

 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You would have to find something else that doesn't need its memory, at the same time the USB code needs the 1024 bytes.  Dynamically allocating memory doesn't give you MORE memory, it just gives you the ability to redistribute it better (more dynamically.)

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

S_K_U_N_X wrote:
I'm cretin

Don't be so hard on yourself - we've all run out of SRAM at some time before.

 

A pinout compatible micro upgrade is probably the easiest fix.

 

If you cannot change the micro then you could add external SRAM. Our lords & masters make some: http://www.microchip.com/design-centers/memory/serial-sram-serial-nvsram.

 

Otherwise you'll could try the malloc / free route but you might still end up running out of SRAM. Bear fragmentation in mind; doing malloc(1024) is certain  to fail after some fragmentation occurs. One solution is to malloc small blocks only and deal with non-contiguous blocks in your code. Dealing with a fragmented struct would be a nightmare but an array not quite so bad. Your USB messages are already fragmented so they are easily handled.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Fragmentation occurs if you alloc/free varying size blocks and free in a different order to allocation. If you only have one block allocated at any time then even if the sizes vary it should not fragment the heap.

 

But as others have implied you can't invent RAM out of no where. Either you do have enough for the worst case of allocation or you don't. 

 

When designing this solution you should have considered that worst case when doing the memory budget and that would then go towards selecting the silicon.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Only you know how it's working, but here is what come into my mind.

100% RAM use: if you know the needed stack, there is no problems in using all the rest. (I don't know your way of coding and the amount of free flash but things like many small function calls (call each other), need a deep stack, here the use of inline functions will help).

I personally don't like malloc and free for this kind of programming, but an other way around it could be to make a union for some of the data, like 768 byte always in your buffer, and the last 256 are shared between other buffers, data etc. that's not needed while you use the "big" buffer. (or after the big buffer have been used can be recreated).

 

Are the 1024 bytes total random? or some bits always 0 or bytes 0 or special sequences are repeated etc. then perhaps make "model" in the the flash and then send tokens. (perhaps simple kind of zip).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm not clear on why you have to hold the whole thing in memory at once. usually things like this work by streaming data in. Say, have two 32-byte blocks, fill one of them while draining the other out to the other device, repeat until you've moved all the data.