Database in EEPROM, with ASM

Go To Last Post
105 posts / 0 new

Pages

Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi guys!

I'm currently facing the problem to create a flexible database within EEPROM memory that should not waste any space because there isn't much available. Basically, I need two tables that will be interlinked.

Table 1 / Modules, up to 4096 entries
-address (12Bit)
-modType (4 Bit)
-pkgSize (8 Bit)
-current Data (1-128 Byte, usually 2-5 Byte)

Table 2 / Module Groups, should be at least 100 entries
-internal ID (min. 8 Bit)
-module adresses (min. 10x = 120 Bit)
-preset 1 (all data for all member modules - much)
-preset 2 (~)
-preset 3 (~)

EEPROM space of any AVR won't be enough, so I'll add some I2C-EEPROMS to extend memory. I'll also have to add external SRAM I guess to hold all these values during operation. Much writing to EEPROM kills it too quickly. Values are only written back on power-down.

Now my question: how would you create this database in assembler?

I thought of creating fixed-size tables which - in worst-case - waste a lot of space. I thought of creating two more tables like a FAT pointing to the start of each entry, which will soon cause fragmentation and forces me to make the FAT bigger to find all the fragments OR again have "holes" that I can't use. I thought of creating fixed-size tables with different sizes of entries, some logic should then choose the right one for each entry.

But after all, there is still no good solution that comes without adding way too much external storage and SRAM and leaving most of it empty.

For the µC, I'd use an ATmega16 unless there are good reasons to use something else.

Background:
My bus system controller (the master module) reads and writes data from/to all connected modules (up to 4096), has to store it *somewhere* and has to know what to do when for example a button is pressed. The above is maybe 1/2 to 2/3 of the total database, the part I thought of yet ;-)

I just don't want to attach a constantly-running computer with Linux/MySQL etc. because that's total overkill and unneccessary.

Any ideas? I'm stuck...

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi Markus,

A couple of suggestions come to mind...
You could use a serial (or parallel) FRAM chip rather than the EEPROM. This functions pretty much the same as a serial EERPOM (SPI and I²C flavours avaliable) but is much faster and effectively doesnt wear out. It could be used as Non-volatile RAM. FRAM is also available with parallel interface and this may give better performance in your app than using serial.
There are also battery backed SRAM chips available (NV RAM).

Regarding the format of the database, I would suggest you use some simple delimiters for your data. There are even some nice ASCII codes that would help such as FS (File Separator), GS (Group Separator), RS (Record Separator) and US (Unit Separator). These combined with the fixed lenght data is all you need.

Cheers
Steve

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

FRAM sound good, thanks!

The format is a problem because I don't have fixed-size data records. So I'd either have to search in a linear way to find the record I need (which takes WAY too long) or I'd have to create a FAT and soon have fragmentation causing available memory to drop fast, which then forces me to do defragmentation etc - it gets nasty. And that's exactly what I don't want ;-)

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sparky23882 wrote:
FRAM sound good, thanks!


Some years ago I used NV-SRAMs. These were funny chips. They had an 32k-Byte RAM shadowed with a EEPROM / Flash can't remeber. There were two types available one that write the NV-Data automatically if the supply was lost and the other one - I used - need a sequence of READ operations to special addresses to perform a full update of the NV-area in aprox. 10ms. If you are interessted I could have a look for the chips and manufacturer.

Knut

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The chips aren't the problem here, I bet if I look for it I'll find a whole bunch of EEPROM, SRAM, FRAM, NVRAM etc. chips. I'll then (once I'm prepared = the theory works) look at my favourite electronics store here in Germany, they have most of the stuff I need.

But for now, I need to get the data structure somehow handle-able without the need for defragmentation.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
reads and writes data from/to all connected modules (up to 4096),
WOW, how much space will each module take up? I can roughly see at some 600 Kbytes to manage!! You may end up needing a proper FAT system in which case you may be well served using ...cough...splatter...gulp... the UNMENTIONABLE high level laguage which start with the 3rd letter of the alphabet....What am I saying?? Who wrote that??

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It's 'C' I heard him!
John recommended 'C'.
Traitor, you are going to get disbarred or something.
John

Resistance is futile…… You will be compiled!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hey John! I'm still stuck to the first letter and happy with it ;-) I also fear having to proceed to the next letters because of bad compilers, loss in speed and then eating up too many cycles and decreasing bus speed. That's really not what I want to do.

I can downsize the amount of modules for *most* uses to some 100 (the maximum 4096 will most definitely never be actually used) but as I want the bus to be flexible, I prefer having the possibilities to manage all modules that are possible.

I'll need something around 1-5MB of storage, non-volatile and with infinite writes. The easiest thing that came to my mind was using EEPROM, Flash or maybe a small SD card and buffer it all in SRAM. It's ugly and it's huge but this way I don't need a computer that drains way too much power.

Most modules (in common installations ALL modules) will need 5-8 Bytes of storage (address included). Most module groups will be below 10 modules and 12-21 Bytes in storage (own address, module addresses and module presets included).

So if I stay in these (too) tight bounds of 5 bytes (data) per module max and 10 modules per group max, limit the number of groups to (too low) 100, I'll need something like 100kB. I'd rather limit the number of modules per group to 100 and the number of groups to at least 500, which then sums up to something definitely over 1MB.

The big problem is bigger modules. My modules can (for now) handle up to 128 Bytes of data (256 nibbles with 4 bit each). Usually, not more than 2-5 are used but in theory...

If I'd limit the maximum number of modules to 100 and the maximum number of groups to something below 100 too, I'd be under 3kB and within integrated storage for at least some AVRs. But that's not what I created 12 bit addresses for.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If you end up using fixed length records, then you will not have to resort to...you know what.. each record will have say a 256 or 512 bytes space, then just use larger memory chips for bigger systems. Getting to the start of the record for each module will be a simple 16 bit multiply, an SPI device will be suitable.

Quote:
Traitor, you are going to get disbarred or something
The ASM secret society does not dis bar, it DIS MEMBERS...ouch :lol:

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I thought about fixed sizes and multiplying addresses, so memory chips with unlimited writing, enough size to fit 4096*256 bytes (= 1 MB) for modules and 512*32 bytes (= 16 kB) will be needed at least.

I found SRAM (many pins = parallel?) up to 512k x8 (which is 512kB, I suppose) and Flash-EPROM (parallel) up to 256k x8 (256kB I guess). Seems like I'll need to find another store shipping to Germany that sells bigger chips.

SRAM only needs a few µA to store data and the system shouldn't be powered-down at any time, so a capacitor should make SRAM safe and I can drop the EEPROM... but one power failure taking too much time and the whole setup is gone. Not a good idea. A small battery should provide the some-µA-current for quite some time. What do you think about this approach? Saves me half of the chips ;-)

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Personally I'm a big fan of Atmel AT45 series SPI connected Dataflash. It's as cheap as chips, you can get megabits of the stuff for half nothing. You can treat it as 256/264/512/528 bytes pages so I'd just pad your records out to page boundaries and avoid the "fragmentation" issue all together just using "wastage" to space things out on neatly divisble boundaries.

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

64 MBits (8 MBytes) actually do sound good, yes! But 10-15 EUR for that size don't. Still, Dataflash is the best option yet. I'll only need this on one module per installation (the main controller) and that'll definitely be pricier than the modules for driving a lamp or reading a light switch, so I guess that's fine.

A few more thoughts on the structure:

1)the module table doesn't need to be permanently stored. All it holds is module type and package size (which are both sent with each time the master talks to the module, to adapt to changes) and the current state (which initially will be 0 for all modules), so SRAM is enough here.

2)the module groups table will need (for example) 10 modules with at least 2 presets (= 115 bytes per group), so with 512 byte pages (for example) I'd get 4 module groups into one memory page without wasting *much* space. Make 20 modules per group (~256b) or 40 (~512b) and it still fits on one page.

3)I'll need a third table like "if button x is pressed, do y", but DataFlash offers enough space for that one to fit in, too.

Damn, this project is getting really ugly!

I'm thinking of selling different versions of main modules for different setups. One very basic one for max. 100 modules and a few groups, a medium one for larger installations for maybe 500 modules and ~100 groups and the ultimate one for 4096 modules. Might be a better solution than selling the ultimate to everyone.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
The ASM secret society does not dis bar, it DIS MEMBERS...ouch

No need to worry John, we will always RE MEMBER you...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

But 10-15 EUR for that size don't.

For a one-off, a little extra cost doesn't matter much.

Maybe I'm spoiled in the US. AT45, 8MB is US$7.30/qty. 1 at DigiKey, and less than $5/qty. 100. The shipping on any small order will dwarf that.

Lee

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
we will always RE MEMBER you...
I think I'll get out on a technicality, didn't actually say the..word..also I could claim temporary luna-C. The thought of being DIS MEMBERered or even dis ASSEMBLED scared the .HEX out of me! :lol:

Quote:
so a capacitor should make SRAM safe and I can drop the EEPROM... but one power failure taking too much time and the whole setup is gone.
Parallel, battery backed up ram can be tricky. The last time I used it...a long time ago in a workshop far far away, I used some MAXIM chip to make sure that the data remained unchanged during power down. Basically the CS pin MUST stay in the diselected mode during power down or the data will be trashed, the BOD is also very important. Serial stuff will be less likey to suffer from this as it has it's own safeguard.

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

DataFlash is serial, right? So this makes it easy to do a battery backup :-D

I'll use DataFlash with capacitor, backup battery and internally with fixed-size data records. Wastes a lot of space but hey, with 8MB I can waste some. I just hope reading and especially writing DataFlash doesn't eat up too many CPU cycles - that's the one thing I don't have enough of. Well, maybe I'll build a network on the main controller PCB with a few µCs instead of a single one (one for bus control, one for memory control and buffering, maybe more) and network them with an 8 bit parallel bus.

Edit: wait, DataFlash is Flash, not SRAM... no need for a battery! This keeps getting better...

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
This keeps getting better...
Except that it wears out :( unlike SRAM. Have a good look at serial FRAM (as Steve mentions above), I haven't used it yet (the useless ex distributors in OZ never got me a sample, will need to BUY one :( ) but I have heard some good reports.

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The only online-orderable FRAM I found in Germany is 256kBit (too small) and 40-50 EUR, based on quantity. Not a bargain at all. I'd need at least 4 of these, which sums up to something close to 200 EUR and that's way too expensive. I'll stick to DataFlash, hope reading is fast and use another µC to buffer reading and writing.

The good thing about the bus is it always accesses data in the same order. Modules 1 through 4096 in a sequence, then start over. Same applies to module groups. So the memory controller can (with its internal small SRAM) buffer all reads and writes before they occur, then push the data into the main µC with a few cycles and it's all good.

And for wearing-out-protection, I'll add a lot of (cheap external) SRAM to the memory controller and only write to DataFlash on power-down.

So I'm currently at two µCs, maybe a third one handling much external SRAM and acting like an SRAM-based DataFlash? ;-)

And I thought I'd be creating *one* bus and network. The controller/master module gets some µCs networked together for managing storage and the bus/slave modules have an extension bus too...

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sounds like you need a bigger processor! Adding AVRs with addons sounds like a hard way to do things. Something like the ARM7s or ARM9/AVR32 running Linux might simplify things a little. Whilst it sounds like an increase in complexity (it is), you move complexity from your code onto code written by others (Linux). That way you don't need to know about the intricacies of storing the data, you just read and write data, Linux takes care of the rest.

As for your 'database' - a database can be simply ordered data like yours up to and beyond a financial system for a large business etc. The choice of solutions is equally as broad.

Writing your database in assembler conceptually is no different to writing in 'c'. Using a higher level language just removes you from writing code to do low-level stuff like calculating indexes, multiplies etc.

Distributed systems like you're describing can get quite complex with the flow of data. This is something you want to consider up front before you design yourself into a big corner!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I thought about linux boards and had a look at their specs, but they are way too expensive and most of them draw too much power.

I did a little more planning and my solution (for now) is a network of 4 µCs (all AVR), each with its own tasks.
1) communication with the bus (just send and receive data)
2) detect changed inputs and change matching outputs (in memory)
3) handle DataFlash and SRAM memory
4) interface for external (e.g. webserver) interface

Everything is linked to a memory bus, each µC has its own interrupt and ack lines (so the memory controller knows who wants to read/write and can then tell the µCs to start) and the µCs define the clock signal for memory access (so own interrupt routines etc. won't be a problem).

Distributing each task to its own µC makes the system a lot more powerful without drawing a lot more power, AVRs don't really need much to run. And because the network is only based on the memory bus on the PCB, it's not really complicated.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

What do you call expensive? The linux boards I use are <$100Euro and draw less than 1W of power! Cost is relative though. - your AVR board may cost less, but how much has it cost to design? For hobby stuff, the time is usually immaterial.

A distributed processor design can either fall together easily or be like pulling teeth. It all comes down to how the data needs to be distributed. Some problems solve easily, other are more difficult. You'll get some experience here I'd expect. I learnt some fundamental lessons many years ago!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The 4 µCs cost 10-15 EUR, SRAM and DataFlash are in this range too (each). So I'll be going with 30-45 EUR (plus some connectors) and save 50% on the linux board price.

DataFlash 64MBit max. 0.09W
4x ATmega16 max. 0.3W
SRAM somewhere around 0.1W
Sums up to ~0.5W and a little for the signals between µCs.

So it's both cheaper (if development isn't considered) and uses less power. The better choice. Development costs are another thing with this project. I started the project officially as a project at university, so it's about education and getting good grades for it. On the other hand, I will be selling it and have already 4 homes that will be wired using this system.

The end price for the main module will be at 100-300 EUR, the components price around 50 EUR, the difference are development costs. At least a part of it.

Last but not least, I'm learning something from it, get more experience and have fun. That's priceless.

So I guess the linux boards aren't the best way in my case. Definitely easier to do most, but not the best way here.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well you don't have to go as far as an ARM9 and Linux. You can get about 60MIPS from a SAM7. Here's a couple of dev boards (they're actually manufactured by Olimex) which have SD/MMC card slots so you can spend another £5 and plug in a 1GB memory card:

http://elmicro.com/de/sam7proto....

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I need about 1MB. A memory card with 1GB is a little oversized for that purpose. And SD cards wear out. So I'd still need the SRAM to buffer it all. In addition to that: if I'd use an SD card, I'd want to be able to read/change values with a computer, so I'd have to use at least FAT16 and test files. Oh, and 54 EUR for the smallest board is way too much.

60 MIPS aren't needed here, either. The whole system runs at 8MHz (each module does) and the bus runs at some 100 kHz. A memory controller with 16 MHz (ATmega16 with external oscillator) will be enough to do the memory management without slowing down the bus. Maybe I'll choose another flavor of ATmega, something with up to 20MHz. I'll most definitely not need more than that.

It's not about calculating scientific stuff, it's about switching power and dimming lights after all. ;-)

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've built lighting systems for buildings that have less than 1MB total of ram! I can't quite understand why you need so much memory for a lighting control system (or cpus - I'd use only 1 for the main controller)? Even if you were logging the lighting changes, a flash card or chip would last years without wearout.

Your comparison is a little flawed as you're comparing chips vs boards. On a chip/chip basis you might find the costs might swing the other way. Integration is cheap. Also, keep in mind that Cliff has extensive experience in designing mass market products and myself have extensive experience in designing architectural lighting control and security systems, so we're not trying to mislead you!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If that system would do only lighting, I'd have it up and running since weeks. It will melt all lights, wall power outlets, light switches, light barriers, pir sensors, light and temperature sensors and motors for sunblinds, heater control, garage door and servos to steer lights etc. into one big system.

The key goals are:
-make it cheap
-make it ultimate (everything you can imagine - it can be done)
-make it easy (anyone should be able to reprogram it, change settings etc.)
-make it on my own (i.e. not buying readily-built dmx lights or similar things - build it from scratch!)

About memory:
Each and every light will be dimmable and have RGB components, so I need 24 Bit per light source. Light switches, wall power outlets etc. only need one bit but I can't always get 8 of them connected to a module, so here data is lost. Servos need at least 8 bit each, sensors 8-16 bit each and if you sum all of this up, the standard module will need at least 5 bytes of storage. For 4096 modules, this sums up to 20kB.

I also need groups (for example "living room light", "tv light", "kitchen light") and in each group, the address and some preset values have to be stored for each module. with 5 bytes per module, 3 presets and 2 bytes address (there are no half bytes!), it's 17 bytes for one module in a group. Let's have for example only 10 modules in a group and we have 170 bytes. With 100 groups (absolute minimum) this sums up to 17kB.

Now where's the rest? Well, a module isn't designed to manage only 5 bytes, they can (for now) manage up to 128 bytes of data each. Later, I'll extend that to 512 bytes. That's the maximum I need. Doing the calculations above with that size, you get well over 1MB.

Sure, I could only use the 512 bytes where I actually need them (a module that needs them is connected) but then I'd have memory fragmentation and I'd need an allocation table for all modules because of the different sizes in memory. No more "multiply and jump".

As DataFlash and SRAM are cheap enough for one module per home (the master), I chose to use them instead of writing a complete file system with defrag to be able to use the whole system and not just a part of it.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

But you only get allocation/fragmentation problems if you store variable sized control blocks. Make all of them as big as your biggest and you won't get fragmentation.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

My biggest block is 512 Bytes. Multiply with 4096 modules and you get 2MB of storage. Only modules.

For only 100 module groups with let's say 3 presets per module and 10 modules per group, you get 1.5MB.

That's how I got over 1MB. Lower memory sizes are only possible WITH fragmentation. That's what I said ;-)

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So why beat yourself up trying to squeeze a quart into a pint pot? - just throw in more memory - it's not like it's expensive (I generally over budget by x2 or sometimes x4 in a design and usually end up using it all in the end!)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well, I always tend to use as little as possible of everything I need (like CPU cycles, memory, power, ...), so the product can be tiny and unsuspicious and I don't need stuff like SD cards, active cooling, special (big) housings etc. That's also the reason why I chose ASM in the first place and not C. I have full control over what the CPU does with ASM, with C I don't.

But hey: seeing the need of an allocation table and a defrag routine made me choose to use big memory to avoid these. The code needs to be fast after all. I'm already way over specification (12 clock cycles for one bus clock is maximum, I currently need >20) and don't want to make this even worse.

Edit: I will be doing something with allocation tables and fragmentation later, just for fun, but not in this product.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sparky23882 wrote:
That's also the reason why I chose ASM in the first place and not C. I have full control over what the CPU does with ASM, with C I don't.

Oh boy is that ever a contentious commment on here?!?

My own personal opinion on that is that it's utter, utter garbage but then that is only one person's PERSONAL opinion!

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sure, ASM is ugly. I totally agree! And it's not easy programming like a higher language. But the free compilers you get for C/AVR produce too bulky code. As clock cycles are very important in this project, I chose ASM because I do get more control on what the µC actually does.

Believe me, I gave C more than one try before I finally accepted it just didn't produce small code for some constructs. And hey: ASM is fun once you got that way of thinking stuffed in between your ears ;-) I wouldn't change back. Okay, maybe for FAT32, LAN or something equal... but I'd try it with ASM.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sparky23882 wrote:
But the free compilers you get for C/AVR produce too bulky code. As clock cycles are very important in this project, I chose ASM because I do get more control on what the µC actually does.

Boy are you ever digging a deeper hole for yourself. None of that is actually true you know?

I generally don't take part in the Asm vs C wars myself as I can see the merits of both (more than the first 10 years of my professional programming life were nothing but Asm - mainly Z80) but as I've said before C is just like a "grown up macro assembler". It's no real different to Asm but allows you to be more productive (US DOD figured a programmer could produce 10 lines of working code per day IN ANY LANGUAGE. You get a lot more happening for your ten lines of C than you do for your ten lines of Asm).

People say they want to avoid C because "they have no control" and "it generates stuff I never asked for". Well OK, I'll give you that there's always about a 60 opcode overhead at the start of the program where it puts the CPU in a known state, copies the .data initialisers and wipes the .bss and that's unavoidable (unless you tell the C compiler not to do it of course!). But the other "hidden overhead" I think that people aren't keen on is the stack frame adjustment on entry and exit of sub routines to make room on the stack for "local variables". But nothing says you have to use locals. If you think about an Asm program then all your variables are either in registers or fixed locations in RAM. Well that's easy in C - just stick to globals. When you do this the C really is nothing more than a grown up macro assembler.

All this just IMHO!

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If your routine for example uses 20 cycles and you need it to run in 17 cycles because you don't have a bigger timeframe for it... how do you do that in C? How do you find out how many cycles it takes and how do you fix that in pure C?

For _ME_, C just doesn't work for µC programming. It produces useable code but the code uses a little (or a little more, based on the C compiler used) more cycles than it should/could. That's okay for most uses but for this project it's not, as the µC will be close to 100% CPU usage.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

A competent C programmer using C for embedded will be looking as much at the generated assembler as they do at the actual source code for anything that's time critical. Equally, when it comes to debugging, they'll probably be looking at the interleaved C/asm view more often than not.

If your C compiler is generating more cycles than it needs to do the job the chances are you are using the wrong C compiler!

Cliff

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

For _ME_, C just doesn't work for µC programming. It produces usable code but the code uses a little (or a little more, based on the C compiler used) more cycles than it should/could.

Don't start that crap again. This was gone through in a quite recent thread: "You CAN'T do this in C!" said the bigots. When proven wrong, "It takes too much time/space!" was the cry. When proven wrong suddenly it became quite quiet.

Yes, indeed, if your algorithm can be significantly improved using rotate or the carry flag or 24-bit arithmetic, then by all means go for it. But to make statements as you are doing is pure blowing smoke harrumph harrumph.

I'll match your work on your supposed 20-cycle routine anytime. Put it up here. And also tell how long it took to get it down to 17.

Quote:

as the µC will be close to 100% CPU usage.

Now if your CPU has 50% usage what is it doing the rest of the time? (And aren't those op codes "usage"?)

Quote:

I have full control over what the CPU does with ASM, with C I don't.

A wide, wide, generalization. Please give a short list to those pieces of the CPU over which I have no control if I write in C.

[Quick answer: None.]

Quote:

I finally accepted it just didn't produce small code for some constructs.

Another overgeneralization. Simply untrue for all but the most intimate of algorithms. Again, let's have a list of these "some constructs". Bet you a cold one the "list" will be more harrumphing.
Quote:
But the free compilers you get for C/AVR produce too bulky code.

There is one free compiler that I believe that you are referring to. It isn't my problem nor C's problem that you get bulky when you use that infinite-value tool--you get exactly what you pay for. Does code get bulky with the free versions of other compilers? If your point is "I want to spend the absolute least on tools, and this is the best toolchain that I can come up with." then I can probably/perhaps agree with you.

Quote:
As clock cycles are very important in this project, I chose ASM because I do get more control on what the µC actually does.
Yet again, let's see a list of these situations to help us be persuaded of the True Way. And yet again, I will predict that I will not be presented with a list or even a good example, but rather with either silence or more generalizations.

Bring it on; your case to prove your point is certainly not going to be this "database driver" app.

Lee

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

Last Edited: Sun. Aug 19, 2007 - 12:24 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
US DOD figured a programmer could produce 10 lines of working code per day IN ANY LANGUAGE.
WOW..and they are in charge of defending a country??? :lol: ...slow typers, hope thay are a little faster at shooting... :lol:

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
US DOD figured a programmer could produce 10 lines of working code per day IN ANY LANGUAGE.
Goverment employess doing their best? :lol:
Great,
John

Resistance is futile…… You will be compiled!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I did quietly suggest to the OP that there are some highly skilled and experienced people here and that his comparisons were not valid. But, we all need to create a hole for ourselves then dig ourselves out. That's real experience. I wish I had someone to give me sage comments with a few of them.

A bit of my business is re-targetting older designs to new micros. Predominantly these designs were written in assembler and poorly structured assembler at that. It's quite a bit of work to unravel tightly written asm code into 'c' so it can be ported to just about any micro. Once done, adding features etc become a lot easier and less error prone.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:
When proven wrong suddenly it became quite quiet.
What was proved wrong? Amazingly ony 2 C champions came forth to try and defend the undefensible out of 80,000+ people, perhaps the enlightened majority uses ASM. It was proved that 100% of ASM programmers would produce small and fast and therefore better code. Only 50% of C programmers could come close, but not better, to the ASM code. This was only after many twist and turns and hair pulling I suspect, and of course having ASM code to copy from.

Dit it also prove the superiority of CAVR vs ICC? Could the ICC programmer improve his code if wanting to try harder?

I would suspect that the majority of C programmers, when faced with a similar task, would simply get a faster and larger chip therefore adding to the power consumption (more greenhouse gas) and more ladfill. C programmers are adding to the destruction of out planet, just like those darned flautulent sheep and cows... :lol: ...I should have been a defence lawyer!!

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Great speach, John...
I'll vote on next secret ASM society meeting that we give your limbs back. :lol:

The last C vs ASM thread convinced me that the major factor when it comes to produce efficient code is the skills of the person writing the stuff. To produce good code you need to know the architecture of the chip.

As Lee said you need to know/check what ASM is produced from C code and know how to use your C-compiler to tweak it to produce what you want. Perhaps there are differences between compilers how well this work.
It feels rather comforting that the skill/experience of the person doing the job still is what makes the difference.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

js wrote:
Quote:
US DOD figured a programmer could produce 10 lines of working code per day IN ANY LANGUAGE.
WOW..and they are in charge of defending a country??? :lol: ...slow typers, hope thay are a little faster at shooting... :lol:

No that's a true figure for ten lines of fully debugged and working code per day. Sure you can churn out several hundred lines in a day but you'll spend several days debugging/testing it so I tend to agree with that 10 line figure.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

What was proved wrong?

C'mon, John. You and all the other guys have done it over and over: the generalizations like sparky, and the "you can't do this..." claims.

And you know as well as I do that it just ain't ture for "general" work: getting the job done and moving on. Yeah, yeah, any program can be made one cell shorter, as one of my first profs used to opine. Well over 90% of the time that just doesn't matter.

If it really matters to you that your program has 80% idle time vs. 78% idle time, then go for it.

If you claim that a critical section cannot, never, no way be written efficiently in C, then you are full of crap.

Quote:

It was proved that 100% of ASM programmers would produce small and fast and therefore better code.

Crap and bull and any other excrement euphemisms.

Quote:

... out of 80,000+ people ...

Any ridiculous claim -- e.g., "the earth is flat" -- will have the 80000 dismissing the rhetoric out of hand and ignoring it, and 2 that will point out that the Emperor has no clothes.

Lee

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi again!

clawson:

Quote:
A competent C programmer using C for embedded will be looking as much at the generated assembler as they do at the actual source code for anything that's time critical. Equally, when it comes to debugging, they'll probably be looking at the interleaved C/asm view more often than not.

lennart:

Quote:
you need to know/check what ASM is produced from C code and know how to use your C-compiler to tweak it to produce what you want

me:

Quote:
I chose ASM because I do get more control on what the µC actually does.

Explanation: Sure, you can write the same (ASM) code in both C and ASM. But to get your program fast (i.e. use as little cycles as possible), you will, even with C, have to work with ASM. That's what you said.

So why should I use C for programming if I still have to analyze the ASM generated to make things work like they are supposed to? Isn't it better to directly do it in ASM because that's where you'll get either way?

The µC doesn't know C. It knows a handful of commands you can write in ASM. So IMO it makes most sense to write the program using these commands the µC actually knows instead of using a higher language that needs to be translated and then tweaked (because of bad translation) to get the same (ASM) result you'd directly get using ASM in the first place.

clawson:

Quote:
If your C compiler is generating more cycles than it needs to do the job the chances are you are using the wrong C compiler!

theusch:

Quote:
It isn't my problem nor C's problem that you get bulky when you use that infinite-value tool--you get exactly what you pay for.

If I can choose between free ASM and good code, a free C compiler and bad code or an expensive C compiler and good code again, I'll definitely choose ASM because I won't be spending extra money for a result I can also get for free. Where is the extra value I pay for? C and ASM are just different languages. Tools to use. Totally un-important. The only thing that matters is the result and the price I'll have to charge the customer in order to pay my bills and still earn something. Less money spent equals more money earned. Simple calculation.

theusch:

Quote:
Now if your CPU has 50% usage what is it doing the rest of the time?

There is a sleep command initiating one of a few low power consumption modes. And the near 100% of usage I was refering to means almost every single clock cycle available being used to do actual work (and not loops or sleeping).

Conclusion: There are a few ways to code for µCs.

1) Do it in C to get portable code and spend quite some time at the generated ASM (and optimization of the C code to produce the ASM result you want).

2) Do it in C and use an expensive compiler if you can afford paying a lot of money for it.

3) Do it in ASM and directly have the resulting ASM code the µC understands. Fast, but non-portable (at least not with a click) code.

There are some tasks that are better done in C, like building a webserver, creating a database, ... just because it's really a lot of ASM code and using C libraries makes it a lot easier to handle all of it. But that either requires an expensive compiler, a too big µC to do the extra work or to spend a lot of time optmizing your C code to get the ASM code you want to get.

IMHO, ASM is better for small code snippets (because you'll optimize ASM in the end either way, as stated above) and C is better for large libraries and complex stuff because it's easier to handle.

All satisfied now? Or are there still people pissed off because they have -after years of experience- all the tweaks in their head and directly write the C code the big majority would get after spending a lot of time optimizing the ASM result in mixed source view?

Sure there are maybe 5 people here that directly write fast code in C. But I bet there are a lot more that can directly write fast code in ASM, at least for small programs. (both without tweaking, kind of "first try to run it")

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think the problem is that the majority of apps written in C don't need any special tweaking to obtain the desired results which would mean that the programmer would have no need to keep monitoring the code produced. So then a time/space critical app does present itself to the programmer, he/she may not be as familiar as he/she should be.

Now the double edged blade here is that it could really cause more headaches coding a critical section using predictive coding due to the fact that a compiler upgrade could always change the code it produces. So IMHO I think just coding that section in asm makes the most since. Then there is no need for trial and error to get the desired output and any compiler upgrades are less likely to cause problems that may not always be very obvious.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Something more:

theusch wrote:
If it really matters to you that your program has 80% idle time vs. 78% idle time, then go for it.

In this project, I am very close to 100% CPU occupation, so getting a few percent better means staying within specification and not slowing down the whole thing. Getting over 100% (I know, that's not possible - the program then only takes longer than it should and timings aren't right anymore) isn't an option for me, nor is overclocking the µC. A bigger µC is also not an option because of power consumption. A few mA in some 100 or even some 1000 modules make up quite some current. Calculate it over a year and you get some actual money saved.

Like I said and now am proved right by a few of you: if you need to get this one or two cycles squeezed out, if you actually need this 1% increase in speed, _then_ choose ASM. That's what I need and what I did.

I didn't say anything like "C programmers have a small dick" or "ASM programmers have up to 5% bigger brains". So why are (some of) you so extremely pissed off? Both languages have their right to exist, there are tasks for each of them where it fits better. Give each other a hand and celebrate the results instead of argueing who did the exact same thing using one or the other language.

All I know that FOR ME (please re-read this!) and for THIS PROJECT, assembler fits better. Another programmer would do it in C, Basic or maybe brainfuck or trifunge (I'm not sure if I spelled the last one right). The way just doesn't matter as long as you get the result you want with the amount of work you expected.

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The example you wanted:
4 variables, an if-elseif-else construct and the main loop. The C compiler I use because it's free stores the variables -in any optimizer setting- in SRAM instead of registers which would be faster for this task (no need to read/write SRAM). Should I now research how to define my variables so with this compiler and for this µC, they are stored in registers? I already have a good result in ASM...

My results for only the main loop, no variable declaration and initialization (all in disassemble view):
C/O0: 24 lines
C/O1: 24 lines
C/O2: 22 lines
C/O3: 22 lines
C/Os: 24 lines
ASM: 11 lines
ASM using SRAM to store the variables: 17 lines

Now please explain to me (and don't tell me something like "use a better compiler" or "code better", give a good explanation!) how I should have written the C code to make it produce less than 11 lines of ASM code. Because getting the same result with more work doesn't prove C to be better for me. I'll have to get better results to think C is actually better for this task (writing this example construct).

Sample code is attached in text file. With disassemble outputs. Enjoy! And yes, please prove me wrong. Prove you can write C code that compiles to something better than the ASM version I have without using much more lines of code in C... using the WinAVR compiler ;-)

I actually do want to learn something. I just didn't find a proof yet that convinced me writing good code for AVRs is easier in C than it is in ASM. Convince me!

P.S: I didn't think a lot for C nor for ASM in this example. I'm sure both can still be optimized.

Attachment(s): 

Markus

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Sparky

Need more info.

What do you want to DO with the database? Do you want to

Add records, delete records, amend records? When you go to fetch a record, what are your search criteria? What field or fields are you searching on? Any other features you need when making use of the data?

What are you speed requirements for fetching a record?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Just a minute, you've made the variables 'volatile'. Of course they aren't going to optimised into register only usage at any optimisation setting. The 'volatile' modifier says exactly "do NOT optimise access to the following variable".

Maybe you just need to get yourself a decent book on learning C?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The C discussion is off-topic, the question is how to structure a 2 table database, not what language or memory chip to use.

Dusty

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Since OP is the one to take it "off-topic", why not?

Pages