SOLVED - Need help / ideas how to pack bits

Go To Last Post
40 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi all,

 

I'm trying to convert 8 bytes of 5 bits each into 5 bytes of 8 bits each (this is to write a custom character bitmap into a VFD (vacuum fluorescent display) similar to what the LiquidCrystal library does, but the bit pattern layout is different (5 bytes of 8 bits each, with 5+3 bits packed into each byte).

 

I have it working with the following code:

 

uint8_t x, y;
uint64_t data;

x = 8;
data = 0;

while (x--) {
    for (y = 0; y < 5; y++) {
        (data <<= 1);
        if (charmap[x] & (1 << y)) {
            data |= 1;
        }
    }
}

for (x = 0; x < 5; x++) {
    _writePort (data);
    data >>= 8;
}

 

...unfortunately, because the total bit count is 40 bits, I need to use a uint64_t variable (data) to do this.  I want to get away from the 64 bit var and hopefully do it a different way.

 

To help clarify what I'm trying to do, I have this 8 byte array of 5 bits each:

 

const uint8_t big_f[] = {
    0b11111, // # # # # #
    0b11111, // # # # # #
    0b11000, // # #
    0b11110, // # # # #
    0b11110, // # # # #
    0b11000, // # #
    0b11000, // # #
    0b11000, // # #
};

 

This needs to be converted into:

 

// 11111111 10001111 11110111 11000110 00011000
//    FF       8F       F7       C6      18

 

This pattern is made up from this:

 

const uint8_t big_f[] = {
    0b11111, // part of F, F
    0b11111, // part of F, part of F
    0b11000, // part of 8, part of F
    0b11110, // part of 7, part of 8
    0b11110, // part of 6, F
    0b11000, // part of C, part of 6
    0b11000, // part of 8, part of C
    0b11000, // 1, part of 8
};

 

Note that it starts from the end and works up to the beginning!

 

Is there a simpler, easier way to do this (specifically, not using the uint64_t)?

 

Thanks... and if you need any more info or clarification, please let me know.

This topic has a solution.

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Tue. Jul 17, 2018 - 04:39 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Is the new, converted, bitmap stored in the AVR's flash and used directly? Can you not do the conversion just once, on a PC, and then use the new data as is?

#1 This forum helps those that help themselves

#2 All grounds are not created equal

#3 How have you proved that your chip is running at xxMHz?

#4 "If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand." - Heater's ex-boss

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

One word:  bitfields.

 

typedef struct {
  union {
    struct {
      unsigned a : 5;
      unsigned b : 5;
      unsigned c : 5;
      unsigned d : 5;
      unsigned e : 5;
      unsigned f : 5;
      unsigned g : 5;
      unsigned h : 5;
    };
    uint8_t foo[5];
  };
} bar_t;
.
.
.
bar_t bar;
.
.
.
  bar.h = 0b11111;
  bar.g = 0b11111;
  bar.f = 0b11000;
  bar.e = 0b11110;
  bar.d = 0b11110;
  bar.c = 0b11000;
  bar.b = 0b11000;
  bar.a = 0b11000;
.
.
.
for (x = 0; x < 5; x++) {
    _writePort (bar.foo[x]);
}

 

OK, three more words.  Anonymous structs and unions.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Last Edited: Fri. Jul 13, 2018 - 01:14 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Brian Fairchild wrote:
Can you not do the conversion just once, on a PC, and then use the new data as is?

+1

 

joeymorin wrote:
One word:  bitfields.

But, if you do follow Brian's suggestion, remember that bitfields are highly implementation-dependant - ie, non-portable...

 

EDIT

 

See, for examples: https://www.avrfreaks.net/commen...

 

 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
Last Edited: Fri. Jul 13, 2018 - 07:00 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

easier way to do this (specifically, not using the uint64_t)?

You shouldn't ever need to deal with more than 16bits at a time...

 

Note that it starts from the end and works up to the beginning!

Ok, I don't understand.  You have bit-reversal going on too?

Can you color-code the 8bit version, or use patterns in your 5bit array that don't repeat?

 

 

Is this right?

// 11111111 10001111 11110111 11000110 00011000
//    FF       8F       F7       C6      18

const uint8_t big_f[] = {
    0b11111, // part of F, F
    0b11111, // part of F, part of F
    0b11000, // part of 8, part of F
    0b11110, // part of 7, part of 8
    0b11110, // part of 6, F
    0b11000, // part of C, part of 6
    0b11000, // part of 8, part of C
    0b11000, // 1, part of 8
};

It'd probably be easier if you got the (SPI?) hardware to do the bit-reversal part.  Is that possible?

 

Are you looking for fast, or small?

 

Last Edited: Fri. Jul 13, 2018 - 08:03 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

I wrote something just like what you're doing back in 1996/97. I had a dual area VFD with a 16 (5x7) character line area and a 7-segment area for readings.

The bits were clocked out over SPI giving rise to an 8-bit packing requirement you also have.

 

I used a Hitachi H8/325 which was an extremely popular micro-controller at the time, and wrote the code in assembly. My characters were stored as bitmaps of 5-bytes in column order but had to be clocked out to the display as a row of 80 pixels. I'm fairly certain I used a loop here because all I had to do was step a byte to retrieve the next pixel.

 

That probably doesn't help you much - perhaps you could re-organise your data model to make this job easier; but I would be tempted to write some table driven code where there is a table line for each character gives a shift; mask; & array offset destination to OR into.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Brian's suggestion,

Joey's ;-)

 

remember that bitfields are highly implementation-dependant - ie, non-portable...

Portability is not always paramount.  And where it is, there's usually a way to handle it.  In all cases, documentation is key.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

I wrote:
Brian's suggestion

joeymorin wrote:
Joey's ;-)

No: Brian's suggestion - the one that I quoted about doing the conversion on a PC.

 

Portability is not always paramount

But it becomes important when you're doing one part of the job on one platform, and another part on a different platform.

 

And where it is, there's usually a way to handle it

Of course. I never said otherwise - just called attention to the fact, and that it would have to be considered.

 

 

Quote:
In all cases, documentation is key.

Absolutely.

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
Last Edited: Fri. Jul 13, 2018 - 01:37 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

No: Brian's suggestion - the one that I quoted about doing the conversion on a PC.

Ah yes I see where you're going.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If this is constant data, do it on a PC.

 

Otherwise, get rid of some looping.

Treat it as 5 separate problems.

Two formulae will have three variable inputs,

the others two.

"Demons after money.
Whatever happened to the still beating heart of a virgin?
No one has any standards anymore." -- Giles

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Brian Fairchild wrote:

Is the new, converted, bitmap stored in the AVR's flash and used directly? Can you not do the conversion just once, on a PC, and then use the new data as is?

 

Unfortunately, no.  This is a "createChar()" function for the VFD, so the bit pattern could be anything (same syntax as found in the LiquidCrystal library).

 

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Fri. Jul 13, 2018 - 10:17 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OK, maybe this diagram will make more sense than what I posted yesterday.  This is how the eight bytes of 5 bits each get "packed" into five bytes of 8 bits each:

 

 

Attachment(s): 

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Fri. Jul 13, 2018 - 10:14 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

This is an interesting problem, I'll think about it tomorrow. In my opinion the original solution is elegant as an algorithm, but when compiled to assembly will probably be big and slow.

 

I think I would start by building a lookup table to do the bit reversing part (5 bits means a 32-entry table). Then doing the merger will be the real challenge.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Krupski wrote:

Brian Fairchild wrote:

Is the new, converted, bitmap stored in the AVR's flash and used directly? Can you not do the conversion just once, on a PC, and then use the new data as is?

 

Unfortunately, no.  This is a "createChar()" function for the VFD, so the bit pattern could be anything (same syntax as found in the LiquidCrystal library).

 


Where is the data coming from?  Is it fixed in the program, or is it literally coming in over some wire?  I would think that "createChar()" would be called with data fixed at compile time.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

bitfields aren't going to reverse the bit order for you...  (although, it looks like avr-gcc does a pretty impressive job of dealing with 5bit bitfields.)

 

This will compile to relatively little code (I'm somewhat impressed by the compiler, actually)

It should work for any input byte size less than 8, although I think it would need some additional work if the output doesn't turn out to be an even number of bytes.

It should also be pretty easy to modify for different bit-ordering.

#define INPUTSIZE 5
#define NINPUTBYTES 8

uint8_t data_in[NINPUTBYTES];
uint8_t data_out[5]; // 5 8-bit outputs, LSB first

void packbytes() {
    uint8_t outp = 0;  // Output byte ptr
    uint8_t inb,       /* Input Byte pointer */
        outb,          /* Currently assmebled output byte */
        outc,          /* Count of bits in current out byte */
        inp,           /* input byte poibter */
        bitcount;      /* Count of bits in each input */

    outb = 0;
    outc = 0;

    for (inp=0; inp < NINPUTBYTES; inp++) {
        inb = data_in[inp];
        for (bitcount = 0; bitcount < INPUTSIZE; bitcount++) {
            if (inb & (1<<(INPUTSIZE-1))) {   // current input bit
                outb |= 0x80;    // set this bit
            }
            inb <<= 1;         // new input bit
            outc += 1;
            if (outc >= 8) {  // if we've collected a full byte
                data_out[outp++] = outb;  // output it/
                outb = 0;     // reset byte
                outc -= 8;    // reset count
            } else {
                outb >>= 1;   // shift output byte (note: opposite direction)
            }
        }
    }
}

 

Attached is the version that has testing and debugging added, and runs on desktops...

 

Attachment(s): 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Krupski wrote:
OK, maybe this diagram will make more sense

Would help to embed it in the post - where we can actually see it - like this:

 

 

See Tip #1

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Krupski wrote:

Is there a simpler, easier way to do this (specifically, not using the uint64_t)?

 

Well, any one output byte is only ever made up from three input 5-bit fields so in theory you don't need the uint64_t. At most a uint16_t will do it. But...

 

In my head I see this as two shift registers. A 5-bit parallel-in serial-out unit which gets loaded every 5 clocks with the input value, whose output is taken from the leftmost bit and who shifts left. An 8-bit serial-in parallel-out unit which takes in the output of the 5-bit unit, shifts left, and is parallel read every 8 clocks.

#1 This forum helps those that help themselves

#2 All grounds are not created equal

#3 How have you proved that your chip is running at xxMHz?

#4 "If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand." - Heater's ex-boss

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

In my head I see this as two shift registers. A 5-bit parallel-in serial-out unit which gets loaded every 5 clocks with the input value, whose output is taken from the leftmost bit and who shifts left. An 8-bit serial-in parallel-out unit which takes in the output of the 5-bit unit, shifts left, and is parallel read every 8 clocks.

 That's essentially what I implemented, except the 8bit register shifts right to implement the bitswap.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Most graphics libraries store fonts in Flash.    After all,  you are unlikely to alter a standard letter at runtime.

So I would guess that your fonts are currently defined in uint8_t letter[8] format and you want uint8_t letter[5] format e.g.

// Standard ASCII 5x7 font

static const unsigned char font[] PROGMEM = {
	0x00, 0x00, 0x00, 0x00, 0x00, // space
	0x3E, 0x5B, 0x4F, 0x5B, 0x3E, // !
	0x3E, 0x6B, 0x4F, 0x6B, 0x3E, // "
	0x1C, 0x3E, 0x7C, 0x3E, 0x1C, // #
	0x18, 0x3C, 0x7E, 0x3C, 0x18,
    ...
};

You can either convert your letter[8] format to letter[5] format on your PC or with an Arduino sketch.

 

My example shows a big one-dimensional uint8_t array

Since you are actually dealing with a two-dimensional array,  a better definition would be:

static const unsigned char font[][5] PROGMEM = {
	{ 0x00, 0x00, 0x00, 0x00, 0x00, }, // space
	{ 0x3E, 0x5B, 0x4F, 0x5B, 0x3E, }, // !
	{ 0x3E, 0x6B, 0x4F, 0x6B, 0x3E, }, // "
	{ 0x1C, 0x3E, 0x7C, 0x3E, 0x1C, }, // #
	{ 0x18, 0x3C, 0x7E, 0x3C, 0x18, },
    ...
};

I am not sure what the fuss is about.   A 96 char font would use 768 bytes as a uint8_t[8] and 480 as uint8_t[5].

Yes the execution time would be faster with a "native" format.

But I doubt if 288  byte saving is significant in a mega32 project.

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I must now admit to being a little confused now - I thought you were writing code to refresh/multiplex a VFD. Instead you seem to be re-arranging bitmaps for user defined characters. for an unknown controller. Is this simply a transposition of bitmap rows to columns ?.

 

What micro is this ? Does it have integrated VFD drivers ?

 

Last Edited: Sat. Jul 14, 2018 - 03:50 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

awneil wrote:

Krupski wrote:

OK, maybe this diagram will make more sense

 

Would help to embed it in the post - where we can actually see it - like this:

 

See Tip #1

 

Sorry. I actually wanted to, but I didn't know how (and I didn't read the tips - yet). My bad.

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

This reply has been marked as the solution. 
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OK, I finally got it!  I decided to write the test code in C (on my PC) then compile and test it. It was a lot faster than constantly re-flashing my Arduino and checking the results.

 

Here's the test program first:

 

// shiftbits.cpp

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <inttypes.h>

const char bigf[] = { // test font
	0b11111,
	0b11111,
	0b11000,
	0b11110,
	0b11110,
	0b11000,
	0b11000,
	0b11000,
};

int main (int argc, char *argv[])
{
	uint8_t x, idx, b1, b2, b3, b4, b5, b6, data;

	fprintf (stdout, "\n");
	fprintf (stdout, "        X     INDEX    B1      B2      B3      B4        B5        B6\n");
	fprintf (stdout, "       ---   -------  ----    ----    ----    ----      ----      ----\n");

	for (x = 0; x < 40; x++) {

		idx = (x / 5); // index into 5 byte font array

		b1 = (x % 8); // command data bits - backwards (0,1,2,3,4,5,6,7)
		b3 = (x % 5); // font data bits - backwards (0,1,2,3,4)

		b2 = (7 - b1); // command data bits - proper direction (7,6,5,4,3,2,1,0)
		b4 = (4 - b3); // font data bits - proper direction (4,3,2,1,0)

		b5 = (1 << b1); // command data bits to bitmask (01,02,04,08,10,20,40,80)
		b6 = (1 << b4); // font data bitmask (10,08,04,02,01)

		// generate the 8 bit command byte
		(*(bigf + idx) & b6) ? data |= b5 : data &= ~b5;

		fprintf (stdout, "       %2u", x);
		fprintf (stdout, "       %u", idx);
		fprintf (stdout, "       %u", b1);
		fprintf (stdout, "       %u", b2);
		fprintf (stdout, "       %u", b3);
		fprintf (stdout, "       %u", b4);
		fprintf (stdout, "       %3u", b5);
		fprintf (stdout, "       %3u", b6);
		fprintf (stdout, "\n");

		if (b2 == 0) { // one byte done, output it
			fprintf (stdout, "\n       write data to VFD: 0x%02X\n\n", data);
			usleep (1e3 * 1000); // sleep 1 sec
		}

		fflush (stdout);
	}

	fprintf (stdout, "\n");

	return 0;
}

 

 

The resulting output:

 

        X     INDEX    B1      B2      B3      B4        B5        B6
       ---   -------  ----    ----    ----    ----      ----      ----
        0       0       0       7       0       4         1        16
        1       0       1       6       1       3         2         8
        2       0       2       5       2       2         4         4
        3       0       3       4       3       1         8         2
        4       0       4       3       4       0        16         1
        5       1       5       2       0       4        32        16
        6       1       6       1       1       3        64         8
        7       1       7       0       2       2       128         4

       write data to VFD: 0xFF

        8       1       0       7       3       1         1         2
        9       1       1       6       4       0         2         1
       10       2       2       5       0       4         4        16
       11       2       3       4       1       3         8         8
       12       2       4       3       2       2        16         4
       13       2       5       2       3       1        32         2
       14       2       6       1       4       0        64         1
       15       3       7       0       0       4       128        16

       write data to VFD: 0x8F

       16       3       0       7       1       3         1         8
       17       3       1       6       2       2         2         4
       18       3       2       5       3       1         4         2
       19       3       3       4       4       0         8         1
       20       4       4       3       0       4        16        16
       21       4       5       2       1       3        32         8
       22       4       6       1       2       2        64         4
       23       4       7       0       3       1       128         2

       write data to VFD: 0xF7

       24       4       0       7       4       0         1         1
       25       5       1       6       0       4         2        16
       26       5       2       5       1       3         4         8
       27       5       3       4       2       2         8         4
       28       5       4       3       3       1        16         2
       29       5       5       2       4       0        32         1
       30       6       6       1       0       4        64        16
       31       6       7       0       1       3       128         8

       write data to VFD: 0xC6

       32       6       0       7       2       2         1         4
       33       6       1       6       3       1         2         2
       34       6       2       5       4       0         4         1
       35       7       3       4       0       4         8        16
       36       7       4       3       1       3        16         8
       37       7       5       2       2       2        32         4
       38       7       6       1       3       1        64         2
       39       7       7       0       4       0       128         1

       write data to VFD: 0x18

 

The final code - simplified and now in my VFD library for Arduino:

 

// Allows us to load up to 16 custom character bitmaps
// in display ram just like LiquidCrystal does.
// Note that "useCustomChar" must be set to true in order
// for the custom bitmap to show (this function does it
// automatically)
void Noritake_CU24063::createChar (uint8_t charcode, const void *charmap)
{
    uint8_t x, idx, b1, b2, b3, data;

    // erase this slot
    clearChar (charcode);

    // RAM user font define command
    _writePort (0x1B);
    _writePort (0x26);
    _writePort (0x01);
    _writePort (charcode);
    _writePort (charcode);
    _writePort (0x05);

    // convert 8 five bit arrays into 5 eight bit arrays
    for (x = 0; x < 40; x++) {

        idx = (x / 5);
        b1 = (7 - (x % 8));
        b2 = (1 << (x % 8));
        b3 = (1 << (4 - (x % 5)));

        ((*(const uint8_t *)(charmap + idx)) & b3) ? data |= b2 : data &= ~b2;

        if (b1 == 0) {
            _writePort (data);
        }
    }

    // enable custom char
    useCustomChar (1);
}

 

Note that some of the Bn vars have been combined together rather than the intermediate values that were used in the test program.

 

And it works!!!!!!!!   laugh

 

Thanks so much for all of your posts. The ideas about bitfields actually steered my mind in the right direction.

 

Thanks again!!

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Sat. Jul 14, 2018 - 06:09 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

BTW, here's the "Big F" bitmap loaded into the VFD:

 

 

jpeg image

 

 

(I used an "F" so that I could easily tell if it was backwards or upside down!)  :)

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Sat. Jul 14, 2018 - 07:06 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Krupski wrote:
OK, I finally got it!

Excellent!

 

Now please mark the solution - see Tip #5

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thread title:

Need help / ideas how to pack bits

Behind the Scenes of 'Pack Your Bits' -- http://blog.31bits.com/blog/behi...

https://ecurrent.fit.edu/blog/ca...

"Don't forget to pack your bits of happiness"

... and many similar

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I can't resist, for your C-freaks that talk about how hard (and slow it is to debug ASM code).

 

The code in #22 is a good example how it's possible to write complete non readable C code that is hard (close to impossible in my opinion) to maintain.    

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You've not heard of the The International Obfuscated C Code Contest, then? 

 

#22 is clarity epitomised by comparison!

 

surprise

 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

yes

No I did not. I love it cheeky 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

All #22 is missing is the array syntax that hopefully should have been fairly obvious. As soon as anyone starts typing "*((type *) base + offset) ?" they should recognize it was base[offset] ?
.
As for the ternary operator, I've no argument against simple use of that.

Last Edited: Mon. Jul 16, 2018 - 08:17 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

awneil wrote:

Krupski wrote:

OK, I finally got it!

 

Excellent!

 

Now please mark the solution - see Tip #5

 

I changed the thread title to "SOLVED - ......"  (isn't that what's supposed to be done?)

 

(edit to add): I got it now. Sorry I'm new to this board format. I'll get it right sooner or later!

 

 

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Tue. Jul 17, 2018 - 04:41 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sparrow2 wrote:

I can't resist, for your C-freaks that talk about how hard (and slow it is to debug ASM code).

 

The code in #22 is a good example how it's possible to write complete non readable C code that is hard (close to impossible in my opinion) to maintain.    

 

 

 

What's wrong with the code?  It's commented fairly well. The crazy stuff like

 

b3 = (1 << (4 - (x % 5)));

 

is just a combination of several simpler operations. In my TEST program, I did it all individually:

idx = (x / 5); // index into 5 byte font array
b1 = (x % 8); // command data bits - backwards (0,1,2,3,4,5,6,7)
b3 = (x % 5); // font data bits - backwards (0,1,2,3,4)
b2 = (7 - b1); // command data bits - proper direction (7,6,5,4,3,2,1,0)
b4 = (4 - b3); // font data bits - proper direction (4,3,2,1,0)
b5 = (1 << b1); // command data bits to bitmask (01,02,04,08,10,20,40,80)
b6 = (1 << b4); // font data bitmask (10,08,04,02,01)

...and the example line above is just b3, b4 and b6 combined together.

 

I suppose I should test both ways and see if there is any difference in code size (flash usage).

 

As far as *(charmap + idx) versus charmap[idx], it's 6 of one, half a dozen of the other. I simply prefer to do it the first way.  It's a habit.

 

(I wonder if there is any difference in the compiled output of those two and/or any speed difference)?

 

 

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Krupski wrote:
What's wrong with the code?
I think the line of complication is:

  ((*(const uint8_t *)(charmap + idx)) & b3) ? data |= b2 : data &= ~b2;

At first glance that does look kind of complicated! The problem actually starts here:

void Noritake_CU24063::createChar (uint8_t charcode, const void *charmap)
{

I wonder why you would pass in something that is almost bound to be const uint8_t * as "const void *" ? You generally use (or rather try to avoid using) void * for when it could be one of a number of pointer types and something else is passed to say how to interpret it and then you later cast the correct interpretation onto it (MISRA will be rolling in their grave at this point!). So if the API had been:

void Noritake_CU24063::createChar (uint8_t charcode, const uint8_t *charmap)
{

then in the "complex" line:

  ((*(const uint8_t *)(charmap + idx)) & b3) ? data |= b2 : data &= ~b2;

the pointer typecast would not be needed. Also as I said above whenever you find yourself using *(pointer + offset) in some form it's probably crying out to be pointer[offset] in fact which is a much cleaner, easier to read (and maintain) syntax. So if the pointer came in a "const uint8_t *" then this could simply have been:

  (charmap[idx] & b3) ? data |= b2 : data &= ~b2;

At this stage I think the line is fairly obvious to most C programmers. I suppose you could drop the ternary operator so it was simply:

  if (charmap[idx] & b3) {
     data |= b2;
  } else {
     data &= ~b2;
  }

but, like I said above I personally don't mind the ternary in there as it's a fairly clear syntax. Ternary only gets "complex" if you start nesting the:

x ? (a ? (u ? v : w) : c) : (m ? n : o)

Now THAT does make for unreadable (unmaintainable!) code. But the simple case is OK. However I think if you compare:

  if (charmap[idx] & b3) {
     data |= b2;
  } else {
     data &= ~b2;
  }

to

  ((*(const uint8_t *)(charmap + idx)) & b3) ? data |= b2 : data &= ~b2;

I think you have to agree that one of these is far more readable (and hence manitainable) than the other.

 

Remember it's not you that you are writing this code for. It's the guy who comes along in 3 years and tries to apply some fix or change to it which necessitates them reading and understanding what's going on. In fact that someone could well be YOU too as you will likely have forgotten what it was all about by then too (this is increasingly true the more years past 50 you are!).

 

This is why simple/clean syntax is always preferred. Trying to build complex, multi-part, nested statements (and I guess that includes using ternary) may be "awfully clever" but it's plain simple, obvious C that someone will be thanking you for in 3+ years.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The weakness of C is bit manipulation. This particular case is actually simpler to code in assembly:

 

C code for testing:

#include <stdint.h>

extern void pack(uint8_t* data_in, uint8_t* data_out);

uint8_t data_in[8] = {
	0x1F, // # # # # #
	0x1F, // # # # # #
	0x18, // # #
	0x1E, // # # # #
	0x1E, // # # # #
	0x18, // # #
	0x18, // # #
	0x18, // # #
};

uint8_t data_out[5];

int main(void)
{
	pack(data_in, data_out);
}

 

pack function in AVR assembly:

 .global pack

 pack:
	movw	X, r24		;data_in
	movw	Z, r22		;data_out
	
	;process data in reverse direction, it's simpler
	adiw	X, 8
	adiw	Z, 5

	ldi	r18, 0x01	;this marker bit gets shifted out when a byte is complete
next_line:
	ld	r0, -X
	ldi	r19, 5
next_bit:
	lsr	r0
	adc	r18, r18
	brcc	skip_store
	st	-Z, r18		;store a complete byte
	ldi	r18, 0x01	;reload marker
skip_store:
	dec	r19
	brne	next_bit	
	cpi	r18, 1		;if we have completed a line and a byte at the same time, we are done
	brne	next_line

	ret

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

El Tangas wrote:
The weakness of C is bit manipulation.
#3 ??

 

(which is the way I would have done it too).

Last Edited: Tue. Jul 17, 2018 - 09:11 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm not saying C can't do bit manipulation, only that in this case it's easier in assembly by using the carry to transfer bits, something C can't do. Just my opinion.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

El Tangas wrote:
something C can't do.
Well that's not entirely true. It's true that are no C operators that mean things like shift/rotate through carry so you, the programmer cannot directly tell it to manipulate the Carry flag. But for some kind of arithmetic operation it's quite possible the code generator might choose to emit a solution that involved the use of C (or sometimes T if it could be useful) though I agree that you have no direct control over what it chooses to use. So it's in the lap of the gods (who wrote the code generator) as to whether they spotted any opportunities.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Yeah, avr-gcc in particular likes to use the T flag to move bits around, and that's fine, but sometimes I'd prefer it would use the carry more.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ok so now I was actually looking of what OP want :)

 

And if speed isn't a problem there no doubt that I would store the data the other way around!

 

So having 5 bytes where one line is bit 0 of each byte, then the next line is bit 1 of each byte .... last would be all the bit 7.

 

at least it would some very clean ASM code, and I guess that it's also possible to make a nice C code for it aswell.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Gosh, I should know better than to post in the real programmer's sub-forum twice in one day, but...

 

(In Basic) I would have inserted the table from Post #16 as a comment in the code and then had 40 lines of simple assignments, one for each bit in the output, similar to:  (Assuming the Input Bytes are Right Justified)

 

OutputByte0.0 = InputByte0.4

OutputByte0.1 = InputByte0.3

OutputByte0.2 = InputByte0.2

...

OutputByte4.7 = InputByte7.4

 

It is neither elegant nor cleaver, but with copy/paste it would take only several minutes to write and would be easily understandable if the format changed down the road.

Not sure how many lines of ASM it would compile into, but it is simple and doesn't require any fancy pointers or loops.

 

Gotta keep it simple for some of us!

 

JC

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I would just brute-force it since it is only 8x5 bits.   I like the suggestion by El Tangas to reverse the 5-bit rows by using a 32-byte table for each of the possible 32 possible binary patterns.  Then copy the 8 bytes of 5-bits each into an array: myStart[8].  Have a second array that is 5 bytes in size : myDestination[5].   Then just chop off pieces from the 8 byte array and slide them into position in the 5-byte array: 

   

     uint8_t temp1;

 

     temp1 = myStart[0] & 0x1f;

    myDestination[0] |= temp1;

    temp1 = (myStart[1] & 0x07) << 5;

    myDestination[0] |= temp1;

    temp1 = (myStart[1] & 0x18) >> 3;

    myDestination[1]  |= temp1;

    temp1 = myStart[2] << 3;

    myDestination[1] |= temp1;

 

      .... etc   ...until all 8 row of 5 bits each have been shifted to the 5 byte [of 8 bits] array.  It's not elegant, but bit-pack/unpack code (for small arrays like this) doesn't need the elegance of complex loops.

 

Thank goodness programmers have stopped (hopefully) using bit-packing strategies like this to save memory.  This was much more common 25-30 years ago.   I have a MIDI tone-module (an Alesis S4) that has hundreds of adjustable parameter values stored internally in a dense bit-packing format like this.  Values in range of 0-3 are packed into two bits and then shifted and rotated into a exact position in the bit matrix.  Then the bozos did a second bit packing in order to get the entire array into a 7-bit per byte format that is used for MIDI bulk transfer.   What a nightmare!