How to promote public domain optimized libraries to Arduino & AVR community?

Go To Last Post
44 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I am looking for advice on how to promote libraries I write for Arduino, as I believe they would benefit the community.
 

I am looking for how to find people interested to get them reviewed, critiqued, and vetted if reviewers agree they're among the better alternatives, and finally promoted so the community can find and use them, and hopefully contribute to them too.

 

I've been playing with AVR Arduino for 5+ years and making code optimized libraries for myself and tiny communities. I am a comparch engineer but my carreer has been in non-public domain programing so I have not yet figured out how to effectively contribute and advertise in the public domain.

 

About my goals:

Being idle with Covid-19, I've decided to write a library to simplify hooking up devices and sensors to the Digital I/O pins to get back in coding for Arduino. This means a simple interface to support all switches, knock sensors, ultrasonic sensors, encoders, with or without interrupt handlers and debounce logic. I write with the mindset the Library user's code should be obvious to read. I also performance oriented. Since I care a lot about minimal memory footprint for my projects, I code those libraries with compiler constant folding in mind to minimize program and RAM space usage. For example the library has an AVR mode that accesses ports directly instead of arduino pin numbers going through PROGMEM tables that don't optimize away causing code bloat and slower performance. 
So right now, I'd like to promote this library: https://github.com/sonyhome/DigitalIO

Before, my main focus has been on writing a programmable LEDs library focussed for optimal memory management since I play a lot with AtTiny85 chips, and got some use through the LED community. However I'm interested in how to promote less fan-based libraries to AVR and Arduino communities, as I have made a few already (task manager, Serial console menu manager, LED library, etc.) and plan to do more of the same, next being Analog I/O devices.  Simple Digital and Analog device support are warm up exercises to get back into writing better code, better documented and presented libraries before I tackle bigger libraries, like a revamp of my LED library and a PWM motor framework  for cosplay builds (another attempt to make a simple framework).
 

I got into writing libraries because I have found that the libraries available on Arduino are often any combination of narrow scoped, kludgy, or inefficient code. There is merit in having so many contributions and working code for almost any chip or device. However, the end result is you often have to hack up code together for projects or scale up to more power hungry controller boards.
Engineers, even in my circle often solve scaling issues with bigger hardware instead of better software, because nobody wants to spend the time to optimize code. Hobbyists in my circle seem to rarely make reusable code and focus on slapping stuff together for one build. As I make projects I like to design for reuse hence why I make libraries. I'd love to see them used by others, and maybe have people contribute.

Anyways I would appreciate any help because I'd love to see this work taken advantage of more broadly. :)

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Overall, this sounds wonderful...will have to give it a try!   Hopefully the word gets out and many peeople use what you have done.  

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

>> I am looking for how to find people interested to get them reviewed, critiqued, and vetted if reviewers agree they're among the better alternatives, and finally promoted so the community can find and use them, and hopefully contribute to them too.

 

Does this really happen ? Call me cynical but 90% of Arduino users just want stuff that works. Very few are qualified to sort the wheat from the chaff.

 

There are a few well-known YouTubers with substantial followings, but I don't how you get to them. And then your code has to be 'interesting' and be whatever the flavour-of-the-month tech is.

 

I think you have to self-promote. Blog about it, post on the usual forums, create YouTube videos and tutorials, be searchable. People will eventually find out about you; some may even collaborate. Then be visible and supportive of people using your code. It takes time. There's a lot of noise out there.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Well I also want stuff that works and works well too. :)
 

I'd rather code or help people do technical fun stuff than blogging or youtubing. Also utility libraries are far from sexy to blog :)

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If you do give it a try, I'd love feedback on usability and features.

Does it indeed make your code simple, easier to read, faster? :) Is it easy to use? Is something missing or awkward to use?
 

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sonyhome wrote:
I am looking for advice on how to promote libraries I write for Arduino

 

sonyhome wrote:
I'd rather code or help people do technical fun stuff than blogging or youtubing

 

But blogging and youtubing is the way to promote stuff these days!

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

how to promote...

partner with a hardware vendor other than Arduino, who is trying to push the envelope.  Find someone doing Xmega or AVR-DA or something that doesn't have "legacy support issues."  Maybe Spence Konde's tiny mega core (for tiny0/tiny1)

 

I also performance oriented. Since I care a lot about minimal memory footprint for my projects, I code those libraries with compiler constant folding in mind to minimize program and RAM space usage. For example the library has an AVR mode that accesses ports directly instead of arduino pin numbers going through PROGMEM tables that don't optimize away causing code bloat and slower performance. 

Oh.  Well, no one seems to care much about performance and memory, and the hardware vendors that I mentioned solve the problem by designing faster hardware.  Adafruit, who I'd put pretty close to the top of the list of vendors "trying to push the envelope" is pushing Python interpreters :-(  Even the chip vendor libraries are infamous for their bloat and inefficiency :-(

 

digitalIo<13, LOW> led;
digitalIoAvr<B,5, LOW> led; /* Pin 13 is AVR port B5 */

Somewhat cynically:

 

  1. I cringe at the idea of explaining template instantiations (is that the right term) to the Arduino target audience.
  2. The abstraction of ports and bits of io ports to board "pin numbers" is probably the single most important thing that the Arduino framework as done.  Please don't undo it, especially in a way that is CPU specific.
  3. Similarly, you probably shouldn't bother unless you're willing to do AVR, SAMD, and ESP implementations all at the same time.  One of the best current aspects of the Arduino environment is its portability between architectures whose vendors probably couldn't come close to agreeing to library APIs.
  4. Oh swell, another implementation of fast digital write.  This one looks a lot like the one Bill Greiman did back in 2013 ( https://github.com/greiman/Digit... )  None of the faster digitalWrite()s have really caught on much (possible exception is PJRC's version in the "teensy" core.)

 

 

Still cynically, and in keeping with the "partner with HW vendors", you might be better off implementing more explicitly named after peripherals.  For example, instead of a "digitalSonar" class, have an "HCSR04" class (that's a sparkfun ultrasonic module part number, originally, I think.)  You can have a more abstracted class underneath, and go ahead and be promiscuous with vendor part numbers as well.  "I just want to read how far away things are; I don't want to know which chip my XYZR04 module uses!"

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 

sonyhome wrote:
how to promote libraries
Think about any Arduino library you ever looked for/used - how did you find that?

 

The chances are that if anyone is looking for library support for "KS0025 Tilt Digital Motion Sensor" or "A3144 Hall sensor" or whatever they are going to type something like "KS20025 arduino library" or "A3144 arduino library" into Google. So your real task is to get a higher presence on google. The techniques for achieving that are well documented across the internet. (spamming the internet with links back to your site may work a bit - but it could also piss off a load of people who's sites you post on - you get the link on this site for free! :-)

 

As an experiment I just tried:

 

 

I guess this may require a little work but if you give it a few days then perhaps the mention right here will make it onto that list as avrfreaks.net is trawled by the google-bots. I'm not sure if github is trawled by Google? It might help to make your own, (trawlable) website somewhere that talks about how to use the stuff you have on Github.

 

Last Edited: Fri. May 1, 2020 - 08:36 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You make a lot of good points.

 

I don't know of  Spence Konde's tiny mega core, is this the repo of the project? If it's to replace hardware/ATTinyCore/*, then maybe I could ask him to fix the pins_arduino.c tables to allow gcc to optimize away all the static pin to port conversions!

 

Few people care about performance and space optimization until they decide to max out their controller specs, like for AtTiny :/ For FAB_LED I had a few folks who found that library and thanked me for providing a memory efficient alternative to FastLED and Adafruit's neopixel library. Maybe I should bug Adafruit folks (I know one of them), that's a good point. They also do show and tell I believe I could try to flounder there.

 

  1. Beginners don't need to understand templates, just do pattern coding the way they all do, with a recipe we provide. The benefit of template constants is it allows gcc to do constant folding and magically remove the bloat of code that handles all use cases to just compile and inline only what the end user actually needs. I could try to do a version with a constructor OR a begin method but I'm not sure I'll get the same level of optimization, and to be fair would just use parenthesis instead of brackets and might require an extra setup function call which reduces flexibility (like: you can't declare it inside loop() - static or not).
  2. I agree yet for some reason they did not bother to provide libraries that can properly optimize away the reference to the translation tables. They hard-coded only one instance of PROGMEM tables to do so and gcc doesn't optimize around it. IDK how PROGMEM is handled nor why we can't get the optimizations I want. I'd go in and fix the Arduino code but IDK that I'd be allowed to.
  3. I care about AVR for my own purpose, but the libraries can be easily adapted for those. I would do that when I have a project with those - or if someone else helps or contributes.
  4. Lol I knoow :) I tried to make sense of Greiman's library for 5mn, as well as a few others. I've looked around a bit and libraries I've found came short of what I wanted. Also usually poorly documented and commented. I'm hoping this time my solution is cleaner. I also build in interrupt handler support, and implement support for all the digital devices that I came across. Having a software toolbox for all the dollar knick knack sensors can be a value add so you don't have to hunt down yet another thing.

 

In the code you quoted from my library, the digitalIo<> version actually should work on all platforms including AVR. It's actually a bit faster for AVR under the covers but it still pays for pin->port conversions. The digitalIoAvr<> version is a drop in alternative, a one line change for folks who cares to fully optimize for AVR.

 

I think naming classes by part numbers they support is a good idea, I'll add that to my to-do list, it's easy and would make it a no-brainer to figure out for a beginner. :)

 

Thank you for taking the time to read my ramblings!

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Good point. This idea will join back with Westfwl's remarks, if I explictly create support classes for part numbers. It will create part numbers to google.

 

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'd try to say I'm too old for this... but I actually do have wordpress and youtube. :P But again digital I/O is really not a sexy topic that will make anyone want to read all about all the cool tricks I nerded on to make a library that compiles like I want.

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sonyhome wrote:
I actually do have wordpress and youtube.
Another technique is to post some short tutorial videos showing how to use your library with KS0025, A3144 etc on Youtube and within the description of those include links back to your Github. People (I know I do) increasingly search and use Youtube when they want to "find out about stuff"(*) and it will all feed back into your overall Google rating too.

 

(*) last night I was looking for videos about how to cut hair when in lockdown! :-)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The repository you found is the the "tinycore."  MegaTinyCore ( https://github.com/SpenceKonde/m... )is more fertile ground (IMO) because:

  1. new chip family with less legacy.
  2. despite that, the current Arduino code is SLOWER than the normal ATmega core. :-(
  3. PROGMEM isn't needed.

 

I managed to get my digitalWriteFast() into MegaCoreX quoting similar logic:  https://github.com/MCUdude/MegaC...
https://github.com/WestfW/Duino-...

 

You might want to look into things like "__attribute__ (( error("") ))" and "#pragma gcc poison" as alternatives to "using a non-existent port writes to PORTB."

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Very good point! Might work with more obscure part numbers.

 

I had done that with my LED library. For example, I wanted to demo you could splice and control different types of LED strips with the FAB_LED library (ws2812 GRB and sk6812 RGBW). However I can't tell if it did bring any users.

 

How to cut hair in lockdown:

 

Flowbee

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

Last Edited: Fri. May 1, 2020 - 09:48 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OOh I'll have to look into this. the non-existent port hack I have bothers me a little bit.

 

The library is fresh out of the press. It should be bug free (hum right!) and that's one orner I cut to get it done and achieve most of the things I wanted from it.

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Okay your fastdigitalIO.h file comments look really interesting.

If I can get gcc to do constant optimizations for the annoying PROGMEM Arduino tables, then I can do away with the digitalIoAvr class altogether!

I was not successful at that but maybe I didn't try hard enough.

 

I can avoid exposing an AVR specific interface, it will be very very cool.

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

If your software is easy to use, works well and is filling a niche where little or no worthy competition exists, it will promote itself to Arduino in time.

IDK if westfw had to promote optiboot for it to become used in Arduino.

I didn't really promote jtag2updi yet it ended up in Arduino in a modified form.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Wiring (the origin of Arduino) had fast digitalWrite for compile-time constants ~10 years ago.

https://github.com/WiringProject...

 

As for simplicity/usability, a common mistake even for experienced developers is mixed up parameters.  For example when you pass 5, 4, 3 to digitalEncoder, is that outA, outB, button, or button, outB, outA, or ...?

The solution I use in TM1638NR makes the pin assignments obvious.

const byte TM1638NR::STROBE = 0;
const byte TM1638NR::CLOCK = 1;
const byte TM1638NR::DATA = 2;

https://github.com/nerdralph/TM1...

 

I'd also discourage the use of templates.  Not only are they hard to debug, many of the template tricks for code optimization are no longer necessary with LTO, since it does constant propagation across translation units.

 

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Wiring (the origin of Arduino) had fast digitalWrite for compile-time constants ~10 years ago.

https://github.com/WiringProject...

 The first mentions I can find of using __builtin_constant_p() to implement a faster digital write were back in 2007, and they reference "earlier messages" which I can't find. 

AFAICT Wiring incorporated some of Paul Stoffregen's code via Michael Margolis.  If you look at the older versions of Wiring you'll find:

/*
 * Michael Margolis
 * this version is based on Paul Stoffregen's Nov13 2009 submission to the
 * Arduino developers list
 */
//inline void digitalWrite(uint8_t, uint8_t) __attribute__((always_inline, unused));
/*inline void digitalWrite(uint8_t P, uint8_t V)
{
       (__builtin_constant_p(P) && __builtin_constant_p(V))
       
       :

 

But strangely for someone who is so bent out-of-shape about not getting proper attribution, there's no such credit in the Wiring 1.0 code.  While PJRC continued onward with highly optimized digital IO code (which I'm glad I don't have to maintain!), Arduino apparently decided they weren't interested.

(also, based on file/directory layout, it looks a lot like Wiring 1.0 started as a fork of Arduino rather than an expansion of the earlier Wiring code, even if Arduino started with the Wiring Code at its beginning.)

 

Remember that Arduino displaced things like the Basic Stamp and BASIC51 boards that were incredibly slow compared even with the Arduino digitalWrite()  (and yet still considered very useful.)

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You raise a good point that the macro params could be mixed up and the end result is not optimal to read at first glance. Hmmm.

 

The issue with your TM1638NR solution though is you can only have one instance of your class.

Might as well define a TM1638NR variable in your header  to make it easy to use.

 

I'm gonna experiment to see if I can achieve that level of expressibility with accessor routines and wesfw's solutions.

 

I'm glad I posted here, you all gave me a lot of food for thought.

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So I added digitalWriteFast to my library and here's the findings for PROGMEM usage on Uno:

 

Note: the program writes a LED and reads a button so both read/write were modified though I only list write.
 

2566 digitalWrite(pin)

2552 const uint8_t portName = digitalPinToPort(pinId);const uint8_t portBit = digitalPinToBitMask(pinId); *portOutputRegister(portName) &= 0xFFU ^ portBit;

2338 digitalWriteFast(pin)

2238 PORTD &= 0xFFU ^ (1U << portPin);

2324 *portOutputRegister(portName) &= 0xFFU ^ (1U << portPin);

 

So your digitalWriteFast really works well for the fact it just uses the pin tables. However for some reason there's still a 100B instruction overhead vs directly writing to the port, so one last thing didn't get optimized out.  I can't look at the ASM to see what happened right now since I have not done that for years and I'm on the road :P

I'd like to push your library into DigitalIO if you're OK with it, but the .h claims BSD, while your repo claims MIT. I've decided to go with MIT since I've had people complain with other licences preventing them from using my libraries before for their projects.

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

For GPIO I have been experimenting with an enum approach. Here are some examples for a board with 328pb and one with 324pb.

 

https://github.com/epccs/Gravimetric/blob/master/Manager/lib/io_enum_bsd.h

 

https://github.com/epccs/Gravimetric/blob/master/Applications/lib/io_enum_bsd.h

 

It is not a general solution, but that is not what I am after. I noticed that VSCode's Intelasense can do some helpful things.

 

 

Once I know the parameter type

 

 

I then edit (or copy/paste) the type to cause VScode to list the options, it is not a perfect solution but I need all the help I can get.

 

I also moved away from Wiring names because Hernando Barragán does not license his software for commercial use, so I am just going to respect that wish going forward.

 

Update: forgot to show mouse over parameter once it is valid

 

Last Edited: Sat. May 2, 2020 - 10:59 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

westfw wrote:

Wiring (the origin of Arduino) had fast digitalWrite for compile-time constants ~10 years ago.

https://github.com/WiringProject...

 The first mentions I can find of using __builtin_constant_p() to implement a faster digital write were back in 2007, and they reference "earlier messages" which I can't find. 

AFAICT Wiring incorporated some of Paul Stoffregen's code via Michael Margolis.  If you look at the older versions of Wiring you'll find:

/*
 * Michael Margolis
 * this version is based on Paul Stoffregen's Nov13 2009 submission to the
 * Arduino developers list
 */
//inline void digitalWrite(uint8_t, uint8_t) __attribute__((always_inline, unused));
/*inline void digitalWrite(uint8_t P, uint8_t V)
{
       (__builtin_constant_p(P) && __builtin_constant_p(V))
       
       :

 

But strangely for someone who is so bent out-of-shape about not getting proper attribution, there's no such credit in the Wiring 1.0 code.  While PJRC continued onward with highly optimized digital IO code (which I'm glad I don't have to maintain!), Arduino apparently decided they weren't interested.

(also, based on file/directory layout, it looks a lot like Wiring 1.0 started as a fork of Arduino rather than an expansion of the earlier Wiring code, even if Arduino started with the Wiring Code at its beginning.)

 

Interesting research.  I'm a bit conflicted Paul.  He seems like a smart developer, but he seems to think he's even smarter than he actually is.

 

After looking at you fast digital IO code, and mulling over your gripes about the macros used in Wiring,  I think the best idea may be to have the digital_pin_to_XXX arrays defined as const __flash, and included by wiring_digital.c.  With LTO, they'd be optimized away when the pins are constant.  By implementing it that way in the core, the regular digitalWrite function would be faster, and you wouldn't need the extra tricks in your code to detect LTO as the core sets the necessary LTO flags in platform.txt.

 

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ron_sutherland wrote:

For GPIO I have been experimenting with an enum approach. Here are some examples for a board with 328pb and one with 324pb.

 

https://github.com/epccs/Gravimetric/blob/master/Manager/lib/io_enum_bsd.h

 

https://github.com/epccs/Gravimetric/blob/master/Applications/lib/io_enum_bsd.h

 

 

I agree enums are much better than #defines, and have been using them where I can in picoCore.

https://github.com/nerdralph/pic...

 

Your ioMap structure looks like it does the same thing as the digital_pin_to_XXX arrays in the Arduino AVR core, except you've combined it into one multi-dimensional array.

 

For your toggle function, I'd recommend writing to the PIN reg instead of reading, negating, and then writing back to the PORT.

 

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ralphd wrote:
For your toggle function, I'd recommend writing to the PIN reg instead of reading, negating, and then writing back to the PORT.

 

I have seen that, but my quick attempt failed so I went with the slow but at least it works method. As I transition to newer parts (AVR128DA) I will be using the port function to toggle, so this problem has a built-in self-destruct.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I think the best idea may be to have the digital_pin_to_XXX arrays defined as const __flash, and included by wiring_digital.c.

"__flash" isn't present in c++  :-(  (and yes, there is various C++ code that accesses the arrays to achieve faster access to run-time-changed pins.)

Before I noticed the LTO hack, I considered having both pgmspace and "static const" versions of the arrays, controlled by ugly preprocessing directives, but I didn't like that very much :-(

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

westfw wrote:

I think the best idea may be to have the digital_pin_to_XXX arrays defined as const __flash, and included by wiring_digital.c.

"__flash" isn't present in c++  :-(  (and yes, there is various C++ code that accesses the arrays to achieve faster access to run-time-changed pins.)

 

 

wiring_digital.c isn't C++, it's C.  C++ code can still call C functions that use __flash data.  Here's a simple example with flash.cpp:

extern "C" int answer(int);

int main()
{
    return answer(0);
}

and flash.c:

const __flash int answers [] = {42, 0};

int answer(int n)
{
    return answers[n];
}

I compiled with "avr-gcc -mmcu=attiny13 -Os -flto flash.cpp flash.c -o flash", and here's the result:

00000022 <main>:
  22:   8a e2           ldi     r24, 0x2A       ; 42
  24:   90 e0           ldi     r25, 0x00       ; 0
  26:   08 95           ret

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

both pgmspace and "static const" versions of the arrays, controlled by ugly preprocessing directives

 

V.V guilty, that's what I would have done :P ... oh and DID for this library.

One thing I'd worry about is gcc specific or arch specific primitives, that makes the library not portable, in case say we'd upgrade to clang/llvm...

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sonyhome wrote:

The issue with your TM1638NR solution though is you can only have one instance of your class.

Might as well define a TM1638NR variable in your header  to make it easy to use.

 

That's not an issue, that's a design intent feature.  There should not be multiple instances of the class.

While in theory someone could have more than one TM1638 module hooked up to an AVR, my library covers the vast majority of the use cases.  Supporting multiple TM1638 connections, without sacrificing any performance when just one is used would make for a lot more work.  I only have one TM1638 module, so I'd need to get another couple before I could properly test a library that supports multiple instances.

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm not criticizing that implementation. I'm pointing out it's not viable in my digitalIO library which can be instantiated for multiple devices. :)

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Final thoughts:

 

The reason link time optimization was not fully working seem to be because  accessor macros use pgm_read_byte(). Using port_to_output_PGM[] directly solved this without requiring GCC specific inline macros. Thanks again to Bill for making me dig into this.

 

#define portOutputRegister(P) ( (volatile uint8_t *)(uint16_t)( pgm_read_byte( port_to_output_PGM + (P))) )

 

I'm now going to think about how to solve the other points raised.

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Just to note that you would not need pgm_read_byte() if this stuff was in C not C++ and you could use "__flash"

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

pgm_read_byte() is unrelated to my code:

It is used within the Arduino architecture specific files pins_arduino.h, which are C files. :)

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sonyhome wrote:

 

I'm now going to think about how to solve the other points raised.

 

Here's something I came up with for type-safe named parameters.  It's a bit verbose, but it works.

 

class SpiSensor
{
    public:
    struct SS {
        int sspin;
        SS(int pin) : sspin(pin) {}
    };
    struct SCK {
        int sckpin;
        SCK(int pin) : sckpin(pin) {}
    };
    SpiSensor(SS ss, SCK sck) : sspin(ss.sspin), sckpin(sck.sckpin) {}

    private:
    SpiSensor(){};
    int sspin;
    int sckpin;
};

void foo()
{
    SpiSensor mySensor(SpiSensor::SS(10), SpiSensor::SCK(11));
}

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sonyhome wrote:

pgm_read_byte() is unrelated to my code:

It is used within the Arduino architecture specific files pins_arduino.h, which are C files. :)

 

Since when is a .h a C or C++ file?  It can be included by a .cpp, which will be compiled as C++.  It can be included by a .c file, which will be compiled as C.

 

Also, I think Bill already pointed out that directly referring to port_to_output_PGM and the other pin-mapping arrays is not guaranteed to work as some cores do not use arrays for the digitalPinToXXX functions.  Those arrays are not part of the Arduino API.

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ralphd wrote:

sonyhome wrote:

 

 

I'm now going to think about how to solve the other points raised.

 

 

Here's something I came up with for type-safe named parameters.  It's a bit verbose, but it works.

 

class SpiSensor
{
    public:
    struct SS {
        int sspin;
        SS(int pin) : sspin(pin) {}
    };
    struct SCK {
        int sckpin;
        SCK(int pin) : sckpin(pin) {}
    };
    SpiSensor(SS ss, SCK sck) : sspin(ss.sspin), sckpin(sck.sckpin) {}

    private:
    SpiSensor(){};
    int sspin;
    int sckpin;
};

void foo()
{
    SpiSensor mySensor(SpiSensor::SS(10), SpiSensor::SCK(11));
}

 

Here's a more concise version:


class SpiSensor
{
    public:
    struct SS { int pin; };
    struct SCK { int pin; };
    SpiSensor(SS ss, SCK sck) : sspin(ss.pin), sckpin(sck.pin) {}

    private:
    SpiSensor(){};
    int sspin;
    int sckpin;
};

void foo()
{
    SpiSensor mySensor(SpiSensor::SS{10}, SpiSensor::SCK{11});
}

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The argument was that template syntax can be daunting for noobs, and opaque since order matters (just like function parameters btw).

 

I'm not sure if I'm sold with this idea in this case... I kinda do something akin to this for my LED library with a set of inherited classes, as it's a way to support RGB, GRB, RGBW etc. In fact a trick I use is define in each a static constant that tells me the type of the parameter.

 

I have to think about switching to a normal constructor and see if that works the same for constant propagation.

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Test results:

 

A template constant uses less memory than a class constant set by the constructor.

Also a class constant doesn't use less RAM than a non constant.

The first way of writing helps the compiler the most .

template <uint8_t property>
class T<property> { T(){}; }

 

class T {const uint8_t property; T(uint8_t p) : property(p) {}; }

 

class T {int8_t property; T(uint8_t p) : property(p) {}; }

 

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sonyhome wrote:

Test results:

 

A template constant uses less memory than a class constant set by the constructor.

Also a class constant doesn't use less RAM than a non constant.

The first way of writing helps the compiler the most .

template <uint8_t property>
class T<property> { T(){}; }

 

class T {const uint8_t property; T(uint8_t p) : property(p) {}; }

 

class T {int8_t property; T(uint8_t p) : property(p) {}; }

 

 

That's strange.  You shouldn't have to use templates to get the minimum code size.  Are you forgetting to turn on LTO?

 

I have no special talents.  I am only passionately curious. - Albert Einstein

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm very interested in this kind of stuff. I wrote an entire library for the configuring stuff on the atmega328p. The basic idea being to break out every option into a define. If the results of everything is non zero (i.e not the default start up value) then the OR'd results are written to the appropriate register. As a side note it's not completely functional and there are things I wrote but then gave up on implementing.

 

https://github.com/harrydylewski/AT328P-Peripherals/tree/version2

 

Ultimately you're gonna run into a ton of questions. A big lingering question is what to, for example, in situations where you're waiting for the I2C /UART to finish transmitting. Do you poll or go to sleep with an interrupt, and then you need to make sure it's an appropriate sleep mode. How would you implement the same thing if you switch to an MCU with DMA?  Ultimately arduino/newbies aren't going to care if they get the results they want. 

 

I suggest take a look at the code posted here to see a good example of GPIO template code. Enumerations help block invalid configurations, right shift by 3 on portx_ determines the port letter. Don't need to have a default state with the constructor, just call an appropriate function, and therefore skip the static_assert for low/high.

Comment from where the code came from:

https://www.avrfreaks.net/comment/2861481#comment-2861481

 

In terms of multi pin writes on the same port, I came up with this macro. 

 

#define PINS(LOW,HIGH) (unsigned char)((1<<(HIGH-LOW+1))-1)<<LOW

 

which will automatically set bits from (and including) the lsb to msb on, for example PINS(1,5) = 0x3E. Just don't try to ~ it.

 

From that little macro, I created a port template class

 

enum PINS : uint8_t  { 
    B0=0,B1,B2,B3,B4,B5,B6,B7,   //B6,7 usually used by the crystal, do not use with arduino 
    C0,C1,C2,C3,C4,C5,C6,C7, //C6,7 are analog input only, only usable when using TQFP-32 packages
    D0,D1,D2,D3,D4,D5,D6,D7	
};

template<PINS lsb, PINS msb>
class PORT
{
	    constexpr static uint8_t portx_{msb>>3}; //B=0,C=1,D=2
	    constexpr static uint8_t shiftbit_ {0x07&lsb};
	    constexpr static uint8_t pins_{( (1<<(msb-lsb+1))-1 ) << shiftbit_  };
	    //constexpr static uint8_t pinbm_{pins_<<lsb}; //0x01,0x02,0x04...0x80

	    constexpr static volatile uint8_t* ddrR(){
		    return portx_ == 0 ? &DDRB : portx_ == 1 ? &DDRC : &DDRD;
	    }
	    constexpr static volatile uint8_t* portR(){
		    return portx_ == 0 ? &PORTB : portx_ == 1 ? &PORTC : &PORTD;
	    }
	    constexpr static volatile uint8_t* pinR(){
		    return portx_ == 0 ? &PINB : portx_ == 1 ? &PINC : &PIND;
	    }
	
	public:
	void setOutput(){ *ddrR() |= pins_; }
	void setInput(){ *ddrR() &=~ pins_; }
		 
	void write(uint8_t input)//{ ( *portR() &=~pins_); (*portR() |=pins_&input);         }	
	{	

	*portR() =  (*portR()&~ pins_) | ((input<<(shiftbit_))&pins_);	
	}
	
};

 

Quite accidentally, because of how pins_ is written it protects against pins being on different ports

 

originally: https://github.com/harrydylewski/LCD-4bit-template/blob/master/avr_gpio_template.h

 

 

 

 

 

Last Edited: Fri. May 8, 2020 - 11:12 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm using the Arduino IDE as-is (well except now warnings are set to high since I looked into it), and I often have a +O2 pragma in my libraries.

 

How would I control that?

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I did not know this C++ construct, instead of using "=". What's the advantage of this syntax?

constexpr static uint8_t portx_{Pin_>>3};

Optimizing for Arduino Pins:
So the problem with the enum approach is it works for a single board, but I want to leverage the arduino pinouts already defined for all the existing boards supported by the platform. So I have to rely on the existing *_PGM tables to do a pin->port conversion. A challenge on AVR Arduino is most people don't know the port nomenclature. It's further muddied with some boards just shuffling the mapping in weird ways.

Optimizing nomenclature for AVR ports:
Your method would look much nicer than my AVR approach <B3> vs <B,3>, at the expense of an enum with at least 10*8 ports (A...K). Note you still need to solve non-existing ports with ifdefs, which you could do inside the enum to error the user trying to access an invalid port (cleaner error). However I could use this trick to avoid declaring two classes digitalIo and digitalIoAvr, simply by hacking the enum to detect if it's an AVR port or an Arduino pin being passed along. That's much better.
 

Multiple pins:

I like the simple solution for that, which allows updating all the pins in one instruction.

template<PINS lsb, PINS msb>
...
constexpr static uint8_t pins_{( (1<<(msb-lsb+1))-1 ) << shiftbit_  };

Interrupts:
I have not tackled I2C or SPI or any other more complex protocol hardware.
I'm not sure if you're talking about implementing those or the risk of gpio interrupts interfering with their proper behavior.

I have added interrupt support for the rotary encoder and ultrasonic sensor classes since they benefit from it directly.

For the simpler digital read I just want to offer the ability to interrupt on a pin event to avoid waiting for a signal.
For a sensor it would allow guaranteed detection independent of the main loop() latency.

... For that, either I provide a callback in the template, OR I write a callback that handles most use cases someone might have. I'll likely go for that since I need to just support stuff like isOn, isOff, transitionUp, transitionDown, debounced and not. I'd clear the state(s) once it is read in the main loop().
Unfortunately, tracking and sharing states would waste 1 byte of RAM, unless I can manage to avoid doing so when interrupts are not requested (I'm gonna have to scratch my head on that, since I hate to waste that memory if it's not used).

 

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

Last Edited: Sat. May 9, 2020 - 11:53 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

sonyhome wrote:

I did not know this C++ construct, instead of using "=". What's the advantage of this syntax?

>You'll have to ask curtvm, he wrote it

 

sonyhome wrote:

..but I want to leverage the arduino pinouts already defined for all the existing boards ...

 

> Big problem, arduino stretches from AVR,ARM and ESP, so how do you create highly portable libraries between micros? Now you understand why this stuff ain't easy... My answer is to create more modular libraries and drivers, adopt a layered approach which is hard to implement because there are thousands of MCUs, it's a ton of work with no real pay off.

 

Taking you're encoder library for example

 

main <- programmer interacts at this  layer

encoder <- provides all the functionality required

mcu architecture <- read/writes to the architecture specific registers (i.e AVR = PORT / ARM = ODR)

mcu specifics <- what pins exists

 

IMO making object libraries that are lightly coupled to their driver library is more important because the functionality required is 90% of the work.

 

sonyhome wrote:

...hacking the enum to detect if it's an AVR port or an Arduino pin ...

 

#ifdef UNO
enum PINS : uint8_t  { 
	D8,D9,D10,D11,D13
	A0,A1,A2,A3,A4,A5	
	D0,D1,D2,D3,D4,D5,D6,D7
};

#elif ATMEGA328P
enum PINS : uint8_t  { 
    B0=0,B1,B2,B3,B4,B5,B6,B7,   //B6,7 usually used by the crystal, do not use with arduino 
    C0,C1,C2,C3,C4,C5,C6,C7,    //C6,7 are analog input only, only usable when using TQFP-32 packages
    D0,D1,D2,D3,D4,D5,D6,D7	
};

#elif MEGA
enum PINS : uint8_t  { 
	D8,D9,D10,D11,D13
	A0,A1,A2,A3,A4,A5	
	D0,D1,D2,D3,D4,D5,D6,D7
	//tons of other IO
};

#endif

or


//uno specific
enum PINS : uint8_t  { 
    PB0=0,PB1,PB2,PB3,PB4,PB5, 
    PC0=8,PC1,PC2,PC3,PC4,PC5,  
    PD0=16,PD1,PD2,PD3,PD4,PD5,PD6,PD7,
	
    D8=0,D9,D10,D11,D13
    A0=8,A1,A2,A3,A4,A5	
    D0=16,D1,D2,D3,D4,D5,D6,D7
};

 

>yes, "hacking" 

 

sonyhome wrote:

I like the simple solution for that, which allows updating all the pins in one instruction.

 

Even this little code has inefficiencies, for example, if you actually need to left shift, then ROL instructions get introduced, which means it's treating the data as 16 bits even when it's 8 bits. However that seems to be a common problem with C++ on AVR, no matter how hard you try it'll treat some stuff as 16 bits. I've seen a few threads regarding this. If you're working on the lower or higher 4 bits though it's a real godsend. 

 

sonyhome wrote:

I have not tackled I2C or SPI or any other more complex protocol hardware.
I'm not sure if you're talking about implementing those or the risk of gpio interrupts interfering with their proper behavior.

 

I'm talking about what happens when you're waiting on I2C/Serial to finish sending a byte

 

I2C is normally 400KHz, 1/400KHz * 8 bits = 20uS 

Uno normal speed is 16MHz so, 20uS / (1/16MHz) = 320 cycles you're wasting waiting for the i2c to finish

A) implement callbacks?

B) go to sleep? If you do go to sleep you need make sure the user doesn't go into too deep a sleep mode otherwise multiple peripherals will stop and not trigger their respective wake up interrupts.

 

These are questions for future you, I thought you already knew all the internals and just wrote a new GPIO library to get started on this new quest.

 

sonyhome wrote:

Unfortunately, tracking and sharing states would waste 1 byte of RAM, unless I can manage to avoid doing so when interrupts are not requested (I'm gonna have to scratch my head on that, since I hate to waste that memory if it's not used).

 

Uno has 2000 bytes, do you think using 1 byte will change anything? Even on an attiny85, you'll have more problems with the limited amount of pins you have than ram usage.

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So using the enum approach works really well, so well that I can dump the "digitalIoAvr" version in my library, and probably tickles the LTO engine the right way!!!

 

The only downside is collisions (aka B0 or PD0 already defined) so I need to use a different prefix. For example: "avr".

#include <DigitalIO.hpp>
digitalIo<13, LOW> led; // On AVR this goes through the pin->avr PROGMEM tables, still way more efficient than digitalWrite().
digitalIo<avrD7, HIGH> button; // On AVR I can reference D7 directly and save memory

Here's the internals (note it creates a specialization for AVR, I could do the same for SAMD or ARM if needed, but AVR is the slow polk that needs the extra cycles and bytes the most in projects I've done).

typedef enum avrPin : uint8_t

{

#ifdef PORTA

 avrA0 = 128, // I haven't seen AVR with more than 128 pins so I can keep using uint8_t, not that it matters much with constants.

        avrA1, avrA2, avrA3, avrA4, avrA5, avrA6, avrA7,

#endif

#ifdef PORTB // The ifdef should allow me to support all boards without customization for each

 avrB0, avrB1, avrB2, avrB3, avrB4, avrB5, avrB6, avrB7,

#endif

#ifdef PORTC

 avrC0, avrC1, avrC2, avrC3, avrC4, avrC5, avrC6, avrC7,

#endif

...
}

inline uint8_t getAvrPort(uint8_t pinId) {

  if (pinId > 128) return pinId>>3;

  else return digital_pin_to_port_PGM[pinId];

}

inline uint8_t getAvrPin(uint8_t pinId) {

  if (pinId > 128) return pinId & 0x07;

  else return digital_pin_to_bit_mask_PGM[pinId];

}

...
template <uint8_t pinId>
class digitalIo {
...

    #if DIGITAL_IO_AVR_PINMODE != DIGITAL_IO_AVR_DEFAULT

      const uint8_t portBit = getAvrPin(pinId); // pinId is from the template constant so all this is constants

      const uint8_t portName = getAvrPort(pinId);

      *portModeRegister(portName) &= 0xFFU ^ portBit;

    #else

      pinMode(pinId, INPUT);

    #endif

 

I'm not a microcontroller expert, no, I am just a systems/OS programmer with a side hobby and I actually have not read all the specs which would require on top of that a little bit of practice to feel at home since there's a million ways to map ports to memory addresses.

 

I definitely don't want to bog down the library into a non portable implementation NOR implement support for every board with custom code. I'm not too worried about a ROL on 16 bits vs 8bits (I assume you could do a *2 instead of a <<1 to tell gcc you don't want 16bit), but I'm more worried about very inefficient jumps to PROGMEM and hopping around functions to handle constants.

 

For I2C, I would offer a non-interrupt based solution that does a busy wait, not a sleep (aka while (!ok) {}) which might use more power than a power down/sleep mode, and an interrupt based callback to send/receive the rest of the buffer, IF there's a case where I2C is not handled by hardware that always has an interrupt attached to it. If there's an interrupt, I'd let the main loop decide to power down. But for low power I have not considered it much as it's not part of my projects, even though I use AtTiny chips... Indeed, future me. :)

My GitHub (FAB_LED bitbang for ws2812, DigitalIO, SerialMenu...).
I make portable, code optimized Arduino libraries as a hobby and use AtTiny85 the most for my builds.
I am a burner and an OS and systems programmer by trade.

Last Edited: Sun. May 10, 2020 - 08:11 AM