Writing a new compiler by using open source compiler compilers

Go To Last Post
22 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm tired of this 64K problem and PROGMEM and __flash...

I think I'm going to write a new compiler by using a compiler template in order to solve this problem.

I've been calculating what kind of pointers would that compiler have.

For small devices:

  1. Small Flash; 16-bit; __flash, __flash0
  2. Small Data; 16-bit; __data, __lildata
  3. Memory X; 24-bit; __memx

For big devices:

  1. Big Flash; 24-bit; __flash, __bigflash
  2. Big Data; 24-bit; __data, __bigdata
  3. Memory X; 24-bit; __memx
  4. Big Memory X; 32-bit; __bigmemx

 

The bold attributes are the attributes that would be perfect for XMEGA projects that go over 64KB in both flash and RAM!

All constant literal strings would automatically be __flash-ified, but the size of __flash will depend on the device. If the device is small, it will be 16-bit, if the device is big, it will be 24-bit.

That way you won't have to use printf_P and PSTR or be scared about not having printf_PF if your program is above 64KB.

Every const variable goes to flash except if you say __flash1, __flash2, __data to put it where you want.

Big Memory X would be consisted of Big Flash and Big Data. It would also be the linker's addressing system.

0x00xxxxxx = Flash

0x01xxxxxx = Data

0x02xxxxxx = EEPROM

Big Data is used so that you can use external RAM up to 16MB! Which is super for making awesome things like PNG readers or video players for XMEGA devices.

 

What do you think how long would it take to use a template compiler to do these things? How much time would it take to write standard libraries? What kind of compiler compiler should I use?

Last Edited: Fri. Feb 27, 2015 - 12:51 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

This thread may form the basis for future dictionary definitions of the phrase "using a sledgehammer to crack a nut" !!

 

Wouldn't it simply be easier to buy a copy of Codevision or IAR or something? cheeky

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Or play with lunaAVR and enlighten me (us?)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

BTW if you wait a while there may be LLVM for AVR. Not sure what the plans are for it supporting anything like __flash/__memx though.

 

EDIT: Realise this is an old message but seems to suggest maybe it doesn't have Harvard support...

 

http://lists.cs.uiuc.edu/piperma...

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I don't like commercial software. It's not portable, it has sopastic license agreements, it costs,...

Are you sure that IAR solves this 64KB problem?

IS THERE A FREE AND OPEN SOURCE AVR COMPILER WITHOUT THE 64KB PROBLEM!?
 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

What "64KB problem" anyway? The fact is that the AVR is an 8bit micro and they typically have 16 address bits so can "easily" address 64KB. Beyond that you are into paging/bank switching and the requirement to split addresses into 8 bit bank pointer (RAMPX, RAMPY)/RAMPZ and then an offset.

 

It's not really any different to the old DS/ES/SS segmented architecture of the 8088/8086 that programmers battled with for decades.

 

There are always "issues" when data objects straddle a 64K boundary. The best idea is probably to make sure nothing does and use padding/alignment to avoid it. This is something the programmer can control as it is her data.

 

If 64K banks irritates you use a micro with a wider address bus (ARMs have 32bit address busses that allow everything to be positioned in a "flat" 4GB address space).

 

It's surely easer to switch to a Cortex processor than write a language compiler (10..20 man years perhaps?).

 

If you really do write a compiler I imagine you will probably fix 1 "problem" and create 10,000 more you never knew you had!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

What do you think about UC3? Would that be a better XMEGA?
 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I wouldn't touch a UC3 with yours!

 

Personally I think UC3 is completely irrelevant. If you want 32bit you choose ARM (and these days that means Cortex). Full Stop.

 

No doubt someone will be along in a minute to tell us about "better" instruction sets or lower power operation or something. Well, OK, yeah. But still not convinced.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

How much do these ARMs cost? If I'm going to make something like that, then I'll have to make an OS for it which would be much more complex than a regular OS for an XMEGA. I mean, MMU, virtualization,... Are there even tutorials for that? I've heard from my mentor that those stuff are so ultra hard that people wasted their jobs over that and even Intel almost bankrupted from the resources used for virtualization.

I need to make a low cost device that acts like a PC for file transfering. Like a physical file manager.
 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for the mentions of "memory models".  PLM86, and x86 in general, and all the far/near combinations...  That was several lives ago.

 

Rather than starting from scratch, if this particular "feature" is a great irritation, I think I'd first look at perhaps pre-processing my big banks of constant data (fonts? logos?  Other graphic images?) into source modules with the desired annotation for sections and the like, and then the corresponding lines for the linker script.  If the compiler is given enough hints about how to access e.g. each font app dev can move on.

 

The second thing I'd look at instead of starting from scratch is to learn how to modify and rebuild GCC.  Then you could make your own subsystem that is analogous to PROGMEM and __flash and associated, but fits your app needs.

You can put lipstick on a pig, but it is still a pig.

I've never met a pig I didn't like, as long as you have some salt and pepper.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Foxcat385 wrote:
Are you sure that IAR solves this 64KB problem?
Yes.

  1. IAR Systems
  2. Support
  3. User Guides

User Guides: IAR Embedded Workbench for Atmel AVR

https://www.iar.com/Support/user-guides/user-guides-iar-embedded-workbench-for-atmel-avr/

ftp://ftp.iar.se/WWWfiles/AVR/webic/doc/EWAVR_CompilerReference.pdf (approx. 5MB)

Part 1. Using the compiler

Data storage

Memory models

The stack max size is 64kB; that's not a problem because should use AVR local RAM for most stack operations.

 

Foxcat385 wrote:
IS THERE A FREE AND OPEN SOURCE AVR COMPILER WITHOUT THE 64KB PROBLEM!?
A conditional no, and, I have not appreciably followed LLVM AVR.

Atmel provides a huge memory module for AVR GCC and 24bit types were added to AVR GCC.

If that's not enough, the XMEGA DMA engine will move up to 16MB in one transaction in block multiples of up to 64kB blocks.


http://asf.atmel.com/docs/latest/doxygen_mainpage/images/atmel.png

Atmel Software Framework

Data in Huge Data Memory Space

http://asf.atmel.com/docs/latest/xmegaau/html/group__hugemem__group.html

GCC Wiki

avr-gcc

Extensions

Types

http://gcc.gnu.org/wiki/avr-gcc#Types

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Thanks for your post; was not aware of the Luna computer language.

http://avr.myluna.de/lib/tpl/luna/images/dashboard.jpg

LunaAVR

start » en » about

http://avr.myluna.de/doku.php?id=en:about

...

Luna is an object-based, modern Basic/Pascal-like programming language, the structure and syntax are based on current development tools.

 fyi, for Germany, Pascal, and AVR there's an AVR Pascal IDE :

  • 16 bit pointers other than for some tiny AVR.
  • Multiple address spaces.
  • Banked pointers (mega1280, mega2560, mega128 (?), etc.; 16 banks, 32kB/bank, 512kB; schematic for external RAM and such to an AVR)
  • Zero price, limited flash size releases.

E-LAB Computers

Pascal-scm for Atmel AVR

http://www.e-lab.de/AVRco/index_en.html

Edit : more info on Pascal banked pointers.

"Dare to be naïve." - Buckminster Fuller

Last Edited: Sat. Feb 28, 2015 - 06:45 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
BTW if you wait a while there may be LLVM for AVR.

clawson wrote:
If you want 32bit you choose ARM (and these days that means Cortex).
If wait some time then LLVM ARM will filter down from Cortex-A to Cortex-M then into open source.

Apple has well created, and creators on iOS have well operated, LLVM ARM.

ARM is also moving to LLVM in ARM Compiler 6.


Home DS-5 Build

http://ds.arm.com/ds-5/build/

"Dare to be naïve." - Buckminster Fuller

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

How much do these ARMs cost? If I'm going to make something like that, then I'll have to make an OS for it which would be much more complex than a regular OS for an XMEGA. I mean, MMU, virtualization,... 

You are confusing your ARMs, there's a lot to choose from, at the top end you have processors like the one in your smartphone. This could well be quad core Cortex A8s running four cores at 1GHz with an OS like Linux using the MMU the chip might cost $10..20.

 

But I was thinking of Cortex M0+, M3 or M4. These can be as small as 8pin DIP. Can have as little as 16K FLASH and 4K RAM. Run at about 50MHz. Have nothing asas exexciting as an MMU. And cost $1or less. They are equivalent to tiny and mega chips (though no EEPROM). A lot of people are starting to use them in preference to AVR8. Atmel have a good range of Cortex M chips on offer. The Atmel ICE can program and debug them just as easily as tiny/mega. They have "flat" memory spaces. arm-xxx-gcc will put "const" into the .rodata section which locates in flash. There is nothing like LPM/ELPM as ARM loads/stores operate identically on RAM or flash. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Foxcat385 wrote:
How much do these ARMs cost?

See reply from clawson - that depends entirely on which particular "ARM" you're talking about! They range from the tiny Cotex-M0 microcontrollers up to multi-core Cortex-A9 microprocessors - not to mention the old-school ARM7 and ARM9.

 

But you could easily compare prices yourself at your favourite distributor...

 

Quote:
If I'm going to make something like that, then I'll have to make an OS for it

Not at all.

 

I would guess that the majority of Cortex-M0 & M3 projects are "bare metal" with no OS at all. possibly also M4.

 

Quote:
which would be much more complex than a regular OS for an XMEGA.

Again, no.

 

Quote:
I mean, MMU, virtualization,

Not on Cortex-M class 

 

Quote:
Are there even tutorials for that?

Is the Pope Catholic?

 

Quote:
I've heard from my mentor that those stuff are so ultra hard that people wasted their jobs over that

Oh dear - time to get a new Mentor!

 

Quote:
I need to make a low cost device that acts like a PC for file transfering.

Eh??

 

clawson wrote:
But I was thinking of Cortex M0+, M3 or M4 ... They are equivalent to tiny and mega chips (though no EEPROM)

Atmel (among others) provide means to emulate EEPROM in the Flash.

 

Quote:
A lot of people are starting to use them in preference to AVR8.

Indeed. My recent projects have involved porting from AVR8 to SAM D/R (Cortex-M0+)

 

Quote:
The Atmel ICE can program and debug them just as easily as tiny/mega.

And there's the EDBG on the Xplained-Pro boards - so no need to buy a separate debug probe.

 

They also work with standard 3rd-party offerings like J-Link and uLink. And, of course, the GCC open stuff.

 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ARMs being 'ultra hard' vs writing a compiler???? With the ARM cortex m series, there's stacks of cheap boards from a variety of vendors complete with a debugger. Then there's the likes of Arduino and MBED that ease the introduction. The 'complexity' is that there is a bit more setup involved to get the chip going - you need to set up the clock system which invariably has a pll, each peripheral block has a power and clock enable and the peripherals are generally a bit more sophisticated than the humble AVR. Luckily most manufacturers give you libraries like ASF to hide a lot of the evil. Past that, it's all much the same as the AVR without the memory size/space issues. The new 'challenges' when moving to ARM are:
Structure packing
Using 32bit vars - being a 32 bit cpu, the code favours using 32 bit variables. The tradeoff is either chewing up more ram or use smaller vars but at a performce cost.
It's generally much faster
Expect to use more flash vs AVR - mainly due to libraries. Considering you can get parts with 1MB+ flash quite economically, it's not really a problem.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

With the ARM cortex m series, there's stacks of cheap boards

I have 2 FREE boards from ..... and used their free IDE .

John Samperi

Ampertronics Pty. Ltd.

www.ampertronics.com.au

* Electronic Design * Custom Products * Contract Assembly

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
there's stacks of cheap boards from a variety of vendors complete with a debugger.

The on-board debug probe is de rigueur these days.

 

Unless you already have and are thoroughly acquainted with a "stand-alone" debug probe, I can't see any point in ever buying a board without one these days.

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ignoramus wrote:

Or play with lunaAVR and enlighten me (us?)

Downloaded and played with the IDE for a bit and it seems pretty straight forward.

- Language looks like it would be pretty easy to pick up.   Ran the TaskKernel demo and it ran without problems and it was easy to trace and understand.

- Downloaded using my USBTiny and had no problems after setup getting it to download.

- I like it and will see what I can do with it when I get more free time.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Mike,

 

Thanks.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Nice to see LunaAVR sparks some interest on avrfreaks.net.

 

I will show you a case.

The following quest is given: simultaneously blink three LEDs at different frequencies: 250ms, 500ms, 750ms (is a fast example to test the non-blocking delays in a new language/compiler). One of the LEDs will be the one provided by Arduino board, the others will be mounted on a breadboard and connected to D6 and D7 pins of Arduino.

 

On Arduino Wiring language, this will look like the following (using millis(), of course):

 

const int ledPin =  13;      // the number of the LED pin
const int ledPin2 =  7;      // the number of the LED2 pin
const int ledPin3 =  6;      // the number of the LED3 pin

// Variables will change:

int ledState = LOW;             // ledState used to set the LED
int ledState2 = LOW;             // ledState used to set the LED2
int ledState3 = LOW;             // ledState used to set the LED3
long previousMillis = 0;        // will store last time LED was updated
long previousMillis2 = 0;        // will store last time LED2 was updated
long previousMillis3 = 0;        // will store last time LED3 was updated

// the follow variables is a long because the time, measured in miliseconds,
// will quickly become a bigger number than can be stored in an int.
long interval = 250;           // interval at which to blink (milliseconds)
long interval2 = 500;           // interval at which to blink (milliseconds)
long interval3 = 750;           // interval at which to blink (milliseconds)

void setup() {
// set the digital pin as output:
pinMode(ledPin, OUTPUT);
pinMode(ledPin2, OUTPUT);
pinMode(ledPin3, OUTPUT);
}

void loop()
{
// here is where you'd put code that needs to be running all the time.
// check to see if it's time to blink the LED; that is, if the
// difference between the current time and last time you blinked
// the LED is bigger than the interval at which you want to
// blink the LED.

unsigned long currentMillis = millis();
if(currentMillis - previousMillis > interval) {

// save the last time you blinked the LED
previousMillis = currentMillis;

// if the LED is off turn it on and vice-versa:
if (ledState == LOW)
    ledState = HIGH;
else
    ledState = LOW; // set the LED with the ledState of the variable:
digitalWrite(ledPin, ledState);
}

if(currentMillis - previousMillis2 > interval2) {
// save the last time you blinked the LED
previousMillis2 = currentMillis;
// if the LED is off turn it on and vice-versa:
if (ledState2 == LOW)
    ledState2 = HIGH;
else
    ledState2 = LOW;// set the LED with the ledState of the variable:
digitalWrite(ledPin2, ledState2);
}

if(currentMillis - previousMillis3 > interval3) {
// save the last time you blinked the LED
previousMillis3 = currentMillis;
// if the LED is off turn it on and vice-versa:
if (ledState3 == LOW)
    ledState3 = HIGH;
else
    ledState3 = LOW;
// set the LED with the ledState of the variable:
digitalWrite(ledPin3, ledState3);
}
}

 

Code size 1282 bytes.

 

 

On Luna, I've came up with something similar and I got a bad result (mainly caused by my Luna ignorance). I complained to the author of the language/compiler and he came up with two different (elegant) solutions:

 

First, using TaskTimer class on timer0 (the advantage against millis() of Arduino is that we can use any of the available timers):

#library "Library/TaskTimer.interface"

const F_CPU = 16000000
avr.device = atmega328p
avr.clock = F_CPU
avr.stack = 32

#define LED1 as PortB.5
#define LED2 as PortD.7
#define LED3 as PortD.6

LED1.mode = output,low
LED2.mode = output,low
LED3.mode = output,low

#define BASE_FREQ as 1   'ms
#define TASK1_TIME as 250
#define TASK2_TIME as 500
#define TASK3_TIME as 750

dim task1cnt,task2cnt,task3cnt as int16
TaskTimer.SaveMode = nosave
TaskTimer.Init(Timer0,1,BASE_FREQ)
TaskTimer.Task(0) = taskclock().Addr

avr.Interrupts.Enable
do
    if task1cnt=TASK1_TIME then
        LED1.toggle
        clr task1cnt
    endif

    if task2cnt=TASK2_TIME then
        LED2.toggle
        clr task2cnt
    endif

    if task3cnt=TASK3_TIME then
        LED3.toggle
        clr task3cnt
    endif
loop

procedure taskclock()
    asm ' save registers
        push _LA0
        xIn _LA0,SREG
        PushA1
        PushZ
    endasm
    task1cnt++
    task2cnt++
    task3cnt++
    asm ' restore registers
        PopZ
        PopA1
        xOut   SREG,_LA0
        pop    _LA0
    endasm
endproc

Code size 499 bytes.

The obvious advantage against Arduino is that you can use an even smaller microcontroller without any effort.

 

Second, we are using the Timer0 class directly:

const F_CPU = 16000000

avr.device = atmega328p
avr.clock  = F_CPU
avr.stack = 8

#define LED1   as PortB.5
#define LED2   as PortD.7
#define LED3   as PortD.6

LED1.mode   = output,low
LED2.mode   = output,low
LED3.mode   = output,low

#define BASE_FREQ   as 1   'ms
#define TASK1_TIME   as 250
#define TASK2_TIME   as 500
#define TASK3_TIME   as 750

dim task1cnt,task2cnt,task3cnt as word

const TIMER_PRESCALE   = 1024
const TIMER_PRELOAD      = 256 - word((F_CPU / TIMER_PRESCALE /(1000 / BASE_FREQ) + 0.5))

timer0.Clock = TIMER_PRESCALE
timer0.Value = TIMER_PRELOAD
timer0.isr = delaytask
timer0.Enable

avr.Interrupts.Enable

do
    if task1cnt=TASK1_TIME then
        LED1.toggle
        clr task1cnt
    end if

    if task2cnt=TASK2_TIME then
        LED2.toggle
        clr task2cnt
    end if

    if task3cnt=TASK3_TIME then
        LED3.toggle
        clr task3cnt
    end if
loop

isr delaytask nosave
    asm
        push   _LA0
        xIn      _LA0,SREG
        PushA1
        PushZ
    endasm
    timer0.value = TIMER_PRELOAD
    task1cnt++
    task2cnt++
    task3cnt++
    asm
        PopZ
        PopA1
        xOut   SREG,_LA0
        pop      _LA0
    endasm
endisr

Code size is 382 bytes.

 

Last Edited: Mon. Mar 2, 2015 - 02:19 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Those are both elegant solutions and the demo I did provides a third using "preemptive(!) RTOS-like tasks"

 

#library "Library/TaskKernel.Interface"

const F_CPU = 16000000
avr.device = atmega328p
avr.clock  = F_CPU
avr.stack = 4		'not more needed, only for initialize the basics

uart.baud = 19200
uart.send.Enable

'the stacks for the different tasks
'all sub-methods called from a task, uses the
'current active stack space of each task.
'** Note: the context switching of each task needs 33 bytes minimum! **
dim stack0(63) as byte
dim stack1(63) as byte

print 12
waitms 500
print "init"
'**********************************************************************
'!! ATTENTION - THIS TASKS ARE TRUELY PREEMPTIVE !!
'!! THE TASKKERNEL IS A TASK SCHEDULER FOR REAL PREEMPTIVE TASKS !!
'**********************************************************************
TaskKernel.Init(Timer1,2,10) '2 Tasks, 10ms TimeSlice between tasks
TaskKernel.Task(0,task0().Addr,stack0(63).Addr)
TaskKernel.Task(1,task1().Addr,stack1(63).Addr)

print "start kernel"
waitms 500
TaskKernel.Start()						'main program ends here! only tasks running!

'*************************************************************************
'* preemptive(!) RTOS-like tasks
'*************************************************************************
procedure task0()
'this is like a single main program
'here you can initialize something once
#define LED1	as PortD.5
LED1.mode	= output,low
'the main loop of this task (never exited)
do
LED1.toggle
waitms 100
loop
endproc
procedure task1()
'this is like a single main program
'here you can initialize something once
#define LED2	as PortD.6
LED2.mode	= output,low
'the main loop of this task (never exited)
do
LED2.toggle
waitms 20
loop
endproc

It really looks like a decent language but right now I'm up to my eyeballs in work rewriting my web site and another project on the block when that's done.

Last Edited: Mon. Mar 2, 2015 - 12:58 AM