AS6 & Multi to speed up the extremely slow build?

Go To Last Post
8 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Is it possible to implement and use multiple makefiles-threads (i don't know what is the correct way to say it, i am not software engineer) to compile a project with many C & CPP files when there is multicore >2 cores CPU in AS6?

The first disadvantage in my opinion for AS5.1/6 is the compilation speed and i think the developers of MplabX had similar compilation speed problem and they fixed by implementing multi make files for the build process.

The second drawback of AS5.1/6 is no data-breakpoints are implemented for all architectures (mega,xmega,uc3,cortex) and no tracing for AV32s and probably ARMs

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The make.exe that ships in 5.1 appears to support -j

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
The make.exe that ships in 5.1 appears to support -j

I think the build process produces and uses a temporal .bat file at users temp folder. Is it that file that uses the make.exe to build the project? If yes how can we mod it to make multiple compilations (multithread?) to boost the compilation process?

I don't know if its possible at all and the Software team of AS6 from Atmel needs to do that mod?

Is it possible if we use our modifed makefile in a AS5.1/6 to do that?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ultimately "make -f Makefile" (or rather an implied -f as that's the file it uses by default) is executed. If "make -j 4 -f Makefile" is used then it says "while processing this Makefile's rules and launching commands feel free to spread them across 4 cores so that 4 commands can be run at once". The structure of the Makefile rule hierarchy may dictate whether this accrues any benefit or not.

BTW I gotta ask why you are concerned? I've never seen an AVR project so complex that it took more than a minute or two to build the entire thing and that's obviously only on the occasion of it remaking all targets (the whole point of make is that not everything will be recompiled on each build). Usually with just a handful of changed files a rebuild takes seconds.

(In another life I work on a 4 core (A8+M3+M3+DSP) target where the DSP build alone can take about half an hour (from "clean"). For that we use jom.exe (that ships with QT) as a replacement for nmake.exe as it does spread the build across cores and we see about a 40% increase across the whole build process on a four core machine. But AFAIK the -j option on GNU make is pretty much doing what jom does that MSVC's nmake cannot so it should be as simple as adding -j to make to get a similar benefit - do not expect to reduce build time to 25% of what it was before though!)

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
Ultimately "make -f Makefile" (or rather an implied -f as that's the file it uses by default) is executed. If "make -j 4 -f Makefile" is used then it says "while processing this Makefile's rules and launching commands feel free to spread them across 4 cores so that 4 commands can be run at once". The structure of the Makefile rule hierarchy may dictate whether this accrues any benefit or not.

BTW I gotta ask why you are concerned?

Quote:
I've never seen an AVR project so complex that it took more than a minute or two to build the entire thing and that's obviously only on the occasion of it remaking all targets (the whole point of make is that not everything will be recompiled on each build). Usually with just a handful of changed files a rebuild takes seconds.

(In another life I work on a 4 core (A8+M3+M3+DSP) target where the DSP build alone can take about half an hour (from "clean"). For that we use jom.exe (that ships with QT) as a replacement for nmake.exe as it does spread the build across cores and we see about a 40% increase across the whole build process on a four core machine. But AFAIK the -j option on GNU make is pretty much doing what jom does that MSVC's nmake cannot so it should be as simple as adding -j to make to get a similar benefit - do not expect to reduce build time to 25% of what it was before though!)

I agree with the point of make, does that means that we should never use clean build (except first time)? It looks bad and its annoying to wait ~1min to compile the example projects the first time if you have quite fast PC.

Try the Control Panel demo for the EVK1100.
It takes more than 2mins to me with i5 2500 4Gb Ram 500GB sata HDD.

I see from Task Manager, the only process that uses 100% from 1 core all the build time (most of the time) is avrstudio so it looks like there is bottleneck of avrstudio to make.exe, cc1.exe and avr32-gcc.exe calls as they never use 100% from any of the other available idle cores. I am not sure for that maybe i am wrong.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Quote:

does that means that we should never use clean build (except first time)?

That is the point of make - otherwise why not simply use a batch file that just compiles everything always?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You will get an option from "Tools -> Options -> Builder -> GNU Make -> MakeParallelExecutionOfBuild" from Atmel Studio 6.1 beta release to enable/disable parallel execution of compilation of files.

You can find a beta release of Atmel Studio 6.1 at
http://www.atmel.com/tools/atmel...

Regards,
Deena

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The AS6.1 beta has just been posted online:
http://www.atmel.com/tools/atmel...

- Dean :twisted:

Make Atmel Studio better with my free extensions. Open source and feedback welcome!