Compiling Magic Lantern on a Macintosh

[dfort]

Setting up a Magic Lantern development environment on a Macintosh

For the terminally impatient try Danne's Magic lantern development compiler tool(Mac OS). It includes the quick install script originally written for this tutorial. You can still get just the install script and skip this tutorial--scroll down to the bottom of this first post.

Newsflash
Apple released macOS Catalina, 10.15 so these instructions no longer work. However, there are some fixes in the works.

Before we begin did you know that all Macs have development tools already installed yet they are not available to the user? If you have a "virgin" Mac that you have never used for development try this, open up a terminal (located in Applications under Utilities) and type:

Code: [Select]

which gcc
Chances are that it will return /usr/bin/gcc, gcc is the compiler to build Mac binaries and one of the tools that we'll use to build Magic Lantern. So we're all set, right? Ok, not that easy try typing in gcc to launch the compiler:

We don't really need the full Xcode development environment so simply click the highlighted "Install" button, accept the licence to activate the developer tools. If you have already installed Xcode you might still need to install the command tools. Use this terminal command:

Code: [Select]

xcode-select --install
This will install the developer tools in /Library/Developer/CommandLineTools/. These tools need to play nicely with other developer tools required by Magic Lantern. If you are using OS 10.14 (Mojave) or newer there's another step we need to take so all these tools work nicely together:

Code: [Select]

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.*.pkg

Next, there are some additional tools needed to compile Magic Lantern and several ways to install them. The simplest is to use one of the popular package managers like Fink (http://www.finkproject.org/) MacPorts (https://www.macports.org/) or Homebrew (http://brew.sh/) Homebrew is the newest of the bunch so we'll be using that. Homebrew installs the tools so they are available in /usr/local/bin which happens to be included in every new Mac's default path. However, your Mac may or may not have a /usr/local directory. Another possibility is that you have a /usr/local but don't have permissions to install anything in there. Do a directory listing to see where you stand:

Rosies-Air:~ Fort$ ls -la /usr
total 8
drwxr-xr-x@   12 root  wheel    408 Oct 12 11:45 .
drwxr-xr-x    31 root  wheel   1122 Sep 30 21:56 ..
drwxr-xr-x     5 root  wheel    170 Aug 22 18:51 X11
lrwxr-xr-x     1 root  wheel      3 Sep 30 21:22 X11R6 -> X11
drwxr-xr-x     3 root  wheel    102 Aug 26 19:17 adic
drwxr-xr-x  1061 root  wheel  36074 Sep 30 21:28 bin
drwxr-xr-x   258 root  wheel   8772 Oct 12 11:45 include
drwxr-xr-x   285 root  wheel   9690 Oct 12 11:45 lib
drwxr-xr-x   183 root  wheel   6222 Sep 30 21:22 libexec
drwxr-xr-x   243 root  wheel   8262 Sep 30 21:22 sbin
drwxr-xr-x    46 root  wheel   1564 Oct 12 11:45 share
drwxr-xr-x     4 root  wheel    136 Sep 17 00:03 standalone

Notice that I used "ls -la" to get a detailed view of everything in the /usr directory. This system doesn't have a /usr/local directory so we'll have to create one. If you have a /usr/local directory but it is owned by root, you need to change permissions. Try this first:

Code: [Select]

sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
Don't be intimidated by long command lines, you're on a Mac so simply copy and paste it into the terminal window. Note that you will have to enter your password.

If it didn't work you are probably on OS-X 10.11 "El Capitan" or 10.12 "Sierra" or even something newer so you need to temporarily turn off disable System Integrity Protection (SIP) in order to create a /usr/local you can use.

1. Restart the computer, while booting hold down Command-R to boot into recovery mode.
2. Once booted, navigate to the "Utilities > Terminal" in the top menu bar.
3. Enter csrutil disable in the terminal window and hit the return key.
4. Restart the machine and System Integrity Protection will now be disabled.

Now in the terminal enter this command:

Code: [Select]

sudo mkdir /usr/local && sudo chflags norestricted /usr/local && sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
What you are looking for is a /usr/local directory with your name on it:

drwxr-xr-x     2 Fort  admin     68 Oct 12 12:52 local

Don't skip this next step -- go back into recovery mode and this time type csrutil enable in the terminal.

Now that we've got a /usr/local that we can use, install Homebrew. Ok--if you are really itching to get up and running you can now just skip to the bottom of this post and run the quick installation script. Go!

If you want to understand what we're doing any why, keep following this tutorial:

First of all this command will download and run a ruby script to install a basic Homebrew setup:

Code: [Select]

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Next start adding some of the tools needed to build Magic Lantern.

Code: [Select]

brew install python wget mercurial
What's the deal with Python, isn't it already installed? Yes, but getting the Apple version configured to work with the ML install scripts is more trouble than it is worth so we'll just use the Homebrew version. In addition, we're going to get some python scripts using the package manager that Apple left out of their version of Python.

Code: [Select]

pip install docutils
pip2 install docutils
pip3 install docutils

That's not a mistake--we want all three versions of docutils.

If for some reason you'd rather use Apple's version of Python, I've included instructions on how to set it up at the bottom of this post.

Although the compiler that comes with the Mac can compile ML there are reasons to use a different version. Technically it isn't really gcc but another compiler called "clang" and if you try using it to compile ML you'll probably get a bunch of clang error messages. In addition, Homebrew switched to version 6 of gcc and that's causing some issues with ML so we'll go with good old version 5.

Code: [Select]

brew install gcc@5
Two versions of gcc? Won't that confuse the system? Homebrew is designed to work with the system so the gcc that you just installed is called gcc-5. This requires some special handling that we'll get to a little later.

We're not not done with compilers, we'll need yet another version of gcc to compile binaries that will run on the ARM architecture used in our Canon cameras. To figure out which ARM compiler to get refer to the ML code, specifically: Makefile.user.default -- ARM_PATH=~/gcc-arm-none-eabi-4_8-2013q4 notice that little squiggly mark after the equal sign? That means that the path starts in your home directory. Ok, so much for the lecture let's get it and put everything in its place.

Code: [Select]

cd ~ && wget -c https://launchpad.net/gcc-arm-embedded/5.0/5-2016-q3-update/+download/gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2 &&  tar -jxf gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2 &&  rm gcc-arm-none-eabi-5_4-2016q3-20160926-mac.tar.bz2
Note that some of the branches will be looking for a different version of the ARM compiler. You can install multiple versions and the build scripts will search out the preferred version. Let's get 2013-q4 because several of the development branches use that version:

Code: [Select]

cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2
Before doing this next step make sure you are in the directory where you want to save the magic-lantern source code.

Finally, time to grab the ML source code and start building. There are plenty of resources on the forum but basically, to get the latest unified branch:

Code: [Select]

hg clone https://bitbucket.org/hudson/magic-lantern
cd magic-lantern
hg update unified
We're almost ready to build Magic Lantern but first we've got to make one small change first. Go into the magic-lantern directory, find the file named Makefile.user.default and you'll see this block of code:

Code: [Select]

#
# Host compiler settings
#
HOST_CC=$(shell which gcc)
HOST_LD=$(shell which ld)
HOST_AR=$(shell which ar)

Now make a new file called Makefile.user and put in just these lines:

Code: [Select]

#
# Host compiler settings
#
HOST_CC=gcc-5
HOST_LD=gcc-5
HOST_AR=$(shell which ar)

Warning: DON'T EDIT Makefile.user.default -- Put the changes you want to make to Makefile.user.default in your Makefile.user file. Think of this as a way to customize the Magic Lantern build environment without modifying the code.

Another area you might want to customize is which modules to include when making a new build. The modules that the developers decided you probably want are in the modules directory in a file called Makefile.modules.default but maybe you'd like to add another one, take for example bulb_nd. To do this without modifying any Magic Lantern code create a file in the modules directory called Makefile.modules.user and put this in it:

Code: [Select]

ML_MODULES_DYNAMIC += bulb_nd
Now let's build Magic Lantern. Here's the way I do it for the EOSM:

Code: [Select]

cd ~/magic-lantern/platform/EOSM.202/
make clean && make zip
So you want to start hacking away on the code? There's lots of choices when it comes to text editors. You can even use TextEdit though I wouldn't recommend it because you have to turn off all the "smart" features so it doesn't destroy your source code. A better choice is Aquamacs. I have no financial interest if you get it or not. Hey, it is free. What about the editor that comes with Xcode? You can certainly install the full Xcode app that includes a very capable editor though it is quite a beast to master. It is also quite a heavy download. Of course if you are a real propeller head there's all the old favorites like vi, emacs, pico and nano already installed for you.

Some other software you might want to look into if you are going to get serious and contribute back to the Magic Lantern project is SourceTree. It supports the bitbucket repositories used by Magic Lantern and it has a nice graphical interface showing the various branches in ML. You're on a Mac, not everything has to be done on the terminal.

Added bonus --  Compiling Magic Lantern command line tools a.k.a. "Module Helpers"

The Magic Lantrn source code comes with some applications that you can run in the terminal or incorporate them in your own programs. Both the cr2hdr Lightroom plugin and Danne's cr2hdr-r workflows use--surprise, cr2hdr which is a part of the dual_iso module. There is also a raw2dng, mlv_dump and a few others. Your system is already configured to compile these tools. In fact if you make a build for your camera the Makefile script builds a few of them for you. For example, in the terminal go to magic-lantern/modules/dual_iso and type:

Code: [Select]

./cr2hdr
We didn't give it any instructions so it simply lists a short list of the available options that it has available. It is beyond the scope of this tutorial how to use the various command line tools but search through the forum and wiki for instructions on how to use the command line tools.

If you find something interesting in the modules you'd like to try out like say raw2dng.c and there isn't a binary for it. Check out the Makefile and see if there is a "rule" to build it. In the case of raw2dng simply type:

Code: [Select]

make raw2dng
and presto, you have a Mac binary to play with.

Extra Added bonus --  Cross compile to Windows

Let you in on a little Magic Lantern secret. Most Windows binaries aren't compiled in Windows. Sure, there are developers using Windows systems but they are either running a Linux distribution inside of a "VirtualBox" or Cygwin which is a self contained environment that functions within Windows. Only a MinGW or MinGW-64 Magic Lantern development setup might be considered Windows native. In fact if you want the best environment to work with open source projects like Magic Lantern set up a Linux system but one issue is that although a Linux system can be easily set up to compile Windows binaries it is not so easy to do Mac binaries. Anyway, to make Windows binaries on a Mac you need yet another gcc compiler like the ones maintained by the MinGW project. MinGW is short for Minimalistic GNU for Windows. What does GNU stand for? GNU is a recursive acronym for GNU's Not Unix. If you check out Apple documentation you will see that they claim that OS-X is Unix. Geek alert - instead of getting caught up in techno-babble how about setting up a cross compiler on your Mac already and start building Windows binaries.

You will need a darwin (OS-X) build of the MinGW compiler. Homebrew to the rescue.

Code: [Select]

brew install mingw-w64
This give you a full 32-bit and 64-bit Windows cross compiling setup ready to go. Now to make a Windows binary all you need to do is add the ".exe" file extension when you invoke make. For example:

make cr2hdr.exe

Being able to create Windows binaries on a Mac is so easy that I am including it on the quick installation script at the bottom of this post, though you'll need to uncomment that line first.

Tip - Removing pesky "._" files from your card

Filenames beginning with a dot are "invisible" and normally you won't even know they are there. Every time you copy files from a Mac file system to another type of filesystem it creates these "dot" files to store some information that is not normally used in other filesystems. If you are using lua on a memory card that is formatted as a FAT16 or FAT32 filesystem (usually cards under 64GB), you may see some strange messages on your screen when your camera starts up with the lua module enabled. It isn't a big deal and dmilligan put in some code to ignore files that begin with a dot but it is rather annoying. Apple actually included a utility to clean up this mess. Assuming your card is named EOS_DIGITAL, here's how to run it:

Code: [Select]

dot_clean -m /Volumes/EOS_DIGITAL
The "-m" option also removes files that start with an underbar.

Tip - Uninstalling Homebrew

Code: [Select]

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"You might also want to clean out your /usr/local directory for any stragglers. Just remember that if you are on "El Capitan" or "Sierra" and you delete /usr/local you will have to do the Command-R csrutil enable/disable song and dance to get it back.

Tip - Configuring Apple's Python to work with ML

Note: I don't use this and things might have changed so you're on your own if you want to experiment with this.

Makefile.user.default specifies that PYTHON=python2. Of course you can change that or make an alias for python2 but if you are comfortable disabling System Integrity Protection (SIP) go ahead and do that then:

Code: [Select]

ln -s /usr/bin/python /usr/bin/python2
The reason for installing the Homebrew version of Python is because Apple left out a Python utility called "pip" that makes it easy to install docutils. Follow these steps to install pip and docutils:

Code: [Select]

wget https://bootstrap.pypa.io/get-pip.py

Code: [Select]

sudo -H python get-pip.py

Code: [Select]

sudo -H pip install docutils
I like to do some cleanup. The docutil scripts are owned by root so let's change that and make sure that we've got permissions for everything in /usr/local:

Code: [Select]

sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
You're done with the get-pip.py

Code: [Select]

rm get-pip.py
Quick installation script.

If you have a Mac with a /usr/local directory and would like to get everything up and running in a hurry, you can use Danne's menu driven compiler tool or download just the quick install script from his repository.

Let's download and run the script using only the terminal. First we'll use curl (copy URL) because it is available on a vanilla Mac. This command copies the script from Danne's repository and saves it to a file named mac_ml.sh

Code: [Select]

curl https://bitbucket.org/Dannephoto/compiler/raw/default/Compiler.app/Contents/mac_ml.sh > mac_ml.sh
In order to make the file executable you need to change the file's properties. Here's an easy way to do it:

Code: [Select]

chmod +x mac_ml.sh
Now it is ready to run. To run it from the current current directory you can start it this way:

Code: [Select]

./mac_ml.sh
You'll need to answer a few questions, enter your password and make decisions like if you want to install the Windows cross compiler and/or QEMU.

Happy coding!

[Danne]

That,s it. I, m going for the El Capitano upgrade. This is too good not to. Thanks for sharing this very comprehensive description.

[DeafEyeJedi]

Damn Daniel... How'd you just all of a sudden get me into this mode when I'm just literally like "That's it -- I'm gonna jump onto the Captain's boat and just sail along with y'all" and THANK YOU for your contributions on this yet another remarkable achievement of yours for this wonderful community.

[dfort]

This is fairly new stuff so I've been making changes to the tutorial. After doing the section on the Widows cross compiler I realized that the stock Apple gcc compiler that comes with "El Capitan" can be used to build ML. It saves a few minutes setting up the system and avoids conflicts between the various compiler versions.

[dmilligan]

The built in Apple compiler is clang/llvm. Last I checked, it would not compile most of the command line tools (cr2hdr, raw2dng, etc.) because those tools use non-byte aligned (aka 'bit field') structs (for processing 14-bit raw data). Supposedly bit-field structs are deprecated in clang/llvm. I'm surprised you actually got it to work. Are you sure they actually compiled and are functioning correctly with clang?

[dfort]

Maybe the 10.11 update addressed that issue? I'm having no problems compiling those command line tools and I just checked cr2hdr and it seems to be working properly.

I've been doing some further testing and although I should have probably stopped once I had a working system I wanted to see how much I could simplify setting up the development environment. This probably isn't for experienced developers like you but rather for someone who just wants to try their hand at compiling ML. The setup requires no changes to Makefile.user.default.

There are a few more adjustments I'd like to make. I found out how to get docutils working without having to install python from Homebrew so the only packages needed are wget and mercurial.

Somewhere along the line I'm getting myself in a situation where everything works, even things that aren't supposed to. On the EOSM the dot_tune module is supposed to fail but it is building without so much as a warning message. Seems that the platform specific features are being ignored?

Figured it out. If you make other platforms that can use dot_tune then compile a platform that cannot use dot_tune it will give you a warning but still give you a dot_tune for the platform that cannot use it. Doing a "make clean" in the magic-lantern root directory fixes this problem.

[Danne]

Finally. I have all three binaries compiling, cr2hdr, mlv_dump, raw2dng. What a thrill! dfort, you,re the man. Thanks dmilligan for shedding light on clang. Exactly my problem also here with El Capitan.

So, I followed dfort steps but skipped the csrutil disable part after running this

Code: [Select]

sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/localGave admin rights and access right away to

Code: [Select]

usr/local/bin and suddenly I can see files in

Code: [Select]

usr/binDon,t think I can place binaries in usr/bin though. Tried copy over raw2dng. Still, compiling worked so does,t matter.
*copying files over to usr/local/bin works but not to usr/bin. I went from Yosemite to El Capitan so usr/local/bin was already there. Don,t know how a fresh El Capitan install behave.

To get rid of this


I had to run this. Thanks dfort!

Code: [Select]

brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv "$f" "${f/-5/}"; done
When you get to the warning:

Code: [Select]

Code: [Select]

override r-xr-xr-x  Fort/admin for /usr/local/bin/gfortran? (y/n [n])
Couldn,t overwrite though it chose not overwritten by itself. Doesn,t matter. Compiling worked anyway, yey!

Now, how will I remember these steps in a month 

[dfort]

Well I'm certainly not going to argue with dmilligan over which compiler to use or ignore Danne's success. I changed the instructions back to using Homebrew's latest gcc compiler.

I also included a tip on how to use the Python that is already installed with the Mac if there are any users out there that might be Apple purists.

[dfort]

Added a quick install script. The whole installation, including downloading the Magic Lantern source code should take about 2 minutes. Scroll to the bottom of the OP.

I also did some small changes to smooth out the installation process.

[Danne]

Very nice with an optional script solution . Thanks for putting it up.

[dfort]

Curious how long it takes to build the Windows cross-compiler? Here are the times it took for each component on a 1.4 GHz Intel Core i5 MacBook Air. It isn't nearly as painful on a more powerful system. Things will get better if the gcc-cross-mingw32 (i686) gets merged and they host a pre-compiled version. As far a I know Homebrew is also a volunteer project so like Magic Lantern, somebody has to do it an post a pull request. In the meantime if you plan on making Windows binaries on your Mac you'll just have to wait until it finishes building or someone builds it for you.

gmp4	4.2 minutes
mpfr2	101 seconds
libmpc08	47 seconds
isl011	34 seconds
cloog01820 seconds	20 seconds
gcc49	264.7 minutes
isl012	39 seconds
binutils-mingw32	2.2 minutes
mingw-headers32	17 seconds
gcc-mingw32	6.7 minutes
runtime-mingw32	61.7 minutes
gcc-cross-mingw32	393.3 minutes

Quite a difference from the 2 minute quick install script.

[DeafEyeJedi]

... this is why I love Macintosh -- even tho no one says it anymore.

[dmilligan]

Maybe the 10.11 update addressed that issue? I'm having no problems compiling those command line tools and I just checked cr2hdr and it seems to be working properly.

I just tried again compiling with clang and as expected I got this:

Code: [Select]

$ make -C modules/dual_iso/ cr2hdr
Updated HGVERSION
[ README   ]   module_strings.h
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [cr2hdr] Error 1

Code: [Select]

$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.0.72)
Target: x86_64-apple-darwin15.0.0
Thread model: posix

Perhaps you though you were compiling with clang when in fact you were using gcc?

[dfort]

Perhaps you though you were compiling with clang when in fact you were using gcc?

That could very well be. I was able to test it on a virgin Mac that never had any development tools installed on it and got the same results as you. I changed to gcc a while back on your recommendation and had not had any problems with it since. I guess if someone is only going to be building ML and none of the command line tools it wouldn't matter because (AFAIK) the only thing that the host system compiler is building is xor_chk.

[dfort]

Added some error checking to the quick install script. Just checks to see if there is a /usr/local directory before starting the installation.

[Danne]

Safety first . Well done.

[DeafEyeJedi]

Finally got it tackled down on the new MBP running El Captain and Thanks Daniel for holding up with me throughout the day!

Here are my very first Compiles ...



... and yet more to come later!

 

[swinxx]

Hello! Great! Could you also make a compile for 5Dmk3 123

[DeafEyeJedi]

Actually I just tried it and it wouldn't produce -- claiming that this file doesn't exist.



Went over to the Magic Lantern Folder in my root and here's what I noticed...



So it looks like until 5D3.123 are back in the Nightlies then I can start compiling them for you or anyone else.

[dfort]

You need to switch to the 5D3.123 branch. Look up the Mercurial command or easier still, download the Sourcetree app and switch branches with a double click on the branch you want to work with.

[DeafEyeJedi]

Ah, I figured that it would probably had to do with needing to switch to a different branch but TBH I wasn't quite sure how exactly to tackle that part just yet.

However, I was able to change the forks and able to use yours in order to compile for EOS-M which worked well.

Thanks for that, D!

[dfort]

@swinxx let's say you followed instructions and cloned the main repository:

Code: [Select]

hg clone -r unified https://bitbucket.org/hudson/magic-lantern
Then you went into the platform directory--

Code: [Select]

cd magic-lantern/platform/
and found out that there is no 5Dmk3 123

Code: [Select]

Rosies-Air:platform Fort$ ls
1100D.105 7D_MASTER.203
500D.111 EOSM.202
50D.109 Makefile
550D.109 Makefile.platform
5D2.212 Makefile.platform.base
5D3.113 Makefile.platform.extras
600D.102 Makefile.platform.map
60D.111 README-unmaintained.txt
650D.104 all
6D.116 firmsum.py
700D.114 unmaintained
7D.203

But if you look at the branches in the main repository, it is there - https://bitbucket.org/hudson/magic-lantern/branches/

As long as you are somewhere in the magic-lantern directory you can switch branches:

Code: [Select]

hg update 5D3-123
If you were doing some experiments and edited a piece of code you'll get this error message:

Code: [Select]

abort: uncommitted changes
(commit or update --clean to discard changes)
so just do what it says to discard your changes:

Code: [Select]

Rosies-Air:platform Fort$ hg update --clean 5D3-123
168 files updated, 0 files merged, 53 files removed, 0 files unresolved
Rosies-Air:platform Fort$ ls
1100D.105 7D.203
500D.111 7D_MASTER.203
50D.109 EOSM.202
550D.109 Makefile
5D2.212 Makefile.platform
5D3.113 Makefile.platform.base
5D3.123 Makefile.platform.extras
600D.102 Makefile.platform.map
60D.111 README-unmaintained.txt
650D.104 all
6D.113 firmsum.py
700D.113 unmaintained

Now you can cd into the 5D3.123 directory and compile it.

Code: [Select]

make clean && make zip
Of course if you are asking someone else to do it for you, that's not the point of this topic--this is for those who want to compile ML themselves. Just download the nightly which is currently:
https://builds.magiclantern.fm/jenkins/job/5D3.123/41/artifact/platform/5D3.123/magiclantern-Nightly.2015Apr28.5D3123.zip

That's right--no changes since April. You want changes? Fork the main repository, apply some of the changes others are doing to see if they work on your platform and if they do then submit a pull request to get that branch updated. You can also try and merge the 5D-123 into unified and see what happens. Once you get something that works, please share it.

How to do all of that is a bit beyond the scope of this topic but that's basically how you can get things get done--instead of asking someone else to do it for you.

[DeafEyeJedi]

Well.. Well.. Well.. How can I say this?

I feel like I've just flown back in Time after spending hours in Terminaland refreshing my memories of how artful those codings can be!

Dreadfully Artful, that is. But hey I admit it was definitely worth the hassle and now I can see why @dfort is the man he is today. His wife is a lucky lady! 

He's an incredible human being that I admire him so much as if he were my 8th grade teacher from Computer Science whom I should've kept in touch with over the years.

But he joined onto this community on February 25, 2014 and I reckon his user name @dfort popping up on certain threads that were rather important so I decided to keep a close eye on him and lord behold he made the threads more valuable to not only to himself but to all of us as well.

I truly appreciate the dedication & eagerness that you proactively provide into this community.

Okay enough of my babbling already and back on topic -- I decided to stick with your latest fork (Thanks for that!) since it has all the "bells & whistles" from the branch that you referred to by PM earlier.

Here's what I was able to accomplished in order to get 5D3.123 to work (somewhat):

Added your Fork in Terminal ...

Code: [Select]

hg clone -r EOSM__working https://bitbucket.org/daniel_fort/magic-lantern
Then I typed this to pull the 5D3.123's Branch ...

Code: [Select]

hg pull && hg update 5D3-123
Then I typed this to make the Compile ...

Code: [Select]

cd magic-lantern/platform/5D3.123/
make clean && make zip
... and VOLIA!!!

Code: [Select]

Seans-Mac-mini-6:~ SMJ$ cd magic-lantern/platform/5D3.123/
Seans-Mac-mini-6:5D3.123 SMJ$ make clean && make zip
[ RM ]  *.o *.d autoexec autoexec.bin magiclantern magiclantern.bin
[ RM ]  5D3_123.sym magiclantern.sym
[ RM ]  magiclantern.bin autoexec autoexec.map location.map
[ RM ]  autoexec-fir autoexec-fir.bin
[ RM ]  magiclantern.lds
[ RM ]  dietlibc.a newlib-libc.a newlib-libm.a gcc-libgcc.a
[ RM ]  version.c version.bin
[ RM dir ]  zip
[ RM ]  *.zip
../../Makefile.inc:79: remove /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ RM dir ]  /Users/SMJ/magic-lantern/platform/5D3.123/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip
[ CPP      ]   magiclantern.lds
[ AS       ]   entry.o
[ AS       ]   ../../platform/5D3.123/stubs.o
[ VERSION  ]   ../../platform/5D3.123/version.c
[ CC       ]   version.o
[ CC       ]   boot-hack.o
[ CC       ]   fio-ml.o
[ CC       ]   mem.o
[ CC       ]   ico.o
[ CC       ]   edmac.o
[ CC       ]   menu.o
[ CC       ]   debug.o
[ CC       ]   rand.o
[ CC       ]   posix.o
[ CC       ]   util.o
[ CC       ]   imath.o
[ CC       ]   electronic_level.o
[ CC       ]   cfn.o
[ CC       ]   gui.o
[ CC       ]   picstyle.o
[ CC       ]   exmem.o
[ CC       ]   bmp.o
[ CC       ]   rbf_font.o
[ CC       ]   config.o
[ CC       ]   stdio.o
[ CC       ]   bitrate-5d3.o
[ CC       ]   lcdsensor.o
[ CC       ]   tweaks.o
[ CC       ]   tweaks-eyefi.o
[ CC       ]   lens.o
[ CC       ]   property.o
[ CC       ]   propvalues.o
[ CC       ]   gui-common.o
[ CC       ]   chdk-gui_draw.o
[ CC       ]   movtweaks.o
[ CC       ]   menuhelp.o
[ CC       ]   menuindex.o
[ CC       ]   focus.o
[ CC       ]   notify_box.o
[ CC       ]   bootflags.o
[ CC       ]   dialog_test.o
[ CC       ]   vram.o
[ CC       ]   greenscreen.o
[ CC       ]   fps-engio.o
[ CC       ]   shoot.o
[ CC       ]   hdr.o
[ CC       ]   lv-img-engio.o
[ CC       ]   state-object.o
[ CC       ]   tasks.o
[ CC       ]   vsync-lite.o
[ CC       ]   tskmon.o
[ CC       ]   battery.o
[ CC       ]   imgconv.o
[ CC       ]   histogram.o
[ CC       ]   falsecolor.o
[ CC       ]   audio-ak.o
[ CC       ]   zebra.o
../../src/zebra.c:1134:12: warning: 'zebra_digic_dirty' defined but not used [-Wunused-variable]
 static int zebra_digic_dirty = 0;
            ^
[ CC       ]   vectorscope.o
[ CC       ]   beep.o
[ CC       ]   crop-mode-hack.o
[ CC       ]   ph_info_disp.o
[ CC       ]   flexinfo.o
[ CC       ]   screenshot.o
[ CC       ]   fileprefix.o
[ CC       ]   lvinfo.o
[ CC       ]   builtin-enforcing.o
[ CC       ]   ml-cbr.o
[ CC       ]   raw.o
[ CC       ]   chdk-dng.o
[ CC       ]   edmac-memcpy.o
[ CC       ]   console.o
[ CC       ]   tcc-glue.o
make -C ../../tcc
make[1]: Nothing to be done for `all'.
[ CC       ]   module.o
[ CC       ]   video_hacks.o
[ CC       ]   misc.o
[ CC       ]   afma.o
[ CC       ]   asm.o
[ AR       ]   strrchr.o
[ AR       ]   dietlibc.a
[ AR       ]   lib_a-setjmp.o
[ AR       ]   newlib-libc.a
[ CP       ]   newlib-libm.a
[ CP       ]   gcc-libgcc.a
[ LD       ]   magiclantern
[ SYMBOLS  ]   magiclantern.sym
[ CP       ]   5D3_123.sym
[ MKDIR    ]   install
[ CP       ]   install
[ VERSION  ]   ../../platform/5D3.123/version.bin
[ OBJCOPY  ]   magiclantern.bin
[ STAT     ]   magiclantern.bin
magiclantern.bin: 471120 bytes
[ CC       ]   reboot.o
[ LD       ]   autoexec

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x073148 0x000dc548 0x000dc548 0x00008 0x00008 R   0x4
  LOAD           0x000100 0x00069500 0x00069500 0x73050 0x870f4 RWE 0x100
[ OBJCOPY  ]   autoexec.bin
[ STAT     ]   autoexec.bin
autoexec.bin: 471760 bytes
cp autoexec.bin /Users/SMJ/magic-lantern/platform/5D3.123/zip/
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/fonts
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/settings
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/data
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/cropmks
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/logs
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/doc
cp ../../data/fonts/*.rbf /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/fonts/
cp ../../data/vram/*.lut /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/data/
cp ../../data/cropmks/*.bmp /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/cropmks/
cp ../../scripts/*.lua /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts/
cp ../../scripts/*.c /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts/
cp ../../scripts/*.py /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/scripts/
make -C ../../modules

Building module raw_rec...

Building module file_man...

Building module pic_view...

Building module ettr...

Building module dual_iso...
Updated HGVERSION
[ README   ]   module_strings.h
[ CC       ]   dual_iso.o
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000136c dual_iso_calc_dr_improvement
000013c4 dual_iso_set_enabled
000013e4 dual_iso_is_enabled
000013f4 dual_iso_is_active
00001414 dual_iso_get_dr_improvement
00001594 dual_iso_get_recovery_iso
000015b4 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3.123
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 5D3.113, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make[4]: *** [cr2hdr] Error 1

********************************************************
WARNING: module dual_iso failed to build, deleting
********************************************************

[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe

Building module autoexpo...

Building module arkanoid...

Building module silent...

Building module dot_tune...

Building module deflick...

Building module mlv_play...

Building module mlv_rec...

Building module mlv_snd...
make -C ../../modules install

Building module raw_rec...

Building module file_man...

Building module pic_view...

Building module ettr...

Building module dual_iso...
Updated HGVERSION
[ README   ]   module_strings.h
[ CC       ]   dual_iso.o
[ MODULE   ]   dual_iso.mo
[ STRIP    ]   dual_iso.mo
[ EXPORTS  ]   dual_iso.sym
0000136c dual_iso_calc_dr_improvement
000013c4 dual_iso_set_enabled
000013e4 dual_iso_is_enabled
000013f4 dual_iso_is_active
00001414 dual_iso_get_dr_improvement
00001594 dual_iso_get_recovery_iso
000015b4 dual_iso_set_recovery_iso
[ DEPENDS  ]   dual_iso.dep
Will load on:
    5D3.123
Not checked (compile ML for these cameras first):
    1100D, 500D, 50D, 550D, 5D2, 5D3.113, 600D, 60D, 650D, 6D, 700D, 7D, EOSM
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make[4]: *** [cr2hdr] Error 1

********************************************************
WARNING: module dual_iso failed to build, deleting
********************************************************

[ RM ]  dual_iso.o dual_iso.mo dual_iso.sym dual_iso.dep module_strings.h *.o *.d *.dep *.sym hgstamp
[ RM ]  cr2hdr cr2hdr.exe

Building module autoexpo...

Building module arkanoid...

Building module silent...

Building module dot_tune...

Building module deflick...

Building module mlv_play...

Building module mlv_rec...

Building module mlv_snd...
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/modules || echo "no problem"
cp ../modules/*/*.mo /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/modules/ || echo "no problem"
mkdir -p /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/doc
cp -v ../../doc/cam/* /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/doc
cp: ../../doc/cam/*: No such file or directory
make: [installdoc] Error 1 (ignored)
cp ../../doc/README.user /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/README
cp ../../doc/INSTALL.pdf /Users/SMJ/magic-lantern/platform/5D3.123/zip || echo "warning Unable to copy INSTALL.pdf"
cp: ../../doc/INSTALL.pdf: No such file or directory
warning Unable to copy INSTALL.pdf
cp ../../doc/UserGuide.pdf /Users/SMJ/magic-lantern/platform/5D3.123/zip || echo "warning Unable to copy UserGuide.pdf"
cp: ../../doc/UserGuide.pdf: No such file or directory
warning Unable to copy UserGuide.pdf
cp ../../src/logo.bmp /Users/SMJ/magic-lantern/platform/5D3.123/zip/ML/docs || echo "warning Unable to copy logo.bmp"
[ LD       ]   autoexec-fir
[ OBJCOPY  ]   autoexec-fir.bin
[ STAT     ]   autoexec-fir.bin
autoexec-fir.bin: 471760 bytes
python ../../../dumper/build_fir.py -r ../../../dumper/5D300123.FIR autoexec-fir.bin ML5D3123.FIR
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file '../../../dumper/build_fir.py': [Errno 2] No such file or directory
make: [build_fir] Error 2 (ignored)
[ RM ]  magiclantern-Nightly.2015Oct27.5D3123.zip
chmod -x /Users/SMJ/magic-lantern/platform/5D3.123/zip/autoexec.bin
cd /Users/SMJ/magic-lantern/platform/5D3.123/zip; zip -z ../magiclantern-Nightly.2015Oct27.5D3123.zip < \
../../../src/zip.txt -r .\
@printf "[ RM dir ] %s\n" " /Users/SMJ/magic-lantern/platform/5D3.123/zip/";
zip warning: name not matched: @printf
zip warning: name not matched: [ RM dir ] %s\n
zip warning: name not matched:  /Users/SMJ/magic-lantern/platform/5D3.123/zip/
  adding: autoexec.bin (deflated 49%)
  adding: ML/ (stored 0%)
  adding: ML/cropmks/ (stored 0%)
  adding: ML/cropmks/CineSco2.bmp (deflated 87%)
  adding: ML/cropmks/CrssMtr2.bmp (deflated 94%)
  adding: ML/cropmks/Passport.bmp (deflated 72%)
  adding: ML/cropmks/PhiPhoto.bmp (deflated 62%)
  adding: ML/cropmks/PhiVideo.bmp (deflated 68%)
  adding: ML/data/ (stored 0%)
  adding: ML/data/apsc8p.lut (deflated 17%)
  adding: ML/data/apsc8r.lut (deflated 7%)
  adding: ML/data/ff8p.lut (deflated 17%)
  adding: ML/data/ff8r.lut (deflated 11%)
  adding: ML/doc/ (stored 0%)
  adding: ML/docs (deflated 94%)
  adding: ML/fonts/ (stored 0%)
  adding: ML/fonts/arghlf22.rbf (deflated 90%)
  adding: ML/fonts/argnor23.rbf (deflated 84%)
  adding: ML/fonts/argnor28.rbf (deflated 84%)
  adding: ML/fonts/argnor32.rbf (deflated 86%)
  adding: ML/fonts/term12.rbf (deflated 64%)
  adding: ML/fonts/term20.rbf (deflated 81%)
  adding: ML/logs/ (stored 0%)
  adding: ML/modules/ (stored 0%)
  adding: ML/modules/5D3_123.sym (deflated 65%)
  adding: ML/modules/arkanoid.mo (deflated 51%)
  adding: ML/modules/autoexpo.mo (deflated 58%)
  adding: ML/modules/deflick.mo (deflated 53%)
  adding: ML/modules/dot_tune.mo (deflated 51%)
  adding: ML/modules/ettr.mo (deflated 56%)
  adding: ML/modules/file_man.mo (deflated 52%)
  adding: ML/modules/mlv_play.mo (deflated 62%)
  adding: ML/modules/mlv_rec.mo (deflated 53%)
  adding: ML/modules/mlv_snd.mo (deflated 56%)
  adding: ML/modules/pic_view.mo (deflated 48%)
  adding: ML/modules/raw_rec.mo (deflated 52%)
  adding: ML/modules/silent.mo (deflated 52%)
  adding: ML/README (deflated 61%)
  adding: ML/scripts/ (stored 0%)
  adding: ML/scripts/brack.c (deflated 52%)
  adding: ML/scripts/clock.c (deflated 60%)
  adding: ML/scripts/dump.lua (deflated 11%)
  adding: ML/scripts/dumpcfg.lua (deflated 28%)
  adding: ML/scripts/hello.c (deflated 27%)
  adding: ML/scripts/hello.py (deflated 42%)
  adding: ML/scripts/img.c (deflated 53%)
  adding: ML/scripts/keys.c (deflated 55%)
  adding: ML/scripts/print.lua (deflated 62%)
  adding: ML/scripts/sokoban.c (deflated 73%)
  adding: ML/scripts/test.c (deflated 69%)
  adding: ML/scripts/wb_shoot.lua (deflated 47%)
  adding: ML/settings/ (stored 0%)
  adding: ML5D3123.FIR (deflated 0%)
enter new zip file comment (end with .):
Seans-Mac-mini-6:5D3.123 SMJ$
So now I am not in the mood to upgrade my 5D3 to 123 because I am just too happy being on 113 so for those who are curious to try... Feel free to check my Dropbox if you like!

However, I noticed the "Dual-ISO Didn't Build" errors so I suspect that this is probably due to change of forks? <-- this only applies to the 5D3.123 build that I just made.

*edit* I just tested it again on another fork (unified branch) and still encountered this "Dual-ISO didn't build" error. Is there something else going on that I don't know about?

It all worked fine few days ago and now this is such a trip to me. Both Mac Mini (Yosemite) and MBP (El Captain) are producing the "Dual-ISO Didn't Build" errors from both branches... Really Strange?!

Sorry guys I'll dig into this some more tomorrow after finally catching up on my much needed sleep! I

[dfort]

You are using the clang compiler. When I was writing this tutorial I was experimenting to see if I could get by without having to install yet another compiler and thought it was working. You might have set up your system when I took out that step but dmilligan, got the same error that you just reported:

I just tried again compiling with clang and as expected I got this:

Code: [Select]

$ make -C modules/dual_iso/ cr2hdr
Updated HGVERSION
[ README   ]   module_strings.h
[ gcc      ]   cr2hdr
clang: error: unknown argument: '-mno-ms-bitfields'
make: *** [cr2hdr] Error 1

Code: [Select]

$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.0.72)
Target: x86_64-apple-darwin15.0.0
Thread model: posix

Perhaps you though you were compiling with clang when in fact you were using gcc?

So make sure you get it:

Code: [Select]

brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done
As long as you are sending out kudos--dmilligan is doing some amazing work and moving ML to a whole new level. And his development platform is a Mac.

[DeafEyeJedi]

Right. Figured. Well this is what I was dealing with all night last night and yet again this morning after following your suggestions (I did @dmilligan's instructions numerously of times) to no avails and it was rather more confusing than frustrating...



SO I have decided to just delete the GCC and Magic-Lantern folders and just rn the script that @Danne created for you in order to start from scratch again to see if that would help clear up the problem or did the recent update for El Captain from Apple somehow caused these errors?

*edit*

Went ahead and deleted those two Folders in root and ran the script from OP which is now giving me this... How exactly can I run Software Update through Terminal?

Code: [Select]

Seans-Mac-mini-6:~ DeafEyeJedi$ cat <<'EOF' > mac_ml.sh
> #!/bin/sh
> set -e
>
> #  mac_ml.sh
> # 
> #  Script to almost automatically install
> #  and configure a Magic Lantern development
> #  environment on a Macintosh.
> # 
> #  Reference:
> #  http://magiclantern.fm/forum/index.php?topic=16012.0
> #
> #  Daniel Fort (dfort) - 2015-10-15
> #  Thanks to Danne for his input
>
> if ! test -d /usr/local; then
>   echo "/usr/local directory not found."
>   echo "Follow the steps at the beginning of"
>   echo "tutorial to create /usr/local and"
>   echo "then re-run this script."
>   exit 1
> fi
>
> echo "/usr/local found -- "
> echo "continuing with installation."
>
> cd ~
> sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local
> sudo mkdir -p /usr/local/ && sudo chflags norestricted /usr/local/ && sudo chown $(whoami):admin /usr/local/ && echo sudo chown -R $(whoami):admin /usr/local/
> xcode-select --install
> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
> brew install python wget mercurial
> pip install docutils
> brew tap homebrew/dupes; brew install gcc --use-llvm && for f in /usr/local/bin/*-5; do mv -n "$f" "${f/-5/}"; done
> cd ~ && wget https://launchpad.net/gcc-arm-embedded/4.8/4.8-2013-q4-major/+download/gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && tar -jxf gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2 && rm gcc-arm-none-eabi-4_8-2013q4-20131218-mac.tar.bz2
> for f in /usr/local/bin/*.py; do ln -s $f "${f/.py/}"; done
> hg clone -r unified https://bitbucket.org/hudson/magic-lantern
> exit 0
> EOF
Seans-Mac-mini-6:~ DeafEyeJedi$ bash mac_ml.sh && rm mac_ml.sh
/usr/local found --
continuing with installation.

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

Password:
sudo chown -R DeafEyeJedi:admin /usr/local/
xcode-select: error: command line tools are already installed, use "Software Update" to install updates
Seans-Mac-mini-6:~ DeafEyeJedi$

Thanks again for the follow-up, D!