Gentoo (and a little Ubuntu) on a MacBook Pro (5,3)

Getting Started

The Google-size summary: I’ve gotten Gentoo working on my MacBook Pro (5,3) again.

I’m going to attempt to recount all the little issues I ran across while installing Gentoo. If you notice I missed something, post it in the comments and I’ll try to figure it out and/or add it here. I also hope to add my findings to the Gentoo wiki, although the wiki is such an outdated mess that I’d almost feel like scratching the whole thing and having the Gentoo+MacBook community write their experiences back in.

As far as boot managers go, I really like rEFIt. It’s simple, has yet to crash, boots anything, and also will not give in to OSX when it gets overwritten.

First, the Gentoo CD doesn’t work. I didn’t bother to figure out why, because I figured it’s going to be some Gentoo CD specific issue, and at that moment I realized that I had better options. I have a copy of Ubuntu 11.04 on a disk already, and I know everything already works with it hardware-wise.

The Ubuntu 11.04 and 11.10 Live CD’s do not boot on a MacBook Pro 5,3 unless you do the following:

1. Boot into OSX, go to System Preferences -> Energy Saver, and select the high performance option (vs. the battery option). This turns on the fancy NVidia 9600M video card.
2. Reboot into the Live CD. When you see the accessibility logos on the bottom of the screen, hit the any key, then F6, and then Esc to get rid of that annoying menu that reminded me of Clippy for some reason.
3. Anyways, in that boot options line, before the “–”, write in “nouveau.noaccel=1″. You need this because nouveau is still in early development, and it thinks it knows how to handle your video card. It doesn’t.
4. For the wireless part, follow these instructions. (If you like the friendly “Additional Drivers”, once called “Restricted Drivers”, you can use that instead for Step 1.) Basically, install the bcmwl-kernel-source package, unload the b43 and ssb kernel modules, and  reload the wl module.

Go ahead and open the Gentoo Handbook, and start following the instructions. The rest of this blog post will be about the exceptions.

Partitioning

If you’re like me, you’ve got OSX and Windows already. You don’t want to mess them up. So why use Linux’s tools, when most of them don’t support GPT, and the others barely know what to do with them? That’s the case, anyway, as of the writing of this blog post.

This section is all thanks to this guy’s fantastic HOWTO on getting Ubuntu set up for EFI booting: http://www.rodsbooks.com/ubuntu-efi/index.html

So instead, use Disk Utility. If you didn’t plan ahead for Linux, resize to your heart’s content. Leave a space blank for Linux. (You can do this by adding a partition, sizing it, and then deleting it.) Don’t fret too much about whether Windows is first, last, or in the middle. I’ve found it really doesn’t matter, at least to Windows 7, contrary to a lot of guides out there.

Install gpt-fdisk 0.7.1 or better (just because those are the versions I used and I know work) in the Live CD:

1. Download gpt-fdisk, the only tool that does a good job: http://www.rodsbooks.com/gdisk/download.html.
2. In Ubuntu, install the development packages uuid-dev, libpopt-dev, and libicu-dev.
3. In the extracted gpt-fdisk directory, run “make” and you’ll have the gdisk binary.

Disk /dev/sda: 625142448 sectors, 298.1 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): [snip]
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 625142414
Partitions will be aligned on 8-sector boundaries
Total free space is 524949 sectors (256.3 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              40          409639   200.0 MiB   EF00  EFI System Partition
   2          409640       312513879   148.8 GiB   AF00  Customer
   3       468961280       625141759   74.5 GiB    0700  Untitled
   4       325358936       468699135   68.3 GiB    8301  Linux reserved
   5       312776024       325358935   6.0 GiB     8200  Linux swap

Partition 1 contains EFI boot files. It’s like the master boot record, except it’s a partition, which is so cool. Partition 2 contains OSX. partition 3 contains Windows, and partition 4 is Linux. Partition 5 was debatable for me. I’ve decided to use 6 precious GiB’s towards swap.

Now, I’m also using a hybrid MBR partition table, because Windows can’t really read GPT just yet (as far as I know; better safe than sorry, especially for existing installations). Here’s what that looks like:

Disk size is 625142448 sectors (298.1 GiB)
MBR disk identifier: [snip]
MBR partitions:

Number  Boot  Start Sector   End Sector   Status      Code
   1                     1       409639   primary     0xEE
   2                409640    312513879   primary     0xAF
   3      *      468961280    625141759   primary     0x07
   4             325358936    468699135   primary     0x83

This table’s job is simple: protect GPT partitions from being stomped on by operating systems that only understand MBR. Partitions 1-4 all correspond to partitions 1-4 in the GPT table.

So, here’s what you do:

1. Go ahead and use gdisk to create the Linux partitions you want. Try to leave 128M gaps between partitions (type “+128M” for start sector when prompted), because apparently OSX might not take major 10.X upgrades.
2. Now, open the recovery side of gdisk with the “r” command.
3. Type the “h” command.
4. At the prompt, type in at most three partition numbers that you want legacy operating systems and boot managers to see. For me, those were the partition numbers for the OSX partition, Windows partition, and the Linux root partition (since I keep /boot in root). If it asks whether you want to “Place EFI GPT (0xEE) partition first in MBR”, say yes.
5. When you’re all done, write out the tables and quit. Partitioning is done. If you were told the kernel wouldn’t get the new partition table, reboot.

Kernel

This was the easy part for me. I started fresh, and built a kernel from there. I’m linking my kernel configuration, which provides me with a working framebuffer, sound, wireless (assuming you install the broadcom-sta ebuild as well), touchpad, bluetooth, etc., etc. Basically, it’s about as compatible a kernel as I’ve ever gotten, while keeping the extravagant selections to a minimum.

gentoo-sources - 2.6.38-gentoo-r6

In order to review kernels, I go to /usr/src/linux, clean out the .config (or move it), run “make menuconfig” to get a default kernel for my system architecture, and then diff the “clean” .config against the downloaded one. That will tell you what kind of choices I made.

When you’ve got your kernel configured, do the usual “make -j3 && make modules_install && make install”. The Gentoo Handbook doesn’t tell you, but “make install” actually puts the kernel and all the other really-good-to-have extras in /boot for you. GRUB2 will actually appreciate those extras, so I strongly recommend “make install”.

GRUB2

Once you’re in your Gentoo chroot, and you’re about to build GRUB, here’s what you do:

1. You need to patch GRUB2′s ebuild so it can handle the “efi” USE flag, like so: http://neuvoo.org/users/javajake/grub-1.99_rc1.ebuild.patch (suggested patch; it’s so easy a caveman could do it).
2. Install this patched GRUB2, and I also highly recommend os-prober.
3. Now, this next part is up to you, but this made it simple for me. In the /boot directory, make an efi directory, and mount /dev/sda1, or wherever your EFI partition is, on /boot/efi. Check to be sure /boot/efi/EFI exists and has an APPLE directory. You might want to make a backup of this just in case.
4. Open /etc/default/grub (which may not exist yet) and add the following line. The important part is to make sure the grub directory is inside the all-capital-letters EFI directory.

   GRUB_PREFIX=/boot/efi/EFI/grub/

   The cool thing about GRUB2 is it auto-detects everything really, really well. As long as you make sure to set GRUB_PREFIX and stay consistent with this setting, GRUB2 will work like a dream. I had so many issues because I didn’t stay consistent.

5. Run grub-install, like this:

   grub-install --boot-directory=/boot/efi/EFI/

   It will install all the modules, fonts, etc., you need. Magical.

6. Right now, and every time you build a new kernel in /boot, run grub-mkconfig like this:

   grub-mkconfig -o /boot/efi/EFI/grub/grub.cfg

   You should get no errors. Again, it just works. I’m serious: if you’re using a simple partition scheme like I am, you don’t have to touch this.

That’s all there is to it. You can now boot Linux. When rEFIt comes up, it’ll give you one or two new options. Ignore the Linux selection (the one with the picture of the penguin) if that appears on your system, and pick the one that mentions “core.efi” instead.

Final Notes

• 3D acceleration is not working. The reason is simple. The nvidia driver conflicts with the efifb driver. However, you can’t just not use a framebuffer, because you need it to see the console. Why? Because GRUB2 auto-detects your framebuffer and configures all text output, from both GRUB2 and the kernel, to output to the framebuffer. Turning this off in GRUB2 is possible, but you need to display to something, and unfortunately that alternative is not obvious. Lastly, I tried both uvesafb and vesafb, and both fail to recognize the Pro’s framebuffer.
• The fans in the MacBook Pro manage themselves, thankfully, and run just like they do in OSX. I wouldn’t change that.
• All the fancy keys on the MacBook Pro are recognized. However, because you and I picked the EFI boot option, which is the cleanest, fastest, and easiest one to set up, the keyboard and screen backlights are not configurable from Linux, as of this writing. Thankfully, there’s a workaround. Simply set your backlights in OSX to something that works in most of your lighting conditions, and reboot into Linux. The hardware remembers.
• I wouldn’t touch pommed. I’ve tried it, and it works, but it conflicts with other automagic tools in KDE and GNOME, and it segfaults frequently. Besides, just about everything works without it. (I think the only thing I needed it for was the keyboard backlight, but I can easily do without to avoid pommed.)
• Because I use a desktop profile, and because I put INPUT_DEVICES=”${INPUT_DEVICES} synaptics” and VIDEO_CARDS=”nvidia nv vesa fbdev” in my make.conf, X configures itself. Again, magic. If you want to pick the video card you’re running on, see the Gentoo Handbook for cofiguring xorg.conf yourself. Use the BusID option in the Device section to pick the video card. Keep in mind that this may be very buggy. (Remember how we had to pick the 9600 card in OSX before Ubuntu would boot?)
• Sound works, if you stick with Intel HD audio drivers (hda-intel). If it’s not working, use “alsamixer -V all” to go on an unmuting spree. Also, the microphone doesn’t work until you turn it on. Don’t be fooled; it’s not the mic volume that’s the issue, it’s that you need to toggle it so it says “CAPTURE” right below the mic volume.