CustomBuilds

Intro

Have you already used Metro to build some Funtoo stages, by following the QuickStartGuide? Great! Now you’re ready to create your own custom builds.

Metro 1.1 has been organized so that creating a custom build is very easy to do. First, let’s look in the /etc/metro/builds directory:

# ls -l /etc/metro/builds
total 8
-rw-r--r-- 1 root root 479 Nov  4 21:13 ~funtoo.conf
-rw-r--r-- 1 root root 479 Nov  4 21:13 funtoo.conf
-rw-r--r-- 1 root root 425 Nov  4 21:13 gentoo.conf

The ~funtoo.conf file contains all information specific to the “unstable” Funtoo builds (with tilde), the funtoo.conf defines the stable Funtoo builds (we’ve just started working on Funtoo stable builds, so this may not work as of March 2009,) and the gentoo.conf file contains all information specific to the official Gentoo builds. Metro will place the results of these builds in /home/mirror/linux/BUILDNAME, or another location if you’ve redefined path/mirror to point to another location.

To create your own custom build of Funtoo, we’ll create a copy of the ~funtoo.conf file and name it my.conf:

# cd /etc/metro/builds
# cp ~funtoo.conf my.conf

Now, let’s look at the contents of my.conf:

[section local]

author: Daniel Robbins <drobbins@funtoo.org>

[section target]

build: ~funtoo

[section portage]

stable: ~
name: portage
MAKEOPTS: -j2
USE:

[when target/arch]

profile: default/linux/$[target/arch]/2008.0

[section emerge]

options: --jobs=4 --load-average=6
packages: dev-util/git net-misc/dhcpcd

[section snapshot]

type: git

[section git]

branch: funtoo.org
name: portage
remote: git://github.com/funtoo/portage.git
options: pull

[section metro]

options/stage: cache/package cache/compiler
target: gentoo

[section multi]

targets: git-snapshot $[multi/targets/core] openvz vserver

[section multi when multi/mode is full]

targets/core: stage1 stage2 stage3

[section multi when multi/mode is quick]

targets/core: stage3-quick

[section multi when multi/mode is freshen]

targets/core: stage3-freshen

Go ahead and change local/author to contain your name and email address. You’ll also see the target/build variable. This variable is used as the prefix for stage directories created in /home/mirror/linux. I use ~funtoo to prefix my funtoo unstable builds. If you are doing your own custom builds, try changing target/build to your username.

Now, let’s look at the portage section. If portage/stable is set to ~, then /etc/make.conf will automatically get configured to use ACCEPT_KEYWORDS="~[arch]". If you want to use the “stable” tree, then simply remove the ~, so the line reads @stable: @ (with nothing after the colon.)

Next, we come to portage/name. This defines both the name that will be used for creating snapshot targets for this build. Right now you can see that our unstable portage snapshots will be named with a file prefix of portage. Normally, it’s OK to keep this as-is. Also note that by default, Metro will place your Portage tree snapshots in $[path/mirror]/$[target/build]/snapshots.

Next, portage/MAKEOPTS. This will become the MAKEOPTS setting and is commonly used to enable parallel make by passing a -jX option, where X is a number. Next, portage/USE. This setting will cause the respective global USE flags to be defined in the target stage’s /etc/make.conf file. You would use it like this:

USE: X perl svn

If you don’t want to set any custom global USE flags, then retain the USE: line as it appears in the file. If you want to add values to the /etc/portage/package.use file, add something like this under [section portage]:

package.use: [
dev-util/git subversion perl cgi curl tk
net-print/cups zeroconf ppds samba
]

And, if you want to add custom entries to /etc/portage/package.keywords, add something like this under [section portage]:

files/package.keywords: [
sys-kernel/openvz-sources ~[target/arch]
sys-apps/portage ~$[target/arch]
app-admin/eselect-news ~$[target/arch]
app-admin/eselect ~$[target/arch]
media-sound/squeezecenter ~$[target/arch]
media-sound/alac_decoder ~$[target/arch]
dev-util/git ~$[target/arch]
]

Above, $[target/arch] will be replaced with either x86 or amd64, depending upon whether you are building a 32-bit or 64-bit version of Gentoo. Cool, eh?

You can also define custom package.unmask entries by defining a portage/files/package.unmask template, similar to portage/files/package.keywords above.

Now, the next variable, portage/profile, will define the Portage system profile that will be used for your build, one that resides in /usr/portage/profiles. Typically, you will only need to change this about once a year when Gentoo moves over to a new system profile – in most cases, you should leave it as-is.

Next, in [section emerge], we have emerge/options and emerge/packages. emerge/options allows you to define command-line options that get passed to the emerge command when it is run. Here, we are using the --jobs and --load-average options available in Portage 2.2 to perform parallel merges (building multiple packages at the same time.) If you are using a 2.1 version of Portage for your build, you’ll want to remove both these options and have a : as the last non-whitespace character on the line.

The emerge/packages option defines extra packages that will be added to your stage3 tarball. This is where you should add any extra stuff you want to build. Specify a package just like you would on the emerge command-line.

Next, we have our snapshot and git sections. I recommend that you use git for your Portage trees. If you want to use another portage tree, change git/remote to point to its location, such as the publicly-cloneable URI on github.com. If you want to use a local git repository that exists on your local fileystem, you can point it to the local filesystem path of the git repo you wish to use. Make sure that git/branch specifies the branch of the repo you want to use.

Also, it’s important to understand the importance of git/name. git/name defines the name of the cached git Portage tree that Metro will create in /var/tmp/metro/cache/cloned-repositories. If you are using your own Portage tree alongside Funtoo’s Portage tree, you will want to change git/name to something besides portage so that Metro will cache your tree to a unique directory.

name: mytree

Then, Metro will look for and cache your Portage tree in /var/tmp/metro/cache/cloned-repositories. I also recommend you keep the git/pull option enabled, as it is in our file right now. This will ensure that Metro will use git to pull the most recent changes into your tree before performing a snapshot. Otherwise you risk having stale information in your snapshot.

Advanced Customization

Now we’ll look at some more advanced functionality that is defined in the build file, above. First, we have the [section metro] part of the file, but let’s look at the second element first. The target: gentoo element tells Metro that the build scripts for any of our targets can be found in /usr/lib/metro/targets/gentoo. The fact that we have control over this parameter is extremely useful. Say, for example, that you decide to fork Gentoo or Funtoo and would like to build the system using another package manager. You could simply copy the contents of /usr/lib/metro/targets/gentoo to /usr/lib/metro/targets/newfork, and then change the metro/target setting to newfork. Metro would then use ’’your’’ custom build scripts rather than the ones that ship with Metro.

The remaining element under [section metro] defines what internal Metro options are enabled for various target types. The build targets of stage1, stage2 and stage3 have a target/type of stage. So to enable features for these target types, we would use the metro/options/stage variable as appears in the file.

The various options supported in Metro 1.3 include:

Building

So, at this point you have your own my.conf file describing your build. But how do you “seed” your build using an already existing funtoo or gentoo stage3? This is actually pretty easy to do – all you need to do is to set up your mirror structure properly. By default, Metro will use /home/mirror/linux as the directory where it looks for things it needs, as well as where it put things it creates — we call this the “mirror” directory, as this directory tree is typically what you copy out to web mirrors. In order to seed from another type of build, all you need to do is to put the stage3 inside an appropriate directory structure in /home/mirror/linux and set your builds’ .control files properly, and Metro will find it and use it. Let’s see how this is done.

Let’s assume that the “seed” stage3 you want to use for your amd64 build is an unstable funtoo amd64 stage3 built on 2009.04.01. You would place this stage3 tarball in a /home/mirror/linux/~funtoo/amd64/~funtoo-amd64-2009.04.01 directory. To get your “my” amd64 build to use this unstable funtoo amd64 stage3 as an initial seed stage3 for your “my” build, you’d do the following:

# install -d /home/mirror/linux/my/amd64/.control
# echo "2009.04.04" > /home/mirror/linux/my/amd64/.control/lastdate
# echo "~funtoo" > /home/mirror/linux/my/amd64/.control/build
# echo "amd64" > /home/mirror/linux/my/amd64/.control/subarch

That’s all there is to it. Now, when you run your “my” amd64 build, it will use the 2009.04.01 amd64 ~funtoo build as a seed stage3:

# /usr/lib/metro/scripts/ezbuild.sh my amd64

After your “my” amd64 successfully completes, Metro will update the control files so that lastdate contains today’s date, build contains “my”, and subarch contains “amd64”. In other words, after the first successful “my” amd64 build completes, Metro will now use your new “my” amd64 stage3 as a seed stage for tomorrow’s “my” amd64 build, rather than continuing to use the unstable funtoo seed stage3. And the day after that, when you create another “my” amd64 release, it will be seeded from the last successfully-built “my” amd64 stage3.

This is default behavior for Metro, and why Metro uses external, dynamic “control” files to define what seed stage to use. This approach ensures that your new stage3’s are utilized as seed stages automatically, without any manual steps required by you.

Happy building!