Building BLAG9002 from BLAG-DVD

From blag.wiki.aktivix.org

Jump to: navigation, search

This page documents how I did the new release of BLAG9002 from version 9001. It documents how to do a completely new release from a BLAGDVD. It should be quite similar to setting up a build based from RedHat 9 or Fedora Core 1.

Contents

Introduction

What you need to know

To build a fresh distro, you should be familiar with compiling applications, rebuilding RPMs, mounting & umounting ISOs, burning CDs, chowning/chmodding, shell scripts, rpm, Linux in general, etc. This doc writes lots of things just as commands without much explanation.... You will want over 20 gigs of space. It can be done in less, but is ever better with far more. A cluster will help with compiling too ;)

Prepare Environment

Install BLAG9001-DVD

Do a complete "Everything" install of BLAG9001 from the DVD.

Tweak your system

Do whatever you do to a fresh system to make it yours...

Import RedHat's GPG key

rpm --import /usr/share/doc/redhat-release-9/RPM-GPG-KEY

Set your $PATH

You need to add this to your $PATH in .bashrc: 

PATH=$PATH:/usr/lib/anaconda-runtime

Set environment variables

Not sure if this is needed (but add it to .bashrc): 

export LD_LIBRARY_PATH=/usr/X11R6/lib:$LD_LIBRARY_PATH

Set up Directories & Files

This setup is quite "personal" and doesn't follow filesystem standards, but is quite convenient for me. Most of the action happens in /BLAG and /usr/src/blag. I couldn't resist the tempation to have a /BLAG/BLAG/BLAG directory. You will want to chown the directories to the user developing BLAG instead of root. Make the following directories and symlinks.

CD/DVD Working Directories

mkdir /BLAG
chown yourusername:yourusername /BLAG
ln -s /BLAG/BLAG/ /BLAG/i386
mkdir -p /BLAG/DVD/i386/

BLAG scripts & misc files

mkdir -p /BLAG/bin
mkdir -p /BLAG/log
mkdir -p /BLAG/iso
mkdir -p /BLAG/docs
chown yourusername:yourusername /usr/src -R
mv /usr/src/redhat /usr/src/blag
ln -s /usr/src/blag /usr/src/redhat
mkdir /usr/src/blag/UPDATES/redhat9
mkdir /mnt/BLAG9001
mkdir /mnt/BLAG
mkdir /mnt/BLAG9001DVD

Copy BLAG9001 Files into Working Directories

Put a copy of BLAG-9001.iso and BLAG-9001-DVD.iso into /BLAG/iso

Mount the images

mount -o loop /BLAG/iso/BLAG-9001.iso /mnt/BLAG9001
mount -o loop /BLAG/iso/BLAG-9001-DVD.iso /mnt/BLAG9001DVD

Copy CD/DVD files into /BLAG dirs

cp -a /mnt/BLAG9001/* /BLAG/BLAG/
cp -a /mnt/BLAG9001DVD/* /BLAG/DVD/i386/

Build Custom RPMs

Set up Build Environment

Files Used

This sets things for compiling:
  • ~/.rpmmacros
This is a simple script that I have running in a terminal when I'm building stuff:
  • blag-compile-one-at-a-time-ok
This just makes a nice little beep sound so I know when a compile is done:
  • beepdone

Directories

For SRPMS I create a few different directories so I know where the SRPM came from. Examples:

  • /usr/src/blag/SRPM/DIST - SRPM provided by main developer
  • /usr/src/blag/SRPM/MDK - SRPM taken from Mandrake
  • /usr/src/blag/SRPM/DAG - SRPM taken from DAG
  • etc...

Install BLAG SRPMS

cd /BLAG/DVD/i386/SRPMS
rpm -Uvh *blag*.rpm

Compile & Build RPMs & SRPMS

Get new SOURCES

Download new source code for BLAG RPMs that need updating.

Edit foo.spec

Increment the Release: line in .spec file. Update changelog in .spec file.

Build RPM

rpmbuild -bb foo.spec

Build SRPM

rpmbuild -bs foo.spec


Update RPMS/SRPMS in CD/DVD working directories

  1. Add new RPMs to CD
  2. Add new RPMs to DVD
  3. Add new SRPMS to DVD

Download RedHat Update RPMs

Each time BLAG is released, it is sync'd to all the latest RedHat updates. Grab all the RedHat 9 updates from ftp://ftp.redhat.com/pub/redhat/linux/updates/9/en/os/ (preferably use a mirror). Download them to: /usr/src/blag/RPMS

Update Custom RPMs

Build the necessary custom RPMs and SRPMS to copy to CD/DVD.

Put new RPMs in CD/DVD working directories

To do this, the old RPMs must be removed and the new ones copied into place. It's quite tedious...a way to do this a bit more automajickly should be scripted up. The way I do it is to open 8 small xterms and do a step in each of them (I have them color coded to so I know whether I should be rm'ing or cp'ing in a particular window).


Here's an example of an update--it has to be repeated for each updated RPM. In this case, libxml RPMs are being updated from version 2.5.4-1 to 2.5.4-2.

Update RPMs on the host system

(Window 1) 

cd /usr/src/blag/RPMS/i386
rpm -Uvh libxml2-2.5.4-2.i386.rpm \ 
libxml2-devel-2.5.4-2.i386.rpm \
libxml2-python-2.5.4-2.i386.rpm

Delete old RPMs from CD Working Dir

(Window 2) 

cd /BLAG/BLAG/BLAG/RPMS
rm libxml2-2.5.4-1.i386.rpm libxml2-python-2.5.4-1.i386.rpm

Delete old RPMs from DVD Working Dir

(Window 3) 

cd /BLAG/DVD/i386/BLAG/RPMS

rm libxml2-2.5.4-1.i386.rpm \ 
libxml2-devel-2.5.4-1.i386.rpm \
libxml2-python-2.5.4-1.i386.rpm

Delete old SRPM from DVD Working Dir

(Window 5) 

cd /BLAG/DVD/i386/SRPMS/
rm libxml2-2.5.4-1.src.rpm


Copy New RPMs to CD Working Dir

(Window 6)


Use cp -p so the timestamp gets preserved. Note that -devel RPMs don't get copied to the CD, just the DVD. 

cp -p libxml2-2.5.4-2.i386.rpm \ 
libxml2-python-2.5.4-2.i386.rpm \
/BLAG/BLAG/BLAG/RPMS/

Copy RPMs to DVD Working Dir

(also Window 6) 

cp -p libxml2-2.5.4-2.i386.rpm \
libxml2-devel-2.5.4-2.i386.rpm \
libxml2-python-2.5.4-2.i386.rpm /BLAG/DVD/i386/BLAG/RPMS/

Copy SRPM to DVD Working Dir

(Window 7) 

cp -p libxml2-2.5.4-2.src.rpm /BLAG/DVD/i386/SRPMS/

Update Changelog

(Window 8)

Edit the changelog to reflect the new RPM. 

vi /BLAG/BLAG/Changelog

Rinse & Repeat

Now do this over & over & over for each RPM. If the RPM is not part of RedHat, you'll need to add it to comps.xml...

Prepare CD/DVD

Update /BLAG/VERSION

Lots of the BLAG scripts reference this file so they know what version they are working on. Create it: 

echo 9002 > /BLAG/VERSION


Test releases have a lower major version number than the target release followed by a decimal place and 5 more digits. Plenty of space to work with. ;) So the first test release, or test master of 9002 would be: 

echo 9001.00001 > /BLAG/VERSION


Then .00002 etc. until it is ready to tag it 9002--the actual release.

Write RELEASE-NOTES and README

  • Rename RedHat's RELEASE-NOTES and README to RELEASE-NOTES-RH and README-RH
  • Write a new RELEASE-NOTES
  • Write a new README 
cp -p /BLAG/BLAG/Changelog \ 
/usr/src/blag/SOURCES/blag-Changelog

cp -p /BLAG/BLAG/RELEASE-NOTES \ 
/usr/src/blag/SOURCES/blag-RELEASE-NOTES

cp -p /BLAG/BLAG/README \
/usr/src/blag/SOURCES/blag-README

Also copy them over to the DVD 

cp -p /BLAG/BLAG/README /BLAG/DVD/i386/
cp -p /BLAG/BLAG/Changelog /BLAG/DVD/i386/Changelog 
cp -p /BLAG/BLAG/RELEASE-NOTES /BLAG/DVD/i386/RELEASE-NOTES

Update release RPMs

Update redhat-release to /BLAG/VERSION, increment Release: line in SPECfile and rebuild redhat-release. RedHat's /etc/redhat-release and Fedora's /etc/fedora-release is kept to insure easy upgrades from BLAG to Fedora.


Update blag-release.

Check various Scripts

For DVD make sure mk-iso-DVD has correct version. In fact every script you run should be checked to make sure it is sane. Lots of things get commented in & out as the process is moving along...

Update BLAG scripts on DVD

The BLAG scripts from /BLAG/bin are copied to DVD so people can re-create BLAG or their own distro based on BLAG.

copy the latest /BLAG/bin and /BLAG/docs to DVD. Remove "old" and "bak" and "cluster" directories and mailing-list.

comps.xml

Welcome to comps.xml

make sure comps.xml is current...

The comps are edited in /BLAG/docs/anaconda/comps/. You edit the file comps-gingin.xml.in.

comps.xml for CD

cd /BLAG/docs/anaconda/comps/
make
cp -p /BLAG/docs/anaconda/comps/comps-gingin.xml \
      /BLAG/BLAG/BLAG/base/comps.xml

comps.xml for DVD

cd /BLAG/docs/anaconda/comps-DVD-9/
make
cp -p /BLAG/docs/anaconda/comps/comps-gingin.xml \
/BLAG/DVD/i386/BLAG/base/comps.xml


The comps.xml file must be further modified. This is done after running the script dependency-check described below.


Prepare CD Image

cd /BLAG/bin/
Generate list of RPMs for hdlist1 & hdlist2: 
./genhdlist-blag
Make sure there are no jacked-up dependencies (cross fingers here): 
./dependency-check

dependency-check will output 2 files and a few lines of shell script to run if everything is fine (yes, this is ugly...). To know if dependency-check ran ok. Look at this file: 

ls -l /BLAG/BLAG/BLAG/base/missing-dependencies

If it is 0 bytes, you are in luck and there are no missing dependencies. If it has content, read the file and fix the errors. If there are no missing dependencies (e.g. a 0 byte file), then you need to run these commands to get the file ready for use: 

cd /BLAG/BLAG/BLAG/base/
grep -v "</comps>" comps.xml > tmpfile
mv tmpfile comps.xml
cat deps.xml >> comps.xml
echo "</comps>" >> comps.xml
rm deps.xml
rm missing-dependencies

Note these commands are output by the script itself, so you can cut & paste them.


Update apt-get's pkglist: 
./blag-genbasedir

Prepare DVD Image

cd /BLAG/bin/
Generate list of RPMs for hdlist1 & hdlist2: 
./genhdlist-blag-DVD
Make sure there are no jacked-up dependencies: 
./dependency-check-DVD

dependency-check will output 2 files and a few lines of shell script to run if everything is fine (yes, this is ugly...). To know if dependency-check ran ok. Look at this file: 

ls -l /BLAG/DVD/i386/BLAG/base/missing-dependencies

If it is 0 bytes, you are in luck and there are no missing dependencies. If it has content, read the file and fix the errors. If there are no missing dependencies (e.g. a 0 byte file), then you need to run these commands to get the file ready for use: 

cd /BLAG/DVD/i386/BLAG/base/
grep -v "</comps>" comps.xml > tmpfile
mv tmpfile comps.xml
cat deps.xml >> comps.xml
echo "</comps>" >> comps.xml
rm deps.xml
rm missing-dependencies

Note these commands are output by the script itself, so you can cut & paste them.

Update apt-get's pkglist: 
./blag-genbasedir-DVD

Build comps.rpm

rpmbuild -bs comps-blag.spec
rpmbuild -bb comps-blag.spec
rm /BLAG/DVD/i386/SRPMS/comps-[whatever it is].rpm
cp -p /usr/src/blag/RPMS/i386/comps-9002-0-[DATE].i386.rpm \ 
/BLAG/BLAG/BLAG/base/comps.rpm
cp -p /usr/src/blag/SRPMS/comps-9002-0-[DATE].src.rpm \
/BLAG/DVD/i386/SRPMS

Master CD/DVD

umount loopbacks

The linux kernel by default can only mount 8 loopback devices. This can be modified when you compile your kernel or when you insmod the loopback module. The script that creates the anaconda installer image uses a few loopback devices, so you need to make sure they are not all in use. Example: 

umount /mnt/boot
umount /mnt/BLAG9001
umount /mnt/BLAG9001DVD

BLAG-it-all

BLAG-it-all creates the actual BLAG .iso that folks download and burn to CD. This is the final product. The script must be run as root--sudo will not work! This script calls a variety of other scripts that:

  1. Sets up some enviroment variables
  2. Whacks a bunch of old directories/files
  3. chowns stuff
  4. Runs genhdlist-blag (again)
  5. Runs buildinstall-blag which creates the anaconda installer files hdstg2.img, netstg2.img, stage2.img.
  6. Runs mk-iso which creates an .iso image of /BLAG/BLAG and also
  7. mounts the .iso and some .img stuff for you to peek at.

BLAG-it-all-DVD

BLAG-it-all creates the actual BLAG DVD .iso that developers download and burn to DVD. This is the final product. The script must be run as root--sudo will not work! This script calls a variety of other scripts that:

  1. Sets up some enviroment variables
  2. Whacks a bunch of old directories/files
  3. chowns stuff
  4. Runs genhdlist-blag (again)
  5. Runs buildinstall-blag which creates the anaconda installer files hdstg2.img, netstg2.img, stage2.img.
  6. Runs mk-iso which creates an .iso image of /BLAG/DVD/i386 and also
  7. mounts the .iso and some .img stuff for you to peek at.

MD5SUM

md5sum BLAG-9002.iso >> /BLAG/iso/MD5SUMS
md5sum BLAG-9002-DVD.iso >> /BLAG/iso/MD5SUMS

Burn CD

cdrecord-blag
OR 
cdrecord speed=4 -v -data dev=cdrom /BLAG/iso/BLAG-9002.iso
Check that the disc burned OK 
/usr/lib/anaconda-runtime/checkisomd5 /dev/cdrom

Burn DVD

cdrecord-blag-DVD
OR 
dvdrecord -v -eject -data -dao dev=cdrom \
/BLAG/iso/BLAG-9002-DVD.iso
OR, if dvdrecord doesn't work with your DVD burner (from dvd+rw-tools which isn't in 9001): 
growisofs -Z /dev/scd0=/BLAG/iso/BLAG-9001-DVD.iso

Test

Install & play.

Upload to servers

Do the stuff needed on the servers...

Announce

Annouce release to:

  • blag-hatch
  • freshmeat
  • lwn
  • distrowatch

Repeat

Start at step one and repeat for 9003. ;)

Retrieved from "http://blag.wiki.aktivix.org/Building_BLAG9002_from_BLAG-DVD"
Personal tools