What ?

I was using debmirror for some really time and pretty depend on it although debmirror just didn't serve some part of mirror obligatory. And i stuck on that now.

What kind of part ?

Dependencies are not handled and the integrity of the host system is not assured, i.e. updating a mirror should automatically affect packages on the serving host as well, but this is not done with debmirror.

So ?

I want to create a partial Debian mirror using the dependency handling of apt. The integrity of the serving host is not affected, as a chroot environment is used.

I found people in debian.org just create old way but nice one, name W. Borgert. I'm referring on his doc to create on my own.

How to create mirrorring system ?

I am doing it all for testing dist.

Create a chrooted debian system that will serving only mirroring purpose.

Directory description (you can change to what your preferrable place):

• The chroot is /var/local/chroot/
• The mirror is under /var/local/mirrors/ inside of the chroot.
• The mirror is symbolically linked to the same place outside of the chroot:

ln -s /var/local/chroot/var/local/mirrors/ /var/local/

All setup things could be done simply with this create_mirror.sh script below:

#!/bin/sh

# the root directory of the mirror system
CHROOT=/var/local/chroot
# the mirror directory inside of the chroot
MIRRORDIR=/var/local/mirrors
# the Debian archive next to you
DEBIAN=http://ftp.de.debian.org/debian

####

OUT=adduser,aptitude,apt-utils,at,base-config,bsdmainutils,\
console-common,console-data,console-tools,cpio,cron,debconf-i18n,\
dhcp-client,ed,exim4,exim4-base,exim4-config,exim4-daemon-light,\
fdutils,gettext-base,groff-base,ifupdown,info,ipchains,iptables,\
iputils-ping,klogd,libconsole,libgcrypt1,libgcrypt7,libgdbm3,\
libgnutls10,libgpg-error0,libident,liblocale-gettext-perl,\
liblockfile1,liblzo1,libnewt0.51,libopencdk8,libpcap0.7,libpcre3,\
libpopt0,libsigc++-1.2-5c102,libssl0.9.7,libtasn1-0,\
libtext-charwidth-perl,libtext-iconv-perl,libtext-wrapi18n-perl,\
libwrap0,lilo,logrotate,mailx,makedev,man-db,manpages,mbr,modconf,\
modutils,nano,netbase,netkit-inetd,net-tools,nvi,pciutils,ppp,\
pppconfig,pppoe,pppoeconf,procps,psmisc,setserial,slang1,sysklogd,\
syslinux,sysvinit,tasksel,tcpd,telnet,wget,whiptail

IN=apt-move,bc,bzip2,dash,debconf-english,libbz2-1.0,libpam0g,\
libreadline6

####

apt-get install debootstrap
mkdir -p $CHROOT$MIRRORDIR/debian
debootstrap --exclude="$OUT" --include="$IN" testing $CHROOT $DEBIAN
mkdir -p $MIRRORDIR
ln -s $CHROOT$MIRRORDIR/debian $MIRRORDIR

Edit the file <chroot dir>/etc/apt/sources.list. The first entry must point to your own mirror inside of the chroot, other entries are the usual http or ftp ones:

deb file:/var/local/mirrors/debian testing main
deb http://ftp.de.debian.org/pub/debian/ testing main

Now edit <chroot dir>/etc/apt-move.conf, especially the values for APTSITES, LOCALDIR, DIST, PKGCOMP, and CONTENTS may need adjustment:

APTSITES="ftp.de.debian.org"
LOCALDIR=/var/local/mirrors/debian
DIST=testing
PKGCOMP="none gzip"
CONTENTS=yes
ARCHS="i386 amd64 ia64"

If you do not have apt-move installed outside of the chroot, you can still read the manual page:

$ zcat <chroot dir>/usr/share/man/man8/apt-move.8.gz|nroff -man|less

Update the apt cache of the mirror:

# chroot <chroot dir> apt-get update

This run will of course issue an error, because the local mirror does not yet exist. Just ignore it.

# chroot <chroot dir> apt-move get

Copy this make-release script to <chroot dir>/root/ and make it executable (chmod 755 is OK):

#!/bin/sh
# generate Release files for a Debian mirror
# 2003-10, W. Borgert <debacle@debian.org>
# Released under the terms of the GNU General Public License

ORIGIN="Debian"
LABEL="Debian"
SUITE="testing"
CODENAME="squeeze"
DESC="Debian Testing distribution - Not Released"
UNPACK=0
ARCHIVE=.

# long options not yet implemented
args=`getopt 'a:c:d:l:o:s:u' $*`
for o
    do case "$o" in
        -a | --arch*) ARCHIVE="$2"; shift; shift;;
        -c | --code*) CODENAME="$2"; shift; shift;;
        -d | --desc*) DESC="$2"; shift; shift;;
        -l | --label) LABEL="$2"; shift; shift;;
        -o | --orig*) ORIGIN="$2"; shift; shift;;
        -s | --suite) SUITE="$2"; shift; shift;;
        -u | --unpa*) UNPACK=1; shift;;
        --) shift; break;;
    esac
done

####

cd $ARCHIVE
DATE=`date -u`
if [ $UNPACK -eq 1 ]; then
    PACKAGGZ=`find . -name Packages.gz -o -name Sources.gz`
    for p in $PACKAGGZ; do
        gunzip -c $p > `dirname $p`/`basename $p .gz`
    done
fi

PACKAGES=`find . -name Packages -o -name Packages.gz -o \
    -name Sources -o -name Sources.gz`
DIRS=`for p in $PACKAGES; do dirname $p; done|sort -u`
COMPS=`for d in $DIRS; do echo $d | \
    sed 's,[^A-Za-z0-9-]*\([A-Za-z0-9-]*\)/.*,\1,'; done|sort -u`
ARCHS=`for d in $DIRS; do echo $d | \
    sed 's,.*binary-\([A-Za-z0-9-]*\),\1,'; done|sort -u`

for d in $DIRS; do
    cd $d
        ARCH=`echo $d | sed 's,.*binary-\([A-Za-z0-9-]*\),\1,'`
        COMP=`echo $d | \
            sed 's,[^A-Za-z0-9-]*\([A-Za-z0-9-]*\)/.*,\1,'`
        echo "Archive: $SUITE
Component: $COMP
Origin: $ORIGIN
Label: $LABEL
Architecture: $ARCH" > Release
    cd -
done

rm -f Release
MD5FILES=`find . -name Packages -o -name Packages.gz -o \
    -name Sources -o -name Sources.gz -o -name Release \
    | sed 's,^[^/]*/,,'`

echo "Origin: $ORIGIN
Label: $LABEL
Suite: $SUITE
Codename: $CODENAME
Date: $DATE
Architectures:" `echo $ARCHS`"
Components:" `echo $COMPS`"
Description: $DESC
MD5Sum:" > Release

for m in $MD5FILES; do
    echo -n " " >> Release
    SIZE=`wc -c $m | sed 's/ *\([0-9]*\).*/\1/'`
    SUM=`md5sum $m | sed 's/^ *\([A-Fa-f0-9]*\) .*/\1/'`
    printf "%s %16d %s\n" $SUM $SIZE $m >> Release
done

echo "SHA1:" >> Release
for m in $MD5FILES; do
    echo -n " " >> Release
    SIZE=`wc -c $m | sed 's/ *\([0-9]*\).*/\1/'`
    SUM=`sha1sum $m | sed 's/^ *\([A-Fa-f0-9]*\) .*/\1/'`
    printf "%s %16d %s\n" $SUM $SIZE $m >> Release
done

It will be used later by the maintenance script.

How to maintain mirrorring system ?

The master list of packages is the file <chroot dir>/root/packages, which contains the package names. The good thing is: Packages that have implicitly installed in the mirror do not appear here. You can always edit this file manually, if you do not wish some packages to be tracked in the future. The maintenance of the mirror itself is done with this update_mirror.sh script below:

#!/bin/sh
# update a partial Debian mirror with apt-move
# 2004-02, W. Borgert <debacle@debian.org>
# Released under the terms of the GNU General Public License

# the root directory of the mirror system
CHROOT=/var/local/chroot
# the mirror directory inside of the chroot
MIRRORDIR=/var/local/mirrors/debian
# master packages file inside of the chroot
MASTER=/root/packages

####

MODE=$1; shift

case "$MODE" in
    clean)
	chroot $CHROOT apt-move --force delete
	exit 0
	;;
    install)
	PACKAGES="$*"
	if [ -f $CHROOT/$MASTER ]; then
	    OLDPACKAGES=`cat $CHROOT/$MASTER`
	else
	    OLDPACKAGES=
	fi
	for p in $OLDPACKAGES $PACKAGES; do echo $p; done | \
	    sort -u > $CHROOT/$MASTER
	;;
    upgrade)
	if [ -f $CHROOT/$MASTER ]; then
	    PACKAGES=`cat $CHROOT/$MASTER`
	else
	    exit 0
	fi
	;;
    *)
	echo "Usage:"
	echo "$0 clean"
	echo "$0 install <packages>"
	echo "$0 upgrade"
	exit -1
	;;
esac

chroot $CHROOT apt-get update
chroot $CHROOT apt-get install \
    --option 'DPkg::Options::=--dry-run' --yes $PACKAGES
chroot $CHROOT apt-move get
chroot $CHROOT apt-move move
chroot $CHROOT apt-move packages
chroot $CHROOT /root/make-release -a $MIRRORDIR
chroot $CHROOT apt-get autoclean

Try it out:

# ./update-mirror install dia ethereal glade-2

Quite a lot of packages will be installed in your mirror.

The following command upgrades the mirror:

# ./update-mirror upgrade

What about other architecture ?

If you like to have not your native, but another architecture mirrored, just specify that in <chroot dir>/etc/apt/apt.conf:

APT::Architecture "arm";

If you want your repository do mirror more than one architecture for example you want arm, i386, ia64, amd64 so add them in <chroot dir>/etc/apt-move.conf:

ARCHS="arm i386 ia64 amd64"

- d