2

I am making a custom ISO w/ autoinstall to automatically install Ubuntu Server onto our machines. We of course need to install some apt packages as part of the customization. The problem is, this installation needs to be done entirely offline, so I can't just add them to the apt package list and have subiquity magic take care of everything.

I am using this method to pack the user-install/autoinstall file into the iso and that is working well.

How can I download apt packages ahead of time, include them on the iso, and have subiquity automatically find and install them?

Mr. T
  • 201

3 Answers3

1

I found a solution for this here

Mr. T
  • 201
1

You can just use the the internal apt repository on the iso.

Deb files are located in pool/main. The manifest of deb files is dists/focal/main/binary-amd64/Packages.gz. SHA256 summary is in dists/focal/Release.

Last problem is that installation checks the signing of the Release file. Usually it is signed by Ubuntu. There are 2 potential ways to solve it:

  1. To set deb [trusted=yes] file:///cdrom focal main so it won't check gpg sign. But I don't know where this is located in the ISO.

  2. To use your own key to sign it and add the pubkey to trusted keys in ISO.
    After modifying them, you can use packages section in user-data as with network.

    The full steps with signing the Release:

    1. Extract iso. I'll call the extracted dir 'iso'.

    2. Add the packages to iso/pool/main/path/to/package.deb. You can use

      url=$(apt-get download --print-uris "$package" | awk '{print $1}')

      This will give you the source so you can download it in the same order as the original repository.

  3. Then to add them to the Packages.gz file that lists all the available packages by calling

    PACKAGES_FILE_PATH="main/binary-amd64/Packages"
PACKAGES_GZ_FILE_PATH="main/binary-amd64/Packages.gz"
# Create Packages file
cd iso
dpkg-scanpackages . /dev/null > dists/focal/$PACKAGES_FILE_PATH
cd dists/focal
# Compress Packages file into Packages.gz
gzip -9c $PACKAGES_FILE_PATH > $PACKAGES_GZ_FILE_PATH

  4. Update the the Release file:

    # Update Release file with new sha256 and size
PACKAGES_INFO=$(apt-ftparchive release . | grep --after-context=10 SHA256 | grep "$PACKAGES_FILE_PATH" | head -n 1)
PACKAGES_GZ_INFO=$(apt-ftparchive release . | grep --after-context=10 SHA256 | grep "$PACKAGES_GZ_FILE_PATH")
# Update the Release file with new values
sed -i "\@$PACKAGES_FILE_PATH@ s@.*@$PACKAGES_INFO@" Release
sed -i "\@$PACKAGES_GZ_FILE_PATH@ s@.*@$PACKAGES_GZ_INFO@" Release

  5. Create a gpg key and sign the Release file:

    GPG_KEY_EMAIL="your.email@example.com"  # Replace with your email address
# Check if GPG key exists; if not, generate a new one
if ! gpg --list-secret-keys --keyid-format LONG "$GPG_KEY_EMAIL" &> /dev/null; then
    echo "Generating a new GPG key..."
    gpg --batch --gen-key <<EOF
    %echo Generating a GPG key
    Key-Type: RSA
    Key-Length: 4096
    Subkey-Type: RSA
    Subkey-Length: 4096
    Name-Real: Your Name
    Name-Email: $GPG_KEY_EMAIL
    Expire-Date: 0
    Passphrase: YourPassphrase
    %commit
    %echo done
EOF
fi
rm -f iso/dists/focal/Release.gpg
# Sign the Release file with GPG
gpg --batch --passphrase "ubuntu" --default-key "$GPG_KEY_EMAIL" -abs -o iso/dists/focal/Release.gpg iso/dists/focal/Release

  6. Add the public key (gpg) to trusted keys in filesystem.squashfs.

    sudo unsquashfs -d filesystem iso/casper/filesystem.squashfs
gpg --export $GPG_KEY_EMAIL > autoinstall.gpg
sudo mv autoinstall.gpg filesystem/etc/apt/trusted.gpg.d/
sudo rm iso/casper/filesystem.squashfs
sudo mksquashfs filesystem/ iso/casper/filesystem.squashfs -comp xz -e boot
sudo rm -rf filesystem/
zx485
  • 2,865
Roman Lozovsky
  • 11
0

Here is what I did on Ubuntu 20.04 live server (focal). The goal is to install Ubuntu without network connection but with some packages not available in the official ISO.

  1. Create a custom ISO file based on the official ubuntu-20.04.6-live-server-amd64.iso
  2. Create a custom GPG key to sign Release.gpg and install it into filesystem.squashfs
  3. Add the *.deb files to the extracted-iso/pool/extra/ directory

Missing *.deb were downloaded from a running target connected to internet and default APT sources to Ubuntu's servers. (Yeah, this is not optimal but couldn't do better). More info here

$ sudo apt update
$ apt-get --print-uris --yes install <PACKAGE_NAMES> | grep ^\' > package-list.txt
$ wget $(cat package-list.txt | cut -d "'" -f 2) -N -P extracted-iso/pool/extra
  1. Add a custom component named extra with all the metadata generated by apt-ftparchive , it gives something like this : 
extracted-iso/dists/
├── focal
│   ├── Release
│   ├── Release.gpg
│   ├── extra
│   │   └── binary-amd64
│   │       ├── Packages
│   │       ├── Packages.gz
│   │       └── Release
│   ├── main
│   │   ├── binary-amd64
│   │   │   ├── Packages
│   │   │   ├── Packages.gz
│   │   │   └── Release
│   │   ├── binary-i386
│   │   │   ├── Packages
│   │   │   ├── Packages.gz
│   │   │   └── Release
│   │   └── source
│   └── restricted
│       ├── binary-amd64
│       │   ├── Packages
│       │   ├── Packages.gz
│       │   └── Release
│       ├── binary-i386
│       │   ├── Packages
│       │   ├── Packages.gz
│       │   └── Release
│       └── source
├── stable -> focal
└── unstable -> focal
  1. Configure autoinstall.yaml to launch a custom script named late-commands.sh (Optionnal but cleaner to me)
#cloud-config
# File : /cdrom/<my_custom_dir_in_iso>/user-data (i.e. autoinstall.yaml)
autoinstall:
  # ...
  # Launch a custom script which regroup all late-commands
  late-commands:
    - /cdrom/<my_custom_dir_in_iso>/late-commands.sh
  1. Install your packages with apt from the ISO mounted on /cdrom
#!/bin/bash
# File : late-commands.sh
# Note "/cdrom" is not mounted anymore at the step of late-commands !

Mount back the cdrom (hopping "/sdb1" is a constant)


curtin in-target -- mkdir -p /cdrom
curtin in-target -- mount -o ro /dev/sdb1 /cdrom


Add a new source for APT pointing to the custom *.deb repo on /cdrom


echo "deb [check-date=no] file:///cdrom focal main restricted extra" > /target/etc/apt/sources.list.d/local_cdrom.list


Update packages list from "local_cdrom.list" uris


curtin in-target -- apt update


Install custom packages and dependencies


curtin in-target -- apt install -y <PACKAGE_NAMES>


Clean APT source and umount cdrom


rm -rf /target/etc/apt/sources.list.d/local_cdrom.list
curtin in-target -- umount /cdrom
curtin in-target -- rm -rf /cdrom

Why the official way did not work ?

The official way of installing packages during an autoinstall procedure is to specify them inside packages:.

In the special case where the repo is local to the ISO (aka offline install), the installer failed during the curtin step where it launches apt update. At this point, /cdrom is not mounted on the target filesystem and APT cannot retreives Release and Packages files.

#cloud-config
autoinstall:
  # ...
  apt:
    # THIS DO NOT WORK !
    # Adding a custom /etc/apt/sources.list.d/local_cdrom.list which appends the 
    # component "extra" to the existing "main" and "restricted" in /etc/apt/sources.list
    sources:
      local_cdrom.list:
        source: "deb [check-date=no] file:///cdrom focal main restricted extra"

Install package from "/cdrom > focal > extra"


packages:
    - <PACKAGE_NAMES>

PS: I didn't use a solution based on apt-offline because you need the package to be installed in the first place on the target

David
  • 1