How to roll your own bootable Linux CD-ROM

When booting a regular Linux system, it just mounts some partition with a nice file system (such as ext4) on your HDD on / read/write and there you go.

When booting from a CD-ROM, it’s not that simple. The CD-ROM file system, ISO 9660, does not support the file names and attributes that a Linux system normally needs. And the CD-ROM is read only.

There are ways to work around these issues. You can use the Rock Ridge extension to ISO 9660 to get the file names and attributes needed. And the read only issue can be handled either as I have described in an earlier post, or by using unionfs.

There is also a different approach which I will describe here. Have an image file of the root file system on the CD which you mount in RAM. This will bypass the restrictions of ISO 9660 since you can use any file system in the image, and it can be read/write after being read into RAM. The obvious disadvantage is that this will consume RAM proportional to the size of the root file system, which can be a problem since you will not have any swap space available. But in several use cases, the size of the root file system can be kept small enough for this to work.

The easiest, and probably best, way to do this is to simply use the initramfs mechanism built into the Linux kernel.

In the initramfs, the Linux kernel will start executing /init as the first process, and this process is not expected to exit. This /init can be (and usually is) a shell script. In a regular root file system, /sbin/init will be executed first. So you can convert a regular root file system into an initramfs by adding an /init like this (don’t forget to make it executable):

#!/bin/sh

exec /sbin/init $@

Then you need to package it as a gzipped cpio archive with these slightly awkward commands (${ROOT_FS} is where you have prepared the root file system, ${ISO_FS} is where you are preparing the file system for the CD-ROM, don’t forget to create the directory ${ISO_FS}/isolinux first):

(cd ${ROOT_FS} && find . -type l -printf '%p %Y\n' | sed -n 's/ [LN]$//p' | xargs -rL1 rm -f)
(cd ${ROOT_FS} && find . | cpio --quiet -o -H newc | gzip -9 >${ISO_FS}/isolinux/initrd.img)

Then put this initramfs image along with the kernel image and pass it to the bootloader. Given that you use isolinux as bootloader, put the kernel image in ${ISO_FS}/isolinux/vmlinuz, copy isolinux.bin to ${ISO_FS}/isolinux/ and create a config file ${ISO_FS}/isolinux/isolinux.cfg:

default linux
label linux
    kernel vmlinuz
    append initrd=initrd.img quiet

Finally build an CD-ROM ISO image with this command:

genisoimage -b isolinux/isolinux.bin -c isolinux/boot.cat \
 -no-emul-boot -boot-load-size 4 -boot-info-table \
 -l -input-charset default -V MyLinuxBoot -A "My Linux Boot" \ 
 -o ${ISO_IMAGE} ${ISO_FS}

Then you can burn the ISO image on a CD-ROM, or use the image directly to test it in a virtual machine.

You can reduce the size of the root file system by placing some large auxiliary files (such as the files to install if you are building an installer) outside, directly on the CD-ROM. Then you need to mount the CD-ROM after boot, which can be a bit tricky. Given that you know that a file install.cgz should be on the CD-ROM, you can do like this to mount the CD-ROM on /media:

mount_cdrom() {
    for CD in /dev/cdrom /dev/cdrom[0-9] /dev/sr[0-9]; do
        if [ -b ${CD} ] ; then 
            if mount -t iso9660 -o ro ${CD} /media 2>/dev/null ; then
                if [ -f ${1}/install.cgz ] ; then
                    return
                else
                    umount /media
                fi
            fi
        fi
    done
    echo "Failed to mount CD!"
}

This is inspired from this article. However, that article uses the older and less efficient initrd instead of initramfs. It also mounts the root file system in a two-step bootstrap process which seems redundant to me. So I have made it simpler and more efficient. The procedure with boot menus and different run levels described in the section Customizing and Adding Scripts can be useful though, I recommend reading that section if you need some kind of menu.

Posted in Linux | Leave a comment

How to roll your own Debian based Linux distro

Goal

To build a minimal Debian based Linux system with a fully functional bash shell, TCP/IP networking with DHCP client and apt setup to be able to install any package from the Debian repositories. The resulting system will use about 157 MB disk space and consume less than 10 MB RAM.

This is now implemented in Bachata Linux.

Prerequisites

A Debian based Linux system to work from (e.g. Ubuntu desktop) with the debootstrap and extlinux packages installed. Some virtualization environment is highly recommended for testing, such as KVM/QEMU.

Install system

This will install the system on a disk mounted at mnt. Choose a hostname for the instance to create, substitute it for ${HOSTNAME}. Substitute the URL to your nearest Debian mirror for ${MIRROR}.

  1. sudo debootstrap --variant=minbase --include=localepurge,netbase,ifupdown,net-tools,isc-dhcp-client,linux-base,linux-image-2.6-686,linux-image-2.6.32-5-686 squeeze mnt ${MIRROR}
  2. sudo rm mnt/etc/udev/rules.d/70-persistent-net.rules
  3. sudo rm mnt/var/cache/apt/archives/*
  4. sudo rm mnt/var/cache/apt/*.bin
  5. sudo rm mnt/var/lib/apt/lists/*
  6. sudo rm mnt/var/log/dpkg.log*
  7. sudo rm mnt/var/log/apt/*
  8. sudo mkdir mnt/boot/extlinux
  9. sudo extlinux --install mnt/boot/extlinux
  10. sudoedit mnt/boot/extlinux/syslinux.cfg
    default linux

        label linux
        kernel /boot/vmlinuz-2.6.32-5-686
        append initrd=/boot/initrd.img-2.6.32-5-686 root=UUID=${UUID} ro quiet

    Only for KVM/QEMU: add console=ttyS0 to the “append” line

  11. sudoedit mnt/etc/inittab
    For KVM/QEMU: uncomment getty ttyS0 and comment out getty tty[1-6]
    For others: comment out getty tty[2-6]

  12. sudoedit mnt/etc/passwd – blank password for root
  13. sudoedit mnt/etc/network/interfaces
    auto lo
    iface lo inet loopback
    allow-hotplug eth0
    iface eth0 inet dhcp
    
  14. sudoedit mnt/etc/fstab
    # /etc/fstab: static file system information.
    #
    # Use 'blkid -o value -s UUID' to print the universally unique identifier
    # for a device; this may be used with UUID= as a more robust way to name
    # devices that works even if disks are added and removed. See fstab(5).
    #
    # <file system> <mount point>   <type>  <options>             <dump>  <pass>
    proc            /proc           proc    nodev,noexec,nosuid   0       0
    UUID=${UUID}        /               ext2    errors=remount-ro     0       1
    
  15. sudoedit mnt/etc/hostname
    ${HOSTNAME}
    
  16. sudoedit mnt/etc/locale.nopurge
    MANDELETE
    
    DONTBOTHERNEWLOCALE
    
    SHOWFREEDSPACE
    
    #QUICKNDIRTYCALC
    
    #VERBOSE
    
    en
    
  17. sudoedit mnt/etc/apt/sources.list
    deb ${MIRROR} squeeze main
    deb-src ${MIRROR} squeeze main
    
    deb http://security.debian.org/ squeeze/updates main
    deb-src http://security.debian.org/ squeeze/updates main
    
    # squeeze-updates, previously known as 'volatile'
    deb ${MIRROR} squeeze-updates main
    deb-src ${MIRROR} squeeze-updates main
    
  18. sudoedit mnt/etc/apt/apt.conf.d/02nocache
    Dir::Cache {
      srcpkgcache "";
      pkgcache "";
    }
    
  19. sudoedit mnt/etc/apt/apt.conf.d/02compress-indexes
    Acquire::GzipIndexes "true";
    Acquire::CompressionTypes::Order:: "gz";
    
  20. sudo chroot mnt localepurge
  21. sudo chroot mnt apt-get update
  22. sudo chroot mnt passwd root

Install on a physical disk

  1. Create a bootable partition of at least 288 MB with system code 83 “Linux” using fdisk
  2. sudo mke2fs -L ${HOSTNAME} -t ext2 /dev/xxxx
  3. sudo UUID=`blkid -o value -s UUID /dev/xxxx`
  4. mkdir mnt
  5. sudo mount /dev/xxxx mnt
  6. Install system as above
  7. sudo umount mnt

Install in KVM/QEMU

  1. dd if=/dev/zero of=${HOSTNAME}.img bs=1024 count=288K
  2. mke2fs -L ${HOSTNAME} -t ext2 ${HOSTNAME}.img
  3. UUID=`blkid -o value -s UUID ${HOSTNAME}.img`
  4. mkdir mnt
  5. sudo mount ${HOSTNAME}.img mnt
  6. Install system as above
  7. sudo umount mnt
  8. virt-install --connect qemu:///system -n ${HOSTNAME} -r 256 --os-type linux --os-variant debiansqueeze --import --disk path=${HOSTNAME}.img --network=network:default --graphics none --virt-type kvm

Use --virt-type qemu if your CPU doesn’t support virtualization.

Then you can start it with virsh start ${HOSTNAME} and connect to it with virsh console ${HOSTNAME}

For some reason, doing reboot from inside the guest doesn’t seem to work properly. virsh reboot and virsh shutdown doesn’t work either (you might get that to work by installing some ACPI stuff in the guest). Use halt from inside the guest to have a clean shutdown, and then restart it with virsh start.

Install in VirtualBox 4.1

First create a virtual machine in VirtualBox with an empty VDI disk ${VDI_FILE} of least 288 MB.

Install VDI mounting tool according to this description.

  1. mkdir vdi-mount
  2. vdfuse-v82a -f ${VDI_FILE} vdi-mount
  3. mke2fs -L ${HOSTNAME} -t ext2 vdi-mount/EntireDisk
  4. mkdir mnt
  5. sudo mount vdi-mount/EntireDisk mnt
  6. Install system as above
  7. sudo umount mnt
  8. fusermount -u vdi-mount

Then start the virtual machine in VirtualBox.

Install in other virtualization environments

Please tell me how!

Posted in Linux | 17 Comments

How to move MySQL data directory in Ubuntu Server

By default, the MySQL data is placed in /var/lib/mysql, which is a reasonable default. However, sometimes you want to place it somewhere else, such as on an other file system. Using a symlink doesn’t seem to work, so you have follow this procedure.

To move the MySQL data directory from /var/lib to /mnt/mydata, run these commands as root:

  1. apt-get install mysql-server
  2. service mysql stop
  3. mv /var/lib/mysql /mnt/mydata/
  4. replace /var/lib/mysql with /mnt/mydata/mysql in
    • /etc/passwd – mysql
    • /etc/mysql/my.cnf – [mysqld] datadir
    • /etc/apparmor.d/usr.sbin.mysqld (twice)
  5. service mysql start
Posted in Uncategorized | 10 Comments

Using ZTE Blade Android phone with Ubuntu 11.10

Using a ZTE Blade Android phone connected with USB is a bit tricky with Ubuntu 11.10.

First you need to apply the patch in this bug, after doing that you should get USB storage to work.

After doing that, it is also possible to get development and debugging to work over USB. Create a group androiddev (addgroup --system androiddev), and add yourself to it (gpasswd -a yourUsername androiddev). Then create a file /etc/udev/rules.d/11-android.rules with this content (4 lines, watch the line breaks):

SUBSYSTEMS==”usb”, ATTRS{idVendor}==”19d2″, ATTRS{idProduct}==”1353″, MODE=”0666″, OWNER=”root”, GROUP=”androiddev” #Normal Blade
SUBSYSTEMS==”usb”, ATTRS{idVendor}==”19d2″, ATTRS{idProduct}==”1350″, MODE=”0666″, OWNER=”root”, GROUP=”androiddev” #Debug Blade
SUBSYSTEMS==”usb”, ATTRS{idVendor}==”19d2″, ATTRS{idProduct}==”1354″, MODE=”0666″, OWNER=”root”, GROUP=”androiddev” #Recovery Blade
SUBSYSTEMS==”usb”, ATTRS{idVendor}==”18d1″, ATTRS{idProduct}==”d00d”, MODE=”0666″, OWNER=”root”, GROUP=”androiddev” #Fastboot Blade

Finally logout and login again. Don’t forget to enable USB debugging on the phone before connecting it.

Depending on the exact model you have, you might need to adjust the file. See what the lsusb command says about your phone and put that product id in the second line.

Posted in Linux | 5 Comments

How to disable activity logging in Ubuntu 11.10 Oneiric Ozelot

Ubuntu has mechanism to log user activity such as used documents. This is used to facilitate searching, but can also be intrusive to your privacy.

Here is a way to disable this logging without breaking Unity or any other part of the system, execute these commands in a terminal:

  1. sudo mv /etc/xdg/autostart/zeitgeist-datahub.desktop /etc/xdg/autostart/zeitgeist-datahub.desktop-inactive
  2. rm ~/.local/share/recently-used.xbel
  3. mkdir ~/.local/share/recently-used.xbel
  4. rm -rf ~/.local/share/zeitgeist

then log out and log in again.

Posted in Linux, Ubuntu | 7 Comments

Backup your mobile phone in Linux

To backup data from a non-smart SonyEricsson mobile phone (such as W890i) in Linux, use the gammu utility.

  1. Install gammu, it is available as a package in the standard repositories for Debian and Ubuntu, just install the gammu package.
  2. Create a ~/.gammurc file with the following content:

    [gammu]
    port = /dev/ttyACM0
    connection = at

  3. Connect your mobile phone to the computer with the USB cable and select Phone mode
  4. Use

    gammu backup <filename>.vcf -yes

    to backup your phone book to a vCard file

  5. Use

    gammu geteachsms

    to get all SMS stored in the phone (both sent and received), they will be written to STDOUT

  6. There is also a lot more you can do with gammu
Posted in Linux | 2 Comments

PHP session timeout

The developers of PHP has, in their infinite wisdom, decided that the default session timeout should be 24 minutes (1440 seconds).

This means that if you have a MediaWiki wiki and are editing a single page for half an hour and then click the save button, you are logged out and all your changes are lost. I just learned this the hard way.

Fortunately, you can change this with the session.gc_maxlifetime parameter in php.ini. So set this to a higher – more sane – value, such as 86400 (24 hours).

Posted in PHP, web | Leave a comment

Upgrade to Ubuntu 11.04 without Unity

The new Ubuntu release 11.04 Natty contains the new Unity desktop environment which is quite controversal. I have tried it for a while and I think it is neat but too buggy and immature. However, it is easy to revert to the old Gnome 2 desktop environment and have things working almost as in 10.10.

First do a normal distribution upgrade to 11.04, then go to System Settings -> Login Screen and select Ubuntu Classic as default session and reboot.

Posted in Linux, Ubuntu | Leave a comment

Using Sveon SNT1020 WiFi USB adapter in Ubuntu

I just brought an Sveon SNT1020 WiFi USB adapter.

It works very well with Ubuntu Linux 10.10, just connect it to an USB 2 port and you can start using it with NetworkManager right away, no drivers or setup necessary. You should possibly disable any built-in WiFi adapters first though.

It cost €35.

Posted in hardware, Linux | 1 Comment

Web application frameworks in Java

When you know which type of web application you are to develop, it’s time to have a look at some possible choices.

I have tried to categorize some modern and popular web application frameworks in Java.

Simple server driven MVC page based

This category contains the traditional frameworks used for developing web applications with purely server driven application logic. They are based on complete HTML pages and uses the Model-View-Controller design pattern.

They do not directly support for client driven application logic, though it can be done if combined with a comprehensive JavaScript library like jQuery.

Full stack server driven MVC page based

This category contains some more advanced full stack frameworks. They are used for server driven application logic and not suitable for client driven application logic. They are inspired by Ruby on Rails.

Server driven component/widget based

Focus on UI components (widgets) instead of HTML pages, make development a bit more like desktop applications in frameworks like Swing. They are used for server driver application logic and not suitable for client driven application logic.

Client driven component/widget based

Useful for developing client driven application logic by automatically generating the client side JavaScript code.

  • Google Web Toolkit
  • Compiles Java to JavaScript and handle the cross-browser issues. Also abstracts away most of HTML. There is a good tutorial.

  • Vaadin
  • Based on GWT, but abstracts away even more HTML. Also handle the server side and automates AJAX based browser-server communication (using a native data format). A programming model which feels much more like Swing than usual web application development.

  • AribaWeb
  • Quite advanced full stack framework.

Posted in AJAX, Java, JavaEE, web | 1 Comment