How to run VirtIO on AGL Reference Hardware

This guide explains how to run virtual AGL on bare-metal AGL (AGL-in-AGL) on AGL Reference Hardware board with help of QEMU.

For simplicity reasons, this guide configures only virtio-blk and virtio-rng.

To run two AGLs, flash storage (SD-Card or eMMC) with two partitions for different root file systems is needed.

NOTE: To be able to use eMMC, a method to partition and flash it is needed. For example, U-Boot can be used for that, but necessary configuration options shall be enabled in U-Boot for that.

  1. Partition SD-Card or eMMC to have two partitions, at least 1 GiB each. Actually, can be less but just rounded up to have nice number.

    1. For example, SD-Card can be partitioned using fdisk utility.

      Execute fdisk:

      sudo fdisk <disk>

      Here <disk> should be SD-Card device file, for example /dev/sdb.

      CAUCTION: Using wrong SD-Card device file can result in data lost on your computer!

      Partition SD-Card:

      $ fdisk /tmp/disk
       
      Welcome to fdisk (util-linux 2.33.1).
      Changes will remain in memory only, until you decide to write them.
      Be careful before using the write command.
       
      Device does not contain a recognized partition table.
      Created a new DOS disklabel with disk identifier 0x8fb74b1e.
       
      Command (m for help): n
      Partition type
         p   primary (0 primary, 0 extended, 4 free)
         e   extended (container for logical partitions)
      Select (default p):
       
      Using default response p.
      Partition number (1-4, default 1):
      First sector (2048-5242879, default 2048):
      Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-5242879, default 5242879): +1GiB
       
      Created a new partition 1 of type 'Linux' and of size 1 GiB.
       
      Command (m for help): n
      Partition type
         p   primary (1 primary, 0 extended, 3 free)
         e   extended (container for logical partitions)
      Select (default p): p
      Partition number (2-4, default 2):
      First sector (2099200-5242879, default 2099200):
      Last sector, +/-sectors or +/-size{K,M,G,T,P} (2099200-5242879, default 5242879): +1GiB
       
      Created a new partition 2 of type 'Linux' and of size 1 GiB.
       
      Command (m for help): w
      The partition table has been altered.
      Syncing disks.

    2. For example, eMMC can be partitioned from U-Boot command line using:

      gpt write mmc 1 "name=linux1_rootfs,size=1GiB;name=linux2_rootfs,size=1GiB;name=misc,size=4KiB"

      NOTE: gpt command should be enable in U-Boot configuration.
      NOTE: partition 'misc' are needed to make fastboot happy when using it to flash eMMC.

  2. Download Koky Koi M2:

    repo init -u https://gerrit.automotivelinux.org/gerrit/AGL/AGL-repo -b master -m koi_10.92.0.xml
    repo sync -c -d

  3. Build AGL for virtio-aarch64 machine:

    source meta-agl/scripts/aglsetup.sh -m virtio-aarch64 -f
    bitbake agl-image-minimal

    Flash root file system to the second partition on SD-Card or eMMC.

    1. Flashing SD-Card:

      cd build/tmp/deploy/images/virtio-aarch64
      sudo dd if=agl-image-minimal-virtio-aarch64.ext4 of=FILE

      Here FILE should be SD-Card second partition device file, for example /dev/sdb2.

      CAUCTION: Using wrong SD-Card partition device file can result in data lost on your computer!
    2. Flashing eMMC via fastboot

      cd build/tmp/deploy/images/virtio-aarch64
      img2simg agl-image-minimal-virtio-aarch64.ext4 agl-image-minimal-virtio-aarch64.ext4.img
      fastboot flash linux2_rootfs agl-image-minimal-virtio-aarch64.ext4.img

      NOTE: U-Boot should have fastboot running.

  4. Build AGL for AGL Reference Hardware.

    NOTE: To build AGL for AGL Reference Hardware and for virtio, the same folder could be reused. It takes a little bit more time when switching between AGL Reference Hardware build and virtio build but this overhead is not critical.

    Initialize build environment:

    source meta-agl/scripts/aglsetup.sh -m h3ulcb -f agl-refhw-h3


    In build/conf/local.conf add:

    AGL_DEFAULT_IMAGE_FSTYPES = "ext4"
    IMAGE_INSTALL_append = "qemu"

    CAUTION: Calling aglsetup.sh with "-f" flag will remove above modification in "local.conf", so they will be needed to be re-applied.

    Build AGL for AGL Reference Hardware:

    bitbake agl-image-minimal

    Add Linux 2 kernel to the Linux 1 rootfs:

    cp tmp/deploy/images/virtio-aarch64/Image tmp/work/h3ulcb-agl-linux/agl-image-minimal/1.0-r0/rootfs/linux2
    bitbake agl-image-minimal -c image_ext4 -f
    bitbake agl-image-minimal -c image_complete

    Flash root file system to the first partition on SD-Card or eMMC.

    1. Flashing SD-Card:

      cd build/tmp/deploy/images/h3ulcb
      sudo dd if=agl-image-minimal-h3ulcb.ext4 of=FILE

      Here FILE should be SD-Card second partition device file, for example /dev/sdb1.

      CAUCTION: Using wrong SD-Card partition device file can result in data lost on your computer!
    2. Flashing eMMC via fastboot:

      cd build/tmp/deploy/images/h3ulcb
      img2simg agl-image-minimal-h3ulcb.ext4 agl-image-minimal-h3ulcb.ext4.img
      fastboot flash linux1_rootfs agl-image-minimal-h3ulcb.ext4.img

      NOTE: U-Boot should have fastboot running.

  5. Boot AGL Reference Hardware board using Linux located on the first partition of SD-Card or eMMC

    setenv bootargs 'console=ttySC0,115200 root=/dev/mmcblk0p1 rootwait ro rootfstype=ext4'
    ext4load mmc 1:1 0x48000000 /boot/r8a7795-agl-refhw.dtb
    ext4load mmc 1:1 0x48080000 /boot/Image
    booti 0x48080000 - 0x48000000

  6. Run QEMU from Linux 1 command line

    qemu-system-aarch64 \
      -machine virt \
      -cpu cortex-a57 \
      -m 2048 \
      -serial mon:stdio \
      -global virtio-mmio.force-legacy=false \
      -drive id=disk0,file=/dev/mmcblk0p2,if=none,format=raw \
      -device virtio-blk-device,drive=disk0 \
      -object rng-random,filename=/dev/urandom,id=rng0 \
      -device virtio-rng-device,rng=rng0 \
      -nographic \
      -kernel /linux2
      -append 'root=/dev/vda rw mem=2048M'

    NOTE: mmcblk0p2 above is used for when root file system is flashed on eMMC. In case of SD-Card, mmcblk1p2 has to be used.

  7. It is possible to exit from QEMU using monitor commands. Enter "Ctrl+a h" for help.

KVM

NOTE: QEMU is a bit slow because KVM is not working. Adding "-enable-kvm" to QEMU command fails with

qemu-system-aarch64: kvm_init_vcpu failed: Invalid argument

Somewhere in the Internet suggested:

  • To use "-cpu host" but it did not help.
  • To use taskset to set to specific CPU core. Not tried since requires busybox reconfiguration to enable taskset.
  • Update QEMU and/or KVM to latest version. There was some kvm_init_vcpu signature changes.