...
Running
...
two
...
virtio-sound
...
devices
...
on
...
top
...
of
...
virtio-loopback
...
on
...
AGL
...
Reference
...
Hardware
...
platform
...
(RefHw)
...
An
...
overview
...
This
...
guide
...
contains
...
instructions
...
and
...
sources/binaries
...
to
...
manually
...
compile,
...
test
...
and
...
experiment
...
virtio-loopback
...
with
...
two
...
vhost-user
...
sound
...
devices
...
on
...
**AGL
...
reference
...
hardware**.
...
This
...
initial
...
multi-device
...
support
...
was
...
developed
...
to
...
demonstrate
...
the
...
flexibility
...
of
...
virtio-loopback,
...
but
...
it
...
is
...
by
...
no
...
means
...
stable
...
and
...
optimized.
...
The
...
yocto
...
integration
...
of
...
virtio-loopback
...
and
...
vhost-user-sound
...
in
...
AGL
...
is
...
ongoing
...
and
...
is
...
targeting
...
single
...
device
...
support
...
only.
...
Future
...
work
...
idea
...
is
...
to
...
properly
...
support
...
multi-devices
...
in
...
2024.
...
This
...
work
...
was
...
presented
...
at
...
OSS
...
Japan
...
by
...
Virtual
...
Open
...
Systems
...
(https://ossjapan2022.sched.com/event/1D127/virtio-loopback-an-hardware-abstraction-layer-hal-for-non-virtualized-environments-michele-paolino-virtual-open-systems).
...
The
...
code
...
of
...
this
...
demo
...
diverges
...
a
...
bit
...
from
...
the
...
single-device
...
that
...
is
...
targeting
...
upstream
...
in
...
AGL.
...
For
...
instance,
...
it
...
contains
...
makefiles
...
to
...
build
...
two
...
identical
...
virtio-loopback-drivers
...
with
...
different
...
names.
...
For
...
this
...
reason,
...
we
...
placed
...
all
...
the
...
sources/binaries
...
in
...
a
...
temporary
...
public
...
repository
...
available
...
[here](https://git.virtualopensystems.com/virtio-loopback/docs/-/tree/multi-sound-guide).
...
Description of two virtio-sound
...
device
...
running
...
on
...
top
...
of
...
virtio-loopback
...
architecture*
...
Components
The architecture we want to put in place is composed by the following components:
- [Virtio Loopback Transport](https://git.virtualopensystems.com/virtio-loopback/loopback_driver/-/tree/multi-sound-demo),
...
a
...
Linux
...
module
...
which
...
acts
...
as
...
a
...
transport
...
for
...
the
...
virtio-devices
...
and
...
redirects
...
the
...
virtio
...
messages
...
to
...
the
...
user-space
...
(virtio-loopback-adapter).
...
-
...
[Virtio
...
Loopback
...
Adapter](https://git.virtualopensystems.com/virtio-loopback/adapter_app/-/tree/multi-sound-demo)
...
is
...
a
...
user-space
...
application
...
which
...
is
...
the
...
intermediate
...
between
...
virtio-loopback-transport
...
and
...
vhost-user-rng
...
device
...
and
...
bridges
...
the
...
communication
...
between
...
them.
...
-
...
[vhost-device-sound](https://github.com/rust-vmm/vhost-device.git)
...
is
...
the
...
upstream
...
vhost-user-sound
...
device
...
#
...
Instructions
...
for
...
reproducing
...
the
...
VOSYS
...
OSS23
...
demo
1. Prerequisites
Before start compiling and running the components of the architecture you need: - **RefHw Linux kernel** dependencies: + The following configuration flags need to be enabled: *CONFIG_VIRTIO*,
...
*CONFIG_SND_VIRTIO*
...
-
...
**RefHw
...
filesystem**
...
dependencies:
...
+
...
**alsa-utils**
...
so
...
to
...
use
...
*aplay*
...
command,
...
in
...
order
...
to
...
feed
...
audio
...
stream
...
to
...
a
...
sound
...
device
...
of
...
the
...
system
...
-
...
**Host
...
system**
...
dependencies
...
+
...
**git**
...
to
...
fetch
...
the
...
sources
...
and
...
patch
...
them
...
before
...
compilation
...
Since
...
RefHw
...
current
...
kernel
...
version
...
is
...
older
...
than
...
v5.13,
...
the
...
virtio-sound
...
driver
...
is
...
not
...
available
...
in
...
the
...
kernel.
...
In
...
that
...
case,
...
patches
...
provided
...
in
...
the
...
*[virtio_snd_patches/](https://git.virtualopensystems.com/virtio-loopback/docs/-/tree/multi-sound-guide/virtio_snd_patches)*
...
subdirectory
...
of
...
the
...
current
...
repo
...
should
...
be
...
applied
...
to
...
the
...
kernel
...
sources.
...
Please
...
find
...
more
...
information
...
about
...
that
...
in
...
the
...
next
...
section
...
(3.1).
...
2.
...
Build
...
virtio-loopback
...
components
...
This
...
section
...
focuses
...
on
...
the
...
steps
...
need
...
to
...
be
...
followed
...
in
...
order
...
all
...
the
...
required
...
components
...
of
...
the
...
architecture
...
be
...
compiled.
...
**NOTE**:
...
For
...
simplicity,
...
we
...
will
...
consider
...
to
...
execute
...
all
...
the
...
operations
...
in
...
"$HOME/virtio-loopback-test".
...
2.1
...
Backport
...
virtio-sound
...
on
...
kernel
...
v5.10
...
Patches
...
which
...
backport
...
the
...
virtio-sound
...
driver
...
on
...
kernel
...
v5.10
...
(used
...
by
...
AGL
...
reference
...
HW)
...
can
...
be
...
found
...
under
...
"virtio_snd_patches/"
...
of
...
the
...
current
...
repo.
...
To
...
apply
...
them
...
to
...
the
...
target
...
kernel
...
(assuming
...
you
...
have
...
sources
...
in
...
the
...
"linux_kernel_source"
...
folder)
...
the
...
following
...
steps
...
are
...
required:
...
-
...
`cd
...
linux_kernel_source`
...
-
...
`git
...
am
...
$HOME/virtio-loopback-test/virtio_loopback_guide/virtio_snd_patches/000*`
...
-
...
`make
...
sound/virtio/virtio_snd.ko`
...
-
...
For
...
cross-compile
...
please
...
use:
...
`make
...
ARCH=arm64
...
CROSS_COMPILE=aarch64-linux-gnu-
...
sound/virtio/virtio_snd.ko`
...
>
...
**NOTE:**
...
A
...
**prebuilt**
...
virtio_snd.ko
...
module
...
can
...
be
...
found
...
under
...
"[prebuilt_binaries](https://git.virtualopensystems.com/virtio-loopback/docs/-/tree/multi-sound-guide/prebuilt_binaries)"
...
directory.
...
2.2
...
virtio-loopback
...
driver
...
The
...
loopback
...
driver
...
is
...
one
...
of
...
two
...
core
...
components
...
of
...
the
...
presented
...
design.
...
It
...
is
...
a
...
kernel
...
module
...
responsible
...
of
...
re-routing
...
virtio
...
messages
...
to
...
the
...
user-space
...
and
...
provides
...
a
...
way
...
to
...
user-space
...
applications
...
to
...
access
...
the
...
vrings.
...
The
...
instructions
...
about
...
how
...
to
...
clone
...
the
...
corresponding
...
repository
...
and
...
build
...
it
...
are
...
the
...
following:
...
Clone
...
and
...
build
...
loopback
...
driver
...
by
...
executing
...
the
...
following
...
commands:
...
-
...
`cd
...
$HOME/virtio-loopback-test`
...
-
...
`git
...
clone
...
git@git.virtualopensystems.com:virtio-loopback/loopback_driver.git`
...
-
...
`git
...
checkout
...
multi-sound-demo`
...
-
...
`cd
...
loopback_driver/`
...
-
...
`./make_loopback.sh`
...
or
...
`./make_loopback.sh
...
1`
...
for
...
enabling
...
logs
...
>More
...
in
...
details,
...
the
...
`make`
...
command
...
(used
...
inside
...
the
...
make_loopback.sh
...
script)
...
takes
...
as
...
an
...
argument:
...
>-
...
'DEBUG=1'
...
which
...
defines
...
if
...
the
...
driver
...
is
...
going
...
to
...
...
logs.
...
**Note:**
...
Please
...
update
...
the
...
variable
...
*LINUX_DIR*
...
in
...
the
...
"Makefile",
...
pointing
...
to
...
the
...
target
...
linux
...
sources.
...
>
...
A
...
prebuilt
...
version
...
of
...
loopback
...
modules
...
can
...
be
...
found
...
under
...
"[prebuilt_binaries](https://git.virtualopensystems.com/virtio-loopback/docs/-/tree/multi-sound-guide/prebuilt_binaries)":
...
>
...
-
...
loopback_driver0.ko,
...
*logs
...
disabled*
...
>
...
-
...
loopback_driver1.ko,
...
*logs
...
disabled*
...
>
...
-
...
loopback_driver0_logs.ko
...
>
...
-
...
loopback_driver1_logs.ko
...
The
...
kernel
...
side
...
component
...
of
...
the
...
virtio-loopback
...
architecture
...
is
...
now
...
ready
...
to
...
be
...
used.
...
Before
...
proceeding,
...
let's
...
build
...
the
...
user-space
...
component
...
virtio-loopback
...
adapter.
...
2.3
...
virtio-loopback
...
adapter
...
The
...
adapter
...
is
...
the
...
second
...
important
...
component
...
of
...
the
...
presented
...
design
...
and
...
it
...
is
...
responsible
...
of
...
bridging
...
the
...
communication
...
between
...
the
...
vhost-user-device
...
(user-space)
...
and
...
the
...
loopback
...
driver
...
(described
...
in
...
the
...
previous
...
section).
...
Instructions
...
to
...
clone
...
and
...
build
...
this
...
component
...
are
...
the
...
following:
...
-
...
`cd
...
$HOME/virtio-loopback-test`
...
-
...
`git
...
clone
...
git@git.virtualopensystems.com:virtio-loopback/adapter_app.git`
...
-
...
`git
...
checkout
...
multi-sound-demo`
...
-
...
`cd
...
adapter_app/`
...
-
...
`make
...
ARCH=arm64`
...
More
...
in
...
details,
...
the
...
`make`
...
command
...
takes
...
as
...
an
...
argument:
...
-
...
'DEBUG=1'
...
which
...
defines
...
if
...
the
...
adapter
...
is
...
going
...
to
...
...
logs.
...
>**NOTE:**
...
A
...
prebuilt
...
version
...
of
...
adapter's
...
binary
...
can
...
be
...
found
...
under
...
"[prebuilt_binaries](https://git.virtualopensystems.com/virtio-loopback/docs/-/tree/multi-sound-guide/prebuilt_binaries)"
...
directory:
...
>
...
-
...
adapter,
...
*logs
...
disabled*
...
>
...
-
...
adapter_logs
...
Next
...
step
...
is
...
to
...
compile
...
vhost-user-sound
...
device
...
to
...
be
...
able
...
to
...
test
...
the
...
architecture.
...
2.4
...
Vhost-user-sound
...
The
...
third
...
component
...
of
...
the
...
virtio-loopback
...
architecture
...
is
...
the
...
vhost-user-device.
...
Vhost-user-device
...
is
...
the
...
component
...
with
...
whom
...
virtio
...
driver
...
communicates
...
via
...
the
...
path
...
"loopback
...
driver"
...
<->
...
"adapter".
...
Essentially,
...
it
...
is
...
a
...
user-space
...
driver
...
which
...
virtio
...
tries
...
communicate
...
with.
...
Instructions
...
to
...
clone
...
and
...
build
...
this
...
modified
...
version
...
are
...
the
...
following:
...
-
...
`cd
...
$HOME/`
...
-
...
`git
...
clone
...
https://github.com/rust-vmm/vhost-device.
...
git`
-
...
`cd
...
./vhost-device/staging/`
...
-
...
`cargo
...
build
...
--bin
...
vhost-device-sound`
...
>**NOTE:**
...
A
...
prebuilt
...
version
...
of
...
vhost-device-sound
...
can
...
be
...
found
...
under
...
"[prebuilt_binaries/vhost-device/](https://git.virtualopensystems.com/virtio-loopback/docs/-/tree/multi-sound-guide/prebuilt_binaries/vhost-device)"
...
directory:
...
>
...
-
...
vhost-device-sound
...
>
...
-
...
vhost-device-sound.d
...
>
...
>Both
...
files
...
needs
...
to
...
be
...
in
...
the
...
same
...
directory
...
After
...
the
...
above
...
process
...
is
...
completed
...
we
...
should
...
be
...
able
...
to
...
locate
...
vhost-device-sound
...
to
...
the
...
following
...
paths:
...
-
...
$HOME/vhost-device/staging/target/debug/vhost-device-sound
...
2.
...
5 Application
...
plays
...
audio
...
stream
...
to
...
a
...
sound
...
system
...
device
...
In
...
order
...
to
...
perform
...
the
...
multi-virtio-sound
...
demo,
...
we
...
need
...
a
...
simple
...
user-space
...
application
...
that
...
plays
...
a
...
sound
...
stream
...
to
...
a
...
sound
...
device.
...
This
...
application
...
is
...
"aplay"
...
command
...
included
...
in
...
the
...
Pike
...
rootfs
...
with
...
target
...
agl-demo-platform.
...
Also
...
sound
...
samples
...
played
...
during
...
that
...
demo
...
can
...
be
...
found
...
in
...
the
...
subdirectory
...
*sound-samples/*.
...
3.
...
Run
...
the
...
demonstration
...
At
...
this
...
point
...
all
...
the
...
components
...
required
...
to
...
start
...
the
...
demo
...
should
...
have
...
been
...
downloaded
...
or
...
built.
...
System
...
Initialization
...
This
...
section
...
is
...
an
...
example
...
of
...
how
...
to
...
run
...
two
...
vhost-user-sound
...
on
...
top
...
of
...
the
...
virtio-loopback
...
architecture.
...
It
...
consists
...
of
...
three
...
steps:
...
-
...
Insert
...
the
...
virtio-sound
...
and
...
virtio-loopback-transport
...
drivers
...
into
...
the
...
kernel
- `insmod` the *virtio_snd.ko*
...
binary
- `insmod $HOME/loopback_driver/virtio-loopback0.ko`
...
- `insmod $HOME/loopback_driver/virtio-loopback1.ko`
...
>
...
By
...
using
...
insmod
...
`virtio-loopback*_logs.ko`
...
prebuilt
...
modules,
...
driver's
...
logs
...
can
...
be
...
seen
...
with
...
`dmesg`
...
but
...
this
...
**will**
...
affect
...
performance.
...
As
...
a
...
result
...
of
...
that
...
process
...
we
...
can
...
see
...
that
...
two
...
new
...
virtio-loopback-driver
...
has
...
been
...
registered
...
in
...
the
...
kernel.
...
This
...
can
...
be
...
verified
...
by
...
running:
...
```
...
$
...
ls
...
/dev/|
...
grep
...
"loopback-"
...
loopback-X
...
loopback-Y
...
```
...
>
...
X
...
and
...
Y
...
are
...
used
...
to
...
discriminate
...
the
...
two
...
virtio-loopback
...
driver
...
instances.
...
-
...
Run
...
the
...
vhost-user-devices
...
- `$HOME/vhost-device/staging/target/debug/vhost-device-sound
...
--socket
...
$HOME/sound.sock0
...
--backend
...
pipewire`
- `$HOME/vhost-device/staging/target/debug/vhost-device-sound
...
--socket
...
$HOME/sound.sock1
...
--backend
...
pipewire`
...
>
...
The
...
"vhost-device-sound"
...
has
...
two
...
arguments
...
"`--socket-path=`"
...
and
...
"`--backend`".
...
The
...
socket
...
argument
...
is
...
the
...
connection
...
between
...
and
...
instance
...
of
...
the
...
vhost-user-device
...
and
...
the
...
virtio-loopback-adapter.
...
Lastly,
...
the
...
backend
...
can
...
be
...
one
...
of
...
the
...
three
...
choices:
...
"null",
...
"alsa",
...
"pipewire".
...
The
...
current
...
demo
...
focuses
...
on
...
"pipewire".
...
>
...
>
...
By
...
using
...
`RUST_LOG=trace`
...
you
...
will
...
be
...
able
...
to
...
see
...
vhost-device-sound's
...
logs
...
but
...
this
...
might
...
affect
...
performance.
...
-
...
Start
...
two
...
instances
...
of
...
virtio-loopback-adapter
...
- `$HOME/adapter_app/adapter
...
-s
...
/home/root/agl/sound.sock0
...
-d
...
vhusnd
...
-l
...
X`
- `$HOME/adapter_app/adapter
...
-s
...
/home/root/agl/sound.sock1
...
-d
...
vhusnd
...
-l
...
Y`
...
>
...
By
...
using
...
`adapter_logs`
...
prebuilt
...
binary,
...
adapter's
...
logs
...
can
...
be
...
seen
...
in
...
the
...
same
...
terminal
...
but
...
this
...
might
...
affect
...
performance.
...
After
...
executing
...
those
...
steps,
...
two
...
new
...
virtio-sound
...
devices
...
should
...
appear
...
in
...
the
...
system.
...
We
...
can
...
verify
...
that
...
and
...
find
...
their
...
names
...
by
...
running:
...
```
...
$
...
aplay
...
-l
...
card
...
1:
...
SoundCard
...
[VirtIO
...
SoundCard],
...
device
...
0:
...
virtio-snd
...
[VirtIO
...
PCM
...
0]
...
Subdevices:
...
1/1
Subdevice #0: subdevice #0
card 2: SoundCard_1 [VirtIO SoundCard], device 0: virtio-snd [VirtIO PCM 0]
Subdevices: 1/1
Subdevice #0: subdevice #0
```
Playing audio on Speakers
All the virtio-loopback infrastructure is now in place. Every request to the virtio-sound driver in kernel space, will be forwarded through virtio-loopback driver and adapter components to the vhost-device-sound device running in user space. At last the vhost-device-sound device via "pipewire" and "ALSA" outputs the audio on speakers.
Playing audio to the speakers is done via two simple commands:
- `aplay -D sysdefault:CARD=SoundCard $HOME/sound-pacthes/agl_demo.wav`
...
+ Playing sound to the speakers via virtio-sound
...
first
...
instance
...
(SoundCard)
...
-
...
`aplay
...
-D
...
sysdefault:CARD=SoundCard_1
...
$HOME/sound-pacthes/agl_ic.wav`
...
+ Playing sound to the speakers via virtio-sound
...
second
...
instance
...
(SoundCard_1)
...