Installing Logical Domains Over ZFS

LDom Manager 1.3 needs to be installed on the top of Solaris 10u8 to setup the virtualization system. See

Download file. unzip it and run the install-ldm script from the unzip folder

# ./Install/install-ldm
# cd $DIR/LDoms_Manager-1_3

Verify the installation with executing following command.

# /opt/SUNWldm/bin/ldm list

Allocate systems resources to the primary (or control) domain

Creating the control domain with 8 vcpu’s and 4 GB RAM.

Note: If you have any cryptographic devices in the control domain, you cannot dynamically reconfigure CPUs. So if you are not using cryptographic devices, set-mau to 0.
Assign one cryptographic resource to the control domain, primary. This leaves the remainder of the cryptographic resources available to a guest domain.

# ldm set-mau 1 primary

Assigning 8 virtual CPUs and 4GB memory to the control domain, primary. This leaves the remainder of the virtual CPUs available to a guest domains.

# ldm set-vcpu 8 primary
# ldm set-memory 4G  primary

Making the modified configuration permanent using list-spconfig option.

# ldm list-spconfig
factory-default [current]

Adding a logical domain machine configuration to the system controller (SC).

# ldm add-spconfig initial
# ldm list-spconfig
factory-default [current]
initial [next]

Reboot the server to come up with the initial configuration.

# shutdown -i6 -g0 -y

Configure the virtual switch on the primary domain.

By default, networking between the control/service domain and other domains in the system is disabled. To enable this, the virtual switch device should be configured as a network device. The virtual switch can either replace the underlying physical device (nxge0 in this example) as the primary interface or be configured as an additional network interface in the domain.

Plumb the virtual switch (vsw0) on the primary domain

# ifconfig vsw0 plumb

Bring down the primary interface

# ifconfig nxge0 down unplumb

Configure virtual switch

# ifconfig vsw0 netmask broadcast + up

Modify the hostname file to make this configuration permanent

# mv /etc/hostname.nxge0 /etc/hostname.vsw0

Enable virtual network terminal server daemon

# svcadm enable vntsd
# svcs vntsd
STATE          STIME    FMRI
online         Jun_17   svc:/ldoms/vntsd:default

Create a template LDOM (Oracle VM for SPARC virtual machine)

Create a ZFS file system that will be used to create virtual disks for VMs. We can use either a local disk or iSCSI target created on the storage for creating ZFS over it.
Create a ZFS pool on the localdisk.

# zpool create ldompool c1t1d0
# zfs list
ldompool    72K   134G    21K  /ldompool
# zpool list
ldompool   136G  76.5K   136G     0%  ONLINE  -

Creating a 25GB ZFS volume for the first disk.

# zfs create -V 25g ldompool/disk1
# zfs list
ldompool        25.0G   109G    21K  /ldompool
ldompool/disk1    25G   134G    16K  -

Specify the device to be exported by the virtual disk server as a virtual disk to the guest domain.

# ldm add-vdiskserverdevice /dev/zvol/dsk/ldompool/disk1 vol1@primary-vds0

Creating a guest domain ld01 with 30 VPUs and 7GB Memory.

# ldm add-domain ld01
# ldm set-vcpu 30 ld01
# ldm set-memory 7G ld01

Add a virtual network device vnet1 to the guest domain ld01

# ldm add-vnet vnet1 primary-vsw0 ld01
# ldm add-vdisk vdisk1 vol1@primary-vds0 ld01

Adding the ISO image as a virtual device that will be used as the installation media.

# ldm add-vdiskserverdevice /installdvd/sol-10-u8-ga-sparc-dvd.iso iso@primary-vds0
# ldm add-vdisk iso iso@primary-vds0 ld01

Set auto-boot and boot-device variables for the guest domain.

# ldm set-variable auto-boot\?=false ld01
# ldm bind-domain ld01
# ldm start-domain ld01

Connect to the guest domain console from control domain and start the installation.

# telnet localhost 5000
Connected to localhost.
Escape character is '^]'.
Connecting to console "ldom1" in group "ldom1" ....
Press ~? for control options ..
{0} ok show-disks
a) /virtual-devices@100/channel-devices@200/disk@1
b) /virtual-devices@100/channel-devices@200/disk@0
Enter Selection, q to quit: a
/virtual-devices@100/channel-devices@200/disk@1 has been selected.
Type ^Y ( Control-Y ) to insert it in the command line. e.g. ok nvalias mydev ^Y
for creating devalias mydev for

{0} ok devalias
iso /virtual-devices@100/channel-devices@200/disk@1
vdisk1 /virtual-devices@100/channel-devices@200/disk@0
vnet0 /virtual-devices@100/channel-devices@200/network@0
net /virtual-devices@100/channel-devices@200/network@0 disk /virtual-devices@100/channel-devices@200/disk@0 virtual-console /virtual-devices/console@1
name aliases

Now boot from the virtual iso image appending the :f (this is to specify the slice 6 of the DVD/ISO image). This can also be done using boot iso:f

{0} ok boot /virtual-devices@100/channel-devices@200/disk@1:f

Follow on screen instructions to complete rest of the solaris installation.

As we have used ZFS pool, New LDOMs can be easily created with ZFS snapshot.
Execute sys-unconfig on ld01. This will halt the guest domain ld01 and allow a snapshot of the base ldom’s disk to be taken.

# sys-unconfig

stop ld01 domain

# ldm stop-domain ld01-db01

Remove the ISO disk from the guest domain, and execute the snap shot of the disk image

# ldm remove-vdisk iso ld01
# zfs snapshot ldompool/disk1@base

# zfs list
ldompool             30.2G   104G    21K  /ldompool
ldompool/disk1       30.2G   129G  5.19G  -
ldompool/disk1@base      0      -  5.19G  -

Clone the gold ldom disk image for the new guest domain ld02

# zfs clone ldompool/disk1@base ldompool/disk2
# zfs list
ldompool             30.2G   104G    21K  /ldompool
ldompool/disk1       30.2G   129G  5.19G  -
ldompool/disk1@base      0      -  5.19G  -
ldompool/disk2           0   104G  5.19G  -

Add a new domain

# ldm add-domain ld02
# ldm set-vcpu 30 ld02
# ldm set-memory 7G ld02
# ldm add-vnet vnet1 primary-vsw0 ld02

# ldm add-vdiskserverdevice /dev/zvol/dsk/ldompool/disk2 vol2@primary-vds0

# ldm add-vdisk vdisk1 vol2@primary-vds0 ld02
# ldm set-variable auto-boot\?=false ld02
# ldm bind-domain ld02
# ldm start-domain ld02

One thought on “Installing Logical Domains Over ZFS

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s