VIO Server - General Overview & Feautures
• VIOS is a special purpose
partition that can serve I/O resources to other partitions. The type of LPAR is
set at
creation. The VIOS LPAR type allows
for the creation of virtual server adapters, where a regular AIX/Linux LPAR
does
not.
• VIOS works by owning a physical
resource and mapping that physical resource to virtual resources. Client LPARs
can
connect to the physical resource via
these mappings.
• VIOS is not a hypervisor, nor is
it required for sub-CPU virtualization. VIOS can be used to manage other
partitions in some situations when a
HMC is not used. This is called IVM (Integrated Virtualization Manager).
• Depending on configurations, VIOS
may or may not be a single point of failure. When client partitions access I/O
via a single path that is delivered
via in a single VIOS, then that VIOS represents a potential single point of
failure for that client partition.
• VIOS is typically configured in
pairs along with various multipathing / failover methods in the client for
virtual
resources to prevent the VIOS from
becoming a single point of failure.
• Active memory sharing and
partition mobility require a VIOS partition. The VIOS partition acts as the
controlling
device for backing store for active
memory sharing. All I/O to a partition capable of partition mobility must be
handled by VIOS as well as the
process of shipping memory between physical systems.
VIO Redundancy
• Fault tolerance in client LPARs in
a VIOS configuration is provided by configuring pairs of VIOS to serve
redundant networking and disk
resources. Additional fault tolerance in the form of NIC link aggregation and /
or
disk multipathing can be provided at
the physical layer to each VIOS server. Multiple paths / aggregations from a
single VIOS to a VIOC do not provide
additional fault tolerance. These multiple paths / aggregations / failover
methods for the client are provided
by multiple VIOS LPARs. In this configuration, an entire VIOS can be lost (ie:
rebooted during an upgrade) without
I/O interruption to the client LPARs.
• In most cases (when using AIX) no
additional configuration is required in the VIOC for this capability. (See
discussion below in regards to SEA
failover vs. NIB for network, and MPIO vs. LVM for disk redundancy.)
• Both virtualized network and disk
redundancy methods tend to be active / passive. For example, it is not possible
to run EtherChannel within the system,
from a VIOC to a single VIOS.
• It is important to understand that
the performance considerations of an active / passive configuration are not
relevant inside the system as all
VIOS can utilize pooled processor resources and therefore gain no significant
(internal) performance benefit by
active / active configurations. Performance benefits of active / active
configurations are realized when
used to connect to outside / physical resources such as EtherChannel (port
aggregation) from the VIOS to a physical switch.
VIO Disk Management
• Disks are presented to VIOC by creating a mapping
between a physical disk or storage pool volume and the vhost adapter
that is associated with the VIOC.
• Best practices configuration suggests that the
connecting VIOS vhost adapter and the VIOC vscsi adapter should use the
same slot number. This makes the typically complex array of virtual SCSI
connections in the system much easier to comprehend.
• The mkvdev command is used to create a mapping between a physical disk and the vhost adapter.
Disk Redundancy in VIO
• LVM mirroring can be used to provide disk
redundancy in a VIOS configuration. One disk should be provided through
each VIOS in a redundant VIOS config to eliminate a VIOS as a single
point of failure.
• LVM mirroring is a client configuration that mirrors data to two different disks presented by two different VIOS.
• Disk path redundancy (assuming an external
storage device is providing disk redundancy) can be provided by a dual
VIOS configuration with MPIO at the client layer.
• Newer NPIV (N Port ID Virtualization) capable cards can be used to provide direct
connectivity of the client to a virtualized FC adapter. Storage
specific multipathing drivers such as PowerPath or HDLM can be used in
the client LPAR. NPIV adapters are virtualized using VIOS, and should be
used in a dual-VIOS configuration.
• MPIO is automatically enabled in AIX if the same disk is presented to a VIOC by two different VIOS.
• LVM mirroring (for client LUNs) is not recommended within
VIOS (ie: mirroring your storage pool in the VIOS). This configuration
would provide no additional protections over LVM mirroring in the VIOC.
• Storage specific multipathing drivers can be used in the VIOS connections to the storage. In this case these drivers should not
then be used on the client. In figure 1, a storage vendor supplied
multipathing driver (such as PowerPath) would be used on VIOS 1 and VIOS
2, and native AIX MPIO would be used in the client. This configuration
gives access to all four paths to the disk and eliminates both VIOS and
any path as a singe point of failure.
Virtual Optical Media
• Optical media can be assigned to an LPAR directly
(by assigning the controller to the LPAR profile), through VIOS (by
presenting the physical CD/DVD on a virtual SCSI controller), or through
file backed virtual optical devices.
• The problem with assigning an optical device to
an LPAR is that it may be difficult to manage in a multiple-LPAR system
and requires explicit DLPAR operations to move it around.
• Assigning an optical device to a VIOS partition
and re-sharing it is much easier as DLPAR operations are not required to
move the device from one partition to another. cfgmgr is simply used to
recognize the device and rmdev is used to "remove" it from the LPAR.
When used in this manner, a physical optical device can only be accessed
by one LPAR at a time.
• Virtual media is a file backed CD/DVD image that
can be "loaded" into a virtual optical device without disruption to the
LPAR configuration. CD/DVD images must be created in a repository before
they can be loaded as virtual media.
• A virtual optical device will show up as a "File-backed Optical" device in lsdev output.
Storage Pools
• Storage pools work much like AIX VGs (Volume
Groups) in that they reside on one or more PVs (Physical Volumes). One
key difference is the concept of a default storage pool. The default
storage pool is the target of storage pool commands where the storage
pool is not explicitly specified.
• The default storage pool is rootvg. If storage pools are used in a configuration then the default storage pool should be changed to something other than rootvg.
Shared Ethernet Adapter (SEA)
• The command used to set up a SEA (Shared Ethernet Adapter) is mkvdev.
• IP addresses cannot be configured on either the
virtual or the physical adapter used in the mkvdev command. IP addresses
are configured either on the SEA itself (created by the mkvdev -sea
command) or another physical or virtual adapter that is not part of a
SEA "bridge". (An example of the latter is seen in Figure 2.)
• Best practices suggest that IP addresses for the VIOS should not be created on the SEA but should be put on another
virtual adapter in the VIOS attached to the same VLAN. This makes the
IP configuration independent of any changes to the SEA. Figure 2, has an
example of an IP address configured on a virtual adapter interface en1
and not any part of the SEA "path". (This is not the case when using SEA
failover).
• The virtual device used in the SEA configuration
should have "Access External Networks" (AKA: "Trunk adapter") checked in
its configuration (in the profile on the HMC). This is the only
interface on the VLAN that should have this checked. Virtual interfaces
with this property set will receive all packets that have addresses
outside the virtual environment. In figure 2, the interface with "Access
External Networks" checked should be ent2.
• If multiple virtual interfaces are used on a
single VLAN as trunk adapters then each must have a different trunk
priority. This is typically done with multiple VIOS servers - with one
virtual adapter from the same VLAN on each VIO server. This is required
when setting up SEA Failover in the next section.
Network Redundancy in VIOS
• The two primary
methods of providing network redundancy for a VIOC in a dual VIOS
configuration are NIB (Network Interface Backup) and SEA Failover.
(These provide protection from the loss of a VIOS or VIOS connectivity
to a physical LAN.)
• NIB creates a link aggregation in the client of a
single virtual NIC with a backup NIC. (Each virtual NIC is on a
separate VLAN and connected to a different VIOS.) This configuration is
done in each client OS. (See Figure 3 for an example of the VIOC that uses NIB to provide redundant connections to the LAN.)
• SEA Failover is a VIOS configuration option that
provides two physical network connections, one from each VIOS. No client
configuration is required as only one virtual interface is presented to
the client.
• Most Power 6 based systems offer IVE (Integrated
Virtual Ethernet) hardware that provides virtual NICs to clients. These
do not provide redundancy and must be used in pairs or with another NIC /
backup path on each client (or VIOS) to provide that capability.
• NIB and SEA Failover are not mutually exclusive
and can be used together or with link aggregation (EtherChannel /
802.3ad) to a physical device in the VIOS. Figure 3 shows a link
aggregation device (ent3) in VIOS 1 as the physical trunk adapter for
the SEA (ent4) in what is seen by the client as a NIB configuration.
• Link aggregation (EtherChannel / 802.3ad) of more
than one virtual adapter is not supported or necessary from the client
as all I/O moves at memory speed in the virtual environment. The more
appropriate method is to direct different kinds of I/O or networks to
particular VIOS servers where they do not compete for CPU time.
• The primary benefit of NIB (on the client) is
that the administrator can choose the path of network traffic. In the
case of figure 3, the administrator would configure the client to use
ent0 as the primary interface and ent1 as the backup. Then more
resources (in the form of aggregate links) can be used on VIOS1 to
handle the traffic with traffic only flowing through VIOS2 in the event
of a failure of VIOS1. The problem with this configuration is that it
requires additional configuration on the client and is not conducive as
SEA Failover to simplistic NIM installs.
• NIB configurations also allow the administrator to balance clients so that all traffic does not
go to a single VIOS. In this case hardware resources would be
configured more evenly than they are in figure 3 with both VIOS servers
having similar physical connections as appropriate for a more "active /
active" configuration.
SEA Failover
• Unlike a regular SEA adapter, a SEA failover configuration has a few settings that are different from stated best practices.
• A SEA failover configuration is a situation when IP addresses should be configured on the SEA adapter.
• A control channel must be configured between the
two VIOS using two virtual ethernet adapters that use that VLAN strictly
for this purpose. The local virtual adapter created for this purpose
should be specified in the ctl_chan attribute in each of the SEA setups.
• Both virtual adapters (on the VLAN with clients)
should be configured to "Access External network", but one should have a
higher priority (lower number) for the "Trunk priority" option. A SEA
failover configuration is the only time that you should have two virtual
adapters on the same VLAN that are configured in this manner.
Best Practices / Additional Notes
• Virtual Ethernet devices should only have 802.1Q
enabled if you intend to run additional VLANs on that interface. (In
most instances this is not the case).
• Only one interface should be configured to
"Access External Networks" on a VLAN, this should be the virtual
interface used for the SEA on the VIOS and not the VIOC. This is the
"gateway" adapter that will receive packets with MAC addresses that are
unknown. (This is also known as a "Trunk adapter" on some versions of
the HMC.)
• VIOS partitions are unique in that they can have
virtual host adapters. Virtual SCSI adapters in VIOC partitions connect
to LUNs shared through VIOS virtual host adapters.
• An organized naming convention to virtual devices
is an important method to simplifying complexity in a VIOS environment.
Several methods are used in this document, but each represents a
self-documenting method that relates what the virtual device is and what
it serves.
• VIOS commands can be run from the HMC. This may
be a convenient alternative to logging into the VIOS LPAR and running
the command.
• Power 6 based systems have an additional LPAR
parameter called the partition "weight". The additional RAS features
will use this value in a resource constrained system to kill a lower priority LPAR in the event of a CPU failure.
• The ratio of virtual CPUs in a partition to the
actual amount of desired / entitled capacity is a "statement" on the
partitions ability to be virtualized. A minimal backing of actual
(physical) CPU entitlement to a virtual CPU suggests that the LPAR will
most likely not be using large amounts of CPU and will relinquish unused
cycles back to the shared pool the majority of the time. This is a
measure of how over-committed the partition is.
• multiple profiles created on the HMC can
represent different configurations such as with and without the physical
CD/DVD ROM. These profiles can be named (for example) as
<partition_name>_prod and <partition_name>_cdrom.
• HMC partition configuration and profiles can be saved to a file and backed up to either other HMCs or remote file systems.
• sysplan files can be created on the HMC or the
SPT (System Planning Tool) and exported to each other. These files are a
good method of expressing explicit configuration intent and can serve
as both documentation as well as a (partial) backup method of
configuration data.
• vhost adapters should be explicitly assigned and
restricted to client partitions. This helps with documentation (viewing
the config in the HMC) as well as preventing trespass of disks by other
client partitions (typically due to user error).
0 comments:
Post a Comment