Accessing Virtual Machines¶
Graphical and Serial Console Access¶
Once a virtual machine is started you are able to connect to the consoles it exposes. Usually there are two types of consoles:
- Serial Console
- Graphical Console (VNC)
Note: You need to have
virtctl
installed to gain access to the VirtualMachineInstance.
Accessing the Serial Console¶
The serial console of a virtual machine can be accessed by using the console
command:
Accessing the Graphical Console (VNC)¶
To access the graphical console of a virtual machine the VNC protocol is
typically used. This requires remote-viewer
to be installed. Once the
tool is installed, you can access the graphical console using:
If you only want to open a vnc-proxy without executing the remote-viewer
command, it can be accomplished with:
This would print the port number on your machine where you can manually connect using any VNC viewer.
Debugging console access¶
If the connection fails, you can use the -v
flag to get more verbose
output from both virtctl
and the remote-viewer
tool to troubleshoot the
problem.
Note: If you are using virtctl via SSH on a remote machine, you need to forward the X session to your machine. Look up the -X and -Y flags of
ssh
if you are not familiar with that. As an alternative you can proxy the API server port with SSH to your machine (either direct or in combination withkubectl proxy
).
SSH Access¶
A common operational pattern used when managing virtual machines is to inject SSH public keys into the virtual machines at boot. This allows automation tools (like Ansible) to provision the virtual machine. It also gives operators a way of gaining secure and passwordless access to a virtual machine.
KubeVirt provides multiple ways to inject SSH public keys into a virtual machine.
In general, these methods fall into two categories: - Static key injection, which places keys on the virtual machine the first time it is booted. - Dynamic key injection, which allows keys to be dynamically updated both at boot and during runtime.
Once a SSH public key is injected into the virtual machine, it can be
accessed via virtctl
.
Static SSH public key injection via cloud-init¶
Users creating virtual machines can provide startup scripts to their virtual machines, allowing multiple customization operations.
One option for injecting public SSH keys into a VM is via cloud-init startup script. However, there are more flexible options available.
The virtual machine's access credential API allows statically injecting SSH
public keys at startup time independently of the cloud-init user data by
placing the SSH public key into a Kubernetes Secret
. This allows keeping the
application data in the cloud-init user data separate from the credentials
used to access the virtual machine.
A Kubernetes Secret
can be created from an SSH public key like this:
# Place SSH public key into a Secret
kubectl create secret generic my-pub-key --from-file=key1=id_rsa.pub
The Secret
containing the public key is then assigned to a virtual machine
using the access credentials API with the noCloud
propagation method.
KubeVirt injects the SSH public key into the virtual machine by using the generated cloud-init metadata instead of the user data. This separates the application user data and user credentials.
Note: The cloud-init
userData
is not touched.
# Create a VM referencing the Secret using propagation method noCloud
kubectl create -f - <<EOF
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: testvm
spec:
running: true
template:
spec:
domain:
devices:
disks:
- disk:
bus: virtio
name: containerdisk
- disk:
bus: virtio
name: cloudinitdisk
rng: {}
resources:
requests:
memory: 1024M
terminationGracePeriodSeconds: 0
accessCredentials:
- sshPublicKey:
source:
secret:
secretName: my-pub-key
propagationMethod:
noCloud: {}
volumes:
- containerDisk:
image: quay.io/containerdisks/fedora:latest
name: containerdisk
- cloudInitNoCloud:
userData: |-
#cloud-config
password: fedora
chpasswd: { expire: False }
name: cloudinitdisk
EOF
Dynamic SSH public key injection via qemu-guest-agent¶
KubeVirt allows the dynamic injection of SSH public keys into a VirtualMachine with the access credentials API.
Utilizing the qemuGuestAgent
propagation method, configured Secrets are attached to a VirtualMachine when the VM is started.
This allows for dynamic injection of SSH public keys at runtime by updating the attached Secrets.
Please note that new Secrets cannot be attached to a running VM: You must restart the VM to attach the new Secret.
Note: This requires the qemu-guest-agent to be installed within the guest.
Note: When using qemuGuestAgent propagation, the
/home/$USER/.ssh/authorized_keys
file will be owned by the guest agent. Changes to the file not made by the guest agent will be lost.Note: More information about the motivation behind the access credentials API can be found in the pull request description that introduced the API.
In the example below the Secret
containing the SSH public key is
attached to the virtual machine via the access credentials API with the
qemuGuestAgent
propagation method. This allows updating the contents of
the Secret
at any time, which will result in the changes getting applied
to the running virtual machine immediately. The Secret
may also contain
multiple SSH public keys.
# Place SSH public key into a secret
kubectl create secret generic my-pub-key --from-file=key1=id_rsa.pub
Now reference this secret in the VirtualMachine
spec with the access
credentials API using
qemuGuestAgent
propagation.
# Create a VM referencing the Secret using propagation method qemuGuestAgent
kubectl create -f - <<EOF
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: testvm
spec:
running: true
template:
spec:
domain:
devices:
disks:
- disk:
bus: virtio
name: containerdisk
- disk:
bus: virtio
name: cloudinitdisk
rng: {}
resources:
requests:
memory: 1024M
terminationGracePeriodSeconds: 0
accessCredentials:
- sshPublicKey:
source:
secret:
secretName: my-pub-key
propagationMethod:
qemuGuestAgent:
users:
- fedora
volumes:
- containerDisk:
image: quay.io/containerdisks/fedora:latest
name: containerdisk
- cloudInitNoCloud:
userData: |-
#cloud-config
password: fedora
chpasswd: { expire: False }
# Disable SELinux for now, so qemu-guest-agent can write the authorized_keys file
# The selinux-policy is too restrictive currently, see open bugs:
# - https://bugzilla.redhat.com/show_bug.cgi?id=1917024
# - https://bugzilla.redhat.com/show_bug.cgi?id=2028762
# - https://bugzilla.redhat.com/show_bug.cgi?id=2057310
bootcmd:
- setenforce 0
name: cloudinitdisk
EOF
Accessing the VMI using virtctl¶
The user can create a websocket backed network tunnel to a port inside the
instance by using the virtualmachineinstances/portforward
subresource of the
VirtualMachineInstance
.
One use-case for this subresource is to forward SSH traffic into
the VirtualMachineInstance
either from the CLI or a web-UI.
To connect to a VirtualMachineInstance
from your local machine, virtctl
provides a lightweight SSH client with the ssh
command, that uses port
forwarding. Refer to the command's help for more details.
To transfer files from or to a VirtualMachineInstance
virtctl
also
provides a lightweight SCP client with the scp
command. Its usage is
similar to the ssh
command. Refer to the command's help for more details.
Using virtctl as proxy¶
If you prefer to use your local OpenSSH client, there are two ways of doing that in combination with virtctl.
Note: Most of this applies to the
virtctl scp
command too.
- The
virtctl ssh
command has a--local-ssh
option. With this optionvirtctl
wraps the local OpenSSH client transparently to the user. The executed SSH command can be viewed by increasing the verbosity (-v 3
).
- The
virtctl port-forward
command provides an option to tunnel a single port to your local stdout/stdin. This allows the command to be used in combination with the OpenSSH client'sProxyCommand
option.
ssh -o 'ProxyCommand=virtctl port-forward --stdio=true vmi/testvm.mynamespace 22' fedora@testvm.mynamespace
To provide easier access to arbitrary virtual machines you can add the following
lines to your SSH config
:
Host vmi/*
ProxyCommand virtctl port-forward --stdio=true %h %p
Host vm/*
ProxyCommand virtctl port-forward --stdio=true %h %p
This allows you to simply call ssh user@vmi/testvmi.mynamespace
and your
SSH config and virtctl will do the rest. Using this method it becomes easy
to set up different identities for different namespaces inside your SSH
config
.
This feature can also be used with Ansible to automate configuration of virtual
machines running on KubeVirt. You can put the snippet above into its own
file (e.g. ~/.ssh/virtctl-proxy-config
) and add the following lines to
your .ansible.cfg
:
Note that all port forwarding traffic will be sent over the Kubernetes
control plane. A high amount of connections and traffic can increase
pressure on the API server. If you regularly need a high amount of connections
and traffic consider using a dedicated Kubernetes Service
instead.
Example¶
-
Create virtual machine and inject SSH public key as explained above
-
SSH into virtual machine
or
ssh -o 'ProxyCommand=virtctl port-forward --stdio=true vmi/testvm.mynamespace 22' -i id_rsa fedora@vmi/testvm.mynamespace
- SCP file to the virtual machine
# Add --local-ssh to transparently use local OpenSSH client
virtctl scp -i id_rsa testfile fedora@testvm:/tmp
or
scp -o 'ProxyCommand=virtctl port-forward --stdio=true vmi/testvm.mynamespace 22' -i id_rsa testfile fedora@testvm.mynamespace:/tmp
RBAC permissions for Console/VNC/SSH access¶
Using default RBAC cluster roles¶
Every KubeVirt installation starting with version v0.5.1 ships a set of default RBAC cluster roles that can be used to grant users access to VirtualMachineInstances.
The kubevirt.io:admin
and kubevirt.io:edit
cluster roles have console,
VNC and SSH respectively port-forwarding access permissions built into them. By
binding either of these roles to a user, they will have the ability to use
virtctl to access the console, VNC and SSH.
Using custom RBAC cluster role¶
The default KubeVirt cluster roles grant access to more than just the
console, VNC and port-forwarding. The ClusterRole
below demonstrates how
to craft a custom role, that only allows access to the console, VNC and
port-forwarding.
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: allow-console-vnc-port-forward-access
rules:
- apiGroups:
- subresources.kubevirt.io
resources:
- virtualmachineinstances/console
- virtualmachineinstances/vnc
verbs:
- get
- apiGroups:
- subresources.kubevirt.io
resources:
- virtualmachineinstances/portforward
verbs:
- update
When bound with a ClusterRoleBinding
the ClusterRole
above grants access
to virtual machines across all namespaces.
In order to reduce the scope to a single namespace, bind this ClusterRole
using a RoleBinding
that targets a single namespace.