Layer 2 User Defined Networks for OpenShift Virtualization¶

April 22, 2025

20 min read

Learn how to set up Layer 2 User Defined Networks (UDN) for VMs in OpenShift Virtualization for isolated tenant networks with multi-node connectivity.

This guide shows how to implement UDNs in OpenShift Virtualization for network isolation between VMs within or across namespaces. UDNs create isolated tenant networks separate from the default pod network, enabling complex network layouts and better security.

Implementation Steps¶

1. Understand User Defined Networks¶

OpenShift Virtualization offers two UDN types:

UserDefinedNetwork (UDN): Creates network isolation within a single namespace.
ClusterUserDefinedNetwork (Cluster UDN): Spans multiple namespaces, allowing VMs in different namespaces to communicate while staying isolated from other networks.

Layer 2 vs Layer 3 User Defined Networks¶

UDNs can use either Layer 2 or Layer 3 topology:

Layer 2 UDN

Creates a virtual switch across all cluster nodes
All VMs connect to the same subnet regardless of node location
Provides a single broadcast domain
Supports VM live migration between nodes
Best for apps needing Layer 2 adjacency
Simpler to configure

Layer 3 UDN

Creates unique Layer 2 segments per node with Layer 3 routing between them
Each node has its own subnet
Controls broadcast traffic within node-specific segments
Better scalability for large deployments
Supports NetworkPolicy for security control
Better for environments with many VMs

Primary vs Secondary Networks

Primary networks handle default traffic
Secondary networks add interfaces for specific traffic
A VM can have one primary but multiple secondary networks
UDNs work as either primary or secondary networks

Important

Compatibility Note:

Only Layer 2 UDNs work as primary networks for VMs
Layer 3 UDNs only work with regular pods, not VMs

This guide focuses on Layer 2 UDNs for VM workloads that need simplicity and migration support.

2. Create a Namespace for User Defined Network¶

Create a namespace with the UDN label:

udn-namespace.yaml¶

apiVersion: v1
kind: Namespace
metadata:
  name: udn-example
  labels:
    k8s.ovn.org/primary-user-defined-network: ""

Important

Add this label when creating the namespace. You cannot add it to existing namespaces.

Apply the namespace:

oc create -f udn-namespace.yaml

3. Create a User Defined Network¶

Create a UDN in your namespace:

udn-example.yaml¶

apiVersion: k8s.ovn.org/v1
kind: UserDefinedNetwork
metadata:
  name: udn-example
  namespace: udn-example
spec:
  layer2:
    ipam:
      lifecycle: Persistent
    role: Primary
    subnets:
    - 10.200.0.0/16
  topology: Layer2

Apply the UDN:

oc create -f udn-example.yaml

Verify creation:

oc get userdefinednetwork -n udn-example

4. Create a VM on the User Defined Network¶

VMs in a namespace with a UDN automatically use the UDN as their primary network:

example-vm.yaml¶

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: example-vm
  namespace: udn-example
spec:
  runStrategy: Always
  template:
    spec:
      domain:
        devices:
          disks:
          - name: rootdisk
            disk:
              bus: virtio
          interfaces:
          - name: default
            binding:
              name: l2bridge
        resources:
          requests:
            memory: 2Gi
      networks:
      - name: default
        pod: {}
      volumes:
      - name: rootdisk
        containerDisk:
          image: quay.io/containerdisks/fedora:latest

Note

Don’t modify the VM’s network config. The UDN system handles it automatically.

Important

Restart or live migrate the VM after network changes:

# Restart the VM:
virtctl restart example-vm

# Or live migrate (no downtime):
virtctl migrate example-vm

5. Create a Cluster User Defined Network¶

For cross-namespace communication, create a Cluster UDN:

Create namespaces with the proper labels:

udn-namespaces.yaml¶

---
apiVersion: v1
kind: Namespace
metadata:
  name: udn-prod
  labels:
    k8s.ovn.org/primary-user-defined-network: ""
    cluster-udn: prod
---
apiVersion: v1
kind: Namespace
metadata:
  name: udn-preprod
  labels:
    k8s.ovn.org/primary-user-defined-network: ""
    cluster-udn: prod

Create the Cluster UDN:

cluster-udn-prod.yaml¶

apiVersion: k8s.ovn.org/v1
kind: ClusterUserDefinedNetwork
metadata:
  name: cluster-udn-prod
spec:
  namespaceSelector:
    matchLabels:
      cluster-udn: prod
  network:
    layer2:
      ipam:
        lifecycle: Persistent
      role: Primary
      subnets:
      - 10.100.0.0/16
    topology: Layer2

Apply the config:
```
oc create -f cluster-udn-prod.yaml
```
Verify creation:
```
oc get clusteruserdefinednetwork
```

Testing and Validation¶

1. Test VM Connectivity within UDN¶

Create multiple VMs in the same namespace with a UDN

Connect to the VM:

# Use console
virtctl console example-vm

# Or SSH if available
virtctl ssh example-vm

Check network config:
```
ip addr show
ip route
```
Test connectivity between VMs:
```
ping <other-vm-ip-address>
```
Verify VMs got IP addresses from the UDN subnet

2. Test VM Connectivity across Cluster UDN¶

Create VMs in different namespaces connected by a Cluster UDN
Check network config in each VM

Test connectivity:

# From VM in namespace 1
ping <vm-ip-in-namespace-2>

3. Test North/South Network Access¶

Test external network access:
```
# From inside the VM
ping 1.1.1.1
```

Important

Don’t manually configure IP addresses. The UDN controller manages IP allocation through IPAM.