To setup and install Hyper v Server on Azure, the recommended way is to use the image available in the marketplace. Deploy on Windows Server with Hyper-V and build as many nested VMs as you need. Make sure to deploy on the supported Azure VM types. Details below.
Supported Hardware types for Nested Virtualization
When deploying Hyper-V on Azure, you will need to make sure the VM size you choose supports Hyper-threaded and capable of running nested virtualization. You can view the supported virtual machine types on the following link.
Setting up Hyper-V on Azure
We now need to get Hyper-V configured for your new nested environment. We will need to do the following in this tutorial to confirm the setup is working in your cloud nested environment. If you dont need to give your nested vms network access to your Azure VMs then you can skip setting up RRAS, Adding a 2nd NIC and setting up Azure defined routes.
- Add 2nd NIC to Hyper-V Host
- Create 2 subnets on Azure vNet
- Enable IP Forwarding on Hyper-V NIC
- Create a NAT’ed vSwitch for outside connectivity
- Setup RRAS (NAT)
- Create a guest virtual machine
- Configure the guest virtual machine
- Configure an IP address on the nested guest virtual machine
- Setup Azure Defined Routes
- Test connectivity
Setup Hyper-V on Windows
The first step is to run the following powershell command to get Hyper-v up and running. It will require a reboot after:
Install-WindowsFeature -Name Hyper-V -IncludeManagementTools -Restart
Add 2nd Nic and enable IP Forwarding
I’m going to create a new Hyper-V switch that will be NAT’ed and also create 2 subnets on my Azure vNet so Azure VMs can communicate with my Hyper-V VMs.
My Azure VNET address range space is 10.0.0.0/16
Two subnets were then configured for the VNET in Azure like so:
- NAT with address space of 10.0.1.0/24 (how the host VM provides connectivity for hosted VMs)
- LAN with address space of 10.0.2.0/24 (how Azure VMs talk to the host and nested VMs)
I will then create a Hyper-V switch later that will use 192.168.0.0/24 address space. This address space can be anything as Azure wont know anything about it.
First we need to power off our Hyper-V host and add an additional IP Address. Within Azure under your Hyper-V host networking properties, click on ‘Attach network interface‘ > Create and attach network interface
Follow the setup wizard making sure its in the resource group of your hyper v host and the subnet is LAN
Once the new second NIC has been created, we now need to enable IP forwarding on the NIC. I’ve called my new nic LanNIC but you can call it anything you like.
On the properties on the LanNIC, select IP configurations and Enable IP Forwarding and press save
Now we can start the VM and login via RDP.
The next first thing that you’re going to want to do is rename your network connections. This will help you later. Run an ipconfig command to see what NIC is on what subnet (your NAT or LAN subnet) and then rename them as NAT or LAN respectively. Also, these interfaces will be referenced throughout the rest of this post so if you don’t rename them now, you’ll most likely get confused later on. They should look something like this before continuing:
Create Virtual Switch
We’re now ready to get started on configuring Hyper-V. Open up Hyper-V manager, right click on your server in the left pane and choose Virtual Switch Manager
Select Internal and then Create virtual switch
Give the new vswitch a name, i will call mine InternalNATSwitch and then hit ok. You will then notice you will temporarily loose network connection to your VM RDP connection and then it will re connect.
In order to configure the NAT’ed IP address, you need to query the interface indexes to know which interface to use for the NAT’ed connection to the virtual machine. To do that, you can use the following PowerShell cmdlet:
Remember that third subnet that wasn’t configured in Azure? It’s time to put it to work. In my example, the nested VMs are going to be on the 192.168.0.0/24 subnet like we talked about at the beginning of this post. You’ll use the first IP in that range as the default gateway for the nested VM switch. Replace my example IP settings below with your own subnet information and ifIndex number and run this PowerShell command to create the IP address that will be used for outside connectivity:
New-NetIPAddress -IPAddress 192.168.0.1 -PrefixLength 24 -InterfaceIndex 32
Setting up RRAS for Hypver-V NAT
In order for Hyper-V to provide connectivity in and out for the nested VMs we need to install the RRAS role on the Hyper-V host and configure NATing.
Run the following powershell command to install RRAS
Install-WindowsFeature -Name Routing -IncludeManagementTools
Once RRAS is installed, Open your Hyper-V host’s new Routing and Remote Access console from Windows Administrative Tools. Your service hasn’t been configured or started yet so you’ll see a red down arrow by your server name. Just right-click on your server name and choose Configure and Enable Routing and Remote Access. In the Routing and Remote Access Server Setup Wizard, select Custom configuration and then NAT and LAN routing, and finish the wizard to start the service.
RRAS is enabled, but it’s not really doing anything because we haven’t told it what interface to use. Right-click NAT under IPv4 and select New Interface. Select your NAT interface, click OK, and then enable both options for Public interface connected to the Internet and its child option to enable NAT on the interface.
To allow the nested VMs to talk to non-Hyper-V, Azure-based VMs on same VNET you created in the beginning of all this, you must now configure a few static routes in RRAS. Remember, your Host VM has two NICs. How does it know what network traffic is for itself and what traffic goes back and forth to nested VMs? Here’s how to let it know.
Still under IPv4, right-click Static Routes, and select New Static Route. You’ll have a few interface options to choose from.
The first, NAT, should be the host VM’s primary NIC—it’s first one. Here’s where you tell the host what network traffic is destined for itself and not for the nested VMs. The next, LAN, is used to configure the host to pass traffic to and from with the hosted VMs and the rest of your Azure VNET. The InternalNATSwitch interface is just for Hyper-V so that one can be ignored at this point. Configure these interfaces according to the subnets you’ve created earlier.
Here is my VNET and subnet address space:
- The virtual network is called hyperv2019-vnet with an address space of 10.0.0.0/16
- Two subnets were then setup on the VNET in Azure with the following config:
- NAT with address space of 10.0.1.0/24
- LAN with address space of 10.0.2.0/24
Now, this gets confusing, but the important thing to think about here is that the VNET has a /16 subnet mask and the subnets end in /24. That means we need to configure the routes so that anything going to a /24 is first picked up by the host and anything destined anywhere else, including our LAN subnet, will be dialing a number through the LAN interface to somewhere in the /16 area code. In my example, these routes are configured as follows:
Nested VMs can now talk to Azure VMs, but what about the other way around, This is where Azure user defined routes come in. We will do that later. First lets get our first nested VM up and running..
Create Guest Virtual Machine
I’m now going to create a Hyper-v nested guest virtual machine using they Hyper-v manager console, Making sure the Configure Networking step is set to use the NAT’ed switch.
I’ll be downloading an evaluation Windows Server 2019 ISO from Microsoft and booting the VM from the ISO in order to install the OS.
Configure Guest VM / Add IP Address
Once the guest VM has booted up, we now need to give it an IP address from the NAT’ed address range. I’m going to use 192.168.0.2 for the IP address and the default gateway of our NAT’ed internal switch (192.168.0.1)
The DNS server address is our Azure DNS server. 22.214.171.124. To find yours, simply rung IPConfig/all on your Hyper-v host. This VM can now access the internet using Azure DNS.
Create Azure User Defined Routes
We now need to setup custom Azure user defined routes so our traffic from Azure can access our nested VMs.
Following the MS guide on setting up a route table – https://docs.microsoft.com/en-us/azure/virtual-network/tutorial-create-route-table-portal#create-a-route-table
I’m going to be routing anything looking for a 192.168.0.0/24 address (my nested VM Hyper-V virtual switch) to my LAN NIC address (not NAT subnet address space) of 10.0.2.4/24. Run IPCONFIG on your host VM to see what your LAN IP is to use here. You’ll also need to use your own Hyper-v resource group, Location, VNET, and subnet names.
Here is my route config:
All traffic to 192.168.0.0/24 goes to next hop type > Virtual Appliance and next hop address is of my LAN NIC 10.0.2.4
Next is to associate this custom route with our LAN Subnet and any production Subnets you want existing Azure VMs to access our nested VMs by clicking on the Subnets > Associate of our Route table. I’ve got another subnet called servers where i have production VMs that will need access to my nested VMs. You may need to reboot your VMs for the new routes to take effect.
For my testing i’ve disabled the Windows firewall on my nested VM, Hyper-V host and also another server i have setup called Jumpbox with an ip address of 10.0.4.4 in my Azure VNet under a new subnet called servers that i’ve associated the Azure user defined route to. I also dont have any Azure Network Security groups setup, so you will need to keep that in mind in your environment and enable any ports that you may need in production.
From my Jumpbox i can ping our nested vm (192.168.0.2)
And from my nested vm (192.168.0.2) i can ping my jump box on azure (10.0.4.4)
If you experience network connectivity issues between VMs, make sure you have enabled ports on your Windows Firewalls and also check your Azure Network Security Groups are not blocking any ports you may need.
To setup Azure firewall rules refer to – Azure Network Security Groups