Azure virtual WAN (vWAN): Lessons Learned

Azure virtual WAN (vWAN ) is undoubtedly a strong choice in the wide-area network (WAN) connectivity model, utilizing third-party backbones. It holds significant potential and functionalities, but it is also a relatively immature and evolving service with certain limitations in use cases that deviate from the Azure reference architecture.

In this post, I would like to share my first-hand experiences and lessons learned from integrating vWAN into my use case. I hope that these insights can assist you in making decisions and implementing workarounds to address challenges arising from certain limitations.

Here we go!🚀

Next, the list of topics covered in this post:

• What is vWAN?
• vWAN Reference Architecture: Hub and Direct Spokes
• Problem Statement
• Topology Description
• Limitations encountered
  • Restriction to only use Default Route Table when BGP is enabled
  • BGP route-maps are in Preview
  • Not possible to change the next-hop IP address for BGP advertisements from an NVA
  • Static routes must be configured in every vHub
  • BGP cannot overwrite certain system routes
  • One Virtual Network Gateway (VGW)
• Conclusion

What is vWAN?

vWAN is a networking service offered by Microsoft Azure that simplifies and optimizes connectivity for businesses with multiple vNETs in different Azure regions, on-prem data centers (DCs), branch offices and remote users. It enables organizations to build a unified and scalable WAN infrastructure by leveraging Microsoft’s global network backbone.

vWAN Reference Architecture: Hub and Direct Spokes

Azure vWAN is a hub-and-spoke architecture. vWAN allows remote locations and users to connect to a central hub in Azure, which in turn connects to other hubs and resources in Azure Cloud.

This architecture enhances network performance, security and management, offering features such as automated VPN and ExpressRoute connections, integrated security through Azure Firewall (secured vHub) and Virtual Network Appliances (NVA), as well as centralized monitoring and management through the Azure portal.

Problem Statement

This reference architecture is well-suited for greenfield environments, small brownfield deployments in the Azure cloud that can be migrated to this “vHubs and direct vNET spokes” reference model and cases where routing and security functionalities are consolidated in vHubs.

vWAN is indeed a powerful option to build our private backbone leveraging Microsoft’s global network. However, if we deviate from “vHubs and direct vNET spokes” reference architecture, we may encounter certain limitations that need to be considered before deployment. It’s important to note that vWAN is a service under constant development, and certain limitations may be addressed in the future.

Topology Description

In this setup, I have Intermediate hubs with a pair of Firewalls. These are NVAs in a “sandwich firewall” topology where load balancers distribute traffic between both FWs in an Active/Active High Availability (HA) deployment. By default, vWAN is capable of automatically learning the IP addressing of directly connected vNETs through Hub virtual network connections. However, it is not able to learn the CIDR range of Indirect Spokes (not directly connected to a vHub), i.e., those behind the direct Spokes or Intermediate Hubs in our topology.

To address this IP range announcement in this architecture, Border Gateway Protocol (BGP) is implemented between vHubs and NVA Firewalls. The Firewalls will advertise, through redistribution into BGP, the networks of the Indirect Spokes to resolve the reachability issue.

Limitations encountered

1. Restriction to only use Default Route Table when BGP is enabled

“The virtual network connection that has the NVA BGP connection endpoint must always be associated and propagating to defaultRouteTable. Custom route tables are not supported at this time.”

vWAN supports the creation of multiple route tables (similar to VRFs). However, this functionality is not supported when BGP sessions are established with NVAs. These BGP sessions are always established using the default route table.

In my scenario, vNETs could belong to different environments that were isolated from each other. Different route tables would have made it much easier to implement this isolation at the routing level.

The alternative was to implement a BGP Community logic to filter between PROD and DEV environments through route-maps on the NVA Firewalls once BGP prefixes were received from the vHubs.

2. BGP route-maps are in Preview

“Route-maps is currently in Public Preview and is provided without a service-level agreement. It shouldn’t be used for production workloads. Certain features might not be supported, might have constrained capabilities, or might not be available in all Azure locations.”

Route-maps is a powerful feature that gives you the ability to control route advertisements and routing for vWAN vHubs. Route-maps lets you have more control of the routing that enters and leaves Azure vWAN S2S VPN connections, User VPN P2S connections, ExpressRoute connections and vNET connections. However, it is not recommended for Production environments at the time of writing this post.

This is why BGP route-maps for BGP Community filtering were applied on NVA Firewalls as shown in the diagram above.

3. Not possible to change the next-hop IP address for BGP advertisements from an NVA.

“The next hop IP address on the routes being advertised from the NVA to the vHub has to be the same as the IP address of the NVA, the IP address configured on the BGP peer. Having a different IP address advertised as next hop IS NOT supported on virtual WAN at the moment.”

In my scenario, the “Firewall Sandwich” model involves directing traffic to the load balancer facing the NVA Firewalls, allowing it to distribute traffic between instances in an Active-Active HA model. However, despite the NVA Firewalls being capable of modifying the next-hop of BGP advertisements exchanged with the vHub, the vHub was unable to understand them. This is because its BGP neighbor is not the load balancer but the Firewalls behind it. The vHub only installs prefixes in its route table whose next-hop matches the IP of its BGP neighbors.

The alternative to overcome this limitation is to configure static routes on each vHub and define the next-hop IP address to be the load balancer IP address to reach prefixes behind the NVA vNET (indirect Spokes).

4. Static routes must be configured in every vHub

“Static routes configured in a vHub are not automatically propagated to the other vHubs in vWAN.”

This is because static routes are not redistributed into BGP in vHubs, therefore they need to be configured in every vHub by the network admin.

5. BGP cannot overwrite certain system routes

“Traffic destined for addresses in the virtual network directly connected to the virtual hub cannot be configured to route through the NVA using BGP peering between the hub and NVA. This is because the virtual hub automatically learns about system routes associated with addresses in the spoke virtual network when the spoke virtual network connection is created. These automatically learned system routes are preferred over routes learned by the hub through BGP.”

In my topology, the direct vNET has a second CIDR used by the NAT device to perform NAT. This NAT device translates the IPs of the Indirect Spokes to this range. Therefore, traffic destined for the CIDR used for NAT must be redirected to the NAT device so can be routed to the final destination.

However, the BGP announcement from the NAT device does not overwrite the system route that is automatically installed due to the direct connectivity with the vNET.

Therefore, this is another example where static routes need to be configured to force the traffic to pass through the NAT device instead of being delivered to the direct vNET, where it would be lost as the destination device does not exist in this vNET.

6. One Virtual Network Gateway (VGW)

“A vNET could have only one Virtual Network Gateway (VGW), the gateway should be either local or remote gateway in the peered virtual network.”

In my use case, it was necessary to connect to Partners via an IPSec tunnel who needed access to very specific resources in certain vNETs, and it was not allowed for them to have connectivity to the rest of the environment.

Therefore, I did not want to connect them to the VGW of the vHubs to minimize risk and comply with security policies. Instead, the initial idea was to connect them to the NVA vNETs to inspect traffic through the NVA Firewalls and access allowed resources in the indirect Spokes.

However, this option is not supported. NVA vNETs (direct Spokes) connected to vWAN through vNET connection cannot have their own VGW.

Instead, I deployed the VGW in the indirect Spoke to support this Partner connectivity and comply with the security policy.

Conclusion

In the world of private WAN connectivity, Azure Virtual WAN emerges as a solid choice, tapping into third-party backbones for a streamlined network infrastructure. Through my exploration, I stumbled upon a few limitations, but understanding these challenges allowed navigate the complexities effectively.

Despite its evolving nature and occasional bumps in the road, vWAN comes packed with significant functionalities that make it a true asset! By embracing these insights, we not only bolster our networks against potential hiccups but also lay the foundation for a resilient and efficient connectivity solution.

May your networks stay robust, your connections seamless and your tech journey enjoyable!

References

About virtual hub routing – Azure Virtual WAN | Microsoft Learn

About BGP peering with a virtual hub – Azure Virtual WAN | Microsoft Learn