Hi, today I'm going to write something about a design question that often pops up when creating an enterprise-grade network design for your Azure connectivity needs.
"When you have an ExpressRoute circuit and need connectivity between multiple VNets, should I go for the VNet "hub and spoke" model with vnet peering, or should I create an ExpressRoute circuit authorization to connect my vnets directly?"
As any good consultant the default (and very cliché) answer would be "it depends"
To make a well-founded decision, let's first dive a bit deeper into our options:
Hub & Spoke with VNet peering
The design principal here is simple: one VNet serves as a hub. This hub will be connected to the ExpressRoute using an ExpressRoute gateway
All other VNets are linked to this hub using vnet peering (read: software defined magic)
A very important part here is that VNet peering is nontransitive, meaning that if:
VNet A peered with VNet B
and
VNet B peered with VNet C
Result: A has no route to C
There are a couple of ways to make VNet A and VNet C connect, like peering them as well.
Doing so however would transform our hub and spoke topology to a mesh, which is not always as desired.
Another option would be to make use of a NGFW (Next Generation Firewall) in the form of a third party appliance or Azure firewall.
Native transitive network flow between peered VNets could possible come in the future.
For more information on how to set up your hub and spoke topology, check out my previous post.
ExpressRoute Circuit Authorization
For this connectivity option, you create an authorization key that you use to connect a VNet to your ExpressRoute circuit.
This can be relevant when your circuit is based in another subscription.
You can easily create an authorization key on the Azure portal and then connect your ExpressRoute gateway to it.
Something that often gets people confused:
This ExpressRoute gateway that you mention, when do I need it?
You will need an ExpressRoute gateway in your VNet - placed in the "gatewaysubnet" subnet - for each VNet you want to directly connect to your circuit.
There is no need for the spoke VNets to have a gateway deployed, they will get their routes and connectivity through their peering with the hub VNet.
So when to choose what connectivity method?
There are 3 main questions that you could ask yourself when creating the design:
What are my security requirements?
If you link all VNets directly to the ExpressRoute circuit and consider your ExpressRoute to be the conceptual hub, then keep in mind that there is no additional security layer in your "hub"
More than often it could make sense to have your traffic flow through a NGFW or NGFW-like component to have security centralized and have a clearer network segmentation.
When connecting the VNets directly, each VNet is responsible for their own security, which, depending on how ownership of the VNets is distributed, is not always an easy thing to achieve.
From a cost perspective, having a firewall in both VNet1, VNet2 and VNet3 to reach your company requirements is also not the most cost effective way of handling things.
What are my network requirements?
As mentioned before, VNet peering is nontransitive.
There are also some limits to what you can do with user defined routes in regards to using the VNet peering as next hop.
So before creating your design, make sure you have a clear picture of what segments need to connect to each other. Managing your routing is easier if you connect each VNet directly to the ExpressRoute circuit & let BGP do its thing without the hassle of user defined routes. Be wary though that this comes with the security tradeoffs.
To keep things like latency or jitter under control it's recommended to bundle all components of an application as closely together as possible from a network perspective. In this case, VNet peering can be considered "better" as this is purely software defined.
Who is going to be responsible for all of this?
In larger environments, it's very common to have multiple parties and vendors involved, each responsible for their own components.
For me, the weight of the answer to this question weights the least: Ownership can change hands fast, but it's still something to take into account.
Ownership often also reflects the business applications, so they can help you to create a design that also makes sense technically.
To give a real life example: Contoso is responsible for the entire SAP stack, consisting of a lot of IaaS components and some PaaS.
They have their own separate subscription linked to their CSP contract.
When you have outbound traffic from the SAP environment, how is it secured?
Does this part of the environment need any additional NGFW component, or is this part of their service?
It could make sense to link the Contoso SAP VNets to the hub VNet that is managed by Fabricam which already has a beefy NGFW already in place with ExpressRoute connectivity up & running.
From a technical and cost perspective this could make most sense, but then the SLAs between all parties need to be crystal clear.
What are technical limitations?
All limitations are clearly defined on the "Azure subscription and service limits, quotas, and constraints" page:
Limits are increasing very fast and at the time of writing this article (September 2020) you shouldn't hit any of them soon in most environments.
One to be very wary about though is this one:
"Number of virtual networks per ExpressRoute circuit".
In the standard ExpressRoute SKUs, your limit is always 10.
If you would completely skip out on the hub & spoke model with VNet peering, you could hit that limit very fast, therefor making the hub & spoke model your primary choice for an enterprise-grade cloud network design.