We have an express route setup with private peering so that our azure networks are now an hybrid extension of our on-prem networks. As all corporations do, we have a single point of exit for all external traffic from on-prem and we publish BGP routes on the express route so that all the internet traffic is routed via our external edge on-prem. This creates an interesting issue in azure networks for incoming traffic from internet.
For, example if you set up a DMZ in azure so that you can host external facing web servers in azure.
The web servers will not be able to communicate with the load balancer. This is because when the traffic hits the web servers, it comes from the load balancer with a public IP address (aka internet). Since, you are forcing all the internet traffic to go out via the external edge on-prem, the reply from the web server is routed via on-prem which causes the asymmetric routing and the traffic between the load balancer and web servers in never successful.
To solve this issue, you can simply route the traffic back to the load balancer by adding a routing table and allowing the web servers to talk directly to the internet.
This will override the BGP route and allow traffic to go directly back to the load balancer.
This is a simple example to demonstrate how the private peering and on-prem edge can cause issues. I will be writing a separate post on how to set up a DMZ with all bells and whistles soon.