DAG (Distributed Anycast Gateway)

Distributed anycast gateway feature for EVPN VXLAN is a default gateway addressing mechanism that enables the use of the same gateway IP addresses across all the leaf switches that are part of a VXLAN network.

Warning

The same subnet mask and IP address must be configured on all the switch virtual interfaces (SVIs) that act as a distributed anycast gateway (DAG).

Inputs

Inventory.yml

In the inventory file, roles (Spine or Leaf), names, and management IP addresses of the nodes are described.

all:
    children:
        leaf:
            hosts:
                Leaf-01:
                    ansible_host: 10.62.149.179
                Leaf-02:
                    ansible_host: 10.62.149.182

        spine:
            hosts:
                Spine-01:
                    ansible_host: 10.62.149.180
                Spine-02:
                    ansible_host: 10.62.149.181

leaf and spine are two roles. Each node should be placed under one of these roles.

Leaf-1 , Spine-01 are the hostnames (nodes). Keep in mind that the names should be with the name of the configuration files in the directory host_vars.

ansible_host is the ip address of the management interface.

group_vars

This directory contains the configurations which are common to all or most of devices.

all.yml

The parameters defined in the file all.yml are applicable to all devices in the network.

General access

This section defines access parameters of the remote devices.

ansible_connection: ansible.netcommon.network_cli
ansible_network_os: cisco.ios.ios
ansible_python_interpreter: "python"
ansible_user: cisco
ansible_ssh_pass: cisco123

<...skip...>

Parameter	Comments
ansible_connection	This option defines thetype for connection to the remote devices. In this project, connection via SSH with implementation of CLI is used: ansible.netcommon.network_cli
ansible_network_os	This option defines the operation system of the remote device. This option is needed if “network_cli” is used for ‘ansible_connection’. In this project, Cat9k with IOS-XE is used, so this option is set to: cisco.ios.ios
ansible_python_interpreter	This option instruct Ansible to use defined python interpreter. This option is set to: python
ansible_user	This option defines a username which is used for access remote devices over SSH. In this project, user must have privilege level 15. This option is set to: cisco
ansible_password	This option defines a password for the user in ‘ansible_user’. In this project, the password is set to: cisco123

Warning

ansible_user must have privildge level 15. Example of the configuration is below

username cisco privilege 15 password 0 cisco123

In this example, unencrypted password is used. Feel free to use HIDDEN (7)

If enable password should be used, check the Enable Mode documentation.

overlay_db.yml

In this file information about EVPN configuration is stored. Let’s check this file gradually step-by-step.

L2VPN EVPN general definition

This section defines global L2VPN EVPN parameters.

l2vpn_global:
    replication_type: 'static'
    router_id: 'Loopback1'
    default_gw: 'yes'

<...skip...>

Parameter	Comments
l2vpn_global / mandatory	This option defines L2VPN EVPN globally.
replication_type / optional	This option defines the type of repliction for the L2 BUM traffic globally. Could be overwritten per vlan by “vlans” -> “vlan_id” -> “replication_type” section. Option static enables to use multicast for the BUM replication. Option ingress enables to use Ingress-replication (unicast) for BUM replication. Choices: static ingress
router_id / optional	This option defines the interface whose IP address will be used for defining router-id of L2VPN. In this project, the interface Loopback1 is used for the router-id of L2VPN, so the option is set to: Loopback1
default_gw / optional	This option defines if Default GW will be advertised or not. In this project it is defined by defualt: default_gw: ‘yes’

VRF definition

This section defines vrf parameters. Lets review parameters for unicast first.

vrfs:
    green:
        rd: '1:1'
            afs:
                ipv4:
                    rt_import:
                        - '1:1'
                        - '1:1 stitching'
                    rt_export:
                        - '1:1'
                        - '1:1 stitching'
                ipv6:
                    rt_import:
                        - '1:1'
                        - '1:1 stitching'
                    rt_export:
                        - '1:1'
                        - '1:1 stitching'
<...skip...>

Parameter	Comments
vrfs / mandatory	This option defines vrf section globally.
<vrf_name> / mandatory	This option defines the vrf name.
rd / mandatory	This option defines the route distinguisher of the vrf.
afs / mandatory	This option defines the address families which will be activated for the vrf. Option ipv4 defines ipv4 address family. Option ipv6 defines ipv6 address family. Choices: ipv4 ipv6
rt_import / mandatory	This option defines the Route Target Import per VRF/AF. This option allows more than one RT to be defined. For EVPN AF additional key is used - “stitching”. In this project next parameter are set by default for both AFs(IPv4 and IPv6): 1:1 1:1 stitching (L2VPN EVPN AF)
rt_export / mandatory	This option defines the Route Target Export per VRF/AF. This option allows more than one RT to be defined. For EVPN AF additional key is used - “stitching”. In this project below parameters are set by default for both AFs(IPv4 and IPv6): 1:1 1:1 stitching (L2VPN EVPN AF)

VLANs section

This section defines VLANs and it stitching with EVIs(EVPN instance) and VNIs(VXLAN network identifier).

vlans:

 101:
  vlan_type: 'access'
  description: 'Access_VLAN_101'
  vni: '10101'
  evi: '101'
  type: 'vlan-based'
  encapsulation: 'vxlan'
  replication_type: 'static'
  replication_mcast: '225.0.0.101'

 102:
  vlan_type: 'access'
  description: 'Access_VLAN_102'
  vni: '10102'
  evi: '102'
  type: 'vlan-based'
  encapsulation: 'vxlan'
  replication_type: 'ingress'

 901:
  vlan_type: 'core'
  description: 'Core_VLAN_VRF_green'
  vni: '50901'
  vrf: 'green'

<...snip...>

Parameter	Comments
vlans / mandatory	This option defines vlan section globally.
<vlan_id> / mandatory	This option defines VLAN ID on the switch. In this example there are 101, 102, 901.
vlan_type / mandatory	This option defines type of the VLAN. Option access is used for L2VNIs. Option core is used for L3VNIs. Option non-vxlan is used for VLANs, which are not extended over Fabric. Choices access core non-vxlan
description / optional	This option defines VLAN description.
vni / mandatory	This option defines the VNI which is stitched with a VLAN ID on the swith.
evi / mandatory	This option defines the EVI which is stitched with a VLAN ID on the swith. This parameter is mandatory for L2VNIs only.
type / mandatory	This option defines the type of EVI. On Cat9k vlan-based is supported for now. This parameter is mandatory for L2VNIs only.
encapsulation / mandatory	This option defines encapsulation for packet is the core. It is set to vxlan. This parameter is mandatory for L2VNIs only.
replication_type / mandatory	This option defines replication type for the BUM for L2VNI. Option static is used for multicast replication. In this case replication_mcast parameter is needed. Option ingress is used for Ingress-replication (unicast). Choices: static ingress This parameter is mandatory for L2VNIs only.
vrf / mandatory	This option defines VRF for which L3VNI is used for encapsulation the routed traffic in the core. For this option vlan_type must be core. This parameter is mandatory for L3VNIs only.

SVIs section

This section defines SVIs configuration.

svis:

 101:
  svi_type: 'access'
  vrf: 'green'
  ipv4: '10.1.101.1 255.255.255.0'
  ipv6:
    - '2001:101::1/64'
  mac: 'dead.beef.abcd'

 102:
  svi_type: 'access'
  vrf: 'green'
  ipv4: '10.1.102.1 255.255.255.0'
  ipv6:
    - '2001:102::1/64'
  mac: 'dead.beef.abcd'

 901:
  svi_type: 'core'
  vrf: 'green'
  src_intf: 'Loopback1'
  ipv6_enable: 'yes


 <...snip...>

Parameter	Comments
svis / mandatory	This option defines SVIs section globally.
<svi_id> / mandatory	This option defines SVI ID on the switch. In this example there are 101, 102, 901.
svi_type / mandatory	This option defines type of the SVI. Option access is used for SVI for vlans stitched to L2VNIs. Option core is used for SVI for vlans stitched to L3VNIs. Option non-vxlan is used for SVI for vlans, which are not extended over Fabric. Choices access core non-vxlan
vrf / mandatory	This option defines vrf which SVI belongs to.
ipv4 / mandatory	This option defines the IPv4 address configured on the SVI. This parameter is applicable for SVIs for L2VNIs only.
ipv6 / optional	This option defines the IPv6 addresses configured on the SVI. This parameter is applicable for SVIs for L2VNIs only.
mac / optional	This option defines the MAC to be configured on SVI. This parameter is applicable for SVIs for L2VNIs only.
src_intf / mandatory	This option defines Source Interface for the SVI for L3VNI. This parameter is applicable for SVIs for L3VNIs only.
ipv6_enable / optional	This option defines enables IPv6 on the SVI. This parameter is applicable for SVIs for L3VNIs only.

NVE section

This section defines NVE interface configuration.

nve_interfaces:
    1:
        source_interface: 'Loopback1'

<...snip...>

Parameter	Comments
nve_interfaces / mandatory	This option defines NVE section globally.
nve_id> / mandatory	This option defines NVE ID on the switch.
source_interface / mandatory	This option defines source interface for corresponding NVE interface.

host_vars

In this directory stored specific to the dedicated device configuration.

<node_name>.yml

In the file <node_name>.yml defined specific to the dedicated node configuration parameters. Usually it is related to interface configuration and underlay configuration in general.

Lets review the configuration options one by one.

Hostname section

In this section hostname of the node is defined.

hostname: 'Leaf-01'

<...snip...>

Parameter	Comments
hostname / optional	This option defines remote device hostname.

Global routing section

In this section parameters of IPv4/IPv6 in GRT are defined.

ç

Parameter	Comments
routing / mandatory	This option defines global routing section.
ipv4_uni / mandatory	This option enables global IPv4 unicast routing on the switch.
ipv6_uni / mandatory	This option enables global IPv6 unicast routing on the switch.
ipv6_multi / mandatory	This option enables global IPv4 multicast routing on the swith.

Interface section

In this section interfaces configuration is defined.

interfaces:

    Loopback0:
        name: 'Routing Loopback'
        ip_address: '172.16.255.3'
        subnet_mask: '255.255.255.255'
        loopback: 'yes'
        pim_enable: 'no'

    Loopback1:
        name: 'NVE Loopback'
        ip_address: '172.16.254.3'
        subnet_mask: '255.255.255.255'
        loopback: 'yes'
        pim_enable: 'yes'

    GigabitEthernet1/0/1:
        name: 'Backbone interface to Spine-01'
        ip_address: '172.16.13.3'
        subnet_mask: '255.255.255.0'
        loopback: 'no'
        pim_enable: 'yes'

    GigabitEthernet1/0/2:
        name: 'Backbone interface to Spine-02'
        ip_address: '172.16.23.3'
        subnet_mask: '255.255.255.0'
        loopback: 'no'
        pim_enable: 'yes'

<...snip...>

Parameter	Comments
interfaces / mandatory	This option defines global interface section.
<interface_name> / mandatory	This option defines interface name i.e. Loopback0 or GigabitEthernet1/0/1
name / optional	This option defines interface description.
ip_address / mandatory	This option defines IPv4 address on the interface.
subnet_mask / mandatory	This option defines subnet mask for the IPv4 address.
loopback / mandatory	This option defines if interface is loopback or not. Choices: yes no
pim_enable / mandatory	This option defines if PIM has to be enabled on the interface. Choices: yes no

OSPF section

This section defines ospf parameters.

By default next OSPF configuration is applied:

• Interface network type - point-to-point

• OSPF process ID - 1

• OSPF area number - 0

OSPF router-id is configurable parameter.

ospf:
    router_id: '172.16.255.3'

<...snip...>

Parameter	Comments
ospf / mandatory	This option defines OSPF section globally.
router_id / mandatory	This option defines OSPF router-id.

PIM section

This section defines global PIM parameters. This section is optional if Ingress-Replication in the core is used.

pim:
    rp_address: '172.16.255.255'

<...skip...>

Parameter	Comments
pim / mandatory	This option defines PIM section globally.
rp_address / mandatory	This option defines RP address.

MSDP section

This section defines MSDP parameters. Usually MSDP is used for configuration RP redundancy in underlay.

This section in general is optional.

msdp:
    '1':
        peer_ip: '172.16.254.2'
        source_interface: 'Loopback1'
        remote_as: '65001'

<...skip...>

Parameter	Comments
msdp / mandatory	This option defines MSDP section globally.
<msdp_neighbor_id> / mandatory	This option defines ID for the MSDP peer. This number is not used in the switch configuration, just index number.
peer_ip / :red: mandatory	This option defines MSDP peer IPv4 address.
source_interface / :red: mandatory	This option defindes source interface which IP address will be used like SRC IP for the MSDP seession.
remote_as / :red: mandatory	This option is used for defining BGP AS number of the MSDP peer.

BGP section

This section defines BGP parameters.

By default next design assumption are made:

• Leafs are Route-Reflector clients

• Two present Spines in the topology are Route-Reflectors

bgp:
  as_number: '65001'
  router_id: 'Loopback0'
  neighbors:
    '172.16.255.1':
        peer_as_number: '65001'
        source_interface: 'Loopback0'

    '172.16.255.2':
        peer_as_number: '65001'
        source_interface: 'Loopback0'

    '172.16.255.3':
        peer_as_number: '65001'
        source_interface: 'Loopback0'
        rrc: 'yes'

<...snip...>

Parameter	Comments
bgp / mandatory	This option defines BGP section globally.
as_number / mandatory	This option defines BGP AS number.
router_id / mandatory	This option defines interface which ip address will be used like BGP router ID.
neighbors / mandatory	This option defines neighbors section.
neigbor_ip_address / mandatory	This option defines BGP neighbor ip address
peer_as_number / mandatory	This option defines BGP neighbor AS number
source_interface / mandatory	This option defines source interface which ip address will be used like a SRC IP for BGP session.
rrc / optional	This option defines the peer like a BGP route-reflector client.

Access interface configuration

This section defines configuration for the customer-facing access interfaces.

By default all access interfaces will be configured like trunks with all L2VNI vlans that are mentioned in group_vars/overlay_db.yml

Trunk configuration

Vlans to be assigned to an interace are taken from the following in increasing order of priority (3 > 2 > 1).

Note

Trunk configuration order of priority (3 > 2 > 1)

1. vlans in group_vars/overlay_db.yml (for playbook_access_add_commit/preview.yml) or access_intf_cli in host_vars/inc_vars/<hostname>.yml

(for playbook_access_incremental_commit/preview.yml)

access_interfaces:
    trunks:
        - GigabitEthernet1/0/6

<...snip...>

2. trunk_vlan_list in access_interfaces dictionary

access_interfaces:
    trunk_vlan_list: 101,102,201
    trunks:
        - GigabitEthernet1/0/6

<...snip...>

3. trunk_vlan_list in specific interface dictionary

access_interfaces:
    trunks:
        - GigabitEthernet1/0/6:
            trunk_vlan_list: 101,102

<...snip...>

Access configuration

Vlan to be assigned to an interace are taken from the following in increasing order of priority (2 > 1).

Note

Access configuration order of priority (2 > 1)

1. access_vlan in access_interfaces dictionary

access_interfaces:
    access_vlan: 101
    access:
        - GigabitEthernet1/0/6

<...snip...>

2. access_vlan in specific interface dictionary

access_interfaces:
    access:
        - GigabitEthernet1/0/6:
            access_vlan: 102

<...snip...>

Examples

There is an assumption, that in group_vars/overlay_db.yml defined next vlans: 101,102,201,202

Example 1

Content of host_vars/access_intf/<hostname>.yml

access_interfaces:
    trunks:
        - GigabitEthernet1/0/7
        - GigabitEthernet1/0/8

Vlans assigned after execution:

GigabitEthernet1/0/7 - 101,102,201,202 (from group_vars/overlay_db.yml or host_vars/inc_vars/<hostname>.yml)

GigabitEthernet1/0/8 - 101,102,201,202 (from group_vars/overlay_db.yml or host_vars/inc_vars/<hostname>.yml)

Example 2

Content of host_vars/access_intf/<hostname>.yml

access_interfaces:
    access_vlan: 202
    access:
        - GigabitEthernet1/0/7
        - GigabitEthernet1/0/8

Vlans assigned after execution:

GigabitEthernet1/0/7 - 202

GigabitEthernet1/0/8 - 202

Example 3

Content of host_vars/access_intf/<hostname>.yml

access_interfaces:
    trunks:
        - GigabitEthernet1/0/6
        - GigabitEthernet1/0/7:
            trunk_vlan_list: 101,102,201
    access:
        - GigabitEthernet1/0/8
        - GigabitEthernet1/0/9
    access_vlan: 202

Vlans assigned after execution:

GigabitEthernet1/0/6 - 101,102,201,202 (from all.yml or host_vars/inc_vars/<hostname>.yml)

GigabitEthernet1/0/7 - 101,102,201

GigabitEthernet1/0/8 - 202

GigabitEthernet1/0/9 - 202

Example 4

Content of host_vars/access_intf/<hostname>.yml

access_interfaces:
    trunks:
        - GigabitEthernet1/0/6
        - GigabitEthernet1/0/7:
            trunk_vlan_list: 101,102,201
    trunk_vlan_list: 101,201
    access:
        - GigabitEthernet1/0/8
        - GigabitEthernet1/0/9:
            access_vlan: 102
    access_vlan: 202

Vlans assigned after execution:

GigabitEthernet1/0/6 - 101,201

GigabitEthernet1/0/7 - 101,102,201

GigabitEthernet1/0/8 - 202

GigabitEthernet1/0/9 - 102

Example 5

Content of host_vars/access_intf/<hostname>.yml

access_interfaces:
    trunks:
        - GigabitEthernet1/0/5
        - GigabitEthernet1/0/6:
            trunk_vlan_list: 101,102,201
        - GigabitEthernet1/0/7
    access:
        - GigabitEthernet1/0/8:
            access_vlan: 201
        - GigabitEthernet1/0/9:
            access_vlan: 102
    access_vlan: 202

Vlans assigned after execution:

GigabitEthernet1/0/5 - 101,102,201,202 (from group_vars/overlay_db.yml or host_vars/inc_vars/<hostname>.yml)

GigabitEthernet1/0/6 - 101,102,201

GigabitEthernet1/0/7 - 101,102,201,202 (from group_vars/overlay_db.yml or host_vars/inc_vars/<hostname>.yml)

GigabitEthernet1/0/8 - 201

GigabitEthernet1/0/9 - 102

Example 6

Content of host_vars/access_intf/<hostname>.yml

access_interfaces:
    trunks:
        - GigabitEthernet1/0/7
access:
    - GigabitEthernet1/0/8:
        access_vlan: 201

Vlans assigned after execution:

GigabitEthernet1/0/7 - 101,102,201,202 (from group_vars/overlay_db.yml or host_vars/inc_vars/<hostname>.yml)

GigabitEthernet1/0/8 - 201