I’ve been meaning to copy this back here but haven’t had the chance until now. I reference this so much, figured it’d have stuck in my mind by now… Anyway, this is one of the best resources for quick analysis troubleshooting of MM_WAIT_MSG errors on VPN tunnels for Cisco ASA / PIX from https://www.tunnelsup.com/isakmp-ike-phase-1-status-messages/.
ISAKMP (IKE Phase 1) Negotiations States
The MM_WAIT_MSG state can be an excellent clue into why a tunnel is not forming. If your firewall is hanging at a specific state review this graph below to find where along the path the VPN is failing.
ASA ISAKMP STATES
These are the possible ISAKMP negotiation states on an ASA firewall. ISAKMP stands for: The Internet Security Association and Key Management Protocol
- MM_WAIT_MSG2 Initiator Initial DH public key sent to responder. Awaiting initial contact reply from other side. Initiator sends encr/hash/dh ike policy details to create initial contact. Initiator will wait at MM_WAIT_MSG2 until it hears back from its peer. If stuck here it usually means the other end is not responding. This could be due to no route to the far end or the far end does not have ISAKMP enabled on the outside or the far end is down.
- MM_WAIT_MSG3 Receiver Receiver is sending back its IKE policy to the initiator. Initiator sends encr/hash/dh ike policy details to create initial contact. Initiator will wait at MM_WAIT_MSG2 until it hears back from its peer. Hang ups here may also be due to mismatch device vendors, a router with a firewall in the way, or even ASA version mismatches.
- MM_WAIT_MSG4 Initiator Initiator is sending the Pre-Shared-Key hash to its peer. Initiator sends a hash of its PSK. Initiator will stay at MSG4 until it gets a PSK back from its peer. If the receiver is missing a tunnel group or PSK the initiator will stay at MM_WAIT_MSG4
- MM_WAIT_MSG5 Receiver Receiver is sending its PSK hash to its peer. Receiver does not yet check if PSK hashes match. If receiver has a tunnel-group and PSK configured for this peer it will send the PSK hash to the peer. If PSKs dont match, receiver will stay at MM_WAIT_MSG5. I have also seen the tunnel stop here when NAT-T was on when it needed to be turned off.
- MM_WAIT_MSG6 Initiator Initiator checks if PSK hashes match. If PSK keys match, Initiator becomes MM_ACTIVE and lets receiver know of match. If PSK doesnt match, initiator stays at MM_WAIT_MSG6. I have also seen the tunnel stop here when NAT-T was on when it needed to be turned off. However, if the state goes to MSG6 then the ISAKMP gets reset that means phase 1 finished but phase 2 failed. Check that IPSEC settings match in phase 2 to get the tunnel to stay at MM_ACTIVE.
- AM_ACTIVE / MM_ACTIVE The ISAKMP negotiations are complete. Phase 1 has successfully completed.
PIX ISAKMP STATES
ISAKMP SA has been created but nothing else has happened yet.
The peers have agreed on parameters for the ISAKMP SA.
The peers have exchanged Diffie-Hellman public keys and have generated a shared secret. The I SAKMP SA remains unauthenticated.
The ISAKMP SA has been authenticated. If the router initiated this exchange, this state trans itions immediately to QM_IDLE and a Quick mode exchange begins.
The ISAKMP SA has been created but nothing else has happened yet.
The peers have done the first exchange in Aggressive mode but the SA is not authenticated.
The ISAKMP SA has been authenticated. If the router initiated this exchange, this state transitions immediately to QM_IDLE and a Quick mode exchange begins.
The ISAKMP negotiations are complete. Phase 1 successfully completed. It remains authenticated with its peer and may be used for subsequent Quick mode exchanges.
What is the difference between MM and AM?
Main mode vs Aggressive mode. Here is a image taken from Cisco’s website to show the difference.
As you can see the Main mode is the same as the flowchart at the top of the page. Aggressive mode only uses 4 steps to establish the tunnel.
Troubleshooting ISAKMP Or Phase 1 VPN connections
When troubleshooting VPNs, a very common problem is phase 1 not establishing correctly. Here’s a quick checksheet to make sure you have the configuration correct.
- Verify ISAKMP parameters match exactly.
- Verify pre-shared-keys match exactly.
- Check that each side has a route to the peer address that you are trying to form a tunnel with.
- Verify ISAKMP is enabled on the outside interfaces.
- Is ESP traffic permitted in through the outside interface?
- Is UDP port 500 open on the outside ACL?
- Some situations require that UDP port 4500 is open for the outside.
- Run the Local Group Policy Editor (Start > type gpedit.msc)
- Navigate to Computer Configuration > Administrative Templates > System
- Select Logon
- Double-click Show first sign-in animation
- In the Show first sign-in animation window, select Disabled and click OK
- Close the Local Group Policy Editor
Some notes on increasing LVM partition in Linux.
- Physical Volume (PV): This can be created on a whole physical disk (think /dev/sda) or a Linux partition.
- Volume Group (VG): This is made up of at least one or more physical volumes.
- Logical Volume (LV): This is sometimes referred to as the partition, it sits within a volume group and has a file system written to it.
- File System: A file system such as ext4 will be on the logical volume.
Increase or Expand Logical Volume
To increase/expand a logical volume (lv from here onward), it can be done without needing to reboot or experiencing any downtime on the system.
My volume group (vg here onward) is debian-vg; it contains all my lv’s.
root@debian:~# vgdisplay --- Volume group --- VG Name debian-vg System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 8 VG Access read/write VG Status resizable MAX LV 0 Cur LV 5 Open LV 5 Max PV 0 Cur PV 1 Act PV 1 VG Size 39.76 GiB PE Size 4.00 MiB Total PE 10178 Alloc PE / Size 7151 / 27.93 GiB Free PE / Size 3027 / 11.82 GiB VG UUID QPsbEO-d7Q4-OlbR-9BQL-4C1k-04oq-R8QcG6
As you can see above, the Free PE / Size indicates how much available to use to increase/expand a lv I have.
To look at the logical volumes, I use
--- Logical volume --- LV Path /dev/debian-vg/home LV Name home VG Name debian-vg LV UUID 61YQXT-wTDM-Fb66-1Fy0-U9dK-tHcn-Kzf1M8 LV Write Access read/write LV Creation host, time debian, 2018-06-11 10:03:17 -0400 LV Status available # open 1 LV Size 10.00 GiB Current LE 2560 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:4
My home logical volume is currently 10GB in size, indicated by LV Size above.
If I want to expand this to 12GB, I would issue the following:
root@debian:~# lvextend -L+2G /dev/debian-vg/home Size of logical volume debian-vg/home changed from 10.00 GiB (2560 extents) to 12.00 GiB (3072 extents). Logical volume debian-vg/home successfully resized.
lvdisplay output again, I see that it is now 12GB, but I need to expand the filesystem now.
--- Logical volume --- LV Path /dev/debian-vg/home LV Name home VG Name debian-vg LV UUID 61YQXT-wTDM-Fb66-1Fy0-U9dK-tHcn-Kzf1M8 LV Write Access read/write LV Creation host, time debian, 2018-06-11 10:03:17 -0400 LV Status available # open 1 LV Size 12.00 GiB Current LE 3072 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:4
This partition is ext4, so I will use
resize2fs as below:
root@debian:~# resize2fs /dev/debian-vg/home resize2fs 1.43.4 (31-Jan-2017) Filesystem at /dev/debian-vg/home is mounted on /home; on-line resizing required old_desc_blocks = 2, new_desc_blocks = 2 The filesystem on /dev/debian-vg/home is now 3145728 (4k) blocks long.
Note: If using xfs, use
xfs_growfs in lieu of
That should do it, now I can issue
df -h and confirm that my /home partition is now 12GB.
root@debian:~# df -h Filesystem Size Used Avail Use% Mounted on udev 991M 0 991M 0% /dev tmpfs 201M 24M 177M 12% /run /dev/mapper/debian--vg-root 7.4G 2.3G 4.7G 33% / tmpfs 1003M 0 1003M 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 1003M 0 1003M 0% /sys/fs/cgroup /dev/mapper/debian--vg-tmp 544M 924K 503M 1% /tmp /dev/sda1 236M 37M 187M 17% /boot /dev/mapper/debian--vg-var 7.7G 2.5G 4.9G 34% /var tmpfs 201M 0 201M 0% /run/user/1000 /dev/mapper/debian--vg-home 12G 41M 12G 1% /home