[Technical] - ACS 'Server Secret Mismatch' Error.

I saw an issue where the following error was being displayed even when the shared key matched. (Or seemed to..)

"Authentication Server not responding: AAA decode failure.. server secret mismatch"

This was seen on an Cisco ASA firewall running 8.0.4 code. The ACS log would either show "Key Mismatch" or "CS password invalid" (using Radius)

The shared key was the same as that in the Network Configuration of ACS, however "Network Device Groups" were being used and the default key for the network device group was different to the specific device key and took priority.


Use the network device group (NDG) key or don't set a NDG key.


Bob

[Technical] - Error Message 'External DB is not operational' seen on Cisco ACS appliance. Wireless users fail to authenticate.

The following is a bit of advice on how to troubleshoot the error message 'External DB is not operational' or the error "Connection to Windows authentication agent established; Failed to connect to any Windows authentication agent." on a CiscoSecure ACS appliance.

This will most likely occur when authentication is being backed off to a Windows Active Directory from a CiscoSecure ACS appliance. The authentication agent software is a small application that's ideally loaded onto a Domain Controller that accepts authentication requests from the ACS and responds after querying the AD.

This issue above will most likely occur where the ACS appliance can't communicate with the agent because;

- The domain controller hosting the agent software is down.
- The agent service "csagent" has stopped responding.
- A firewall is blocking traffic between the ACS and Domain Controller.
- The IP address of the domain controller hosting the agent has changed.
- The software version of ACS and the agent need to match.
- The certificate has expired on the ACS appliance or is now invalid for the domain.



The above error message will likely be seen in the ACS "failed attempts" log file.

To troubleshoot this issue do the following;

1 - Check connectivity from the ACS appliance to the domain controller.
2 - Check that the csagent service is running on the domain controller.
3 - Stop the csagent service and restart it from DOS in debug mode.

To use debug mode open a command prompt and go to the csagent directory. Then start the agent with the -z and -p options.

C:\Program Files\Cisco\CiscoSecure ACS Agent\bin>csagent -z -p

Two windows will open, a successfully communication will look as follows:

C:\Program Files\Cisco\CiscoSecure ACS Agent\bin>csagent -z -p
Running ACSRemoteAgent server from command line..
Debug printing on..
ACSRemoteAgent server starting ==============================
Running as console application.
Will listen on port 2004
Configuration will be fetched from 192.x.x.103:2003
Agents: CSWinAgent
CSWinAgent File: ..\bin\CSWinAgent.exe
CSWinAgent Port: 2005
1 agents configured
Permitted CSAgent Clients: *.*.*.*
Hit Return/Enter to stop...

Listener activated
Watchdog activated
CSWinAgent launched
Client connecting from 192.x.x.103:1159
RPC: Info request received
RPC: Info reply sent
Client disconnected, thread 2628 terminating

--------- In the second window ----------

CSWinAgent server starting ==============================
Running as console application.
Will listen on port 2005
Permitted CSWinAgent Clients: *.*.*.*
NTLIB: Library behaviour mode 2
NTLIB: Initialising locally
NTLIB: The local computer name is MYDC12
NTLIB: The 'insist on domain' feature is enabled
NTLIB: We ARE a domain controller
NTLIB: We are a backup domain controller for domain MY.COM
NTLIB: Found 0 trusted domains
Listener activated
Client connecting from 192.x.x.103:1160
RPC: NT_MSCHAPAuthenticateUser received
NTLIB: Attempting Windows authentication for user JIM.SMITH$
NTLIB: Windows authentication SUCCESSFUL (by MYDC12)
RPC: NT_MSCHAPAuthenticateUser reply sent

Please feedback any other experiences.

Some useful URLs are;

• Remote Agent Installation Guide
• Another Similar Example

Bob