My work computer recently crashed with a BSOD, when disconnecting or reconnecting Cisco AnyConnect Secure Mobility Client.
Considering the circumstances I suspected that Cisco AnyConnect was the culprit, but I wanted to confirm this.
I started looking for information in Event Viewer and found:
Log Name: System Source: Microsoft-Windows-WER-SystemErrorReporting Event ID: 1001 Task Category: None Level: Error Keywords: Classic Description: The computer has rebooted from a bugcheck. The bugcheck was: 0x000000c5 (0x00000000760e0002, 0x0000000000000002, 0x0000000000000000, 0xfffff802fba61850). A dump was saved in: C:\WINDOWS\MEMORY.DMP.
Needed to examine the memory dump, so started WinDbg (x64) and opened:
C:\Windows\Memory.dmp
This message was displayed:
BugCheck C5, {760e0002, 2, 0, fffff802fba61850}
*** ERROR: Module load completed but symbols could not be loaded for acsock64.sys
Probably caused by : memory_corruption
Checked for more details with:
!analyze -v
Part of the result:
DRIVER_CORRUPTED_EXPOOL (c5) An attempt was made to access a pageable (or completely invalid) address at an interrupt request level (IRQL) that is too high. This is caused by drivers that have corrupted the system pool. Run the driver verifier against any new (or suspect) drivers, and if that doesn't turn up the culprit, then use gflags to enable special pool. Arguments: Arg1: 00000000760e0002, memory referenced Arg2: 0000000000000002, IRQL Arg3: 0000000000000000, value 0 = read operation, 1 = write operation Arg4: fffff802fba61850, address which referenced memory
The crash definitely seemed to be caused by a driver bug.
Now I wanted to identify which driver caused the problem.
Checked the call stack with:
kc
Result:
# Call Site 00 nt!KeBugCheckEx 01 nt!KiBugCheckDispatch 02 nt!KiPageFault 03 nt!ExDeferredFreePool 04 nt!ExFreePoolWithTag 05 acsock64 06 acsock64 07 acsock64 08 NETIO!ProcessCallout 09 NETIO!ProcessFastCalloutClassify 0a NETIO!KfdClassify 0b tcpip!AleNotifyEndpointTeardown 0c tcpip!UdpCleanupEndpointWorkQueueRoutine 0d tcpip!UdpCloseEndpoint 0e afd!AfdTLCloseEndpoint 0f afd!AfdCloseTransportEndpoint 10 afd!AfdCleanupCore 11 afd!AfdDispatch 12 nt!IopCloseFile 13 nt!ObCloseHandleTableEntry 14 nt!NtClose 15 nt!KiSystemServiceCopyEnd 16 0x0
Noticed that the acsock64 calls occured just before the crash.
Examined available information about acsock64 with:
lmv m acsock64
Part of the result:
start end module name fffff807`453d0000 fffff807`45406000 acsock64 (no symbols) Loaded symbol image file: acsock64.sys Image path: \SystemRoot\system32\DRIVERS\acsock64.sys Image name: acsock64.sys Timestamp: Thu Oct 08 17:12:56 2015 (561687F8) CheckSum: 0003DD8B ImageSize: 00036000 Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
Noticed that the driver was around 14 months old at the time of writing.
Found the driver file in Windows explorer under:
C:\Windows\system32\DRIVERS\acsock64.sys
Then checked Properties. The details view revealed that the driver was:
Cisco AnyConnect Kernel Driver Framework Socket Layer Interceptor
Version: 4.2.1009.0
This confirmed my suspicion, Cisco AnyConnect most likely caused the BSOD.
At this point I would have updated to the latest version, if updates to Cisco AnyConnect were freely available.
Instead I decided to install and use the Windows port of OpenConnect as an alternative to Cisco AnyConnect.