Wireshark can hang when stopping capture

I have experienced problems with recent Wireshark versions on Windows, including the current latest version 2.2.3.

 

The way I normally use Wireshark is to capture the traffic of interest, then stop the capture and finally analyze the traffic.

However Wireshark on Windows can hang when stopping the capture…

 

When it happens the Wireshark UI becomes non-responsive:

 

The main thread runs at 100% CPU utilization.

 

And Process Monitor shows that the Wireshark process reads the “End Of File” from the same temporary file over and over:

 

Started debugging the issue by taking multiple memory dumps of the wireshark.exe process using procdump with:

procdump -ma -n 10 -s 1 wireshark.exe

 

Opened the first memory dump in WinDbg (x64).

Checked the call stack for the main thread (0) with:

~0 kp

Result:

# Child-SP          RetAddr           Call Site
00 00000039`a00f9fc8 00007ffc`e5dbc264 ntdll!NtReadFile+0x14
01 00000039`a00f9fd0 00007ffc`d9ca9aa7 KERNELBASE!ReadFile+0x74
02 00000039`a00fa050 00007ffc`d9ca9782 msvcr120!_read_nolock(int fh = 0n8, void * inputbuf = 0x000001a8`3e57bd20, unsigned int cnt = 0x1000)+0x2cf [f:\dd\vctools\crt\crtw32\lowio\read.c @ 256]
03 00000039`a00fa0f0 00007ffc`a14f09b3 msvcr120!_read(int fh = 0n8, void * buf = 0x000001a8`3e57bd20, unsigned int cnt = 0x1000)+0xc6 [f:\dd\vctools\crt\crtw32\lowio\read.c @ 92]
04 00000039`a00fa140 00007ffc`a14efde4 libwiretap!file_tell+0xc93
05 00000039`a00fa170 00007ffc`a14ef98c libwiretap!file_tell+0xc4
06 00000039`a00fa1a0 00007ffc`a1517c8d libwiretap!file_read+0xac
07 00000039`a00fa1e0 00007ffc`a150abb3 libwiretap!wtap_read_bytes_or_eof+0x2d
08 00000039`a00fa210 00007ffc`a150a9a9 libwiretap!wtap_wtap_encap_to_pcap_encap+0xbd3
09 00000039`a00fa290 00007ffc`a1517b77 libwiretap!wtap_wtap_encap_to_pcap_encap+0x9c9
0a 00000039`a00fa310 00007ff7`c7929fd3 libwiretap!wtap_read+0x37
0b 00000039`a00fa350 00007ff7`c7b7cfd7 Wireshark+0x9fd3
0c 00000039`a00fa3a0 00007ff7`c7b963dd Wireshark+0x25cfd7
0d 00000039`a00fa3e0 00007ff7`c798889a Wireshark+0x2763dd
0e 00000039`a00fb470 00007ff7`c79d5144 Wireshark+0x6889a
0f 00000039`a00fb4c0 00000000`5b94f906 Wireshark+0xb5144
10 00000039`a00fb5c0 00000000`5b9c4d66 Qt5Core!QMetaObject::activate+0x5a6
11 00000039`a00fb6d0 00000000`5b95413a Qt5Core!QTimer::timeout+0x16
12 00000039`a00fb700 00000000`5bcf7d12 Qt5Core!QObject::event+0x6a
13 00000039`a00fb8b0 00000000`5bcf6c2f Qt5Widgets!QApplicationPrivate::notify_helper+0x112
14 00000039`a00fb8e0 00000000`5b92f689 Qt5Widgets!QApplication::notify+0x1b3f
15 00000039`a00fc000 00000000`5b977a8c Qt5Core!QCoreApplication::notifyInternal2+0xb9
16 00000039`a00fc080 00000000`5b976a32 Qt5Core!QEventDispatcherWin32Private::sendTimerEvent+0x10c
17 00000039`a00fc0e0 00007ffc`e6851c24 Qt5Core!QEventDispatcherWin32::processEvents+0xd82
18 00000039`a00fc1f0 00007ffc`e685156c user32!UserCallWinProcCheckWow+0x274
19 00000039`a00fc350 00000000`5b9761d9 user32!DispatchMessageWorker+0x1ac
1a 00000039`a00fc3d0 00007ffc`a14029b9 Qt5Core!QEventDispatcherWin32::processEvents+0x529
1b 00000039`a00ff760 00000000`5b92bf91 qwindows!qt_plugin_query_metadata+0x2499
1c 00000039`a00ff790 00000000`5b92e477 Qt5Core!QEventLoop::exec+0x1b1
1d 00000039`a00ff810 00007ff7`c7929ccd Qt5Core!QCoreApplication::exec+0x147
1e 00000039`a00ff880 00007ff7`c7ba2ac5 Wireshark+0x9ccd
1f 00000039`a00ffd50 00007ff7`c7ba22fd Wireshark+0x282ac5
20 00000039`a00ffde0 00007ffc`e87c8364 Wireshark+0x2822fd
21 00000039`a00ffe20 00007ffc`e8f470d1 kernel32!BaseThreadInitThunk+0x14
22 00000039`a00ffe50 00000000`00000000 ntdll!RtlUserThreadStart+0x21

 

Checked the main thread (0) call stack for all the memory dumps by scripting CDB, the console version of WinDbg.

Used this PowerShell script:

$dmpPath = 'C:\Bin\Procdump\Wireshark\'
$dmpFiles = Get-ChildItem -Path $dmpPath -Recurse -Include *.dmp

foreach ($dmpFile in $dmpFiles)
{
    & "C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe" -z $dmpFile -c "~0 kp; Q"
}

 

Noticed that all the memory dumps had the same call stack for thread 0, which further indicated that WireShark was stuck or running in an endless loop.

 

Now I wanted to identify if this was a hang or endless loop.

Attached to the running, but non-responsive Wireshark process with WinDbg (x64).

Experimented by setting breakpoints from the initial call stack.

 

Set a single breakpoint at:

MSVCR120!_read_nolock(...)+0x2cf

With:

bc *
bp MSVCR120!_read_nolock+0x2cf

Result after continue: Breakpoint was continually hit.

 

This definitely indicated an endless loop.

 

Set a single breakpoint at:

libwiretap!wtap_read_bytes_or_eof+0x2d

With:

bc *
bp libwiretap!wtap_read_bytes_or_eof+0x2d

Result after continue: Breakpoint was continually hit.

 

Set a single breakpoint at:

Qt5Core!QTimer::timeout+0x16

With:

bc *
bp Qt5Core!QTimer::timeout+0x16

Result after continue: Breakpoint was not hit (within 1 minute)

 

Set single breakpoints at the various Wireshark functions.

The breakpoint was not hit until setting a single breakpoint at:

Wireshark+0x9fd3

With:

bc *
bp Wireshark+0x9fd3

Result after continue: The breakpoint was continually hit.

 

This indicated that the endless loop occured in the Wireshark module.

 

Had a quick look at the disassembly and tried single stepping with:

p

The code was definitely looping, but the problem was not obvious by looking at the machine code.

 

Went looking for debug symbols and found them here:

https://1.eu.dl.wireshark.org/win64/all-versions/Wireshark-pdb-win64-2.2.3.zip

Downloaded the debug symbols and unpacked them to a temporary folder.

 

Then modified the symbol path in WinDbg with:

.sympath C:\Temp\WSSymbols\;srv*c:\SymbolsCache*https://msdl.microsoft.com/download/symbols

And reloaded the symbols with:

.reload /f

 

Checked the call stack again with:

~0 kp

Result:

# Child-SP          RetAddr           Call Site
00 00000039`a00f9fc8 00007ffc`e5dbc264 ntdll!NtReadFile+0x14
01 00000039`a00f9fd0 00007ffc`d9ca9aa7 KERNELBASE!ReadFile+0x74
02 00000039`a00fa050 00007ffc`d9ca9782 MSVCR120!_read_nolock(int fh = 0n8, void * inputbuf = 0x000001a8`3e57bd20, unsigned int cnt = 0x1000)+0x2cf [f:\dd\vctools\crt\crtw32\lowio\read.c @ 256]
03 00000039`a00fa0f0 00007ffc`a14f09b3 MSVCR120!_read(int fh = 0n8, void * buf = 0x000001a8`3e57bd20, unsigned int cnt = 0x1000)+0xc6 [f:\dd\vctools\crt\crtw32\lowio\read.c @ 92]
04 00000039`a00fa140 00007ffc`a14efde4 libwiretap!raw_read(struct wtap_reader * state = 0x000001a8`4ad0ebd0, unsigned char * buf = 0x000001a8`3e57bd20 "vided by dumpcap???", unsigned int count = 0x1000, unsigned int * have = 0x000001a8`4ad0ec08)+0x43 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wiretap\file_wrappers.c @ 133]
05 00000039`a00fa170 00007ffc`a14ef98c libwiretap!fill_out_buffer(struct wtap_reader * state = 0x000001a8`4ad0ebd0)+0x44 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wiretap\file_wrappers.c @ 704]
06 00000039`a00fa1a0 00007ffc`a1517c8d libwiretap!file_read(void * buf = 0x00000039`a00fa250, unsigned int len = 8, struct wtap_reader * file = 0x000001a8`4ad0ebd0)+0xac [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wiretap\file_wrappers.c @ 1237]
07 00000039`a00fa1e0 00007ffc`a150abb3 libwiretap!wtap_read_bytes_or_eof(struct wtap_reader * fh = 0x000001a8`4ad0ebd0, void * buf = <Value unavailable error>, unsigned int count = 8, int * err = 0x00000039`a00fa3a0, char ** err_info = 0x00000039`a00fa3b0)+0x2d [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wiretap\wtap.c @ 1291]
08 00000039`a00fa210 00007ffc`a150a9a9 libwiretap!pcapng_read_block(struct wtap * wth = 0x000001a8`41f48790, struct wtap_reader * fh = 0x000001a8`4ad0ebd0, struct pcapng_t * pn = 0x000001a8`41f1ca40, struct wtapng_block_s * wblock = 0x00000039`a00fa2c0, int * err = 0x00000039`a00fa3a0, char ** err_info = 0x00000039`a00fa3b0)+0x53 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wiretap\pcapng.c @ 2303]
09 00000039`a00fa290 00007ffc`a1517b77 libwiretap!pcapng_read(struct wtap * wth = 0x000001a8`41f48790, int * err = 0x00000039`a00fa3a0, char ** err_info = 0x00000039`a00fa3b0, int64 * data_offset = 0x00000039`a00fa3a8)+0x79 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wiretap\pcapng.c @ 2589]
0a 00000039`a00fa310 00007ff7`c7929fd3 libwiretap!wtap_read(struct wtap * wth = 0x000001a8`41f48790, int * err = 0x00000039`a00fa3a0, char ** err_info = 0x00000039`a00fa3b0, int64 * data_offset = <Value unavailable error>)+0x37 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wiretap\wtap.c @ 1237]
0b 00000039`a00fa350 00007ff7`c7b7cfd7 Wireshark!capture_info_new_packets(int to_read = 0n14, struct _info_data * cap_info = 0x000001a8`3f1838b8)+0x43 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\capture_info.c @ 211]
0c 00000039`a00fa3a0 00007ff7`c7b963dd Wireshark!capture_input_new_packets(struct _capture_session * cap_session = 0x000001a8`3f183878, int to_read = 0n23)+0xa7 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\ui\capture.c @ 407]
0d 00000039`a00fa3e0 00007ff7`c798889a Wireshark!sync_pipe_input_cb(int source = 0n4, void * user_data = 0x000001a8`3f183878)+0x1bd [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\capchild\capture_sync.c @ 1775]
0e 00000039`a00fb470 00007ff7`c79d5144 Wireshark!MainWindow::pipeTimeout(void)+0x8a [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\ui\qt\main_window_slots.cpp @ 949]
0f 00000039`a00fb4c0 00000000`5b94f906 Wireshark!MainWindow::qt_static_metacall(class QObject * _o = 0x000001a8`3f1836e0, QMetaObject::Call _c = <Value unavailable error>, int _id = <Value unavailable error>, void ** _a = 0x00000039`a00fb5f8)+0x4c4 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\cmbuild\ui\qt\moc_main_window.cpp @ 1436]
10 00000039`a00fb5c0 00000000`5b9c4d66 Qt5Core!QMetaObject::activate+0x5a6
11 00000039`a00fb6d0 00000000`5b95413a Qt5Core!QTimer::timeout+0x16
12 00000039`a00fb700 00000000`5bcf7d12 Qt5Core!QObject::event+0x6a
13 00000039`a00fb8b0 00000000`5bcf6c2f Qt5Widgets!QApplicationPrivate::notify_helper+0x112
14 00000039`a00fb8e0 00000000`5b92f689 Qt5Widgets!QApplication::notify+0x1b3f
15 00000039`a00fc000 00000000`5b977a8c Qt5Core!QCoreApplication::notifyInternal2+0xb9
16 00000039`a00fc080 00000000`5b976a32 Qt5Core!QEventDispatcherWin32Private::sendTimerEvent+0x10c
17 00000039`a00fc0e0 00007ffc`e6851c24 Qt5Core!QEventDispatcherWin32::processEvents+0xd82
18 00000039`a00fc1f0 00007ffc`e685156c USER32!UserCallWinProcCheckWow+0x274
19 00000039`a00fc350 00000000`5b9761d9 USER32!DispatchMessageWorker+0x1ac
1a 00000039`a00fc3d0 00007ffc`a14029b9 Qt5Core!QEventDispatcherWin32::processEvents+0x529
1b 00000039`a00ff760 00000000`5b92bf91 qwindows!qt_plugin_query_metadata+0x2499
1c 00000039`a00ff790 00000000`5b92e477 Qt5Core!QEventLoop::exec+0x1b1
1d 00000039`a00ff810 00007ff7`c7929ccd Qt5Core!QCoreApplication::exec+0x147
1e 00000039`a00ff880 00007ff7`c7ba2ac5 Wireshark!main(int argc = 0n1, char ** qt_argv = <Value unavailable error>)+0xe3d [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\wireshark-qt.cpp @ 853]
1f 00000039`a00ffd50 00007ff7`c7ba22fd Wireshark!WinMain+0x155
20 00000039`a00ffde0 00007ffc`e87c8364 Wireshark!__tmainCRTStartup(void)+0x149 [f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c @ 618]
21 00000039`a00ffe20 00007ffc`e8f470d1 KERNEL32!BaseThreadInitThunk+0x14
22 00000039`a00ffe50 00000000`00000000 ntdll!RtlUserThreadStart+0x21

 

The symbols seemed to be correct and functional.

 

Wanted to verify the location of the endless loop by setting breakpoints again.

 

Set a single breakpoint at:

Wireshark!capture_input_new_packets

With:

bc *
bp Wireshark!capture_input_new_packets

Result after continue: Breakpoint was not hit (within 1 minute)

 

Set a single breakpoint at:

Wireshark!capture_info_new_packets

With:

bc *
bp Wireshark!capture_info_new_packets

Result after continue: Breakpoint was not hit (within 1 minute)

 

Set a single breakpoint at:

libwiretap!wtap_read

With:

bc *
bp libwiretap!wtap_read

Result after continue: The breakpoint was continually hit.

 

This result indicated that the endless loop was in Wireshark!capture_info_new_packets(…) calling libwiretap!wtap_read(…), specifically here:

Wireshark!capture_info_new_packets(int to_read = 0n14, struct _info_data * cap_info = 0x000001a8`3f1838b8)+0x43 [c:\buildbot\wireshark\wireshark-2.2-64\windows-2012r2-x64\build\capture_info.c @ 211]

 

Downloaded the source code for Wireshark from:

https://1.eu.dl.wireshark.org/src/wireshark-2.2.3.tar.bz2

 

Looked for and opened: capture_info.c

Found the method, which has been included here:

/* new packets arrived */
void capture_info_new_packets(int to_read, info_data_t* cap_info)
{
    int err;
    gchar *err_info;
    gint64 data_offset;
    struct wtap_pkthdr *phdr;
    union wtap_pseudo_header *pseudo_header;
    int wtap_linktype;
    const guchar *buf;

    cap_info->ui.new_packets = to_read;

    /*g_warning("new packets: %u", to_read);*/

    while (to_read > 0) {
        wtap_cleareof(cap_info->wtap);
        if (wtap_read(cap_info->wtap, &err, &err_info, &data_offset)) {
            phdr = wtap_phdr(cap_info->wtap);
            pseudo_header = &phdr->pseudo_header;
            wtap_linktype = phdr->pkt_encap;
            buf = wtap_buf_ptr(cap_info->wtap);

            capture_info_packet(cap_info, wtap_linktype, buf, phdr->caplen, pseudo_header);

            /*g_warning("new packet");*/
            to_read--;
        }
    }

    capture_info_ui_update(&cap_info->ui);
}

 

I have marked some points of interest with bold.

 

The while loop continues as long as to_read is greater than 0.

Noticed that the to_read variable continued to have the value 14.

to_read is only decremented in case wtap_read returns a non-zero value.

It seems that this never happens, when the endless loop occurs in Wireshark.

 

I have noticed that the problem apparently doesn’t occur with Wireshark Legacy based on GTK (at least with default settings).

I wanted to determine the cause for the difference.

 

Examined symbols for Wireshark (QT) with:

x /D /f Wireshark!capture_info_new_packets

Result:

00007ff7`c7929f90 Wireshark!capture_info_new_packets (int, struct _info_data *)

 

Examined symbols for Wireshark Legacy (GTK) with:

x /D /f Wireshark_gtk!capture_info_new_packets

Result:

00007ff7`c03310f0 Wireshark_gtk!capture_info_new_packets (int, struct _info_data *)

 

Both versions include the function capture_info_new_packets (…)

 

Checked where the function was mentioned in the source code with grep (from Cygwin):

grep "capture_info_new_packets" -r

Result:

capture_info.c:void capture_info_new_packets(int to_read, info_data_t* cap_info)
capture_info.h:extern void capture_info_new_packets(int to_read, info_data_t* cap_info);
ui/capture.c:    capture_info_new_packets(to_read, cap_session->cap_data_info);

 

Checked the content of ui/capture.c and found:

if(capture_opts->show_info)
    capture_info_new_packets(to_read, cap_session->cap_data_info);

 

So the capture_info_new_packets(…) function will only be called if the show_info option is true…

 

Looked for places where show_info was used with:

grep "show_info" -r

Part of the result was:

ui/gtk/capture_dlg.c:  gtk_toggle_button_set_active(GTK_TOGGLE_BUTTON(hide_info_cb), !global_capture_opts.show_info);
ui/gtk/capture_dlg.c:  global_capture_opts.show_info =
ui/qt/capture_interfaces_dialog.cpp:    global_capture_opts.show_info = checked;
ui/qt/capture_interfaces_dialog.cpp:    ui->cbExtraCaptureInfo->setChecked(global_capture_opts.show_info);

 

For Wireshark Legacy (GTK) I checked the content of ui/gtk/capture_dlg.c and found:

global_capture_opts.show_info =
    !gtk_toggle_button_get_active(GTK_TOGGLE_BUTTON(hide_info_cb));

 

For Wireshark Legacy (GTK) the endless loop problem can be avoided by enabling this option in the Capture Options dialog :

Hide capture info dialog   (default setting)

 

For Wireshark (QT) I checked the content of ui/qt/capture_interfaces_dialog.cpp and found:

void CaptureInterfacesDialog::on_cbExtraCaptureInfo_toggled(bool checked)
{
    global_capture_opts.show_info = checked;
}

 

For Wireshark (QT) the endless loop problem can be avoided by disabling this option in the Capture Interfaces dialog :

Show extra capture information dialog   (not default setting)

 

Conclusion

The endless loop problem occurs consistently depending on the identified settings, so the problem cause has likely been found along with a workaround.

Until the endless loop bug has been fixed in Wireshark, it is recommended to set the Wireshark options like described above.