UltraVNC not responding to connection attempts

A user reported he was unable to connect to a Microsoft Windows 8 system functioning as a VNC server using UltraVNC. The remote vncviewer program was reporting "Failed to connect to server !" I logged into the system and verified that the system was listening on the registered port for VNC, port 5900 using the netstat command.
C:\Users\Collen>netstat -a | find "5900"
  TCP    0.0.0.0:5900           Speleologist:0         LISTENING

C:\Users\Collen>

I used windump, a free packet capture tool for Microsoft Windows systems, similar to the tcpdump tool found on OS X/macOS and Linux systems, to verify that the system was seeing the incoming connections to network port 5900.

C:\Program Files (x86)\Network\>windump port 5900
windump: listening on \Device\NPF_{B5B6E80B-A4BF-4B04-90ED-573BA531BD03}
19:05:00.749658 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,nop,wscale 5,nop,nop,timestamp 459255717 0,sackOK,eo
l>
19:05:01.739199 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,nop,wscale 5,nop,nop,timestamp 459256717 0,sackOK,eo
l>
19:05:02.737463 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,nop,wscale 5,nop,nop,timestamp 459257718 0,sackOK,eo
l>
19:05:03.746881 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,nop,wscale 5,nop,nop,timestamp 459258719 0,sackOK,eo
l>
19:05:04.741430 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,nop,wscale 5,nop,nop,timestamp 459259719 0,sackOK,eo
l>
19:05:05.759667 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,nop,wscale 5,nop,nop,timestamp 459260720 0,sackOK,eo
l>
19:05:07.759922 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,nop,wscale 5,nop,nop,timestamp 459262720 0,sackOK,eo
l>
19:05:11.776950 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,sackOK,eol>
19:05:19.770063 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,sackOK,eol>
19:05:35.784238 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,sackOK,eol>
19:06:07.793203 IP 192.168.226.54.49672 > Speleologist.5900: S 3356145059:335614
5059(0) win 65535 <mss 1360,sackOK,eol>

11 packets captured
3360 packets received by filter
0 packets dropped by kernel

C:\Program Files (x86)\Network\Monitoring>

I could see the incoming packets, but I didn't see any responses from the VNC server, so I stopped and restarted the UltraVNC service. Note: if you try to stop the service from a regular command prompt rather than an administrator command prompt, you will receive the message below:

C:\Program Files (x86)\Network>net stop uvnc_service
System error 5 has occurred.

Access is denied.


C:\Program Files (x86)\Network>

When I opened an administrator command prompt, I was able to successfully restart the service.

Anonymous Online: The Ultimate Guide to Online Privacy
Anonymous Online
The Ultimate Guide to Online Privacy
1x1 px

C:\Windows\system32>net stop uvnc_service
The uvnc_service service is stopping...
The uvnc_service service was stopped successfully.


C:\Windows\system32>net start uvnc_service
The uvnc_service service is starting.
The uvnc_service service was started successfully.


C:\Windows\system32>

But the same problem occurred. I.e., remote VNC connections failed and, using windump, I could see incoming packets to port 5900, but no outgoing packets. So at that point I surmised it was probably a host-based firewall issue, since the system uses the Windows Firewall. I stopped the firewall, which also has to be done from an administrator command prompt, and then attempted a remote VNC connection. That worked. When I turned the firewall back on, connectivity to port 5900 with VNC again failed.

C:\Windows\system32>netsh advfirewall set currentprofile state off
Ok.


C:\Windows\system32>

C:\Windows\system32>netsh advfirewall set currentprofile state on
Ok.

When I looked for any firewall rules referencing "vnc", I saw the following:

C:\Windows\system32>netsh advfirewall firewall show rule name=all dir=in | find /i "vnc"
Rule Name:                            vncviewer.exe
Rule Name:                            vncviewer.exe
Rule Name:                            winvnc.exe
Rule Name:                            winvnc.exe

C:\Windows\system32>

Since I wasn't concerned about connectivity to the viewer application, I looked at the rules for winvnc.exe and saw the following:




C:\Windows\system32>netsh advfirewall firewall show rule name=winvnc.exe dir=in

Rule Name:                            winvnc.exe
----------------------------------------------------------------------
Enabled:                              Yes
Direction:                            In
Profiles:                             Private
Grouping:
LocalIP:                              Any
RemoteIP:                             Any
Protocol:                             UDP
LocalPort:                            Any
RemotePort:                           Any
Edge traversal:                       No
Action:                               Allow

Rule Name:                            winvnc.exe
----------------------------------------------------------------------
Enabled:                              Yes
Direction:                            In
Profiles:                             Private
Grouping:
LocalIP:                              Any
RemoteIP:                             Any
Protocol:                             TCP
LocalPort:                            Any
RemotePort:                           Any
Edge traversal:                       No
Action:                               Allow
Ok.


C:\Windows\system32>

That revealed that there were two rules, one for TCP and the other for UDP, but both only applied to "private" network profiles. I could also see that through the graphical user interface (GUI) for managing firewall rules by selecting Control Panel, System and Security, Windows Firewall, and then Allow an app or feature through Windows Firewall.

Udemy Generic Category (English)120x600

Windows Firewall allowed apps

At the prior Windows Firewall window, I could see that the system believed it was connected to a "guest or public network".

Windows Firewall public network

You can also see whether the Windows firewall believes you are currently connected to a public, private, or work network using the command netsh advfirewall show currentprofile at a command prompt.

C:\Windows\system32>netsh advfirewall show currentprofile

Public Profile Settings:
----------------------------------------------------------------------
State                                 OFF
Firewall Policy                       BlockInbound,AllowOutbound
LocalFirewallRules                    N/A (GPO-store only)
LocalConSecRules                      N/A (GPO-store only)
InboundUserNotification               Enable
RemoteManagement                      Disable
UnicastResponseToMulticast            Enable

Logging:
LogAllowedConnections                 Disable
LogDroppedConnections                 Disable
FileName                              %systemroot%\system32\LogFiles\Firewall\pf
irewall.log
MaxFileSize                           4096

Ok.

C:\Windows\system32>

The system had only one possible network connection. I chose to enable the firewall rules for winvnc.exe by clicking on the Change settings button at the "Allow apps to communicate through Windows Firewall" window, which then allowed me to put a check in the public check box next to winvnc.exe.

Windows Firewall winvnc public network

I then clicked on OK. I then re-enabled the Windows Firewall from the command line for the current profile, which was a public profile with the command netsh advfirewall set currentprofile state on. Remote VNC connections then worked. They had been working previously, so, perhaps, the system changed its view of the network after a reboot regarding the network connection to be a public rather than private network.