You can use the following troubleshooting steps in order to fix the issue:

1.Confirm that the client is connecting via the external IP of the instance. Test network connectivity by pinging ICMP it from the client machine to ensure that it can reach the remote instance. Before doing this test make sure that ICMP traffic is allowed in Google network.

2.Check that a GCP firewall rule is already created to allow connections to the instance on the RDP port (tcp:3389 by default).

3.Establish a connection to the machine using one of the available serial console methods to run advanced troubleshooting via command line:

GCP serial console (port 2)

Open the dropdown menu and select port 2 to access via an interactive serial console.

Commands:

cmd
ch -sn cmd0001 (or the name of the channel that was created if not 'cmd0001')

4.Check that your RDP service is listening (Default port 3389)

Command: netstat -an | find "[PORT NUMBER]"
Pass: TCP 0.0.0.0:[PORT NUMBER] 0.0.0.0:0 LISTENING
Fail: (unexpected results, no output)
Solution: (continue with steps below)   

5.Check to see that the 'Remote Desktop Service' is running:

Command: net start | find "Remote Desktop Services"

Pass: Remote Desktop Service

Fail: (Remote Desktop Service missing from output)

Solution: net start "Remote Desktop Services"

Additionally, It's possible that internal firewall of Windows Server is blocking the non-standard port that you configured for RDP service. You can try turning off Windows firewall to verify if this is the cause of the issue. To discard this is not the root cause, you could also spin a new VM using a public GCP public image. By default public Windows images allow communication to RDP port.

If the your testing points to the internal firewall (Windows firewall) you could disable it by running a startup script. For example, add “windows-startup-script-cmd” custom metadata key with a value of “netsh advfirewall set allprofiles state off” to your Windows VM and then reset the machine. Startup script will run and turn off Windows firewall. At this point, you can try connecting to your Windows machine through RDP to verify if the internal firewall was blocking the RDP connection.