Update: UltraVNC 1.4.3.6 and UltraVNC SC 1.4.3.6: viewtopic.php?t=37885
Important: Please update to latest version before to create a reply, a topic or an issue: viewtopic.php?t=37864

Join us on social networks and share our announcements:
- Website: https://uvnc.com/
- GitHub: https://github.com/ultravnc
- Mastodon: https://mastodon.social/@ultravnc
- Facebook: https://www.facebook.com/ultravnc1
- X/Twitter: https://twitter.com/ultravnc1
- Reddit community: https://www.reddit.com/r/ultravnc
- OpenHub: https://openhub.net/p/ultravnc

CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machine

Should you have problems with the DSM plugin, here's the place to look for help or report issues
Post Reply
alpha1
8
8
Posts: 20
Joined: 2007-03-16 12:41

CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machine

Post by alpha1 »

Hello,

during this past year i've been conecting to a vnc server without any problem,
I was using viewer on a WinXP SP3 x32

today I installed the winVISTA SP2, (dual boot)
but can´t connect anymore:

I ve this message:

"CONNECTION FAILED - INVALID PROTOCOL
possible causes: YOu've forgotten to select a DSM plugin and server uses a DSM plugin"

Server:
WIN XP SP2 x32 (pentium4 cpu)
winvnc 1.0.5.3
MSRC4Plugin_NoReg

Viewer:
WIN VISTA HOME basic SP2 x32 (pentium4 cpu)
vncviewer 1.0.5.6 (also tried 1.0.8.2)
MSRC4Plugin_NoReg

I´m using the same rc4 file on both sides
i check to use plugin here on viewer side

curiouse:
If I try to conncet using my laptop running 1.05.6 viewer WIN XP SP2, everything is ok.

so:
if I connect using viewer Windows VIsta, can't connect,
if using viewer on Win XP, no problem, everything ok

thanks in advance
Ol.
redge
1000
1000
Posts: 6797
Joined: 2004-07-03 17:05
Location: Switzerland - Geneva

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by redge »

you need to use vncviewer 1.0.8.2 on vista/7/2008 computer with msrc4plugin.dsm version 1.2.4.0
UltraVNC 1.0.9.6.1 (built 20110518)
OS Win: xp home + vista business + 7 home
only experienced user, not developer
alpha1
8
8
Posts: 20
Joined: 2007-03-16 12:41

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by alpha1 »

HI,

thanks for your answer.
do you think is good idea to mantain the MSRC4Plugin.dsm version 1.2.4.0?
or it´s time to change to the SecureVNCPlugin.dsm?

what´s the diference between: Server_SecureVNC.pkey and Server_ClientAuth.pubkey ?

I tried to read all post about the new 3 possible plug in, I also saw this:
[topic=15881][/topic]

can you tell me , in simple:

what Encryption Plugins is best on a XP server?
what Encryption Plugins is best on a Win7 or Vista server?

thnaks again
redge
1000
1000
Posts: 6797
Joined: 2004-07-03 17:05
Location: Switzerland - Geneva

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by redge »

is my opinion...

if you have pentium4 HT enabled or higher or Athlon XP or higher
you can use SecureVNCPlugin.dsm (AESv3)
otherwise, on olders computers, use MSRC4Plugin.dsm (MSRC4)
adzm wrote:ARC4/RC4 is significantly faster (hundreds of times) than AES, but CPU speed is not much of a factor for newer computers. Still, this should be considered when dealing with large terminal server environments or low-spec machines
adzm wrote: Installation and setup:

Download the SecureVNCPlugin.dsm and ensure your viewer and server applications are set to use it.

That is all!

Optionally, you can generate a key on the server. Otherwise, a new key will be generated for each connection. The pre-generated key is only used on the server! If you create a pre-generated key, do not share it! Pre-generated keys should be avoided when used on multiple servers.
...
more detail about SecureVNC
http://adamwalling.com/SecureVNC/

alpha1 wrote:what´s the diference between: Server_SecureVNC.pkey and Server_ClientAuth.pubkey ?

pkey - private key processing tool
pubkey - public key processing tool
Last edited by redge on 2010-02-07 20:24, edited 2 times in total.
UltraVNC 1.0.9.6.1 (built 20110518)
OS Win: xp home + vista business + 7 home
only experienced user, not developer
Cobra
20
20
Posts: 36
Joined: 2005-05-19 18:44

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by Cobra »

And in my humble opinion, the AES encryption is safer than the rest of the plugins. Even tough we are talking about theoretical possibilities, I like it more. :wink:

Regards,

Cobra
I know nothing except the fact of my ignorance. (Socrates 470 - 399 B.C.)
alpha1
8
8
Posts: 20
Joined: 2007-03-16 12:41

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by alpha1 »

thanks for your help.

can you be more specific about this? what do you mean by this?
pkey - private key processing tool
pubkey - public key processing tool

I went to http://adamwalling.com/SecureVNC/ , but its not so clear to me.

do you think its a good idea (more secure) to use those 3files? it gives me in fact a more secure connection?
Server_ClientAuth.pubkey
Server_SecureVNC.pkey
Viewer_ClientAuth.pkey



using the MSRC4Plugin I used to use before, we need to have the same rc4.key file on sever and viewer.
what about the new AESV3PluginMT.dsm (you suggested me), does this still needs the rc4.key file?

thanks again.
Ol
Cobra
20
20
Posts: 36
Joined: 2005-05-19 18:44

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by Cobra »

@Ol,
alpha1 wrote:do you think its a good idea (more secure) to use those 3files? it gives me in fact a more secure connection?
Server_ClientAuth.pubkey
Server_SecureVNC.pkey
Viewer_ClientAuth.pkey

using the MSRC4Plugin I used to use before, we need to have the same rc4.key file on sever and viewer.
what about the new AESV3PluginMT.dsm (you suggested me), does this still needs the rc4.key file?
No, it's not depending on how many files you got, but it's depending on the underlying encryption.

You don't need the rc4.key file anymore if you're using the AESv3 plugin - the only file, which is needed is the AESv3PluginMT.dsm, nothing more.

Give it a try and tell us if you like it. It's very easy to set up and it works perfectly! :wink:

Cobra
I know nothing except the fact of my ignorance. (Socrates 470 - 399 B.C.)
alpha1
8
8
Posts: 20
Joined: 2007-03-16 12:41

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by alpha1 »

hi,

I only asked about those 3 files, in order to know if it´s safer to use them.
so the question is: is there any advantage using them? is the conncetion more protected?
(i´m using also using the vnc password authentication)

i think I understood the all idea:

viewer side should use:Viewer_ClientAuth.pkey

but, the server should be using wich of them?: Server_ClientAuth.pubkey or Server_SecureVNC.pkey?

sorry, I still don´t undertstand the diferences betewen them...

(sorry my english)

many thanks!
Cobra
20
20
Posts: 36
Joined: 2005-05-19 18:44

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by Cobra »

Ol,

Sorry, but I don't get it what exactly you're trying to do. Why do you use an "old" encryption plugin from 2006? Don't you know, that scovel (the programer of those plugins) doesn't update those anymore? :tomato:

Now it's admz, who took a look into them. And he wrote the new plugins. They are much easier to use. Go ahead with the new one: :wink:

http://adamwalling.com/SecureVNC/SecureVNCPlugin.dsm

Regards,

Cobra
I know nothing except the fact of my ignorance. (Socrates 470 - 399 B.C.)
alpha1
8
8
Posts: 20
Joined: 2007-03-16 12:41

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by alpha1 »

Hi,

I was refering to this:
(wroted at http://adamwalling.com/SecureVNC/

"[i]As of 1.0.2.0, you now have the option of generating a key for authenticating the client. The server should have the public key (named *ClientAuth.pubkey), and the viewer should have the private key (named *ClientAuth.pkey). By cryptographically signing the communication, this prevents an active man-in-the-middle attack, since the communication key cannot be tampered with without invalidating the signature. This is ideal for remote support / single click situations, since the server can be publicly distributed with the public key for authentication. The viewer's private key must remain protected"[/i]

if I want to use the SecureVNCPlugin.dsm.
what abou using this extra "protection"?(generating those 2 key files)

is there any advantage using them? is the conncetion more protected?
(i´m using also using the vnc password authentication)

i think I understood the all idea:

viewer side should use:Viewer_ClientAuth.pkey

but, the server should be using wich of them?: Server_ClientAuth.pubkey or Server_SecureVNC.pkey?

sorry, I still don´t undertstand the diferences betewen them...
Cobra
20
20
Posts: 36
Joined: 2005-05-19 18:44

Re: CONNECTION FAILED - INVALID PROTOCOL -Vista viewer machi

Post by Cobra »

Hi Ol,

It's funny that you're still stuck with the old plugin. Adam (scovel) wrote those some years ago, they still needed those public and private keys. In my opinion not very handy, it makes the whole thing more complicated.

The new SecureVNCPlugin is easier to use and already implements those keys. You now have only two files - the plugin itself which has to be installed on the server side as well as on the client side. It's as safe as the plugin from Adam. Perhaps it's written in a more sophisticated way so that you don't need those keys anymore. And if you are paranoia, there's no secure way of communicating. Either it's the man-in-the-middle attack or they hack your viewer or server... Everything is possible. :D

But don't worry, I think that the communication of UltraVNC is secure. The question is more if your computer is as secure as this? :wink:

Regards,

Cobra

P.S. Your question is answered already in the text of Adam:

As of 1.0.2.0, you now have the option of generating a key for authenticating the client. The server should have the public key (named *ClientAuth.pubkey), and the viewer should have the private key (named *ClientAuth.pkey). The viewer's private key must remain protected"

viewer side should use:Viewer_ClientAuth.pkey

but, the server should be using wich of them?: Server_ClientAuth.pubkey or Server_SecureVNC.pkey?

Server_ClientAuth.pubkey can be distributed, Server_SecureVNC.pkey must remain protected and guarded somewhere safe.
Last edited by Cobra on 2010-02-19 20:30, edited 1 time in total.
I know nothing except the fact of my ignorance. (Socrates 470 - 399 B.C.)
Post Reply