Update: UltraVNC 1.4.3.6 and UltraVNC SC 1.4.3.6: viewtopic.php?t=37885
Important: Please update to latest version before to create a reply, a topic or an issue: viewtopic.php?t=37864

Join us on social networks and share our announcements:
- Website: https://uvnc.com/
- GitHub: https://github.com/ultravnc
- Mastodon: https://mastodon.social/@ultravnc
- Facebook: https://www.facebook.com/ultravnc1
- X/Twitter: https://twitter.com/ultravnc1
- Reddit community: https://www.reddit.com/r/ultravnc
- OpenHub: https://openhub.net/p/ultravnc

MSRC4Plugin.dsm Plug-In FAQ

Should you have problems with the DSM plugin, here's the place to look for help or report issues
Post Reply
balubeto
40
40
Posts: 82
Joined: 2004-11-05 15:35
Contact:

MSRC4Plugin.dsm Plug-In FAQ

Post by balubeto »

HI

I have need of a FAQ that me aids to configure MSRC4Plugin.dsm Plug-In in how much, when I configure it and I close the window "WinVNC: Default Local System Properties" of Ultr@VNC Server (RC19.4), me appears the error window: <The plugin was unable to locate the key file. Please ensure that the registry entry is properly set and that the key file exists>. How Never?

THANKS

HI

P.S: I use Windows XP Pro SP2
balubeto
Ipsec
Former moderator
Former moderator
Posts: 565
Joined: 2004-09-20 18:56
Contact:

Post by Ipsec »

Interesting - same post twice lol JK!

The DSM Plugin FAQ just tells you how to know if you are or arent using the Security plugin, not how to configure it. I think they expect you to actually read the DSM Security Module home page for that help / information.

How to use the plugin -

http://home.comcast.net/%7Emsrc4plugin/ ... _docs.html

He has a very good instruction set on how to actually download, configure and use it.

But basic instructions -
Copy - MSRC4Plugin.dsm to your UltraVNC directory (usually C:\program files\UltraVnc\ )
Server Admin Properties or Default Properties - Select DSM Plugin - click Configure.
On the dialogue box that opens click before you click Genereate Key - select encryption and select the name that you wish to use. (very last line)
By default the plugin name stays in the UltraVNC dir if you only change the name. And The encryption is the lowest. So customize it then hit the Gen Key button.
It doesnt tell you anything however if you did it right it is in your VNC directory under the name you specified.
Change the name in the server line if you customized it. Otherwise leave it alone.
****Your server is now set up to use the DSM plugin and any viewer connecting needs the Key file you just created.*** Of course if you configured it properly.


Copy the MSRC4Plugin.dsm & RC4.key file to the viewers computer under the UltraVNC dir. (Should be same as above)
When using the viewer you must select the DSM plugin, click the checkbox to use the plugin then click configure. This is where you specify the Key file. That key file can be one of many if you connect to other networks / computers.

When connecting you then use the FAQ you read to ensure you are actually using the security plugin - the easiest one is - check the top bar and it says you are or arent using the security e.g. it says + MSRC4 Plugin V.1.1.0.

Hope this helps.
~Disclaimer - if you didnt configure it right it wont work and if you arent using it properly its not my fault, the fault of the devs, or anyone else besides you..
balubeto
40
40
Posts: 82
Joined: 2004-11-05 15:35
Contact:

Post by balubeto »

HI

The error <The plugin was unable to locate the key file. Please ensure that the registry entry is properly set and that the key file exists> has been exceeded thanks to you but, now, I have an other problem.

Now, I explain you my situation:

I have two computer A and B where I have installed the RC19.4 complete of VNC Server and of VNC Viewer in how much I must be able to control computer A from B but also from B to A.

Therefore, I have configured MSRC4Plugin.dsm Plug-In on both computer putting in all the four text box the path of file rc4.key that it is found in the directory of installation of UltraVNC and selecting the option 128bit. End here, OK.

Now, when I try the remote connection from Ato B or B to A (having cure to select the use of MSRC4Plugin.dsm PlugIn in the Viewer) me appears the following message of error: <Connection failed. Invalid protocol !>. Like never?

THANKS

HI
balubeto
Ipsec
Former moderator
Former moderator
Posts: 565
Joined: 2004-09-20 18:56
Contact:

Post by Ipsec »

From the viewer machine - dont use the plugin.. JUST TO TEST..

it sounds like you dont have the plugin selected at the server machine.

For ease lets call computer A your server and computer B your viewer -

When computer B connects and gets the invalid protocol its one of the following - you either dont have the right key file, or DSM selected OR your Server Machine (computer A) isnt running the encryption plugin like you intend it to.

I would check that out.. The error comes up with the most likely causes of what is wrong.
balubeto
40
40
Posts: 82
Joined: 2004-11-05 15:35
Contact:

Post by balubeto »

HI

Therefore, as I must make in order to create an right key file on the computer A and B so that this key file functions?

THANKS

HI
balubeto
Ipsec
Former moderator
Former moderator
Posts: 565
Joined: 2004-09-20 18:56
Contact:

Post by Ipsec »

Not sure if you were trying to say this but -

The key file you create on the Server computer (A)
Must be on the Viewer computer (b).

You can not generate two different keys only one.
balubeto
40
40
Posts: 82
Joined: 2004-11-05 15:35
Contact:

Post by balubeto »

HI

How I make to make that?

THANKS

HI
balubeto
Sean

Registry problem

Post by Sean »

This is probably due to a change in UltraVNC that is causing a problem with the plugin. The plugin is trying to get read/write access to the registry under HKLM, and probably doesn't have permission to write to it. In the past it DID have write access when running as a service. Now it doesn't.

I'm working on a fix for this with Sam and Rudi.

Sean
redge
1000
1000
Posts: 6797
Joined: 2004-07-03 17:05
Location: Switzerland - Geneva

Post by redge »

winvnc rc19.6 as system service on XP SP2 US
C:\Program Files\UltraVNC\MSRC4Plugin_noreg

[x]Use the DSM Plugin: MSRC4Plugin_noreg
Configuration
Gen Key (closing without any comment and don't create any rc4.key )
OK
(popup: The plugin was unable to locate the key file. Please ensure that the registry entry is properly set and that the key file exists)
rc4.key

Delete container : An unrecoverable error occured
Delete Container failed (normal, not exist !)

unfriendly plugin application for secure connection
can you add some pictures step by step for non english users
i read documentation and I don't understanding everything !

crypto -l give to much possibilty !

RC4 default 1
RC4 default 12
RC4 default 13
RC4 default 18
RC4 Microsoft....

restart computer...try it...restart computer ...without success !
i'm lamer ! :oops:
UltraVNC 1.0.9.6.1 (built 20110518)
OS Win: xp home + vista business + 7 home
only experienced user, not developer
scovel
100
100
Posts: 307
Joined: 2004-07-12 11:56
Location: CT, USA
Contact:

Help is on the way...

Post by scovel »

Redge,

The "noreg" version is new so I'm sure not all of the documentation is up-to-date. Sorry!

When you create the key from the SERVER, the key file will probably end up in your system32 directory. If you still can't find it, search for it. If you STILL can't find it, then probably your best bet is to create the key file from the viewer. If you STILL can't find it, Email me and I'll help you turn on logging so we can see what is happening. msrc4plugin AT comcast.net

Tell me where you get lost in my directions so I can help you out, and so I can make the documentation better. I have made a number of improvements to the plugin for non-English users. I'm more than willing to make improvements to the documentation too.

Sean
Joe Hoffman

Followon comments

Post by Joe Hoffman »

I had some similar problems and worked around most of them. Here's a few comments.

1) When using the no_reg plugin, I found that the path to the keys was not editable in the plugin window, however the key file was being generated on my desktop and that's where the plugin wanted to find it when I ran the server/viewer.

2) Since I didn't like the desktop location, I started using the other plug that was not "no_reg". I assumed this meant the key was going to be in the registry, however that was not the case. I found the key file path WAS editable in the plugin window and I specified c:\program files\ultravnc\rc4.key. Then VNC was able to find the key and all worked fine.

3) I also found that when using the plugin, VNC would not accept a connection after the first connection, even after the first connection is terminated. To get around this, I went into the server VNC properties screen and just selected "Apply". Then the connection was accepted. However again, after this connnection was termianted, no further connections were accepted by the server until the "apply" was applied again.

Hope this helps.

This was using RC18.

Joe Hoffman
joe at intelopment dot com
redge
1000
1000
Posts: 6797
Joined: 2004-07-03 17:05
Location: Switzerland - Geneva

Post by redge »

Joe Hoffman

You must use => UltraVNC RC19.5 for use DSMPlugin_no_reg
UltraVNC 1.0.9.6.1 (built 20110518)
OS Win: xp home + vista business + 7 home
only experienced user, not developer
scovel
100
100
Posts: 307
Joined: 2004-07-12 11:56
Location: CT, USA
Contact:

RC19-5 or later

Post by scovel »

Joe,

The noreg plugin looks in a few places for the key file when it is encrypting the session.

If the key file got generated on the desktop, I assume you used the viewer to create the key, and that the "working directory" for the shortcut wasn't set, or was set to the desktop. If you change the viewer's working directory to the UltraVNC directory it might help.

RC19-5 introduced additional plugin handling code to re-initialize the plugin after each connection. The symptoms you are seeing is because RC18 doesn't reset the server after each connection. Switch to RC19-5 and it will work more than once in a row.

See the documentation on the plugin website for where the plugin looks for the keyfile, and in what order it checks them.

Sean
Guest

Now running RC19.5, but still other problems

Post by Guest »

Thanks guys. I also noticed the need for RC19.5. So here's an update:

1) I cleanly uninstalled RC18 and installed RC19.5 (along with the plugin).

2) I connect fine with first connection attempt. Then disconnect

3) I attempt connection a second time and get the following sequence:
- Connection request (I response 'accept')
- Password prompt on client ( I respond with valid password)
- Status window on client shows password accepted
- UltraVNC splash screen shows up on client. Then splash screen goes away.
- Status window on client never goes away, and remote desktop is never displayed on client.
- If I use the servers tray icon "Close VNC", the tray icon goes away, but the process is still running. The only option is to kill the process. (I am not using the service)

Last three entries in log file are:
30/1/2005 11:01 Connection received from 192.168.1.120
30/1/2005 11:02 Client 192.168.1.120 disconnected
30/1/2005 11:02 Connection received from 192.168.1.120

Any suggestion?

Joe Hoffman
Joe Hoffman

Further observables.

Post by Joe Hoffman »

A couple more points:

- I am using the 112MSRC4Plugin_reg.dsm plugin.
- This problem happens even if I don't use any encryption plugin.

This makes me think it's something to do with RC19.5.

Comments?

Thanks

Joe Hoffman
redge
1000
1000
Posts: 6797
Joined: 2004-07-03 17:05
Location: Switzerland - Geneva

Post by redge »

Scovel

i fund the c:\windows\system32\key.txt
but why this name ? and not rc4.key ?

i copy key.txt to my computer host at same folder but i cannot connect from viewer to host with same dsmplugin !

i use default C:\Windows\...
and default C:\Program Files\UltraVNC
UltraVNC 1.0.9.6.1 (built 20110518)
OS Win: xp home + vista business + 7 home
only experienced user, not developer
Joe Hoffman

keygen tool and filename must match

Post by Joe Hoffman »

Change the name of the file to match what is in the plugin configuration page, OR change the plugin configuration page to match the name of the file. key.txt is just the default filename specified in the keygen tool.

Note: There is a field in the keygen config screen to specify the name and path of the generated key file. Edit as you need.

If you can't edit the field, this means (as I understand it) you are using the no-reg version of the plugin. Instead use the reg version of the plugin and then you can edit the fields.
Joe Hoffman

keygen tool and filename must match

Post by Joe Hoffman »

Change the name of the file to match what is in the plugin configuration page, OR change the plugin configuration page to match the name of the file. key.txt is just the default filename specified in the keygen tool.

Note: There is a field in the keygen config screen to specify the name and path of the generated key file. Edit as you need.

If you can't edit the field, this means (as I understand it) you are using the no-reg version of the plugin. Instead use the reg version of the plugin and then you can edit the fields.
scovel
100
100
Posts: 307
Joined: 2004-07-12 11:56
Location: CT, USA
Contact:

Key.txt -> rc4.key

Post by scovel »

Redge, Joe,

Since a user might want to have multiple keys, the key file does get named key.txt. That way it doesn't over-write any key you might already be using. You need to rename it rc4.key. Since the "noreg" plugin has no place to store configuration options, it will always look for rc4.key.

The version of the plugin that uses the registry allows you to change the path and name of the key, and stores that information in the registry.

Sean

P.S. That could be made more clear in the documentation.
redge
1000
1000
Posts: 6797
Joined: 2004-07-03 17:05
Location: Switzerland - Geneva

Post by redge »

scovel

dsmplugin no_reg ver 1.1.4
--- That's work after renaming rc4.txt --> rc4.key --- I can use security too ! Very happy ! :-)
Thanks you a lot for your so nice quick support and good information (february 8 2005)

good help for no english "dummies" can now use security feature dsmplugin ! ;-)

Have a so beautiful nice day ! :D
sorry for my poor english ! :-]
UltraVNC 1.0.9.6.1 (built 20110518)
OS Win: xp home + vista business + 7 home
only experienced user, not developer
Post Reply