winvnc.exe crashes with two monitors, analysis & patch

Developers may discuss here.

winvnc.exe crashes with two monitors, analysis & patch

Postby hsmeier » 2017-03-02 10:07

Hi everybody,

we have

- ultravnc code 1068
- server machine is win7
- two monitors, secondary monitor ABOVE primary monitor
- both monitors 1920x1080
- mirror driver on

As soon as a viewer connects, winvnc crashes with an access violation in vncbuffer.cpp:803

memcpy(&pScaled[0], &pMain[0], nBytesPerPixel*(ScaledRect.br.x - ScaledRect.tl.x));

where pMain causes the access violation. This is a result of

void vncBuffer::ScaleRect(rfb::Rect &rect)

beeing called wit a rect with tl/br = 0,-1080,1920,2160 (see negative value of tl.y) which finally results in reading with a negative offset of 1080*m_bytesPerRow from m_mainbuff.

After some investigation I located the source of the problem and I think I fixed it (hunk 3 of patch below). Additionally I modified things according to hunk 2 because this looks like causing similar trouble in some different situation - but maybe this is wrong. After that, the crash disappeared, but when viewing both monitors with the viewer, the primary monitor (below) stayed grey. Mouse already moved correctly. This finally could be fixed by hunk 1 of the patch below. The original code returned a size of 3840x1080 instead of 1920x2160 as size of the combined screen.

Well, I am not sure whether this patch really is correct for all scenarios. Here, it works with 1 monitor and with 2 monitors in all 4 possible arrangements. So please review it!

Thanks,
Best regards
Hans

Code: Select all
--- ultravnc-code-1068-orig\ultravnc-code-1068\UltraVNC Project Root\UltraVNC\winvnc\winvnc\vncDesktopSW.cpp   2013-10-10 20:36:06.000000000 +0200
+++ ultravnc-code-1068\ultravnc-code-1068\UltraVNC Project Root\UltraVNC\winvnc\winvnc\vncDesktopSW.cpp   2017-03-02 10:01:10.918616800 +0100
@@ -185,8 +185,8 @@
            switch (nr_monitors) {
            case 2:
               {
-               nWidth=mymonitor[0].Width+mymonitor[1].Width;
-               nHeight=max(mymonitor[0].Height, mymonitor[1].Height);
+                  nWidth=m_Cliprect.br.x;
+                  nHeight=m_Cliprect.br.y;
               } break;
            case 3:
               {
@@ -262,8 +262,8 @@
   }
   else
   {
-   m_SWOffsetx=0;
-   m_SWOffsety=0;
+   m_SWOffsetx=m_bmrect.tl.x;
+   m_SWOffsety=m_bmrect.tl.y;
   m_Cliprect.tl.x=0;
   m_Cliprect.tl.y=0;
   m_Cliprect.br.x=m_bmrect.br.x;
@@ -276,8 +276,8 @@
   {   
   m_SWOffsetx=m_bmrect.tl.x;
   m_SWOffsety=m_bmrect.tl.y;
-   m_Cliprect.tl.x=m_bmrect.tl.x;
-   m_Cliprect.tl.y=m_bmrect.tl.y;
+   m_Cliprect.tl.x=0;
+   m_Cliprect.tl.y=0;
   m_Cliprect.br.x=m_bmrect.br.x;
   m_Cliprect.br.y=m_bmrect.br.y;

hsmeier
 
Posts: 3
Joined: 2014-03-25 08:33

Re: winvnc.exe crashes with two monitors, analysis & patch

Postby Rudi De Vos » 2017-03-06 18:57

Thanks, i will take a look at it...
Lang time ago someone contributed a patch.

Forum is current getting a lot of spam... loosing a lot of time on checking posts.
If your post get delete by accident ( sometimes i just need to select all and delete to see the end) please resend.
Rudi De Vos
Admin & Developer
Admin & Developer
 
Posts: 5350
Joined: 2004-04-23 10:21


Return to Developer discussions (mainly user-mode)

Who is online

Users browsing this forum: No registered users and 1 guest

cron