I think you're missing my point. If you are, as I infer, using a mobile cellular based network and/or wireless mesh arrangement, you are using a very unusual network (relatively) and you problems are going to be unique.
Whether you can "connect to the modem" locally means <b>nothing</b>. That's just the first hop, and will only help you reach other stations connecting to that same "modem" (wireless router/gateway?).
I suggest you consult with the people responsible for putting in your network and ask about the limits of its technology for full-time streaming. I am guessing they will advise against doing it, at all.
You can ask whether full time uninterrupted videoconferencing, VoIP, and video serving are supported across the network. These have similar usage profiles to trying to maintain long-term VNC connections.
You really should have mentioned this unique arrangement in your original post.