[geeks] virtual machine routing

[Joshua Boyd]

On Tue, Oct 04, 2016 at 08:55:30AM -0400, Charles Shannon Hendrix wrote:

> I have two main servers: UNRAID and QNAP.
> 
> Originally the QNAP was to replace the Unraid, but I find it too useful and
> capable to get rid of. The QNAP has hardware accelerated media, so its great
> for actually serving music and video, but I like Unraid for actually running
> most of the virtual machines.
> 
> The QNAP is running a VPN client, which connects via
> privateinternetaccess.com <http://privateinternetaccess.com/>. However, I do
> NOT have it set up as my LANs default
> gateway. Its slower, and some services have to originate where I really am.
> 
> Of course, its relatively easy to point any given machine to the QNAP if I
> want its traffic encrypted, just make it the default gateway instead of the
> main router. The QNAP also has a fancy virtual interface configuration tool
> which supposedly lets you do a lot of creative routing for your VMs and
> containers.
> 
> I tend to be conservative with the QNAP so its performance is dedicated to
> media serving and backup, so the UNRAID server is my playground and heavy
> lifter.
> 
> By default, UNRAID creates virtbr0, assigned to another private LAN
> (192.168.100/24), which is then routed through the hosts br0 interface and
> the
> main gateway. Works great. Port forwarding gives my LAN access to virtualized
> services.
> 
> However, now I want to make some services that use the VPN gateway.
> 
> Since by default, UNRAID puts new VMs on virtbr0 which is routed through the
> main gateway, I created a new one called virtbr1, with a new private LAN.
> 
> That also works great.
> 
> With that in place and tested, I figured it should be easy to have any hosts
> on virtbr1 use my QNAP as the default gateway, but so far this seemingly
> simple task has eluded me.
> 
> Unraid is mostly Linux, so it has all the routing tools, and I have the
> secondary 'private' virtual bridge set up so it is persistent now. Like I
> said, it works great, it just isn't going via the gateway I want it to.
> 
> I understand that by default Linux is going to route virtual machines through
> its own main gateway, I just want those hosted on virtbr1 to go through an
> alternate.

I haven't used unRAID, but I have used KVM based systems with a few
different networking setups.

I would have expected that which gateway a VM uses would be determined
by the gateway setting in the VM, not the virtual bridge that VM is
using.  As I have it set up at home, virtual machines are bridged
straight to ethernet with no host routing. They get their network
configuration from my dhcp server, which could individually set the
gateway seperately for each VM.

It almost sounds like you are trying to have unRAID do its own NATing,
which I would only do (assumming that unRAID is reasonably flexible
like other VM setups) if the host was on wifi (for instance, I do that
for VMs laptops, possibly in addition to local only networking and
bridged to ethernet networking).  If that is the case, then routing
rules would have to be added I guess.  Could they be based on outbound
destination rather than source VM?