Bdtyktl

I have the following setup:

      Hypervisor host 1              |            Hypervisor host 2

                                     |

------------------                   |

| VM             |                   |

| 10.244.140.10  |                   |

| GW 10.244.140.1|                   |    Route to 10.244.8.0/24 

------------------                   |    goes through br1

        |                            |

 /-------------- /-------------    |   /-------------  /--------------

 |     br0      | |     br1     |    |   |     br1     |  |     br0      |

 |10.244.140.254| | 10.244.8.21 |    |   | 10.244.8.1  |  | 10.244.140.1 |

 |              | | ....        |    |   | 10.244.8.33 |  |              |

 --------------/ -------------/    |   -------------/  --------------/

        |                |           |         |                 |

       eth0             eth1------------------eth1              eth0

        |                            |                           |

        --------------------------------------------------------/

Two hosts with virtual machines. Several subnets are defined: 10.244.140.0/24 on the br0 bridges, 10.244.8.0/24 on the br1 bridges. The br1 bridges have several IP addresses. No netfilter rules. Netfilter policies are ACCEPT. ip_forward is set.

A VM is connected to br0 on host 1. It pings 10.244.8.21, an address that is in a different subnet but resides on the same host 1. The VM's default gateway is 10.244.140.1 and resides on host 2.

My naive expectation: The echo request packet is sent to br0 on host2, since br0 has the gateway address. The packet is then routed to br1 and back to host1.

Reality: The packet arrives as expected. However, it is not routed. On the other hand, packets sent to 10.244.8.33 arrive at br1 on host 2, and echo replies go back.

Where am I wrong? And more importantly: How can I fix this?

The hypervisor hosts run OpenSuse Leaf 15 and KVM.

EDIT: Since I don't have enough reputation to add comments, I put answers in the question. Thanks for your concern.

Fang: your packet arrives "as expected", it has passed subnet borders and thus, was routed. Sorry, my description was incomplete. It arrives at br0 on host 2, in the same subnet, and is not routed further.

Grawity: I guess your first question is answered now.
Which destination MAC address does the packet have when it traverses br0 on host1?
The router's MAC, i.e the MAC of br0 on host 2. It knows it because of an ARP for which it does get a response.

Reverse path filtering.

I had known that it existed, but I didn't make the connection to my problem. RPF limits reply packets to be sent out through the receiving interface. in my case, replies would go through a different interface and are therefore blocked.

The solution is to disable reverse path filtering on both hypervisor hosts, for example

# sysctl net.ipv4.conf.all.rp_filter=0

This makes hosts vulnerable to certain network attacks, but my environment is isolated from the big bad internet and I don't care.

For more information, see the rp_filter paragraph on the kernel documentation page for network parmeters.