Serveurs dédiés - Additional IPs on vRack and Virtual Machines
... / Additional IPs on vRack a...
BMPCreated with Sketch.BMPZIPCreated with Sketch.ZIPXLSCreated with Sketch.XLSTXTCreated with Sketch.TXTPPTCreated with Sketch.PPTPNGCreated with Sketch.PNGPDFCreated with Sketch.PDFJPGCreated with Sketch.JPGGIFCreated with Sketch.GIFDOCCreated with Sketch.DOC Error Created with Sketch.
Frage

Additional IPs on vRack and Virtual Machines

Von
Apex
Erstellungsdatum 2024-05-29 07:01:58 (edited on 2024-09-04 14:02:39) in Serveurs dédiés

Hi everyone !
Need help about Networking and VLAN as I can't ping the Gateway for Additional IP block from my VM nor reach the public IP from my laptop.

**Products:**
- 4 dedicated servers Advance-1 with each having 2 NICs for Proxmox cluster
- Additional IP, initially tied to one of the the DS (let's say DS-1), and actually the block is tied to the vRack.

**Goals:**
Here is the configuration I am trying to achieve :
- Create a hyperconverged proxmox cluster ( done )
- Deploy a Kubernetes cluster on the VMs managed by Proxmox ( not yet done )
- Having cilium (CNI) to manage a Load Balancer using one of my IP adress from the Additional IP block allowing my application to be be reachable from the Internet ( not yet done )
- Using ceph-csi (CSI) for k8s volumes

**Actual Configurations:**
Below is the description of my Proxmox cluster configuration:
- All of the Dedicated servers are tied into the vRack
- VLAN1 on the private NIC for Ceph cluster
- VLAN2 on the private NIC for Ceph public network and VM network via *vmbr20 bridge*
- Dedicated Servers Public NICs for VM migration and Proxmox cluster network
- Created a *bridge vmbr1* attached to some VMs acting as VM gateways to the Internet using public IP address from the Additional IP block (those VMs are also attached to vmbr20)

Actually ***everything is working*** great as expected!

But here is the issue, I can't use or don't known how to configure one of the Additional IP as a secondary IP of the VM NIC (let's say AdditionalIP-3 on nic0) that is attached to vmbr20 ( not using vmbr1 here )
I am not using vmbr1 here for testing purpose as my k8s nodes (VMs) will not use it and so do the VM that will be hosting the LB managed by cillium.

*Note that: AdditionalIP-1 and AdditionalIP-2 are already in use by the VMs acting as gateway*

Here is the configuration in the VM for the test:

allow-hotplug nic0
auto nic0
iface nic0 inet static
address 192.168.x.111
gateway 192.168.x.254

iface nic0 inet static
address AddionalIP-3
netmask AdditionalIP-Mask
broadcast AdditionalIP-BROADCAST
post-up ip route add AdditionalIP-Network/Mask dev nic0 table vrack
post-up ip route add default via AdditionalIP-GW dev nic0 table vrack
post-up ip rule add from AdditionalIP-Network/Mask table vrack
post-up ip rule add to AdditionalIP-Network/Mask table vrack

NIC configuration on the Host:

iface enPub inet manual
iface enPriv inet manual

#VLAN For VMs and Ceph public network
auto enPriv.2
iface enPriv.2 inet manual
auto vmbr20
iface vmbr20 inet static
address 192.168.x.xx/24
bridge-ports enPriv.2
bridge-stp off
bridge-fd

#VLAN For Ceph Cluster
auto enPriv.10
iface enPriv.10 inet static
address 192.168.y.yy/24

#IFACE for additional IP
auto vmbr1
iface vmbr1 inet manual
bridge-ports enPriv (just the DS NIC name without .vlanId )
bridge-stp off
bridge-fd 0

# Dedicated Servers NIC
auto vmbr0
iface vmbr0 inet static
address DS-Pub-IP/24
gateway DS-Pub-GW
bridge-ports enPub
bridge-stp off
bridge-fd 0
hwaddress NicMAC

**Test results:**
1. Ping on *AddionalIP-1* and *AddionalIP-2* from this VM having AddionalIP-3 **succeeds**.
2. Ping to *AddionalIP-GW* from the VM failed with: `From 192.168.x.111 icmp_seq=2 Destination Host Unreachable`
3. Ping or curl to AddionalIP-3 from external network (my laptop) failed ( no response )
4. traceroute from the VM to *AddionalIP-GW* show the following:

traceroute to AddionalIP-GW (AddionalIP-GW), 30 hops max, 60 byte packets
1 192.168.x.111 (192.168.x.111) 3066.148 ms !H 3066.133 ms !H 3066.125 ms !H

Here are my routing tables:

root@test:~# ip rule show table vrack
32764: from all to AdditionalIP-Network/Mask lookup vrack
32765: from AdditionalIP-Network/Mask lookup vrack

root@test:~# ip route show table vrack
default via AdditionalIP-GW dev nic0
AdditionalIP-Network/Mask dev nic0 scope link

root@test:~# ip route
default via VIP-VM-Gateway dev nic0 onlink
AdditionalIP-Network/Mask dev nic0 proto kernel scope link src AddionalIP-3
AdditionalIP-Network/Mask dev nic0 scope link src 192.168.x.111

**My Questions:**
Is it only possible to configure *nic0* (VM interface) that is attached to *vmbr20* (Proxmox) to have one additional IP configured on it next to the LAN IP (192.168.x.111) for it to be reachable from the Internet ? (Given that the LAN IP of this interface is tied to the VLAN2)
Or hopefully, Am I just missing something ?

Note:
- DS-1 is actually down

Many thanks in advance !!

*PS: I am far from being a network expert*

Edit:
- 1: ++More context and typo