Load Balancing W/linux Router

[Mr-Protocol]

I happen to have a nice neighbor who has allowed the world to use his access point. I have flashed a WRT54G router with DD-WRT to act as a repeater bridge.

I am going to attempt to combine my service with his to see how it improves. When I feel up for demolishing my network and setting it up in this configuration.

Routing for multiple uplinks/providers
A common configuration is the following, in which there are two  
providers that connect a  local network (or even a single 
machine) to the big Internet. 

                          ISP1
Localnetwork -- LinuxBox <         > Internet
                          ISP2


There are usually two questions given this setup. 

4.2.1. Split access
The first is how to route answers to packets coming in over a 
particular provider, say Provider 1, back out again over that 
same provider. 

Let us first set some symbolical names. Let $IF1 be the name of 
the first interface (if1 in the picture above) and $IF2 the name 
of the second interface. Then let $IP1 be the IP address 
associated with $IF1 and $IP2 the IP address associated with 
$IF2. Next, let $P1 be the IP address of the gateway at Provider 
1, and $P2 the IP address of the gateway at provider 2. Finally, 
let $P1_NET be the IP network $P1 is in, and $P2_NET the IP 
network $P2 is in. 

One creates two additional routing tables, say T1 and T2. These 
are added in /etc/iproute2/rt_tables. Then you set up routing in 
these tables as follows: 

      ip route add $P1_NET dev $IF1 src $IP1 table T1
      ip route add default via $P1 table T1
      ip route add $P2_NET dev $IF2 src $IP2 table T2
      ip route add default via $P2 table T2
    

Nothing spectacular, just build a route to the gateway and build 
a default route via that gateway, as you would do in the case of 
a single upstream provider, but put the routes in a separate 
table per provider. Note that the network route suffices, as it 
tells you how to find any host in that network, which includes 
the gateway, as specified above. 

Next you set up the main routing table. It is a good idea to 
route things to the direct neighbour through the interface 
connected to that neighbour. Note the `src' arguments, they make 
sure the right outgoing IP address is chosen.         

            ip route add $P1_NET dev $IF1 src $IP1
        ip route add $P2_NET dev $IF2 src $IP2
      

Then, your  preference for default route:         

             ip route add default via $P1
      

Next, you set up the routing rules. These actually choose what 
routing table to route with. You want to make sure that you 
route out a given interface if you already have the  
corresponding source address:         

            ip rule add from $IP1 table T1
        ip rule add from $IP2 table T2
      

This set of commands makes sure all answers to traffic coming in 
on a particular interface get answered from that interface. 

Now, this is just the very basic setup. It will work for all 
processes running on the router itself, and for the local 
network, if it is masqueraded. If it is not, then you either have 
IP space from both providers or you are going to want to 
masquerade to one of the two providers. In both cases you will 
want to add rules selecting which provider to route out from 
based on the IP address of the machine in the local network. 

4.2.2. Load balancing
The second question is how to balance traffic going out over the 
two providers. This is actually not hard if you already have set 
up split access as above. 

Instead of choosing one of the two providers as your default 
route, you now set up the default route to be a multipath route. 
In the default kernel this will balance routes over 
the two providers. It is done as follows (once more building on 
the example in the section on split-access):         
   
ip route add default scope global nexthop via $P1 dev $IF1 weight 1 \
        nexthop via $P2 dev $IF2 weight 1
      

This will balance the routes over both providers. The weight 
parameters can be tweaked to favor one provider over the other. 

Note that balancing will not be perfect, as it is route based, 
and routes are cached. This means that routes to often-used 
sites will always be over the same provider

Side note: Not sure why this post ended up in Questions. Was supposed to be in "Everything Else"

Edited July 8, 2010 by Mr-Protocol

[Infiltrator]

It looks complex to me, but I like the idea.

[Mr-Protocol]

The concept is simple. To make a connection a route needs to be made. If in my case the first ISP is about double the speed of the second one. So i will weight 2/1 the routes. So it will put twice as many routes on the faster as the slower one.

[Mr-Protocol]

I installed PFsense on my old desktop. I gave this doc a try and the network seemed to run SLOOOOOOOOOW. The exact Opposite of what I wanted...

Web browsing took way too long to load pages.

http://doc.pfsense.org/index.php/Multi_WAN_/_Load_Balancing

Any suggestions?

Edited July 10, 2010 by Mr-Protocol

[Infiltrator]

Can you tell if load is really getting split across the modems. How would you go about testing that out?

[Mr-Protocol]

only way to test would be to run a packet sniffer on the pfsense box or after it to see.. but the thing was i was web browsing so slow i felt like i was on dialup. Not sure wtf went wrong. I followed that guide as best i could. Has anyone done something like this before?