Learning All About Router

Introduction

Bandwidth management is an essential part of everyday operation for typcial ISPs, businesses, and even everday home users. There are many different types of management tools available to RouterOS users, QOS, rate-limiting, packet-limiting, to name a few.

I personally operate a wireless ISP, in an area that has no other type of convential high-speed internet (ie. cable, fibre, or DSL). Not having access to fibre myself, I am in a situation where the single fastest backbone connection I can get does not provide enough bandwidth for me to have only one connection. As a result of this limitation load-balancing multiple internet backbone connections is very important.

In the past I have used ECMP, persistent per connection styled load-balancing (see http://wiki.mikrotik.com/wiki/Load_Balancing), as well as various other methods. However, I found all of them lacking in various different areas (not load-balancing correctly, broken large HTTP downloads, IM problems, to name a few issues). I then investigated a way to give me more control over my bandwidth while minimizing the potential problems. The end result was a per-traffic type of load-balancing. This tutorial is designed address that one specific area in depth, at a later date, I may expand/add additional info on fail-over, QOS, and other topics.

Functions of RouterOS Used

Main functions of RouterOS used in this tutorial

• Firewall mangle rules
• Firewall address-lists
• Routing

Step 1 - How to break up to traffic

Before you even log onto your RouterOS box, you should have an idea of how you want to divide your traffic, and have an understanding of what traffic can and what traffic can't be broken up.

Here is an example of what you may want to attempt to seperate (in no particular order):

• HTTP traffic (port 80)
• SSL traffic (port 443)
• POP3 traffic (port 110)
• SMTP traffic (port 25)
• P2P traffic (various ports)
• Unknown traffic (various ports)

After making a list of the traffic type, and the ports they operate on, you need to look at the list and decide if it is at all even capable of being forced out a certain internet connection.

Using the above list as an example here is what I came up with:

• HTTP traffic (no problems found yet)
• SSL traffic (some issues, for normal SSL websites and 90% of all software it is not a problem, I'll explain why issues can arise later)
• POP3 traffic (no problems found yet)
• SMTP traffic (no problems found yet)
• P2P traffic (must go out same internet connection as Unknown traffic, I'll explain why this is later)
• Unknown traffic (must go out same internet connection as P2P traffic, I'll explain why this is later)

Now for a quick explanation of why there can be some issues depending on the traffic type. Some website/programs do not play friendly with multiple requests from different IP addresses, this is the reason why ECMP has so many problems. I will provide a simple solution to the rare conditions where load-balancing can't be completed.

Again using the above as an example lets explain where and why issues can arise. SSL - The beauty of a website is that there are seperate requests for different data, ie. loading pictures from 3 different sources would be a request to each respective server. The result is if a we have a website that uses both SSL and HTTP traffic, we know that in most cases the website will just answer the request without care of the originating IP address. However, and this is a special case, if the website/program developer checks where the requests are originating and they find that the IPs are different they may not successfully answer those requests (this can be by design or by accident). I have had 2 cases of this, in both casees it was secured medical websites that were using it as a method of protecting their data.

P2P and Unknown - I am addressing these together because the issue is one and the same. RouterOS doesn't identify P2P based on any single condition, but instead analyzes the packets, this means RouterOS needs time to watch the data before it realizes that is in fact P2P traffic. As a result RouterOS doesn't know the data is P2P until AFTER the connection is made. This is important becuase the only way to send data out a specific internet connection you must know what that traffic is BEFORE the connection is made. So as with P2P traffic, unknown traffic is just that, it is unknown. By marking the unknown traffic though you can control what internet connection is used for both P2P and the left over unknown traffic (very useful).

Step 2 - Setting up the network

Before we get to the real point of this tutorial we need a fictious network we can use as an example.

• Client Computers (172.18.1.0/24)
• Internet Gateways (10.0.1.1/24, 10 0.2.1/24)
• RouterOS IPs (10.0.1.2/24, 10.0.2.2/24)

Assuming that the IPs, default routes, and DNS settings are already in place the following allows users to get internet access.

Create Address-List for permitted use of internet

/ ip firewall address-list
add list="Allowed - Internet" address=172.18.1.0/24 comment="" disabled=no

Create Address-List to bypass load-balancing

/ ip firewall address-list
add list="WAN - 01" address=172.18.1.24/32 comment="" disabled=no
add list="WAN - 02" address=172.18.1.76/32 comment="" disabled=no

Apply 'Masquerading' to the traffic leaving the WAN interfaces

/ ip firewall nat
add chain=srcnat action=masquerade out-interface="WAN - 01" src-address-list="Allowed - Internet" comment="Gateway 10.0.1.1/24" disabled=no
add chain=srcnat action=masquerade out-interface="WAN - 02" src-address-list="Allowed - Internet" comment="Gateway 10.0.2.1/24" disabled=no

Clients should be able to browse the internet, however only one internet connection would be used (the current default route in RouterOS)

Step 3 - Using RouterOS's Mangle tool to mark specific traffic

This following is the necessary RouterOS commands to mark particular traffic for a certain route, we are using the same example for traffic types as in the beginning, HTTP, SSL, POP3, SMTP, P2P, and Uknown

/ ip firewall mangle
add chain=prerouting action=mark-routing new-routing-mark="WAN - 01" src-address-list="WAN - 01" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="WAN - 02" src-address-list="WAN - 02" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="HTTP Traffic" passthrough=no dst-port=80 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SSL Traffic" passthrough=no dst-port=443 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="POP3 Traffic" passthrough=no dst-port=110 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SMTP Traffic" passthrough=no dst-port=25 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="P2P Traffic" passthrough=no p2p=all-p2p comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="Unknown Traffic" passthrough=no comment="" disabled=no

The first two lines provide a method of marking particular clients to bypass the load-balancing and use only a certain internet connection for all of their traffic. The following lines with mark traffic based on the the dst-port, notice how we are not passing though, also notice that we are marking all traffic, even if its not known, this always for a different internet connection to be specified for the Unknown and P2P instead of using the router's defult route. I also mark P2P seperately even though it must go out the same internet connection as Unknown traffic, I do this for a couple reasons, one is that I could easily stop all P2P traffic by disabling a route, and the other is in my RouterOS configuration I use alot of QOS, it it very easy to remember how everything is configured if the QOS mirrors the load-balancing.

So now we are marking traffic for their respective routes, next is to add those actual routes.

[edit] Step 4 - Using the Routing fuctions of RouterOS to force traffic out certain internet connections

This following is the necessary RouterOS commands to provide routes for the marked HTTP, SSL, POP3, SMTP, P2P, and Uknown Traffic

/ ip route
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="WAN - 01" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="WAN - 02" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="HTTP Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="SSL Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="POP3 Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="SMTP Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="P2P Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="Unknown Traffic" comment="" disabled=no

The first two line provide the routes necessary to give the clients that are not being load-balanced via traffic type the correct gateway to the internet. The remaining lines are the routes necessary to provide the appropiate gateway based on traffic type.

Step 5 - Review what you've created

What did you do:

• You intelligently broke up your internet traffic into different types
• You marked that traffic using the Mangle tool of RouterOS
• You created a bypass list to allow certain IP to bypass the Per-Traffic Load Balancing
• You assigned particular routes based on the traffic you marked with Mangle

[edit] Conclusion

What you have have done is very powerful and this tutorial provides just the tip of the iceberg so to say. The traffic types I have listed here are only a small amount of the total and you may want to add others (eg. DNS, terminal services, ICMP, etc). This can be used is a great deal of different ways be it in a small office enviroment (seperating your email usage from your browsing) or a large wireless ISP (for load-balancing). Some tips and final thoughts:

1. Think though what you want to achieve before you start
2. Pay attention to the type of traffic on each line (is it incoming intensive or outgoing intensive)
3. Finally experiment, nothing better then finding better, faster and more intelligent ways to improve your services with little capital costs