Different limits for Local/Overseas traffic for 3 bandwitch rates using pcq and queue tree
Tuesday, November 27, 20071. Introdution
Let's consider the scenario, when you want to apply different limits for many users to Local and Oversea traffic and don`t want load cpu with a lot of simple queues and you want sell for customers 3 bandwitch rates.
1.12/6 Mbps Latvian Traffic 1Mbps/512Kbps Overseas traffic
2.6/3 Mbps Latvian Traffic 512kbps/256Kbps Overseas traffic
3.4/2 Mbps Latvian Traffic 256Kbps/128Kbps Overseas traffic
Queue trees will limit data rate for the Local country traffic and Oversea traffic In this scenario local country is Latvia. List of all Latvian subnets located at http://www.nic.lv/local.net
2. Configuration
/ ip firewall address-list
add list=12/6 address=192.168.0.2 comment="12/6mbps Local traffic 1mbps/512kbps oversea"
add list=6/3 address=192.168.0.3 comment="6/3mbps Local traffic 512kbps/256kbps oversea"
add list=4/2 address=192.168.0.4 comment="4/2mbps Local traffic 256kbps/128kbps oversea"
add list=Latvia address=159.148.0.0/16 comment="" disabled=no
add list=Latvia address=193.41.195.0/24 comment="" disabled=no
add list=Latvia address=193.41.33.0/24 comment="" disabled=no
add list=Latvia address=193.41.45.0/24 comment="" disabled=no
add list=Latvia address=193.68.64.0/19 comment="" disabled=no
add list=Latvia address=193.108.29.0/24 comment="" disabled=no
add list=Latvia address=193.108.144.0/22 comment="" disabled=no
add list=Latvia address=193.108.185.0/24 comment="" disabled=no
/ ip firewall mangle
add chain=forward action=mark-packet new-packet-mark=LV_DL_6M passthrough=yes \
in-interface=public src-address-list=latvia comment="Default mangle for \
Latvia Download" disabled=no
add chain=forward action=mark-packet new-packet-mark=LV_UL_3M passthrough=yes \
in-interface=local dst-address-list=latvia comment="Default mangle for \
Latvia Upload" disabled=no
add chain=forward action=mark-packet new-packet-mark=OS_DL_512k \
passthrough=yes in-interface=public src-address-list=!latvia \
comment="Default mangle for Oversea Download" disabled=no
add chain=forward action=mark-packet new-packet-mark=OS_UL_256k \
passthrough=yes in-interface=local dst-address-list=!latvia \
comment="Default mangle for Oversea Upload" disabled=no
add chain=forward action=mark-packet new-packet-mark=LV_DL_12M passthrough=no \
in-interface=public src-address-list=latvia dst-address-list=12/6 \
comment="Tarif 12/6 Latvia Download" disabled=no
add chain=forward action=mark-packet new-packet-mark=LV_UL_6M passthrough=no \
in-interface=local src-address-list=12/6 dst-address-list=latvia \
comment="Tarif 12/6 Latvia Upload" disabled=no
add chain=forward action=mark-packet new-packet-mark=OS_DL_1M passthrough=no \
in-interface=public src-address-list=!latvia dst-address-list=12/6 \
comment="Tarif 12/6 Oversea Download" disabled=no
add chain=forward action=mark-packet new-packet-mark=OS_UL_512k passthrough=no \
in-interface=local src-address-list=12/6 dst-address-list=!latvia \
comment="Tarif 12/6 Oversea Upload" disabled=no
add chain=forward action=mark-packet new-packet-mark=LV_DL_4M passthrough=no \
in-interface=public src-address-list=latvia dst-address-list=4/2 \
comment="Tarif 4/2 Latvia Download" disabled=no
add chain=forward action=mark-packet new-packet-mark=LV_UL_2M passthrough=no \
in-interface=local src-address-list=4/2 dst-address-list=latvia \
comment="Tarif 4/2 Latvia Upload" disabled=no
add chain=forward action=mark-packet new-packet-mark=OS_DL_256k passthrough=no \
in-interface=public src-address-list=!latvia dst-address-list=4/2 \
comment="Tarif 4/2 Oversea Download" disabled=no
add chain=forward action=mark-packet new-packet-mark=OS_UL_128k passthrough=no \
in-interface=local src-address-list=4/2 dst-address-list=!latvia \
comment="Tarif 4/2 Oversea Upload" disabled=no
/ queue type
add name="LV_DL_12M" kind=pcq pcq-rate=12000000 pcq-limit=50 \
pcq-classifier=dst-address pcq-total-limit=2000
add name="LV_DL_6M" kind=pcq pcq-rate=6000000 pcq-limit=50 \
pcq-classifier=dst-address pcq-total-limit=2000
add name="LV_DL_4M" kind=pcq pcq-rate=4000000 pcq-limit=50 \
pcq-classifier=dst-address pcq-total-limit=2000
add name="LV_UL_6M" kind=pcq pcq-rate=6000000 pcq-limit=50 \
pcq-classifier=src-address pcq-total-limit=2000
add name="LV_UL_3M" kind=pcq pcq-rate=3000000 pcq-limit=50 \
pcq-classifier=src-address pcq-total-limit=2000
add name="LV_UL_2M" kind=pcq pcq-rate=2000000 pcq-limit=50 \
pcq-classifier=src-address pcq-total-limit=2000
add name="OS_DL_1M" kind=pcq pcq-rate=1000000 pcq-limit=50 \
pcq-classifier=dst-address pcq-total-limit=2000
add name="OS_DL_512k" kind=pcq pcq-rate=512000 pcq-limit=50 \
pcq-classifier=dst-address pcq-total-limit=2000
add name="OS_DL_256k" kind=pcq pcq-rate=256000 pcq-limit=50 \
pcq-classifier=dst-address pcq-total-limit=2000
add name="OS_UL_512k" kind=pcq pcq-rate=512000 pcq-limit=50 \
pcq-classifier=src-address pcq-total-limit=2000
add name="OS_UL_256k" kind=pcq pcq-rate=256000 pcq-limit=50 \
pcq-classifier=src-address pcq-total-limit=2000
add name="OS_UL_128k" kind=pcq pcq-rate=128000 pcq-limit=50 \
pcq-classifier=src-address pcq-total-limit=2000
/ queue tree
add name="LV_DL_12M" parent=local packet-mark=LV_DL_12M limit-at=0 queue=LV_DL_12M priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="LV_DL_6M" parent=local packet-mark=LV_DL_6M limit-at=0 queue=LV_DL_6M priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="LV_UL_2M" parent=public packet-mark=LV_UL_2M limit-at=0 queue=LV_UL_2M priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="LV_DL_4M" parent=local packet-mark=LV_DL_4M limit-at=0 queue=LV_DL_4M priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="LV_UL_3M" parent=public packet-mark=LV_UL_3M limit-at=0 queue=LV_UL_3M priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="LV_UL_6M" parent=public packet-mark=LV_UL_6M limit-at=0 queue=LV_UL_6M priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="OS_DL_1M" parent=local packet-mark=OS_DL_1M limit-at=0 queue=OS_DL_1M priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="OS_DL_256k" parent=local packet-mark=OS_DL_256k limit-at=0 queue=OS_DL_256k priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="OS_DL_512k" parent=local packet-mark=OS_DL_512k limit-at=0 queue=OS_DL_512k priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="OS_UL_128k" parent=public packet-mark=OS_UL_128k limit-at=0 queue=OS_UL_128k priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="OS_UL_256k" parent=public packet-mark=OS_UL_256k limit-at=0 queue=OS_UL_256k priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
add name="OS_UL_512k" parent=public packet-mark=OS_UL_512k limit-at=0 queue=OS_UL_512k priority=8 max-limit=0 burst-limit=0 \
burst-threshold=0 burst-time=0s disabled=no
3. Explanation
In /ip firewall address-list we created 3 lists for bandwitch profiles and list with name Latvia with latvian subnets In /ip firewall mangle we created address list based packet marks for bandwitch profiles. All ips witch is not included in any address list will have second bandwitch profiles rates. In /queue types we created pcq types for bandwitch profiles. In /queue tree we created rules with Local/Overseas packet marks and queue types for bandwitch profiles.
Now customer with ip 192।168.0.2 have 1 profile rates, customer with ip 192.168.0.3 have 2 profile rates, client with ip 192.168.0.4 have 3 profile rates and all others have 2 bandwitch rates, so all traffic will be limited and your cpu power will be free for other tasks.
source = http://wiki.mikrotik.com
How to apply different limits for Local/Overseas traffic
Introduction
Let's consider the scenario, when you want to apply different limit to Local and Oversea traffic. Oversea traffic - traffic that doesn't belong to the Local country traffic.
To distinguish oversea traffic from Local country traffic, we will use 'mangle marks' and 'address-list' features. It will place appropriate marks to the packets to/from the Local country and Oversea networks. Local traffic is 'latvian traffic' in the particular example, list of network numbers belonging to ISPs in Latvia can be extracted from file: http://www.nic.lv/local.net
Note, 'address-list' entries should be replaced with respective addresses, if your router isn't located in Latvia. To find the actual list of network numbers belonging to your country, use Google or any other resources.
Simple queues will limit data rate for the Local country traffic and Oversea traffic.
Quick Start for Impatient
Configuration export from the router:
/ ip firewall address-list
add list=Latvia address=159.148.0.0/16 comment="" disabled=no
add list=Latvia address=193.41.195.0/24 comment="" disabled=no
add list=Latvia address=193.41.33.0/24 comment="" disabled=no
add list=Latvia address=193.41.45.0/24 comment="" disabled=no
add list=Latvia address=193.68.64.0/19 comment="" disabled=no
add list=Latvia address=193.108.29.0/24 comment="" disabled=no
add list=Latvia address=193.108.144.0/22 comment="" disabled=no
add list=Latvia address=193.108.185.0/24 comment="" disabled=no
add list=Latvia address=193.109.211.0/24 comment="" disabled=no
add list=Latvia address=193.109.85.0/24 comment="" disabled=no
add list=Latvia address=193.110.8.0/23 comment="" disabled=no
add list=Latvia address=193.110.164.0/23 comment="" disabled=no
...
add list=Latvia address=193.111.244.0/22 comment="" disabled=no
/ ip firewall mangle
add chain=prerouting src-address=192.168.100.0/24 action=mark-connection \
new-connection-mark="Con Entire Traffic" passthrough=yes \
comment="Mark-connection All Traffic" disabled=no
add chain=prerouting src-address=192.168.100.0/24 connection-mark="Con Entire \
Traffic" dst-address-list=!Latvia action=mark-connection \
new-connection-mark="Con Oversea" passthrough=yes comment="Mark-connection \
Oversea Traffic" disabled=no
add chain=prerouting connection-mark="Con Oversea" action=mark-packet \
new-packet-mark="Oversea traffic" passthrough=no comment="Mark-packet \
Oversea Traffic" disabled=no
add chain=prerouting action=mark-packet new-packet-mark="Local Country Traffic" \
passthrough=no comment="Mark-packet Local Country Traffic" disabled=no
/ queue simple
add name="Oversea" target-addresses=192.168.100.254/32 dst-address=0.0.0.0/0 \
interface=all parent=none packet-marks="Oversea traffic" direction=both \
priority=8 queue=default-small/default-small limit-at=0/0 \
max-limit=256000/256000 total-queue=default-small disabled=no
add name="Local Country" target-addresses=192.168.100.254/32 dst-address=0.0.0.0/0 \
interface=all parent=none packet-marks="Local Country Traffic" direction=both \
priority=8 queue=default-small/default-small limit-at=0/0 \
max-limit=1024000/1024000 total-queue=default-small disabled=no
Explanation
Address-list
First we create Local country address-list, where are placed list of network numbers belonging to ISPs in Latvia (any other country network addresses can be used instead). Full address-list configuration is not included (too many address-list entries), but address-list idea is clear. Networks added to the list 'Latvia':
/ ip firewall address-list
add list=Latvia address=159.148.0.0/16 comment="" disabled=no
add list=Latvia address=193.41.195.0/24 comment="" disabled=no
add list=Latvia address=193.41.33.0/24 comment="" disabled=no
add list=Latvia address=193.41.45.0/24 comment="" disabled=no
add list=Latvia address=193.68.64.0/19 comment="" disabled=no
add list=Latvia address=193.108.29.0/24 comment="" disabled=no
add list=Latvia address=193.108.144.0/22 comment="" disabled=no
add list=Latvia address=193.108.185.0/24 comment="" disabled=no
add list=Latvia address=193.109.211.0/24 comment="" disabled=no
add list=Latvia address=193.109.85.0/24 comment="" disabled=no
add list=Latvia address=193.110.8.0/23 comment="" disabled=no
add list=Latvia address=193.110.164.0/23 comment="" disabled=no
...
add list=Latvia address=193.111.244.0/22 comment="" disabled=no
Note, it's much easier to create/edit such list with Excel or any other similar program.
Mangle
First we add rule to mark connections that belong to local router's subnet (192.168.100.0/24). Second rule marks connections between local subnet and overseas networks. Third rule marks oversea packets and exclude them from mangle table (passtrough=no). Finally, the last rule places packet mark on all packets that belong to Local country traffic.
/ ip firewall mangle
add chain=prerouting src-address=192.168.100.0/24 action=mark-connection \
new-connection-mark="Con Entire Traffic" passthrough=yes \
comment="Mark-connection All Traffic" disabled=no
add chain=prerouting src-address=192.168.100.0/24 connection-mark="Con Entire \
Traffic" dst-address-list=!Latvia action=mark-connection \
new-connection-mark="Con Oversea" passthrough=yes comment="Mark-connection \
Oversea Traffic" disabled=no
add chain=prerouting connection-mark="Con Oversea" action=mark-packet \
new-packet-mark="Oversea traffic" passthrough=no comment="Mark-packet \
Oversea Traffic" disabled=no
add chain=prerouting action=mark-packet new-packet-mark="Local Country Traffic" \
passthrough=no comment="Mark-packet Local Country Traffic" disabled=no
Simple Queue
Queue configuration is quite simple in the particular case. 192.168.100.254 is the local network host. First rule sets limit 256k/256k to Oversea traffic for the particular host. Respectively second simple queue set limit 1M/1M for Local country traffic.
/ queue simple
add name="Oversea" target-addresses=192.168.100.254/32 dst-address=0.0.0.0/0 \
interface=all parent=none packet-marks="Oversea traffic" direction=both \
priority=8 queue=default-small/default-small limit-at=0/0 \
max-limit=256000/256000 total-queue=default-small disabled=no
add name="Local Country" target-addresses=192.168.100.254/32 dst-address=0.0.0.0/0 \
interface=all parent=none packet-marks="Local Country Traffic" direction=both \
priority=8 queue=default-small/default-small limit-at=0/0 \
max-limit=1024000/1024000 total-queue=default-small disabled=no
Per-Traffic Load Balancing
Introduction
Bandwidth management is an essential part of everyday operation for typcial ISPs, businesses, and even everday home users. There are many different types of management tools available to RouterOS users, QOS, rate-limiting, packet-limiting, to name a few.
I personally operate a wireless ISP, in an area that has no other type of convential high-speed internet (ie. cable, fibre, or DSL). Not having access to fibre myself, I am in a situation where the single fastest backbone connection I can get does not provide enough bandwidth for me to have only one connection. As a result of this limitation load-balancing multiple internet backbone connections is very important.
In the past I have used ECMP, persistent per connection styled load-balancing (see http://wiki.mikrotik.com/wiki/Load_Balancing), as well as various other methods. However, I found all of them lacking in various different areas (not load-balancing correctly, broken large HTTP downloads, IM problems, to name a few issues). I then investigated a way to give me more control over my bandwidth while minimizing the potential problems. The end result was a per-traffic type of load-balancing. This tutorial is designed address that one specific area in depth, at a later date, I may expand/add additional info on fail-over, QOS, and other topics.
Functions of RouterOS Used
Main functions of RouterOS used in this tutorial
- Firewall mangle rules
- Firewall address-lists
- Routing
Step 1 - How to break up to traffic
Before you even log onto your RouterOS box, you should have an idea of how you want to divide your traffic, and have an understanding of what traffic can and what traffic can't be broken up.
Here is an example of what you may want to attempt to seperate (in no particular order):
- HTTP traffic (port 80)
- SSL traffic (port 443)
- POP3 traffic (port 110)
- SMTP traffic (port 25)
- P2P traffic (various ports)
- Unknown traffic (various ports)
After making a list of the traffic type, and the ports they operate on, you need to look at the list and decide if it is at all even capable of being forced out a certain internet connection.
Using the above list as an example here is what I came up with:
- HTTP traffic (no problems found yet)
- SSL traffic (some issues, for normal SSL websites and 90% of all software it is not a problem, I'll explain why issues can arise later)
- POP3 traffic (no problems found yet)
- SMTP traffic (no problems found yet)
- P2P traffic (must go out same internet connection as Unknown traffic, I'll explain why this is later)
- Unknown traffic (must go out same internet connection as P2P traffic, I'll explain why this is later)
Now for a quick explanation of why there can be some issues depending on the traffic type. Some website/programs do not play friendly with multiple requests from different IP addresses, this is the reason why ECMP has so many problems. I will provide a simple solution to the rare conditions where load-balancing can't be completed.
Again using the above as an example lets explain where and why issues can arise. SSL - The beauty of a website is that there are seperate requests for different data, ie. loading pictures from 3 different sources would be a request to each respective server. The result is if a we have a website that uses both SSL and HTTP traffic, we know that in most cases the website will just answer the request without care of the originating IP address. However, and this is a special case, if the website/program developer checks where the requests are originating and they find that the IPs are different they may not successfully answer those requests (this can be by design or by accident). I have had 2 cases of this, in both casees it was secured medical websites that were using it as a method of protecting their data.
P2P and Unknown - I am addressing these together because the issue is one and the same. RouterOS doesn't identify P2P based on any single condition, but instead analyzes the packets, this means RouterOS needs time to watch the data before it realizes that is in fact P2P traffic. As a result RouterOS doesn't know the data is P2P until AFTER the connection is made. This is important becuase the only way to send data out a specific internet connection you must know what that traffic is BEFORE the connection is made. So as with P2P traffic, unknown traffic is just that, it is unknown. By marking the unknown traffic though you can control what internet connection is used for both P2P and the left over unknown traffic (very useful).
Step 2 - Setting up the network
Before we get to the real point of this tutorial we need a fictious network we can use as an example.
- Client Computers (172.18.1.0/24)
- Internet Gateways (10.0.1.1/24, 10 0.2.1/24)
- RouterOS IPs (10.0.1.2/24, 10.0.2.2/24)
Assuming that the IPs, default routes, and DNS settings are already in place the following allows users to get internet access.
Create Address-List for permitted use of internet
/ ip firewall address-list
add list="Allowed - Internet" address=172.18.1.0/24 comment="" disabled=no
Create Address-List to bypass load-balancing
/ ip firewall address-list
add list="WAN - 01" address=172.18.1.24/32 comment="" disabled=no
add list="WAN - 02" address=172.18.1.76/32 comment="" disabled=no
Apply 'Masquerading' to the traffic leaving the WAN interfaces
/ ip firewall nat
add chain=srcnat action=masquerade out-interface="WAN - 01" src-address-list="Allowed - Internet" comment="Gateway 10.0.1.1/24" disabled=no
add chain=srcnat action=masquerade out-interface="WAN - 02" src-address-list="Allowed - Internet" comment="Gateway 10.0.2.1/24" disabled=no
Clients should be able to browse the internet, however only one internet connection would be used (the current default route in RouterOS)
Step 3 - Using RouterOS's Mangle tool to mark specific traffic
This following is the necessary RouterOS commands to mark particular traffic for a certain route, we are using the same example for traffic types as in the beginning, HTTP, SSL, POP3, SMTP, P2P, and Uknown
/ ip firewall mangle
add chain=prerouting action=mark-routing new-routing-mark="WAN - 01" src-address-list="WAN - 01" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="WAN - 02" src-address-list="WAN - 02" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="HTTP Traffic" passthrough=no dst-port=80 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SSL Traffic" passthrough=no dst-port=443 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="POP3 Traffic" passthrough=no dst-port=110 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SMTP Traffic" passthrough=no dst-port=25 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="P2P Traffic" passthrough=no p2p=all-p2p comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="Unknown Traffic" passthrough=no comment="" disabled=no
The first two lines provide a method of marking particular clients to bypass the load-balancing and use only a certain internet connection for all of their traffic. The following lines with mark traffic based on the the dst-port, notice how we are not passing though, also notice that we are marking all traffic, even if its not known, this always for a different internet connection to be specified for the Unknown and P2P instead of using the router's defult route. I also mark P2P seperately even though it must go out the same internet connection as Unknown traffic, I do this for a couple reasons, one is that I could easily stop all P2P traffic by disabling a route, and the other is in my RouterOS configuration I use alot of QOS, it it very easy to remember how everything is configured if the QOS mirrors the load-balancing.
So now we are marking traffic for their respective routes, next is to add those actual routes.
[edit] Step 4 - Using the Routing fuctions of RouterOS to force traffic out certain internet connections
This following is the necessary RouterOS commands to provide routes for the marked HTTP, SSL, POP3, SMTP, P2P, and Uknown Traffic
/ ip route
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="WAN - 01" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="WAN - 02" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="HTTP Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="SSL Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="POP3 Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="SMTP Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="P2P Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="Unknown Traffic" comment="" disabled=no
The first two line provide the routes necessary to give the clients that are not being load-balanced via traffic type the correct gateway to the internet. The remaining lines are the routes necessary to provide the appropiate gateway based on traffic type.
Step 5 - Review what you've created
What did you do:
- You intelligently broke up your internet traffic into different types
- You marked that traffic using the Mangle tool of RouterOS
- You created a bypass list to allow certain IP to bypass the Per-Traffic Load Balancing
- You assigned particular routes based on the traffic you marked with Mangle
[edit] Conclusion
What you have have done is very powerful and this tutorial provides just the tip of the iceberg so to say. The traffic types I have listed here are only a small amount of the total and you may want to add others (eg. DNS, terminal services, ICMP, etc). This can be used is a great deal of different ways be it in a small office enviroment (seperating your email usage from your browsing) or a large wireless ISP (for load-balancing). Some tips and final thoughts:
- Think though what you want to achieve before you start
- Pay attention to the type of traffic on each line (is it incoming intensive or outgoing intensive)
- Finally experiment, nothing better then finding better, faster and more intelligent ways to improve your services with little capital costs
Per-Traffic Load Balancing
Introduction
Bandwidth management is an essential part of everyday operation for typcial ISPs, businesses, and even everday home users. There are many different types of management tools available to RouterOS users, QOS, rate-limiting, packet-limiting, to name a few.
I personally operate a wireless ISP, in an area that has no other type of convential high-speed internet (ie. cable, fibre, or DSL). Not having access to fibre myself, I am in a situation where the single fastest backbone connection I can get does not provide enough bandwidth for me to have only one connection. As a result of this limitation load-balancing multiple internet backbone connections is very important.
In the past I have used ECMP, persistent per connection styled load-balancing (see http://wiki.mikrotik.com/wiki/Load_Balancing), as well as various other methods. However, I found all of them lacking in various different areas (not load-balancing correctly, broken large HTTP downloads, IM problems, to name a few issues). I then investigated a way to give me more control over my bandwidth while minimizing the potential problems. The end result was a per-traffic type of load-balancing. This tutorial is designed address that one specific area in depth, at a later date, I may expand/add additional info on fail-over, QOS, and other topics.
Functions of RouterOS Used
Main functions of RouterOS used in this tutorial
- Firewall mangle rules
- Firewall address-lists
- Routing
Step 1 - How to break up to traffic
Before you even log onto your RouterOS box, you should have an idea of how you want to divide your traffic, and have an understanding of what traffic can and what traffic can't be broken up.
Here is an example of what you may want to attempt to seperate (in no particular order):
- HTTP traffic (port 80)
- SSL traffic (port 443)
- POP3 traffic (port 110)
- SMTP traffic (port 25)
- P2P traffic (various ports)
- Unknown traffic (various ports)
After making a list of the traffic type, and the ports they operate on, you need to look at the list and decide if it is at all even capable of being forced out a certain internet connection.
Using the above list as an example here is what I came up with:
- HTTP traffic (no problems found yet)
- SSL traffic (some issues, for normal SSL websites and 90% of all software it is not a problem, I'll explain why issues can arise later)
- POP3 traffic (no problems found yet)
- SMTP traffic (no problems found yet)
- P2P traffic (must go out same internet connection as Unknown traffic, I'll explain why this is later)
- Unknown traffic (must go out same internet connection as P2P traffic, I'll explain why this is later)
Now for a quick explanation of why there can be some issues depending on the traffic type. Some website/programs do not play friendly with multiple requests from different IP addresses, this is the reason why ECMP has so many problems. I will provide a simple solution to the rare conditions where load-balancing can't be completed.
Again using the above as an example lets explain where and why issues can arise. SSL - The beauty of a website is that there are seperate requests for different data, ie. loading pictures from 3 different sources would be a request to each respective server. The result is if a we have a website that uses both SSL and HTTP traffic, we know that in most cases the website will just answer the request without care of the originating IP address. However, and this is a special case, if the website/program developer checks where the requests are originating and they find that the IPs are different they may not successfully answer those requests (this can be by design or by accident). I have had 2 cases of this, in both casees it was secured medical websites that were using it as a method of protecting their data.
P2P and Unknown - I am addressing these together because the issue is one and the same. RouterOS doesn't identify P2P based on any single condition, but instead analyzes the packets, this means RouterOS needs time to watch the data before it realizes that is in fact P2P traffic. As a result RouterOS doesn't know the data is P2P until AFTER the connection is made. This is important becuase the only way to send data out a specific internet connection you must know what that traffic is BEFORE the connection is made. So as with P2P traffic, unknown traffic is just that, it is unknown. By marking the unknown traffic though you can control what internet connection is used for both P2P and the left over unknown traffic (very useful).
Step 2 - Setting up the network
Before we get to the real point of this tutorial we need a fictious network we can use as an example.
- Client Computers (172.18.1.0/24)
- Internet Gateways (10.0.1.1/24, 10 0.2.1/24)
- RouterOS IPs (10.0.1.2/24, 10.0.2.2/24)
Assuming that the IPs, default routes, and DNS settings are already in place the following allows users to get internet access.
Create Address-List for permitted use of internet
/ ip firewall address-list
add list="Allowed - Internet" address=172.18.1.0/24 comment="" disabled=no
Create Address-List to bypass load-balancing
/ ip firewall address-list
add list="WAN - 01" address=172.18.1.24/32 comment="" disabled=no
add list="WAN - 02" address=172.18.1.76/32 comment="" disabled=no
Apply 'Masquerading' to the traffic leaving the WAN interfaces
/ ip firewall nat
add chain=srcnat action=masquerade out-interface="WAN - 01" src-address-list="Allowed - Internet" comment="Gateway 10.0.1.1/24" disabled=no
add chain=srcnat action=masquerade out-interface="WAN - 02" src-address-list="Allowed - Internet" comment="Gateway 10.0.2.1/24" disabled=no
Clients should be able to browse the internet, however only one internet connection would be used (the current default route in RouterOS)
Step 3 - Using RouterOS's Mangle tool to mark specific traffic
This following is the necessary RouterOS commands to mark particular traffic for a certain route, we are using the same example for traffic types as in the beginning, HTTP, SSL, POP3, SMTP, P2P, and Uknown
/ ip firewall mangle
add chain=prerouting action=mark-routing new-routing-mark="WAN - 01" src-address-list="WAN - 01" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="WAN - 02" src-address-list="WAN - 02" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="HTTP Traffic" passthrough=no dst-port=80 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SSL Traffic" passthrough=no dst-port=443 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="POP3 Traffic" passthrough=no dst-port=110 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SMTP Traffic" passthrough=no dst-port=25 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="P2P Traffic" passthrough=no p2p=all-p2p comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="Unknown Traffic" passthrough=no comment="" disabled=no
The first two lines provide a method of marking particular clients to bypass the load-balancing and use only a certain internet connection for all of their traffic. The following lines with mark traffic based on the the dst-port, notice how we are not passing though, also notice that we are marking all traffic, even if its not known, this always for a different internet connection to be specified for the Unknown and P2P instead of using the router's defult route. I also mark P2P seperately even though it must go out the same internet connection as Unknown traffic, I do this for a couple reasons, one is that I could easily stop all P2P traffic by disabling a route, and the other is in my RouterOS configuration I use alot of QOS, it it very easy to remember how everything is configured if the QOS mirrors the load-balancing.
So now we are marking traffic for their respective routes, next is to add those actual routes.
Step 4 - Using the Routing fuctions of RouterOS to force traffic out certain internet connections
This following is the necessary RouterOS commands to provide routes for the marked HTTP, SSL, POP3, SMTP, P2P, and Uknown Traffic
/ ip route
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="WAN - 01" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="WAN - 02" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="HTTP Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="SSL Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="POP3 Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="SMTP Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="P2P Traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="Unknown Traffic" comment="" disabled=no
The first two line provide the routes necessary to give the clients that are not being load-balanced via traffic type the correct gateway to the internet. The remaining lines are the routes necessary to provide the appropiate gateway based on traffic type.
Step 5 - Review what you've created
What did you do:
- You intelligently broke up your internet traffic into different types
- You marked that traffic using the Mangle tool of RouterOS
- You created a bypass list to allow certain IP to bypass the Per-Traffic Load Balancing
- You assigned particular routes based on the traffic you marked with Mangle
Conclusion
What you have have done is very powerful and this tutorial provides just the tip of the iceberg so to say. The traffic types I have listed here are only a small amount of the total and you may want to add others (eg. DNS, terminal services, ICMP, etc). This can be used is a great deal of different ways be it in a small office enviroment (seperating your email usage from your browsing) or a large wireless ISP (for load-balancing). Some tips and final thoughts:
- Think though what you want to achieve before you start
- Pay attention to the type of traffic on each line (is it incoming intensive or outgoing intensive)
- Finally experiment, nothing better then finding better, faster and more intelligent ways to improve your services with little capital costs
Load Balancing
With NAT and policy routing, with working HTTPS, IM and large downloads
Consider the following network layout:
Quick Start for Impatient
Configuration export from the gateway router:
/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local comment="" \
disabled=no
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan2 \
comment="" disabled=no
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan1 \
comment="" disabled=no
/ ip firewall mangle
add chain=prerouting in-interface=Local connection-state=new nth=1,1,0 \
action=mark-connection new-connection-mark=odd passthrough=yes comment="" \
disabled=no
add chain=prerouting in-interface=Local connection-mark=odd action=mark-routing \
new-routing-mark=odd passthrough=no comment="" disabled=no
add chain=prerouting in-interface=Local connection-state=new nth=1,1,1 \
action=mark-connection new-connection-mark=even passthrough=yes comment="" \
disabled=no
add chain=prerouting in-interface=Local connection-mark=even action=mark-routing \
new-routing-mark=even passthrough=no comment="" disabled=no
/ ip firewall nat
add chain=srcnat connection-mark=odd action=src-nat to-addresses=10.111.0.2 \
to-ports=0-65535 comment="" disabled=no
add chain=srcnat connection-mark=even action=src-nat to-addresses=10.112.0.2 \
to-ports=0-65535 comment="" disabled=no
/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 routing-mark=odd \
comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 routing-mark=even \
comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 comment="" \
disabled=no
Explanation
First we give a code snippet and then explain what it actually does.
Mangle
/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local comment="" \
disabled=no
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan2 \
comment="" disabled=no
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan1 \
comment="" disabled=no
The router has two upstream (WAN) interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24. The LAN interface has the name "Local" and IP address of 192.168.0.1/24.
/ ip firewall mangle
add chain=prerouting in-interface=Local connection-state=new nth=1,1,0 \
action=mark-connection new-connection-mark=odd passthrough=yes comment="" \
disabled=no
First we take every second packet that establishes new session (note connection-state=new), and mark it with connection mark "odd". Consequently all successive packets belonging to the same session will carry the connection mark "odd". Note that we are passing these packets to the second rule (passthrough=yes) to place a routing mark on these packets in addition to the connection mark.
add chain=prerouting in-interface=Local connection-mark=odd action=mark-routing \
new-routing-mark=odd passthrough=no comment="" disabled=no
The rule above places the routing mark "odd" on all packets that belong to the "odd" connection and stops processing all other mangle in prerouting chain rules for these packets.
add chain=prerouting in-interface=Local connection-state=new nth=1,1,1 \
action=mark-connection new-connection-mark=even passthrough=yes comment="" \
disabled=no
add chain=prerouting in-interface=Local connection-mark=even action=mark-routing \
new-routing-mark=even passthrough=no comment="" disabled=no
These rules do the same for the remaining half of the traffic as the first two rules for the first half of the traffic.
The code above effectively means that each new connection initiated through the router from the local network will be marked as either "odd" or "even" with both routing and connection marks.
NAT
/ ip firewall nat
add chain=srcnat connection-mark=odd action=src-nat to-addresses=10.111.0.2 \
to-ports=0-65535 comment="" disabled=no
add chain=srcnat connection-mark=even action=src-nat to-addresses=10.112.0.2 \
to-ports=0-65535 comment="" disabled=no
All traffic marked "odd" is being NATted to source IP address of 10.111.0.2, while traffic marked "even" gets "10.112.0.2" source IP address.
Routing
/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 routing-mark=odd \
comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 routing-mark=even \
comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 comment="" \
disabled=no comment="gateway for the router itself"
For all traffic marked "odd" (consequently having 10।111.0.2 translated source address) we use 10.111.0.1 gateway. In the same manner all traffic marked "even" is routed through the 10.112.0.1 gateway. Finally, we have one additional entry specifying that traffic from the router itself (the traffic without any routing marks) should go to 10.112.0.1 gateway.
सोर्स = ह्त्त्प://विकी.मिक्रोतिक.कॉम
PCQ Examples
Per Connection Queue (PCQ) is a queuing discipline that can be used to dynamically equalize or shape traffic for multiple users, using little administration.
Equal Bandwidth for a Number of Users
Use PCQ type queue when you need to equalize the bandwidth [and set max limit] for a number of users. We will set the 64kbps download and 32kbps upload limits.
There are two ways how to make this: using mangle and queue trees, or, using simple queues.
1. Mark all packets with packet-mark all:
/ip firewall mangle add chain=prerouting action=mark-packet new-packet-mark=all passthrough=no
2. Setup two PCQ queue types - one for download and one for upload. dst-address is classifier for user's download traffic, src-address for upload traffic:
/queue type add name="PCQ_download" kind=pcq pcq-rate=64000 pcq-classifier=dst-address
/queue type add name="PCQ_upload" kind=pcq pcq-rate=32000 pcq-classifier=src-address
3. Finally, two queue rules are required, one for download and one for upload:
/queue tree add parent=global-in queue=PCQ_download packet-mark=all
/queue tree add parent=global-out queue=PCQ_upload packet-mark=all
If you don't like using mangle and queue trees, you can skip step 1, do step 2, and step 3 would be to create one simple queue as shown here:
/queue simple add queue=PCQ_upload/PCQ_download target-addresses=192.168.0.0/२४
सोर्स = ह्त्त्प://विकी.मिक्रोतिक.कॉम
TransparentTrafficShaper
This example shows how to configure a transparent traffic shaper. The transparent traffic shaper is essentially a bridge that is able to differentiate and prioritize traffic that passes through it.
Consider the following network layout:
We will configure one queue limiting the total throughput to the client and three sub-queues that limit HTTP, P2P and all other traffic separately.
Configuration snippet from the MikroTik router:
/ interface bridge
add name="bridge1"
/ interface bridge port
add interface=ether2 bridge=bridge1
add interface=ether3 bridge=bridge1
/ ip firewall mangle
add chain=prerouting protocol=tcp dst-port=80 action=mark-connection \
new-connection-mark=http_conn passthrough=yes
add chain=prerouting connection-mark=http_conn action=mark-packet \
new-packet-mark=http passthrough=no
add chain=prerouting p2p=all-p2p action=mark-connection \
new-connection-mark=p2p_conn passthrough=yes
add chain=prerouting connection-mark=p2p_conn action=mark-packet \
new-packet-mark=p2p passthrough=no
add chain=prerouting action=mark-connection new-connection-mark=other_conn \
passthrough=yes
add chain=prerouting connection-mark=other_conn action=mark-packet \
new-packet-mark=other passthrough=no
/ queue simple
add name="main" target-addresses=10.0.0.12/32 max-limit=256000/512000
add name="http" parent=main packet-marks=http max-limit=240000/500000
add name="p2p" parent=main packet-marks=p2p max-limit=64000/64000
add name="other" parent=main packet-marks=other max-limit=128000/128000
Each piece of code is followed by the explanation of what it actually does.
/ interface bridge
add name="bridge1"
/ interface bridge port
add interface=ether2 bridge=bridge1
add interface=ether3 bridge=bridge1
We create a new bridge interface and assign two ethernet interfaces to it. Thus the prospective traffic shaper will be completely transparent to the client.
/ ip firewall mangle
add chain=prerouting protocol=tcp dst-port=80 action=mark-connection \
new-connection-mark=http_conn passthrough=yes
add chain=prerouting connection-mark=http_conn action=mark-packet \
new-packet-mark=http passthrough=no
All traffic destined to TCP port 80 is likely to be HTTP traffic and therefore is being marked with the packet mark http. Note, that the first rule has passthrough=yes while the second one has passthrough=no. (You can obtain additional information about mangle at http://www.mikrotik.com/docs/ros/2.9/ip/mangle)
/ ip firewall mangle
add chain=prerouting p2p=all-p2p action=mark-connection \
new-connection-mark=p2p_conn passthrough=yes
add chain=prerouting connection-mark=p2p_conn action=mark-packet \
new-packet-mark=p2p passthrough=no
add chain=prerouting action=mark-connection new-connection-mark=other_conn \
passthrough=yes
add chain=prerouting connection-mark=other_conn action=mark-packet \
new-packet-mark=other passthrough=no
Same as above, P2P traffic is marked with the packet mark p2p and all other traffic is marked with the packet mark other.
/ queue simple
add name="main" target-addresses=10.0.0.12/32 max-limit=256000/512000
We create a queue that limits all the traffic going to/from the client (specified by the target-address) to 256k/512k.
/ queue simple
add name="http" parent=main packet-marks=http max-limit=240000/500000
add name="p2p" parent=main packet-marks=p2p max-limit=64000/64000
add name="other" parent=main packet-marks=other max-limit=128000/128000
All sub-queues have the main queue as the parent, thus the aggregate data rate could not exceed limits specified in the main queue. Note, that http queue has higher priority than other queues, meaning that HTTP downloads are prioritized।
सोर्स=ह्त्त्प://विकी.मिक्रोतिकी.com
14 Step Configuration of mikrotik using ADSL speedy internet
This articel i explain how to configuration mikrotik using pc for router।
1. Configuration position of ADSL connection
modem —Mikrotik—-Switch —– Client
Client
a. Modem IP –>> 192.168.1.1
b. Mikrotik Using 2 Interface - Ether1 —->> 192.168.1.2
- Ether1 —->> 192.168.0.254
c. Switch — Client ip 192.168.0.1-253
2. Configuration step router pc mikrotik
a. add ip address to interface 1
/ip address add interface=ether1 address = 192.168.1.1 netmask= 255.255.255.0
b. add ip address to interface2
/ip address add interface=ether2 address = 192.168.0.1 netmask= 255.255.255.0
c. to see input of interface
/interface print
d. Set Value name off interface
/inteface set 0 name=”Public” –>> For Modem
/inteface set 1 name=”Lan” –>> For Lan
e. set routing for net ( modem )
/ip route add gateway=192.168.0.1
f. set dns server ISP
/ip dns set primary-dns=203.130.193.74 secondary-dns=202.134.0.155
/ip dns set allow-remote-requests=yes
g. set of nat the router pc to sharing bw to lan
/ip firewall nat add chain=srcnat out-inteface=Public action=masquerade
