Mikrotik – QOS Per File

Today’s article will be a bit more complex.  If you played with your Mikrotik enough, you should know how to use QOS to limit connection speeds for different things (if not, maybe I will do a quick article on that).  Well we will build on that today to show you how you can limit HTTP download speeds on particular files or file types (for instance all Zip files).  To make this work, we will be utilizing Queues, Mangle Rules (for Connection and Packet Marking), and Layer7 Protocols.

This is something I currently use to limit the download speed of my CactiEZ CD that I release.  This one download uses an ungodly amount of bandwidth a month, and would use even more if it wasn’t tightly controlled.  The problem comes in that all my downloads were previously QOSed solely by destination IP (the download site as a static IP).  If several users were downloading the CD (560 Megs a pop each) it would slow down and limit the speed of all my other plugins could be downloaded.  So I had to figure out how to separate them out.

So to begin with, we are going to be making a Layer7 regex, which will allow us to determine if the file being downloaded is the particular file we want to limit.   To create this, goto IP >> Firewall and then click on the Layer7 Protocols tab.  Create a new rule that looks something like this.  This one is a simple one that matches the name of my CactiEZ cd.  How to create regex’s is beyond this arctile, but just remember that in the Mikrotik, everything must be lowercase.  This will match the GET request for any file name CactiEZ.  You will probably want to make your rule a bit more complex, as on a normal web server, this could match all sorts of things (forum searches, etc…) but it will work for me since I will also limit this down by IP later.

l7rule1

Now that we have our L7 Rule, we need to create a Mangle rule to Mark the connection.  The reason we mark the connection instead of the packet, is that you should only receive one GET request per download, and just marking the packet would in turn only apply QOS to the GET request, not to the entire download.  So click over to the Mangle tab, and create a new rule.  We will be limiting this to only TCP 80 to the particular IP that are downloads reside on.  This is to limit the amount of CPU required for the Layer7 scanning.  Narrow it down as far as possible because the less traffic that needs to be scanned, the faster everything will run.

mangle1

Now click on the Advanced tab, and we want to select the Layer7 rule we just created.

mangle2

And the next step is to tell it to mark the connection, so click on the Action tab.  You will notice I clicked the Passthrough checkbox.  This is to allow the rule to be processed by the rest of the rules also.  This allows the 1st packet with the GET request to match the next rule we will be setting up which will mark the packet also.  Not completely necessary, but ensures that even the GET request is QOSed.

mangle3

Now that have the connection marked, we need mark every packet in that connection.  This is only necessary since the QOS rules has a limitation that only allow us to use Packet Marks and not Connection Marks to categorize the traffic.  Even though, this is simple enough to do.  Create another Mangle rule, but this time on the General tab only set the “Connection Mark” like so.

mangle-packet1And on the Action tab, tell it to mark the Packet.  Notice the Passthrough is not checked on this one.

mangle-packet2

So now the beginning work is done and every download of the file is marked (you can click over the Connections tab, download the file, and see the actual connections being marked).

The next step is creating the actual rule to do the bandwidth limiting.  For this, we will on Queues on the left hand menu of Winbox.  Create a new Simple Queue and set your Max Limit.  For this particular I limited it to 6M.  On the Advanced tab, you will need to select the Packet Mark we setup in the last step.

queue1

Save that, and you should now be in business.  Attempt to download the file, and you show see the counters increment on the Mangle rules, see a connection marked in the Connections tab, and see the counters increment on the Queue.

From here you could do all sorts of fun stuff.  Another thing I do in particular is limit the download to 1 connection.  I actually allow up to 3, but if you happen to use any more than that (I’ve had people try to hit me with 100 connections) I add you to an address list which drops all but the 1 connection.  If you only use 2-3, I do not block the other connections but instead use connection marking to mark the excess connections into their own queue with only 64K of bandwidth allocated.  This stops people with File Managers from hammering my site with multiple connections (to download the file in many chunks at once) but still allows those excess connections to share a very small pool of bandwidth to stay alive to continue downloading their chunk once the 1st connection has finished.

May 27, 2009 · Jimmy · 2 Comments
Posted in: Mikrotik

2 Responses

  1. Mikrotik - QOS Per File - Web Server Limiting « Inside Greg’s Brain - May 27, 2009

    […] Mikrotik – QOS Per File – Web Server Limiting Jimmy has released another great little article on his personal experience with Mikrotik. His cactiEZ cd gets a LOT of downloads, so this is his method with dealing with such problems. He marks connections using a L7 regex statement, then wrangles those packets into a queue. You can find the article HERE. […]

  2. Tom Waterloo - May 27, 2009

    Nice article. Here is an excellent video on how to properly configure a Mikrotik router for other Quality of Service Issues. http://www.tiktube.com/?video=214