Squid proxy and custom URL filtering

A proxy server, is a computer/appliance that acts as a gateway between a local network (e.g., all the computers at one company or in one building) and a larger-scale network such as the Internet. There are great benefits of using a proxy server such as the ability to hide the IP address of the client computer so that it can surf anonymous or cache web pages; but most importantly the ability to provide certain security features (such as URL filtering, content scanning and so on)

Enterprise organisations are turning to vendors such as bluecoat, Cisco (ex ironport) , websense, barracuda (to name a few ) to get proxy appliance that would deliver their requirements. However, Small-medium businesses or even home users can’t afford such products due to the cost involved.

This is where Squid comes in. Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and much much more! More importantly though Squid is provided as free, open source software and can be used under the GNU General Public License (GPL) of the Free Software Foundation.

This guide will assume that a squid server has already been installed and is functional in your environment and will focus on delivering security features such as URL filtering

Unfortunately, Proxy vendors do not provide a list of malicious domains to end users (it’s their ‘secret sauce’ and users can only benefit by using their appliances)

However, several organizations maintain and more importantly publish blocklists (a.k.a blacklists) of IP addresses and URLs of systems and networks suspected in malicious activities. Utilising these lists in your squid configuration can (and will) protect your clients

A Blocklists of Suspected Malicious IPs and URLs is hosted by Lenny Zeltser.

Once you have identified which list(s) you would like to use, the process is quite easy

Step 1 : Create a script to merge the lists you would like to us

The script below will use 3 different lists identified in Lenny’s blog and after some basic manipulation will store it in a text file named malicious_domains_and_IPs.txt

Step 2 : Create a cronjob to update the text file created with the latest entries

Based on my parameters above the script would run at 05:35 every day. This will ensure that the text file will include the latest malicious domain that I would like to have blocked. If you are not sure about the specific fields you can use crontab Guru

Step 3: Point your Squid configuration to the text file created

Once you reload the squid configuration or ask squid to just reread the config, your clients would get an “Access Denied” message.


I hope this guide helps and please do leave a comment if you have any questions

Leave a Reply