Setup open proxy

This tutorial shows how to setup an open proxy in ubuntu. I am using such a setup for scraping purposes and it works quite well.

Open a terminal and connect to your server (if you want to install it on a remote machine).

I am using squid, because it's a famous package and it supports great security settings (if you don't need authentication you can use lightproxy). The second package (apache2-utils) is used to manage users for authentication.

apt-get install squid3 apache2-utils

Now you have to set new rules for your firewall. In my case I used the following command (tested on digitalocean and scaleway VPS):

ufw allow 3128/tcp

The following command adds a new user (named chris) the promt will ask you for a proper password.

htpasswd -c /etc/squid/passwords chris
Attention: Different names are used for the configuration file. In ubuntu it's /etc/squid3/squid.conf or /etc/squid/squid.conf. In my case it was the second one, so I will use it in this tutorial.

The configuration file of squid is well documented, but bloated. My configuration file looks like this (I removed most comments), you should adapt it or modify it accordingly.

http_port 3128

#disable cache
cache deny all

#use http-basic auth
auth_param basic program /usr/lib/squid3/basic_ncsa_auth /etc/squid/passwords
auth_param basic realm proxy
acl authenticated proxy_auth REQUIRED
http_access allow authenticated

acl SSL_ports port 443
acl Safe_ports port 80      
acl Safe_ports port 21      
acl Safe_ports port 443     
acl Safe_ports port 70      
acl Safe_ports port 210     
acl Safe_ports port 1025-65535  
acl Safe_ports port 280     
acl Safe_ports port 488     
acl Safe_ports port 591     
acl Safe_ports port 777     
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
http_access allow localhost
http_access deny all
coredump_dir /var/spool/squid
refresh_pattern ^ftp:       1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .       0   20% 4320

Afterwords you have to restart the service. This can be done with the following command:

service squid restart

Check configuration

Now you can test the installation on the server. Just call the following command and you should see a squid process, which listens on port 3128.

netstat -tulpn

Curl has also a build-in proxy support so it's quite easy to test it in a "real" setting.

curl -x http://<username>:<password>@<domain>:3128