Changing from Squid2 to Squid3 inline / transparent / interception and NAT/dNat on separate boxes

If you are used to doing http inline caching or filtering with Squid2 on a different machine to your router and previously used DNAT in iptables then you are in for a shock when you try to move your stuff to Squid3!

See the first comment below this article for further information on the changes made. I appear to have assumed wrong, read wrong or misinterpreted information about the cause of the errors.

Squid3 changes the interception mode to MASQUERADE as the source IP that NATTED the request in the first place. This means instead of squid going “oh, a request” and processing it and then sending the GET request to the website from the IP of the squid box… squid tries to set the source IP of the GET request to the IP of the machine that made the request. (Basically spoofing the IP).

However if you just fire up squid in the configuration you expect then you get errors similar to this in the cache.log:
2015/10/19 19:24:12| NF getsockopt(SO_ORIGINAL_DST) failed on local=10.10.10.141:3128 remote=10.10.10.1:53238 FD 6 flags=33: (92) Protocol not available

This is because squid doesn’t have the ability to set the from IP.
This issue is simply solved by allowing Masquerade on the squid machine:
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

This then triggers another problem.. the IPTables rule on the router then can’t tell that the request is from the Squid box and just routes the request back to squid. Squid then rejects the requests with:
WARNING: Forwarding loop detected for: (and then prints the request)

Which is a pretty annoying pain in the ass.

It seems that due to a security flaw they changed the way NAT is handled in Squid3.
Further reading here: https://squidproxy.wordpress.com/2014/12/19/squid-3-2-mythbusting-nat/

The squid documentation has very complicated instructions on how you are supposed to get around this..
http://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute

For some reason I didn’t have much luck with it. I think the documentation is lacking on what each step actually does or means.

I then came across the following helpful post on Reddit:
https://www.reddit.com/r/linux/comments/1dv2x2/on_mitigating_the_great_squid_31x_33x_upheaval/

But before I could read and digest it.. a friend also came across the following stackexchange post which is even simpler!
http://unix.stackexchange.com/questions/138013/routing-internet-on-nat-server-thru-squid-proxy

I’m reposting the commands I used here in case the above url goes down!

On your router (in my example 10.10.10.141 is my squid box):

#!/bin/bash

iptables -t mangle -A PREROUTING -j ACCEPT -p tcp --dport 80 -s 10.10.10.141
iptables -t mangle -A PREROUTING -j MARK --set-mark 8 -p tcp --dport 80
ip rule add fwmark 8 table 2
ip route add default via 10.10.10.141 dev ETH00 table 2

You then need the following on the squid box:

iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

The squid config needs to have http_port set to “intercept” like so:
http_port 3128 intercept

Remember you also need a line without an intercept for Squid to function:
http_port 9898
Otherwise you get this error in cache.log:
“ERROR: No forward-proxy ports configured.”

Finally.. success.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Changing from Squid2 to Squid3 inline / transparent / interception and NAT/dNat on separate boxes

  1. MadCache says:

    Not sure where you got your info from to begin with. But from the 2nd to 5th paragraphs for this article every explanatory sentence is just plain wrong about the technical operations happening. Thankfully you do find the real solution at the end in StackExchange.

    I would just like to clear up where you went wrong, so if you have other issues the solution might be easier to find:

    “Squid3 changes the interception mode to MASQUERADE as the source IP that NATTED the request in the first place.”

    No. Your “iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE” rule does that, not Squid3.

    Squid2 did something similar called “application layer gateway” which has the appearance of MASQUERADE. But was not actually NAT based, and that disappearing is part of what changed in Squid3.

    ” This means instead of squid going “oh, a request” and processing it ”

    Squid does go “oh, a new request” and processes it. Seriously. The how of the processing part is all that has changed.

    “and then sending the GET request to the website”

    Squid sends the request to the web server (aka original destination IP) that NAT tells it the client was connecting to before NAT mangled the IPs. The hint (dst / destination) is in the option name ORIGINAL_DST which you see later.

    The difference is that the NAT information is actively used now. So it needs checking.

    ” from the IP of the squid box… squid tries to set the source IP of the GET request to the IP of the machine that made the request.”

    No. Squid tries to send the request TO some destination (not source). The destination that a remote router (secretly) changed to be the Squid boxes own IP.

    “(Basically spoofing the IP).”

    No spoofing. The source IP with NAT is the Squid box IP. Always.

    Other features do have spoofing, but that is not being discussed here. Just NAT.

    “However if you just fire up squid in the configuration you expect then you get errors similar to this in the cache.log:
    2015/10/19 19:24:12| NF getsockopt(SO_ORIGINAL_DST) failed on local=10.10.10.141:3128 remote=10.10.10.1:53238 FD 6 flags=33: (92) Protocol not available”

    This is about the inbound client to Squid TCP connection. Notice how the local IP (supposed to be the web server IP) is the Squid box IP, and the remote IP is the client.

    This failure is telilng Squid the connection was not NATed by the Squid box. So the local IP is the actual web server the client was contacting.
    If it had worked the local IP would be changed to be the original dst-IP the client was connecting to. So Squid could use it to relay the request to the right web server, and not some other DNS listed IP the client did not want to connect to.

    “This is because squid doesn’t have the ability to set the from IP.”

    Wrong. This is because the router NATed the IPs and told Squid it was the webserver.

    “This issue is simply solved by allowing Masquerade on the squid machine:
    iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

    Wrong. That is not solving anything. The traffic was already from the Squid box IP and is not going to anywhere outside the Squid box…

    “This then triggers another problem.. the IPTables rule on the router then can’t tell that the request is from the Squid box and just routes the request back to squid.”

    Note that the traffic is probably not even going to the router. It is simply looping inside the Squid box from a Squid outgoing port to Squid listening http_port. It should be clear at this point why the MASQUERADE or any other NAT operation is not useful.

    ” Squid then rejects the requests with:
    WARNING: Forwarding loop detected for: (and then prints the request)”

    As it should. Since it is sending itself a request to fetch from itself a request to fetch from itself a request to …

    The alternative is that every TCP socket or every byte of memory on your Squid box gets consumed (whichever happens first). Then something crashes. Maybe Squid, maybe the OS kernel, maybe something else.

    “Which is a pretty annoying pain in the ass.”

    Quite. Luckily you found the solution was to make the router actually do routing (what a concept), and leaving the Squid box to do the NATing.

    Strangely this is how routers and NAT was designed to operate from the very beginning. All Squid3 is doing is enforcing that routing and NATing do what they were designed to do.

  2. Thank you for the incredibly detailed reply.
    The information I got was gathered mainly from the only source I could find for the SO_ORIGINAL_DST error – which were many mailing list posts dispersed around the place. Until my friend found the squid blog post (referenced in the page above) there wasn’t any sensible source of information.

    I am yet to digest the information about what the squid2 + nat vulnerability is, I might understand it better once I find time to do so.

    I think I already understand slightly more. Squid3 now uses the DST IP instead of the Host: header? Which causes the loop (or SO_ORIGINAL_DST error)? And is my red herring making me think it was an issue with NAT stuff.

    Whilst I am wrong – your comment demeanour comes across quite angry or aggressive. While it is difficult to communicate emotions or feelings in text on the internet I’d be happier if you somehow changed your tone to sound friendlier.

Comment on this topic

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s