Squid3, Cygwin, diladele, external acl helper programs and .net

Recently I had the need to try and interface a .net program with Squid 3.5+.

The idea is pretty simple.. Squid is configured to have an external_acl_type sending %URI to a .exe program.

Upon web browsing Squid passes, for example, http://www.no-exist-example-pornsite.com/image.jpg to the .net program.
My .net program can lookup, regex match, or any other function and then return OK or ERR to allow or deny access to the website.

There is not very much documentation on building helper applications for Squid but I found that:

  1. Squid should be sending stdout with the URL or parameters you select.
  2. Squid expects stdin with the result such as OK, ERR and a limited set of other fields like the reason the request was rejected (or allowed).

I spent many, many hours trying to figure out why my program wasn’t working. Squid would start it when squid got loaded but when I browsed to a website the following would happen:

  1. The browser would never complete the query.
  2. Squid wouldn’t log the query or errors.
  3. My .net application wouldn’t log _any_ stdin input at all.
  4. Eventually the browser would time out (or in Firefox’s case.. memory leak and crash after several hours of not progressing the page load).

Eventually I tried Squid 2.7 instead of Squid3. Suddenly my .net program could see the requests and also pass back the OK or ERR responses. The blocking and allowing work exactly as required!

The only issue is – Squid 2.7 doesn’t have SSL interception. This feature only exists in Squid3!

As far as I could work out it potentially might have been down to the compilation of Squid3. Squid3 seems to have been compiled using CygWin. Squid 2.7 was compiled using MinGW.

Sadly nobody seems to maintain Squid3 compilations for windows using MinGW. I’ve also failed massively trying to compile it myself.

After _lots_ and lots of googling I finally have found this:

https://cygwin.com/ml/cygwin/2012-03/msg00298.html

Which I will quote in case the above mailing archive ever goes down:

Actually, the problem can be reproduced as follows from a C++ console program. The issue is not specific to .NET. It appears that Cygwin croaks if you give it a null write (writing zero bytes):

#include <windows.h>

int _tmain(int argc, _TCHAR* argv[])
{
	char * test = "AB";
	DWORD written;

	// Get standard output file handle
	HANDLE h = GetStdHandle(STD_OUTPUT_HANDLE);

	// Do a null write.  This breaks Cygwin when output is redirected.
	WriteFile(h, test, 0, &written, NULL);

	// Print an "A" and a "B" in two write operations
	WriteFile(h, test, 1, &written, NULL);
	WriteFile(h, test+1, 1, &written, NULL);
	return 0;
}

The correct output of “AB” is not printed when running “echo
`./HelloCPP.exe`” from Cygwin. If you remove the null write (WriteFile of zero bytes), then Cygwin captures the output of “AB” as expected. Also note that if the null write is moved between the printing of “A” and the printing of “B”, then the output is also as expected; it seems to be having problems only when null write is the first write.

How is this related to .NET? It turns out this is the API sequence used when .NET decides to open standard output:

1. Call GetStdHandle to get standard output file handle.
2. Do a null write to test if the handle was opened successfully. If it wasn’t, then a null stream is used instead of standard output.

There’s really no way to disable #2 from a .NET programmer’s perspective. So this new Cygwin appears to have broken every .NET program writing to standard output, and also every Windows program that does null writes to standard output.

By the way, maybe these issues exist with standard error. I didn’t test.

1. I assume this is a bug in Cygwin. Anything more I should do for
reporting it? (e.g. creating an issue/ticket)
2. Workaround suggestions?
3. Which part of Cygwin do I need to roll back to fix this issue for now?

Best regards,

James Johnston

It seems I was right! Something to do with Cygwin compilation is breaking stdin / stdout “Standard Input”. But only for .net applications or any other applications which may test a call-back using a “null write”.

What a massive waste of time :( Now I’ve got to work out how to either compile, successfully, Squid3 using MinGW or fix the Cygwin issue or stop .net bugging it out.

Every day is a learning day. I will post the solution; if I ever find it.

A couple of other interesting messages and statements from Cygwyn developers are here:

https://www.cygwin.com/ml/cygwin/2012-04/msg00631.html

http://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app

https://www.cygwin.com/ml/cygwin/2012-06/msg00450.html

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

Comment on this topic

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s