diff --git a/.cvsignore b/.cvsignore index e4eaa76..675db2b 100644 --- a/.cvsignore +++ b/.cvsignore @@ -1 +1 @@ -squid-2.4.STABLE1-src.tar.gz +squid-2.4.STABLE3-src.tar.gz diff --git a/FAQ.sgml b/FAQ.sgml index b8be7e5..86d1a5c 100644 --- a/FAQ.sgml +++ b/FAQ.sgml @@ -237,7 +237,7 @@ gives information on our operational mesh of caches. (uh, you're reading it). -. +. @@ -417,7 +417,7 @@ The following people have made contributions to this document: - + @@ -1864,7 +1864,7 @@ You can use the With Squid-2.4 and later you can use the ``null'' storage module: - cache_dir null -1 1000 + cache_dir null /null Can I prevent users from downloading large files? @@ -2023,7 +2023,7 @@ for each user. For example: gopher_proxy:http://mycache.example.com:3128/ -Redundant Auto-Proxy-Configuration +Redundant Proxy Auto-Configuration

There's one nasty side-effect to using auto-proxy scripts: if you start @@ -2089,6 +2089,13 @@ DHCP. name="Rodney van den Oever"> +Proxy Auto-Configuration with URL Hashing +

+The + +contains a lot of good information about hash-based proxy auto-configuration +scripts. With these you can distribute the load between a number +of caching proxies. Microsoft Internet Explorer configuration

@@ -3473,10 +3480,10 @@ Squid does not allow you to purge objects unless it is configured with access controls in - acl PURGE method purge + acl PURGE method PURGE acl localhost src 127.0.0.1 - http_access allow purge localhost - http_access deny purge + http_access allow PURGE localhost + http_access deny PURGE The above only allows purge requests which come from the local host and denies all other purge requests. @@ -4020,7 +4027,7 @@ represents the maximum size your Squid process has reached.

-by +by

Messages like "FATAL: xcalloc: Unable to allocate 4096 blocks of 1 bytes!" @@ -5296,16 +5303,35 @@ the client's IP address. The I set up my access controls, but they don't work! why? +I set up my access controls, but they don't work! why?

-You can debug your access control configuration by setting the - +If ACLs are giving you problems and you don't know why they +aren't working, you can use this tip to debug them. +

+In squid.conf enable debugging for section 33 at level 2. +For example: - debug_options ALL,1 28,9 +debug_options ALL,1 33,2 +Then restart or reconfigure squid. +

+From now on, your +If this does not give you sufficient information to nail down the +problem you can also enable detailed debug information on ACL processing + +debug_options ALL,1 33,2 28,9 + +Then restart or reconfigure squid as above. +

+From now on, your See also Proxy-authentication and neighbor caches

@@ -5546,20 +5572,9 @@ http_access deny all Debugging ACLs +

-If ACLs are giving you problems and you don't know why they -aren't working, you can use this tip to debug them. -

-In squid.conf enable debugging for section 32 at level 2. -For example: - -debug_options ALL,1 32,2 - -The restart or reconfigure squid. -

-From now on, your and . Can I limit the number of connections from a client?

@@ -5601,12 +5616,6 @@ you should write acl yuck dstdomain .foo.com http_access deny yuck -To be safe, you probably want to list both forms in your -access lists, for example: - -acl yuck dstdomain .foo.com foo.com -http_access deny yuck - I want to customize, or make my own error messages.

@@ -5926,7 +5935,7 @@ If the peer is multihomed, it is sending packets out an interface which is not advertised in the DNS. Unfortunately, this is a configuration problem at the peer site. You can tell them to either add the IP address interface to their DNS, or use Squid's -'udp_outgoing_address' option to force the replies +"udp_outgoing_address" option to force the replies out a specific interface. For example:

Resource Limits. The shell has limits on the size of a coredump file. You may need to increase the limit. + + sysctl options. On FreeBSD, you won't get a coredump from + programs that call setuid() and/or setgid() (like Squid sometimes does) + unless you enable this option: + +# sysctl -w kern.sugid_coredump=1 + No debugging symbols. The Squid binary must have debugging symbols in order to get @@ -6435,7 +6451,50 @@ If possible, you might keep the coredump file around for a day or two. It is often helpful if we can ask you to send additional debugger output, such as the contents of some variables. -Debugging Squid +

If you CANNOT get Squid to leave a core file for you then one of +the following approaches can be used

First alternative is to start Squid under the contol of GDB + + + % gdb /path/to/squid + handle SIGPIPE pass nostop noprint + run -DNYCd3 + [wait for crash] + backtrace + quit + + +

The drawback from the above is that it isn't really suitable to run on a +production system as Squid then won't restart automatically if it +crashes. The good news is that it is fully possible to automate the +process above to automatically get the stack trace and then restart +Squid. Here is a short automated script that should work: + + + #!/bin/sh + trap "rm -f $$.gdb" 0 + cat <$$.gdb + handle SIGPIPE pass nostop noprint + run -DNYCd3 + backtrace + quit + EOF + while sleep 2; do + gdb -x $$.gdb /path/to/squid 2>&1 | tee -a squid.out + done + + +

Other options if the above cannot be done is to: + +

a) Build Squid with the --enable-stacktraces option, if support exists for your OS (exists for Linux glibc on Intel, and Solaris with some extra libraries..) + +

b) Run Squid using the "catchsegv" tool. (Linux glibc Intel) + +

but these approaches does not by far provide as much details as using +gdb. + +Debugging Squid

If you believe you have found a non-fatal bug (such as incorrect HTTP @@ -6677,7 +6736,7 @@ A forwarding loop is when a request passes through one proxy more than once. You can get a forwarding loop if a cache forwards requests to itself. This might happen with - transparent caching (or server acceleration) configurations. + interception caching (or server acceleration) configurations. a pair or group of caches forward requests to each other. This can happen when Squid uses ICP, Cache Digests, or the ICMP RTT database to select a next-hop cache. @@ -6949,7 +7008,7 @@ than a proper close. You probably don't need to worry about them, unless you receive a lot of user complaints relating to SSL sites.

- notes that + notes that if the server is running a Microsoft TCP stack, clients receive RST segments whenever the listen queue overflows. In other words, if the server is really busy, new connections receive the reset message. @@ -7061,12 +7120,10 @@ Mikael Andersson reports that clicking on Webmin's -Changing the path to use Squid's own Segment Violation at startup or upon first request @@ -7121,6 +7178,29 @@ only use the proxy.pac. Cydoor aps will use both and will generate the errors. Disabling the old proxy settings in IE is not enought, you should delete them completely and only use the proxy.pac for example. +Requests for international domain names does not work +

+By Henrik Nordström +

+Some people have asked why requests for domain names using national +symbols as "supported" by the certain domain registrars does not work +in Squid. This is because there as of yet is no standard on how to +manage national characters in the current Internet protocols such +as HTTP or DNS. The current Internet standards is very strict +on what is an acceptable hostname and only accepts A-Z a-z 0-9 and - +in Internet hostname labels. Anything outside this is outside +the current Internet standards and will cause interoperability +issues such as the problems seen with such names and Squid. +

+When there is a consensus in the DNS and HTTP standardization groups +on how to handle international domain names Squid will be changed to +support this if any changes to Squid will be required. +

+If you are interested in the progress of the standardization process +for international domain names please see the +working group or it's . @@ -8040,7 +8120,16 @@ some other replacement algorithms by using the -The heap replacement code was contributed by John Dilley and others +With Squid version 2.4 and later you should use this configure option: + +./configure --enable-removal-policies=heap + +

+Then, in +The LFUDA and GDS replacement code was contributed by John Dilley and others from Hewlett-Packard. Their work is described in these papers: @@ -8174,7 +8263,7 @@ the -So why bind in that way? If you know you are transparent proxying, then why +So why bind in that way? If you know you are interception proxying, then why not bind the local endpoint to the host's (intranet) IP address? Why make the masses suffer needlessly? @@ -8631,7 +8720,7 @@ for more than 4 hours.

Because the problem appears to be with IPFilter I would guess that you would only run into this issue if you are trying to run Squid as a -transparent proxy using IPFilter. That makes sense. If there is anyone +interception proxy using IPFilter. That makes sense. If there is anyone with information that would indicate my finding are incorrect I am willing to investigate further. @@ -8737,8 +8826,8 @@ diff -p -u -r1.40 -r1.41 * SUCH DAMAGE. * * @(#)uipc_socket.c 8.3 (Berkeley) 4/15/94 -- * $Id: FAQ.sgml,v 1.3 2004/09/09 12:36:55 cvsdist Exp $ -+ * $Id: FAQ.sgml,v 1.3 2004/09/09 12:36:55 cvsdist Exp $ +- * $Id: FAQ.sgml,v 1.4 2004/09/09 12:37:50 cvsdist Exp $ ++ * $Id: FAQ.sgml,v 1.4 2004/09/09 12:37:50 cvsdist Exp $ */ #include @@ -8829,6 +8918,19 @@ to the following will appear: /dev/da2a on /usr/local/squid/cache (ufs, local, noatime, soft-updates, writes: sync 70 async 225) +Internal DNS problems with jail environment +

+Some users report problems with running Squid in the jail environment. Specifically, +Squid logs messages like: + +2001/10/12 02:08:49| comm_udp_sendto: FD 4, 192.168.1.3, port 53: (22) Invalid argument +2001/10/12 02:08:49| idnsSendQuery: FD 4: sendto: (22) Invalid argument + +

+You can eliminate the problem by putting the jail's network interface +address in the 'udp_outgoing_addr' configuration option +in squid.conf. + OSF1/3.2

@@ -8985,15 +9087,45 @@ When using Squid, some sites may give erorrs such as although these sites work fine without going through Squid.

Some versions of linux implement - (ECN) and this can cause -some TCP connections to fail. You can disable ECN with +some TCP connections to fail when contacting some sites with broken firewalls +or broken TCP/IP implementations. + +To work around such broken sites you can disable ECN with the following command: echo 0 >/proc/sys/net/ipv4/tcp_ecn

-See also the . +Found this on the FreeBSD mailing list: + +

+From: Robert Watson +

+As Bill Fumerola has indicated, and I thought I'd follow up in with a bit +more detail, the behavior you're seeing is the result of a bug in the +FreeBSD IPFW code. FreeBSD did a direct comparison of the TCP header flag +field with an internal field in the IPFW rule description structure. +Unfortunately, at some point, someone decided to overload the IPFW rule +description structure field to add a flag representing "ESTABLISHED". They +used a flag value that was previously unused by the TCP protocol (which +doesn't make it safer, just less noticeable). Later, when that flag was +allocated for ECN (Endpoint Congestion Notification) in TCP, and Linux +began using ECN by default, the packets began to match ESTABLISHED rules +regardless of the other TCP header flags. This bug was corrected on the +RELENG_4 branch, and security advisory for the bug was released. This +was, needless to say, a pretty serious bug, and good example of why you +should be very careful to compare only the bits you really mean to, and +should seperate packet state from protocol state in management structures, +as well as make use of extensive testing to make sure rules actually have +the effect you describe. + +

+See also the , + +or + HP-UX @@ -9055,7 +9187,7 @@ their own implementation.

A redirector allows the administrator to control the locations to which -his users goto. Using this in conjunction with transparent proxies +his users goto. Using this in conjunction with interception proxies allows simple but effective porn control. How does it work? @@ -9103,7 +9235,7 @@ it may be desirable to return an HTTP "301" or "302" redirect message to the client. This is now possible with Squid version 1.1.19.

-Simply modify your redirector program to append either "301:" or "302:" +Simply modify your redirector program to prepend either "301:" or "302:" before the new URL. For example, the following script might be used to direct external clients to a secure Web server for internal documents: @@ -9878,7 +10010,7 @@ information. However, the following instructions are correct as of this writing (July 1999.)

-Getting transparent caching to work requires four distinct steps: +Getting interception caching to work requires four distinct steps: @@ -9950,17 +10082,17 @@ forwarding commands. In the The The You @@ -10195,7 +10327,7 @@ not host names and demon aren't generally asked for IP addresses by other users; -Linux kernel 2.0.30 is a no-no as transparent proxying is broken (I use +Linux kernel 2.0.30 is a no-no as interception proxying is broken (I use 2.0.29); @@ -10380,7 +10512,7 @@ Apply the route map to the ethernet interface.

notes that -there is a Cisco bug relating to transparent proxying using IP +there is a Cisco bug relating to interception proxying using IP policy route maps, that causes NFS and other applications to break. Apparently there are two bug reports raised in Cisco, but they are not available for public dissemination. @@ -10426,7 +10558,7 @@ Conversely, this set has worse performance, but works for all protocols:

Just for kicks, here's an email message posted to squid-users -on how to make transparent proxying work with a Cisco router +on how to make interception proxying work with a Cisco router and Squid running on Linux.

@@ -10439,7 +10571,7 @@ running Linux 2.0.33.

Many thanks to the following individuals and the squid-users list for -helping me get redirection and transparent proxying working on my +helping me get redirection and interception proxying working on my Cisco/Linux box. @@ -10529,16 +10661,16 @@ this in /etc/rc.d/rc.local

I am using with - installed. You will want to install this patch if using a setup similar to mine. The cache is trying to connect to itself...

-by +by

-I think almost everyone who have tried to build a transparent proxy +I think almost everyone who have tried to build a interception proxy setup have been bitten by this one.

@@ -10548,7 +10680,7 @@ Measures you can take: Deny Squid from fetching objects from itself (using ACL lists). Apply a small patch that prevents Squid from looping infinitely -(available from ) +(available from ) Don't run Squid on port 80, and redirect port 80 not destined for the local machine to Squid (redirection == ipfilter/ipfw/ipfadm). This @@ -10564,7 +10696,7 @@ front of Squid. Squid does not yet know how to interface to ipfilter

by Duane Wessels

-I set out yesterday to make transparent caching work with Squid and +I set out yesterday to make interception caching work with Squid and FreeBSD. It was, uh, fun.

It was relatively easy to configure a cisco to divert port 80 @@ -10650,7 +10782,7 @@ and the by

-This is to do with configuring transparent proxy +This is to do with configuring interception proxy for an ACC Tigris digital access server (like a CISCO 5200/5300 or an Ascend MAX 4000). I've found that doing this in the NAS reduces traffic on the LAN and reduces processing load on the @@ -10836,7 +10968,7 @@ Finally add "OPTION GRE" to your kernel config file and rebuild your kernel. Note, the . +. Configuring Linux 2.2 @@ -10918,7 +11050,7 @@ Finally you will need to load the module:

The machine should now be striping the GRE encapsulation from any packets recieved and requeuing them. The system will also need to be configured -for transparent proxying, either with +for interception proxying, either with or with . Configuring Others @@ -10948,12 +11080,12 @@ is welcome to code it up and contribute to the Squid project.

by .

-First, configure Squid for transparent caching as detailed +First, configure Squid for interception caching as detailed at the .

Next, configure the Foundry layer 4 switch to -transparently redirect traffic to your Squid box or boxes. By default, +redirect traffic to your Squid box or boxes. By default, the Foundry redirects to port 80 of your squid box. This can be changed to a different port if needed, but won't be covered @@ -10975,7 +11107,7 @@ squid2.foo.com 192.168.1.11

We will assume you have various workstations, customers, etc, plugged -into the switch for which you want them to be transparently proxied. +into the switch for which you want them to be intercepted and sent to Squid. The squid caches themselves should be plugged into the switch as well. Only the interface that the router is connected to is important. Where you put the squid caches or other connections does not matter. @@ -11037,6 +11169,12 @@ howto that would apply for most people, not meant to be a comprehensive manual of how to configure a Foundry switch. I can however revise this with any information necessary if people feel it should be included. +Can I use +No, you cannot. With interception proxying, the client thinks +it is talking to an origin server and would never send the + SNMP @@ -11279,14 +11417,14 @@ new disk and leave the existing ones in place. Squid 2 performs badly on Linux

-by +by

You may have enabled Asyncronous I/O with the You should also know that For our local access we use a dstdomain ACL, and for delay pool exceptions we use a dst ACL as well since the delay pool ACL processing is done using -'fast lookups', which means (among other things) it won't wait for a DNS +"fast lookups", which means (among other things) it won't wait for a DNS lookup if it would need one.

@@ -12081,7 +12219,7 @@ squid.conf.

is a program used in conjunction with the Linux Transparent Proxy -networking feature, and ipfwadm, to transparently proxy HTTP and +networking feature, and ipfwadm, to intercept HTTP and other requests. Transproxy is written by . Iain's redirector package @@ -12207,32 +12345,45 @@ queue parameters for Squid. Message queue implementations normally have the following parameters:

-The messages between Squid and diskd are 32 bytes. Thus, MSGMAX -should be 32 or greater. You may want to set it to a larger -value, just to be safe. +The messages between Squid and diskd are 32 bytes for 32-bit CPUs +and 40 bytes for 64-bit CPUs. Thus, MSGSSZ should be 32 or greater. +You may want to set it to a larger value, just to be safe.

We'll have two queues for each +I've found that 75 messages per queue is about the limit of decent performance. +If each diskd message consists of just one segment (depending on your +value of MSGSSZ), then MSGSEG should be greater than 75. +

MSGMNB and MSGTQL affect how many messages can be in the queues -at one time. I've found that 75 messages per queue is about -the limit of decent performance. Thus, MSGMNB must be -at least 75*MSGMAX, and MSGTQL must be at least 75 times -the number of +MSGTQL should be at least 75 times the number of FreeBSD

@@ -12245,11 +12396,11 @@ options SYSVMSG You can set the parameters in the kernel as follows. This is just an example. Make sure the values are appropriate for your system: -options MSGMNB=16384 # max # of bytes in a queue -options MSGMNI=41 # number of message queue identifiers -options MSGSEG=2049 # number of message segments +options MSGMNB=8192 # max # of bytes in a queue +options MSGMNI=40 # number of message queue identifiers +options MSGSEG=512 # number of message segments per queue options MSGSSZ=64 # size of a message segment -options MSGTQL=512 # max messages in system +options MSGTQL=2048 # max messages in system Digital Unix @@ -12258,9 +12409,9 @@ Message queue support seems to be in the kernel by default. Setting the options is as follows: options MSGMNB="8192" # max # bytes on queue -options MSGMNI="31" # # of message queue identifiers -options MSGMAX="2049" # max message size -options MSGTQL="1024" # # of system message headers +options MSGMNI="40" # # of message queue identifiers +options MSGMAX="2048" # max message size +options MSGTQL="2048" # # of system message headers

@@ -12274,9 +12425,9 @@ If you have a newer version (DU64), then you can probably use To change them make a file like this called ipc.stanza: ipc: - msg-max = 2049 - msg-mni = 31 - msg-tql = 1024 + msg-max = 2048 + msg-mni = 40 + msg-tql = 2048 msg-mnb = 8192 then run @@ -12312,11 +12463,11 @@ name="Demangling Message Queues"> in Sunworld Magazine. I don't think the above article really tells you how to set the parameters. You do it in /etc/system with lines like this: -set msgsys:msginfo_msgmax=2049 +set msgsys:msginfo_msgmax=2048 set msgsys:msginfo_msgmnb=8192 -set msgsys:msginfo_msgmni=31 +set msgsys:msginfo_msgmni=40 set msgsys:msginfo_msgssz=64 -set msgsys:msginfo_msgtql=1024 +set msgsys:msginfo_msgtql=2048

Of course, you must reboot whenever you modify /etc/system @@ -12673,7 +12824,7 @@ want to make a cron job that regularly verifies that your proxy blocks access to port 25. -$Id: FAQ.sgml,v 1.3 2004/09/09 12:36:55 cvsdist Exp $ +$Id: FAQ.sgml,v 1.4 2004/09/09 12:37:50 cvsdist Exp $