Differences

This shows you the differences between two versions of the page.

--- pfsense:squid:options_which_affect_the_cache [2020/03/30 09:26] – peter
+++ pfsense:squid:options_which_affect_the_cache [2020/07/15 09:30] (current) – external edit 127.0.0.1
@@ Line 22: / Line 22: @@
 ----
-===== Memory Cache Size (cache_mem) =====
+===== Low-Water Mark in % (cache_swap_low) =====
-Specifies the ideal amount of physical RAM (in megabytes) to be used for In-Transit objects, Hot Objects and Negative-Cached objects.
+The low-water mark for AUFS/UFS/diskd cache object eviction by the cache_replacement_policy algorithm.
-  * Default is : **64MB**.
+  * Default: **90**.
-Data for these objects are stored in 4 KB blocks.  This parameter specifies the ideal upper limit on the total size of 4 KB blocks allocated.
+This defines when the replacement should take place.
-In-transit objects have priority over the others.  When additional space is needed for incoming data, Negative-cached and Hot objects will be released.  In other words, the negative-cached and hot objects will fill up any unused space not needed for In-transit objects.
+Replacement begins when the swap (disk) usage is above the low-water mark and attempts to maintain utilization near the low-water mark.
-If circumstances require, this limit will be exceeded.  Specifically, if your incoming request rate requires more than **cache_mem** of memory to hold In-transit objects, Squid will exceed this limit to satisfy the new requests.  When the load decreases, blocks will be freed until the high-water mark is reached.  Thereafter, blocks will be used to store hot objects.
+As swap utilization gets close to high-water mark object eviction becomes more aggressive.
-<WRAP info>
+If utilization is close to the low-water mark less replacement is done each time.
-**NOTE:**  This tag does not specify the maximum process size.
-It places a limit on one aspect of squid's memory usage.
+<WRAP important>
+**WARNING:**  If you have a large cache, 5% could be hundreds of MB. If this is the case you may wish to set this number closer together.
+</WRAP>
-Squid uses memory for other things as well.
-Process will probably become twice or three times bigger than the value put here.
+----
+===== High-Water Mark in % (cache_swap_high) =====
+The high-water mark for AUFS/UFS/diskd cache object eviction by the cache_replacement_policy algorithm.
+  * Default: **95**.
+This defines when the replacement should take place.
+Replacement begins when the swap (disk) usage is above the low-water mark and attempts to maintain utilization near the low-water mark.
+As swap utilization gets close to high-water mark object eviction becomes more aggressive.
+If utilization is close to the low-water mark less replacement is done each time.
+<WRAP important>
+**WARNING:**  If you have a large cache, 5% could be hundreds of MB.  If this is the case you may wish to set these numbers closer together.
 </WRAP>
+----
+===== Hard Disk Cache Size =====
+Amount of disk space (in megabytes) to use for cached objects.
+Set to 50% of the hard disk size.
+----
+===== Hard Disk Cache System (cache_dir) =====
+This specifies the kind of storage system to use.
+^Storage System^Description^Comment^
+|usf|The old well-known Squid storage format that has always been there.| |
+|aufs|Uses POSIX threads to avoid blocking the main Squid process on disk I/O. (Formerly known as async-io.)| |
+|diskd|Uses a separate process to avoid blocking the main Squid process on disk I/O.| |
+|null|Does not use any storage. Ideal for embedded.| |
+----
+===== Level 1 Directories =====
+Specifies the number of Level 1 directories for the hard disk cache.
+  * Default: **16**.
+  * My setting: **64**.
+Each level-1 directory contains 256 subdirectories, so a value of 256 level-1 directories will use a total of 65536 directories for the hard disk cache.
+This will significantly slow down the startup process of the proxy service, but can speed up the caching under certain conditions.
 ----
@@ Line 59: / Line 111: @@
   * Default: **4 (MB)**.
+  * My setting: **1024**.
+Objects larger than this size will NOT be saved on disk.
+The value is specified in kilobytes.
+If you wish to get a high BYTES hit ratio, you should probably increase this (one 32 MB object hit counts for 3200 10KB hits).
+Leave this value low if you wish to increase the speed more than what you want to save bandwidth.
+<WRAP info>
+**NOTE:**  If using the LFUDA replacement policy you should increase this value to maximize the byte hit rate improvement of LFUDA!
+See **replacement_policy** section for a discussion of this policy.
+</WRAP>
 ----
-===== Memory Cache Size (Cache memory size) =====
+===== Memory Cache Size (cache_mem) =====
 Specifies the ideal amount of physical RAM (in megabytes) to be used for In-Transit objects, Hot Objects and Negative-Cached objects.
-  * Minimum value: **1 (MB)**.
+  * Default is : **64MB**.
-  * Default: **64 (MB)**.
+  * My setting: **2048**.
+Data for these objects are stored in 4 KB blocks.  This parameter specifies the ideal upper limit on the total size of 4 KB blocks allocated.
+In-transit objects have priority over the others.  When additional space is needed for incoming data, Negative-cached and Hot objects will be released.  In other words, the negative-cached and hot objects will fill up any unused space not needed for In-transit objects.
+If circumstances require, this limit will be exceeded.  Specifically, if your incoming request rate requires more than **cache_mem** of memory to hold In-transit objects, Squid will exceed this limit to satisfy the new requests.  When the load decreases, blocks will be freed until the high-water mark is reached.  Thereafter, blocks will be used to store hot objects.
+|In-transit objects or current requests|These are the objects related to the current requests and they have the highest priority to be kept in the cache space in RAM.  These objects must be kept in RAM and if there is a situation where the incoming request rate is quite high and we are about to overflow the cache space in RAM, Squid will try to keep the served part (the part which has already been sent to the client) on the disk to create free space in RAM.|
+|Hot or popular objects|These objects or web documents are popular and are requested quite frequently compared to others.  These are stored in the cache space left after storing the in-transit objects as these have a lower priority than in-transit objects.  These objects are generally pushed to disk when there is a need to generate more in RAM cache space for storing the in-transit objects.|
+|Negatively cached objects|Negatively cached objects are error messages which Squid has encountered while fetching a page or web document on behalf of a client.  For example, if a request to a web page has resulted in a HTTP error 404 (page not found), and Squid receives a subsequent request for the same web page, then Squid will check if the response is still fresh and will return a reply from the cache itself.  If there is a request for the same page after the negatively cached object corresponding to that page has expired, Squid will check again if the page is available.  Negatively cached objects have the same priority as hot or popular objects and they can be pushed to disk at any time in favor of in-transit objects.|
+<WRAP important>
+**WARNING:**  This tag does not specify the maximum process size.
+It places a limit on one aspect of squid's memory usage.
+Squid uses memory for other things as well.
+Process will probably become twice or three times bigger than the value put here.
+While deciding the RAM space for caching, we should be neither greedy nor paranoid.  If we specify a large percentage of RAM for caching, the overall system performance will suffer as the system will start swapping processes in case there is no free RAM left for other processes.  If we use a very low percentage of RAM for caching, then we’ll not be able to take full advantage of Squid’s caching mechanism.
+</WRAP>
 ----
@@ Line 76: / Line 169: @@
   * Default: **256 (KB)**.
+  * My setting:  **1024**.
+This should be set high enough to keep objects accessed frequently in memory to improve performance at the same time low enough to keep larger objects from hoarding **cache_mem**.
 ----
@@ Line 84: / Line 180: @@
 The memory replacement policy determines which objects are purged from memory when space is needed.
-  * Default: **heap GDSF**.
+  * Default: **Heap GDSF**.
+  * My setting: **Heap LFUDA**.
+The cache replacement policies are same as **cache_replacemen_policy**:
+^Policy^Explanation^Comment^
+|LRU|Squid's original list based LRU policy|Keeps recently referenced objects.|
+|Heap GDSF|Greedy-Dual Size Frequency|Optimizes object hit rate by keeping smaller popular objects in cache so it has a better chance of getting a hit.  It achieves a lower byte hit rate than LFUDA though since it evicts larger (possibly popular) objects.|
+|Heap LFUDA|Least Frequently Used with Dynamic Aging|Keeps popular objects in cache regardless of their size and thus optimizes byte hit rate at the expense of hit rate since one large, popular object will prevent many smaller, slightly less popular objects from being cached.  If using the LFUDA replacement policy you should increase the value of **maximum_object_size** above its default of 4096 KB to maximize the potential byte hit rate improvement of LFUDA.|
+|Heap LRU|LRU policy implemented using a heap| |
+The policies utilize a dynamic aging mechanism that prevents cache pollution that can otherwise occur with frequency-based replacement policies.
+See https://www.hpl.hp.com/techreports/1999/HPL-1999-69.html and https://fog.hpl.external.hp.com/techreports/98/HPL-98-173.html.