Differences

This shows you the differences between two versions of the page.

--- pfsense:pfblockerng:block_ads [2021/02/07 17:29] – peter
+++ pfsense:pfblockerng:block_ads [2021/02/07 18:34] (current) – removed peter
@@ Line 1: / Line 1: @@
-====== PFSense - pfBlockingNG - Block Ads ======
-Use the **DNSBL** portion of pfBlockerNG to remove advertising.
-===== Testing By Browsing =====
-So what does the finished product look like?
-On many sites, you’ll see gray boxes where an ad normally would have been.
-A browser add-on like uBlock Origin further cleans this up by removing the gray box entirely and it also provides some secondary protections.
-Also keep in mind that some ads are still served such as video ads on YouTube.
-Those ads cannot be blocked via pfBlockerNG since the ad content is served from the same domain names (DNS) as the video content.
-----
-===== Testing from the command line =====
-Normally, pinging a site will return the sites actual IP address.
-However, with pfBlockerNG properly setup you may instead see a reply of **10.10.10.1**, which is the default virtual IP address DNSBL creates.
-{{:pfsense:pfblockerng:pfsense_pfblockerng_dnsbl_ping_block.png?800|}}
-----
-===== Statistics and graphs =====
-pfBlockerNG has some really fantastic graphs built-in as shown below.
-You can even see the top blocked domains, source IPs with the most blocks, blocked user agent strings, TLDs, and much more.
-Also helpful is you need to whittle down the number of feeds you are using, i.e. this feed accounts for 50% of your blocks and it’s a third the size of these other two feeds combined.  Just go to **Firewall -> pfBlockerNG -> Reports -> DNSBL Stats** to see all the DNSBL eye candy, aka graphs/stats.
-{{:pfsense:pfsense_pfblockerng_reports_dnsbl_stats.png?800|}}
-----
-===== Troubleshooting - Whitelisting =====
-You can remove the offending list entirely (**DNSBL -> DNSBL Groups -> Edit the list in question**) or more preferably, you can just whitelist the domain.
-The absolute easiest way to do this is by going to the **Reports** tab and scrolling down to the DNSBL section.
-Clicking on the **red** lock will temporarily unlock the domain so you can verify if it is indeed the domain that needs to be whitelisted.
-Clicking the **+** will add the domain to the DNSBL whitelist.
-{{:pfsense:pfblockerng:pfsense_pfblockerng_dnsbl_whitelist.png?800|}}
-When clicking the **+** you will then receive a prompt about whether you want to perform a wildcard whitelist or just a whitelist.
-Read the explanation, but typically use whitelist because it is more exact and less prone to letting something past.
-<WRAP tip>
-Adding a description so you know what was broken and/or why you fixed it, i.e. today it makes perfect sense, but it might not 6 months from now.
-</WRAP>
-If you go back to the main DNSBL tab and expand the DNSBL Whitelist section toward the bottom, you should now see the domain you whitelisted.
-You might also notice that if the domain you are whitelisting has CNAME records, pfBlockerNG is smart enough to add those too.
-Simply type each domain in on a separate line and then click **Save** if you know which domains to whitelist.  If you want the whitelist additions/changes to occur sooner rather than later, you will also need to go back to the **Update** tab and click **Run**.  If you don’t want to do the trial and error on your own see some whitelist recommendations below.
-It's also worth mentioning that if a system already resolved the domain name on your system and it is previously resolved to 10.10.10.1, then you may need to clear your local DNS cache, your browser cache, or both.  To clear your machine’s cache, from a command line on Windows, type in **ipconfig /flushdns** and that should take care of it.  You can run a similar command on a Linux system, although the commands can vary from one installation to the next.  More often than not, simply restarting your network interface will work; on most distributions, **service networking restart** or **systemctl restart network** should take care of it for you.  Each browser has a slightly different way to clear the cache, however, all of them allow you to pull a new version of the website if you hold down **Shift** while clicking on the refresh/reload button.
-If ads are not getting blocked and the ping commands above don’t return the virtual IP address, it’s also possible your local machine is not using pfSense for its DNS settings. If you are using Windows, check your network settings and make sure it is set to your pfSense IP address. On Linux/*nix, check your /etc/resolv.conf or even Network Manager (if using a GUI).  If you are not using pfSense for your DHCP server, you may need to do some digging.
-Browsers can also get in the way especially with the advent of DNS over HTTPS.  If you find your ping tests work, but your browser doesn’t, then that is most likely your issue.  Although somewhat uncommon, some anti-virus packages and endpoint protection can mess with your DNS settings too.  Furthermore, those changes may not necessarily be reflected in your operating system’s DNS settings.  For example, Avast Premier has a Secure DNS feature that will force your browser to use Avast specified DNS servers in an effort to prevent DNS hijacking.  If you find that other devices on your network are blocking ads and one particular device doesn’t, then your anti-virus or endpoint protection very well may be the culprit.  When all else fails, you can always fire up Wireshark for a packet capture to ensure your system is querying the DNS server(s) you specify.
-----
-===== Whitelist Recommendations =====
-These are a few domains that cause issues if they end up on the various DNSBLs.
-You can easily copy and paste them into the "custom list" as described above.  If you ended up using the pfBlockerNG wizard, BBCan actually incorporated these recommendations already.  If you have no plans to use some of them (based off their name alone), you can and should omit them from your whitelist.
-<code>
-s3.amazonaws.com
-s3-1.amazonaws.com # CNAME for (s3.amazonaws.com)
-.github.com
-.githubusercontent.com
-github.map.fastly.net # CNAME for (raw.githubusercontent.com)
-.apple.com
-.sourceforge.net
-.fls-na.amazon.com # alexa
-.control.kochava.com # alexa 2
-.device-metrics-us-2.amazon.com # alexa 3
-.amazon-adsystem.com # amazon app ads
-.px.moatads.com # amazon app 2
-.wildcard.moatads.com.edgekey.net # CNAME for (px.moatads.com)
-.e13136.g.akamaiedge.net # CNAME for (px.moatads.com)
-.secure-gl.imrworldwide.com # amazon app 3
-.pixel.adsafeprotected.com # amazon app 4
-.anycast.pixel.adsafeprotected.com # CNAME for (pixel.adsafeprotected.com)
-.bs.serving-sys.com # amazon app 5
-.bs.eyeblaster.akadns.net # CNAME for (bs.serving-sys.com)
-.bsla.eyeblaster.akadns.net # CNAME for (bs.serving-sys.com)
-.adsafeprotected.com # amazon app 6
-.anycast.static.adsafeprotected.com # CNAME for (static.adsafeprotected.com)
-google.com
-www.google.com
-youtube.com
-www.youtube.com
-youtube-ui.l.google.com # CNAME for (youtube.com)
-stackoverflow.com
-www.stackoverflow.com
-dropbox.com
-www.dropbox.com
-www.dropbox-dns.com # CNAME for (dropbox.com)
-.adsafeprotected.com
-control.kochava.com
-secure-gl.imrworldwide.com
-pbs.twimg.com # twitter images
-www.pbs.twimg.com # twitter images
-cs196.wac.edgecastcdn.net # CNAME for (pbs.twimg.com)
-cs2-wac.apr-8315.edgecastdns.net # CNAME for (pbs.twimg.com)
-cs2-wac-us.8315.ecdns.net # CNAME for (pbs.twimg.com)
-cs45.wac.edgecastcdn.net # CNAME for (pbs.twimg.com)
-</code>
-----
-===== TLD Blacklisting =====
-TLD (top-level domain) blacklisting is another option in DNSBL.
-Don’t forget you need to **Enable** the TLD option at the top of the DNSBL configuration page to use the features discussed here.
-Static blacklisting is not normally advocated because the bad guys will simply move around it, TLD blacklisting is a rare instance where you can eliminate some potential attack vectors although its usefulness depends entirely on your situation.  TLDs are the characters after the last dot on a domain name, e.g. com, net, and biz are some common ones.  The number of TLDs has skyrocketed and there were well over 1,500.  Over time, some TLDs have become wastelands for nefarious activity such as command and control servers.  If you no plans to connect with a particular TLD and it has shown to be less than reputable, i.e. most sane companies wouldn’t bother trying to use it for legitimate businesses, you can just go to the main DNSBL tab and block it outright.
-Some TLDs are used extensively for typosquatting — Omitting the "o" in .com could Be costly.
-If you’re looking for a little more guidance of what is ‘bad’ then look no further than Spamhaus and the website link below.  Spamhaus is constantly updating this list and related statistics so check it directly for the most up-to-date information.
-https://www.spamhaus.org/statistics/tlds/
-Suggest adding the top 3 TLDs, as they are used often for
-<code>
-cm
-party
-click
-link
-</code>
-Adding these others would likely not cause too many issues, although keep in mind that you will see false positives:
-<code>
-technology
-gdn
-study
-men
-biz
-reise
-stream
-</code>
-----
-===== References =====
-https://www.linuxincluded.com/block-ads-malvertising-on-pfsense-using-pfblockerng-dnsbl/
-https://github.com/jmdugan/blocklists/blob/master/corporations/facebook/all