User Tools

Site Tools


pfsense:pfblockerng:block_ads

This is an old revision of the document!


PFSense - pfBlockingNG - Block Ads

Use the DNSBL portion of pfBlockerNG to remove advertising.

It essentially creates functionality similar to the pi-Hole project just using your pfSense + pfBlockerNG.

WARNING: DO NOT install the latest version of pfBlockerNG unless you are on the most up-to-date version of pfSense.

This is especially important if you are on a pfSense before 2.4.4.

Version 2.4.4 introduced PHP 7.2 and it broke a lot of packages, not just pfBlockerNG.

Upgrading from a previous version

If you installing pfBlockerNG for the first time, skip this step and go to installation.

If you have quite a few custom settings such as rules, IPv4 lists, and DNSBL lists and you want to keep all of your settings, go to Firewall → pfBlockerNG (General) and make sure Keep Settings is checked. If it's not, put a check there and click 'Save' at the bottom.

NOTE: Unless you have a very complex setup, it might be better to not use the 'Keep settings' options and setup pfBlockerNG from scratch.

Remove the package: Go to System → Package Manager and delete the package.


Install pfBlockerNG

Go to System → Package Manager → Available Packages and type pfblocker into the search criteria and then click search.

To have the latest version, suggestion is to get the development version; which still works very well even in production environments.

In this case click install on the version with -devel at the end of it or the package or you will be installing might be an older one!

NOTE: If you do not see pfBlockerNG-devel in the list of available packages, you can also try running pkg update -f from the command line.

Click Confirm and let the package install. This will take a bit of time as it has to download several files and databases. Wait a few minutes and you should see “success” once the installation is complete.

The package should be installed.


Configure pfBlockingNG

Go to the configuration page Firewall → pfBlockerNG.

Assuming you didn’t upgrade, you will receive the config wizard; which is literally 4 steps and will get you started.

WARNING: Pay special attention to the interface step/page if you have a non-standard setup or if you want to enable pfBlockerNG on multiple interfaces.

Finish up the wizard and you will be automatically directed to the update page (Firewall → pfBlockingNG → Update).

The update will likely take a little bit to complete as it is downloading the various IP and DNSBL feeds associated with the wizard setup.

Once the feeds are downloaded, the text in the gray box will stop scrolling and you will see “UPDATE PROCESS ENDED” at the very bottom along with your current date and time.

Now go back to the main pfBlockerNG page by going to General or by clicking Firewall → pfBlockerNG.

The Enable and Keep Settings checkboxes should already be checked.


Further configuring DNSBL

Go to the DNSBL tab and it will take you to the main DNSBL landing page.

You should also notice there is already a checkmark in Enable next to DNSBL.

If you only have one internal interface such as LAN, then you shouldn't need to do anything else.

If you have multiple internal interfaces and you would like to protect them with DNSBL, then you will need to pay attention to the Permit Firewall Rules section below. You may have already selected the “extra” interfaces when you went through the wizard above. Either way, keep this in mind should you ever add interfaces or VLANs in the future!

If you do need to add interfaces, place a checkmark in the Enable box (Permit Firewall Rules). Then, select the various interfaces (to the right) by holding down the Ctrl key and left-clicking. Don’t forget to hit Save DNSBL settings and move to the DNSBL feeds section.


Configure TLD (Optional)

If your pfSense has plenty of memory, another really amazing feature to consider is TLD (below the DNSBL option).

This option is required for the TLD blacklists.

What does the TLD feature provide? Normally, DNSBL (and other DNS blackhole software) block the domains specified in the feeds and that's that. What TLD does differently is it will block the domain specified in addition to all of a domain’s subdomains. As a result, a bad guy can’t circumvent the blacklist by creating a random subdomain name.

pfBlockingNG is one of the few DNS blackholing software that does this.

You can get an idea on memory requirements by clicking on the blue info icon next to TLD. If you have less than 2GB of memory on your pfSense, I would skip it. If you’re unsure on your memory, this might be a feature to come back to after you get your feeds and everything else configured.

TLD can definitely add several layers of protection.


Configuring DNSBL feeds

Before we go adding additional feeds, we should at least understand what the wizard provided us.

Go to DNSBL → DNSBL Groups to see the current (post-wizard) configuration.

if you want to add more, go to Feeds (not DNSBL Groups) at the top menu. Here you will see all of the pre-configured feeds for the IPv4, IPv6, and DNSBL categories. And yes, there are a bunch of them! You’ll also see custom, user defined feeds at the very bottom if you performed an upgrade and it was unable to match a feed to an existing feed.

NOTE: If you don’t have a Feeds sub-menu, that most likely means you are still on the older version of pfBlockerNG.

Another way to check is if you have Alerts instead of Reports along the top row of pfBlockerNG options… That too means you are still on the old version.

You can either follow the walkthrough for the older version of pfBlockerNG or better yet, delete the old pfBlockerNG and install the pfBlockerNG-devel package.

Scroll down to the DNSBL Category header, which is *after* all of the IPv4 and IPv6 sections. The first DNSBL sub-category you should see is labeled EasyList.

Note that EasyList has a checkbox near the top left. This means the alias/group or category already exists and is being used.

If you look toward the right, you will see another checkbox. This means the individual feed is enabled.

This subtle distinction is extremely important to understanding how aliases and feeds work. In addition, if a category ever has a problematic feed, you can always disable that feed instead of the entire category, i.e. we do not need to enable every feed for a particular category.

For example, if you want to add the EasyList Adware Filter or one of the language specific feeds, you would click the + sign to the far right and that would add the individual feed to the already existing EasyList group.

Ensure you switch OFF to ON and then click Save at the bottom of the screen.

WARNING: You can add as many feeds as you like, but keep in mind that too many feeds can potentially slow down your firewall.

It’s quite possible just adding a few categories by themselves is too much for a resource starved firewall!

This is because feeds are periodically downloaded and likewise, unbound is reloaded regularly.

If you using a system with limited resources (mainly RAM), you need to be extra careful.

When in doubt, add feeds slowly and keep an eye on memory, CPU, etc.


Add Feed hphosts

If we go back to the Feeds, a category (group) recommend adding is hpHosts. Click the + next to the hpHosts header (top left) to add all the feeds related to this category.

After clicking the + next to the hpHosts category, you are taken to a DNSBL feeds page with all of the feeds under that category pre-populated. All of the feeds in the list will initially be in the OFF state. You can go through and enable each one individually or you can click Enable All at the bottom of the list.

Make sure you switch the Action from Disabled to Unbound (below).

Click Save DNSBL Settings at the bottom of the page and you should receive a message at the top along the lines of Saved [ Type:DNSBL, Name:hpHosts ] configuration.

Click on the DNSBL Groups tab and you will be taken to the DNSBL feeds summary. Assuming everything went as planned, your feeds summary should include the hphosts.


Other items worth mentioning

If you take a look at the Malicious category, you will notice that some feeds have selectable options, such as such as the SANS Internet Storm Center feeds (bullet points). I personally recommend switching the feed from ISC_SDH (high) to ISC_SDL (low) as the high feed has under 20 entries and the low feed includes the high feed.

I addition, I haven’t seen many false positives when using the expanded (low) list.

Take note of the door-arrow graphic icons next to several feeds. The door-arrow graphic means the feed is a subscription feed, which at the very least means you need to register for it. Some subscription feeds also have a fee associated with them. Subscription feeds can have a lower false positive rate and are typically updated on a more frequent basis. You will see selectable options and subscription feeds throughout the DNSBL feeds so it is important to understand what these graphics mean.


  • hpHosts (all of them) – From MalwareBytes.
  • BBcan177 – From the creator of pfBlockerNG.
  • BBC (BBC_DGA_Agr) – From Bambenek Consulting ← This feed is extremely large.
  • Cryptojackers (all of them) – This blocks cryptojacking software and in-browser miners, but it also blocks various coin exchanges.

ALERT: You can add as many feeds as you like, but keep in mind that too many feeds can potentially slow down your firewall!

If you using a system with limited resources (mainly RAM), you need to be extra careful.

When in doubt, add feeds slowly and keep an eye on memory, CPU, etc


Problem Solving a Feed

If you ever experience issues with a particular feed, go to DNSBL → DNSBL Groups and then click the pencil/edit icon next to that particular category.

Once in the category edit screen, simply switch those feeds to OFF and then click save at the bottom.

You could also delete those feeds.


Forcing DNSBL feed updates

Anytime you make changes, you can either wait for the next update or you can force the changes yourself.

To force the changes, go over to the Update tab within pfBlockerNG.

Heed the warning and make sure you are not going to run the updates near the time your cron job would automatically run. If the countdown timer is less than 5 minutes, I would not recommend running it and instead just wait for the system to run it automatically.

Assuming you are good on the time, go ahead and click the Run button. You will see progress updates in the gray window below including the number of domains downloaded for each list, when the list was last updated, etc. Also note that pfBlockerNG is smart enough to check for and eliminate duplicate DNS (# Dups) entries between the lists.


Testing By Browsing

So what does the finished product look like?

On many sites, you’ll see gray boxes where an ad normally would have been.

A browser add-on like uBlock Origin further cleans this up by removing the gray box entirely and it also provides some secondary protections.

Also keep in mind that some ads are still served such as video ads on YouTube.

Those ads cannot be blocked via pfBlockerNG since the ad content is served from the same domain names (DNS) as the video content.


Testing from the command line

Normally, pinging a site will return the sites actual IP address.

However, with pfBlockerNG properly setup you may instead see a reply of 10.10.10.1, which is the default virtual IP address DNSBL creates.


Statistics and graphs

pfBlockerNG has some really fantastic graphs built-in as shown below.

You can even see the top blocked domains, source IPs with the most blocks, blocked user agent strings, TLDs, and much more.

Also helpful is you need to whittle down the number of feeds you are using, i.e. this feed accounts for 50% of your blocks and it’s a third the size of these other two feeds combined. Just go to Firewall → pfBlockerNG → Reports → DNSBL Stats to see all the DNSBL eye candy, aka graphs/stats.


Troubleshooting - Whitelisting

You can remove the offending list entirely (DNSBL → DNSBL Groups → Edit the list in question) or more preferably, you can just whitelist the domain.

The absolute easiest way to do this is by going to the Reports tab and scrolling down to the DNSBL section.

Clicking on the red lock will temporarily unlock the domain so you can verify if it is indeed the domain that needs to be whitelisted.

Clicking the + will add the domain to the DNSBL whitelist.

When clicking the + you will then receive a prompt about whether you want to perform a wildcard whitelist or just a whitelist.

Read the explanation, but typically use whitelist because it is more exact and less prone to letting something past.

Adding a description so you know what was broken and/or why you fixed it, i.e. today it makes perfect sense, but it might not 6 months from now.

If you go back to the main DNSBL tab and expand the DNSBL Whitelist section toward the bottom, you should now see the domain you whitelisted.

You might also notice that if the domain you are whitelisting has CNAME records, pfBlockerNG is smart enough to add those too.

Simply type each domain in on a separate line and then click Save if you know which domains to whitelist. If you want the whitelist additions/changes to occur sooner rather than later, you will also need to go back to the Update tab and click Run. If you don’t want to do the trial and error on your own see some whitelist recommendations below.

It's also worth mentioning that if a system already resolved the domain name on your system and it is previously resolved to 10.10.10.1, then you may need to clear your local DNS cache, your browser cache, or both. To clear your machine’s cache, from a command line on Windows, type in ipconfig /flushdns and that should take care of it. You can run a similar command on a Linux system, although the commands can vary from one installation to the next. More often than not, simply restarting your network interface will work; on most distributions, service networking restart or systemctl restart network should take care of it for you. Each browser has a slightly different way to clear the cache, however, all of them allow you to pull a new version of the website if you hold down Shift while clicking on the refresh/reload button.

If ads are not getting blocked and the ping commands above don’t return the virtual IP address, it’s also possible your local machine is not using pfSense for its DNS settings. If you are using Windows, check your network settings and make sure it is set to your pfSense IP address. On Linux/*nix, check your /etc/resolv.conf or even Network Manager (if using a GUI). If you are not using pfSense for your DHCP server, you may need to do some digging.

Browsers can also get in the way especially with the advent of DNS over HTTPS. If you find your ping tests work, but your browser doesn’t, then that is most likely your issue. Although somewhat uncommon, some anti-virus packages and endpoint protection can mess with your DNS settings too. Furthermore, those changes may not necessarily be reflected in your operating system’s DNS settings. For example, Avast Premier has a Secure DNS feature that will force your browser to use Avast specified DNS servers in an effort to prevent DNS hijacking. If you find that other devices on your network are blocking ads and one particular device doesn’t, then your anti-virus or endpoint protection very well may be the culprit. When all else fails, you can always fire up Wireshark for a packet capture to ensure your system is querying the DNS server(s) you specify.


Whitelist Recommendations

These are a few domains that cause issues if they end up on the various DNSBLs.

You can easily copy and paste them into the “custom list” as described above. If you ended up using the pfBlockerNG wizard, BBCan actually incorporated these recommendations already. If you have no plans to use some of them (based off their name alone), you can and should omit them from your whitelist.

s3.amazonaws.com
s3-1.amazonaws.com # CNAME for (s3.amazonaws.com)
.github.com
.githubusercontent.com 
github.map.fastly.net # CNAME for (raw.githubusercontent.com)
.apple.com 
.sourceforge.net
.fls-na.amazon.com # alexa
.control.kochava.com # alexa 2
.device-metrics-us-2.amazon.com # alexa 3
.amazon-adsystem.com # amazon app ads
.px.moatads.com # amazon app 2
.wildcard.moatads.com.edgekey.net # CNAME for (px.moatads.com)
.e13136.g.akamaiedge.net # CNAME for (px.moatads.com)
.secure-gl.imrworldwide.com # amazon app 3
.pixel.adsafeprotected.com # amazon app 4
.anycast.pixel.adsafeprotected.com # CNAME for (pixel.adsafeprotected.com)
.bs.serving-sys.com # amazon app 5
.bs.eyeblaster.akadns.net # CNAME for (bs.serving-sys.com)
.bsla.eyeblaster.akadns.net # CNAME for (bs.serving-sys.com)
.adsafeprotected.com # amazon app 6
.anycast.static.adsafeprotected.com # CNAME for (static.adsafeprotected.com)
google.com
www.google.com
youtube.com
www.youtube.com
youtube-ui.l.google.com # CNAME for (youtube.com)
stackoverflow.com
www.stackoverflow.com
dropbox.com
www.dropbox.com
www.dropbox-dns.com # CNAME for (dropbox.com)
.adsafeprotected.com
control.kochava.com
secure-gl.imrworldwide.com
pbs.twimg.com # twitter images
www.pbs.twimg.com # twitter images
cs196.wac.edgecastcdn.net # CNAME for (pbs.twimg.com)
cs2-wac.apr-8315.edgecastdns.net # CNAME for (pbs.twimg.com)
cs2-wac-us.8315.ecdns.net # CNAME for (pbs.twimg.com)
cs45.wac.edgecastcdn.net # CNAME for (pbs.twimg.com)

TLD Blacklisting

TLD (top-level domain) blacklisting is another option in DNSBL.

Don’t forget you need to Enable the TLD option at the top of the DNSBL configuration page to use the features discussed here.

Static blacklisting is not normally advocated because the bad guys will simply move around it, TLD blacklisting is a rare instance where you can eliminate some potential attack vectors although its usefulness depends entirely on your situation. TLDs are the characters after the last dot on a domain name, e.g. com, net, and biz are some common ones. The number of TLDs has skyrocketed and there were well over 1,500. Over time, some TLDs have become wastelands for nefarious activity such as command and control servers. If you no plans to connect with a particular TLD and it has shown to be less than reputable, i.e. most sane companies wouldn’t bother trying to use it for legitimate businesses, you can just go to the main DNSBL tab and block it outright.

Some TLDs are used extensively for typosquatting — Omitting the “o” in .com could Be costly.

If you’re looking for a little more guidance of what is ‘bad’ then look no further than Spamhaus and the website link below. Spamhaus is constantly updating this list and related statistics so check it directly for the most up-to-date information.

https://www.spamhaus.org/statistics/tlds/

Suggest adding the top 3 TLDs, as they are used often for

cm
party
click
link

Adding these others would likely not cause too many issues, although keep in mind that you will see false positives:

technology
gdn
study
men
biz
reise
stream

References

pfsense/pfblockerng/block_ads.1585064578.txt.gz · Last modified: 2020/07/15 09:30 (external edit)

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki