X-Pi-hole removed from blocking page...

Signed-off-by: Adam Warner <me@adamwarner.co.uk>

.github/ISSUE_TEMPLATE.md

@@ -1,37 +0,0 @@
-**In raising this issue, I confirm the following:** `{please fill the checkboxes, e.g: [X]}`
-
-- [] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md).
-- [] The issue I am reporting can be *replicated*.
-- [] The issue I am reporting isn't a duplicate (see [FAQs](https://github.com/pi-hole/pi-hole/wiki/FAQs), [closed issues](https://github.com/pi-hole/pi-hole/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), and [open issues](https://github.com/pi-hole/pi-hole/issues)).
-
-**How familiar are you with the the source code relevant to this issue?:**
-
-`{Replace this with a number from 1 to 10. 1 being not familiar, and 10 being very familiar}`
-
----
-**Expected behavior:**
-
-`{A detailed description of what you expect to see}`
-
-**Actual behavior:**
-
-`{A detailed description and/or screenshots of what you do see}`
-
-**Steps to reproduce:**
-
-`{Detailed steps of how we can reproduce this}`
-
-**Debug token provided by [uploading `pihole -d` log](https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#debug):**
-
-`{Alphanumeric token}`
-
-**Troubleshooting undertaken, and/or other relevant information:**
-
-`{Steps of what you have done to fix this}`
-
-> * `{Please delete this quoted section when opening your issue}`
-> * You must follow the template instructions. Failure to do so will result in your issue being closed.
-> * Please [submit any feature requests here](https://discourse.pi-hole.net/c/feature-requests), so it is votable and trackable by the community.
-> * Please respect that Pi-hole is developed by volunteers, who can only reply in their spare time.
-> * Detail helps us understand and resolve an issue quicker, but please ensure it's relevant.
-> * _This template was created based on the work of [`udemy-dl`](https://github.com/nishad/udemy-dl/blob/master/LICENSE)._

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,31 +0,0 @@
-**By submitting this pull request, I confirm the following:** 
-*please fill any appropriate checkboxes, e.g: [X]*
-
-- [ ] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md), as well as this entire template.
-- [ ] I have made only one major change in my proposed changes.
-- [ ] I have commented my proposed changes within the code.
-- [ ] I have tested my proposed changes, and have included unit tests where possible.
-- [ ] I am willing to help maintain this change if there are issues with it later.
-- [ ] I give this submission freely and claim no ownership.
-- [ ] It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1)
-- [ ] I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
-
-Please make sure you [Sign Off](https://docs.pi-hole.net/guides/github/how-to-signoff/) all commits. Pi-hole enforces the [DCO](https://docs.pi-hole.net/guides/github/contributing/).
-
----
-**What does this PR aim to accomplish?:**
-*A detailed description, screenshots (if necessary), as well as links to any relevant GitHub issues*
-
-
-**How does this PR accomplish the above?:**
-*A detailed description (such as a changelog) and screenshots (if necessary) of the implemented fix*
-
-
-**What documentation changes (if any) are needed to support this PR?:**
-*A detailed list of any necessary changes*
-
-
----
-* You must follow the template instructions. Failure to do so will result in your pull request being closed.
-* Please respect that Pi-hole is developed by volunteers, who can only reply in their spare time.
-

.github/dependabot.yml

@@ -0,0 +1,10 @@
+version: 2
+updates:
+- package-ecosystem: github-actions
+  directory: "/"
+  schedule:
+    interval: weekly
+    day: saturday
+    time: "10:00"
+  open-pull-requests-limit: 10
+  target-branch: developement

.github/workflows/codeql-analysis.yml

@@ -0,0 +1,40 @@
+name: "CodeQL"
+
+on:
+  push:
+    branches:
+      - master
+      - development
+  pull_request:
+    branches:
+      - master
+      - development
+  schedule:
+    - cron: '32 11 * * 6'
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: ubuntu-latest
+
+    permissions:
+      actions: read
+      contents: read
+      security-events: write
+
+    steps:
+    -
+      name: Checkout repository
+      uses: actions/checkout@v2
+    # Initializes the CodeQL tools for scanning.
+    -
+      name: Initialize CodeQL
+      uses: github/codeql-action/init@v1
+      with:
+        languages: 'python'
+    -
+      name: Autobuild
+      uses: github/codeql-action/autobuild@v1
+    -
+      name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v1

.github/workflows/test.yml

@@ -0,0 +1,48 @@
+name: Test Supported Distributions
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+
+jobs:
+  smoke-test:
+    if: github.event.pull_request.draft == false
+    runs-on: ubuntu-latest
+    steps:
+    -
+      name: Checkout repository
+      uses: actions/checkout@v2
+    -
+      name: Run Smoke Tests
+      run: |
+        # Ensure scripts in repository are executable
+        IFS=$'\n';
+        for f in $(find . -name '*.sh'); do if [[ ! -x $f ]]; then echo "$f is not executable" && FAIL=1; fi ;done
+        unset IFS;
+        # If FAIL is 1 then we fail.
+        [[ $FAIL == 1 ]] && exit 1 || echo "Smoke Tests Passed"
+
+  distro-test:
+    if: github.event.pull_request.draft == false
+    runs-on: ubuntu-latest
+    needs: smoke-test
+    strategy:
+      matrix:
+        distro: [debian_9, debian_10, debian_11, ubuntu_16, ubuntu_18, ubuntu_20, ubuntu_21, centos_7, centos_8, fedora_32, fedora_33]
+    env:
+      DISTRO: ${{matrix.distro}}
+    steps:
+    -
+      name: Checkout repository
+      uses: actions/checkout@v2
+    -
+      name: Set up Python 3.7
+      uses: actions/setup-python@v2
+      with:
+        python-version: 3.7
+    -
+      name: Install dependencies
+      run: pip install -r test/requirements.txt
+    -
+      name: Test with tox
+      run: tox -c test/tox.${DISTRO}.ini

.gitignore

@@ -7,70 +7,5 @@ __pycache__
 .tox
 .eggs
 *.egg-info
-
+.idea/
-
-# Created by https://www.gitignore.io/api/jetbrains+iml
-
-### JetBrains+iml ###
-# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
-# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
-
-# All idea files, with exceptions
-.idea
-!.idea/codeStyles/*
-!.idea/codeStyleSettings.xml
-
-
-# Sensitive or high-churn files:
-.idea/**/dataSources/
-.idea/**/dataSources.ids
-.idea/**/dataSources.xml
-.idea/**/dataSources.local.xml
-.idea/**/sqlDataSources.xml
-.idea/**/dynamic.xml
-.idea/**/uiDesigner.xml
-
-# Gradle:
-.idea/**/gradle.xml
-.idea/**/libraries
-
-# CMake
-cmake-build-debug/
-
-# Mongo Explorer plugin:
-.idea/**/mongoSettings.xml
-
-## File-based project format:
-*.iws
-
-## Plugin-specific files:
-
-# IntelliJ
-/out/
-
-# mpeltonen/sbt-idea plugin
-.idea_modules/
-
-# JIRA plugin
-atlassian-ide-plugin.xml
-
-# Cursive Clojure plugin
-.idea/replstate.xml
-
-# Ruby plugin and RubyMine
-/.rakeTasks
-
-# Crashlytics plugin (for Android Studio and IntelliJ)
-com_crashlytics_export_strings.xml
-crashlytics.properties
-crashlytics-build.properties
-fabric.properties
-
-### JetBrains+iml Patch ###
-# Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-249601023
-
 *.iml
-.idea/misc.xml
-*.ipr
-
-# End of https://www.gitignore.io/api/jetbrains+iml

.idea/codeStyleSettings.xml

@@ -1,25 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<project version="4">
-    <component name="ProjectCodeStyleSettingsManager">
-        <option name="PER_PROJECT_SETTINGS">
-            <value>
-                <option name="OTHER_INDENT_OPTIONS">
-                    <value>
-                        <option name="INDENT_SIZE" value="2" />
-                        <option name="CONTINUATION_INDENT_SIZE" value="8" />
-                        <option name="TAB_SIZE" value="2" />
-                        <option name="USE_TAB_CHARACTER" value="false" />
-                        <option name="SMART_TABS" value="false" />
-                        <option name="LABEL_INDENT_SIZE" value="0" />
-                        <option name="LABEL_INDENT_ABSOLUTE" value="false" />
-                        <option name="USE_RELATIVE_INDENTS" value="false" />
-                    </value>
-                </option>
-                <MarkdownNavigatorCodeStyleSettings>
-                    <option name="RIGHT_MARGIN" value="72" />
-                </MarkdownNavigatorCodeStyleSettings>
-            </value>
-        </option>
-        <option name="USE_PER_PROJECT_SETTINGS" value="true" />
-    </component>
-</project>

.idea/codeStyles/Project.xml

@@ -1,7 +0,0 @@
-<component name="ProjectCodeStyleConfiguration">
-  <code_scheme name="Project" version="173">
-    <MarkdownNavigatorCodeStyleSettings>
-      <option name="RIGHT_MARGIN" value="72" />
-    </MarkdownNavigatorCodeStyleSettings>
-  </code_scheme>
-</component>

.idea/codeStyles/codeStyleConfig.xml

@@ -1,5 +0,0 @@
-<component name="ProjectCodeStyleConfiguration">
-  <state>
-    <option name="USE_PER_PROJECT_SETTINGS" value="true" />
-  </state>
-</component>

.stickler.yml

@@ -3,3 +3,4 @@ linters:
     shell: bash
   phpcs:
   flake8:
+    max-line-length: 120

.travis.yml

@@ -1,12 +0,0 @@
-sudo: required
-services:
-  - docker
-language: python
-python:
-  - "3.6"
-install:
-  - pip install -r requirements.txt
-
-script:
-  # tox.ini handles setup, ordering of docker build first, and then run tests
-  - tox

CONTRIBUTING.md

@@ -4,8 +4,8 @@ Please read and understand the contribution guide before creating an issue or pu
 
 ## Etiquette
 
-- Our goal for Pi-hole is **stability before features**.  This means we focus on squashing critical bugs before adding new features.  Often, we can do both in tandem, but bugs will take priority over a new feature.
+- Our goal for Pi-hole is **stability before features**. This means we focus on squashing critical bugs before adding new features. Often, we can do both in tandem, but bugs will take priority over a new feature.
-- Pi-hole is open source and [powered by donations](https://pi-hole.net/donate/), and as such, we give our **free time** to build, maintain, and **provide user support** for this project.  It would be extremely unfair for us to suffer abuse or anger for our hard work, so please take a moment to consider that.
+- Pi-hole is open source and [powered by donations](https://pi-hole.net/donate/), and as such, we give our **free time** to build, maintain, and **provide user support** for this project. It would be extremely unfair for us to suffer abuse or anger for our hard work, so please take a moment to consider that.
 - Please be considerate towards the developers and other users when raising issues or presenting pull requests.
 - Respect our decision(s), and do not be upset or abusive if your submission is not used.
 
@@ -26,7 +26,7 @@ When requesting or submitting new features, first consider whether it might be u
 
 - Check the codebase to ensure that your feature doesn't already exist.
 - Check the pull requests to ensure that another person hasn't already submitted the feature or fix.
-- Read and understand the [DCO guidelines](https://github.com/pi-hole/pi-hole/wiki/Contributing-to-the-project) for the project.
+- Read and understand the [DCO guidelines](https://docs.pi-hole.net/guides/github/contributing/) for the project.
 
 ## Technical Requirements
 
@@ -36,3 +36,77 @@ When requesting or submitting new features, first consider whether it might be u
 - Commit Unix line endings.
 - Please use the Pi-hole brand: **Pi-hole** (Take a special look at the capitalized 'P' and a low 'h' with a hyphen)
 - (Optional fun) keep to the theme of Star Trek/black holes/gravity.
+
+## Forking and Cloning from GitHub to GitHub
+
+1. Fork <https://github.com/pi-hole/pi-hole/> to a repo under a namespace you control, or have permission to use, for example: `https://github.com/<your_namespace>/<your_repo_name>/`. You can do this from the github.com website.
+2. Clone `https://github.com/<your_namespace>/<your_repo_name>/` with the tool of you choice.
+3. To keep your fork in sync with our repo, add an upstream remote for pi-hole/pi-hole to your repo.
+
+    ```bash
+    git remote add upstream https://github.com/pi-hole/pi-hole.git
+    ```
+
+4. Checkout the `development` branch from your fork `https://github.com/<your_namespace>/<your_repo_name>/`.
+5. Create a topic/branch, based on the `development` branch code. *Bonus fun to keep to the theme of Star Trek/black holes/gravity.*
+6. Make your changes and commit to your topic branch in your repo.
+7. Rebase your commits and squash any insignificant commits. See the notes below for an example.
+8. Merge `development` your branch and fix any conflicts.
+9. Open a Pull Request to merge your topic branch into our repo's `development` branch.
+
+- Keep in mind the technical requirements from above.
+
+## Forking and Cloning from GitHub to other code hosting sites
+
+- Forking is a GitHub concept and cannot be done from GitHub to other git-based code hosting sites. However, those sites may be able to mirror a GitHub repo.
+
+1. To contribute from another code hosting site, you must first complete the steps above to fork our repo to a GitHub namespace you have permission to use, for example: `https://github.com/<your_namespace>/<your_repo_name>/`.
+2. Create a repo in your code hosting site, for example: `https://gitlab.com/<your_namespace>/<your_repo_name>/`
+3. Follow the instructions from your code hosting site to create a mirror between `https://github.com/<your_namespace>/<your_repo_name>/` and `https://gitlab.com/<your_namespace>/<your_repo_name>/`.
+4. When you are ready to create a Pull Request (PR), follow the steps `(starting at step #6)` from [Forking and Cloning from GitHub to GitHub](#forking-and-cloning-from-github-to-github) and create the PR from `https://github.com/<your_namespace>/<your_repo_name>/`.
+
+## Notes for squashing commits with rebase
+
+- To rebase your commits and squash previous commits, you can use:
+
+    ```bash
+    git rebase -i your_topic_branch~(number of commits to combine)
+    ```
+
+- For more details visit [gitready.com](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html)
+
+1. The following would combine the last four commits in the branch `mytopic`.
+
+    ```bash
+    git rebase -i mytopic~4
+    ```
+
+2. An editor window opens with the most recent commits indicated: (edit the commands to the left of the commit ID)
+
+    ```gitattributes
+    pick 9dff55b2 existing commit comments
+    squash ebb1a730 existing commit comments
+    squash 07cc5b50 existing commit comments
+    reword 9dff55b2 existing commit comments
+    ```
+
+3. Save and close the editor. The next editor window opens: (edit the new commit message). *If you select reword for a commit, an additional editor window will open for you to edit the comment.*
+
+    ```bash
+    new commit comments
+    Signed-off-by: yourname <your email address>
+    ```
+
+4. Save and close the editor for the rebase process to execute. The terminal output should say something like the following:
+
+    ```bash
+    Successfully rebased and updated refs/heads/mytopic.
+    ```
+
+5. Once you have a successful rebase, and before you sync your local clone, you have to force push origin to update your repo:
+
+    ```bash
+    git push -f origin
+    ```
+
+6. Continue on from step #7 from [Forking and Cloning from GitHub to GitHub](#forking-and-cloning-from-github-to-github)

README.md

@@ -1,5 +1,7 @@
 <!-- markdownlint-configure-file { "MD004": { "style": "consistent" } } -->
 <!-- markdownlint-disable MD033 -->
+#
+
 <p align="center">
     <a href="https://pi-hole.net/">
         <img src="https://pi-hole.github.io/graphics/Vortex/Vortex_with_Wordmark.svg" width="150" height="260" alt="Pi-hole">
@@ -9,11 +11,9 @@
 </p>
 <!-- markdownlint-enable MD033 -->
 
-#
+The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) that protects your devices from unwanted content without installing any client-side software.
 
-The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) that protects your devices from unwanted content, without installing any client-side software.
+- **Easy-to-install**: our versatile installer walks you through the process and takes less than ten minutes
-
-- **Easy-to-install**: our versatile installer walks you through the process, and [takes less than ten minutes](https://www.youtube.com/watch?v=vKWjx1AQYgs)
 - **Resolute**: content is blocked in _non-browser locations_, such as ad-laden mobile apps and smart TVs
 - **Responsive**: seamlessly speeds up the feel of everyday browsing by caching DNS queries
 - **Lightweight**: runs smoothly with [minimal hardware and software requirements](https://docs.pi-hole.net/main/prerequisites/)
@@ -22,12 +22,10 @@ The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) th
 - **Versatile**: can optionally function as a [DHCP server](https://discourse.pi-hole.net/t/how-do-i-use-pi-holes-built-in-dhcp-server-and-why-would-i-want-to/3026), ensuring *all* your devices are protected automatically
 - **Scalable**: [capable of handling hundreds of millions of queries](https://pi-hole.net/2017/05/24/how-much-traffic-can-pi-hole-handle/) when installed on server-grade hardware
 - **Modern**: blocks ads over both IPv4 and IPv6
-- **Free**: open source software which helps ensure _you_ are the sole person in control of your privacy
+- **Free**: open source software that helps ensure _you_ are the sole person in control of your privacy
 
 -----
 
-Master [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=master)](https://travis-ci.com/pi-hole/pi-hole) Development [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=development)](https://travis-ci.com/pi-hole/pi-hole)
-
 ## One-Step Automated Install
 
 Those who want to get started quickly and conveniently may install Pi-hole using the following command:
@@ -52,42 +50,46 @@ sudo bash basic-install.sh
 wget -O basic-install.sh https://install.pi-hole.net
 sudo bash basic-install.sh
 ```
+### Method 3: Using Docker to deploy Pi-hole
+Please refer to the [Pi-hole docker repo](https://github.com/pi-hole/docker-pi-hole) to use the Official Docker Images.
 
 ## [Post-install: Make your network take advantage of Pi-hole](https://docs.pi-hole.net/main/post-install/)
 
 Once the installer has been run, you will need to [configure your router to have **DHCP clients use Pi-hole as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) which ensures that all devices connecting to your network will have content blocked without any further intervention.
 
-If your router does not support setting the DNS server, you can [use Pi-hole's built-in DHCP server](https://discourse.pi-hole.net/t/how-do-i-use-pi-holes-built-in-dhcp-server-and-why-would-i-want-to/3026); just be sure to disable DHCP on your router first (if it has that feature available).
+If your router does not support setting the DNS server, you can [use Pi-hole's built-in DHCP server](https://discourse.pi-hole.net/t/how-do-i-use-pi-holes-built-in-dhcp-server-and-why-would-i-want-to/3026); be sure to disable DHCP on your router first (if it has that feature available).
 
-As a last resort, you can always manually set each device to use Pi-hole as their DNS server.
+As a last resort, you can manually set each device to use Pi-hole as their DNS server.
 
 -----
 
-## Pi-hole is free, but powered by your support
+## Pi-hole is free but powered by your support
 
-There are many reoccurring costs involved with maintaining free, open source, and privacy-respecting software; expenses which [our volunteer developers](https://github.com/orgs/pi-hole/people) pitch in to cover out-of-pocket. This is just one example of how strongly we feel about our software, as well as the importance of keeping it maintained.
+There are many reoccurring costs involved with maintaining free, open source, and privacy-respecting software; expenses which [our volunteer developers](https://github.com/orgs/pi-hole/people) pitch in to cover out-of-pocket. This is just one example of how strongly we feel about our software and the importance of keeping it maintained.
 
 Make no mistake: **your support is absolutely vital to help keep us innovating!**
 
 ### [Donations](https://pi-hole.net/donate)
 
-Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses:
+Donating using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses:
 
 ### Alternative support
 
 If you'd rather not donate (_which is okay!_), there are other ways you can help support us:
 
-- [Patreon](https://patreon.com/pihole) _Become a patron for rewards_
+- [GitHub Sponsors](https://github.com/sponsors/pi-hole/)
+- [Patreon](https://patreon.com/pihole)
+- [Hetzner Cloud](https://hetzner.cloud/?ref=7aceisRX3AzA) _affiliate link_
 - [Digital Ocean](https://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
 - [Stickermule](https://www.stickermule.com/unlock?ref_id=9127301701&utm_medium=link&utm_source=invite) _earn a $10 credit after your first purchase_
-- [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
+- [Amazon US](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
-- Spreading the word about our software, and how you have benefited from it
+- Spreading the word about our software and how you have benefited from it
 
 ### Contributing via GitHub
 
 We welcome _everyone_ to contribute to issue reports, suggest new features, and create pull requests.
 
-If you have something to add - anything from a typo through to a whole new feature, we're happy to check it out! Just make sure to fill out our template when submitting your request; the questions that it asks will help the volunteers quickly understand what you're aiming to achieve.
+If you have something to add - anything from a typo through to a whole new feature, we're happy to check it out! Just make sure to fill out our template when submitting your request; the questions it asks will help the volunteers quickly understand what you're aiming to achieve.
 
 You'll find that the [install script](https://github.com/pi-hole/pi-hole/blob/master/automated%20install/basic-install.sh) and the [debug script](https://github.com/pi-hole/pi-hole/blob/master/advanced/Scripts/piholeDebug.sh) have an abundance of comments, which will help you better understand how Pi-hole works. They're also a valuable resource to those who want to learn how to write scripts or code a program! We encourage anyone who likes to tinker to read through it and submit a pull request for us to review.
 
@@ -95,7 +97,9 @@ You'll find that the [install script](https://github.com/pi-hole/pi-hole/blob/ma
 
 ## Getting in touch with us
 
-While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets. **Please be sure to check the FAQ's** before starting a new discussion, as we do not have the spare time to reply to every request for assistance.
+While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on various social media outlets.
+
+**Please be sure to check the FAQs** before starting a new discussion, as we do not have the spare time to reply to every request for assistance.
 
 - [Frequently Asked Questions](https://discourse.pi-hole.net/c/faqs)
 - [Feature Requests](https://discourse.pi-hole.net/c/feature-requests?order=votes)
@@ -106,15 +110,30 @@ While we are primarily reachable on our [Discourse User Forum](https://discourse
 
 ## Breakdown of Features
 
+### [Faster-than-light Engine](https://github.com/pi-hole/ftl)
+
+[FTLDNS](https://github.com/pi-hole/ftl) is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
+
+Some of the statistics you can integrate include:
+
+- Total number of domains being blocked
+- Total number of DNS queries today
+- Total number of ads blocked today
+- Percentage of ads blocked
+- Unique domains
+- Queries forwarded (to your chosen upstream DNS server)
+- Queries cached
+- Unique clients
+
+Access the API via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).
+
 ### The Command Line Interface
 
-The [pihole](https://docs.pi-hole.net/core/pihole-command/) command has all the functionality necessary to be able to fully administer the Pi-hole, without the need of the Web Interface. It's fast, user-friendly, and auditable by anyone with an understanding of `bash`.
+The [pihole](https://docs.pi-hole.net/core/pihole-command/) command has all the functionality necessary to fully administer the Pi-hole, without the need of the Web Interface. It's fast, user-friendly, and auditable by anyone with an understanding of `bash`.
-
-![Pi-hole Blacklist Demo](https://pi-hole.github.io/graphics/Screenshots/blacklist-cli.gif)
 
 Some notable features include:
 
-- [Whitelisting, Blacklisting and Regex](https://docs.pi-hole.net/core/pihole-command/#whitelisting-blacklisting-and-regex)
+- [Whitelisting, Blacklisting, and Regex](https://docs.pi-hole.net/core/pihole-command/#whitelisting-blacklisting-and-regex)
 - [Debugging utility](https://docs.pi-hole.net/core/pihole-command/#debugger)
 - [Viewing the live log file](https://docs.pi-hole.net/core/pihole-command/#tail)
 - [Updating Ad Lists](https://docs.pi-hole.net/core/pihole-command/#gravity)
@@ -128,11 +147,9 @@ You can read our [Core Feature Breakdown](https://docs.pi-hole.net/core/pihole-c
 
 This [optional dashboard](https://github.com/pi-hole/AdminLTE) allows you to view stats, change settings, and configure your Pi-hole. It's the power of the Command Line Interface, with none of the learning curve!
 
-![Pi-hole Dashboard](https://pi-hole.github.io/graphics/Screenshots/pihole-dashboard.png)
-
 Some notable features include:
 
-- Mobile friendly interface
+- Mobile-friendly interface
 - Password protection
 - Detailed graphs and doughnut charts
 - Top lists of domains and clients
@@ -145,21 +162,3 @@ There are several ways to [access the dashboard](https://discourse.pi-hole.net/t
 
 1. `http://pi.hole/admin/` (when using Pi-hole as your DNS server)
 2. `http://<IP_ADDPRESS_OF_YOUR_PI_HOLE>/admin/`
-3. `http://pi.hole/` (when using Pi-hole as your DNS server)
-
-## Faster-than-light Engine
-
-FTLDNS is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
-
-Some of the statistics you can integrate include:
-
-- Total number of domains being blocked
-- Total number of DNS queries today
-- Total number of ads blocked today
-- Percentage of ads blocked
-- Unique domains
-- Queries forwarded (to your chosen upstream DNS server)
-- Queries cached
-- Unique clients
-
-The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can out find [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).

advanced/01-pihole.conf

@@ -34,11 +34,9 @@ server=@DNS2@
 
 interface=@INT@
 
-cache-size=10000
+cache-size=@CACHE_SIZE@
 
 log-queries
 log-facility=/var/log/pihole.log
 
-local-ttl=2
-
 log-async

advanced/06-rfc6761.conf

@@ -0,0 +1,41 @@
+# Pi-hole: A black hole for Internet advertisements
+# (c) 2021 Pi-hole, LLC (https://pi-hole.net)
+# Network-wide ad blocking via your own hardware.
+#
+# RFC 6761 config file for Pi-hole
+#
+# This file is copyright under the latest version of the EUPL.
+# Please see LICENSE file for your rights under this license.
+
+###############################################################################
+#      FILE AUTOMATICALLY POPULATED BY PI-HOLE INSTALL/UPDATE PROCEDURE.      #
+# ANY CHANGES MADE TO THIS FILE AFTER INSTALL WILL BE LOST ON THE NEXT UPDATE #
+#                                                                             #
+#             CHANGES SHOULD BE MADE IN A SEPARATE CONFIG FILE                #
+#                    WITHIN /etc/dnsmasq.d/yourname.conf                      #
+###############################################################################
+
+# RFC 6761: Caching DNS servers SHOULD recognize
+#     test, localhost, invalid
+# names as special and SHOULD NOT attempt to look up NS records for them, or
+# otherwise query authoritative DNS servers in an attempt to resolve these
+# names.
+server=/test/
+server=/localhost/
+server=/invalid/
+
+# The same RFC requests something similar for
+#     16.172.in-addr.arpa.  22.172.in-addr.arpa.  27.172.in-addr.arpa.
+#     17.172.in-addr.arpa.  30.172.in-addr.arpa.  28.172.in-addr.arpa.
+#     18.172.in-addr.arpa.  23.172.in-addr.arpa.  29.172.in-addr.arpa.
+#     19.172.in-addr.arpa.  24.172.in-addr.arpa.  31.172.in-addr.arpa.
+#     20.172.in-addr.arpa.  25.172.in-addr.arpa.  168.192.in-addr.arpa.
+# Pi-hole implements this via the dnsmasq option "bogus-priv" (see
+# 01-pihole.conf) because this also covers IPv6.
+
+# OpenWRT furthermore blocks    bind, local, onion    domains
+# see https://git.openwrt.org/?p=openwrt/openwrt.git;a=blob_plain;f=package/network/services/dnsmasq/files/rfc6761.conf;hb=HEAD
+# and https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml
+# We do not include the ".local" rule ourselves, see https://github.com/pi-hole/pi-hole/pull/4282#discussion_r689112972
+server=/bind/
+server=/onion/

advanced/Scripts/chronometer.sh

@@ -13,6 +13,7 @@ LC_NUMERIC=C
 
 # Retrieve stats from FTL engine
 pihole-FTL() {
+    local ftl_port LINE
     ftl_port=$(cat /run/pihole-FTL.port 2> /dev/null)
     if [[ -n "$ftl_port" ]]; then
         # Open connection to FTL
@@ -20,12 +21,13 @@ pihole-FTL() {
 
         # Test if connection is open
         if { "true" >&3; } 2> /dev/null; then
-            # Send command to FTL
+            # Send command to FTL and ask to quit when finished
-            echo -e ">$1" >&3
+            echo -e ">$1 >quit" >&3
 
-            # Read input
+            # Read input until we received an empty string and the connection is
+            # closed
             read -r -t 1 LINE <&3
-            until [[ ! $? ]] || [[ "$LINE" == *"EOM"* ]]; do
+            until [[ -z "${LINE}" ]] && [[ ! -t 3 ]]; do
                 echo "$LINE" >&1
                 read -r -t 1 LINE <&3
             done
@@ -228,15 +230,21 @@ get_sys_stats() {
         mapfile -t ph_ver_raw < <(pihole -v -c 2> /dev/null | sed -n 's/^.* v/v/p')
         if [[ -n "${ph_ver_raw[0]}" ]]; then
             ph_core_ver="${ph_ver_raw[0]}"
-            ph_lte_ver="${ph_ver_raw[1]}"
+            if [[ ${#ph_ver_raw[@]} -eq 2 ]]; then
-            ph_ftl_ver="${ph_ver_raw[2]}"
+                # AdminLTE not installed
+                ph_lte_ver="(not installed)"
+                ph_ftl_ver="${ph_ver_raw[1]}"
+            else
+                ph_lte_ver="${ph_ver_raw[1]}"
+                ph_ftl_ver="${ph_ver_raw[2]}"
+            fi
         else
             ph_core_ver="-1"
         fi
 
         sys_name=$(hostname)
 
-        [[ -n "$TEMPERATUREUNIT" ]] && temp_unit="$TEMPERATUREUNIT" || temp_unit="c"
+        [[ -n "$TEMPERATUREUNIT" ]] && temp_unit="${TEMPERATUREUNIT^^}" || temp_unit="C"
 
         # Get storage stats for partition mounted on /
         read -r -a disk_raw <<< "$(df -B1 / 2> /dev/null | awk 'END{ print $3,$2,$5 }')"
@@ -490,10 +498,6 @@ chronoFunc() {
         printFunc " RAM usage: " "$ram_perc%" "$ram_info"
         printFunc " HDD usage: " "$disk_perc" "$disk_info"
 
-        if [[ "$scr_lines" -gt 17 ]] && [[ "$chrono_width" != "small" ]]; then
-            printFunc "  LAN addr: " "${IPV4_ADDRESS/\/*/}" "$lan_info"
-        fi
-
         if [[ "$DHCP_ACTIVE" == "true" ]]; then
             printFunc "DHCP usage: " "$ph_dhcp_percent%" "$dhcp_info"
         fi
@@ -551,7 +555,7 @@ Calculates stats and displays to an LCD
 Options:
   -j, --json          Output stats as JSON formatted string
   -r, --refresh       Set update frequency (in seconds)
-  -e, --exit          Output stats and exit witout refreshing
+  -e, --exit          Output stats and exit without refreshing
   -h, --help          Display this help text"
   fi
 

advanced/Scripts/database_migration/gravity-db.sh

@@ -110,4 +110,16 @@ upgrade_gravityDB(){
 		sqlite3 "${database}" < "${scriptPath}/11_to_12.sql"
 		version=12
 	fi
+	if [[ "$version" == "12" ]]; then
+		# Add column date_updated to adlist table
+		echo -e "  ${INFO} Upgrading gravity database from version 12 to 13"
+		sqlite3 "${database}" < "${scriptPath}/12_to_13.sql"
+		version=13
+	fi
+	if [[ "$version" == "13" ]]; then
+		# Add columns number and status to adlist table
+		echo -e "  ${INFO} Upgrading gravity database from version 13 to 14"
+		sqlite3 "${database}" < "${scriptPath}/13_to_14.sql"
+		version=14
+	fi
 }

advanced/Scripts/database_migration/gravity/12_to_13.sql

@@ -0,0 +1,18 @@
+.timeout 30000
+
+PRAGMA FOREIGN_KEYS=OFF;
+
+BEGIN TRANSACTION;
+
+ALTER TABLE adlist ADD COLUMN date_updated INTEGER;
+
+DROP TRIGGER tr_adlist_update;
+
+CREATE TRIGGER tr_adlist_update AFTER UPDATE OF address,enabled,comment ON adlist
+    BEGIN
+      UPDATE adlist SET date_modified = (cast(strftime('%s', 'now') as int)) WHERE id = NEW.id;
+    END;
+
+UPDATE info SET value = 13 WHERE property = 'version';
+
+COMMIT;

advanced/Scripts/database_migration/gravity/13_to_14.sql

@@ -0,0 +1,13 @@
+.timeout 30000
+
+PRAGMA FOREIGN_KEYS=OFF;
+
+BEGIN TRANSACTION;
+
+ALTER TABLE adlist ADD COLUMN number INTEGER NOT NULL DEFAULT 0;
+ALTER TABLE adlist ADD COLUMN invalid_domains INTEGER NOT NULL DEFAULT 0;
+ALTER TABLE adlist ADD COLUMN status INTEGER NOT NULL DEFAULT 0;
+
+UPDATE info SET value = 14 WHERE property = 'version';
+
+COMMIT;

advanced/Scripts/list.sh

@@ -1,4 +1,6 @@
 #!/usr/bin/env bash
+# shellcheck disable=SC1090
+
 # Pi-hole: A black hole for Internet advertisements
 # (c) 2017 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
@@ -9,11 +11,19 @@
 # Please see LICENSE file for your rights under this license.
 
 # Globals
-basename=pihole
+piholeDir="/etc/pihole"
-piholeDir=/etc/"${basename}"
+GRAVITYDB="${piholeDir}/gravity.db"
-gravityDBfile="${piholeDir}/gravity.db"
+# Source pihole-FTL from install script
+pihole_FTL="${piholeDir}/pihole-FTL.conf"
+if [[ -f "${pihole_FTL}" ]]; then
+  source "${pihole_FTL}"
+fi
 
-reload=false
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"
+
+noReloadRequested=false
 addmode=true
 verbose=true
 wildcard=false
@@ -25,6 +35,7 @@ typeId=""
 comment=""
 declare -i domaincount
 domaincount=0
+reload=false
 
 colfile="/opt/pihole/COL_TABLE"
 source ${colfile}
@@ -112,7 +123,7 @@ ProcessDomainList() {
     for dom in "${domList[@]}"; do
         # Format domain into regex filter if requested
         if [[ "${wildcard}" == true ]]; then
-            dom="(^|\\.)${dom//\./\\.}$"
+            dom="(\\.|^)${dom//\./\\.}$"
         fi
 
         # Logic: If addmode then add to desired list and remove from the other;
@@ -231,7 +242,15 @@ Displaylist() {
 }
 
 NukeList() {
-    sqlite3 "${gravityDBfile}" "DELETE FROM domainlist WHERE type = ${typeId};"
+    count=$(sqlite3 "${gravityDBfile}" "SELECT COUNT(1) FROM domainlist WHERE type = ${typeId};")
+    listname="$(GetListnameFromTypeId "${typeId}")"
+    if [ "$count" -gt 0 ];then
+        sqlite3 "${gravityDBfile}" "DELETE FROM domainlist WHERE type = ${typeId};"
+        echo "  ${TICK} Removed ${count} domain(s) from the ${listname}"
+    else
+        echo "  ${INFO} ${listname} already empty. Nothing to do!"
+    fi
+    exit 0;
 }
 
 GetComment() {
@@ -250,7 +269,7 @@ while (( "$#" )); do
         "--white-wild" | "white-wild" ) typeId=2; wildcard=true;;
         "--wild" | "wildcard" ) typeId=3; wildcard=true;;
         "--regex" | "regex"   ) typeId=3;;
-        "-nr"| "--noreload"  ) reload=false;;
+        "-nr"| "--noreload"  ) noReloadRequested=true;;
         "-d" | "--delmode"   ) addmode=false;;
         "-q" | "--quiet"     ) verbose=false;;
         "-h" | "--help"      ) helpFunc;;
@@ -276,6 +295,6 @@ if $web; then
 echo "DONE"
 fi
 
-if [[ "${reload}" != false ]]; then
+if [[ ${reload} == true && ${noReloadRequested} == false ]]; then
     pihole restartdns reload-lists
 fi

advanced/Scripts/piholeARPTable.sh

@@ -38,7 +38,7 @@ flushARP(){
 
     # Truncate network_addresses table in pihole-FTL.db
     # This needs to be done before we can truncate the network table due to
-    # foreign key contraints
+    # foreign key constraints
     if ! output=$(sqlite3 "${DBFILE}" "DELETE FROM network_addresses" 2>&1); then
         echo -e "${OVER}  ${CROSS} Failed to truncate network_addresses table"
         echo "  Database location: ${DBFILE}"

advanced/Scripts/piholeCheckout.sh

@@ -166,12 +166,15 @@ checkout() {
         checkout_pull_branch "${webInterfaceDir}" "${2}"
     elif [[ "${1}" == "ftl" ]] ; then
         local path
+        local oldbranch
         path="${2}/${binary}"
+        oldbranch="$(pihole-FTL -b)"
 
         if check_download_exists "$path"; then
             echo "  ${TICK} Branch ${2} exists"
             echo "${2}" > /etc/pihole/ftlbranch
             chmod 644 /etc/pihole/ftlbranch
+            echo -e "  ${INFO} Switching to branch: \"${2}\" from \"${oldbranch}\""
             FTLinstall "${binary}"
             restart_service pihole-FTL
             enable_service pihole-FTL

advanced/Scripts/piholeDebug.sh

@@ -46,8 +46,9 @@ OBFUSCATED_PLACEHOLDER="<DOMAIN OBFUSCATED>"
 # FAQ URLs for use in showing the debug log
 FAQ_UPDATE_PI_HOLE="${COL_CYAN}https://discourse.pi-hole.net/t/how-do-i-update-pi-hole/249${COL_NC}"
 FAQ_CHECKOUT_COMMAND="${COL_CYAN}https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#checkout${COL_NC}"
-FAQ_HARDWARE_REQUIREMENTS="${COL_CYAN}https://discourse.pi-hole.net/t/hardware-software-requirements/273${COL_NC}"
+FAQ_HARDWARE_REQUIREMENTS="${COL_CYAN}https://docs.pi-hole.net/main/prerequisites/${COL_NC}"
-FAQ_HARDWARE_REQUIREMENTS_PORTS="${COL_CYAN}https://discourse.pi-hole.net/t/hardware-software-requirements/273#ports${COL_NC}"
+FAQ_HARDWARE_REQUIREMENTS_PORTS="${COL_CYAN}https://docs.pi-hole.net/main/prerequisites/#ports${COL_NC}"
+FAQ_HARDWARE_REQUIREMENTS_FIREWALLD="${COL_CYAN}https://docs.pi-hole.net/main/prerequisites/#firewalld${COL_NC}"
 FAQ_GATEWAY="${COL_CYAN}https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546${COL_NC}"
 FAQ_ULA="${COL_CYAN}https://discourse.pi-hole.net/t/use-ipv6-ula-addresses-for-pi-hole/2127${COL_NC}"
 FAQ_FTL_COMPATIBILITY="${COL_CYAN}https://github.com/pi-hole/FTL#compatibility-list${COL_NC}"
@@ -55,11 +56,6 @@ FAQ_BAD_ADDRESS="${COL_CYAN}https://discourse.pi-hole.net/t/why-do-i-see-bad-add
 
 # Other URLs we may use
 FORUMS_URL="${COL_CYAN}https://discourse.pi-hole.net${COL_NC}"
-TRICORDER_CONTEST="${COL_CYAN}https://pi-hole.net/2016/11/07/crack-our-medical-tricorder-win-a-raspberry-pi-3/${COL_NC}"
-
-# Port numbers used for uploading the debug log
-TRICORDER_NC_PORT_NUMBER=9999
-TRICORDER_SSL_PORT_NUMBER=9998
 
 # Directories required by Pi-hole
 # https://discourse.pi-hole.net/t/what-files-does-pi-hole-use/1684
@@ -77,15 +73,12 @@ HTML_DIRECTORY="/var/www/html"
 WEB_GIT_DIRECTORY="${HTML_DIRECTORY}/admin"
 #BLOCK_PAGE_DIRECTORY="${HTML_DIRECTORY}/pihole"
 SHM_DIRECTORY="/dev/shm"
+ETC="/etc"
 
 # Files required by Pi-hole
 # https://discourse.pi-hole.net/t/what-files-does-pi-hole-use/1684
 PIHOLE_CRON_FILE="${CRON_D_DIRECTORY}/pihole"
 
-PIHOLE_DNS_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/01-pihole.conf"
-PIHOLE_DHCP_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/02-pihole-dhcp.conf"
-PIHOLE_WILDCARD_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/03-wildcard.conf"
-
 WEB_SERVER_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/lighttpd.conf"
 WEB_SERVER_CUSTOM_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/external.conf"
 
@@ -124,6 +117,8 @@ get_ftl_conf_value() {
 
 PIHOLE_GRAVITY_DB_FILE="$(get_ftl_conf_value "GRAVITYDB" "${PIHOLE_DIRECTORY}/gravity.db")"
 
+PIHOLE_FTL_DB_FILE="$(get_ftl_conf_value "DBFILE" "${PIHOLE_DIRECTORY}/pihole-FTL.db")"
+
 PIHOLE_COMMAND="${BIN_DIRECTORY}/pihole"
 PIHOLE_COLTABLE_FILE="${BIN_DIRECTORY}/COL_TABLE"
 
@@ -138,6 +133,9 @@ PIHOLE_FTL_LOG="$(get_ftl_conf_value "LOGFILE" "${LOG_DIRECTORY}/pihole-FTL.log"
 PIHOLE_WEB_SERVER_ACCESS_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/access.log"
 PIHOLE_WEB_SERVER_ERROR_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/error.log"
 
+RESOLVCONF="${ETC}/resolv.conf"
+DNSMASQ_CONF="${ETC}/dnsmasq.conf"
+
 # An array of operating system "pretty names" that we officially support
 # We can loop through the array at any time to see if it matches a value
 #SUPPORTED_OS=("Raspbian" "Ubuntu" "Fedora" "Debian" "CentOS")
@@ -162,9 +160,6 @@ PIHOLE_PROCESSES=( "lighttpd" "pihole-FTL" )
 
 # Store the required directories in an array so it can be parsed through
 REQUIRED_FILES=("${PIHOLE_CRON_FILE}"
-"${PIHOLE_DNS_CONFIG_FILE}"
-"${PIHOLE_DHCP_CONFIG_FILE}"
-"${PIHOLE_WILDCARD_CONFIG_FILE}"
 "${WEB_SERVER_CONFIG_FILE}"
 "${WEB_SERVER_CUSTOM_CONFIG_FILE}"
 "${PIHOLE_INSTALL_LOG_FILE}"
@@ -182,7 +177,9 @@ REQUIRED_FILES=("${PIHOLE_CRON_FILE}"
 "${PIHOLE_DEBUG_LOG}"
 "${PIHOLE_FTL_LOG}"
 "${PIHOLE_WEB_SERVER_ACCESS_LOG_FILE}"
-"${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}")
+"${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}"
+"${RESOLVCONF}"
+"${DNSMASQ_CONF}")
 
 DISCLAIMER="This process collects information from your Pi-hole, and optionally uploads it to a unique and random directory on tricorder.pi-hole.net.
 
@@ -232,6 +229,7 @@ copy_to_debug_log() {
 }
 
 initialize_debug() {
+    local system_uptime
     # Clear the screen so the debug log is readable
     clear
     show_disclaimer
@@ -239,9 +237,13 @@ initialize_debug() {
     log_write "${COL_PURPLE}*** [ INITIALIZING ]${COL_NC}"
     # Timestamp the start of the log
     log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initialized."
+    # Uptime of the system
+    # credits to https://stackoverflow.com/questions/28353409/bash-format-uptime-to-show-days-hours-minutes
+    system_uptime=$(uptime | awk -F'( |,|:)+' '{if ($7=="min") m=$6; else {if ($7~/^day/){if ($9=="min") {d=$6;m=$8} else {d=$6;h=$8;m=$9}} else {h=$6;m=$7}}} {print d+0,"days,",h+0,"hours,",m+0,"minutes"}')
+    log_write "${INFO} System has been running for ${system_uptime}"
 }
 
-# This is a function for visually displaying the curent test that is being run.
+# This is a function for visually displaying the current test that is being run.
 # Accepts one variable: the name of what is being diagnosed
 # Colors do not show in the dasboard, but the icons do: [i], [✓], and [✗]
 echo_current_diagnostic() {
@@ -331,7 +333,17 @@ compare_local_version_to_git_version() {
             return 1
         fi
     else
-        :
+        # There is no git directory so check if the web interface was disabled
+        local setup_vars_web_interface
+        setup_vars_web_interface=$(< ${PIHOLE_SETUP_VARS_FILE} grep ^INSTALL_WEB_INTERFACE | cut -d '=' -f2)
+        if [[ "${pihole_component}" == "Web" ]] && [[ "${setup_vars_web_interface}" == "false" ]]; then
+            log_write "${INFO} ${pihole_component}: Disabled in setupVars.conf via INSTALL_WEB_INTERFACE=false"
+        else
+            # Return an error message
+            log_write "${COL_RED}Directory ${git_dir} doesn't exist${COL_NC}"
+            # and exit with a non zero code
+            return 1
+        fi
     fi
 }
 
@@ -366,11 +378,11 @@ get_program_version() {
     # Create a local variable so this function can be safely reused
     local program_version
     echo_current_diagnostic "${program_name} version"
-    # Evalutate the program we are checking, if it is any of the ones below, show the version
+    # Evaluate the program we are checking, if it is any of the ones below, show the version
     case "${program_name}" in
-        "lighttpd") program_version="$(${program_name} -v |& head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)"
+        "lighttpd") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)"
                     ;;
-        "php") program_version="$(${program_name} -v |& head -n1 | cut -d '-' -f1 | cut -d ' ' -f2)"
+        "php") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '-' -f1 | cut -d ' ' -f2)"
                 ;;
         # If a match is not found, show an error
         *) echo "Unrecognized program";
@@ -396,49 +408,54 @@ check_critical_program_versions() {
 os_check() {
     # This function gets a list of supported OS versions from a TXT record at versions.pi-hole.net
     # and determines whether or not the script is running on one of those systems
-    local remote_os_domain valid_os valid_version detected_os_pretty detected_os detected_version
+    local remote_os_domain valid_os valid_version detected_os detected_version cmdResult digReturnCode response
-    remote_os_domain="versions.pi-hole.net"
+    remote_os_domain=${OS_CHECK_DOMAIN_NAME:-"versions.pi-hole.net"}
-    valid_os=false
-    valid_version=false
 
-    detected_os_pretty=$(cat /etc/*release | grep PRETTY_NAME | cut -d '=' -f2- | tr -d '"')
+    detected_os=$(grep "\bID\b" /etc/os-release | cut -d '=' -f2 | tr -d '"')
-    detected_os="${detected_os_pretty%% *}"
+    detected_version=$(grep VERSION_ID /etc/os-release | cut -d '=' -f2 | tr -d '"')
-    detected_version=$(cat /etc/*release | grep VERSION_ID | cut -d '=' -f2- | tr -d '"')
 
-    IFS=" " read -r -a supportedOS < <(dig +short -t txt ${remote_os_domain} | tr -d '"')
+    cmdResult="$(dig +short -t txt "${remote_os_domain}" @ns1.pi-hole.net 2>&1; echo $?)"
+    #Get the return code of the previous command (last line)
+    digReturnCode="${cmdResult##*$'\n'}"
 
-    for i in "${supportedOS[@]}"
+    # Extract dig response
+    response="${cmdResult%%$'\n'*}"
+
+    IFS=" " read -r -a supportedOS < <(echo "${response}" | tr -d '"')
+    for distro_and_versions in "${supportedOS[@]}"
     do
-        os_part=$(echo "$i" | cut -d '=' -f1)
+        distro_part="${distro_and_versions%%=*}"
-        versions_part=$(echo "$i" | cut -d '=' -f2-)
+        versions_part="${distro_and_versions##*=}"
 
-        if [[ "${detected_os}" =~ ${os_part} ]]; then
+        if [[ "${detected_os^^}" =~ ${distro_part^^} ]]; then
-          valid_os=true
+            valid_os=true
-          IFS="," read -r -a supportedVer <<<"${versions_part}"
+            IFS="," read -r -a supportedVer <<<"${versions_part}"
-          for x in "${supportedVer[@]}"
+            for version in "${supportedVer[@]}"
-          do
+            do
-            if [[ "${detected_version}" =~ $x ]];then
+                if [[ "${detected_version}" =~ $version ]]; then
-              valid_version=true
+                    valid_version=true
-              break
+                    break
-            fi
+                fi
-          done
+            done
-          break
+            break
         fi
     done
 
-    # Display findings back to the user
+    log_write "${INFO} dig return code:  ${digReturnCode}"
+    log_write "${INFO} dig response:  ${response}"
+
     if [ "$valid_os" = true ]; then
-        log_write "${TICK} Distro:  ${COL_GREEN}${detected_os}${COL_NC}"
+        log_write "${TICK} Distro:  ${COL_GREEN}${detected_os^}${COL_NC}"
 
         if [ "$valid_version" = true ]; then
             log_write "${TICK} Version: ${COL_GREEN}${detected_version}${COL_NC}"
         else
             log_write "${CROSS} Version: ${COL_RED}${detected_version}${COL_NC}"
-            log_write "${CROSS} Error: ${COL_RED}${detected_os} is supported but version ${detected_version} is currently unsupported (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
+            log_write "${CROSS} Error: ${COL_RED}${detected_os^} is supported but version ${detected_version} is currently unsupported (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
         fi
     else
-        log_write "${CROSS} Distro:  ${COL_RED}${detected_os}${COL_NC}"
+        log_write "${CROSS} Distro:  ${COL_RED}${detected_os^}${COL_NC}"
-        log_write "${CROSS} Error: ${COL_RED}${detected_os} is not a supported distro (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
+        log_write "${CROSS} Error: ${COL_RED}${detected_os^} is not a supported distro (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
     fi
 }
 
@@ -488,6 +505,58 @@ check_selinux() {
     fi
 }
 
+check_firewalld() {
+    # FirewallD ships by default on Fedora/CentOS/RHEL and enabled upon clean install
+    # FirewallD is not configured by the installer and is the responsibility of the user
+    echo_current_diagnostic "FirewallD"
+    # Check if FirewallD service is enabled
+    if command -v systemctl &> /dev/null; then
+        # get its status via systemctl
+        local firewalld_status
+        firewalld_status=$(systemctl is-active firewalld)
+        log_write "${INFO} ${COL_GREEN}Firewalld service ${firewalld_status}${COL_NC}";
+        if [ "${firewalld_status}" == "active" ]; then
+            # test common required service ports
+            local firewalld_enabled_services
+            firewalld_enabled_services=$(firewall-cmd --list-services)
+            local firewalld_expected_services=("http" "dns" "dhcp" "dhcpv6")
+            for i in "${firewalld_expected_services[@]}"; do
+                if [[ "${firewalld_enabled_services}" =~ ${i} ]]; then
+                    log_write "${TICK} ${COL_GREEN}  Allow Service: ${i}${COL_NC}";
+                else
+                    log_write "${CROSS} ${COL_RED}  Allow Service: ${i}${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+                fi
+            done
+            # check for custom FTL FirewallD zone
+            local firewalld_zones
+            firewalld_zones=$(firewall-cmd --get-zones)
+            if [[ "${firewalld_zones}" =~ "ftl" ]]; then
+                log_write "${TICK} ${COL_GREEN}FTL Custom Zone Detected${COL_NC}";
+                # check FTL custom zone interface: lo
+                local firewalld_ftl_zone_interfaces
+                firewalld_ftl_zone_interfaces=$(firewall-cmd --zone=ftl --list-interfaces)
+                if [[ "${firewalld_ftl_zone_interfaces}" =~ "lo" ]]; then
+                    log_write "${TICK} ${COL_GREEN}  Local Interface Detected${COL_NC}";
+                else
+                    log_write "${CROSS} ${COL_RED}  Local Interface Not Detected${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+                fi
+                # check FTL custom zone port: 4711
+                local firewalld_ftl_zone_ports
+                firewalld_ftl_zone_ports=$(firewall-cmd --zone=ftl --list-ports)
+                if [[ "${firewalld_ftl_zone_ports}" =~ "4711/tcp" ]]; then
+                    log_write "${TICK} ${COL_GREEN}  FTL Port 4711/tcp Detected${COL_NC}";
+                else
+                    log_write "${CROSS} ${COL_RED}  FTL Port 4711/tcp Not Detected${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+                fi
+            else
+                log_write "${CROSS} ${COL_RED}FTL Custom Zone Not Detected${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+            fi
+        fi
+    else
+        log_write "${TICK} ${COL_GREEN}Firewalld service not detected${COL_NC}";
+    fi
+}
+
 processor_check() {
     echo_current_diagnostic "Processor"
     # Store the processor type in a variable
@@ -500,7 +569,7 @@ processor_check() {
     else
         # Check if the architecture is currently supported for FTL
         case "${PROCESSOR}" in
-            "amd64") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
+            "amd64" | "x86_64") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
                 ;;
             "armv6l") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
                 ;;
@@ -535,43 +604,11 @@ parse_locale() {
     parse_file "${pihole_locale}"
 }
 
-does_ip_match_setup_vars() {
-    # Check for IPv4 or 6
-    local protocol="${1}"
-    # IP address to check for
-    local ip_address="${2}"
-    # See what IP is in the setupVars.conf file
-    local setup_vars_ip
-    setup_vars_ip=$(< ${PIHOLE_SETUP_VARS_FILE} grep IPV"${protocol}"_ADDRESS | cut -d '=' -f2)
-    # If it's an IPv6 address
-    if [[ "${protocol}" == "6" ]]; then
-        # Strip off the / (CIDR notation)
-        if [[ "${ip_address%/*}" == "${setup_vars_ip%/*}" ]]; then
-            # if it matches, show it in green
-            log_write "   ${COL_GREEN}${ip_address%/*}${COL_NC} matches the IP found in ${PIHOLE_SETUP_VARS_FILE}"
-        else
-            # otherwise show it in red with an FAQ URL
-            log_write "   ${COL_RED}${ip_address%/*}${COL_NC} does not match the IP found in ${PIHOLE_SETUP_VARS_FILE} (${FAQ_ULA})"
-        fi
-
-    else
-        # if the protocol isn't 6, it's 4 so no need to strip the CIDR notation
-        # since it exists in the setupVars.conf that way
-        if [[ "${ip_address}" == "${setup_vars_ip}" ]]; then
-            # show in green if it matches
-            log_write "   ${COL_GREEN}${ip_address}${COL_NC} matches the IP found in ${PIHOLE_SETUP_VARS_FILE}"
-        else
-            # otherwise show it in red
-            log_write "   ${COL_RED}${ip_address}${COL_NC} does not match the IP found in ${PIHOLE_SETUP_VARS_FILE} (${FAQ_ULA})"
-        fi
-    fi
-}
-
 detect_ip_addresses() {
     # First argument should be a 4 or a 6
     local protocol=${1}
     # Use ip to show the addresses for the chosen protocol
-    # Store the values in an arry so they can be looped through
+    # Store the values in an array so they can be looped through
     # Get the lines that are in the file(s) and store them in an array for parsing later
     mapfile -t ip_addr_list < <(ip -"${protocol}" addr show dev "${PIHOLE_INTERFACE}" | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }')
 
@@ -583,8 +620,7 @@ detect_ip_addresses() {
         log_write "${TICK} IPv${protocol} address(es) bound to the ${PIHOLE_INTERFACE} interface:"
         # Since there may be more than one IP address, store them in an array
         for i in "${!ip_addr_list[@]}"; do
-            # For each one in the list, print it out
+            log_write "    ${ip_addr_list[$i]}"
-            does_ip_match_setup_vars "${protocol}" "${ip_addr_list[$i]}"
         done
         # Print a blank line just for formatting
         log_write ""
@@ -593,13 +629,6 @@ detect_ip_addresses() {
         log_write "${CROSS} ${COL_RED}No IPv${protocol} address(es) found on the ${PIHOLE_INTERFACE}${COL_NC} interface.\\n"
         return 1
     fi
-    # If the protocol is v6
-    if [[ "${protocol}" == "6" ]]; then
-        # let the user know that as long as there is one green address, things should be ok
-        log_write "   ^ Please note that you may have more than one IP address listed."
-        log_write "   As long as one of them is green, and it matches what is in ${PIHOLE_SETUP_VARS_FILE}, there is no need for concern.\\n"
-        log_write "   The link to the FAQ is for an issue that sometimes occurs when the IPv6 address changes, which is why we check for it.\\n"
-    fi
 }
 
 ping_ipv4_or_ipv6() {
@@ -625,7 +654,7 @@ ping_gateway() {
     # Check if we are using IPv4 or IPv6
     # Find the default gateway using IPv4 or IPv6
     local gateway
-    gateway="$(ip -"${protocol}" route | grep default | cut -d ' ' -f 3)"
+    gateway="$(ip -"${protocol}" route | grep default | grep "${PIHOLE_INTERFACE}" | cut -d ' ' -f 3)"
 
     # If the gateway variable has a value (meaning a gateway was found),
     if [[ -n "${gateway}" ]]; then
@@ -748,36 +777,20 @@ check_x_headers() {
     # server is operating correctly
     echo_current_diagnostic "Dashboard and block page"
     # Use curl -I to get the header and parse out just the X-Pi-hole one
-    local block_page
-    block_page=$(curl -Is localhost | awk '/X-Pi-hole/' | tr -d '\r')
-    # Do it for the dashboard as well, as the header is different than above
     local dashboard
     dashboard=$(curl -Is localhost/admin/ | awk '/X-Pi-hole/' | tr -d '\r')
-    # Store what the X-Header shoud be in variables for comparison later
+    # Store what the X-Header should be in variables for comparison later
-    local block_page_working
-    block_page_working="X-Pi-hole: A black hole for Internet advertisements."
     local dashboard_working
     dashboard_working="X-Pi-hole: The Pi-hole Web interface is working!"
-    local full_curl_output_block_page
-    full_curl_output_block_page="$(curl -Is localhost)"
     local full_curl_output_dashboard
     full_curl_output_dashboard="$(curl -Is localhost/admin/)"
-    # If the X-header found by curl matches what is should be,
+    
-    if [[ $block_page == "$block_page_working" ]]; then
+    # Same logic applies to the dashboard as above, if the X-Header matches what a working system should have,
-        # display a success message
-        log_write "$TICK Block page X-Header: ${COL_GREEN}${block_page}${COL_NC}"
-    else
-        # Otherwise, show an error
-        log_write "$CROSS Block page X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
-        log_write "${COL_RED}${full_curl_output_block_page}${COL_NC}"
-    fi
-
-    # Same logic applies to the dashbord as above, if the X-Header matches what a working system shoud have,
     if [[ $dashboard == "$dashboard_working" ]]; then
         # then we can show a success
         log_write "$TICK Web interface X-Header: ${COL_GREEN}${dashboard}${COL_NC}"
     else
-        # Othewise, it's a failure since the X-Headers either don't exist or have been modified in some way
+        # Otherwise, it's a failure since the X-Headers either don't exist or have been modified in some way
         log_write "$CROSS Web interface X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
         log_write "${COL_RED}${full_curl_output_dashboard}${COL_NC}"
     fi
@@ -789,13 +802,13 @@ dig_at() {
 
     # Store the arguments as variables with names
     local protocol="${1}"
-    local IP="${2}"
     echo_current_diagnostic "Name resolution (IPv${protocol}) using a random blocked domain and a known ad-serving domain"
     # Set more local variables
     # We need to test name resolution locally, via Pi-hole, and via a public resolver
     local local_dig
-    local pihole_dig
     local remote_dig
+    local interfaces
+    local addresses
     # Use a static domain that we know has IPv4 and IPv6 to avoid false positives
     # Sometimes the randomly chosen domains don't use IPv6, or something else is wrong with them
     local remote_url="doubleclick.com"
@@ -804,15 +817,15 @@ dig_at() {
     if [[ ${protocol} == "6" ]]; then
         # Set the IPv6 variables and record type
         local local_address="::1"
-        local pihole_address="${IP}"
         local remote_address="2001:4860:4860::8888"
+        local sed_selector="inet6"
         local record_type="AAAA"
-    # Othwerwise, it should be 4
+    # Otherwise, it should be 4
     else
         # so use the IPv4 values
         local local_address="127.0.0.1"
-        local pihole_address="${IP}"
         local remote_address="8.8.8.8"
+        local sed_selector="inet"
         local record_type="A"
     fi
 
@@ -822,32 +835,55 @@ dig_at() {
     local random_url
     random_url=$(sqlite3 "${PIHOLE_GRAVITY_DB_FILE}" "SELECT domain FROM vw_gravity ORDER BY RANDOM() LIMIT 1")
 
-    # First, do a dig on localhost to see if Pi-hole can use itself to block a domain
-    if local_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @${local_address} +short "${record_type}"); then
-        # If it can, show success
-        log_write "${TICK} ${random_url} ${COL_GREEN}is ${local_dig}${COL_NC} via ${COL_CYAN}localhost$COL_NC (${local_address})"
-    else
-        # Otherwise, show a failure
-        log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_RED}localhost${COL_NC} (${local_address})"
-    fi
-
     # Next we need to check if Pi-hole can resolve a domain when the query is sent to it's IP address
     # This better emulates how clients will interact with Pi-hole as opposed to above where Pi-hole is
     # just asing itself locally
-    # The default timeouts and tries are reduced in case the DNS server isn't working, so the user isn't waiting for too long
+    # The default timeouts and tries are reduced in case the DNS server isn't working, so the user isn't
+    # waiting for too long
+    #
+    # Turn off history expansion such that the "!" in the sed command cannot do silly things
+    set +H
+    # Get interfaces
+    # sed logic breakdown:
+    #     / master /d;
+    #          Removes all interfaces that are slaves of others (e.g. virtual docker interfaces)
+    #     /UP/!d;
+    #          Removes all interfaces which are not UP
+    #     s/^[0-9]*: //g;
+    #          Removes interface index
+    #     s/: <.*//g;
+    #          Removes everything after the interface name
+    interfaces="$(ip link show | sed "/ master /d;/UP/!d;s/^[0-9]*: //g;s/: <.*//g;")"
 
-    # If Pi-hole can dig itself from it's IP (not the loopback address)
+    while IFS= read -r iface ; do
-    if pihole_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @"${pihole_address}" +short "${record_type}"); then
+        # Get addresses of current interface
-        # show a success
+        # sed logic breakdown:
-        log_write "${TICK} ${random_url} ${COL_GREEN}is ${pihole_dig}${COL_NC} via ${COL_CYAN}Pi-hole${COL_NC} (${pihole_address})"
+        #     /inet(|6) /!d;
-    else
+        #          Removes all lines from ip a that do not contain either "inet " or "inet6 "
-        # Othewise, show a failure
+        #     s/^.*inet(|6) //g;
-        log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_RED}Pi-hole${COL_NC} (${pihole_address})"
+        #          Removes all leading whitespace as well as the "inet " or "inet6 " string
-    fi
+        #     s/\/.*$//g;
+        #          Removes CIDR and everything thereafter (e.g., scope properties)
+        addresses="$(ip address show dev "${iface}" | sed "/${sed_selector} /!d;s/^.*${sed_selector} //g;s/\/.*$//g;")"
+        if [ -n "${addresses}" ]; then
+          while IFS= read -r local_address ; do
+              # Check if Pi-hole can use itself to block a domain
+              if local_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @"${local_address}" +short "${record_type}"); then
+                  # If it can, show success
+                  log_write "${TICK} ${random_url} ${COL_GREEN}is ${local_dig}${COL_NC} on ${COL_CYAN}${iface}${COL_NC} (${COL_CYAN}${local_address}${COL_NC})"
+              else
+                  # Otherwise, show a failure
+                  log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} on ${COL_RED}${iface}${COL_NC} (${COL_RED}${local_address}${COL_NC})"
+              fi
+          done <<< "${addresses}"
+        else
+          log_write "${TICK} No IPv${protocol} address available on ${COL_CYAN}${iface}${COL_NC}"
+        fi
+    done <<< "${interfaces}"
 
     # Finally, we need to make sure legitimate queries can out to the Internet using an external, public DNS server
     # We are using the static remote_url here instead of a random one because we know it works with IPv4 and IPv6
-    if remote_dig=$(dig +tries=1 +time=2 -"${protocol}" "${remote_url}" @${remote_address} +short "${record_type}" | head -n1); then
+    if remote_dig=$(dig +tries=1 +time=2 -"${protocol}" "${remote_url}" @"${remote_address}" +short "${record_type}" | head -n1); then
         # If successful, the real IP of the domain will be returned instead of Pi-hole's IP
         log_write "${TICK} ${remote_url} ${COL_GREEN}is ${remote_dig}${COL_NC} via ${COL_CYAN}a remote, public DNS server${COL_NC} (${remote_address})"
     else
@@ -888,6 +924,18 @@ process_status(){
     done
 }
 
+ftl_full_status(){
+    # if using systemd print the full status of pihole-FTL
+    echo_current_diagnostic "Pi-hole-FTL full status"
+    local FTL_status
+    if command -v systemctl &> /dev/null; then
+      FTL_status=$(systemctl status --full --no-pager pihole-FTL.service)
+      log_write "   ${FTL_status}"
+    else
+      log_write "${INFO} systemctl:  command not found"
+    fi
+}
+
 make_array_from_file() {
     local filename="${1}"
     # The second argument can put a limit on how many line should be read from the file
@@ -905,7 +953,7 @@ make_array_from_file() {
         # Otherwise, read the file line by line
         while IFS= read -r line;do
             # Othwerise, strip out comments and blank lines
-            new_line=$(echo "${line}" | sed -e 's/#.*$//' -e '/^$/d')
+            new_line=$(echo "${line}" | sed -e 's/^\s*#.*$//' -e '/^$/d')
             # If the line still has content (a non-zero value)
             if [[ -n "${new_line}" ]]; then
                 # Put it into the array
@@ -950,7 +998,7 @@ parse_file() {
     local file_lines
     # For each line in the file,
     for file_lines in "${file_info[@]}"; do
-        if [[ ! -z "${file_lines}" ]]; then
+        if [[ -n "${file_lines}" ]]; then
             # don't include the Web password hash
             [[ "${file_lines}" =~ ^\#.*$  || ! "${file_lines}" || "${file_lines}" == "WEBPASSWORD="* ]] && continue
             # otherwise, display the lines of the file
@@ -962,14 +1010,10 @@ parse_file() {
 }
 
 check_name_resolution() {
-    # Check name resoltion from localhost, Pi-hole's IP, and Google's name severs
+    # Check name resolution from localhost, Pi-hole's IP, and Google's name severs
     # using the function we created earlier
-    dig_at 4 "${IPV4_ADDRESS%/*}"
+    dig_at 4
-    # If IPv6 enabled,
+    dig_at 6
-    if [[ "${IPV6_ADDRESS}" ]]; then
-        # check resolution
-        dig_at 6 "${IPV6_ADDRESS%/*}"
-    fi
 }
 
 # This function can check a directory exists
@@ -1012,17 +1056,21 @@ list_files_in_dir() {
             :
         elif [[ "${dir_to_parse}" == "${SHM_DIRECTORY}" ]]; then
             # SHM file - we do not want to see the content, but we want to see the files and their sizes
-            log_write "$(ls -ld "${dir_to_parse}"/"${each_file}")"
+            log_write "$(ls -lhd "${dir_to_parse}"/"${each_file}")"
+        elif [[ "${dir_to_parse}" == "${DNSMASQ_D_DIRECTORY}" ]]; then
+            # in case of the dnsmasq directory inlcuede all files in the debug output
+            log_write "\\n${COL_GREEN}$(ls -lhd "${dir_to_parse}"/"${each_file}")${COL_NC}"
+            make_array_from_file "${dir_to_parse}/${each_file}"
         else
             # Then, parse the file's content into an array so each line can be analyzed if need be
             for i in "${!REQUIRED_FILES[@]}"; do
                 if [[ "${dir_to_parse}/${each_file}" == "${REQUIRED_FILES[$i]}" ]]; then
                     # display the filename
-                    log_write "\\n${COL_GREEN}$(ls -ld "${dir_to_parse}"/"${each_file}")${COL_NC}"
+                    log_write "\\n${COL_GREEN}$(ls -lhd "${dir_to_parse}"/"${each_file}")${COL_NC}"
                     # Check if the file we want to view has a limit (because sometimes we just need a little bit of info from the file, not the entire thing)
                     case "${dir_to_parse}/${each_file}" in
-                        # If it's Web server error log, just give the first 25 lines
+                        # If it's Web server error log, give the first and last 25 lines
-                        "${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}") make_array_from_file "${dir_to_parse}/${each_file}" 25
+                        "${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}") head_tail_log "${dir_to_parse}/${each_file}" 25
                             ;;
                         # Same for the FTL log
                         "${PIHOLE_FTL_LOG}") head_tail_log "${dir_to_parse}/${each_file}" 35
@@ -1057,6 +1105,7 @@ show_content_of_pihole_files() {
     show_content_of_files_in_dir "${WEB_SERVER_LOG_DIRECTORY}"
     show_content_of_files_in_dir "${LOG_DIRECTORY}"
     show_content_of_files_in_dir "${SHM_DIRECTORY}"
+    show_content_of_files_in_dir "${ETC}"
 }
 
 head_tail_log() {
@@ -1112,27 +1161,71 @@ show_db_entries() {
     IFS="$OLD_IFS"
 }
 
+show_FTL_db_entries() {
+    local title="${1}"
+    local query="${2}"
+    local widths="${3}"
+
+    echo_current_diagnostic "${title}"
+
+    OLD_IFS="$IFS"
+    IFS=$'\r\n'
+    local entries=()
+    mapfile -t entries < <(\
+        sqlite3 "${PIHOLE_FTL_DB_FILE}" \
+            -cmd ".headers on" \
+            -cmd ".mode column" \
+            -cmd ".width ${widths}" \
+            "${query}"\
+    )
+
+    for line in "${entries[@]}"; do
+        log_write "   ${line}"
+    done
+
+    IFS="$OLD_IFS"
+}
+
+check_dhcp_servers() {
+    echo_current_diagnostic "Discovering active DHCP servers (takes 10 seconds)"
+
+    OLD_IFS="$IFS"
+    IFS=$'\n'
+    local entries=()
+    mapfile -t entries < <(pihole-FTL dhcp-discover)
+
+    for line in "${entries[@]}"; do
+        log_write "   ${line}"
+    done
+
+    IFS="$OLD_IFS"
+}
+
 show_groups() {
     show_db_entries "Groups" "SELECT id,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,name,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,description FROM \"group\"" "4 7 50 19 19 50"
 }
 
 show_adlists() {
-    show_db_entries "Adlists" "SELECT id,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,GROUP_CONCAT(adlist_by_group.group_id) group_ids,address,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM adlist LEFT JOIN adlist_by_group ON adlist.id = adlist_by_group.adlist_id GROUP BY id;" "4 7 12 100 19 19 50"
+    show_db_entries "Adlists" "SELECT id,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,GROUP_CONCAT(adlist_by_group.group_id) group_ids,address,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM adlist LEFT JOIN adlist_by_group ON adlist.id = adlist_by_group.adlist_id GROUP BY id;" "5 7 12 100 19 19 50"
 }
 
 show_domainlist() {
-    show_db_entries "Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)" "SELECT id,CASE type WHEN '0' THEN '0   ' WHEN '1' THEN ' 1  ' WHEN '2' THEN '  2 ' WHEN '3' THEN '   3' ELSE type END type,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,GROUP_CONCAT(domainlist_by_group.group_id) group_ids,domain,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM domainlist LEFT JOIN domainlist_by_group ON domainlist.id = domainlist_by_group.domainlist_id GROUP BY id;" "4 4 7 12 100 19 19 50"
+    show_db_entries "Domainlist (0/1 = exact white-/blacklist, 2/3 = regex white-/blacklist)" "SELECT id,CASE type WHEN '0' THEN '0   ' WHEN '1' THEN ' 1  ' WHEN '2' THEN '  2 ' WHEN '3' THEN '   3' ELSE type END type,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,GROUP_CONCAT(domainlist_by_group.group_id) group_ids,domain,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM domainlist LEFT JOIN domainlist_by_group ON domainlist.id = domainlist_by_group.domainlist_id GROUP BY id;" "5 4 7 12 100 19 19 50"
 }
 
 show_clients() {
     show_db_entries "Clients" "SELECT id,GROUP_CONCAT(client_by_group.group_id) group_ids,ip,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM client LEFT JOIN client_by_group ON client.id = client_by_group.client_id GROUP BY id;" "4 12 100 19 19 50"
 }
 
+show_messages() {
+    show_FTL_db_entries "Pi-hole diagnosis messages" "SELECT id,datetime(timestamp,'unixepoch','localtime') timestamp,type,message,blob1,blob2,blob3,blob4,blob5 FROM message;" "4 19 20 60 20 20 20 20 20"
+}
+
 analyze_gravity_list() {
-    echo_current_diagnostic "Gravity List and Database"
+    echo_current_diagnostic "Gravity Database"
 
     local gravity_permissions
-    gravity_permissions=$(ls -ld "${PIHOLE_GRAVITY_DB_FILE}")
+    gravity_permissions=$(ls -lhd "${PIHOLE_GRAVITY_DB_FILE}")
     log_write "${COL_GREEN}${gravity_permissions}${COL_NC}"
 
     show_db_entries "Info table" "SELECT property,value FROM info" "20 40"
@@ -1155,77 +1248,88 @@ analyze_gravity_list() {
     IFS="$OLD_IFS"
 }
 
-analyze_pihole_log() {
+obfuscated_pihole_log() {
-    echo_current_diagnostic "Pi-hole log"
+  local pihole_log=("$@")
-    local head_line
+  local line
-    # Put the current Internal Field Separator into another variable so it can be restored later
+  local error_to_check_for
-    OLD_IFS="$IFS"
+  local line_to_obfuscate
-    # Get the lines that are in the file(s) and store them in an array for parsing later
+  local obfuscated_line
-    IFS=$'\r\n'
+  for line in "${pihole_log[@]}"; do
-    local pihole_log_permissions
+      # A common error in the pihole.log is when there is a non-hosts formatted file
-    pihole_log_permissions=$(ls -ld "${PIHOLE_LOG}")
+      # that the DNS server is attempting to read.  Since it's not formatted
-    log_write "${COL_GREEN}${pihole_log_permissions}${COL_NC}"
+      # correctly, there will be an entry for "bad address at line n"
-    local pihole_log_head=()
+      # So we can check for that here and highlight it in red so the user can see it easily
-    mapfile -t pihole_log_head < <(head -n 20 ${PIHOLE_LOG})
+      error_to_check_for=$(echo "${line}" | grep 'bad address at')
-    log_write "   ${COL_CYAN}-----head of $(basename ${PIHOLE_LOG})------${COL_NC}"
+      # Some users may not want to have the domains they visit sent to us
-    local error_to_check_for
+      # To that end, we check for lines in the log that would contain a domain name
-    local line_to_obfuscate
+      line_to_obfuscate=$(echo "${line}" | grep ': query\|: forwarded\|: reply')
-    local obfuscated_line
+      # If the variable contains a value, it found an error in the log
-    for head_line in "${pihole_log_head[@]}"; do
+      if [[ -n ${error_to_check_for} ]]; then
-        # A common error in the pihole.log is when there is a non-hosts formatted file
+          # So we can print it in red to make it visible to the user
-        # that the DNS server is attempting to read.  Since it's not formatted
+          log_write "   ${CROSS} ${COL_RED}${line}${COL_NC} (${FAQ_BAD_ADDRESS})"
-        # correctly, there will be an entry for "bad address at line n"
+      else
-        # So we can check for that here and highlight it in red so the user can see it easily
+          # If the variable does not a value (the current default behavior), so do not obfuscate anything
-        error_to_check_for=$(echo "${head_line}" | grep 'bad address at')
+          if [[ -z ${OBFUSCATE} ]]; then
-        # Some users may not want to have the domains they visit sent to us
+              log_write "   ${line}"
-        # To that end, we check for lines in the log that would contain a domain name
+          # Othwerise, a flag was passed to this command to obfuscate domains in the log
-        line_to_obfuscate=$(echo "${head_line}" | grep ': query\|: forwarded\|: reply')
+          else
-        # If the variable contains a value, it found an error in the log
+              # So first check if there are domains in the log that should be obfuscated
-        if [[ -n ${error_to_check_for} ]]; then
+              if [[ -n ${line_to_obfuscate} ]]; then
-            # So we can print it in red to make it visible to the user
+                  # If there are, we need to use awk to replace only the domain name (the 6th field in the log)
-            log_write "   ${CROSS} ${COL_RED}${head_line}${COL_NC} (${FAQ_BAD_ADDRESS})"
+                  # so we substitute the domain for the placeholder value
-        else
+                  obfuscated_line=$(echo "${line_to_obfuscate}" | awk -v placeholder="${OBFUSCATED_PLACEHOLDER}" '{sub($6,placeholder); print $0}')
-            # If the variable does not a value (the current default behavior), so do not obfuscate anything
+                  log_write "   ${obfuscated_line}"
-            if [[ -z ${OBFUSCATE} ]]; then
+              else
-                log_write "   ${head_line}"
+                  log_write "   ${line}"
-            # Othwerise, a flag was passed to this command to obfuscate domains in the log
+              fi
-            else
+          fi
-                # So first check if there are domains in the log that should be obfuscated
+      fi
-                if [[ -n ${line_to_obfuscate} ]]; then
+  done
-                    # If there are, we need to use awk to replace only the domain name (the 6th field in the log)
-                    # so we substitute the domain for the placeholder value
-                    obfuscated_line=$(echo "${line_to_obfuscate}" | awk -v placeholder="${OBFUSCATED_PLACEHOLDER}" '{sub($6,placeholder); print $0}')
-                    log_write "   ${obfuscated_line}"
-                else
-                    log_write "   ${head_line}"
-                fi
-            fi
-        fi
-    done
-    log_write ""
-    # Set the IFS back to what it was
-    IFS="$OLD_IFS"
 }
 
-tricorder_use_nc_or_curl() {
+analyze_pihole_log() {
-    # Users can submit their debug logs using nc (unencrypted) or curl (encrypted) if available
+  echo_current_diagnostic "Pi-hole log"
-    # Check for curl first since encryption is a good thing
+  local pihole_log_head=()
-    if command -v curl &> /dev/null; then
+  local pihole_log_tail=()
-        # If the command exists,
+  local pihole_log_permissions
-        log_write "    * Using ${COL_GREEN}curl${COL_NC} for transmission."
+  local logging_enabled
-        # transmit he log via TLS and store the token returned in a variable
+
-        tricorder_token=$(curl --silent --upload-file ${PIHOLE_DEBUG_LOG} https://tricorder.pi-hole.net:${TRICORDER_SSL_PORT_NUMBER})
+  logging_enabled=$(grep -c "^log-queries" /etc/dnsmasq.d/01-pihole.conf)
-        if [ -z "${tricorder_token}" ]; then
+  if [[ "${logging_enabled}" == "0" ]]; then
-         # curl failed, fallback to nc
+      # Inform user that logging has been disabled and pihole.log does not contain queries
-         log_write "    * ${COL_GREEN}curl${COL_NC} failed, falling back to ${COL_YELLOW}netcat${COL_NC} for transmission."
+      log_write "${INFO} Query logging is disabled"
-         tricorder_token=$(< ${PIHOLE_DEBUG_LOG} nc tricorder.pi-hole.net ${TRICORDER_NC_PORT_NUMBER})
+      log_write ""
+  fi
+  # Put the current Internal Field Separator into another variable so it can be restored later
+  OLD_IFS="$IFS"
+  # Get the lines that are in the file(s) and store them in an array for parsing later
+  IFS=$'\r\n'
+  pihole_log_permissions=$(ls -lhd "${PIHOLE_LOG}")
+  log_write "${COL_GREEN}${pihole_log_permissions}${COL_NC}"
+  mapfile -t pihole_log_head < <(head -n 20 ${PIHOLE_LOG})
+  log_write "   ${COL_CYAN}-----head of $(basename ${PIHOLE_LOG})------${COL_NC}"
+  obfuscated_pihole_log "${pihole_log_head[@]}"
+  log_write ""
+  mapfile -t pihole_log_tail < <(tail -n 20 ${PIHOLE_LOG})
+  log_write "   ${COL_CYAN}-----tail of $(basename ${PIHOLE_LOG})------${COL_NC}"
+  obfuscated_pihole_log "${pihole_log_tail[@]}"
+  log_write ""
+  # Set the IFS back to what it was
+  IFS="$OLD_IFS"
+}
+
+curl_to_tricorder() {
+    # Users can submit their debug logs using curl (encrypted)
+    log_write "    * Using ${COL_GREEN}curl${COL_NC} for transmission."
+    # transmit the log via TLS and store the token returned in a variable
+    tricorder_token=$(curl --silent --fail --show-error --upload-file ${PIHOLE_DEBUG_LOG} https://tricorder.pi-hole.net 2>&1)
+    if [[ "${tricorder_token}" != "https://tricorder.pi-hole.net/"* ]]; then
+        log_write "    * ${COL_GREEN}curl${COL_NC} failed, contact Pi-hole support for assistance."
+        # Log curl error (if available)
+        if [ -n "${tricorder_token}" ]; then
+            log_write "    * Error message: ${COL_RED}${tricorder_token}${COL_NC}\\n"
+            tricorder_token=""
         fi
-    # Otherwise,
-    else
-        # use net cat
-        log_write "${INFO} Using ${COL_YELLOW}netcat${COL_NC} for transmission."
-        # Save the token returned by our server in a variable
-        tricorder_token=$(< ${PIHOLE_DEBUG_LOG} nc tricorder.pi-hole.net ${TRICORDER_NC_PORT_NUMBER})
     fi
 }
 
@@ -1244,14 +1348,13 @@ upload_to_tricorder() {
 
     # Provide information on what they should do with their token
     log_write "    * The debug log can be uploaded to tricorder.pi-hole.net for sharing with developers only."
-    log_write "    * For more information, see: ${TRICORDER_CONTEST}"
+
-    log_write "    * If available, we'll use openssl to upload the log, otherwise it will fall back to netcat."
     # If pihole -d is running automatically (usually through the dashboard)
     if [[ "${AUTOMATED}" ]]; then
         # let the user know
         log_write "${INFO} Debug script running in automated mode"
         # and then decide again which tool to use to submit it
-        tricorder_use_nc_or_curl
+        curl_to_tricorder
         # If we're not running in automated mode,
     else
         echo ""
@@ -1260,26 +1363,26 @@ upload_to_tricorder() {
         read -r -p "[?] Would you like to upload the log? [y/N] " response
         case ${response} in
             # If they say yes, run our function for uploading the log
-            [yY][eE][sS]|[yY]) tricorder_use_nc_or_curl;;
+            [yY][eE][sS]|[yY]) curl_to_tricorder;;
             # If they choose no, just exit out of the script
             *) log_write "    * Log will ${COL_GREEN}NOT${COL_NC} be uploaded to tricorder.\\n    * A local copy of the debug log can be found at: ${COL_CYAN}${PIHOLE_DEBUG_LOG}${COL_NC}\\n";exit;
         esac
-    fi232623
+    fi
     # Check if tricorder.pi-hole.net is reachable and provide token
     # along with some additional useful information
     if [[ -n "${tricorder_token}" ]]; then
         # Again, try to make this visually striking so the user realizes they need to do something with this information
         # Namely, provide the Pi-hole devs with the token
         log_write ""
-        log_write "${COL_PURPLE}***********************************${COL_NC}"
+        log_write "${COL_PURPLE}*****************************************************************${COL_NC}"
-        log_write "${COL_PURPLE}***********************************${COL_NC}"
+        log_write "${COL_PURPLE}*****************************************************************${COL_NC}\\n"
         log_write "${TICK} Your debug token is: ${COL_GREEN}${tricorder_token}${COL_NC}"
-        log_write "${COL_PURPLE}***********************************${COL_NC}"
+        log_write "${INFO}${COL_RED} Logs are deleted 48 hours after upload.${COL_NC}\\n"
-        log_write "${COL_PURPLE}***********************************${COL_NC}"
+        log_write "${COL_PURPLE}*****************************************************************${COL_NC}"
+        log_write "${COL_PURPLE}*****************************************************************${COL_NC}"
         log_write ""
-        log_write "   * Provide the token above to the Pi-hole team for assistance at"
+        log_write "   * Provide the token above to the Pi-hole team for assistance at ${FORUMS_URL}"
-        log_write "   * ${FORUMS_URL}"
+
-        log_write "   * Your log will self-destruct on our server after ${COL_RED}48 hours${COL_NC}."
     # If no token was generated
     else
         # Show an error and some help instructions
@@ -1300,10 +1403,13 @@ check_component_versions
 check_critical_program_versions
 diagnose_operating_system
 check_selinux
+check_firewalld
 processor_check
 check_networking
 check_name_resolution
+check_dhcp_servers
 process_status
+ftl_full_status
 parse_setup_vars
 check_x_headers
 analyze_gravity_list
@@ -1312,6 +1418,7 @@ show_domainlist
 show_clients
 show_adlists
 show_content_of_pihole_files
+show_messages
 parse_locale
 analyze_pihole_log
 copy_to_debug_log

advanced/Scripts/piholeLogFlush.sh

@@ -11,6 +11,11 @@
 colfile="/opt/pihole/COL_TABLE"
 source ${colfile}
 
+# In case we're running at the same time as a system logrotate, use a
+# separate logrotate state file to prevent stepping on each other's
+# toes.
+STATEFILE="/var/lib/logrotate/pihole"
+
 # Determine database location
 # Obtain DBFILE=... setting from pihole-FTL.db
 # Constructed to return nothing when
@@ -32,7 +37,7 @@ if [[ "$@" == *"once"* ]]; then
     # Nightly logrotation
     if command -v /usr/sbin/logrotate >/dev/null; then
         # Logrotate once
-        /usr/sbin/logrotate --force /etc/pihole/logrotate
+        /usr/sbin/logrotate --force --state "${STATEFILE}" /etc/pihole/logrotate
     else
         # Copy pihole.log over to pihole.log.1
         # and empty out pihole.log
@@ -47,8 +52,8 @@ else
     # Manual flushing
     if command -v /usr/sbin/logrotate >/dev/null; then
         # Logrotate twice to move all data out of sight of FTL
-        /usr/sbin/logrotate --force /etc/pihole/logrotate; sleep 3
+        /usr/sbin/logrotate --force --state "${STATEFILE}" /etc/pihole/logrotate; sleep 3
-        /usr/sbin/logrotate --force /etc/pihole/logrotate
+        /usr/sbin/logrotate --force --state "${STATEFILE}" /etc/pihole/logrotate
     else
         # Flush both pihole.log and pihole.log.1 (if existing)
         echo " " > /var/log/pihole.log

advanced/Scripts/query.sh

@@ -1,5 +1,6 @@
 #!/usr/bin/env bash
 # shellcheck disable=SC1090
+
 # Pi-hole: A black hole for Internet advertisements
 # (c) 2018 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
@@ -11,12 +12,21 @@
 
 # Globals
 piholeDir="/etc/pihole"
-gravityDBfile="${piholeDir}/gravity.db"
+GRAVITYDB="${piholeDir}/gravity.db"
 options="$*"
 all=""
 exact=""
 blockpage=""
 matchType="match"
+# Source pihole-FTL from install script
+pihole_FTL="${piholeDir}/pihole-FTL.conf"
+if [[ -f "${pihole_FTL}" ]]; then
+  source "${pihole_FTL}"
+fi
+
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"
 
 colfile="/opt/pihole/COL_TABLE"
 source "${colfile}"

advanced/Scripts/update.sh

@@ -95,6 +95,10 @@ main() {
     # shellcheck disable=1090,2154
     source "${setupVars}"
 
+    # Install packages used by this installation script (necessary if users have removed e.g. git from their systems)
+    package_manager_detect
+    install_dependent_packages "${INSTALLER_DEPS[@]}"
+
     # This is unlikely
     if ! is_repo "${PI_HOLE_FILES_DIR}" ; then
         echo -e "\\n  ${COL_LIGHT_RED}Error: Core Pi-hole repo is missing from system!"

advanced/Scripts/version.sh

@@ -153,7 +153,7 @@ versionOutput() {
     if [[ -n "$current" ]] && [[ -n "$latest" ]]; then
         output="${1^} version is $branch$current (Latest: $latest)"
     elif [[ -n "$current" ]] && [[ -z "$latest" ]]; then
-        output="Current ${1^} version is $branch$current."
+        output="Current ${1^} version is $branch$current"
     elif [[ -z "$current" ]] && [[ -n "$latest" ]]; then
         output="Latest ${1^} version is $latest"
     elif [[ "$curHash" == "N/A" ]] || [[ "$latHash" == "N/A" ]]; then

advanced/Scripts/webpage.sh

@@ -44,7 +44,8 @@ Options:
   -e, email           Set an administrative contact address for the Block Page
   -h, --help          Show this help dialog
   -i, interface       Specify dnsmasq's interface listening behavior
-  -l, privacylevel    Set privacy level (0 = lowest, 4 = highest)"
+  -l, privacylevel    Set privacy level (0 = lowest, 3 = highest)
+  -t, teleporter      Backup configuration as an archive"
     exit 0
 }
 
@@ -53,7 +54,7 @@ add_setting() {
 }
 
 delete_setting() {
-    sed -i "/${1}/d" "${setupVars}"
+    sed -i "/^${1}/d" "${setupVars}"
 }
 
 change_setting() {
@@ -66,7 +67,7 @@ addFTLsetting() {
 }
 
 deleteFTLsetting() {
-    sed -i "/${1}/d" "${FTLconf}"
+    sed -i "/^${1}/d" "${FTLconf}"
 }
 
 changeFTLsetting() {
@@ -83,7 +84,7 @@ add_dnsmasq_setting() {
 }
 
 delete_dnsmasq_setting() {
-    sed -i "/${1}/d" "${dnsmasqconfig}"
+    sed -i "/^${1}/d" "${dnsmasqconfig}"
 }
 
 SetTemperatureUnit() {
@@ -167,9 +168,11 @@ ProcessDNSSettings() {
     fi
 
     delete_dnsmasq_setting "domain-needed"
+    delete_dnsmasq_setting "expand-hosts"
 
     if [[ "${DNS_FQDN_REQUIRED}" == true ]]; then
         add_dnsmasq_setting "domain-needed"
+        add_dnsmasq_setting "expand-hosts"
     fi
 
     delete_dnsmasq_setting "bogus-priv"
@@ -215,6 +218,12 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423
 
     if [[ "${CONDITIONAL_FORWARDING}" == true ]]; then
         # Convert legacy "conditional forwarding" to rev-server configuration
+        # Remove any existing REV_SERVER settings
+        delete_setting "REV_SERVER"
+        delete_setting "REV_SERVER_DOMAIN"
+        delete_setting "REV_SERVER_TARGET"
+        delete_setting "REV_SERVER_CIDR"
+
         REV_SERVER=true
         add_setting "REV_SERVER" "true"
 
@@ -224,30 +233,63 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423
         REV_SERVER_TARGET="${CONDITIONAL_FORWARDING_IP}"
         add_setting "REV_SERVER_TARGET" "${REV_SERVER_TARGET}"
 
+        #Convert CONDITIONAL_FORWARDING_REVERSE if necessary e.g:
+        #          1.1.168.192.in-addr.arpa to 192.168.1.1/32
+        #          1.168.192.in-addr.arpa to 192.168.1.0/24
+        #          168.192.in-addr.arpa to 192.168.0.0/16
+        #          192.in-addr.arpa to 192.0.0.0/8
+        if [[ "${CONDITIONAL_FORWARDING_REVERSE}" == *"in-addr.arpa" ]];then
+            arrRev=("${CONDITIONAL_FORWARDING_REVERSE//./ }")
+            case ${#arrRev[@]} in
+                6   )   REV_SERVER_CIDR="${arrRev[3]}.${arrRev[2]}.${arrRev[1]}.${arrRev[0]}/32";;
+                5   )   REV_SERVER_CIDR="${arrRev[2]}.${arrRev[1]}.${arrRev[0]}.0/24";;
+                4   )   REV_SERVER_CIDR="${arrRev[1]}.${arrRev[0]}.0.0/16";;
+                3   )   REV_SERVER_CIDR="${arrRev[0]}.0.0.0/8";;
+            esac
+        else
+          # Set REV_SERVER_CIDR to whatever value it was set to
+          REV_SERVER_CIDR="${CONDITIONAL_FORWARDING_REVERSE}"
+        fi
+
+        # If REV_SERVER_CIDR is not converted by the above, then use the REV_SERVER_TARGET variable to derive it
+        if [ -z "${REV_SERVER_CIDR}" ]; then
+            # Convert existing input to /24 subnet (preserves legacy behavior)
+            # This sed converts "192.168.1.2" to "192.168.1.0/24"
+            # shellcheck disable=2001
+            REV_SERVER_CIDR="$(sed "s+\\.[0-9]*$+\\.0/24+" <<< "${REV_SERVER_TARGET}")"
+        fi
+        add_setting "REV_SERVER_CIDR" "${REV_SERVER_CIDR}"
+
         # Remove obsolete settings from setupVars.conf
         delete_setting "CONDITIONAL_FORWARDING"
         delete_setting "CONDITIONAL_FORWARDING_REVERSE"
         delete_setting "CONDITIONAL_FORWARDING_DOMAIN"
         delete_setting "CONDITIONAL_FORWARDING_IP"
-
-        # Convert existing input to /24 subnet (preserves legacy behavior)
-        # This sed converts "192.168.1.2" to "192.168.1.0/24"
-        # shellcheck disable=2001
-        REV_SERVER_CIDR="$(sed "s+\\.[0-9]*$+\\.0/24+" <<< "${REV_SERVER_TARGET}")"
-        add_setting "REV_SERVER_CIDR" "${REV_SERVER_CIDR}"
     fi
 
+    delete_dnsmasq_setting "rev-server"
+
     if [[ "${REV_SERVER}" == true ]]; then
         add_dnsmasq_setting "rev-server=${REV_SERVER_CIDR},${REV_SERVER_TARGET}"
         if [ -n "${REV_SERVER_DOMAIN}" ]; then
+            # Forward local domain names to the CF target, too
             add_dnsmasq_setting "server=/${REV_SERVER_DOMAIN}/${REV_SERVER_TARGET}"
         fi
+
+        if [[ "${DNS_FQDN_REQUIRED}" != true ]]; then
+            # Forward unqualified names to the CF target only when the "never
+            # forward non-FQDN" option is unticked
+            add_dnsmasq_setting "server=//${REV_SERVER_TARGET}"
+        fi
+
     fi
 
-    # Prevent Firefox from automatically switching over to DNS-over-HTTPS
+    # We need to process DHCP settings here as well to account for possible
-    # This follows https://support.mozilla.org/en-US/kb/configuring-networks-disable-dns-over-https
+    # changes in the non-FQDN forwarding. This cannot be done in 01-pihole.conf
-    # (sourced 7th September 2019)
+    # as we don't want to delete all local=/.../ lines so it's much safer to
-    add_dnsmasq_setting "server=/use-application-dns.net/"
+    # simply rewrite the entire corresponding config file (which is what the
+    # DHCP settings subroutie is doing)
+    ProcessDHCPSettings
 }
 
 SetDNSServers() {
@@ -370,6 +412,14 @@ dhcp-leasefile=/etc/pihole/dhcp.leases
 
     if [[ "${PIHOLE_DOMAIN}" != "none" ]]; then
         echo "domain=${PIHOLE_DOMAIN}" >> "${dhcpconfig}"
+
+        # When there is a Pi-hole domain set and "Never forward non-FQDNs" is
+        # ticked, we add `local=/domain/` to tell FTL that this domain is purely
+        # local and FTL may answer queries from /etc/hosts or DHCP but should
+        # never forward queries on that domain to any upstream servers
+        if  [[ "${DNS_FQDN_REQUIRED}" == true ]]; then
+          echo "local=/${PIHOLE_DOMAIN}/" >> "${dhcpconfig}"
+        fi
     fi
 
     # Sourced from setupVars
@@ -382,7 +432,7 @@ dhcp-leasefile=/etc/pihole/dhcp.leases
         echo "#quiet-dhcp6
 #enable-ra
 dhcp-option=option6:dns-server,[::]
-dhcp-range=::100,::1ff,constructor:${interface},ra-names,slaac,${leasetime}
+dhcp-range=::100,::1ff,constructor:${interface},ra-names,slaac,64,3600
 ra-param=*,0,0
 " >> "${dhcpconfig}"
     fi
@@ -442,10 +492,15 @@ SetWebUITheme() {
 }
 
 CheckUrl(){
-    local regex
+    local regex check_url
     # Check for characters NOT allowed in URLs
-    regex="[^a-zA-Z0-9:/?&%=~._-]"
+    regex="[^a-zA-Z0-9:/?&%=~._()-;]"
-    if [[ "${1}" =~ ${regex} ]]; then
+
+    # this will remove first @ that is after schema and before domain
+    # \1 is optional schema, \2 is userinfo
+    check_url="$( sed -re 's#([^:/]*://)?([^/]+)@#\1\2#' <<< "$1" )"
+
+    if [[ "${check_url}" =~ ${regex} ]]; then
         return 1
     else
         return 0
@@ -515,7 +570,13 @@ AddDHCPStaticAddress() {
 
 RemoveDHCPStaticAddress() {
     mac="${args[2]}"
-    sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
+    if [[ "$mac" =~ ^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$ ]]; then
+        sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
+    else
+        echo "  ${CROSS} Invalid Mac Passed!"
+        exit 1
+    fi
+
 }
 
 SetAdminEmail() {
@@ -587,8 +648,11 @@ Interfaces:
 
 Teleporter() {
     local datetimestamp
+    local host
     datetimestamp=$(date "+%Y-%m-%d_%H-%M-%S")
-    php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-teleporter_${datetimestamp}.tar.gz"
+    host=$(hostname)
+    host="${host//./_}"
+    php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-${host:-noname}-teleporter_${datetimestamp}.tar.gz"
 }
 
 checkDomain()
@@ -633,8 +697,8 @@ clearAudit()
 }
 
 SetPrivacyLevel() {
-    # Set privacy level. Minimum is 0, maximum is 4
+    # Set privacy level. Minimum is 0, maximum is 3
-    if [ "${args[2]}" -ge 0 ] && [ "${args[2]}" -le 4 ]; then
+    if [ "${args[2]}" -ge 0 ] && [ "${args[2]}" -le 3 ]; then
         changeFTLsetting "PRIVACYLEVEL" "${args[2]}"
         pihole restartdns reload-lists
     fi
@@ -656,7 +720,13 @@ RemoveCustomDNSAddress() {
 
     ip="${args[2]}"
     host="${args[3]}"
-    sed -i "/${ip} ${host}/d" "${dnscustomfile}"
+
+    if valid_ip "${ip}" || valid_ip6 "${ip}" ; then
+        sed -i "/^${ip} ${host}$/d" "${dnscustomfile}"
+    else
+        echo -e "  ${CROSS} Invalid IP has been passed"
+        exit 1
+    fi
 
     # Restart dnsmasq to update removed custom DNS entries
     RestartDNS
@@ -667,6 +737,7 @@ AddCustomCNAMERecord() {
 
     domain="${args[2]}"
     target="${args[3]}"
+
     echo "cname=${domain},${target}" >> "${dnscustomcnamefile}"
 
     # Restart dnsmasq to load new custom CNAME records
@@ -678,7 +749,20 @@ RemoveCustomCNAMERecord() {
 
     domain="${args[2]}"
     target="${args[3]}"
-    sed -i "/cname=${domain},${target}/d" "${dnscustomcnamefile}"
+
+    validDomain="$(checkDomain "${domain}")"
+    if [[ -n "${validDomain}" ]]; then
+        validTarget="$(checkDomain "${target}")"
+        if [[ -n "${validDomain}" ]]; then
+            sed -i "/cname=${validDomain},${validTarget}$/d" "${dnscustomcnamefile}"
+        else
+            echo "  ${CROSS} Invalid Target Passed!"
+            exit 1
+        fi
+    else
+        echo "  ${CROSS} Invalid Domain passed!"
+        exit 1
+    fi
 
     # Restart dnsmasq to update removed custom CNAME records
     RestartDNS

advanced/Scripts/wildcard_regex_converter.sh

@@ -1,28 +0,0 @@
-#!/usr/bin/env bash
-# Pi-hole: A black hole for Internet advertisements
-# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
-# Network-wide ad blocking via your own hardware.
-#
-# Provides an automated migration subroutine to convert Pi-hole v3.x wildcard domains to Pi-hole v4.x regex filters
-#
-# This file is copyright under the latest version of the EUPL.
-# Please see LICENSE file for your rights under this license.
-
-# regexFile set in gravity.sh
-
-wildcardFile="/etc/dnsmasq.d/03-pihole-wildcard.conf"
-
-convert_wildcard_to_regex() {
-    if [ ! -f "${wildcardFile}" ]; then
-        return
-    fi
-    local addrlines domains uniquedomains
-    # Obtain wildcard domains from old file
-    addrlines="$(grep -oE "/.*/" ${wildcardFile})"
-    # Strip "/" from domain names and convert "." to regex-compatible "\."
-    domains="$(sed 's/\///g;s/\./\\./g' <<< "${addrlines}")"
-    # Remove repeated domains (may have been inserted two times due to A and AAAA blocking)
-    uniquedomains="$(uniq <<< "${domains}")"
-    # Automatically generate regex filters and remove old wildcards file
-    awk '{print "(^|\\.)"$0"$"}' <<< "${uniquedomains}" >> "${regexFile:?}" && rm "${wildcardFile}"
-}

advanced/Templates/gravity.db.sql

@@ -31,7 +31,11 @@ CREATE TABLE adlist
 	enabled BOOLEAN NOT NULL DEFAULT 1,
 	date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
-	comment TEXT
+	comment TEXT,
+	date_updated INTEGER,
+	number INTEGER NOT NULL DEFAULT 0,
+	invalid_domains INTEGER NOT NULL DEFAULT 0,
+	status INTEGER NOT NULL DEFAULT 0
 );
 
 CREATE TABLE adlist_by_group
@@ -53,7 +57,7 @@ CREATE TABLE info
 	value TEXT NOT NULL
 );
 
-INSERT INTO "info" VALUES('version','12');
+INSERT INTO "info" VALUES('version','14');
 
 CREATE TABLE domain_audit
 (
@@ -72,7 +76,7 @@ CREATE TABLE domainlist_by_group
 CREATE TABLE client
 (
 	id INTEGER PRIMARY KEY AUTOINCREMENT,
-	ip TEXT NOL NULL UNIQUE,
+	ip TEXT NOT NULL UNIQUE,
 	date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	comment TEXT
@@ -85,9 +89,9 @@ CREATE TABLE client_by_group
 	PRIMARY KEY (client_id, group_id)
 );
 
-CREATE TRIGGER tr_adlist_update AFTER UPDATE ON adlist
+CREATE TRIGGER tr_adlist_update AFTER UPDATE OF address,enabled,comment ON adlist
     BEGIN
-      UPDATE adlist SET date_modified = (cast(strftime('%s', 'now') as int)) WHERE address = NEW.address;
+      UPDATE adlist SET date_modified = (cast(strftime('%s', 'now') as int)) WHERE id = NEW.id;
     END;
 
 CREATE TRIGGER tr_client_update AFTER UPDATE ON client

advanced/Templates/pihole-FTL.service

@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/usr/bin/env sh
 ### BEGIN INIT INFO
 # Provides:          pihole-FTL
 # Required-Start:    $remote_fs $syslog $network
@@ -9,22 +9,8 @@
 # Description:       Enable service provided by pihole-FTL daemon
 ### END INIT INFO
 
-FTLUSER=pihole
-PIDFILE=/run/pihole-FTL.pid
-
-get_pid() {
-    # First, try to obtain PID from PIDFILE
-    if [ -s "${PIDFILE}" ]; then
-        cat "${PIDFILE}"
-        return
-    fi
-
-    # If the PIDFILE is empty or not available, obtain the PID using pidof
-    pidof "pihole-FTL" | awk '{print $(NF)}'
-}
-
 is_running() {
-    ps "$(get_pid)" > /dev/null 2>&1
+  pgrep -xo "pihole-FTL" > /dev/null
 }
 
 
@@ -34,27 +20,18 @@ start() {
     echo "pihole-FTL is already running"
   else
     # Touch files to ensure they exist (create if non-existing, preserve if existing)
-    touch /var/log/pihole-FTL.log /var/log/pihole.log
+    mkdir -pm 0755 /run/pihole
-    touch /run/pihole-FTL.pid /run/pihole-FTL.port
+    touch /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole-FTL.log /var/log/pihole.log /etc/pihole/dhcp.leases
-    touch /etc/pihole/dhcp.leases
-    mkdir -p /run/pihole
-    mkdir -p /var/log/pihole
-    chown pihole:pihole /run/pihole /var/log/pihole
-    # Remove possible leftovers from previous pihole-FTL processes
-    rm -f /dev/shm/FTL-* 2> /dev/null
-    rm /run/pihole/FTL.sock 2> /dev/null
     # Ensure that permissions are set so that pihole-FTL can edit all necessary files
-    chown pihole:pihole /run/pihole-FTL.pid /run/pihole-FTL.port
+    chown pihole:pihole /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole-FTL.log /var/log/pihole.log /etc/pihole/dhcp.leases /run/pihole /etc/pihole
-    chown pihole:pihole /etc/pihole /etc/pihole/dhcp.leases 2> /dev/null
+    chmod 0644 /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole-FTL.log /var/log/pihole.log /etc/pihole/dhcp.leases /etc/pihole/macvendor.db
-    chown pihole:pihole /var/log/pihole-FTL.log /var/log/pihole.log
-    chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole.log
     # Chown database files to the user FTL runs as. We ignore errors as the files may not (yet) exist
-    chown pihole:pihole /etc/pihole/pihole-FTL.db /etc/pihole/gravity.db 2> /dev/null
+    chown -f pihole:pihole /etc/pihole/pihole-FTL.db /etc/pihole/gravity.db /etc/pihole/macvendor.db
-    if setcap CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_NET_ADMIN,CAP_SYS_NICE+eip "$(which pihole-FTL)"; then
+    if setcap CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_NET_ADMIN,CAP_SYS_NICE,CAP_IPC_LOCK,CAP_CHOWN+eip "/usr/bin/pihole-FTL"; then
-      su -s /bin/sh -c "/usr/bin/pihole-FTL" "$FTLUSER"
+      su -s /bin/sh -c "/usr/bin/pihole-FTL" pihole
     else
       echo "Warning: Starting pihole-FTL as root because setting capabilities is not supported on this system"
-      pihole-FTL
+      /usr/bin/pihole-FTL
     fi
     echo
   fi
@@ -63,20 +40,20 @@ start() {
 # Stop the service
 stop() {
   if is_running; then
-    kill "$(get_pid)"
+    pkill -xo "pihole-FTL"
-    for i in {1..5}; do
+    for i in 1 2 3 4 5; do
       if ! is_running; then
         break
       fi
 
-      echo -n "."
+      printf "."
       sleep 1
     done
     echo
 
     if is_running; then
       echo "Not stopped; may still be shutting down or shutdown may have failed, killing now"
-      kill -9 "$(get_pid)"
+      pkill -xo -9 "pihole-FTL"
       exit 1
     else
       echo "Stopped"
@@ -84,6 +61,8 @@ stop() {
   else
     echo "Not running"
   fi
+  # Cleanup
+  rm -f /run/pihole/FTL.sock /dev/shm/FTL-*
   echo
 }
 
@@ -112,7 +91,7 @@ case "$1" in
         start
         ;;
   *)
-        echo $"Usage: $0 {start|stop|restart|reload|status}"
+        echo "Usage: $0 {start|stop|restart|reload|status}"
         exit 1
 esac
 

advanced/Templates/pihole.cron

@@ -18,19 +18,19 @@
 #          early morning. Download any updates from the adlists
 #          Squash output to log, then splat the log to stdout on error to allow for
 #          standard crontab job error handling.
-59 1    * * 7   root    PATH="$PATH:/usr/local/bin/" pihole updateGravity >/var/log/pihole_updateGravity.log || cat /var/log/pihole_updateGravity.log
+59 1    * * 7   root    PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole updateGravity >/var/log/pihole_updateGravity.log || cat /var/log/pihole_updateGravity.log
 
 # Pi-hole: Flush the log daily at 00:00
 #          The flush script will use logrotate if available
 #          parameter "once": logrotate only once (default is twice)
 #          parameter "quiet": don't print messages
-00 00   * * *   root    PATH="$PATH:/usr/local/bin/" pihole flush once quiet
+00 00   * * *   root    PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole flush once quiet
 
-@reboot root /usr/sbin/logrotate /etc/pihole/logrotate
+@reboot root /usr/sbin/logrotate --state /var/lib/logrotate/pihole /etc/pihole/logrotate
 
 # Pi-hole: Grab local version and branch every 10 minutes
-*/10 *  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker local
+*/10 *  * * *   root    PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole updatechecker local
 
 # Pi-hole: Grab remote version every 24 hours
-59 17  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker remote
+59 17  * * *   root    PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole updatechecker remote
-@reboot root    PATH="$PATH:/usr/local/bin/" pihole updatechecker remote reboot
+@reboot root    PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole updatechecker remote reboot

advanced/bash-completion/pihole

@@ -56,11 +56,11 @@ _pihole() {
 		;;
 		"privacylevel")
 			if ( [[ "$prev2" == "admin" ]] || [[ "$prev2" == "-a" ]] ); then
-				opts_privacy="0 1 2 3 4"
+				opts_privacy="0 1 2 3"
 				COMPREPLY=( $(compgen -W "${opts_privacy}" -- ${cur}) )
-			else 
+			else
 				return 1
-			fi	
+			fi
 		;;
 		"core"|"admin"|"ftl")
 			if [[ "$prev2" == "checkout" ]]; then

advanced/blockingpage.css

@@ -145,7 +145,17 @@ body {
 }
 
 /* User is greeted with a splash page when browsing to Pi-hole IP address */
-#splashpage { background: #222; color: rgba(255, 255, 255, 0.7); text-align: center; }
+#splashpage {
+  background: #222;
+  color: rgba(255, 255, 255, 0.7);
+  text-align: center;
+  width: 100%;
+  height: 100%;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+}
+
 #splashpage img { margin: 5px; width: 256px; }
 #splashpage b { color: inherit; }
 
@@ -196,6 +206,26 @@ header #bpAlt label {
   display: block;
 }
 
+html, body {
+    height: 100%;
+}
+
+#pihole_card {
+    width: 400px;
+    height: auto;
+    max-width: 400px;
+}
+
+    #pihole_card p, #pihole_card a {
+        font-size: 13pt;
+        text-align: center;
+    }
+
+#pihole_logo_splash {
+    height: auto;
+    width: 100%;
+}
+
 /* Click anywhere else on screen to hide #bpAbout */
 #bpAboutToggle:checked {
   display: block;
@@ -382,12 +412,44 @@ footer {
 
 /* Responsive Content */
 @media only screen and (max-width: 500px) {
-  h1 a { font-size: 1.8rem; min-width: 170px; }
+  h1 a {
-  footer span::before { content: "Generated "; }
+        font-size: 1.8rem;
-  footer span { display: block; }
+        min-width: 170px;
+    }
+
+    footer span::before {
+        content: "Generated ";
+    }
+
+    footer span {
+        display: block;
+    }
 }
 
 @media only screen and (min-width: 1251px) {
-  #bpWrapper, footer { border-radius: 0 0 5px 5px; }
+  #bpWrapper, footer {
-  #bpAbout { border-right-width: 1px; }
+        border-radius: 0 0 5px 5px;
+    }
+
+    #bpAbout {
+        border-right-width: 1px;
+    }
+}
+
+@media only screen and (max-width: 400px) {
+    #pihole_card {
+        width: 100%;
+        height: auto;
+    }
+
+        #pihole_card p, #pihole_card a {
+            font-size: 100%;
+        }
+}
+
+@media only screen and (max-width: 256px) {
+    #pihole_logo_splash {
+        width: 90% !important;
+        height: auto;
+    }
 }

advanced/index.php

@@ -11,20 +11,11 @@ $serverName = htmlspecialchars($_SERVER["SERVER_NAME"]);
 // Remove external ipv6 brackets if any
 $serverName = preg_replace('/^\[(.*)\]$/', '${1}', $serverName);
 
-if (!is_file("/etc/pihole/setupVars.conf"))
-  die("[ERROR] File not found: <code>/etc/pihole/setupVars.conf</code>");
-
-// Get values from setupVars.conf
-$setupVars = parse_ini_file("/etc/pihole/setupVars.conf");
-$svPasswd = !empty($setupVars["WEBPASSWORD"]);
-$svEmail = (!empty($setupVars["ADMIN_EMAIL"]) && filter_var($setupVars["ADMIN_EMAIL"], FILTER_VALIDATE_EMAIL)) ? $setupVars["ADMIN_EMAIL"] : "";
-unset($setupVars);
-
 // Set landing page location, found within /var/www/html/
 $landPage = "../landing.php";
 
 // Define array for hostnames to be accepted as self address for splash page
-$authorizedHosts = [];
+$authorizedHosts = [ "localhost" ];
 if (!empty($_SERVER["FQDN"])) {
     // If setenv.add-environment = ("fqdn" => "true") is configured in lighttpd,
     // append $serverName to $authorizedHosts
@@ -34,37 +25,36 @@ if (!empty($_SERVER["FQDN"])) {
     array_push($authorizedHosts, $_SERVER["VIRTUAL_HOST"]);
 }
 
-// Set which extension types render as Block Page (Including "" for index.ext)
-$validExtTypes = array("asp", "htm", "html", "php", "rss", "xml", "");
-
-// Get extension of current URL
-$currentUrlExt = pathinfo($_SERVER["REQUEST_URI"], PATHINFO_EXTENSION);
-
-// Set mobile friendly viewport
-$viewPort = '<meta name="viewport" content="width=device-width, initial-scale=1">';
-
-// Set response header
-function setHeader($type = "x") {
-    header("X-Pi-hole: A black hole for Internet advertisements.");
-    if (isset($type) && $type === "js") header("Content-Type: application/javascript");
-}
-
 // Determine block page type
 if ($serverName === "pi.hole"
     || (!empty($_SERVER["VIRTUAL_HOST"]) && $serverName === $_SERVER["VIRTUAL_HOST"])) {
     // Redirect to Web Interface
     exit(header("Location: /admin"));
 } elseif (filter_var($serverName, FILTER_VALIDATE_IP) || in_array($serverName, $authorizedHosts)) {
-    // Set Splash Page output
+    // When directly browsing via IP or authorized hostname
-    $splashPage = "
+    // Render splash/landing page based off presence of $landPage file
+    // Unset variables so as to not be included in $landPage or $splashPage
+    unset($svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt);
+    // If $landPage file is present
+    if (is_file(getcwd()."/$landPage")) {
+        unset($serverName, $viewPort); // unset extra variables not to be included in $landpage
+        include $landPage;
+        exit();
+    }
+    // If $landPage file was not present, Set Splash Page output
+    $splashPage = <<<EOT
     <!doctype html>
     <html lang='en'>
         <head>
             <meta charset='utf-8'>
-            $viewPort
+            <meta name='viewport' content='width=device-width, initial-scale=1'>
             <title>● $serverName</title>
-            <link rel='stylesheet' href='pihole/blockingpage.css'>
             <link rel='shortcut icon' href='admin/img/favicons/favicon.ico' type='image/x-icon'>
+            <style>
+              #splashpage { background: #222; color: rgba(255, 255, 255, 0.7); text-align: center; }
+              #splashpage img { margin: 5px; width: 256px; }
+              #splashpage b { color: inherit; }
+            </style>
         </head>
         <body id='splashpage'>
             <img src='admin/img/logo.svg' alt='Pi-hole logo' width='256' height='377'>
@@ -73,320 +63,10 @@ if ($serverName === "pi.hole"
             <a href='/admin'>Did you mean to go to the admin panel?</a>
         </body>
     </html>
-    ";
+EOT;
-
+    exit($splashPage);
-    // Set splash/landing page based off presence of $landPage
-    $renderPage = is_file(getcwd()."/$landPage") ? include $landPage : "$splashPage";
-
-    // Unset variables so as to not be included in $landPage
-    unset($serverName, $svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt, $viewPort);
-
-    // Render splash/landing page when directly browsing via IP or authorized hostname
-    exit($renderPage);
-} elseif ($currentUrlExt === "js") {
-    // Serve Pi-hole JavaScript for blocked domains requesting JS
-    exit(setHeader("js").'var x = "Pi-hole: A black hole for Internet advertisements."');
-} elseif (strpos($_SERVER["REQUEST_URI"], "?") !== FALSE && isset($_SERVER["HTTP_REFERER"])) {
-    // Serve blank image upon receiving REQUEST_URI w/ query string & HTTP_REFERRER
-    // e.g: An iframe of a blocked domain
-    exit(setHeader().'<!doctype html>
-    <html lang="en">
-        <head>
-            <meta charset="utf-8"><script>window.close();</script>
-        </head>
-        <body>
-            <img src="data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=">
-        </body>
-    </html>');
-} elseif (!in_array($currentUrlExt, $validExtTypes) || substr_count($_SERVER["REQUEST_URI"], "?")) {
-    // Serve SVG upon receiving non $validExtTypes URL extension or query string
-    // e.g: Not an iframe of a blocked domain, such as when browsing to a file/query directly
-    // QoL addition: Allow the SVG to be clicked on in order to quickly show the full Block Page
-    $blockImg = '<a href="/">
-    <svg xmlns="http://www.w3.org/2000/svg" width="110" height="16">
-      <circle cx="8" cy="8" r="7" fill="none" stroke="rgba(152,2,2,.5)" stroke-width="2"/>
-      <path fill="rgba(152,2,2,.5)" d="M11.526 3.04l1.414 1.415-8.485 8.485-1.414-1.414z"/>
-      <text x="19.3" y="12" opacity=".3" style="font:11px Arial">
-        Blocked by Pi-hole
-      </text>
-    </svg>
-    </a>';
-    exit(setHeader()."<!doctype html>
-    <html lang='en'>
-        <head>
-            <meta charset='utf-8'>
-            $viewPort
-        </head>
-        <body>$blockImg</body>
-    </html>");
 }
 
-/* Start processing Block Page from here */
+exit(header("HTTP/1.1 404 Not Found"));
 
-// Define admin email address text based off $svEmail presence
-$bpAskAdmin = !empty($svEmail) ? '<a href="mailto:'.$svEmail.'?subject=Site Blocked: '.$serverName.'"></a>' : "<span/>";
-
-// Get possible non-standard location of FTL's database
-$FTLsettings = parse_ini_file("/etc/pihole/pihole-FTL.conf");
-if (isset($FTLsettings["GRAVITYDB"])) {
-    $gravityDBFile = $FTLsettings["GRAVITYDB"];
-} else {
-    $gravityDBFile = "/etc/pihole/gravity.db";
-}
-
-// Connect to gravity.db
-try {
-    $db = new SQLite3($gravityDBFile, SQLITE3_OPEN_READONLY);
-} catch (Exception $exception) {
-    die("[ERROR]: Failed to connect to gravity.db");
-}
-
-// Get all adlist addresses
-$adlistResults = $db->query("SELECT address FROM vw_adlist");
-$adlistsUrls = array();
-while ($row = $adlistResults->fetchArray()) {
-    array_push($adlistsUrls, $row[0]);
-}
-
-if (empty($adlistsUrls))
-    die("[ERROR]: There are no adlists enabled");
-
-// Get total number of blocklists (Including Whitelist, Blacklist & Wildcard lists)
-$adlistsCount = count($adlistsUrls) + 3;
-
-// Set query timeout
-ini_set("default_socket_timeout", 3);
-
-// Logic for querying blocklists
-function queryAds($serverName) {
-    // Determine the time it takes while querying adlists
-    $preQueryTime = microtime(true)-$_SERVER["REQUEST_TIME_FLOAT"];
-    $queryAdsURL = sprintf(
-        "http://127.0.0.1:%s/admin/scripts/pi-hole/php/queryads.php?domain=%s&bp",
-        $_SERVER["SERVER_PORT"],
-        $serverName
-    );
-    $queryAds = file($queryAdsURL, FILE_IGNORE_NEW_LINES);
-    $queryAds = array_values(array_filter(preg_replace("/data:\s+/", "", $queryAds)));
-    $queryTime = sprintf("%.0f", (microtime(true)-$_SERVER["REQUEST_TIME_FLOAT"]) - $preQueryTime);
-
-    // Exception Handling
-    try {
-        // Define Exceptions
-        if (strpos($queryAds[0], "No exact results") !== FALSE) {
-            // Return "none" into $queryAds array
-            return array("0" => "none");
-        } else if ($queryTime >= ini_get("default_socket_timeout")) {
-            // Connection Timeout
-            throw new Exception ("Connection timeout (".ini_get("default_socket_timeout")."s)");
-        } elseif (!strpos($queryAds[0], ".") !== false) {
-            // Unknown $queryAds output
-            throw new Exception ("Unhandled error message (<code>$queryAds[0]</code>)");
-        }
-        return $queryAds;
-    } catch (Exception $e) {
-        // Return exception as array
-        return array("0" => "error", "1" => $e->getMessage());
-    }
-}
-
-// Get results of queryads.php exact search
-$queryAds = queryAds($serverName);
-
-// Pass error through to Block Page
-if ($queryAds[0] === "error")
-    die("[ERROR]: Unable to parse results from <i>queryads.php</i>: <code>".$queryAds[1]."</code>");
-
-// Count total number of matching blocklists
-$featuredTotal = count($queryAds);
-
-// Place results into key => value array
-$queryResults = null;
-foreach ($queryAds as $str) {
-    $value = explode(" ", $str);
-    @$queryResults[$value[0]] .= "$value[1]";
-}
-
-// Determine if domain has been blacklisted, whitelisted, wildcarded or CNAME blocked
-if (strpos($queryAds[0], "blacklist") !== FALSE) {
-    $notableFlagClass = "blacklist";
-    $adlistsUrls = array("π" => substr($queryAds[0], 2));
-} elseif (strpos($queryAds[0], "whitelist") !== FALSE) {
-    $notableFlagClass = "noblock";
-    $adlistsUrls = array("π" => substr($queryAds[0], 2));
-    $wlInfo = "recentwl";
-} elseif (strpos($queryAds[0], "wildcard") !== FALSE) {
-    $notableFlagClass = "wildcard";
-    $adlistsUrls = array("π" => substr($queryAds[0], 2));
-} elseif ($queryAds[0] === "none") {
-    $featuredTotal = "0";
-    $notableFlagClass = "noblock";
-
-    // QoL addition: Determine appropriate info message if CNAME exists
-    // Suggests to the user that $serverName has a CNAME (alias) that may be blocked
-    $dnsRecord = dns_get_record("$serverName")[0];
-    if (array_key_exists("target", $dnsRecord)) {
-        $wlInfo = $dnsRecord['target'];
-    } else {
-        $wlInfo = "unknown";
-    }
-}
-
-// Set #bpOutput notification
-$wlOutputClass = (isset($wlInfo) && $wlInfo === "recentwl") ? $wlInfo : "hidden";
-$wlOutput = (isset($wlInfo) && $wlInfo !== "recentwl") ? "<a href='http://$wlInfo'>$wlInfo</a>" : "";
-
-// Get Pi-hole Core version
-$phVersion = exec("cd /etc/.pihole/ && git describe --long --tags");
-
-// Print $execTime on development branches
-// Testing for - is marginally faster than "git rev-parse --abbrev-ref HEAD"
-if (explode("-", $phVersion)[1] != "0")
-  $execTime = microtime(true)-$_SERVER["REQUEST_TIME_FLOAT"];
-
-// Please Note: Text is added via CSS to allow an admin to provide a localized
-// language without the need to edit this file
-
-setHeader();
 ?>
-<!doctype html>
-<!-- Pi-hole: A black hole for Internet advertisements
-*  (c) 2017 Pi-hole, LLC (https://pi-hole.net)
-*  Network-wide ad blocking via your own hardware.
-*
-*  This file is copyright under the latest version of the EUPL. -->
-<html>
-<head>
-  <meta charset="utf-8">
-  <?=$viewPort ?>
-  <meta name="robots" content="noindex,nofollow">
-  <meta http-equiv="x-dns-prefetch-control" content="off">
-  <link rel="stylesheet" href="pihole/blockingpage.css">
-  <link rel="shortcut icon" href="admin/img/favicons/favicon.ico" type="image/x-icon">
-  <title>● <?=$serverName ?></title>
-  <script src="admin/scripts/vendor/jquery.min.js"></script>
-  <script>
-    window.onload = function () {
-      <?php
-      // Remove href fallback from "Back to safety" button
-      if ($featuredTotal > 0) {
-        echo '$("#bpBack").removeAttr("href");';
-
-        // Enable whitelisting if JS is available
-        echo '$("#bpWhitelist").prop("disabled", false);';
-
-        // Enable password input if necessary
-        if (!empty($svPasswd)) {
-          echo '$("#bpWLPassword").attr("placeholder", "Password");';
-          echo '$("#bpWLPassword").prop("disabled", false);';
-        }
-        // Otherwise hide the input
-        else {
-          echo '$("#bpWLPassword").hide();';
-        }
-      }
-      ?>
-    }
-  </script>
-</head>
-<body id="blockpage"><div id="bpWrapper">
-<header>
-  <h1 id="bpTitle">
-    <a class="title" href="/"><?php //Website Blocked ?></a>
-  </h1>
-  <div class="spc"></div>
-
-  <input id="bpAboutToggle" type="checkbox">
-  <div id="bpAbout">
-    <div class="aboutPH">
-      <div class="aboutImg"></div>
-      <p>Open Source Ad Blocker
-        <small>Designed for Raspberry Pi</small>
-      </p>
-    </div>
-    <div class="aboutLink">
-      <a class="linkPH" href="https://github.com/pi-hole/pi-hole/wiki/What-is-Pi-hole%3F-A-simple-explanation"><?php //About PH ?></a>
-      <?php if (!empty($svEmail)) echo '<a class="linkEmail" href="mailto:'.$svEmail.'"></a>'; ?>
-    </div>
-  </div>
-
-  <div id="bpAlt">
-    <label class="altBtn" for="bpAboutToggle"><?php //Why am I here? ?></label>
-  </div>
-</header>
-
-<main>
-  <div id="bpOutput" class="<?=$wlOutputClass ?>"><?=$wlOutput ?></div>
-  <div id="bpBlock">
-    <p class="blockMsg"><?=$serverName ?></p>
-  </div>
-  <?php if(isset($notableFlagClass)) { ?>
-    <div id="bpFlag">
-        <p class="flagMsg <?=$notableFlagClass ?>"></p>
-    </div>
-  <?php } ?>
-  <div id="bpHelpTxt"><?=$bpAskAdmin ?></div>
-  <div id="bpButtons" class="buttons">
-    <a id="bpBack" onclick="javascript:history.back()" href="about:home"></a>
-    <?php if ($featuredTotal > 0) echo '<label id="bpInfo" for="bpMoreToggle"></label>'; ?>
-  </div>
-  <input id="bpMoreToggle" type="checkbox">
-  <div id="bpMoreInfo">
-    <span id="bpFoundIn"><span><?=$featuredTotal ?></span><?=$adlistsCount ?></span>
-    <pre id='bpQueryOutput'><?php if ($featuredTotal > 0) foreach ($queryResults as $num => $value) { echo "<span>[$num]:</span>$adlistsUrls[$num]\n"; } ?></pre>
-
-    <form id="bpWLButtons" class="buttons">
-      <input id="bpWLDomain" type="text" value="<?=$serverName ?>" disabled>
-      <input id="bpWLPassword" type="password" placeholder="JavaScript disabled" disabled>
-      <button id="bpWhitelist" type="button" disabled></button>
-    </form>
-  </div>
-</main>
-
-<footer><span><?=date("l g:i A, F dS"); ?>.</span> Pi-hole <?=$phVersion ?> (<?=gethostname()."/".$_SERVER["SERVER_ADDR"]; if (isset($execTime)) printf("/%.2fs", $execTime); ?>)</footer>
-</div>
-
-<script>
-  function add() {
-    $("#bpOutput").removeClass("hidden error exception");
-    $("#bpOutput").addClass("add");
-    var domain = "<?=$serverName ?>";
-    var pw = $("#bpWLPassword");
-    if(domain.length === 0) {
-      return;
-    }
-    $.ajax({
-      url: "/admin/scripts/pi-hole/php/add.php",
-      method: "post",
-      data: {"domain":domain, "list":"white", "pw":pw.val()},
-      success: function(response) {
-        if(response.indexOf("Pi-hole blocking") !== -1) {
-          setTimeout(function(){window.location.reload(1);}, 10000);
-          $("#bpOutput").removeClass("add");
-          $("#bpOutput").addClass("success");
-          $("#bpOutput").html("");
-        } else {
-          $("#bpOutput").removeClass("add");
-          $("#bpOutput").addClass("error");
-          $("#bpOutput").html(""+response+"");
-        }
-      },
-      error: function(jqXHR, exception) {
-        $("#bpOutput").removeClass("add");
-        $("#bpOutput").addClass("exception");
-        $("#bpOutput").html("");
-      }
-    });
-  }
-  <?php if ($featuredTotal > 0) { ?>
-    $(document).keypress(function(e) {
-        if(e.which === 13 && $("#bpWLPassword").is(":focus")) {
-            add();
-        }
-    });
-    $("#bpWhitelist").on("click", function() {
-        add();
-    });
-  <?php } ?>
-</script>
-</body></html>

advanced/lighttpd.conf.debian

@@ -20,7 +20,6 @@ server.modules = (
     "mod_accesslog",
     "mod_auth",
     "mod_expire",
-    "mod_compress",
     "mod_redirect",
     "mod_setenv",
     "mod_rewrite"
@@ -41,26 +40,6 @@ index-file.names            = ( "index.php", "index.html", "index.lighttpd.html"
 url.access-deny             = ( "~", ".inc", ".md", ".yml", ".ini" )
 static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
 
-compress.cache-dir = "/var/cache/lighttpd/compress/"
-compress.filetype  = (
-    "application/json",
-    "application/vnd.ms-fontobject",
-    "application/xml",
-    "font/eot",
-    "font/opentype",
-    "font/otf",
-    "font/ttf",
-    "image/bmp",
-    "image/svg+xml",
-    "image/vnd.microsoft.icon",
-    "image/x-icon",
-    "text/css",
-    "text/html",
-    "text/javascript",
-    "text/plain",
-    "text/xml"
-)
-
 mimetype.assign = (
     ".ico"   => "image/x-icon",
     ".jpeg"  => "image/jpeg",
@@ -81,6 +60,10 @@ mimetype.assign = (
     ".woff2" => "font/woff2"
 )
 
+# Add user chosen options held in external file
+# This uses include_shell instead of an include wildcard for compatibility
+include_shell "cat external.conf 2>/dev/null"
+
 # default listening port for IPv6 falls back to the IPv4 port
 include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
 
@@ -95,11 +78,6 @@ $HTTP["url"] =~ "^/admin/" {
         "X-Pi-hole" => "The Pi-hole Web interface is working!",
         "X-Frame-Options" => "DENY"
     )
-
-    $HTTP["url"] =~ "\.(eot|otf|tt[cf]|woff2?)$" {
-        # Allow Block Page access to local fonts
-        setenv.add-response-header = ( "Access-Control-Allow-Origin" => "*" )
-    }
 }
 
 # Block . files from being served, such as .git, .github, .gitignore
@@ -109,7 +87,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {
 
 # Default expire header
 expire.url = ( "" => "access plus 0 seconds" )
-
-# Add user chosen options held in external file
-# This uses include_shell instead of an include wildcard for compatibility
-include_shell "cat external.conf 2>/dev/null"

advanced/lighttpd.conf.fedora

@@ -21,7 +21,6 @@ server.modules = (
     "mod_expire",
     "mod_fastcgi",
     "mod_accesslog",
-    "mod_compress",
     "mod_redirect",
     "mod_setenv",
     "mod_rewrite"
@@ -42,26 +41,6 @@ index-file.names            = ( "index.php", "index.html", "index.lighttpd.html"
 url.access-deny             = ( "~", ".inc", ".md", ".yml", ".ini" )
 static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
 
-compress.cache-dir = "/var/cache/lighttpd/compress/"
-compress.filetype  = (
-    "application/json",
-    "application/vnd.ms-fontobject",
-    "application/xml",
-    "font/eot",
-    "font/opentype",
-    "font/otf",
-    "font/ttf",
-    "image/bmp",
-    "image/svg+xml",
-    "image/vnd.microsoft.icon",
-    "image/x-icon",
-    "text/css",
-    "text/html",
-    "text/javascript",
-    "text/plain",
-    "text/xml"
-)
-
 mimetype.assign = (
     ".ico"   => "image/x-icon",
     ".jpeg"  => "image/jpeg",
@@ -82,6 +61,10 @@ mimetype.assign = (
     ".woff2" => "font/woff2"
 )
 
+# Add user chosen options held in external file
+# This uses include_shell instead of an include wildcard for compatibility
+include_shell "cat external.conf 2>/dev/null"
+
 # default listening port for IPv6 falls back to the IPv4 port
 #include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
 #include_shell "/usr/share/lighttpd/create-mime.assign.pl"
@@ -103,11 +86,6 @@ $HTTP["url"] =~ "^/admin/" {
         "X-Pi-hole" => "The Pi-hole Web interface is working!",
         "X-Frame-Options" => "DENY"
     )
-
-    $HTTP["url"] =~ "\.(eot|otf|tt[cf]|woff2?)$" {
-        # Allow Block Page access to local fonts
-        setenv.add-response-header = ( "Access-Control-Allow-Origin" => "*" )
-    }
 }
 
 # Block . files from being served, such as .git, .github, .gitignore
@@ -117,7 +95,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {
 
 # Default expire header
 expire.url = ( "" => "access plus 0 seconds" )
-
-# Add user chosen options held in external file
-# This uses include_shell instead of an include wildcard for compatibility
-include_shell "cat external.conf 2>/dev/null"

automated install/uninstall.sh

@@ -31,7 +31,7 @@ else
     else
         echo -e "  ${CROSS} ${str}
             Script called with non-root privileges
-            The Pi-hole requires elevated privleges to uninstall"
+            The Pi-hole requires elevated privileges to uninstall"
         exit 1
     fi
 fi
@@ -42,8 +42,8 @@ source "${PI_HOLE_FILES_DIR}/automated install/basic-install.sh"
 # setupVars set in basic-install.sh
 source "${setupVars}"
 
-# distro_check() sourced from basic-install.sh
+# package_manager_detect() sourced from basic-install.sh
-distro_check
+package_manager_detect
 
 # Install packages used by the Pi-hole
 DEPS=("${INSTALLER_DEPS[@]}" "${PIHOLE_DEPS[@]}")
@@ -113,7 +113,7 @@ removeNoPurge() {
         fi
     fi
     echo -e "${OVER}  ${TICK} Removed Web Interface"
- 
+
     # Attempt to preserve backwards compatibility with older versions
     # to guarantee no additional changes were made to /etc/crontab after
     # the installation of pihole, /etc/crontab.pihole should be permanently
@@ -145,6 +145,7 @@ removeNoPurge() {
 
     ${SUDO} rm -f /etc/dnsmasq.d/adList.conf &> /dev/null
     ${SUDO} rm -f /etc/dnsmasq.d/01-pihole.conf &> /dev/null
+    ${SUDO} rm -f /etc/dnsmasq.d/06-rfc6761.conf &> /dev/null
     ${SUDO} rm -rf /var/log/*pihole* &> /dev/null
     ${SUDO} rm -rf /etc/pihole/ &> /dev/null
     ${SUDO} rm -rf /etc/.pihole/ &> /dev/null
@@ -206,11 +207,7 @@ removeNoPurge() {
 }
 
 ######### SCRIPT ###########
-if command -v vcgencmd &> /dev/null; then
+echo -e "  ${INFO} Be sure to confirm if any dependencies should not be removed"
-    echo -e "  ${INFO} All dependencies are safe to remove on Raspbian"
-else
-    echo -e "  ${INFO} Be sure to confirm if any dependencies should not be removed"
-fi
 while true; do
     echo -e "  ${INFO} ${COL_YELLOW}The following dependencies may have been added by the Pi-hole install:"
     echo -n "    "

autotest

@@ -1 +0,0 @@
-py.test -v -f test/

gravity.sh

@@ -15,8 +15,6 @@ export LC_ALL=C
 
 coltable="/opt/pihole/COL_TABLE"
 source "${coltable}"
-regexconverter="/opt/pihole/wildcard_regex_converter.sh"
-source "${regexconverter}"
 # shellcheck disable=SC1091
 source "/etc/.pihole/advanced/Scripts/database_migration/gravity-db.sh"
 
@@ -35,30 +33,18 @@ localList="${piholeDir}/local.list"
 VPNList="/etc/openvpn/ipp.txt"
 
 piholeGitDir="/etc/.pihole"
-gravityDBfile="${piholeDir}/gravity.db"
+gravityDBfile_default="${piholeDir}/gravity.db"
-gravityTEMPfile="${piholeDir}/gravity_temp.db"
+# GRAVITYDB may be overwritten by source pihole-FTL.conf below
+GRAVITYDB="${gravityDBfile_default}"
 gravityDBschema="${piholeGitDir}/advanced/Templates/gravity.db.sql"
 gravityDBcopy="${piholeGitDir}/advanced/Templates/gravity_copy.sql"
-optimize_database=false
 
 domainsExtension="domains"
 
-resolver="pihole-FTL"
-
 # Source setupVars from install script
 setupVars="${piholeDir}/setupVars.conf"
 if [[ -f "${setupVars}" ]];then
   source "${setupVars}"
-
-  # Remove CIDR mask from IPv4/6 addresses
-  IPV4_ADDRESS="${IPV4_ADDRESS%/*}"
-  IPV6_ADDRESS="${IPV6_ADDRESS%/*}"
-
-  # Determine if IPv4/6 addresses exist
-  if [[ -z "${IPV4_ADDRESS}" ]] && [[ -z "${IPV6_ADDRESS}" ]]; then
-    echo -e "  ${COL_LIGHT_RED}No IP addresses found! Please run 'pihole -r' to reconfigure${COL_NC}"
-    exit 1
-  fi
 else
   echo -e "  ${COL_LIGHT_RED}Installation Failure: ${setupVars} does not exist! ${COL_NC}
   Please run 'pihole -r', and choose the 'reconfigure' option to fix."
@@ -71,6 +57,13 @@ if [[ -f "${pihole_FTL}" ]]; then
   source "${pihole_FTL}"
 fi
 
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"
+gravityTEMPfile="${GRAVITYDB}_temp"
+gravityDIR="$(dirname -- "${gravityDBfile}")"
+gravityOLDfile="${gravityDIR}/gravity_old.db"
+
 if [[ -z "${BLOCKINGMODE}" ]] ; then
   BLOCKINGMODE="NULL"
 fi
@@ -87,11 +80,11 @@ generate_gravity_database() {
 
 # Copy data from old to new database file and swap them
 gravity_swap_databases() {
-  local str
+  local str copyGravity
   str="Building tree"
   echo -ne "  ${INFO} ${str}..."
 
-  # The index is intentionally not UNIQUE as prro quality adlists may contain domains more than once
+  # The index is intentionally not UNIQUE as poor quality adlists may contain domains more than once
   output=$( { sqlite3 "${gravityTEMPfile}" "CREATE INDEX idx_gravity ON gravity (domain, adlist_id);"; } 2>&1 )
   status="$?"
 
@@ -104,7 +97,14 @@ gravity_swap_databases() {
   str="Swapping databases"
   echo -ne "  ${INFO} ${str}..."
 
-  output=$( { sqlite3 "${gravityTEMPfile}" < "${gravityDBcopy}"; } 2>&1 )
+  # Gravity copying SQL script
+  copyGravity="$(cat "${gravityDBcopy}")"
+  if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
+    # Replace default gravity script location by custom location
+    copyGravity="${copyGravity//"${gravityDBfile_default}"/"${gravityDBfile}"}"
+  fi
+
+  output=$( { sqlite3 "${gravityTEMPfile}" <<< "${copyGravity}"; } 2>&1 )
   status="$?"
 
   if [[ "${status}" -ne 0 ]]; then
@@ -113,8 +113,19 @@ gravity_swap_databases() {
   fi
   echo -e "${OVER}  ${TICK} ${str}"
 
-  # Swap databases and remove old database
+  # Swap databases and remove or conditionally rename old database
-  rm "${gravityDBfile}"
+  # Number of available blocks on disk
+  availableBlocks=$(stat -f --format "%a" "${gravityDIR}")
+  # Number of blocks, used by gravity.db
+  gravityBlocks=$(stat --format "%b" ${gravityDBfile})
+  # Only keep the old database if available disk space is at least twice the size of the existing gravity.db.
+  # Better be safe than sorry...
+  if [ "${availableBlocks}" -gt "$((gravityBlocks * 2))" ] && [ -f "${gravityDBfile}" ]; then
+    echo -e "  ${TICK} The old database remains available."
+    mv "${gravityDBfile}" "${gravityOLDfile}"
+  else
+    rm "${gravityDBfile}"
+  fi
   mv "${gravityTEMPfile}" "${gravityDBfile}"
 }
 
@@ -179,7 +190,7 @@ database_table_from_file() {
         echo "${rowid},\"${domain}\",${timestamp}" >> "${tmpFile}"
       elif [[ "${table}" == "adlist" ]]; then
         # Adlist table format
-        echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"" >> "${tmpFile}"
+        echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\",,0,0,0" >> "${tmpFile}"
       else
         # White-, black-, and regexlist table format
         echo "${rowid},${type},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"" >> "${tmpFile}"
@@ -209,6 +220,59 @@ database_table_from_file() {
     echo -e "  ${CROSS} Unable to remove ${tmpFile}"
 }
 
+# Update timestamp of last update of this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_updated() {
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET date_updated = (cast(strftime('%%s', 'now') as int)) WHERE id = %i;\\n" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update timestamp of adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
+# Check if a column with name ${2} exists in gravity table with name ${1}
+gravity_column_exists() {
+  output=$( { printf ".timeout 30000\\nSELECT EXISTS(SELECT * FROM pragma_table_info('%s') WHERE name='%s');\\n" "${1}" "${2}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  if [[ "${output}" == "1" ]]; then
+    return 0 # Bash 0 is success
+  fi
+
+  return 1 # Bash non-0 is failure
+}
+
+# Update number of domain on this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_number() {
+  # Only try to set number of domains when this field exists in the gravity database
+  if ! gravity_column_exists "adlist" "number"; then
+    return;
+  fi
+
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET number = %i, invalid_domains = %i WHERE id = %i;\\n" "${num_lines}" "${num_invalid}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update number of domains in adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
+# Update status of this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_status() {
+  # Only try to set the status when this field exists in the gravity database
+  if ! gravity_column_exists "adlist" "status"; then
+    return;
+  fi
+
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET status = %i WHERE id = %i;\\n" "${2}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update status of adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
 # Migrate pre-v5.0 list files to database-based Pi-hole versions
 migrate_to_database() {
   # Create database file only if not present
@@ -284,7 +348,7 @@ gravity_CheckDNSResolutionAvailable() {
   fi
 
   # Determine error output message
-  if pidof ${resolver} &> /dev/null; then
+  if pgrep pihole-FTL &> /dev/null; then
     echo -e "  ${CROSS} DNS resolution is currently unavailable"
   else
     echo -e "  ${CROSS} DNS service is not running"
@@ -309,6 +373,10 @@ gravity_CheckDNSResolutionAvailable() {
 gravity_DownloadBlocklists() {
   echo -e "  ${INFO} ${COL_BOLD}Neutrino emissions detected${COL_NC}..."
 
+  if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
+    echo -e "  ${INFO} Storing gravity database in ${COL_BOLD}${gravityDBfile}${COL_NC}"
+  fi
+
   # Retrieve source URLs from gravity database
   # We source only enabled adlists, sqlite3 stores boolean values as 0 (false) or 1 (true)
   mapfile -t sources <<< "$(sqlite3 "${gravityDBfile}" "SELECT address FROM vw_adlist;" 2> /dev/null)"
@@ -336,7 +404,7 @@ gravity_DownloadBlocklists() {
     return 1
   fi
 
-  local url domain agent cmd_ext str target
+  local url domain agent cmd_ext str target compression
   echo ""
 
   # Prepare new gravity database
@@ -355,13 +423,24 @@ gravity_DownloadBlocklists() {
 
   target="$(mktemp -p "/tmp" --suffix=".gravity")"
 
+  # Use compression to reduce the amount of data that is transferred
+  # between the Pi-hole and the ad list provider. Use this feature
+  # only if it is supported by the locally available version of curl
+  if curl -V | grep -q "Features:.* libz"; then
+    compression="--compressed"
+    echo -e "  ${INFO} Using libz compression\n"
+  else
+      compression=""
+      echo -e "  ${INFO} Libz compression not available\n"
+    fi
   # Loop through $sources and download each one
   for ((i = 0; i < "${#sources[@]}"; i++)); do
     url="${sources[$i]}"
     domain="${sourceDomains[$i]}"
+    id="${sourceIDs[$i]}"
 
     # Save the file as list.#.domain
-    saveLocation="${piholeDir}/list.${i}.${domain}.${domainsExtension}"
+    saveLocation="${piholeDir}/list.${id}.${domain}.${domainsExtension}"
     activeDomains[$i]="${saveLocation}"
 
     # Default user-agent (for Cloudflare's Browser Integrity Check: https://support.cloudflare.com/hc/en-us/articles/200170086-What-does-the-Browser-Integrity-Check-do-)
@@ -374,13 +453,18 @@ gravity_DownloadBlocklists() {
     esac
 
     echo -e "  ${INFO} Target: ${url}"
-    local regex
+    local regex check_url
     # Check for characters NOT allowed in URLs
     regex="[^a-zA-Z0-9:/?&%=~._()-;]"
-    if [[ "${url}" =~ ${regex} ]]; then
+
+    # this will remove first @ that is after schema and before domain
+    # \1 is optional schema, \2 is userinfo
+    check_url="$( sed -re 's#([^:/]*://)?([^/]+)@#\1\2#' <<< "$url" )"
+
+    if [[ "${check_url}" =~ ${regex} ]]; then
         echo -e "  ${CROSS} Invalid Target"
     else
-       gravity_DownloadBlocklistFromUrl "${url}" "${cmd_ext}" "${agent}" "${sourceIDs[$i]}" "${saveLocation}" "${target}"
+       gravity_DownloadBlocklistFromUrl "${url}" "${cmd_ext}" "${agent}" "${sourceIDs[$i]}" "${saveLocation}" "${target}" "${compression}"
     fi
     echo ""
   done
@@ -420,6 +504,8 @@ gravity_DownloadBlocklists() {
 }
 
 total_num=0
+num_lines=0
+num_invalid=0
 parseList() {
   local adlistID="${1}" src="${2}" target="${3}" incorrect_lines
   # This sed does the following things:
@@ -430,7 +516,7 @@ parseList() {
   # Find (up to) five domains containing invalid characters (see above)
   incorrect_lines="$(sed -e "/[^a-zA-Z0-9.\_-]/!d" "${src}" | head -n 5)"
 
-  local num_lines num_target_lines num_correct_lines num_invalid
+  local num_target_lines num_correct_lines num_invalid
   # Get number of lines in source file
   num_lines="$(grep -c "^" "${src}")"
   # Get number of lines in destination file
@@ -439,9 +525,9 @@ parseList() {
   total_num="$num_target_lines"
   num_invalid="$(( num_lines-num_correct_lines ))"
   if [[ "${num_invalid}" -eq 0 ]]; then
-    echo "  ${INFO} Received ${num_lines} domains"
+    echo "  ${INFO} Analyzed ${num_lines} domains"
   else
-    echo "  ${INFO} Received ${num_lines} domains, ${num_invalid} domains invalid!"
+    echo "  ${INFO} Analyzed ${num_lines} domains, ${num_invalid} domains invalid!"
   fi
 
   # Display sample of invalid lines if we found some
@@ -452,11 +538,34 @@ parseList() {
     done <<< "${incorrect_lines}"
   fi
 }
+compareLists() {
+  local adlistID="${1}" target="${2}"
+
+  # Verify checksum when an older checksum exists
+  if [[ -s "${target}.sha1" ]]; then
+    if ! sha1sum --check --status --strict "${target}.sha1"; then
+      # The list changed upstream, we need to update the checksum
+      sha1sum "${target}" > "${target}.sha1"
+      echo "  ${INFO} List has been updated"
+      database_adlist_status "${adlistID}" "1"
+      database_adlist_updated "${adlistID}"
+    else
+      echo "  ${INFO} List stayed unchanged"
+      database_adlist_status "${adlistID}" "2"
+    fi
+  else
+    # No checksum available, create one for comparing on the next run
+    sha1sum "${target}" > "${target}.sha1"
+    # We assume here it was changed upstream
+    database_adlist_status "${adlistID}" "1"
+    database_adlist_updated "${adlistID}"
+  fi
+}
 
 # Download specified URL and perform checks on HTTP status and file content
 gravity_DownloadBlocklistFromUrl() {
-  local url="${1}" cmd_ext="${2}" agent="${3}" adlistID="${4}" saveLocation="${5}" target="${6}"
+  local url="${1}" cmd_ext="${2}" agent="${3}" adlistID="${4}" saveLocation="${5}" target="${6}" compression="${7}"
-  local heisenbergCompensator="" patternBuffer str httpCode success=""
+  local heisenbergCompensator="" patternBuffer str httpCode success="" ip
 
   # Create temp file to store content on disk instead of RAM
   patternBuffer=$(mktemp -p "/tmp" --suffix=".phgpb")
@@ -474,7 +583,10 @@ gravity_DownloadBlocklistFromUrl() {
   blocked=false
   case $BLOCKINGMODE in
     "IP-NODATA-AAAA"|"IP")
-        if [[ $(dig "${domain}" +short | grep "${IPV4_ADDRESS}" -c) -ge 1 ]]; then
+        # Get IP address of this domain
+        ip="$(dig "${domain}" +short)"
+        # Check if this IP matches any IP of the system
+        if [[ -n "${ip}" && $(grep -Ec "inet(|6) ${ip}" <<< "$(ip a)") -gt 0 ]]; then
           blocked=true
         fi;;
     "NXDOMAIN")
@@ -504,8 +616,9 @@ gravity_DownloadBlocklistFromUrl() {
     echo -ne "  ${INFO} ${str} Pending..."
     cmd_ext="--resolve $domain:$port:$ip $cmd_ext"
   fi
+
   # shellcheck disable=SC2086
-  httpCode=$(curl -s -L ${cmd_ext} ${heisenbergCompensator} -w "%{http_code}" -A "${agent}" "${url}" -o "${patternBuffer}" 2> /dev/null)
+  httpCode=$(curl -s -L ${compression} ${cmd_ext} ${heisenbergCompensator} -w "%{http_code}" -A "${agent}" "${url}" -o "${patternBuffer}" 2> /dev/null)
 
   case $url in
     # Did we "download" a local file?
@@ -534,29 +647,49 @@ gravity_DownloadBlocklistFromUrl() {
       esac;;
   esac
 
+  local done="false"
   # Determine if the blocklist was downloaded and saved correctly
   if [[ "${success}" == true ]]; then
     if [[ "${httpCode}" == "304" ]]; then
       # Add domains to database table file
       parseList "${adlistID}" "${saveLocation}" "${target}"
+      database_adlist_status "${adlistID}" "2"
+      database_adlist_number "${adlistID}"
+      done="true"
     # Check if $patternbuffer is a non-zero length file
     elif [[ -s "${patternBuffer}" ]]; then
       # Determine if blocklist is non-standard and parse as appropriate
       gravity_ParseFileIntoDomains "${patternBuffer}" "${saveLocation}"
       # Add domains to database table file
       parseList "${adlistID}" "${saveLocation}" "${target}"
+      # Compare lists, are they identical?
+      compareLists "${adlistID}" "${saveLocation}"
+      # Update gravity database table (status and updated timestamp are set in
+      # compareLists)
+      database_adlist_number "${adlistID}"
+      done="true"
     else
       # Fall back to previously cached list if $patternBuffer is empty
-      echo -e "  ${INFO} Received empty file: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
+      echo -e "  ${INFO} Received empty file"
     fi
-  else
+  fi
+
+  # Do we need to fall back to a cached list (if available)?
+  if [[ "${done}" != "true" ]]; then
     # Determine if cached list has read permission
     if [[ -r "${saveLocation}" ]]; then
       echo -e "  ${CROSS} List download failed: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
       # Add domains to database table file
       parseList "${adlistID}" "${saveLocation}" "${target}"
+      database_adlist_number "${adlistID}"
+      database_adlist_status "${adlistID}" "3"
     else
       echo -e "  ${CROSS} List download failed: ${COL_LIGHT_RED}no cached list available${COL_NC}"
+      # Manually reset these two numbers because we do not call parseList here
+      num_lines=0
+      num_invalid=0
+      database_adlist_number "${adlistID}"
+      database_adlist_status "${adlistID}" "4"
     fi
   fi
 }
@@ -568,7 +701,7 @@ gravity_ParseFileIntoDomains() {
   # Determine if we are parsing a consolidated list
   #if [[ "${source}" == "${piholeDir}/${matterAndLight}" ]]; then
     # Remove comments and print only the domain name
-    # Most of the lists downloaded are already in hosts file format but the spacing/formating is not contiguous
+    # Most of the lists downloaded are already in hosts file format but the spacing/formatting is not contiguous
     # This helps with that and makes it easier to read
     # It also helps with debugging so each stage of the script can be researched more in depth
     # 1) Remove carriage returns
@@ -656,43 +789,12 @@ gravity_ShowCount() {
   gravity_Table_Count "vw_regex_whitelist" "regex whitelist filters"
 }
 
-# Parse list of domains into hosts format
-gravity_ParseDomainsIntoHosts() {
-  awk -v ipv4="$IPV4_ADDRESS" -v ipv6="$IPV6_ADDRESS" '{
-    # Remove windows CR line endings
-    sub(/\r$/, "")
-    # Parse each line as "ipaddr domain"
-    if(ipv6 && ipv4) {
-      print ipv4" "$0"\n"ipv6" "$0
-    } else if(!ipv6) {
-      print ipv4" "$0
-    } else {
-      print ipv6" "$0
-    }
-  }' >> "${2}" < "${1}"
-}
-
 # Create "localhost" entries into hosts format
 gravity_generateLocalList() {
-  local hostname
-
-  if [[ -s "/etc/hostname" ]]; then
-    hostname=$(< "/etc/hostname")
-  elif command -v hostname &> /dev/null; then
-    hostname=$(hostname -f)
-  else
-    echo -e "  ${CROSS} Unable to determine fully qualified domain name of host"
-    return 0
-  fi
-
-  echo -e "${hostname}\\npi.hole" > "${localList}.tmp"
-
   # Empty $localList if it already exists, otherwise, create it
-  : > "${localList}"
+  echo "### Do not modify this file, it will be overwritten by pihole -g" > "${localList}"
   chmod 644 "${localList}"
 
-  gravity_ParseDomainsIntoHosts "${localList}.tmp" "${localList}"
-
   # Add additional LAN hosts provided by OpenVPN (if available)
   if [[ -f "${VPNList}" ]]; then
     awk -F, '{printf $2"\t"$1".vpn\n"}' "${VPNList}" >> "${localList}"
@@ -730,23 +832,8 @@ gravity_Cleanup() {
 
   echo -e "${OVER}  ${TICK} ${str}"
 
-  if ${optimize_database} ; then
-    str="Optimizing domains database"
-    echo -ne "  ${INFO} ${str}..."
-    # Run VACUUM command on database to optimize it
-    output=$( { sqlite3 "${gravityDBfile}" "VACUUM;"; } 2>&1 )
-    status="$?"
-
-    if [[ "${status}" -ne 0 ]]; then
-      echo -e "\\n  ${CROSS} Unable to optimize gravity database ${gravityDBfile}\\n  ${output}"
-      error="error"
-    else
-      echo -e "${OVER}  ${TICK} ${str}"
-    fi
-  fi
-
   # Only restart DNS service if offline
-  if ! pidof ${resolver} &> /dev/null; then
+  if ! pgrep pihole-FTL &> /dev/null; then
     "${PIHOLE_COMMAND}" restartdns
     dnsWasOffline=true
   fi
@@ -771,12 +858,16 @@ Options:
 for var in "$@"; do
   case "${var}" in
     "-f" | "--force" ) forceDelete=true;;
-    "-o" | "--optimize" ) optimize_database=true;;
     "-r" | "--recreate" ) recreate_database=true;;
     "-h" | "--help" ) helpFunc;;
   esac
 done
 
+# Remove OLD (backup) gravity file, if it exists
+if [[ -f "${gravityOLDfile}" ]]; then
+  rm "${gravityOLDfile}"
+fi
+
 # Trap Ctrl-C
 gravity_Trap
 

manpages/pihole-FTL.8

@@ -1,4 +1,4 @@
-.TH "Pihole-FTL" "8" "pihole-FTL" "Pi-hole" "June 2018"
+.TH "Pihole-FTL" "8" "pihole-FTL" "Pi-hole" "November 2020"
 .SH "NAME"
 pihole-FTL - Pi-hole : The Faster-Than-Light (FTL) Engine
 .br
@@ -10,7 +10,7 @@ pihole-FTL - Pi-hole : The Faster-Than-Light (FTL) Engine
 .br
 \fBpihole-FTL test\fR
 .br
-\fBpihole-FTL -v\fR
+\fBpihole-FTL -v|-vv\fR
 .br
 \fBpihole-FTL -t\fR
 .br
@@ -22,6 +22,16 @@ pihole-FTL - Pi-hole : The Faster-Than-Light (FTL) Engine
 .br
 \fBpihole-FTL dnsmasq-test\fR
 .br
+\fBpihole-FTL regex-test str\fR
+.br
+\fBpihole-FTL regex-test str rgx\fR
+.br
+\fBpihole-FTL lua\fR
+.br
+\fBpihole-FTL luac\fR
+.br
+\fBpihole-FTL dhcp-discover\fR
+.br
 \fBpihole-FTL --\fR (\fBoptions\fR)
 .br
 
@@ -65,6 +75,11 @@ Command line arguments
     Don't start FTL, show only version
 .br
 
+\fB-vv\fR
+.br
+    Don't start FTL, show verbose version information of embedded applications
+.br
+
 \fB-t, tag\fR
 .br
     Don't start FTL, show only git tag
@@ -90,6 +105,31 @@ Command line arguments
     Test resolver config file syntax
 .br
 
+\fBregex-test str\fR
+.br
+    Test str against all regular expressions in the database
+.br
+
+\fBregex-test str rgx\fR
+.br
+    Test str against regular expression given by rgx
+.br
+
+\fBlua\fR
+.br
+    Start the embedded Lua interpreter
+.br
+
+\fBluac\fR
+.br
+    Execute the embedded Lua compiler
+.br
+
+\fBdhcp-discover\fR
+.br
+    Discover DHCP servers in the local network
+.br
+
 \fB--\fR  (options)
 .br
     Pass options to internal dnsmasq resolver

manpages/pihole-FTL.conf.5

@@ -1,4 +1,4 @@
-.TH "pihole-FTL.conf" "5" "pihole-FTL.conf" "pihole-FTL.conf" "June 2018"
+.TH "pihole-FTL.conf" "5" "pihole-FTL.conf" "pihole-FTL.conf" "November 2020"
 .SH "NAME"
 
 pihole-FTL.conf - FTL's config file
@@ -7,49 +7,32 @@ pihole-FTL.conf - FTL's config file
 
 /etc/pihole/pihole-FTL.conf will be read by \fBpihole-FTL(8)\fR on startup.
 .br
-
+For each setting the option shown first is the default.
-\fBSOCKET_LISTENING=localonly|all\fR
-.br
-    Listen only for local socket connections or permit all connections
 .br
 
-\fBQUERY_DISPLAY=yes|no\fR
+\fBBLOCKINGMODE=IP|IP-AAAA-NODATA|NODATA|NXDOMAIN|NULL\fR
 .br
-    Display all queries? Set to no to hide query display
+    How should FTL reply to blocked queries?
+
+    IP             - Pi-hole's IPs for blocked domains
+
+    IP-AAAA-NODATA - Pi-hole's IP + NODATA-IPv6 for blocked domains
+
+    NODATA         - Using NODATA for blocked domains
+
+    NXDOMAIN       - NXDOMAIN for blocked domains
+
+    NULL           - Null IPs for blocked domains
 .br
 
-\fBAAAA_QUERY_ANALYSIS=yes|no\fR
+\fBCNAME_DEEP_INSPECT=true|false\fR
 .br
-    Allow FTL to analyze AAAA queries from pihole.log?
+    Use this option to disable deep CNAME inspection. This might be beneficial for very low-end devices.
 .br
 
-\fBRESOLVE_IPV6=yes|no\fR
+\fBBLOCK_ESNI=true|false\fR
 .br
-    Should FTL try to resolve IPv6 addresses to host names?
+    Block requests to _esni.* sub-domains.
-.br
-
-\fBRESOLVE_IPV4=yes|no\fR
-.br
-    Should FTL try to resolve IPv4 addresses to host names?
-.br
-
-\fBMAXDBDAYS=365\fR
-.br
-    How long should queries be stored in the database?
-.br
-    Setting this to 0 disables the database
-.br
-
-\fBDBINTERVAL=1.0\fR
-.br
-    How often do we store queries in FTL's database [minutes]?
-.br
-
-\fBDBFILE=/etc/pihole/pihole-FTL.db\fR
-.br
-    Specify path and filename of FTL's SQLite long-term database.
-.br
-    Setting this to DBFILE= disables the database altogether
 .br
 
 \fBMAXLOGAGE=24.0\fR
@@ -59,14 +42,9 @@ pihole-FTL.conf - FTL's config file
     Maximum is 744 (31 days)
 .br
 
-\fBFTLPORT=4711\fR
-.br
-    On which port should FTL be listening?
-.br
-
 \fBPRIVACYLEVEL=0|1|2|3|4\fR
 .br
-    Which privacy level is used?
+    Privacy level used to collect Pi-hole statistics.
 .br
     0 - show everything
 .br
@@ -84,13 +62,244 @@ pihole-FTL.conf - FTL's config file
     Should FTL ignore queries coming from the local machine?
 .br
 
-\fBBLOCKINGMODE=IP|IP-AAAA-NODATA|NXDOMAIN|NULL\fR
+\fBAAAA_QUERY_ANALYSIS=yes|no\fR
 .br
-    How should FTL reply to blocked queries?
+    Should FTL analyze AAAA queries?
 .br
 
-For each setting, the option shown first is the default.
+\fBANALYZE_ONLY_A_AND_AAAA=false|true\fR
 .br
+    Should FTL only analyze A and AAAA queries?
+.br
+
+\fBSOCKET_LISTENING=localonly|all\fR
+.br
+    Listen only for local socket connections on the API port or permit all connections.
+.br
+
+\fBFTLPORT=4711\fR
+.br
+    On which port should FTL be listening?
+.br
+
+\fBRESOLVE_IPV6=yes|no\fR
+.br
+    Should FTL try to resolve IPv6 addresses to hostnames?
+.br
+
+\fBRESOLVE_IPV4=yes|no\fR
+.br
+    Should FTL try to resolve IPv4 addresses to hostnames?
+.br
+
+\fBDELAY_STARTUP=0\fR
+.br
+    Time in seconds (between 0 and 300) to delay FTL startup.
+.br
+
+\fBNICE=-10\fR
+.br
+    Set the niceness of the Pi-hole FTL process.
+.br
+    Can be disabled altogether by setting a value of -999.
+.br
+
+\fBNAMES_FROM_NETDB=true|false\fR
+.br
+    Control whether FTL should use a fallback option and try to obtain client names from checking the network table.
+.br
+    E.g. IPv6 clients without a hostname will be compared via MAC address to known clients.
+.br
+
+\fB\fBREFRESH_HOSTNAMES=IPV4|ALL|NONE\fR
+.br
+    Change how (and if) hourly PTR requests are made to check for changes in client and upstream server hostnames:
+.br
+    IPV4 - Do the hourly PTR lookups only for IPv4 addresses resolving issues in networks with many short-lived PE IPv6 addresses.
+.br
+    ALL  - Do the hourly PTR lookups for all addresses. This can create a lot of PTR queries in networks with many IPv6 addresses.
+.br
+    NONE - Don't do hourly PTR lookups. Look up hostnames once (when first seeing a client) and never again. Future hostname changes may be missed.
+.br
+
+\fBMAXNETAGE=365\fR
+.br
+    IP addresses (and associated host names) older than the specified number of days are removed.
+.br
+    This avoids dead entries in the network overview table.
+.br
+
+\fBEDNS0_ECS=true|false\fR
+.br
+    Should we overwrite the query source when client information is provided through EDNS0 client subnet (ECS) information?
+.br
+
+\fBPARSE_ARP_CACHE=true|false\fR
+.br
+    Parse ARP cache to fill network overview table.
+.br
+
+\fBDBIMPORT=yes|no\fR
+.br
+    Should FTL load information from the database on startup to be aware of the most recent history?
+.br
+
+\fBMAXDBDAYS=365\fR
+.br
+    How long should queries be stored in the database? Setting this to 0 disables the database
+.br
+
+\fBDBINTERVAL=1.0\fR
+.br
+    How often do we store queries in FTL's database [minutes]?
+.br
+    Accepts value between 0.1 (6 sec) and 1440 (1 day)
+.br
+
+\fBDBFILE=/etc/pihole/pihole-FTL.db\fR
+.br
+    Specify path and filename of FTL's SQLite long-term database.
+.br
+    Setting this to DBFILE= disables the database altogether
+.br
+
+\fBLOGFILE=/var/log/pihole-FTL.log\fR
+.br
+    The location of FTL's log file.
+.br
+
+\fBPIDFILE=/run/pihole-FTL.pid\fR
+.br
+    The file which contains the PID of FTL's main process.
+.br
+
+\fBPORTFILE=/run/pihole-FTL.port\fR
+.br
+    Specify path and filename where the FTL process will write its API port number.
+.br
+
+\fBSOCKETFILE=/run/pihole/FTL.sock\fR
+.br
+    The file containing the socket FTL's API is listening on.
+.br
+
+\fBSETUPVARSFILE=/etc/pihole/setupVars.conf\fR
+.br
+    The config file of Pi-hole containing, e.g., the current blocking status (do not change).
+.br
+
+\fBMACVENDORDB=/etc/pihole/macvendor.db\fR
+.br
+    The database containing MAC -> Vendor information for the network table.
+.br
+
+\fBGRAVITYDB=/etc/pihole/gravity.db\fR
+.br
+    Specify path and filename of FTL's SQLite3 gravity database. This database contains all domains relevant for Pi-hole's DNS blocking.
+.br
+
+\fBDEBUG_ALL=false|true\fR
+.br
+    Enable all debug flags. If this is set to true, all other debug config options are ignored.
+.br
+
+\fBDEBUG_DATABASE=false|true\fR
+.br
+    Print debugging information about database actions such as SQL statements and performance.
+.br
+
+\fBDEBUG_NETWORKING=false|true\fR
+.br
+    Prints a list of the detected network interfaces on the startup of FTL.
+.br
+
+\fBDEBUG_LOCKS=false|true\fR
+.br
+    Print information about shared memory locks.
+.br
+    Messages will be generated when waiting, obtaining, and releasing a lock.
+.br
+
+\fBDEBUG_QUERIES=false|true\fR
+.br
+    Print extensive DNS query information (domains, types, replies, etc.).
+.br
+
+\fBDEBUG_FLAGS=false|true\fR
+.br
+    Print flags of queries received by the DNS hooks.
+.br
+    Only effective when \fBDEBUG_QUERIES\fR is enabled as well.
+
+\fBDEBUG_SHMEM=false|true\fR
+.br
+    Print information about shared memory buffers.
+.br
+    Messages are either about creating or enlarging shmem objects or string injections.
+.br
+
+\fBDEBUG_GC=false|true\fR
+.br
+    Print information about garbage collection (GC):
+.br
+    What is to be removed, how many have been removed and how long did GC take.
+.br
+
+\fBDEBUG_ARP=false|true\fR
+.br
+    Print information about ARP table processing:
+.br
+    How long did parsing take, whether read MAC addresses are valid, and if the macvendor.db file exists.
+.br
+
+\fBDEBUG_REGEX=false|true\fR
+.br
+    Controls if FTL should print extended details about regex matching.
+.br
+
+\fBDEBUG_API=false|true\fR
+.br
+    Print extra debugging information during telnet API calls.
+.br
+    Currently only used to send extra information when getting all queries.
+.br
+
+\fBDEBUG_OVERTIME=false|true\fR
+.br
+    Print information about overTime memory operations, such as initializing or moving overTime slots.
+.br
+
+\fBDEBUG_EXTBLOCKED=false|true\fR
+.br
+    Print information about why FTL decided that certain queries were recognized as being externally blocked.
+.br
+
+\fBDEBUG_CAPS=false|true\fR
+.br
+    Print information about POSIX capabilities granted to the FTL process.
+.br
+    The current capabilities are printed on receipt of SIGHUP i.e. after executing `killall -HUP pihole-FTL`.
+.br
+
+\fBDEBUG_DNSMASQ_LINES=false|true\fR
+.br
+    Print file and line causing a dnsmasq event into FTL's log files.
+.br
+    This is handy to implement additional hooks missing from FTL.
+.br
+
+\fBDEBUG_VECTORS=false|true\fR
+.br
+    FTL uses dynamically allocated vectors for various tasks.
+.br
+    This config option enables extensive debugging information such as information about allocation, referencing, deletion, and appending.
+.br
+
+\fBDEBUG_RESOLVER=false|true\fR
+.br
+    Extensive information about hostname resolution like which DNS servers are used in the first and second hostname resolving tries.
+.br
+
 .SH "SEE ALSO"
 
 \fBpihole\fR(8), \fBpihole-FTL\fR(8)

manpages/pihole.8

@@ -56,7 +56,7 @@ Available commands and options:
 
 \fB-w, whitelist\fR [options] [<domain1> <domain2 ...>]
 .br
-    Adds or removes specified domain or domains tho the Whitelist
+    Adds or removes specified domain or domains to the Whitelist
 .br
 
 \fB-b, blacklist\fR [options] [<domain1> <domain2 ...>]
@@ -139,7 +139,7 @@ Available commands and options:
       -i, interface     Specify dnsmasq's interface listening behavior
 .br
       -l, privacylevel  <level> Set privacy level
-                        (0 = lowest, 4 = highest)
+                        (0 = lowest, 3 = highest)
 .br
 
 \fB-c, chronometer\fR	[options]
@@ -153,7 +153,7 @@ Available commands and options:
 .br
       -r, --refresh     Set update frequency (in seconds)
 .br
-      -e, --exit        Output stats and exit witout refreshing
+      -e, --exit        Output stats and exit without refreshing
 .br
 
 \fB-g, updateGravity\fR

pihole

@@ -16,12 +16,11 @@ readonly PI_HOLE_SCRIPT_DIR="/opt/pihole"
 # error due to modifying a readonly variable.
 setupVars="/etc/pihole/setupVars.conf"
 PI_HOLE_BIN_DIR="/usr/local/bin"
+readonly FTL_PID_FILE="/run/pihole-FTL.pid"
 
 readonly colfile="${PI_HOLE_SCRIPT_DIR}/COL_TABLE"
 source "${colfile}"
 
-resolver="pihole-FTL"
-
 webpageFunc() {
   source "${PI_HOLE_SCRIPT_DIR}/webpage.sh"
   main "$@"
@@ -100,27 +99,61 @@ versionFunc() {
   exit 0
 }
 
+# Get PID of main pihole-FTL process
+getFTLPID() {
+  local pid
+
+  if [ -s "${FTL_PID_FILE}" ]; then
+    # -s: FILE exists and has a size greater than zero
+    pid="$(<"$FTL_PID_FILE")"
+    # Exploit prevention: unset the variable if there is malicious content
+    # Verify that the value read from the file is numeric
+    [[ "$pid" =~ [^[:digit:]] ]] && unset pid
+  fi
+
+  # If FTL is not running, or the PID file contains malicious stuff, substitute
+  # negative PID to signal this to the caller
+  echo "${pid:=-1}"
+}
+
 restartDNS() {
-  local svcOption svc str output status
+  local svcOption svc str output status pid icon
   svcOption="${1:-restart}"
 
   # Determine if we should reload or restart
   if [[ "${svcOption}" =~ "reload-lists" ]]; then
     # Reloading of the lists has been requested
-    # Note: This will NOT re-read any *.conf files
+    # Note 1: This will NOT re-read any *.conf files
     # Note 2: We cannot use killall here as it does
     #         not know about real-time signals
-    svc="kill -SIGRTMIN $(pidof ${resolver})"
+    pid="$(getFTLPID)"
-    str="Reloading DNS lists"
+    if [[ "$pid" -eq "-1" ]]; then
+      svc="true"
+      str="FTL is not running"
+      icon="${INFO}"
+    else
+      svc="kill -RTMIN ${pid}"
+      str="Reloading DNS lists"
+      icon="${TICK}"
+    fi
   elif [[ "${svcOption}" =~ "reload" ]]; then
     # Reloading of the DNS cache has been requested
     # Note: This will NOT re-read any *.conf files
-    svc="killall -s SIGHUP ${resolver}"
+    pid="$(getFTLPID)"
-    str="Flushing DNS cache"
+    if [[ "$pid" -eq "-1" ]]; then
+      svc="true"
+      str="FTL is not running"
+      icon="${INFO}"
+    else
+      svc="kill -HUP ${pid}"
+      str="Flushing DNS cache"
+      icon="${TICK}"
+    fi
   else
     # A full restart has been requested
-    svc="service ${resolver} restart"
+    svc="service pihole-FTL restart"
     str="Restarting DNS server"
+    icon="${TICK}"
   fi
 
   # Print output to Terminal, but not to Web Admin
@@ -130,7 +163,7 @@ restartDNS() {
   status="$?"
 
   if [[ "${status}" -eq 0 ]]; then
-    [[ -t 1 ]] && echo -e "${OVER}  ${TICK} ${str}"
+    [[ -t 1 ]] && echo -e "${OVER}  ${icon} ${str}"
     return 0
   else
     [[ ! -t 1 ]] && local OVER=""
@@ -209,7 +242,7 @@ Time:
     echo "BLOCKING_ENABLED=true" >> "${setupVars}"
   fi
 
-  restartDNS reload
+  restartDNS reload-lists
 
   echo -e "${OVER}  ${TICK} ${str}"
 }
@@ -251,16 +284,47 @@ Options:
   echo -e "${OVER}  ${TICK} ${str}"
 }
 
+analyze_ports() {
+  # FTL is listening at least on at least one port when this
+  # function is getting called
+  echo -e "  ${TICK} DNS service is listening"
+  # Check individual address family/protocol combinations
+  # For a healthy Pi-hole, they should all be up (nothing printed)
+  if grep -q "IPv4.*UDP" <<< "${1}"; then
+      echo -e "     ${TICK} UDP (IPv4)"
+  else
+      echo -e "     ${CROSS} UDP (IPv4)"
+  fi
+  if grep -q "IPv4.*TCP" <<< "${1}"; then
+      echo -e "     ${TICK} TCP (IPv4)"
+  else
+      echo -e "     ${CROSS} TCP (IPv4)"
+  fi
+  if grep -q "IPv6.*UDP" <<< "${1}"; then
+      echo -e "     ${TICK} UDP (IPv6)"
+  else
+      echo -e "     ${CROSS} UDP (IPv6)"
+  fi
+  if grep -q "IPv6.*TCP" <<< "${1}"; then
+      echo -e "     ${TICK} TCP (IPv6)"
+  else
+      echo -e "     ${CROSS} TCP (IPv6)"
+  fi
+  echo ""
+}
+
 statusFunc() {
-  # Determine if service is running on port 53 (Cr: https://superuser.com/a/806331)
+  # Determine if there is a pihole service is listening on port 53
-  if (echo > /dev/tcp/127.0.0.1/53) >/dev/null 2>&1; then
+  local listening
+  listening="$(lsof -Pni:53)"
+  if grep -q "pihole" <<< "${listening}"; then
     if [[ "${1}" != "web" ]]; then
-      echo -e "  ${TICK} DNS service is running"
+      analyze_ports "${listening}"
     fi
   else
     case "${1}" in
       "web") echo "-1";;
-      *) echo -e "  ${CROSS} DNS service is NOT running";;
+      *) echo -e "  ${CROSS} DNS service is NOT listening";;
     esac
     return 0
   fi
@@ -270,13 +334,13 @@ statusFunc() {
     # A config is commented out
     case "${1}" in
       "web") echo 0;;
-      *) echo -e "  ${CROSS} Pi-hole blocking is Disabled";;
+      *) echo -e "  ${CROSS} Pi-hole blocking is disabled";;
     esac
   elif grep -q "BLOCKING_ENABLED=true" /etc/pihole/setupVars.conf;  then
     # Configs are set
     case "${1}" in
       "web") echo 1;;
-      *) echo -e "  ${TICK} Pi-hole blocking is Enabled";;
+      *) echo -e "  ${TICK} Pi-hole blocking is enabled";;
     esac
   else
     # No configs were found
@@ -299,16 +363,13 @@ tailFunc() {
   fi
   echo -e "  ${INFO} Press Ctrl-C to exit"
 
-  # Retrieve IPv4/6 addresses
-  source /etc/pihole/setupVars.conf
-
   # Strip date from each line
   # Color blocklist/blacklist/wildcard entries as red
   # Color A/AAAA/DHCP strings as white
   # Color everything else as gray
-  tail -f /var/log/pihole.log | sed -E \
+  tail -f /var/log/pihole.log | grep --line-buffered "${1}" | sed -E \
     -e "s,($(date +'%b %d ')| dnsmasq\[[0-9]*\]),,g" \
-    -e "s,(.*(blacklisted |gravity blocked ).* is (0.0.0.0|::|NXDOMAIN|${IPV4_ADDRESS%/*}|${IPV6_ADDRESS:-NULL}).*),${COL_RED}&${COL_NC}," \
+    -e "s,(.*(blacklisted |gravity blocked ).*),${COL_RED}&${COL_NC}," \
     -e "s,.*(query\\[A|DHCP).*,${COL_NC}&${COL_NC}," \
     -e "s,.*,${COL_GRAY}&${COL_NC},"
   exit 0
@@ -338,34 +399,24 @@ Branches:
 }
 
 tricorderFunc() {
+  local tricorder_token
   if [[ ! -p "/dev/stdin" ]]; then
     echo -e "  ${INFO} Please do not call Tricorder directly"
     exit 1
   fi
 
-  if ! (echo > /dev/tcp/tricorder.pi-hole.net/9998) >/dev/null 2>&1; then
+  tricorder_token=$(curl --silent --fail --show-error --upload-file "-" https://tricorder.pi-hole.net/upload < /dev/stdin 2>&1)
-    echo -e "  ${CROSS} Unable to connect to Pi-hole's Tricorder server"
+  if [[ "${tricorder_token}" != "https://tricorder.pi-hole.net/"* ]]; then
-    exit 1
+      echo -e "${CROSS} uploading failed, contact Pi-hole support for assistance."
-  fi
+      # Log curl error (if available)
-
+      if [ -n "${tricorder_token}" ]; then
-  if command -v openssl &> /dev/null; then
+          echo -e "${INFO} Error message: ${COL_RED}${tricorder_token}${COL_NC}\\n"
-    openssl s_client -quiet -connect tricorder.pi-hole.net:9998 2> /dev/null < /dev/stdin
+          tricorder_token=""
-    exit "$?"
+      fi
-  else
+      exit 1
-    echo -e "  ${INFO} ${COL_YELLOW}Security Notice${COL_NC}: ${COL_WHITE}openssl${COL_NC} is not installed
-       Your debug log will be transmitted unencrypted via plain-text
-       There is a possibility that this could be intercepted by a third party
-       If you wish to cancel, press Ctrl-C to exit within 10 seconds"
-    secs="10"
-    while [[ "$secs" -gt "0" ]]; do
-       echo -ne "."
-       sleep 1
-       : $((secs--))
-    done
-    echo " "
-    nc tricorder.pi-hole.net 9999 < /dev/stdin
-    exit "$?"
   fi
+  echo "Upload successful, your token is: ${COL_GREEN}${tricorder_token}${COL_NC}"
+  exit 0
 }
 
 updateCheckFunc() {
@@ -389,10 +440,13 @@ Whitelist/Blacklist Options:
 
 Debugging Options:
   -d, debug           Start a debugging session
-                        Add '-a' to enable automated debugging
+                        Add '-a' to automatically upload the log to tricorder.pi-hole.net
   -f, flush           Flush the Pi-hole log
   -r, reconfigure     Reconfigure or Repair Pi-hole subsystems
-  -t, tail            View the live output of the Pi-hole log
+  -t, tail [arg]      View the live output of the Pi-hole log.
+                      Add an optional argument to filter the log
+                      (regular expressions are supported)
+
 
 Options:
   -a, admin           Web interface options
@@ -466,7 +520,7 @@ case "${1}" in
   "status"                      ) statusFunc "$2";;
   "restartdns"                  ) restartDNS "$2";;
   "-a" | "admin"                ) webpageFunc "$@";;
-  "-t" | "tail"                 ) tailFunc;;
+  "-t" | "tail"                 ) tailFunc "$2";;
   "checkout"                    ) piholeCheckoutFunc "$@";;
   "tricorder"                   ) tricorderFunc;;
   "updatechecker"               ) updateCheckFunc "$@";;

supportedos.txt

@@ -1,5 +0,0 @@
-Raspbian=9,10
-Ubuntu=16,18,20
-Debian=9,10
-Fedora=31,32
-CentOS=7,8

test/_centos_7.Dockerfile

@@ -12,5 +12,6 @@ RUN true && \
     chmod +x $SCRIPTDIR/*
 
 ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
 
 #sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_centos_8.Dockerfile

@@ -1,4 +1,4 @@
-FROM buildpack-deps:jessie-scm
+FROM centos:8
 
 ENV GITDIR /etc/.pihole
 ENV SCRIPTDIR /opt/pihole
@@ -12,5 +12,6 @@ RUN true && \
     chmod +x $SCRIPTDIR/*
 
 ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
 
 #sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_debian_10.Dockerfile

@@ -0,0 +1,17 @@
+FROM buildpack-deps:buster-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_debian_11.Dockerfile

@@ -0,0 +1,17 @@
+FROM buildpack-deps:bullseye-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_debian_9.Dockerfile

@@ -0,0 +1,17 @@
+FROM buildpack-deps:stretch-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_fedora_32.Dockerfile

@@ -1,4 +1,4 @@
-FROM fedora:30
+FROM fedora:32
 
 ENV GITDIR /etc/.pihole
 ENV SCRIPTDIR /opt/pihole
@@ -12,5 +12,6 @@ RUN true && \
     chmod +x $SCRIPTDIR/*
 
 ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
 
 #sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_fedora_33.Dockerfile

@@ -0,0 +1,17 @@
+FROM fedora:33
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_ubuntu_16.Dockerfile

@@ -0,0 +1,17 @@
+FROM buildpack-deps:xenial-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_ubuntu_18.Dockerfile

@@ -0,0 +1,17 @@
+FROM buildpack-deps:bionic-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_ubuntu_20.Dockerfile

@@ -0,0 +1,18 @@
+FROM buildpack-deps:focal-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/_ubuntu_21.Dockerfile

@@ -0,0 +1,18 @@
+FROM buildpack-deps:hirsute-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+ENV OS_CHECK_DOMAIN_NAME dev-supportedos.pi-hole.net
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/conftest.py

@@ -8,8 +8,6 @@ check_output = testinfra.get_backend(
 
 SETUPVARS = {
     'PIHOLE_INTERFACE': 'eth99',
-    'IPV4_ADDRESS': '1.1.1.1',
-    'IPV6_ADDRESS': 'FE80::240:D0FF:FE48:4672',
     'PIHOLE_DNS_1': '4.2.2.1',
     'PIHOLE_DNS_2': '4.2.2.2'
 }
@@ -69,7 +67,9 @@ def args(request):
     return '-t -d'
 
 
-@pytest.fixture(params=['debian', 'centos', 'fedora'])
+@pytest.fixture(params=[
+    'test_container'
+])
 def tag(request):
     '''
     consumed by image to make the test matrix

test/test_000_build_containers.py

@@ -1,23 +0,0 @@
-''' This file starts with 000 to make it run first '''
-import pytest
-import testinfra
-
-run_local = testinfra.get_backend(
-    "local://"
-).get_module("Command").run
-
-
-@pytest.mark.parametrize("image,tag", [
-    ('test/debian.Dockerfile', 'pytest_pihole:debian'),
-    ('test/centos.Dockerfile', 'pytest_pihole:centos'),
-    ('test/fedora.Dockerfile', 'pytest_pihole:fedora'),
-])
-# mark as 'build_stage' so we can ensure images are build first when tests
-# are executed in parallel. (not required when tests are executed serially)
-@pytest.mark.build_stage
-def test_build_pihole_image(image, tag):
-    build_cmd = run_local('docker build -f {} -t {} .'.format(image, tag))
-    if build_cmd.rc != 0:
-        print(build_cmd.stdout)
-        print(build_cmd.stderr)
-    assert build_cmd.rc == 0

test/test_automated_install.py

@@ -11,20 +11,20 @@ from .conftest import (
 )
 
 
-def test_supported_operating_system(Pihole):
+def test_supported_package_manager(Pihole):
     '''
-    confirm installer exists on unsupported distribution
+    confirm installer exits when no supported package manager found
     '''
-    # break supported package managers to emulate an unsupported distribution
+    # break supported package managers
     Pihole.run('rm -rf /usr/bin/apt-get')
     Pihole.run('rm -rf /usr/bin/rpm')
-    distro_check = Pihole.run('''
+    package_manager_detect = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    distro_check
+    package_manager_detect
     ''')
-    expected_stdout = cross_box + ' OS distribution not supported'
+    expected_stdout = cross_box + ' No supported package manager found'
-    assert expected_stdout in distro_check.stdout
+    assert expected_stdout in package_manager_detect.stdout
-    # assert distro_check.rc == 1
+    # assert package_manager_detect.rc == 1
 
 
 def test_setupVars_are_sourced_to_global_scope(Pihole):
@@ -45,8 +45,6 @@ def test_setupVars_are_sourced_to_global_scope(Pihole):
         # Currently debug test function only
         echo "Outputting sourced variables"
         echo "PIHOLE_INTERFACE=${PIHOLE_INTERFACE}"
-        echo "IPV4_ADDRESS=${IPV4_ADDRESS}"
-        echo "IPV6_ADDRESS=${IPV6_ADDRESS}"
         echo "PIHOLE_DNS_1=${PIHOLE_DNS_1}"
         echo "PIHOLE_DNS_2=${PIHOLE_DNS_2}"
     }
@@ -114,9 +112,9 @@ def test_installPiholeWeb_fresh_install_no_errors(Pihole):
     source /opt/pihole/basic-install.sh
     installPiholeWeb
     ''')
-    expected_stdout = info_box + ' Installing blocking page...'
+    expected_stdout = info_box + ' Installing 404 page...'
     assert expected_stdout in installWeb.stdout
-    expected_stdout = tick_box + (' Creating directory for blocking page, '
+    expected_stdout = tick_box + (' Creating directory for 404 page, '
                                   'and copying files')
     assert expected_stdout in installWeb.stdout
     expected_stdout = info_box + ' Backing up index.lighttpd.html'
@@ -128,7 +126,6 @@ def test_installPiholeWeb_fresh_install_no_errors(Pihole):
     assert expected_stdout in installWeb.stdout
     web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
     assert 'index.php' in web_directory
-    assert 'blockingpage.css' in web_directory
 
 
 def test_update_package_cache_success_no_errors(Pihole):
@@ -137,7 +134,7 @@ def test_update_package_cache_success_no_errors(Pihole):
     '''
     updateCache = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    distro_check
+    package_manager_detect
     update_package_cache
     ''')
     expected_stdout = tick_box + ' Update local cache of available packages'
@@ -152,7 +149,7 @@ def test_update_package_cache_failure_no_errors(Pihole):
     mock_command('apt-get', {'update': ('', '1')}, Pihole)
     updateCache = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    distro_check
+    package_manager_detect
     update_package_cache
     ''')
     expected_stdout = cross_box + ' Update local cache of available packages'
@@ -187,7 +184,55 @@ def test_FTL_detect_aarch64_no_errors(Pihole):
     ''')
     expected_stdout = info_box + ' FTL Checks...'
     assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + ' Detected ARM-aarch64 architecture'
+    expected_stdout = tick_box + ' Detected AArch64 (64 Bit ARM) processor'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Downloading and Installing FTL'
+    assert expected_stdout in detectPlatform.stdout
+
+
+def test_FTL_detect_armv4t_no_errors(Pihole):
+    '''
+    confirms only armv4t package is downloaded for FTL engine
+    '''
+    # mock uname to return armv4t platform
+    mock_command('uname', {'-m': ('armv4t', '0')}, Pihole)
+    # mock ldd to respond with ld-linux shared library
+    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    create_pihole_user
+    funcOutput=$(get_binary_name)
+    binary="pihole-FTL${funcOutput##*pihole-FTL}"
+    theRest="${funcOutput%pihole-FTL*}"
+    FTLdetect "${binary}" "${theRest}"
+    ''')
+    expected_stdout = info_box + ' FTL Checks...'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + (' Detected ARMv4 processor')
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Downloading and Installing FTL'
+    assert expected_stdout in detectPlatform.stdout
+
+
+def test_FTL_detect_armv5te_no_errors(Pihole):
+    '''
+    confirms only armv5te package is downloaded for FTL engine
+    '''
+    # mock uname to return armv5te platform
+    mock_command('uname', {'-m': ('armv5te', '0')}, Pihole)
+    # mock ldd to respond with ld-linux shared library
+    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    create_pihole_user
+    funcOutput=$(get_binary_name)
+    binary="pihole-FTL${funcOutput##*pihole-FTL}"
+    theRest="${funcOutput%pihole-FTL*}"
+    FTLdetect "${binary}" "${theRest}"
+    ''')
+    expected_stdout = info_box + ' FTL Checks...'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + (' Detected ARMv5 (or newer) processor')
     assert expected_stdout in detectPlatform.stdout
     expected_stdout = tick_box + ' Downloading and Installing FTL'
     assert expected_stdout in detectPlatform.stdout
@@ -199,7 +244,7 @@ def test_FTL_detect_armv6l_no_errors(Pihole):
     '''
     # mock uname to return armv6l platform
     mock_command('uname', {'-m': ('armv6l', '0')}, Pihole)
-    # mock ldd to respond with aarch64 shared library
+    # mock ldd to respond with ld-linux-armhf shared library
     mock_command('ldd', {'/bin/ls': ('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
     detectPlatform = Pihole.run('''
     source /opt/pihole/basic-install.sh
@@ -211,8 +256,8 @@ def test_FTL_detect_armv6l_no_errors(Pihole):
     ''')
     expected_stdout = info_box + ' FTL Checks...'
     assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + (' Detected ARM-hf architecture '
+    expected_stdout = tick_box + (' Detected ARMv6 processor '
-                                  '(armv6 or lower)')
+                                  '(with hard-float support)')
     assert expected_stdout in detectPlatform.stdout
     expected_stdout = tick_box + ' Downloading and Installing FTL'
     assert expected_stdout in detectPlatform.stdout
@@ -224,7 +269,7 @@ def test_FTL_detect_armv7l_no_errors(Pihole):
     '''
     # mock uname to return armv7l platform
     mock_command('uname', {'-m': ('armv7l', '0')}, Pihole)
-    # mock ldd to respond with aarch64 shared library
+    # mock ldd to respond with ld-linux-armhf shared library
     mock_command('ldd', {'/bin/ls': ('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
     detectPlatform = Pihole.run('''
     source /opt/pihole/basic-install.sh
@@ -236,7 +281,32 @@ def test_FTL_detect_armv7l_no_errors(Pihole):
     ''')
     expected_stdout = info_box + ' FTL Checks...'
     assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + ' Detected ARM-hf architecture (armv7+)'
+    expected_stdout = tick_box + (' Detected ARMv7 processor '
+                                  '(with hard-float support)')
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Downloading and Installing FTL'
+    assert expected_stdout in detectPlatform.stdout
+
+
+def test_FTL_detect_armv8a_no_errors(Pihole):
+    '''
+    confirms only armv8a package is downloaded for FTL engine
+    '''
+    # mock uname to return armv8a platform
+    mock_command('uname', {'-m': ('armv8a', '0')}, Pihole)
+    # mock ldd to respond with ld-linux-armhf shared library
+    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    create_pihole_user
+    funcOutput=$(get_binary_name)
+    binary="pihole-FTL${funcOutput##*pihole-FTL}"
+    theRest="${funcOutput%pihole-FTL*}"
+    FTLdetect "${binary}" "${theRest}"
+    ''')
+    expected_stdout = info_box + ' FTL Checks...'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Detected ARMv8 (or newer) processor'
     assert expected_stdout in detectPlatform.stdout
     expected_stdout = tick_box + ' Downloading and Installing FTL'
     assert expected_stdout in detectPlatform.stdout
@@ -256,7 +326,7 @@ def test_FTL_detect_x86_64_no_errors(Pihole):
     ''')
     expected_stdout = info_box + ' FTL Checks...'
     assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + ' Detected x86_64 architecture'
+    expected_stdout = tick_box + ' Detected x86_64 processor'
     assert expected_stdout in detectPlatform.stdout
     expected_stdout = tick_box + ' Downloading and Installing FTL'
     assert expected_stdout in detectPlatform.stdout
@@ -274,7 +344,7 @@ def test_FTL_detect_unknown_no_errors(Pihole):
     theRest="${funcOutput%pihole-FTL*}"
     FTLdetect "${binary}" "${theRest}"
     ''')
-    expected_stdout = 'Not able to detect architecture (unknown: mips)'
+    expected_stdout = 'Not able to detect processor (unknown: mips)'
     assert expected_stdout in detectPlatform.stdout
 
 
@@ -286,7 +356,7 @@ def test_FTL_download_aarch64_no_errors(Pihole):
     mock_command('whiptail', {'*': ('', '0')}, Pihole)
     Pihole.run('''
     source /opt/pihole/basic-install.sh
-    distro_check
+    package_manager_detect
     install_dependent_packages ${INSTALLER_DEPS[@]}
     ''')
     download_binary = Pihole.run('''
@@ -350,10 +420,9 @@ def test_IPv6_only_link_local(Pihole):
     )
     detectPlatform = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    useIPv6dialog
+    find_IPv6_information
     ''')
-    expected_stdout = ('Unable to find IPv6 ULA/GUA address, '
+    expected_stdout = ('Unable to find IPv6 ULA/GUA address')
-                       'IPv6 adblocking will not be enabled')
     assert expected_stdout in detectPlatform.stdout
 
 
@@ -374,9 +443,9 @@ def test_IPv6_only_ULA(Pihole):
     )
     detectPlatform = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    useIPv6dialog
+    find_IPv6_information
     ''')
-    expected_stdout = 'Found IPv6 ULA address, using it for blocking IPv6 ads'
+    expected_stdout = 'Found IPv6 ULA address'
     assert expected_stdout in detectPlatform.stdout
 
 
@@ -397,9 +466,9 @@ def test_IPv6_only_GUA(Pihole):
     )
     detectPlatform = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    useIPv6dialog
+    find_IPv6_information
     ''')
-    expected_stdout = 'Found IPv6 GUA address, using it for blocking IPv6 ads'
+    expected_stdout = 'Found IPv6 GUA address'
     assert expected_stdout in detectPlatform.stdout
 
 
@@ -421,9 +490,9 @@ def test_IPv6_GUA_ULA_test(Pihole):
     )
     detectPlatform = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    useIPv6dialog
+    find_IPv6_information
     ''')
-    expected_stdout = 'Found IPv6 ULA address, using it for blocking IPv6 ads'
+    expected_stdout = 'Found IPv6 ULA address'
     assert expected_stdout in detectPlatform.stdout
 
 
@@ -445,46 +514,125 @@ def test_IPv6_ULA_GUA_test(Pihole):
     )
     detectPlatform = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    useIPv6dialog
+    find_IPv6_information
     ''')
-    expected_stdout = 'Found IPv6 ULA address, using it for blocking IPv6 ads'
+    expected_stdout = 'Found IPv6 ULA address'
     assert expected_stdout in detectPlatform.stdout
 
 
-def test_validate_ip_valid(Pihole):
+def test_validate_ip(Pihole):
     '''
-    Given a valid IP address, valid_ip returns success
+    Tests valid_ip for various IP addresses
     '''
 
+    def test_address(addr, success=True):
+        output = Pihole.run('''
+        source /opt/pihole/basic-install.sh
+        valid_ip "{addr}"
+        '''.format(addr=addr))
+
+        assert output.rc == 0 if success else 1
+
+    test_address('192.168.1.1')
+    test_address('127.0.0.1')
+    test_address('255.255.255.255')
+    test_address('255.255.255.256', False)
+    test_address('255.255.256.255', False)
+    test_address('255.256.255.255', False)
+    test_address('256.255.255.255', False)
+    test_address('1092.168.1.1', False)
+    test_address('not an IP', False)
+    test_address('8.8.8.8#', False)
+    test_address('8.8.8.8#0')
+    test_address('8.8.8.8#1')
+    test_address('8.8.8.8#42')
+    test_address('8.8.8.8#888')
+    test_address('8.8.8.8#1337')
+    test_address('8.8.8.8#65535')
+    test_address('8.8.8.8#65536', False)
+    test_address('8.8.8.8#-1', False)
+    test_address('00.0.0.0', False)
+    test_address('010.0.0.0', False)
+    test_address('001.0.0.0', False)
+    test_address('0.0.0.0#00', False)
+    test_address('0.0.0.0#01', False)
+    test_address('0.0.0.0#001', False)
+    test_address('0.0.0.0#0001', False)
+    test_address('0.0.0.0#00001', False)
+
+
+def test_os_check_fails(Pihole):
+    ''' Confirms install fails on unsupported OS '''
+    Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    install_dependent_packages ${OS_CHECK_DEPS[@]}
+    install_dependent_packages ${INSTALLER_DEPS[@]}
+    cat <<EOT > /etc/os-release
+    ID=UnsupportedOS
+    VERSION_ID="2"
+    EOT
+    ''')
+    detectOS = Pihole.run('''t
+    source /opt/pihole/basic-install.sh
+    os_check
+    ''')
+    expected_stdout = 'Unsupported OS detected: UnsupportedOS'
+    assert expected_stdout in detectOS.stdout
+
+
+def test_os_check_passes(Pihole):
+    ''' Confirms OS meets the requirements '''
+    Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    install_dependent_packages ${OS_CHECK_DEPS[@]}
+    install_dependent_packages ${INSTALLER_DEPS[@]}
+    ''')
+    detectOS = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    os_check
+    ''')
+    expected_stdout = 'Supported OS detected'
+    assert expected_stdout in detectOS.stdout
+
+
+def test_package_manager_has_installer_deps(Pihole):
+    ''' Confirms OS is able to install the required packages for the installer'''
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
     output = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    valid_ip "192.168.1.1"
+    package_manager_detect
+    install_dependent_packages ${INSTALLER_DEPS[@]}
     ''')
 
+    assert 'No package' not in output.stdout  # centos7 still exits 0...
     assert output.rc == 0
 
 
-def test_validate_ip_invalid_octet(Pihole):
+def test_package_manager_has_pihole_deps(Pihole):
-    '''
+    ''' Confirms OS is able to install the required packages for Pi-hole '''
-    Given an invalid IP address (large octet), valid_ip returns an error
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    '''
-
     output = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    valid_ip "1092.168.1.1"
+    package_manager_detect
+    select_rpm_php
+    install_dependent_packages ${PIHOLE_DEPS[@]}
     ''')
 
-    assert output.rc == 1
+    assert 'No package' not in output.stdout  # centos7 still exits 0...
+    assert output.rc == 0
 
 
-def test_validate_ip_invalid_letters(Pihole):
+def test_package_manager_has_web_deps(Pihole):
-    '''
+    ''' Confirms OS is able to install the required packages for web '''
-    Given an invalid IP address (contains letters), valid_ip returns an error
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    '''
-
     output = Pihole.run('''
     source /opt/pihole/basic-install.sh
-    valid_ip "not an IP"
+    package_manager_detect
+    select_rpm_php
+    install_dependent_packages ${PIHOLE_WEB_DEPS[@]}
     ''')
 
-    assert output.rc == 1
+    assert 'No package' not in output.stdout  # centos7 still exits 0...
+    assert output.rc == 0

test/test_centos_7_support.py

@@ -0,0 +1,63 @@
+from .conftest import (
+    tick_box,
+    info_box,
+    mock_command,
+)
+
+
+def test_php_upgrade_default_optout_centos_eq_7(Pihole):
+    '''
+    confirms the default behavior to opt-out of installing PHP7 from REMI
+    '''
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in package_manager_detect.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optout_centos_eq_7(Pihole):
+    '''
+    confirms installer behavior when user opt-out of installing PHP7 from REMI
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*': ('', '1')}, Pihole)
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in package_manager_detect.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optin_centos_eq_7(Pihole):
+    '''
+    confirms installer behavior when user opt-in to installing PHP7 from REMI
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Continue for user prompt
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    assert 'opt-out' not in package_manager_detect.stdout
+    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
+                                  '(https://rpms.remirepo.net)')
+    assert expected_stdout in package_manager_detect.stdout
+    expected_stdout = tick_box + (' Remi\'s RPM repository has '
+                                  'been enabled for PHP7')
+    assert expected_stdout in package_manager_detect.stdout
+    remi_package = Pihole.package('remi-release')
+    assert remi_package.is_installed

test/test_centos_8_support.py

@@ -0,0 +1,68 @@
+from .conftest import (
+    tick_box,
+    info_box,
+    mock_command,
+)
+
+
+def test_php_upgrade_default_continue_centos_gte_8(Pihole):
+    '''
+    confirms the latest version of CentOS continues / does not optout
+    (should trigger on CentOS7 only)
+    '''
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    unexpected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS.'
+                                    ' Deprecated PHP may be in use.')
+    assert unexpected_stdout not in package_manager_detect.stdout
+    # ensure remi was not installed on latest CentOS
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optout_skipped_centos_gte_8(Pihole):
+    '''
+    confirms installer skips user opt-out of installing PHP7 from REMI on
+    latest CentOS (should trigger on CentOS7 only)
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*': ('', '1')}, Pihole)
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    unexpected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS.'
+                                    ' Deprecated PHP may be in use.')
+    assert unexpected_stdout not in package_manager_detect.stdout
+    # ensure remi was not installed on latest CentOS
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optin_skipped_centos_gte_8(Pihole):
+    '''
+    confirms installer skips user opt-in to installing PHP7 from REMI on
+    latest CentOS (should trigger on CentOS7 only)
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Continue for user prompt
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    assert 'opt-out' not in package_manager_detect.stdout
+    unexpected_stdout = info_box + (' Enabling Remi\'s RPM repository '
+                                    '(https://rpms.remirepo.net)')
+    assert unexpected_stdout not in package_manager_detect.stdout
+    unexpected_stdout = tick_box + (' Remi\'s RPM repository has '
+                                    'been enabled for PHP7')
+    assert unexpected_stdout not in package_manager_detect.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed

test/test_centos_common_support.py

@@ -0,0 +1,125 @@
+import pytest
+from .conftest import (
+    tick_box,
+    info_box,
+    cross_box,
+    mock_command,
+)
+
+
+def test_release_supported_version_check_centos(Pihole):
+    '''
+    confirms installer exits on unsupported releases of CentOS
+    '''
+    # modify /etc/redhat-release to mock an unsupported CentOS release
+    Pihole.run('echo "CentOS Linux release 6.9" > /etc/redhat-release')
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    expected_stdout = cross_box + (' CentOS 6 is not supported.')
+    assert expected_stdout in package_manager_detect.stdout
+    expected_stdout = 'Please update to CentOS release 7 or later'
+    assert expected_stdout in package_manager_detect.stdout
+
+
+def test_enable_epel_repository_centos(Pihole):
+    '''
+    confirms the EPEL package repository is enabled when installed on CentOS
+    '''
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    expected_stdout = info_box + (' Enabling EPEL package repository '
+                                  '(https://fedoraproject.org/wiki/EPEL)')
+    assert expected_stdout in package_manager_detect.stdout
+    expected_stdout = tick_box + ' Installed epel-release'
+    assert expected_stdout in package_manager_detect.stdout
+    epel_package = Pihole.package('epel-release')
+    assert epel_package.is_installed
+
+
+def test_php_version_lt_7_detected_upgrade_default_optout_centos(Pihole):
+    '''
+    confirms the default behavior to opt-out of upgrading to PHP7 from REMI
+    '''
+    # first we will install the default php version to test installer behavior
+    php_install = Pihole.run('yum install -y php')
+    assert php_install.rc == 0
+    php_package = Pihole.package('php')
+    default_centos_php_version = php_package.version.split('.')[0]
+    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
+        pytest.skip("Test deprecated . Detected default PHP version >= 7")
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in package_manager_detect.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_version_lt_7_detected_upgrade_user_optout_centos(Pihole):
+    '''
+    confirms installer behavior when user opt-out to upgrade to PHP7 via REMI
+    '''
+    # first we will install the default php version to test installer behavior
+    php_install = Pihole.run('yum install -y php')
+    assert php_install.rc == 0
+    php_package = Pihole.package('php')
+    default_centos_php_version = php_package.version.split('.')[0]
+    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
+        pytest.skip("Test deprecated . Detected default PHP version >= 7")
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*': ('', '1')}, Pihole)
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in package_manager_detect.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_version_lt_7_detected_upgrade_user_optin_centos(Pihole):
+    '''
+    confirms installer behavior when user opt-in to upgrade to PHP7 via REMI
+    '''
+    # first we will install the default php version to test installer behavior
+    php_install = Pihole.run('yum install -y php')
+    assert php_install.rc == 0
+    php_package = Pihole.package('php')
+    default_centos_php_version = php_package.version.split('.')[0]
+    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
+        pytest.skip("Test deprecated . Detected default PHP version >= 7")
+    # Whiptail dialog returns Continue for user prompt
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    install_dependent_packages PIHOLE_WEB_DEPS[@]
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout not in package_manager_detect.stdout
+    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
+                                  '(https://rpms.remirepo.net)')
+    assert expected_stdout in package_manager_detect.stdout
+    expected_stdout = tick_box + (' Remi\'s RPM repository has '
+                                  'been enabled for PHP7')
+    assert expected_stdout in package_manager_detect.stdout
+    remi_package = Pihole.package('remi-release')
+    assert remi_package.is_installed
+    updated_php_package = Pihole.package('php')
+    updated_php_version = updated_php_package.version.split('.')[0]
+    assert int(updated_php_version) == 7

test/test_centos_fedora_common_support.py

@@ -0,0 +1,65 @@
+from .conftest import (
+    tick_box,
+    cross_box,
+    mock_command,
+)
+
+
+def mock_selinux_config(state, Pihole):
+    '''
+    Creates a mock SELinux config file with expected content
+    '''
+    # validate state string
+    valid_states = ['enforcing', 'permissive', 'disabled']
+    assert state in valid_states
+    # getenforce returns the running state of SELinux
+    mock_command('getenforce', {'*': (state.capitalize(), '0')}, Pihole)
+    # create mock configuration with desired content
+    Pihole.run('''
+    mkdir /etc/selinux
+    echo "SELINUX={state}" > /etc/selinux/config
+    '''.format(state=state.lower()))
+
+
+def test_selinux_enforcing_exit(Pihole):
+    '''
+    confirms installer prompts to exit when SELinux is Enforcing by default
+    '''
+    mock_selinux_config("enforcing", Pihole)
+    check_selinux = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    checkSelinux
+    ''')
+    expected_stdout = cross_box + ' Current SELinux: Enforcing'
+    assert expected_stdout in check_selinux.stdout
+    expected_stdout = 'SELinux Enforcing detected, exiting installer'
+    assert expected_stdout in check_selinux.stdout
+    assert check_selinux.rc == 1
+
+
+def test_selinux_permissive(Pihole):
+    '''
+    confirms installer continues when SELinux is Permissive
+    '''
+    mock_selinux_config("permissive", Pihole)
+    check_selinux = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    checkSelinux
+    ''')
+    expected_stdout = tick_box + ' Current SELinux: Permissive'
+    assert expected_stdout in check_selinux.stdout
+    assert check_selinux.rc == 0
+
+
+def test_selinux_disabled(Pihole):
+    '''
+    confirms installer continues when SELinux is Disabled
+    '''
+    mock_selinux_config("disabled", Pihole)
+    check_selinux = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    checkSelinux
+    ''')
+    expected_stdout = tick_box + ' Current SELinux: Disabled'
+    assert expected_stdout in check_selinux.stdout
+    assert check_selinux.rc == 0

test/test_centos_fedora_support.py

@@ -1,264 +0,0 @@
-import pytest
-from .conftest import (
-    tick_box,
-    info_box,
-    cross_box,
-    mock_command,
-)
-
-
-def mock_selinux_config(state, Pihole):
-    '''
-    Creates a mock SELinux config file with expected content
-    '''
-    # validate state string
-    valid_states = ['enforcing', 'permissive', 'disabled']
-    assert state in valid_states
-    # getenforce returns the running state of SELinux
-    mock_command('getenforce', {'*': (state.capitalize(), '0')}, Pihole)
-    # create mock configuration with desired content
-    Pihole.run('''
-    mkdir /etc/selinux
-    echo "SELINUX={state}" > /etc/selinux/config
-    '''.format(state=state.lower()))
-
-
-@pytest.mark.parametrize("tag", [('centos'), ('fedora'), ])
-def test_selinux_enforcing_exit(Pihole):
-    '''
-    confirms installer prompts to exit when SELinux is Enforcing by default
-    '''
-    mock_selinux_config("enforcing", Pihole)
-    check_selinux = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    checkSelinux
-    ''')
-    expected_stdout = cross_box + ' Current SELinux: Enforcing'
-    assert expected_stdout in check_selinux.stdout
-    expected_stdout = 'SELinux Enforcing detected, exiting installer'
-    assert expected_stdout in check_selinux.stdout
-    assert check_selinux.rc == 1
-
-
-@pytest.mark.parametrize("tag", [('centos'), ('fedora'), ])
-def test_selinux_permissive(Pihole):
-    '''
-    confirms installer continues when SELinux is Permissive
-    '''
-    mock_selinux_config("permissive", Pihole)
-    check_selinux = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    checkSelinux
-    ''')
-    expected_stdout = tick_box + ' Current SELinux: Permissive'
-    assert expected_stdout in check_selinux.stdout
-    assert check_selinux.rc == 0
-
-
-@pytest.mark.parametrize("tag", [('centos'), ('fedora'), ])
-def test_selinux_disabled(Pihole):
-    '''
-    confirms installer continues when SELinux is Disabled
-    '''
-    mock_selinux_config("disabled", Pihole)
-    check_selinux = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    checkSelinux
-    ''')
-    expected_stdout = tick_box + ' Current SELinux: Disabled'
-    assert expected_stdout in check_selinux.stdout
-    assert check_selinux.rc == 0
-
-
-@pytest.mark.parametrize("tag", [('fedora'), ])
-def test_epel_and_remi_not_installed_fedora(Pihole):
-    '''
-    confirms installer does not attempt to install EPEL/REMI repositories
-    on Fedora
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    assert distro_check.stdout == ''
-
-    epel_package = Pihole.package('epel-release')
-    assert not epel_package.is_installed
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_release_supported_version_check_centos(Pihole):
-    '''
-    confirms installer exits on unsupported releases of CentOS
-    '''
-    # modify /etc/redhat-release to mock an unsupported CentOS release
-    Pihole.run('echo "CentOS Linux release 6.9" > /etc/redhat-release')
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = cross_box + (' CentOS 6 is not supported.')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = 'Please update to CentOS release 7 or later'
-    assert expected_stdout in distro_check.stdout
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_enable_epel_repository_centos(Pihole):
-    '''
-    confirms the EPEL package repository is enabled when installed on CentOS
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' Enabling EPEL package repository '
-                                  '(https://fedoraproject.org/wiki/EPEL)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + ' Installed epel-release'
-    assert expected_stdout in distro_check.stdout
-    epel_package = Pihole.package('epel-release')
-    assert epel_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_upgrade_default_optout_centos(Pihole):
-    '''
-    confirms the default behavior to opt-out of installing PHP7 from REMI
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_upgrade_user_optout_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-out of installing PHP7 from REMI
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_upgrade_user_optin_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-in to installing PHP7 from REMI
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Continue for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    assert 'opt-out' not in distro_check.stdout
-    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
-                                  '(https://rpms.remirepo.net)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + (' Remi\'s RPM repository has '
-                                  'been enabled for PHP7')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_version_lt_7_detected_upgrade_default_optout_centos(Pihole):
-    '''
-    confirms the default behavior to opt-out of upgrading to PHP7 from REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_version_lt_7_detected_upgrade_user_optout_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-out to upgrade to PHP7 via REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_version_lt_7_detected_upgrade_user_optin_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-in to upgrade to PHP7 via REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    # Whiptail dialog returns Continue for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    install_dependent_packages PIHOLE_WEB_DEPS[@]
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout not in distro_check.stdout
-    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
-                                  '(https://rpms.remirepo.net)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + (' Remi\'s RPM repository has '
-                                  'been enabled for PHP7')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert remi_package.is_installed
-    updated_php_package = Pihole.package('php')
-    updated_php_version = updated_php_package.version.split('.')[0]
-    assert int(updated_php_version) == 7

test/test_fedora_support.py

@@ -0,0 +1,16 @@
+def test_epel_and_remi_not_installed_fedora(Pihole):
+    '''
+    confirms installer does not attempt to install EPEL/REMI repositories
+    on Fedora
+    '''
+    package_manager_detect = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    package_manager_detect
+    select_rpm_php
+    ''')
+    assert package_manager_detect.stdout == ''
+
+    epel_package = Pihole.package('epel-release')
+    assert not epel_package.is_installed
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed

test/test_shellcheck.py

@@ -1,18 +0,0 @@
-import testinfra
-
-run_local = testinfra.get_backend(
-    "local://"
-).get_module("Command").run
-
-
-def test_scripts_pass_shellcheck():
-    '''
-    Make sure shellcheck does not find anything wrong with our shell scripts
-    '''
-    shellcheck = ("find . -type f -name 'update.sh' "
-                  "| while read file; do "
-                  "shellcheck -x \"$file\" -e SC1090,SC1091; "
-                  "done;")
-    results = run_local(shellcheck)
-    print(results.stdout)
-    assert '' == results.stdout

test/tox.centos_7.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _centos_7.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_common_support.py ./test_centos_7_support.py

test/tox.centos_8.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _centos_8.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_common_support.py ./test_centos_8_support.py

test/tox.debian_10.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _debian_10.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py

test/tox.debian_11.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _debian_11.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py

test/tox.debian_9.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _debian_9.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py

test/tox.fedora_32.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _fedora_32.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py

test/tox.fedora_33.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _fedora_33.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py

test/tox.ubuntu_16.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _ubuntu_16.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py

test/tox.ubuntu_18.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _ubuntu_18.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py

test/tox.ubuntu_20.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _ubuntu_20.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py

test/tox.ubuntu_21.ini

@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _ubuntu_21.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py

tox.ini

@@ -1,10 +0,0 @@
-[tox]
-envlist = py36
-
-[testenv]
-whitelist_externals = docker
-deps = -rrequirements.txt
-commands = docker build -f test/debian.Dockerfile -t pytest_pihole:debian .
-           docker build -f test/centos.Dockerfile -t pytest_pihole:centos .
-           docker build -f test/fedora.Dockerfile -t pytest_pihole:fedora .
-           pytest {posargs:-vv -n auto} -m "not build_stage" ./test/