Initial work on stubbing for lua.

Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
Merge pull request #4175 from ashuio/development
2021-06-21 10:49:46 -07:00 · 2021-05-20 18:49:01 +01:00 · 2021-05-20 22:34:19 +05:30 · 2021-05-01 00:09:19 +01:00 · 2021-04-30 20:48:12 +02:00 · 2021-04-25 17:47:40 +01:00
62 changed files with 2141 additions and 936 deletions
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -0,0 +1,25 @@
+name: Test Supported Distributions
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+
+jobs:
+  distro-test:
+    if: github.event.pull_request.draft == false
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        distro: [debian_9, debian_10, ubuntu_16, ubuntu_18, ubuntu_20, centos_7, centos_8, fedora_32, fedora_33]
+    env:
+      DISTRO: ${{matrix.distro}}
+    steps:
+    - uses: actions/checkout@v1
+    - name: Set up Python 3.7
+      uses: actions/setup-python@v2
+      with:
+        python-version: 3.7
+    - name: Install dependencies
+      run: pip install -r test/requirements.txt
+    - name: Test with tox
+      run: tox -c test/tox.${DISTRO}.ini
--- a/.stickler.yml
+++ b/.stickler.yml
@@ -3,3 +3,4 @@ linters:
    shell: bash
  phpcs:
  flake8:
+    max-line-length: 120
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +0,0 @@
-import:
-  - source: pi-hole/.github:/build-configs/core.yml@main
-    if: branch = master
-  - source: pi-hole/.github:/build-configs/core.yml@latest
-    if: branch != master
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -4,8 +4,8 @@ Please read and understand the contribution guide before creating an issue or pu

 ## Etiquette

- Our goal for Pi-hole is **stability before features**.  This means we focus on squashing critical bugs before adding new features.  Often, we can do both in tandem, but bugs will take priority over a new feature.
- Pi-hole is open source and [powered by donations](https://pi-hole.net/donate/), and as such, we give our **free time** to build, maintain, and **provide user support** for this project.  It would be extremely unfair for us to suffer abuse or anger for our hard work, so please take a moment to consider that.
+- Our goal for Pi-hole is **stability before features**. This means we focus on squashing critical bugs before adding new features. Often, we can do both in tandem, but bugs will take priority over a new feature.
+- Pi-hole is open source and [powered by donations](https://pi-hole.net/donate/), and as such, we give our **free time** to build, maintain, and **provide user support** for this project. It would be extremely unfair for us to suffer abuse or anger for our hard work, so please take a moment to consider that.
 - Please be considerate towards the developers and other users when raising issues or presenting pull requests.
 - Respect our decision(s), and do not be upset or abusive if your submission is not used.

@@ -26,7 +26,7 @@ When requesting or submitting new features, first consider whether it might be u

 - Check the codebase to ensure that your feature doesn't already exist.
 - Check the pull requests to ensure that another person hasn't already submitted the feature or fix.
- Read and understand the [DCO guidelines](https://github.com/pi-hole/pi-hole/wiki/Contributing-to-the-project) for the project.
+- Read and understand the [DCO guidelines](https://docs.pi-hole.net/guides/github/contributing/) for the project.

 ## Technical Requirements

@@ -36,3 +36,77 @@ When requesting or submitting new features, first consider whether it might be u
 - Commit Unix line endings.
 - Please use the Pi-hole brand: **Pi-hole** (Take a special look at the capitalized 'P' and a low 'h' with a hyphen)
 - (Optional fun) keep to the theme of Star Trek/black holes/gravity.
+
+## Forking and Cloning from GitHub to GitHub
+
+1. Fork <https://github.com/pi-hole/pi-hole/> to a repo under a namespace you control, or have permission to use, for example: `https://github.com/<your_namespace>/<your_repo_name>/`. You can do this from the github.com website.
+2. Clone `https://github.com/<your_namespace>/<your_repo_name>/` with the tool of you choice.
+3. To keep your fork in sync with our repo, add an upstream remote for pi-hole/pi-hole to your repo.
+
+    ```bash
+    git remote add upstream https://github.com/pi-hole/pi-hole.git
+    ```
+
+4. Checkout the `development` branch from your fork `https://github.com/<your_namespace>/<your_repo_name>/`.
+5. Create a topic/branch, based on the `development` branch code. *Bonus fun to keep to the theme of Star Trek/black holes/gravity.*
+6. Make your changes and commit to your topic branch in your repo.
+7. Rebase your commits and squash any insignificant commits. See the notes below for an example.
+8. Merge `development` your branch and fix any conflicts.
+9. Open a Pull Request to merge your topic branch into our repo's `development` branch.
+
+- Keep in mind the technical requirements from above.
+
+## Forking and Cloning from GitHub to other code hosting sites
+
+- Forking is a GitHub concept and cannot be done from GitHub to other git-based code hosting sites. However, those sites may be able to mirror a GitHub repo.
+
+1. To contribute from another code hosting site, you must first complete the steps above to fork our repo to a GitHub namespace you have permission to use, for example: `https://github.com/<your_namespace>/<your_repo_name>/`.
+2. Create a repo in your code hosting site, for example: `https://gitlab.com/<your_namespace>/<your_repo_name>/`
+3. Follow the instructions from your code hosting site to create a mirror between `https://github.com/<your_namespace>/<your_repo_name>/` and `https://gitlab.com/<your_namespace>/<your_repo_name>/`.
+4. When you are ready to create a Pull Request (PR), follow the steps `(starting at step #6)` from [Forking and Cloning from GitHub to GitHub](#forking-and-cloning-from-github-to-github) and create the PR from `https://github.com/<your_namespace>/<your_repo_name>/`.
+
+## Notes for squashing commits with rebase
+
+- To rebase your commits and squash previous commits, you can use:
+
+    ```bash
+    git rebase -i your_topic_branch~(number of commits to combine)
+    ```
+
+- For more details visit [gitready.com](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html)
+
+1. The following would combine the last four commits in the branch `mytopic`.
+
+    ```bash
+    git rebase -i mytopic~4
+    ```
+
+2. An editor window opens with the most recent commits indicated: (edit the commands to the left of the commit ID)
+
+    ```gitattributes
+    pick 9dff55b2 existing commit comments
+    squash ebb1a730 existing commit comments
+    squash 07cc5b50 existing commit comments
+    reword 9dff55b2 existing commit comments
+    ```
+
+3. Save and close the editor. The next editor window opens: (edit the new commit message). *If you select reword for a commit, an additional editor window will open for you to edit the comment.*
+
+    ```bash
+    new commit comments
+    Signed-off-by: yourname <your email address>
+    ```
+
+4. Save and close the editor for the rebase process to execute. The terminal output should say something like the following:
+
+    ```bash
+    Successfully rebased and updated refs/heads/mytopic.
+    ```
+
+5. Once you have a successful rebase, and before you sync your local clone, you have to force push origin to update your repo:
+
+    ```bash
+    git push -f origin
+    ```
+
+6. Continue on from step #7 from [Forking and Cloning from GitHub to GitHub](#forking-and-cloning-from-github-to-github)
--- a/README.md
+++ b/README.md
@@ -1,5 +1,7 @@
 <!-- markdownlint-configure-file { "MD004": { "style": "consistent" } } -->
 <!-- markdownlint-disable MD033 -->
+#
+
 <p align="center">
    <a href="https://pi-hole.net/">
        <img src="https://pi-hole.github.io/graphics/Vortex/Vortex_with_Wordmark.svg" width="150" height="260" alt="Pi-hole">
@@ -9,11 +11,9 @@
 </p>
 <!-- markdownlint-enable MD033 -->

-#
-
 The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) that protects your devices from unwanted content, without installing any client-side software.

- **Easy-to-install**: our versatile installer walks you through the process, and [takes less than ten minutes](https://www.youtube.com/watch?v=vKWjx1AQYgs)
+- **Easy-to-install**: our versatile installer walks you through the process, and takes less than ten minutes
 - **Resolute**: content is blocked in _non-browser locations_, such as ad-laden mobile apps and smart TVs
 - **Responsive**: seamlessly speeds up the feel of everyday browsing by caching DNS queries
 - **Lightweight**: runs smoothly with [minimal hardware and software requirements](https://docs.pi-hole.net/main/prerequisites/)
@@ -26,8 +26,6 @@ The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) th

 -----

-Master [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=master)](https://travis-ci.com/pi-hole/pi-hole) Development [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=development)](https://travis-ci.com/pi-hole/pi-hole)
-
 ## One-Step Automated Install

 Those who want to get started quickly and conveniently may install Pi-hole using the following command:
@@ -52,6 +50,8 @@ sudo bash basic-install.sh
 wget -O basic-install.sh https://install.pi-hole.net
 sudo bash basic-install.sh
 ```
+### Method 3: Using Docker to deploy Pi-hole
+Please refer to the [Pi-hole docker repo](https://github.com/pi-hole/docker-pi-hole) to use the Official Docker Images.

 ## [Post-install: Make your network take advantage of Pi-hole](https://docs.pi-hole.net/main/post-install/)

@@ -71,16 +71,18 @@ Make no mistake: **your support is absolutely vital to help keep us innovating!*

 ### [Donations](https://pi-hole.net/donate)

-Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses:
+Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses and rewarding our dedicated development team:

 ### Alternative support

 If you'd rather not donate (_which is okay!_), there are other ways you can help support us:

- [Patreon](https://patreon.com/pihole) _Become a patron for rewards_
+- [GitHub Sponsors](https://github.com/sponsors/pi-hole/)
+- [Patreon](https://patreon.com/pihole)
+- [Hetzner Cloud](https://hetzner.cloud/?ref=7aceisRX3AzA) _affiliate link_
 - [Digital Ocean](https://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
 - [Stickermule](https://www.stickermule.com/unlock?ref_id=9127301701&utm_medium=link&utm_source=invite) _earn a $10 credit after your first purchase_
- [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
+- [Amazon US](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
 - Spreading the word about our software, and how you have benefited from it

 ### Contributing via GitHub
@@ -95,7 +97,9 @@ You'll find that the [install script](https://github.com/pi-hole/pi-hole/blob/ma

 ## Getting in touch with us

-While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets. **Please be sure to check the FAQ's** before starting a new discussion, as we do not have the spare time to reply to every request for assistance.
+While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets.
+
+**Please be sure to check the FAQ's** before starting a new discussion. Many user questions already have answers and can be solved without any additional assistance.

 - [Frequently Asked Questions](https://discourse.pi-hole.net/c/faqs)
 - [Feature Requests](https://discourse.pi-hole.net/c/feature-requests?order=votes)
@@ -106,12 +110,27 @@ While we are primarily reachable on our [Discourse User Forum](https://discourse

 ## Breakdown of Features

+### [Faster-than-light Engine](https://github.com/pi-hole/ftl)
+
+[FTLDNS](https://github.com/pi-hole/ftl) is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
+
+Some of the statistics you can integrate include:
+
+- Total number of domains being blocked
+- Total number of DNS queries today
+- Total number of ads blocked today
+- Percentage of ads blocked
+- Unique domains
+- Queries forwarded (to your chosen upstream DNS server)
+- Queries cached
+- Unique clients
+
+The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).
+
 ### The Command Line Interface

 The [pihole](https://docs.pi-hole.net/core/pihole-command/) command has all the functionality necessary to be able to fully administer the Pi-hole, without the need of the Web Interface. It's fast, user-friendly, and auditable by anyone with an understanding of `bash`.

-![Pi-hole Blacklist Demo](https://pi-hole.github.io/graphics/Screenshots/blacklist-cli.gif)
-
 Some notable features include:

 - [Whitelisting, Blacklisting and Regex](https://docs.pi-hole.net/core/pihole-command/#whitelisting-blacklisting-and-regex)
@@ -128,8 +147,6 @@ You can read our [Core Feature Breakdown](https://docs.pi-hole.net/core/pihole-c

 This [optional dashboard](https://github.com/pi-hole/AdminLTE) allows you to view stats, change settings, and configure your Pi-hole. It's the power of the Command Line Interface, with none of the learning curve!

-![Pi-hole Dashboard](https://pi-hole.github.io/graphics/Screenshots/pihole-dashboard.png)
-
 Some notable features include:

 - Mobile friendly interface
@@ -145,21 +162,3 @@ There are several ways to [access the dashboard](https://discourse.pi-hole.net/t

 1. `http://pi.hole/admin/` (when using Pi-hole as your DNS server)
 2. `http://<IP_ADDPRESS_OF_YOUR_PI_HOLE>/admin/`
-3. `http://pi.hole/` (when using Pi-hole as your DNS server)
-
-## Faster-than-light Engine
-
-FTLDNS is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
-
-Some of the statistics you can integrate include:
-
- Total number of domains being blocked
- Total number of DNS queries today
- Total number of ads blocked today
- Percentage of ads blocked
- Unique domains
- Queries forwarded (to your chosen upstream DNS server)
- Queries cached
- Unique clients
-
-The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can out find [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).
--- a/advanced/01-pihole.conf
+++ b/advanced/01-pihole.conf
@@ -34,7 +34,7 @@ server=@DNS2@

 interface=@INT@

-cache-size=10000
+cache-size=@CACHE_SIZE@

 log-queries
 log-facility=/var/log/pihole.log
--- a/advanced/Scripts/chronometer.sh
+++ b/advanced/Scripts/chronometer.sh
@@ -13,6 +13,7 @@ LC_NUMERIC=C

 # Retrieve stats from FTL engine
 pihole-FTL() {
+    local ftl_port LINE
    ftl_port=$(cat /run/pihole-FTL.port 2> /dev/null)
    if [[ -n "$ftl_port" ]]; then
        # Open connection to FTL
@@ -20,12 +21,13 @@ pihole-FTL() {

        # Test if connection is open
        if { "true" >&3; } 2> /dev/null; then
-            # Send command to FTL
-            echo -e ">$1" >&3
+            # Send command to FTL and ask to quit when finished
+            echo -e ">$1 >quit" >&3

-            # Read input
+            # Read input until we received an empty string and the connection is
+            # closed
            read -r -t 1 LINE <&3
-            until [[ ! $? ]] || [[ "$LINE" == *"EOM"* ]]; do
+            until [[ -z "${LINE}" ]] && [[ ! -t 3 ]]; do
                echo "$LINE" >&1
                read -r -t 1 LINE <&3
            done
@@ -228,15 +230,21 @@ get_sys_stats() {
        mapfile -t ph_ver_raw < <(pihole -v -c 2> /dev/null | sed -n 's/^.* v/v/p')
        if [[ -n "${ph_ver_raw[0]}" ]]; then
            ph_core_ver="${ph_ver_raw[0]}"
-            ph_lte_ver="${ph_ver_raw[1]}"
-            ph_ftl_ver="${ph_ver_raw[2]}"
+            if [[ ${#ph_ver_raw[@]} -eq 2 ]]; then
+                # AdminLTE not installed
+                ph_lte_ver="(not installed)"
+                ph_ftl_ver="${ph_ver_raw[1]}"
+            else
+                ph_lte_ver="${ph_ver_raw[1]}"
+                ph_ftl_ver="${ph_ver_raw[2]}"
+            fi
        else
            ph_core_ver="-1"
        fi

        sys_name=$(hostname)

-        [[ -n "$TEMPERATUREUNIT" ]] && temp_unit="$TEMPERATUREUNIT" || temp_unit="c"
+        [[ -n "$TEMPERATUREUNIT" ]] && temp_unit="${TEMPERATUREUNIT^^}" || temp_unit="C"

        # Get storage stats for partition mounted on /
        read -r -a disk_raw <<< "$(df -B1 / 2> /dev/null | awk 'END{ print $3,$2,$5 }')"
@@ -551,7 +559,7 @@ Calculates stats and displays to an LCD
 Options:
  -j, --json          Output stats as JSON formatted string
  -r, --refresh       Set update frequency (in seconds)
-  -e, --exit          Output stats and exit witout refreshing
+  -e, --exit          Output stats and exit without refreshing
  -h, --help          Display this help text"
  fi

--- a/advanced/Scripts/database_migration/gravity-db.sh
+++ b/advanced/Scripts/database_migration/gravity-db.sh
@@ -110,4 +110,16 @@ upgrade_gravityDB(){
 		sqlite3 "${database}" < "${scriptPath}/11_to_12.sql"
 		version=12
 	fi
+	if [[ "$version" == "12" ]]; then
+		# Add column date_updated to adlist table
+		echo -e "  ${INFO} Upgrading gravity database from version 12 to 13"
+		sqlite3 "${database}" < "${scriptPath}/12_to_13.sql"
+		version=13
+	fi
+	if [[ "$version" == "13" ]]; then
+		# Add columns number and status to adlist table
+		echo -e "  ${INFO} Upgrading gravity database from version 13 to 14"
+		sqlite3 "${database}" < "${scriptPath}/13_to_14.sql"
+		version=14
+	fi
 }
--- a/advanced/Scripts/database_migration/gravity/12_to_13.sql
+++ b/advanced/Scripts/database_migration/gravity/12_to_13.sql
@@ -0,0 +1,18 @@
+.timeout 30000
+
+PRAGMA FOREIGN_KEYS=OFF;
+
+BEGIN TRANSACTION;
+
+ALTER TABLE adlist ADD COLUMN date_updated INTEGER;
+
+DROP TRIGGER tr_adlist_update;
+
+CREATE TRIGGER tr_adlist_update AFTER UPDATE OF address,enabled,comment ON adlist
+    BEGIN
+      UPDATE adlist SET date_modified = (cast(strftime('%s', 'now') as int)) WHERE id = NEW.id;
+    END;
+
+UPDATE info SET value = 13 WHERE property = 'version';
+
+COMMIT;
--- a/advanced/Scripts/database_migration/gravity/13_to_14.sql
+++ b/advanced/Scripts/database_migration/gravity/13_to_14.sql
@@ -0,0 +1,13 @@
+.timeout 30000
+
+PRAGMA FOREIGN_KEYS=OFF;
+
+BEGIN TRANSACTION;
+
+ALTER TABLE adlist ADD COLUMN number INTEGER NOT NULL DEFAULT 0;
+ALTER TABLE adlist ADD COLUMN invalid_domains INTEGER NOT NULL DEFAULT 0;
+ALTER TABLE adlist ADD COLUMN status INTEGER NOT NULL DEFAULT 0;
+
+UPDATE info SET value = 14 WHERE property = 'version';
+
+COMMIT;
--- a/advanced/Scripts/list.sh
+++ b/advanced/Scripts/list.sh
@@ -1,4 +1,6 @@
 #!/usr/bin/env bash
+# shellcheck disable=SC1090
+
 # Pi-hole: A black hole for Internet advertisements
 # (c) 2017 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
@@ -9,9 +11,17 @@
 # Please see LICENSE file for your rights under this license.

 # Globals
-basename=pihole
-piholeDir=/etc/"${basename}"
-gravityDBfile="${piholeDir}/gravity.db"
+piholeDir="/etc/pihole"
+GRAVITYDB="${piholeDir}/gravity.db"
+# Source pihole-FTL from install script
+pihole_FTL="${piholeDir}/pihole-FTL.conf"
+if [[ -f "${pihole_FTL}" ]]; then
+  source "${pihole_FTL}"
+fi
+
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"

 reload=false
 addmode=true
@@ -112,7 +122,7 @@ ProcessDomainList() {
    for dom in "${domList[@]}"; do
        # Format domain into regex filter if requested
        if [[ "${wildcard}" == true ]]; then
-            dom="(^|\\.)${dom//\./\\.}$"
+            dom="(\\.|^)${dom//\./\\.}$"
        fi

        # Logic: If addmode then add to desired list and remove from the other;
@@ -231,7 +241,15 @@ Displaylist() {
 }

 NukeList() {
-    sqlite3 "${gravityDBfile}" "DELETE FROM domainlist WHERE type = ${typeId};"
+    count=$(sqlite3 "${gravityDBfile}" "SELECT COUNT(1) FROM domainlist WHERE type = ${typeId};")
+    listname="$(GetListnameFromTypeId "${typeId}")"    
+    if [ "$count" -gt 0 ];then
+        sqlite3 "${gravityDBfile}" "DELETE FROM domainlist WHERE type = ${typeId};"
+        echo "  ${TICK} Removed ${count} domain(s) from the ${listname}"
+    else
+        echo "  ${INFO} ${listname} already empty. Nothing to do!"
+    fi    
+    exit 0;
 }

 GetComment() {
--- a/advanced/Scripts/piholeARPTable.sh
+++ b/advanced/Scripts/piholeARPTable.sh
@@ -38,7 +38,7 @@ flushARP(){

    # Truncate network_addresses table in pihole-FTL.db
    # This needs to be done before we can truncate the network table due to
-    # foreign key contraints
+    # foreign key constraints
    if ! output=$(sqlite3 "${DBFILE}" "DELETE FROM network_addresses" 2>&1); then
        echo -e "${OVER}  ${CROSS} Failed to truncate network_addresses table"
        echo "  Database location: ${DBFILE}"
--- a/advanced/Scripts/piholeDebug.sh
+++ b/advanced/Scripts/piholeDebug.sh
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
 # Pi-hole: A black hole for Internet advertisements
-# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
+# (c) 2021 Pi-hole (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
 #
 # Generates pihole_debug.log to be used for troubleshooting.
@@ -48,6 +48,7 @@ FAQ_UPDATE_PI_HOLE="${COL_CYAN}https://discourse.pi-hole.net/t/how-do-i-update-p
 FAQ_CHECKOUT_COMMAND="${COL_CYAN}https://discourse.pi-hole.net/t/the-pihole-command-with-examples/738#checkout${COL_NC}"
 FAQ_HARDWARE_REQUIREMENTS="${COL_CYAN}https://docs.pi-hole.net/main/prerequisites/${COL_NC}"
 FAQ_HARDWARE_REQUIREMENTS_PORTS="${COL_CYAN}https://docs.pi-hole.net/main/prerequisites/#ports${COL_NC}"
+FAQ_HARDWARE_REQUIREMENTS_FIREWALLD="${COL_CYAN}https://docs.pi-hole.net/main/prerequisites/#firewalld${COL_NC}"
 FAQ_GATEWAY="${COL_CYAN}https://discourse.pi-hole.net/t/why-is-a-default-gateway-important-for-pi-hole/3546${COL_NC}"
 FAQ_ULA="${COL_CYAN}https://discourse.pi-hole.net/t/use-ipv6-ula-addresses-for-pi-hole/2127${COL_NC}"
 FAQ_FTL_COMPATIBILITY="${COL_CYAN}https://github.com/pi-hole/FTL#compatibility-list${COL_NC}"
@@ -124,6 +125,8 @@ get_ftl_conf_value() {

 PIHOLE_GRAVITY_DB_FILE="$(get_ftl_conf_value "GRAVITYDB" "${PIHOLE_DIRECTORY}/gravity.db")"

+PIHOLE_FTL_DB_FILE="$(get_ftl_conf_value "DBFILE" "${PIHOLE_DIRECTORY}/pihole-FTL.db")"
+
 PIHOLE_COMMAND="${BIN_DIRECTORY}/pihole"
 PIHOLE_COLTABLE_FILE="${BIN_DIRECTORY}/COL_TABLE"

@@ -195,6 +198,33 @@ show_disclaimer(){
    log_write "${DISCLAIMER}"
 }

+check_for_ftl(){
+    echo_current_diagnostic "Checking for pihole-FTL binary"
+
+    declare -g FTL_PATH
+    read -r FTL_PATH < <(which pihole-FTL)
+    if [ -z "${FTL_PATH}" ]; then
+        log_write "${CROSS} ${COL_RED} Unable to find pihole-FTL binary.${COL_NC}"
+        # Non-zero return value
+        return 2
+    else
+        log_write "${TICK} pihole-FTL: ${COL_GREEN}${FTL_PATH}${COL_NC}"
+    fi
+
+}
+
+check_for_lua(){
+    echo_current_diagnostic "Checking for lua capabilities"
+
+    if ! (${FTL_PATH} lua -v &>/dev/null); then
+        log_write "${CROSS} ${COL_RED} pihole-FTL binary does not have lua capabilites.${COL_NC}"
+        # Non-zero return value
+        return
+    else
+        log_write "${TICK} pihole-FTL: ${COL_GREEN}lua found!${COL_NC}"
+    fi
+}
+
 source_setup_variables() {
    # Display the current test that is running
    log_write "\\n${COL_PURPLE}*** [ INITIALIZING ]${COL_NC} Sourcing setup variables"
@@ -241,7 +271,7 @@ initialize_debug() {
    log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initialized."
 }

-# This is a function for visually displaying the curent test that is being run.
+# This is a function for visually displaying the current test that is being run.
 # Accepts one variable: the name of what is being diagnosed
 # Colors do not show in the dasboard, but the icons do: [i], [✓], and [✗]
 echo_current_diagnostic() {
@@ -331,7 +361,17 @@ compare_local_version_to_git_version() {
            return 1
        fi
    else
-        :
+        # There is no git directory so check if the web interface was disabled
+        local setup_vars_web_interface
+        setup_vars_web_interface=$(< ${PIHOLE_SETUP_VARS_FILE} grep ^INSTALL_WEB_INTERFACE | cut -d '=' -f2)
+        if [[ "${pihole_component}" == "Web" ]] && [[ "${setup_vars_web_interface}" == "false" ]]; then
+            log_write "${INFO} ${pihole_component}: Disabled in setupVars.conf via INSTALL_WEB_INTERFACE=false"
+        else
+            # Return an error message
+            log_write "${COL_RED}Directory ${git_dir} doesn't exist${COL_NC}"
+            # and exit with a non zero code
+            return 1
+        fi
    fi
 }

@@ -366,11 +406,11 @@ get_program_version() {
    # Create a local variable so this function can be safely reused
    local program_version
    echo_current_diagnostic "${program_name} version"
-    # Evalutate the program we are checking, if it is any of the ones below, show the version
+    # Evaluate the program we are checking, if it is any of the ones below, show the version
    case "${program_name}" in
-        "lighttpd") program_version="$(${program_name} -v |& head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)"
+        "lighttpd") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)"
                    ;;
-        "php") program_version="$(${program_name} -v |& head -n1 | cut -d '-' -f1 | cut -d ' ' -f2)"
+        "php") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '-' -f1 | cut -d ' ' -f2)"
                ;;
        # If a match is not found, show an error
        *) echo "Unrecognized program";
@@ -396,49 +436,54 @@ check_critical_program_versions() {
 os_check() {
    # This function gets a list of supported OS versions from a TXT record at versions.pi-hole.net
    # and determines whether or not the script is running on one of those systems
-    local remote_os_domain valid_os valid_version detected_os_pretty detected_os detected_version
+    local remote_os_domain valid_os valid_version detected_os detected_version cmdResult digReturnCode response
    remote_os_domain="versions.pi-hole.net"
-    valid_os=false
-    valid_version=false

-    detected_os_pretty=$(cat /etc/*release | grep PRETTY_NAME | cut -d '=' -f2- | tr -d '"')
-    detected_os="${detected_os_pretty%% *}"
-    detected_version=$(cat /etc/*release | grep VERSION_ID | cut -d '=' -f2- | tr -d '"')
+    detected_os=$(grep "\bID\b" /etc/os-release | cut -d '=' -f2 | tr -d '"')
+    detected_version=$(grep VERSION_ID /etc/os-release | cut -d '=' -f2 | tr -d '"')

-    IFS=" " read -r -a supportedOS < <(dig +short -t txt ${remote_os_domain} | tr -d '"')
+    cmdResult="$(dig +short -t txt ${remote_os_domain} @ns1.pi-hole.net 2>&1; echo $?)"
+    #Get the return code of the previous command (last line)
+    digReturnCode="${cmdResult##*$'\n'}"

-    for i in "${supportedOS[@]}"
+    # Extract dig response
+    response="${cmdResult%%$'\n'*}"
+
+    IFS=" " read -r -a supportedOS < <(echo "${response}" | tr -d '"')
+    for distro_and_versions in "${supportedOS[@]}"
    do
-        os_part=$(echo "$i" | cut -d '=' -f1)
-        versions_part=$(echo "$i" | cut -d '=' -f2-)
+        distro_part="${distro_and_versions%%=*}"
+        versions_part="${distro_and_versions##*=}"

-        if [[ "${detected_os}" =~ ${os_part} ]]; then
-          valid_os=true
-          IFS="," read -r -a supportedVer <<<"${versions_part}"
-          for x in "${supportedVer[@]}"
-          do
-            if [[ "${detected_version}" =~ $x ]];then
-              valid_version=true
-              break
-            fi
-          done
-          break
+        if [[ "${detected_os^^}" =~ ${distro_part^^} ]]; then
+            valid_os=true
+            IFS="," read -r -a supportedVer <<<"${versions_part}"
+            for version in "${supportedVer[@]}"
+            do
+                if [[ "${detected_version}" =~ $version ]]; then
+                    valid_version=true
+                    break
+                fi
+            done
+            break
        fi
    done

-    # Display findings back to the user
+    log_write "${INFO} dig return code:  ${digReturnCode}"
+    log_write "${INFO} dig response:  ${response}"
+
    if [ "$valid_os" = true ]; then
-        log_write "${TICK} Distro:  ${COL_GREEN}${detected_os}${COL_NC}"
+        log_write "${TICK} Distro:  ${COL_GREEN}${detected_os^}${COL_NC}"

        if [ "$valid_version" = true ]; then
            log_write "${TICK} Version: ${COL_GREEN}${detected_version}${COL_NC}"
        else
            log_write "${CROSS} Version: ${COL_RED}${detected_version}${COL_NC}"
-            log_write "${CROSS} Error: ${COL_RED}${detected_os} is supported but version ${detected_version} is currently unsupported (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
+            log_write "${CROSS} Error: ${COL_RED}${detected_os^} is supported but version ${detected_version} is currently unsupported (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
        fi
    else
-        log_write "${CROSS} Distro:  ${COL_RED}${detected_os}${COL_NC}"
-        log_write "${CROSS} Error: ${COL_RED}${detected_os} is not a supported distro (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
+        log_write "${CROSS} Distro:  ${COL_RED}${detected_os^}${COL_NC}"
+        log_write "${CROSS} Error: ${COL_RED}${detected_os^} is not a supported distro (${FAQ_HARDWARE_REQUIREMENTS})${COL_NC}"
    fi
 }

@@ -488,6 +533,58 @@ check_selinux() {
    fi
 }

+check_firewalld() {
+    # FirewallD ships by default on Fedora/CentOS/RHEL and enabled upon clean install
+    # FirewallD is not configured by the installer and is the responsibility of the user
+    echo_current_diagnostic "FirewallD"
+    # Check if FirewallD service is enabled
+    if command -v systemctl &> /dev/null; then
+        # get its status via systemctl
+        local firewalld_status
+        firewalld_status=$(systemctl is-active firewalld)
+        log_write "${INFO} ${COL_GREEN}Firewalld service ${firewalld_status}${COL_NC}";
+        if [ "${firewalld_status}" == "active" ]; then
+            # test common required service ports
+            local firewalld_enabled_services
+            firewalld_enabled_services=$(firewall-cmd --list-services)
+            local firewalld_expected_services=("http" "dns" "dhcp" "dhcpv6")
+            for i in "${firewalld_expected_services[@]}"; do
+                if [[ "${firewalld_enabled_services}" =~ ${i} ]]; then
+                    log_write "${TICK} ${COL_GREEN}  Allow Service: ${i}${COL_NC}";
+                else
+                    log_write "${CROSS} ${COL_RED}  Allow Service: ${i}${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+                fi
+            done
+            # check for custom FTL FirewallD zone
+            local firewalld_zones
+            firewalld_zones=$(firewall-cmd --get-zones)
+            if [[ "${firewalld_zones}" =~ "ftl" ]]; then
+                log_write "${TICK} ${COL_GREEN}FTL Custom Zone Detected${COL_NC}";
+                # check FTL custom zone interface: lo
+                local firewalld_ftl_zone_interfaces
+                firewalld_ftl_zone_interfaces=$(firewall-cmd --zone=ftl --list-interfaces)
+                if [[ "${firewalld_ftl_zone_interfaces}" =~ "lo" ]]; then
+                    log_write "${TICK} ${COL_GREEN}  Local Interface Detected${COL_NC}";
+                else
+                    log_write "${CROSS} ${COL_RED}  Local Interface Not Detected${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+                fi
+                # check FTL custom zone port: 4711
+                local firewalld_ftl_zone_ports
+                firewalld_ftl_zone_ports=$(firewall-cmd --zone=ftl --list-ports)
+                if [[ "${firewalld_ftl_zone_ports}" =~ "4711/tcp" ]]; then
+                    log_write "${TICK} ${COL_GREEN}  FTL Port 4711/tcp Detected${COL_NC}";
+                else
+                    log_write "${CROSS} ${COL_RED}  FTL Port 4711/tcp Not Detected${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+                fi
+            else
+                log_write "${CROSS} ${COL_RED}FTL Custom Zone Not Detected${COL_NC} (${FAQ_HARDWARE_REQUIREMENTS_FIREWALLD})"
+            fi
+        fi
+    else
+        log_write "${TICK} ${COL_GREEN}Firewalld service not detected${COL_NC}";
+    fi
+}
+
 processor_check() {
    echo_current_diagnostic "Processor"
    # Store the processor type in a variable
@@ -500,7 +597,7 @@ processor_check() {
    else
        # Check if the architecture is currently supported for FTL
        case "${PROCESSOR}" in
-            "amd64") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
+            "amd64" | "x86_64") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
                ;;
            "armv6l") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
                ;;
@@ -571,7 +668,7 @@ detect_ip_addresses() {
    # First argument should be a 4 or a 6
    local protocol=${1}
    # Use ip to show the addresses for the chosen protocol
-    # Store the values in an arry so they can be looped through
+    # Store the values in an array so they can be looped through
    # Get the lines that are in the file(s) and store them in an array for parsing later
    mapfile -t ip_addr_list < <(ip -"${protocol}" addr show dev "${PIHOLE_INTERFACE}" | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }')

@@ -625,7 +722,7 @@ ping_gateway() {
    # Check if we are using IPv4 or IPv6
    # Find the default gateway using IPv4 or IPv6
    local gateway
-    gateway="$(ip -"${protocol}" route | grep default | cut -d ' ' -f 3)"
+    gateway="$(ip -"${protocol}" route | grep default | grep "${PIHOLE_INTERFACE}" | cut -d ' ' -f 3)"

    # If the gateway variable has a value (meaning a gateway was found),
    if [[ -n "${gateway}" ]]; then
@@ -753,7 +850,7 @@ check_x_headers() {
    # Do it for the dashboard as well, as the header is different than above
    local dashboard
    dashboard=$(curl -Is localhost/admin/ | awk '/X-Pi-hole/' | tr -d '\r')
-    # Store what the X-Header shoud be in variables for comparison later
+    # Store what the X-Header should be in variables for comparison later
    local block_page_working
    block_page_working="X-Pi-hole: A black hole for Internet advertisements."
    local dashboard_working
@@ -772,12 +869,12 @@ check_x_headers() {
        log_write "${COL_RED}${full_curl_output_block_page}${COL_NC}"
    fi

-    # Same logic applies to the dashbord as above, if the X-Header matches what a working system shoud have,
+    # Same logic applies to the dashboard as above, if the X-Header matches what a working system should have,
    if [[ $dashboard == "$dashboard_working" ]]; then
        # then we can show a success
        log_write "$TICK Web interface X-Header: ${COL_GREEN}${dashboard}${COL_NC}"
    else
-        # Othewise, it's a failure since the X-Headers either don't exist or have been modified in some way
+        # Otherwise, it's a failure since the X-Headers either don't exist or have been modified in some way
        log_write "$CROSS Web interface X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
        log_write "${COL_RED}${full_curl_output_dashboard}${COL_NC}"
    fi
@@ -807,7 +904,7 @@ dig_at() {
        local pihole_address="${IP}"
        local remote_address="2001:4860:4860::8888"
        local record_type="AAAA"
-    # Othwerwise, it should be 4
+    # Otherwise, it should be 4
    else
        # so use the IPv4 values
        local local_address="127.0.0.1"
@@ -841,7 +938,7 @@ dig_at() {
        # show a success
        log_write "${TICK} ${random_url} ${COL_GREEN}is ${pihole_dig}${COL_NC} via ${COL_CYAN}Pi-hole${COL_NC} (${pihole_address})"
    else
-        # Othewise, show a failure
+        # Otherwise, show a failure
        log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_RED}Pi-hole${COL_NC} (${pihole_address})"
    fi

@@ -888,6 +985,18 @@ process_status(){
    done
 }

+ftl_full_status(){
+    # if using systemd print the full status of pihole-FTL
+    echo_current_diagnostic "Pi-hole-FTL full status"
+    local FTL_status
+    if command -v systemctl &> /dev/null; then
+      FTL_status=$(systemctl status --full --no-pager pihole-FTL.service)
+      log_write "   ${FTL_status}"
+    else
+      log_write "${INFO} systemctl:  command not found"
+    fi
+}
+
 make_array_from_file() {
    local filename="${1}"
    # The second argument can put a limit on how many line should be read from the file
@@ -905,7 +1014,7 @@ make_array_from_file() {
        # Otherwise, read the file line by line
        while IFS= read -r line;do
            # Othwerise, strip out comments and blank lines
-            new_line=$(echo "${line}" | sed -e 's/#.*$//' -e '/^$/d')
+            new_line=$(echo "${line}" | sed -e 's/^\s*#.*$//' -e '/^$/d')
            # If the line still has content (a non-zero value)
            if [[ -n "${new_line}" ]]; then
                # Put it into the array
@@ -962,7 +1071,7 @@ parse_file() {
 }

 check_name_resolution() {
-    # Check name resoltion from localhost, Pi-hole's IP, and Google's name severs
+    # Check name resolution from localhost, Pi-hole's IP, and Google's name severs
    # using the function we created earlier
    dig_at 4 "${IPV4_ADDRESS%/*}"
    # If IPv6 enabled,
@@ -1021,8 +1130,8 @@ list_files_in_dir() {
                    log_write "\\n${COL_GREEN}$(ls -ld "${dir_to_parse}"/"${each_file}")${COL_NC}"
                    # Check if the file we want to view has a limit (because sometimes we just need a little bit of info from the file, not the entire thing)
                    case "${dir_to_parse}/${each_file}" in
-                        # If it's Web server error log, just give the first 25 lines
-                        "${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}") make_array_from_file "${dir_to_parse}/${each_file}" 25
+                        # If it's Web server error log, give the first and last 25 lines
+                        "${PIHOLE_WEB_SERVER_ERROR_LOG_FILE}") head_tail_log "${dir_to_parse}/${each_file}" 25
                            ;;
                        # Same for the FTL log
                        "${PIHOLE_FTL_LOG}") head_tail_log "${dir_to_parse}/${each_file}" 35
@@ -1112,6 +1221,46 @@ show_db_entries() {
    IFS="$OLD_IFS"
 }

+show_FTL_db_entries() {
+    local title="${1}"
+    local query="${2}"
+    local widths="${3}"
+
+    echo_current_diagnostic "${title}"
+
+    OLD_IFS="$IFS"
+    IFS=$'\r\n'
+    local entries=()
+    mapfile -t entries < <(\
+        sqlite3 "${PIHOLE_FTL_DB_FILE}" \
+            -cmd ".headers on" \
+            -cmd ".mode column" \
+            -cmd ".width ${widths}" \
+            "${query}"\
+    )
+
+    for line in "${entries[@]}"; do
+        log_write "   ${line}"
+    done
+
+    IFS="$OLD_IFS"
+}
+
+check_dhcp_servers() {
+    echo_current_diagnostic "Discovering active DHCP servers (takes 10 seconds)"
+
+    OLD_IFS="$IFS"
+    IFS=$'\n'
+    local entries=()
+    mapfile -t entries < <(pihole-FTL dhcp-discover)
+
+    for line in "${entries[@]}"; do
+        log_write "   ${line}"
+    done
+
+    IFS="$OLD_IFS"
+}
+
 show_groups() {
    show_db_entries "Groups" "SELECT id,CASE enabled WHEN '0' THEN '   0' WHEN '1' THEN '      1' ELSE enabled END enabled,name,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,description FROM \"group\"" "4 7 50 19 19 50"
 }
@@ -1128,6 +1277,10 @@ show_clients() {
    show_db_entries "Clients" "SELECT id,GROUP_CONCAT(client_by_group.group_id) group_ids,ip,datetime(date_added,'unixepoch','localtime') date_added,datetime(date_modified,'unixepoch','localtime') date_modified,comment FROM client LEFT JOIN client_by_group ON client.id = client_by_group.client_id GROUP BY id;" "4 12 100 19 19 50"
 }

+show_messages() {
+    show_FTL_db_entries "Pi-hole diagnosis messages" "SELECT id,datetime(timestamp,'unixepoch','localtime') timestamp,type,message,blob1,blob2,blob3,blob4,blob5 FROM message;" "4 19 20 60 20 20 20 20 20"
+}
+
 analyze_gravity_list() {
    echo_current_diagnostic "Gravity List and Database"

@@ -1295,24 +1448,40 @@ make_temporary_log
 initialize_debug
 # setupVars.conf needs to be sourced before the networking so the values are
 # available to the other functions
-source_setup_variables
-check_component_versions
-check_critical_program_versions
-diagnose_operating_system
-check_selinux
-processor_check
-check_networking
-check_name_resolution
-process_status
-parse_setup_vars
-check_x_headers
-analyze_gravity_list
-show_groups
-show_domainlist
-show_clients
-show_adlists
-show_content_of_pihole_files
-parse_locale
-analyze_pihole_log
-copy_to_debug_log
-upload_to_tricorder
+if ! check_for_ftl; then
+    log_write "${COL_RED}Unable to complete debug run. Please contact support for assistance."
+    log_write "Please note the error that is displayed above.${COL_NC}"
+    #Non-zero return value
+    exit 2
+fi
+if ! check_for_lua; then
+    log_write "${COL_RED}Unable to complete debug run. Please contact support for assistance."
+    log_write "Please note the error that is displayed above.${COL_NC}"
+    #Non-zero return value
+    exit 2
+fi
+# source_setup_variables
+# check_component_versions
+# check_critical_program_versions
+# diagnose_operating_system
+# check_selinux
+# check_firewalld
+# processor_check
+# check_networking
+# check_name_resolution
+# check_dhcp_servers
+# process_status
+# ftl_full_status
+# parse_setup_vars
+# check_x_headers
+# analyze_gravity_list
+# show_groups
+# show_domainlist
+# show_clients
+# show_adlists
+# show_content_of_pihole_files
+# show_messages
+# parse_locale
+# analyze_pihole_log
+# copy_to_debug_log
+# upload_to_tricorder
--- a/advanced/Scripts/query.sh
+++ b/advanced/Scripts/query.sh
@@ -1,5 +1,6 @@
 #!/usr/bin/env bash
 # shellcheck disable=SC1090
+
 # Pi-hole: A black hole for Internet advertisements
 # (c) 2018 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
@@ -11,12 +12,21 @@

 # Globals
 piholeDir="/etc/pihole"
-gravityDBfile="${piholeDir}/gravity.db"
+GRAVITYDB="${piholeDir}/gravity.db"
 options="$*"
 all=""
 exact=""
 blockpage=""
 matchType="match"
+# Source pihole-FTL from install script
+pihole_FTL="${piholeDir}/pihole-FTL.conf"
+if [[ -f "${pihole_FTL}" ]]; then
+  source "${pihole_FTL}"
+fi
+
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"

 colfile="/opt/pihole/COL_TABLE"
 source "${colfile}"
--- a/advanced/Scripts/version.sh
+++ b/advanced/Scripts/version.sh
@@ -153,7 +153,7 @@ versionOutput() {
    if [[ -n "$current" ]] && [[ -n "$latest" ]]; then
        output="${1^} version is $branch$current (Latest: $latest)"
    elif [[ -n "$current" ]] && [[ -z "$latest" ]]; then
-        output="Current ${1^} version is $branch$current."
+        output="Current ${1^} version is $branch$current"
    elif [[ -z "$current" ]] && [[ -n "$latest" ]]; then
        output="Latest ${1^} version is $latest"
    elif [[ "$curHash" == "N/A" ]] || [[ "$latHash" == "N/A" ]]; then
--- a/advanced/Scripts/webpage.sh
+++ b/advanced/Scripts/webpage.sh
@@ -44,7 +44,7 @@ Options:
  -e, email           Set an administrative contact address for the Block Page
  -h, --help          Show this help dialog
  -i, interface       Specify dnsmasq's interface listening behavior
-  -l, privacylevel    Set privacy level (0 = lowest, 4 = highest)"
+  -l, privacylevel    Set privacy level (0 = lowest, 3 = highest)"
    exit 0
 }

@@ -167,9 +167,11 @@ ProcessDNSSettings() {
    fi

    delete_dnsmasq_setting "domain-needed"
+    delete_dnsmasq_setting "expand-hosts"

    if [[ "${DNS_FQDN_REQUIRED}" == true ]]; then
        add_dnsmasq_setting "domain-needed"
+        add_dnsmasq_setting "expand-hosts"
    fi

    delete_dnsmasq_setting "bogus-priv"
@@ -215,6 +217,12 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423

    if [[ "${CONDITIONAL_FORWARDING}" == true ]]; then
        # Convert legacy "conditional forwarding" to rev-server configuration
+        # Remove any existing REV_SERVER settings
+        delete_setting "REV_SERVER"
+        delete_setting "REV_SERVER_DOMAIN"
+        delete_setting "REV_SERVER_TARGET"
+        delete_setting "REV_SERVER_CIDR"
+
        REV_SERVER=true
        add_setting "REV_SERVER" "true"

@@ -224,17 +232,38 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423
        REV_SERVER_TARGET="${CONDITIONAL_FORWARDING_IP}"
        add_setting "REV_SERVER_TARGET" "${REV_SERVER_TARGET}"

+        #Convert CONDITIONAL_FORWARDING_REVERSE if necessary e.g:
+        #          1.1.168.192.in-addr.arpa to 192.168.1.1/32
+        #          1.168.192.in-addr.arpa to 192.168.1.0/24
+        #          168.192.in-addr.arpa to 192.168.0.0/16
+        #          192.in-addr.arpa to 192.0.0.0/8
+        if [[ "${CONDITIONAL_FORWARDING_REVERSE}" == *"in-addr.arpa" ]];then
+            arrRev=("${CONDITIONAL_FORWARDING_REVERSE//./ }")
+            case ${#arrRev[@]} in
+                6   )   REV_SERVER_CIDR="${arrRev[3]}.${arrRev[2]}.${arrRev[1]}.${arrRev[0]}/32";;
+                5   )   REV_SERVER_CIDR="${arrRev[2]}.${arrRev[1]}.${arrRev[0]}.0/24";;
+                4   )   REV_SERVER_CIDR="${arrRev[1]}.${arrRev[0]}.0.0/16";;
+                3   )   REV_SERVER_CIDR="${arrRev[0]}.0.0.0/8";;
+            esac
+        else
+          # Set REV_SERVER_CIDR to whatever value it was set to
+          REV_SERVER_CIDR="${CONDITIONAL_FORWARDING_REVERSE}"
+        fi
+
+        # If REV_SERVER_CIDR is not converted by the above, then use the REV_SERVER_TARGET variable to derive it
+        if [ -z "${REV_SERVER_CIDR}" ]; then
+            # Convert existing input to /24 subnet (preserves legacy behavior)
+            # This sed converts "192.168.1.2" to "192.168.1.0/24"
+            # shellcheck disable=2001
+            REV_SERVER_CIDR="$(sed "s+\\.[0-9]*$+\\.0/24+" <<< "${REV_SERVER_TARGET}")"
+        fi
+        add_setting "REV_SERVER_CIDR" "${REV_SERVER_CIDR}"
+
        # Remove obsolete settings from setupVars.conf
        delete_setting "CONDITIONAL_FORWARDING"
        delete_setting "CONDITIONAL_FORWARDING_REVERSE"
        delete_setting "CONDITIONAL_FORWARDING_DOMAIN"
        delete_setting "CONDITIONAL_FORWARDING_IP"
-
-        # Convert existing input to /24 subnet (preserves legacy behavior)
-        # This sed converts "192.168.1.2" to "192.168.1.0/24"
-        # shellcheck disable=2001
-        REV_SERVER_CIDR="$(sed "s+\\.[0-9]*$+\\.0/24+" <<< "${REV_SERVER_TARGET}")"
-        add_setting "REV_SERVER_CIDR" "${REV_SERVER_CIDR}"
    fi

    if [[ "${REV_SERVER}" == true ]]; then
@@ -248,6 +277,13 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423
    # This follows https://support.mozilla.org/en-US/kb/configuring-networks-disable-dns-over-https
    # (sourced 7th September 2019)
    add_dnsmasq_setting "server=/use-application-dns.net/"
+
+    # We need to process DHCP settings here as well to account for possible
+    # changes in the non-FQDN forwarding. This cannot be done in 01-pihole.conf
+    # as we don't want to delete all local=/.../ lines so it's much safer to
+    # simply rewrite the entire corresponding config file (which is what the
+    # DHCP settings subroutie is doing)
+    ProcessDHCPSettings
 }

 SetDNSServers() {
@@ -370,6 +406,14 @@ dhcp-leasefile=/etc/pihole/dhcp.leases

    if [[ "${PIHOLE_DOMAIN}" != "none" ]]; then
        echo "domain=${PIHOLE_DOMAIN}" >> "${dhcpconfig}"
+
+        # When there is a Pi-hole domain set and "Never forward non-FQDNs" is
+        # ticked, we add `local=/domain/` to tell FTL that this domain is purely
+        # local and FTL may answer queries from /etc/hosts or DHCP but should
+        # never forward queries on that domain to any upstream servers
+        if  [[ "${DNS_FQDN_REQUIRED}" == true ]]; then
+          echo "local=/${PIHOLE_DOMAIN}/" >> "${dhcpconfig}"
+        fi
    fi

    # Sourced from setupVars
@@ -442,10 +486,15 @@ SetWebUITheme() {
 }

 CheckUrl(){
-    local regex
+    local regex check_url
    # Check for characters NOT allowed in URLs
-    regex="[^a-zA-Z0-9:/?&%=~._-]"
-    if [[ "${1}" =~ ${regex} ]]; then
+    regex="[^a-zA-Z0-9:/?&%=~._()-;]"
+
+    # this will remove first @ that is after schema and before domain
+    # \1 is optional schema, \2 is userinfo
+    check_url="$( sed -re 's#([^:/]*://)?([^/]+)@#\1\2#' <<< "$1" )"
+
+    if [[ "${check_url}" =~ ${regex} ]]; then
        return 1
    else
        return 0
@@ -515,7 +564,13 @@ AddDHCPStaticAddress() {

 RemoveDHCPStaticAddress() {
    mac="${args[2]}"
-    sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
+    if [[ "$mac" =~ ^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$ ]]; then
+        sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
+    else
+        echo "  ${CROSS} Invalid Mac Passed!"
+        exit 1
+    fi
+
 }

 SetAdminEmail() {
@@ -587,8 +642,11 @@ Interfaces:

 Teleporter() {
    local datetimestamp
+    local host
    datetimestamp=$(date "+%Y-%m-%d_%H-%M-%S")
-    php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-teleporter_${datetimestamp}.tar.gz"
+    host=$(hostname)
+    host="${host//./_}"
+    php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-${host:-noname}-teleporter_${datetimestamp}.tar.gz"
 }

 checkDomain()
@@ -633,8 +691,8 @@ clearAudit()
 }

 SetPrivacyLevel() {
-    # Set privacy level. Minimum is 0, maximum is 4
-    if [ "${args[2]}" -ge 0 ] && [ "${args[2]}" -le 4 ]; then
+    # Set privacy level. Minimum is 0, maximum is 3
+    if [ "${args[2]}" -ge 0 ] && [ "${args[2]}" -le 3 ]; then
        changeFTLsetting "PRIVACYLEVEL" "${args[2]}"
        pihole restartdns reload-lists
    fi
@@ -656,7 +714,13 @@ RemoveCustomDNSAddress() {

    ip="${args[2]}"
    host="${args[3]}"
-    sed -i "/${ip} ${host}/d" "${dnscustomfile}"
+
+    if valid_ip "${ip}" || valid_ip6 "${ip}" ; then
+        sed -i "/${ip} ${host}/d" "${dnscustomfile}"
+    else
+        echo -e "  ${CROSS} Invalid IP has been passed"
+        exit 1
+    fi

    # Restart dnsmasq to update removed custom DNS entries
    RestartDNS
@@ -667,6 +731,7 @@ AddCustomCNAMERecord() {

    domain="${args[2]}"
    target="${args[3]}"
+
    echo "cname=${domain},${target}" >> "${dnscustomcnamefile}"

    # Restart dnsmasq to load new custom CNAME records
@@ -678,7 +743,20 @@ RemoveCustomCNAMERecord() {

    domain="${args[2]}"
    target="${args[3]}"
-    sed -i "/cname=${domain},${target}/d" "${dnscustomcnamefile}"
+
+    validDomain="$(checkDomain "${domain}")"
+    if [[ -n "${validDomain}" ]]; then
+        validTarget="$(checkDomain "${target}")"
+        if [[ -n "${validDomain}" ]]; then
+            sed -i "/cname=${validDomain},${validTarget}/d" "${dnscustomcnamefile}"
+        else
+            echo "  ${CROSS} Invalid Target Passed!"
+            exit 1
+        fi
+    else
+        echo "  ${CROSS} Invalid Domain passed!"
+        exit 1
+    fi

    # Restart dnsmasq to update removed custom CNAME records
    RestartDNS
--- a/advanced/Templates/gravity.db.sql
+++ b/advanced/Templates/gravity.db.sql
@@ -31,7 +31,11 @@ CREATE TABLE adlist
 	enabled BOOLEAN NOT NULL DEFAULT 1,
 	date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
-	comment TEXT
+	comment TEXT,
+	date_updated INTEGER,
+	number INTEGER NOT NULL DEFAULT 0,
+	invalid_domains INTEGER NOT NULL DEFAULT 0,
+	status INTEGER NOT NULL DEFAULT 0
 );

 CREATE TABLE adlist_by_group
@@ -53,7 +57,7 @@ CREATE TABLE info
 	value TEXT NOT NULL
 );

-INSERT INTO "info" VALUES('version','12');
+INSERT INTO "info" VALUES('version','14');

 CREATE TABLE domain_audit
 (
@@ -72,7 +76,7 @@ CREATE TABLE domainlist_by_group
 CREATE TABLE client
 (
 	id INTEGER PRIMARY KEY AUTOINCREMENT,
-	ip TEXT NOL NULL UNIQUE,
+	ip TEXT NOT NULL UNIQUE,
 	date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	comment TEXT
@@ -85,9 +89,9 @@ CREATE TABLE client_by_group
 	PRIMARY KEY (client_id, group_id)
 );

-CREATE TRIGGER tr_adlist_update AFTER UPDATE ON adlist
+CREATE TRIGGER tr_adlist_update AFTER UPDATE OF address,enabled,comment ON adlist
    BEGIN
-      UPDATE adlist SET date_modified = (cast(strftime('%s', 'now') as int)) WHERE address = NEW.address;
+      UPDATE adlist SET date_modified = (cast(strftime('%s', 'now') as int)) WHERE id = NEW.id;
    END;

 CREATE TRIGGER tr_client_update AFTER UPDATE ON client
--- a/advanced/bash-completion/pihole
+++ b/advanced/bash-completion/pihole
@@ -56,11 +56,11 @@ _pihole() {
 		;;
 		"privacylevel")
 			if ( [[ "$prev2" == "admin" ]] || [[ "$prev2" == "-a" ]] ); then
-				opts_privacy="0 1 2 3 4"
+				opts_privacy="0 1 2 3"
 				COMPREPLY=( $(compgen -W "${opts_privacy}" -- ${cur}) )
-			else 
+			else
 				return 1
-			fi	
+			fi
 		;;
 		"core"|"admin"|"ftl")
 			if [[ "$prev2" == "checkout" ]]; then
--- a/advanced/blockingpage.css
+++ b/advanced/blockingpage.css
@@ -145,7 +145,17 @@ body {
 }

 /* User is greeted with a splash page when browsing to Pi-hole IP address */
-#splashpage { background: #222; color: rgba(255, 255, 255, 0.7); text-align: center; }
+#splashpage {
+  background: #222;
+  color: rgba(255, 255, 255, 0.7);
+  text-align: center;
+  width: 100%;
+  height: 100%;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+}
+
 #splashpage img { margin: 5px; width: 256px; }
 #splashpage b { color: inherit; }

@@ -196,6 +206,26 @@ header #bpAlt label {
  display: block;
 }

+html, body {
+    height: 100%;
+}
+
+#pihole_card {
+    width: 400px;
+    height: auto;
+    max-width: 400px;
+}
+
+    #pihole_card p, #pihole_card a {
+        font-size: 13pt;
+        text-align: center;
+    }
+
+#pihole_logo_splash {
+    height: auto;
+    width: 100%;
+}
+
 /* Click anywhere else on screen to hide #bpAbout */
 #bpAboutToggle:checked {
  display: block;
@@ -382,12 +412,44 @@ footer {

 /* Responsive Content */
@media only screen and (max-width: 500px) {
-  h1 a { font-size: 1.8rem; min-width: 170px; }
-  footer span::before { content: "Generated "; }
-  footer span { display: block; }
+  h1 a {
+        font-size: 1.8rem;
+        min-width: 170px;
+    }
+
+    footer span::before {
+        content: "Generated ";
+    }
+
+    footer span {
+        display: block;
+    }
 }

@media only screen and (min-width: 1251px) {
-  #bpWrapper, footer { border-radius: 0 0 5px 5px; }
-  #bpAbout { border-right-width: 1px; }
+  #bpWrapper, footer {
+        border-radius: 0 0 5px 5px;
+    }
+
+    #bpAbout {
+        border-right-width: 1px;
+    }
+}
+
+@media only screen and (max-width: 400px) {
+    #pihole_card {
+        width: 100%;
+        height: auto;
+    }
+
+        #pihole_card p, #pihole_card a {
+            font-size: 100%;
+        }
+}
+
+@media only screen and (max-width: 256px) {
+    #pihole_logo_splash {
+        width: 90% !important;
+        height: auto;
+    }
 }
--- a/advanced/index.php
+++ b/advanced/index.php
@@ -24,7 +24,7 @@ unset($setupVars);
 $landPage = "../landing.php";

 // Define array for hostnames to be accepted as self address for splash page
-$authorizedHosts = [];
+$authorizedHosts = [ "localhost" ];
 if (!empty($_SERVER["FQDN"])) {
    // If setenv.add-environment = ("fqdn" => "true") is configured in lighttpd,
    // append $serverName to $authorizedHosts
@@ -55,8 +55,18 @@ if ($serverName === "pi.hole"
    // Redirect to Web Interface
    exit(header("Location: /admin"));
 } elseif (filter_var($serverName, FILTER_VALIDATE_IP) || in_array($serverName, $authorizedHosts)) {
-    // Set Splash Page output
-    $splashPage = "
+    // When directly browsing via IP or authorized hostname
+    // Render splash/landing page based off presence of $landPage file
+    // Unset variables so as to not be included in $landPage or $splashPage
+    unset($svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt);
+    // If $landPage file is present
+    if (is_file(getcwd()."/$landPage")) {
+        unset($serverName, $viewPort); // unset extra variables not to be included in $landpage
+        include $landPage;
+        exit();
+    }
+    // If $landPage file was not present, Set Splash Page output
+    $splashPage = <<<EOT
    <!doctype html>
    <html lang='en'>
        <head>
@@ -67,22 +77,15 @@ if ($serverName === "pi.hole"
            <link rel='shortcut icon' href='admin/img/favicons/favicon.ico' type='image/x-icon'>
        </head>
        <body id='splashpage'>
-            <img src='admin/img/logo.svg' alt='Pi-hole logo' width='256' height='377'>
-            <br>
-            <p>Pi-<strong>hole</strong>: Your black hole for Internet advertisements</p>
-            <a href='/admin'>Did you mean to go to the admin panel?</a>
+            <div id="pihole_card">
+              <img src='admin/img/logo.svg' alt='Pi-hole logo' id="pihole_logo_splash" />
+              <p>Pi-<strong>hole</strong>: Your black hole for Internet advertisements</p>
+              <a href='/admin'>Did you mean to go to the admin panel?</a>
+            </div>
        </body>
    </html>
-    ";
-
-    // Set splash/landing page based off presence of $landPage
-    $renderPage = is_file(getcwd()."/$landPage") ? include $landPage : "$splashPage";
-
-    // Unset variables so as to not be included in $landPage
-    unset($serverName, $svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt, $viewPort);
-
-    // Render splash/landing page when directly browsing via IP or authorized hostname
-    exit($renderPage);
+EOT;
+    exit($splashPage);
 } elseif ($currentUrlExt === "js") {
    // Serve Pi-hole JavaScript for blocked domains requesting JS
    exit(setHeader("js").'var x = "Pi-hole: A black hole for Internet advertisements."');
@@ -305,7 +308,7 @@ setHeader();
      </p>
    </div>
    <div class="aboutLink">
-      <a class="linkPH" href="https://github.com/pi-hole/pi-hole/wiki/What-is-Pi-hole%3F-A-simple-explanation"><?php //About PH ?></a>
+      <a class="linkPH" href="https://docs.pi-hole.net/"><?php //About PH ?></a>
      <?php if (!empty($svEmail)) echo '<a class="linkEmail" href="mailto:'.$svEmail.'"></a>'; ?>
    </div>
  </div>
--- a/advanced/lighttpd.conf.debian
+++ b/advanced/lighttpd.conf.debian
@@ -81,6 +81,10 @@ mimetype.assign = (
    ".woff2" => "font/woff2"
 )

+# Add user chosen options held in external file
+# This uses include_shell instead of an include wildcard for compatibility
+include_shell "cat external.conf 2>/dev/null"
+
 # default listening port for IPv6 falls back to the IPv4 port
 include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port

@@ -109,7 +113,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {

 # Default expire header
 expire.url = ( "" => "access plus 0 seconds" )
-
-# Add user chosen options held in external file
-# This uses include_shell instead of an include wildcard for compatibility
-include_shell "cat external.conf 2>/dev/null"
--- a/advanced/lighttpd.conf.fedora
+++ b/advanced/lighttpd.conf.fedora
@@ -82,6 +82,10 @@ mimetype.assign = (
    ".woff2" => "font/woff2"
 )

+# Add user chosen options held in external file
+# This uses include_shell instead of an include wildcard for compatibility
+include_shell "cat external.conf 2>/dev/null"
+
 # default listening port for IPv6 falls back to the IPv4 port
 #include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
 #include_shell "/usr/share/lighttpd/create-mime.assign.pl"
@@ -117,7 +121,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {

 # Default expire header
 expire.url = ( "" => "access plus 0 seconds" )
-
-# Add user chosen options held in external file
-# This uses include_shell instead of an include wildcard for compatibility
-include_shell "cat external.conf 2>/dev/null"
--- a/install/basic-install.sh
+++ b/install/basic-install.sh
--- a/install/uninstall.sh
+++ b/install/uninstall.sh
@@ -31,7 +31,7 @@ else
    else
        echo -e "  ${CROSS} ${str}
            Script called with non-root privileges
-            The Pi-hole requires elevated privleges to uninstall"
+            The Pi-hole requires elevated privileges to uninstall"
        exit 1
    fi
 fi
--- a/1
+++ b/1
@@ -1 +0,0 @@
-py.test -v -f test/
--- a/gravity.sh
+++ b/gravity.sh
@@ -35,11 +35,11 @@ localList="${piholeDir}/local.list"
 VPNList="/etc/openvpn/ipp.txt"

 piholeGitDir="/etc/.pihole"
-gravityDBfile="${piholeDir}/gravity.db"
-gravityTEMPfile="${piholeDir}/gravity_temp.db"
+gravityDBfile_default="${piholeDir}/gravity.db"
+# GRAVITYDB may be overwritten by source pihole-FTL.conf below
+GRAVITYDB="${gravityDBfile_default}"
 gravityDBschema="${piholeGitDir}/advanced/Templates/gravity.db.sql"
 gravityDBcopy="${piholeGitDir}/advanced/Templates/gravity_copy.sql"
-optimize_database=false

 domainsExtension="domains"

@@ -69,6 +69,11 @@ if [[ -f "${pihole_FTL}" ]]; then
  source "${pihole_FTL}"
 fi

+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"
+gravityTEMPfile="${GRAVITYDB}_temp"
+
 if [[ -z "${BLOCKINGMODE}" ]] ; then
  BLOCKINGMODE="NULL"
 fi
@@ -85,11 +90,11 @@ generate_gravity_database() {

 # Copy data from old to new database file and swap them
 gravity_swap_databases() {
-  local str
+  local str copyGravity
  str="Building tree"
  echo -ne "  ${INFO} ${str}..."

-  # The index is intentionally not UNIQUE as prro quality adlists may contain domains more than once
+  # The index is intentionally not UNIQUE as poor quality adlists may contain domains more than once
  output=$( { sqlite3 "${gravityTEMPfile}" "CREATE INDEX idx_gravity ON gravity (domain, adlist_id);"; } 2>&1 )
  status="$?"

@@ -102,7 +107,14 @@ gravity_swap_databases() {
  str="Swapping databases"
  echo -ne "  ${INFO} ${str}..."

-  output=$( { sqlite3 "${gravityTEMPfile}" < "${gravityDBcopy}"; } 2>&1 )
+  # Gravity copying SQL script
+  copyGravity="$(cat "${gravityDBcopy}")"
+  if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
+    # Replace default gravity script location by custom location
+    copyGravity="${copyGravity//"${gravityDBfile_default}"/"${gravityDBfile}"}"
+  fi
+
+  output=$( { sqlite3 "${gravityTEMPfile}" <<< "${copyGravity}"; } 2>&1 )
  status="$?"

  if [[ "${status}" -ne 0 ]]; then
@@ -177,7 +189,7 @@ database_table_from_file() {
        echo "${rowid},\"${domain}\",${timestamp}" >> "${tmpFile}"
      elif [[ "${table}" == "adlist" ]]; then
        # Adlist table format
-        echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"" >> "${tmpFile}"
+        echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\",,0,0,0" >> "${tmpFile}"
      else
        # White-, black-, and regexlist table format
        echo "${rowid},${type},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"" >> "${tmpFile}"
@@ -207,6 +219,59 @@ database_table_from_file() {
    echo -e "  ${CROSS} Unable to remove ${tmpFile}"
 }

+# Update timestamp of last update of this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_updated() {
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET date_updated = (cast(strftime('%%s', 'now') as int)) WHERE id = %i;\\n" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update timestamp of adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
+# Check if a column with name ${2} exists in gravity table with name ${1}
+gravity_column_exists() {
+  output=$( { printf ".timeout 30000\\nSELECT EXISTS(SELECT * FROM pragma_table_info('%s') WHERE name='%s');\\n" "${1}" "${2}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  if [[ "${output}" == "1" ]]; then
+    return 0 # Bash 0 is success
+  fi
+
+  return 1 # Bash non-0 is failure
+}
+
+# Update number of domain on this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_number() {
+  # Only try to set number of domains when this field exists in the gravity database
+  if ! gravity_column_exists "adlist" "number"; then
+    return;
+  fi
+
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET number = %i, invalid_domains = %i WHERE id = %i;\\n" "${num_lines}" "${num_invalid}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update number of domains in adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
+# Update status of this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_status() {
+  # Only try to set the status when this field exists in the gravity database
+  if ! gravity_column_exists "adlist" "status"; then
+    return;
+  fi
+
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET status = %i WHERE id = %i;\\n" "${2}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update status of adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
 # Migrate pre-v5.0 list files to database-based Pi-hole versions
 migrate_to_database() {
  # Create database file only if not present
@@ -307,6 +372,10 @@ gravity_CheckDNSResolutionAvailable() {
 gravity_DownloadBlocklists() {
  echo -e "  ${INFO} ${COL_BOLD}Neutrino emissions detected${COL_NC}..."

+  if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
+    echo -e "  ${INFO} Storing gravity database in ${COL_BOLD}${gravityDBfile}${COL_NC}"
+  fi
+
  # Retrieve source URLs from gravity database
  # We source only enabled adlists, sqlite3 stores boolean values as 0 (false) or 1 (true)
  mapfile -t sources <<< "$(sqlite3 "${gravityDBfile}" "SELECT address FROM vw_adlist;" 2> /dev/null)"
@@ -334,7 +403,7 @@ gravity_DownloadBlocklists() {
    return 1
  fi

-  local url domain agent cmd_ext str target
+  local url domain agent cmd_ext str target compression
  echo ""

  # Prepare new gravity database
@@ -353,13 +422,24 @@ gravity_DownloadBlocklists() {

  target="$(mktemp -p "/tmp" --suffix=".gravity")"

+  # Use compression to reduce the amount of data that is transferred
+  # between the Pi-hole and the ad list provider. Use this feature
+  # only if it is supported by the locally available version of curl
+  if curl -V | grep -q "Features:.* libz"; then
+    compression="--compressed"
+    echo -e "  ${INFO} Using libz compression\n"
+  else
+      compression=""
+      echo -e "  ${INFO} Libz compression not available\n"
+    fi
  # Loop through $sources and download each one
  for ((i = 0; i < "${#sources[@]}"; i++)); do
    url="${sources[$i]}"
    domain="${sourceDomains[$i]}"
+    id="${sourceIDs[$i]}"

    # Save the file as list.#.domain
-    saveLocation="${piholeDir}/list.${i}.${domain}.${domainsExtension}"
+    saveLocation="${piholeDir}/list.${id}.${domain}.${domainsExtension}"
    activeDomains[$i]="${saveLocation}"

    # Default user-agent (for Cloudflare's Browser Integrity Check: https://support.cloudflare.com/hc/en-us/articles/200170086-What-does-the-Browser-Integrity-Check-do-)
@@ -372,13 +452,18 @@ gravity_DownloadBlocklists() {
    esac

    echo -e "  ${INFO} Target: ${url}"
-    local regex
+    local regex check_url
    # Check for characters NOT allowed in URLs
    regex="[^a-zA-Z0-9:/?&%=~._()-;]"
-    if [[ "${url}" =~ ${regex} ]]; then
+
+    # this will remove first @ that is after schema and before domain
+    # \1 is optional schema, \2 is userinfo
+    check_url="$( sed -re 's#([^:/]*://)?([^/]+)@#\1\2#' <<< "$url" )"
+
+    if [[ "${check_url}" =~ ${regex} ]]; then
        echo -e "  ${CROSS} Invalid Target"
    else
-       gravity_DownloadBlocklistFromUrl "${url}" "${cmd_ext}" "${agent}" "${sourceIDs[$i]}" "${saveLocation}" "${target}"
+       gravity_DownloadBlocklistFromUrl "${url}" "${cmd_ext}" "${agent}" "${sourceIDs[$i]}" "${saveLocation}" "${target}" "${compression}"
    fi
    echo ""
  done
@@ -418,6 +503,8 @@ gravity_DownloadBlocklists() {
 }

 total_num=0
+num_lines=0
+num_invalid=0
 parseList() {
  local adlistID="${1}" src="${2}" target="${3}" incorrect_lines
  # This sed does the following things:
@@ -428,7 +515,7 @@ parseList() {
  # Find (up to) five domains containing invalid characters (see above)
  incorrect_lines="$(sed -e "/[^a-zA-Z0-9.\_-]/!d" "${src}" | head -n 5)"

-  local num_lines num_target_lines num_correct_lines num_invalid
+  local num_target_lines num_correct_lines num_invalid
  # Get number of lines in source file
  num_lines="$(grep -c "^" "${src}")"
  # Get number of lines in destination file
@@ -437,9 +524,9 @@ parseList() {
  total_num="$num_target_lines"
  num_invalid="$(( num_lines-num_correct_lines ))"
  if [[ "${num_invalid}" -eq 0 ]]; then
-    echo "  ${INFO} Received ${num_lines} domains"
+    echo "  ${INFO} Analyzed ${num_lines} domains"
  else
-    echo "  ${INFO} Received ${num_lines} domains, ${num_invalid} domains invalid!"
+    echo "  ${INFO} Analyzed ${num_lines} domains, ${num_invalid} domains invalid!"
  fi

  # Display sample of invalid lines if we found some
@@ -450,10 +537,33 @@ parseList() {
    done <<< "${incorrect_lines}"
  fi
 }
+compareLists() {
+  local adlistID="${1}" target="${2}"
+
+  # Verify checksum when an older checksum exists
+  if [[ -s "${target}.sha1" ]]; then
+    if ! sha1sum --check --status --strict "${target}.sha1"; then
+      # The list changed upstream, we need to update the checksum
+      sha1sum "${target}" > "${target}.sha1"
+      echo "  ${INFO} List has been updated"
+      database_adlist_status "${adlistID}" "1"
+      database_adlist_updated "${adlistID}"
+    else
+      echo "  ${INFO} List stayed unchanged"
+      database_adlist_status "${adlistID}" "2"
+    fi
+  else
+    # No checksum available, create one for comparing on the next run
+    sha1sum "${target}" > "${target}.sha1"
+    # We assume here it was changed upstream
+    database_adlist_status "${adlistID}" "1"
+    database_adlist_updated "${adlistID}"
+  fi
+}

 # Download specified URL and perform checks on HTTP status and file content
 gravity_DownloadBlocklistFromUrl() {
-  local url="${1}" cmd_ext="${2}" agent="${3}" adlistID="${4}" saveLocation="${5}" target="${6}"
+  local url="${1}" cmd_ext="${2}" agent="${3}" adlistID="${4}" saveLocation="${5}" target="${6}" compression="${7}"
  local heisenbergCompensator="" patternBuffer str httpCode success=""

  # Create temp file to store content on disk instead of RAM
@@ -502,8 +612,9 @@ gravity_DownloadBlocklistFromUrl() {
    echo -ne "  ${INFO} ${str} Pending..."
    cmd_ext="--resolve $domain:$port:$ip $cmd_ext"
  fi
+
  # shellcheck disable=SC2086
-  httpCode=$(curl -s -L ${cmd_ext} ${heisenbergCompensator} -w "%{http_code}" -A "${agent}" "${url}" -o "${patternBuffer}" 2> /dev/null)
+  httpCode=$(curl -s -L ${compression} ${cmd_ext} ${heisenbergCompensator} -w "%{http_code}" -A "${agent}" "${url}" -o "${patternBuffer}" 2> /dev/null)

  case $url in
    # Did we "download" a local file?
@@ -532,29 +643,49 @@ gravity_DownloadBlocklistFromUrl() {
      esac;;
  esac

+  local done="false"
  # Determine if the blocklist was downloaded and saved correctly
  if [[ "${success}" == true ]]; then
    if [[ "${httpCode}" == "304" ]]; then
      # Add domains to database table file
      parseList "${adlistID}" "${saveLocation}" "${target}"
+      database_adlist_status "${adlistID}" "2"
+      database_adlist_number "${adlistID}"
+      done="true"
    # Check if $patternbuffer is a non-zero length file
    elif [[ -s "${patternBuffer}" ]]; then
      # Determine if blocklist is non-standard and parse as appropriate
      gravity_ParseFileIntoDomains "${patternBuffer}" "${saveLocation}"
      # Add domains to database table file
      parseList "${adlistID}" "${saveLocation}" "${target}"
+      # Compare lists, are they identical?
+      compareLists "${adlistID}" "${saveLocation}"
+      # Update gravity database table (status and updated timestamp are set in
+      # compareLists)
+      database_adlist_number "${adlistID}"
+      done="true"
    else
      # Fall back to previously cached list if $patternBuffer is empty
-      echo -e "  ${INFO} Received empty file: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
+      echo -e "  ${INFO} Received empty file"
    fi
-  else
+  fi
+
+  # Do we need to fall back to a cached list (if available)?
+  if [[ "${done}" != "true" ]]; then
    # Determine if cached list has read permission
    if [[ -r "${saveLocation}" ]]; then
      echo -e "  ${CROSS} List download failed: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
      # Add domains to database table file
      parseList "${adlistID}" "${saveLocation}" "${target}"
+      database_adlist_number "${adlistID}"
+      database_adlist_status "${adlistID}" "3"
    else
      echo -e "  ${CROSS} List download failed: ${COL_LIGHT_RED}no cached list available${COL_NC}"
+      # Manually reset these two numbers because we do not call parseList here
+      num_lines=0
+      num_invalid=0
+      database_adlist_number "${adlistID}"
+      database_adlist_status "${adlistID}" "4"
    fi
  fi
 }
@@ -566,7 +697,7 @@ gravity_ParseFileIntoDomains() {
  # Determine if we are parsing a consolidated list
  #if [[ "${source}" == "${piholeDir}/${matterAndLight}" ]]; then
    # Remove comments and print only the domain name
-    # Most of the lists downloaded are already in hosts file format but the spacing/formating is not contiguous
+    # Most of the lists downloaded are already in hosts file format but the spacing/formatting is not contiguous
    # This helps with that and makes it easier to read
    # It also helps with debugging so each stage of the script can be researched more in depth
    # 1) Remove carriage returns
@@ -728,21 +859,6 @@ gravity_Cleanup() {

  echo -e "${OVER}  ${TICK} ${str}"

-  if ${optimize_database} ; then
-    str="Optimizing domains database"
-    echo -ne "  ${INFO} ${str}..."
-    # Run VACUUM command on database to optimize it
-    output=$( { sqlite3 "${gravityDBfile}" "VACUUM;"; } 2>&1 )
-    status="$?"
-
-    if [[ "${status}" -ne 0 ]]; then
-      echo -e "\\n  ${CROSS} Unable to optimize gravity database ${gravityDBfile}\\n  ${output}"
-      error="error"
-    else
-      echo -e "${OVER}  ${TICK} ${str}"
-    fi
-  fi
-
  # Only restart DNS service if offline
  if ! pgrep pihole-FTL &> /dev/null; then
    "${PIHOLE_COMMAND}" restartdns
@@ -769,7 +885,6 @@ Options:
 for var in "$@"; do
  case "${var}" in
    "-f" | "--force" ) forceDelete=true;;
-    "-o" | "--optimize" ) optimize_database=true;;
    "-r" | "--recreate" ) recreate_database=true;;
    "-h" | "--help" ) helpFunc;;
  esac
--- a/manpages/pihole-FTL.8
+++ b/manpages/pihole-FTL.8
@@ -1,4 +1,4 @@
-.TH "Pihole-FTL" "8" "pihole-FTL" "Pi-hole" "June 2018"
+.TH "Pihole-FTL" "8" "pihole-FTL" "Pi-hole" "November 2020"
 .SH "NAME"
 pihole-FTL - Pi-hole : The Faster-Than-Light (FTL) Engine
 .br
@@ -10,7 +10,7 @@ pihole-FTL - Pi-hole : The Faster-Than-Light (FTL) Engine
 .br
 \fBpihole-FTL test\fR
 .br
-\fBpihole-FTL -v\fR
+\fBpihole-FTL -v|-vv\fR
 .br
 \fBpihole-FTL -t\fR
 .br
@@ -22,6 +22,16 @@ pihole-FTL - Pi-hole : The Faster-Than-Light (FTL) Engine
 .br
 \fBpihole-FTL dnsmasq-test\fR
 .br
+\fBpihole-FTL regex-test str\fR
+.br
+\fBpihole-FTL regex-test str rgx\fR
+.br
+\fBpihole-FTL lua\fR
+.br
+\fBpihole-FTL luac\fR
+.br
+\fBpihole-FTL dhcp-discover\fR
+.br
 \fBpihole-FTL --\fR (\fBoptions\fR)
 .br

@@ -65,6 +75,11 @@ Command line arguments
    Don't start FTL, show only version
 .br

+\fB-vv\fR
+.br
+    Don't start FTL, show verbose version information of embedded applications
+.br
+
 \fB-t, tag\fR
 .br
    Don't start FTL, show only git tag
@@ -90,6 +105,31 @@ Command line arguments
    Test resolver config file syntax
 .br

+\fBregex-test str\fR
+.br
+    Test str against all regular expressions in the database
+.br
+
+\fBregex-test str rgx\fR
+.br
+    Test str against regular expression given by rgx
+.br
+
+\fBlua\fR
+.br
+    Start the embedded Lua interpreter
+.br
+
+\fBluac\fR
+.br
+    Execute the embedded Lua compiler
+.br
+
+\fBdhcp-discover\fR
+.br
+    Discover DHCP servers in the local network
+.br
+
 \fB--\fR  (options)
 .br
    Pass options to internal dnsmasq resolver
--- a/manpages/pihole-FTL.conf.5
+++ b/manpages/pihole-FTL.conf.5
@@ -1,4 +1,4 @@
-.TH "pihole-FTL.conf" "5" "pihole-FTL.conf" "pihole-FTL.conf" "June 2018"
+.TH "pihole-FTL.conf" "5" "pihole-FTL.conf" "pihole-FTL.conf" "November 2020"
 .SH "NAME"

 pihole-FTL.conf - FTL's config file
@@ -7,49 +7,32 @@ pihole-FTL.conf - FTL's config file

 /etc/pihole/pihole-FTL.conf will be read by \fBpihole-FTL(8)\fR on startup.
 .br
-
-\fBSOCKET_LISTENING=localonly|all\fR
-.br
-    Listen only for local socket connections or permit all connections
+For each setting the option shown first is the default.
 .br

-\fBQUERY_DISPLAY=yes|no\fR
+\fBBLOCKINGMODE=IP|IP-AAAA-NODATA|NODATA|NXDOMAIN|NULL\fR
 .br
-    Display all queries? Set to no to hide query display
+    How should FTL reply to blocked queries?
+
+    IP             - Pi-hole's IPs for blocked domains
+
+    IP-AAAA-NODATA - Pi-hole's IP + NODATA-IPv6 for blocked domains
+
+    NODATA         - Using NODATA for blocked domains
+
+    NXDOMAIN       - NXDOMAIN for blocked domains
+
+    NULL           - Null IPs for blocked domains
 .br

-\fBAAAA_QUERY_ANALYSIS=yes|no\fR
+\fBCNAME_DEEP_INSPECT=true|false\fR
 .br
-    Allow FTL to analyze AAAA queries from pihole.log?
+    Use this option to disable deep CNAME inspection. This might be beneficial for very low-end devices.
 .br

-\fBRESOLVE_IPV6=yes|no\fR
+\fBBLOCK_ESNI=true|false\fR
 .br
-    Should FTL try to resolve IPv6 addresses to host names?
-.br
-
-\fBRESOLVE_IPV4=yes|no\fR
-.br
-    Should FTL try to resolve IPv4 addresses to host names?
-.br
-
-\fBMAXDBDAYS=365\fR
-.br
-    How long should queries be stored in the database?
-.br
-    Setting this to 0 disables the database
-.br
-
-\fBDBINTERVAL=1.0\fR
-.br
-    How often do we store queries in FTL's database [minutes]?
-.br
-
-\fBDBFILE=/etc/pihole/pihole-FTL.db\fR
-.br
-    Specify path and filename of FTL's SQLite long-term database.
-.br
-    Setting this to DBFILE= disables the database altogether
+    Block requests to _esni.* sub-domains.
 .br

 \fBMAXLOGAGE=24.0\fR
@@ -59,14 +42,9 @@ pihole-FTL.conf - FTL's config file
    Maximum is 744 (31 days)
 .br

-\fBFTLPORT=4711\fR
-.br
-    On which port should FTL be listening?
-.br
-
 \fBPRIVACYLEVEL=0|1|2|3|4\fR
 .br
-    Which privacy level is used?
+    Privacy level used to collect Pi-hole statistics.
 .br
    0 - show everything
 .br
@@ -84,13 +62,244 @@ pihole-FTL.conf - FTL's config file
    Should FTL ignore queries coming from the local machine?
 .br

-\fBBLOCKINGMODE=IP|IP-AAAA-NODATA|NXDOMAIN|NULL\fR
+\fBAAAA_QUERY_ANALYSIS=yes|no\fR
 .br
-    How should FTL reply to blocked queries?
+    Should FTL analyze AAAA queries?
 .br

-For each setting, the option shown first is the default.
+\fBANALYZE_ONLY_A_AND_AAAA=false|true\fR
 .br
+    Should FTL only analyze A and AAAA queries?
+.br
+
+\fBSOCKET_LISTENING=localonly|all\fR
+.br
+    Listen only for local socket connections on the API port or permit all connections.
+.br
+
+\fBFTLPORT=4711\fR
+.br
+    On which port should FTL be listening?
+.br
+
+\fBRESOLVE_IPV6=yes|no\fR
+.br
+    Should FTL try to resolve IPv6 addresses to hostnames?
+.br
+
+\fBRESOLVE_IPV4=yes|no\fR
+.br
+    Should FTL try to resolve IPv4 addresses to hostnames?
+.br
+
+\fBDELAY_STARTUP=0\fR
+.br
+    Time in seconds (between 0 and 300) to delay FTL startup.
+.br
+
+\fBNICE=-10\fR
+.br
+    Set the niceness of the Pi-hole FTL process.
+.br
+    Can be disabled altogether by setting a value of -999.
+.br
+
+\fBNAMES_FROM_NETDB=true|false\fR
+.br
+    Control whether FTL should use a fallback option and try to obtain client names from checking the network table.
+.br
+    E.g. IPv6 clients without a hostname will be compared via MAC address to known clients.
+.br
+
+\fB\fBREFRESH_HOSTNAMES=IPV4|ALL|NONE\fR
+.br
+    Change how (and if) hourly PTR requests are made to check for changes in client and upstream server hostnames:
+.br
+    IPV4 - Do the hourly PTR lookups only for IPv4 addresses resolving issues in networks with many short-lived PE IPv6 addresses.
+.br
+    ALL  - Do the hourly PTR lookups for all addresses. This can create a lot of PTR queries in networks with many IPv6 addresses.
+.br
+    NONE - Don't do hourly PTR lookups. Look up hostnames once (when first seeing a client) and never again. Future hostname changes may be missed.
+.br
+
+\fBMAXNETAGE=365\fR
+.br
+    IP addresses (and associated host names) older than the specified number of days are removed.
+.br
+    This avoids dead entries in the network overview table.
+.br
+
+\fBEDNS0_ECS=true|false\fR
+.br
+    Should we overwrite the query source when client information is provided through EDNS0 client subnet (ECS) information?
+.br
+
+\fBPARSE_ARP_CACHE=true|false\fR
+.br
+    Parse ARP cache to fill network overview table.
+.br
+
+\fBDBIMPORT=yes|no\fR
+.br
+    Should FTL load information from the database on startup to be aware of the most recent history?
+.br
+
+\fBMAXDBDAYS=365\fR
+.br
+    How long should queries be stored in the database? Setting this to 0 disables the database
+.br
+
+\fBDBINTERVAL=1.0\fR
+.br
+    How often do we store queries in FTL's database [minutes]?
+.br
+    Accepts value between 0.1 (6 sec) and 1440 (1 day)
+.br
+
+\fBDBFILE=/etc/pihole/pihole-FTL.db\fR
+.br
+    Specify path and filename of FTL's SQLite long-term database.
+.br
+    Setting this to DBFILE= disables the database altogether
+.br
+
+\fBLOGFILE=/var/log/pihole-FTL.log\fR
+.br
+    The location of FTL's log file.
+.br
+
+\fBPIDFILE=/run/pihole-FTL.pid\fR
+.br
+    The file which contains the PID of FTL's main process.
+.br
+
+\fBPORTFILE=/run/pihole-FTL.port\fR
+.br
+    Specify path and filename where the FTL process will write its API port number.
+.br
+
+\fBSOCKETFILE=/run/pihole/FTL.sock\fR
+.br
+    The file containing the socket FTL's API is listening on.
+.br
+
+\fBSETUPVARSFILE=/etc/pihole/setupVars.conf\fR
+.br
+    The config file of Pi-hole containing, e.g., the current blocking status (do not change).
+.br
+
+\fBMACVENDORDB=/etc/pihole/macvendor.db\fR
+.br
+    The database containing MAC -> Vendor information for the network table.
+.br
+
+\fBGRAVITYDB=/etc/pihole/gravity.db\fR
+.br
+    Specify path and filename of FTL's SQLite3 gravity database. This database contains all domains relevant for Pi-hole's DNS blocking.
+.br
+
+\fBDEBUG_ALL=false|true\fR
+.br
+    Enable all debug flags. If this is set to true, all other debug config options are ignored.
+.br
+
+\fBDEBUG_DATABASE=false|true\fR
+.br
+    Print debugging information about database actions such as SQL statements and performance.
+.br
+
+\fBDEBUG_NETWORKING=false|true\fR
+.br
+    Prints a list of the detected network interfaces on the startup of FTL.
+.br
+
+\fBDEBUG_LOCKS=false|true\fR
+.br
+    Print information about shared memory locks.
+.br
+    Messages will be generated when waiting, obtaining, and releasing a lock.
+.br
+
+\fBDEBUG_QUERIES=false|true\fR
+.br
+    Print extensive DNS query information (domains, types, replies, etc.).
+.br
+
+\fBDEBUG_FLAGS=false|true\fR
+.br
+    Print flags of queries received by the DNS hooks.
+.br
+    Only effective when \fBDEBUG_QUERIES\fR is enabled as well.
+
+\fBDEBUG_SHMEM=false|true\fR
+.br
+    Print information about shared memory buffers.
+.br
+    Messages are either about creating or enlarging shmem objects or string injections.
+.br
+
+\fBDEBUG_GC=false|true\fR
+.br
+    Print information about garbage collection (GC):
+.br
+    What is to be removed, how many have been removed and how long did GC take.
+.br
+
+\fBDEBUG_ARP=false|true\fR
+.br
+    Print information about ARP table processing:
+.br
+    How long did parsing take, whether read MAC addresses are valid, and if the macvendor.db file exists.
+.br
+
+\fBDEBUG_REGEX=false|true\fR
+.br
+    Controls if FTL should print extended details about regex matching.
+.br
+
+\fBDEBUG_API=false|true\fR
+.br
+    Print extra debugging information during telnet API calls.
+.br
+    Currently only used to send extra information when getting all queries.
+.br
+
+\fBDEBUG_OVERTIME=false|true\fR
+.br
+    Print information about overTime memory operations, such as initializing or moving overTime slots.
+.br
+
+\fBDEBUG_EXTBLOCKED=false|true\fR
+.br
+    Print information about why FTL decided that certain queries were recognized as being externally blocked.
+.br
+
+\fBDEBUG_CAPS=false|true\fR
+.br
+    Print information about POSIX capabilities granted to the FTL process.
+.br
+    The current capabilities are printed on receipt of SIGHUP i.e. after executing `killall -HUP pihole-FTL`.
+.br
+
+\fBDEBUG_DNSMASQ_LINES=false|true\fR
+.br
+    Print file and line causing a dnsmasq event into FTL's log files.
+.br
+    This is handy to implement additional hooks missing from FTL.
+.br
+
+\fBDEBUG_VECTORS=false|true\fR
+.br
+    FTL uses dynamically allocated vectors for various tasks.
+.br
+    This config option enables extensive debugging information such as information about allocation, referencing, deletion, and appending.
+.br
+
+\fBDEBUG_RESOLVER=false|true\fR
+.br
+    Extensive information about hostname resolution like which DNS servers are used in the first and second hostname resolving tries.
+.br
+
 .SH "SEE ALSO"

 \fBpihole\fR(8), \fBpihole-FTL\fR(8)
--- a/manpages/pihole.8
+++ b/manpages/pihole.8
@@ -139,7 +139,7 @@ Available commands and options:
      -i, interface     Specify dnsmasq's interface listening behavior
 .br
      -l, privacylevel  <level> Set privacy level
-                        (0 = lowest, 4 = highest)
+                        (0 = lowest, 3 = highest)
 .br

 \fB-c, chronometer\fR	[options]
@@ -153,7 +153,7 @@ Available commands and options:
 .br
      -r, --refresh     Set update frequency (in seconds)
 .br
-      -e, --exit        Output stats and exit witout refreshing
+      -e, --exit        Output stats and exit without refreshing
 .br

 \fB-g, updateGravity\fR
--- a/92
+++ b/92
@@ -16,6 +16,7 @@ readonly PI_HOLE_SCRIPT_DIR="/opt/pihole"
 # error due to modifying a readonly variable.
 setupVars="/etc/pihole/setupVars.conf"
 PI_HOLE_BIN_DIR="/usr/local/bin"
+readonly FTL_PID_FILE="/run/pihole-FTL.pid"

 readonly colfile="${PI_HOLE_SCRIPT_DIR}/COL_TABLE"
 source "${colfile}"
@@ -98,8 +99,25 @@ versionFunc() {
  exit 0
 }

+# Get PID of main pihole-FTL process
+getFTLPID() {
+  local pid
+
+  if [ -s "${FTL_PID_FILE}" ]; then
+    # -s: FILE exists and has a size greater than zero
+    pid="$(<"$FTL_PID_FILE")"
+    # Exploit prevention: unset the variable if there is malicious content
+    # Verify that the value read from the file is numeric
+    [[ "$pid" =~ [^[:digit:]] ]] && unset pid
+  fi
+
+  # If FTL is not running, or the PID file contains malicious stuff, substitute
+  # negative PID to signal this to the caller
+  echo "${pid:=-1}"
+}
+
 restartDNS() {
-  local svcOption svc str output status
+  local svcOption svc str output status pid icon
  svcOption="${1:-restart}"

  # Determine if we should reload or restart
@@ -108,17 +126,34 @@ restartDNS() {
    # Note 1: This will NOT re-read any *.conf files
    # Note 2: We cannot use killall here as it does
    #         not know about real-time signals
-    svc="pkill -RTMIN pihole-FTL"
-    str="Reloading DNS lists"
+    pid="$(getFTLPID)"
+    if [[ "$pid" -eq "-1" ]]; then
+      svc="true"
+      str="FTL is not running"
+      icon="${INFO}"
+    else
+      svc="kill -RTMIN ${pid}"
+      str="Reloading DNS lists"
+      icon="${TICK}"
+    fi
  elif [[ "${svcOption}" =~ "reload" ]]; then
    # Reloading of the DNS cache has been requested
    # Note: This will NOT re-read any *.conf files
-    svc="pkill -HUP pihole-FTL"
-    str="Flushing DNS cache"
+    pid="$(getFTLPID)"
+    if [[ "$pid" -eq "-1" ]]; then
+      svc="true"
+      str="FTL is not running"
+      icon="${INFO}"
+    else
+      svc="kill -HUP ${pid}"
+      str="Flushing DNS cache"
+      icon="${TICK}"
+    fi
  else
    # A full restart has been requested
    svc="service pihole-FTL restart"
    str="Restarting DNS server"
+    icon="${TICK}"
  fi

  # Print output to Terminal, but not to Web Admin
@@ -128,7 +163,7 @@ restartDNS() {
  status="$?"

  if [[ "${status}" -eq 0 ]]; then
-    [[ -t 1 ]] && echo -e "${OVER}  ${TICK} ${str}"
+    [[ -t 1 ]] && echo -e "${OVER}  ${icon} ${str}"
    return 0
  else
    [[ ! -t 1 ]] && local OVER=""
@@ -249,16 +284,47 @@ Options:
  echo -e "${OVER}  ${TICK} ${str}"
 }

+analyze_ports() {
+  # FTL is listening at least on at least one port when this
+  # function is getting called
+  echo -e "  ${TICK} DNS service is listening"
+  # Check individual address family/protocol combinations
+  # For a healthy Pi-hole, they should all be up (nothing printed)
+  if grep -q "IPv4.*UDP" <<< "${1}"; then
+      echo -e "     ${TICK} UDP (IPv4)"
+  else
+      echo -e "     ${CROSS} UDP (IPv4)"
+  fi
+  if grep -q "IPv4.*TCP" <<< "${1}"; then
+      echo -e "     ${TICK} TCP (IPv4)"
+  else
+      echo -e "     ${CROSS} TCP (IPv4)"
+  fi
+  if grep -q "IPv6.*UDP" <<< "${1}"; then
+      echo -e "     ${TICK} UDP (IPv6)"
+  else
+      echo -e "     ${CROSS} UDP (IPv6)"
+  fi
+  if grep -q "IPv6.*TCP" <<< "${1}"; then
+      echo -e "     ${TICK} TCP (IPv6)"
+  else
+      echo -e "     ${CROSS} TCP (IPv6)"
+  fi
+  echo ""
+}
+
 statusFunc() {
-  # Determine if service is running on port 53 (Cr: https://superuser.com/a/806331)
-  if (echo > /dev/tcp/127.0.0.1/53) >/dev/null 2>&1; then
+  # Determine if there is a pihole service is listening on port 53
+  local listening
+  listening="$(lsof -Pni:53)"
+  if grep -q "pihole" <<< "${listening}"; then
    if [[ "${1}" != "web" ]]; then
-      echo -e "  ${TICK} DNS service is running"
+      analyze_ports "${listening}"
    fi
  else
    case "${1}" in
      "web") echo "-1";;
-      *) echo -e "  ${CROSS} DNS service is NOT running";;
+      *) echo -e "  ${CROSS} DNS service is NOT listening";;
    esac
    return 0
  fi
@@ -268,13 +334,13 @@ statusFunc() {
    # A config is commented out
    case "${1}" in
      "web") echo 0;;
-      *) echo -e "  ${CROSS} Pi-hole blocking is Disabled";;
+      *) echo -e "  ${CROSS} Pi-hole blocking is disabled";;
    esac
  elif grep -q "BLOCKING_ENABLED=true" /etc/pihole/setupVars.conf;  then
    # Configs are set
    case "${1}" in
      "web") echo 1;;
-      *) echo -e "  ${TICK} Pi-hole blocking is Enabled";;
+      *) echo -e "  ${TICK} Pi-hole blocking is enabled";;
    esac
  else
    # No configs were found
@@ -387,7 +453,7 @@ Whitelist/Blacklist Options:

 Debugging Options:
  -d, debug           Start a debugging session
-                        Add '-a' to enable automated debugging
+                        Add '-a' to automatically upload the log to tricorder.pi-hole.net
  -f, flush           Flush the Pi-hole log
  -r, reconfigure     Reconfigure or Repair Pi-hole subsystems
  -t, tail            View the live output of the Pi-hole log
--- a/supportedos.txt
+++ b/supportedos.txt
@@ -1,5 +1,5 @@
 Raspbian=9,10
 Ubuntu=16,18,20
 Debian=9,10
-Fedora=31,32
+Fedora=32,33
 CentOS=7,8
--- a/test/_centos_7.Dockerfile
+++ b/test/_centos_7.Dockerfile
--- a/test/_centos_8.Dockerfile
+++ b/test/_centos_8.Dockerfile
@@ -1,4 +1,4 @@
-FROM fedora:30
+FROM centos:8

 ENV GITDIR /etc/.pihole
 ENV SCRIPTDIR /opt/pihole
--- a/test/_debian_10.Dockerfile
+++ b/test/_debian_10.Dockerfile
@@ -1,4 +1,4 @@
-FROM buildpack-deps:jessie-scm
+FROM buildpack-deps:buster-scm

 ENV GITDIR /etc/.pihole
 ENV SCRIPTDIR /opt/pihole
--- a/test/_debian_9.Dockerfile
+++ b/test/_debian_9.Dockerfile
@@ -0,0 +1,16 @@
+FROM buildpack-deps:stretch-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \
--- a/test/_fedora_32.Dockerfile
+++ b/test/_fedora_32.Dockerfile
@@ -0,0 +1,16 @@
+FROM fedora:32
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \
--- a/test/_fedora_33.Dockerfile
+++ b/test/_fedora_33.Dockerfile
@@ -0,0 +1,16 @@
+FROM fedora:33
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \
--- a/test/_ubuntu_16.Dockerfile
+++ b/test/_ubuntu_16.Dockerfile
@@ -0,0 +1,16 @@
+FROM buildpack-deps:xenial-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \
--- a/test/_ubuntu_18.Dockerfile
+++ b/test/_ubuntu_18.Dockerfile
@@ -0,0 +1,16 @@
+FROM buildpack-deps:bionic-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \
--- a/test/_ubuntu_20.Dockerfile
+++ b/test/_ubuntu_20.Dockerfile
@@ -0,0 +1,17 @@
+FROM buildpack-deps:focal-scm
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \
--- a/test/conftest.py
+++ b/test/conftest.py
@@ -69,7 +69,9 @@ def args(request):
    return '-t -d'


-@pytest.fixture(params=['debian', 'centos', 'fedora'])
+@pytest.fixture(params=[
+    'test_container'
+])
 def tag(request):
    '''
    consumed by image to make the test matrix
--- a/test/requirements.txt
+++ b/test/requirements.txt
--- a/test/setup.py
+++ b/test/setup.py
--- a/test/test_000_build_containers.py
+++ b/test/test_000_build_containers.py
@@ -1,23 +0,0 @@
-''' This file starts with 000 to make it run first '''
-import pytest
-import testinfra
-
-run_local = testinfra.get_backend(
-    "local://"
-).get_module("Command").run
-
-
-@pytest.mark.parametrize("image,tag", [
-    ('test/debian.Dockerfile', 'pytest_pihole:debian'),
-    ('test/centos.Dockerfile', 'pytest_pihole:centos'),
-    ('test/fedora.Dockerfile', 'pytest_pihole:fedora'),
-])
-# mark as 'build_stage' so we can ensure images are build first when tests
-# are executed in parallel. (not required when tests are executed serially)
-@pytest.mark.build_stage
-def test_build_pihole_image(image, tag):
-    build_cmd = run_local('docker build -f {} -t {} .'.format(image, tag))
-    if build_cmd.rc != 0:
-        print(build_cmd.stdout)
-        print(build_cmd.stderr)
-    assert build_cmd.rc == 0
--- a/test/test_automated_install.py
+++ b/test/test_automated_install.py
@@ -187,7 +187,55 @@ def test_FTL_detect_aarch64_no_errors(Pihole):
    ''')
    expected_stdout = info_box + ' FTL Checks...'
    assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + ' Detected ARM-aarch64 architecture'
+    expected_stdout = tick_box + ' Detected AArch64 (64 Bit ARM) processor'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Downloading and Installing FTL'
+    assert expected_stdout in detectPlatform.stdout
+
+
+def test_FTL_detect_armv4t_no_errors(Pihole):
+    '''
+    confirms only armv4t package is downloaded for FTL engine
+    '''
+    # mock uname to return armv4t platform
+    mock_command('uname', {'-m': ('armv4t', '0')}, Pihole)
+    # mock ldd to respond with ld-linux shared library
+    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    create_pihole_user
+    funcOutput=$(get_binary_name)
+    binary="pihole-FTL${funcOutput##*pihole-FTL}"
+    theRest="${funcOutput%pihole-FTL*}"
+    FTLdetect "${binary}" "${theRest}"
+    ''')
+    expected_stdout = info_box + ' FTL Checks...'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + (' Detected ARMv4 processor')
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Downloading and Installing FTL'
+    assert expected_stdout in detectPlatform.stdout
+
+
+def test_FTL_detect_armv5te_no_errors(Pihole):
+    '''
+    confirms only armv5te package is downloaded for FTL engine
+    '''
+    # mock uname to return armv5te platform
+    mock_command('uname', {'-m': ('armv5te', '0')}, Pihole)
+    # mock ldd to respond with ld-linux shared library
+    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    create_pihole_user
+    funcOutput=$(get_binary_name)
+    binary="pihole-FTL${funcOutput##*pihole-FTL}"
+    theRest="${funcOutput%pihole-FTL*}"
+    FTLdetect "${binary}" "${theRest}"
+    ''')
+    expected_stdout = info_box + ' FTL Checks...'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + (' Detected ARMv5 (or newer) processor')
    assert expected_stdout in detectPlatform.stdout
    expected_stdout = tick_box + ' Downloading and Installing FTL'
    assert expected_stdout in detectPlatform.stdout
@@ -199,7 +247,7 @@ def test_FTL_detect_armv6l_no_errors(Pihole):
    '''
    # mock uname to return armv6l platform
    mock_command('uname', {'-m': ('armv6l', '0')}, Pihole)
-    # mock ldd to respond with aarch64 shared library
+    # mock ldd to respond with ld-linux-armhf shared library
    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
    detectPlatform = Pihole.run('''
    source /opt/pihole/basic-install.sh
@@ -211,8 +259,8 @@ def test_FTL_detect_armv6l_no_errors(Pihole):
    ''')
    expected_stdout = info_box + ' FTL Checks...'
    assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + (' Detected ARM-hf architecture '
-                                  '(armv6 or lower)')
+    expected_stdout = tick_box + (' Detected ARMv6 processor '
+                                  '(with hard-float support)')
    assert expected_stdout in detectPlatform.stdout
    expected_stdout = tick_box + ' Downloading and Installing FTL'
    assert expected_stdout in detectPlatform.stdout
@@ -224,7 +272,7 @@ def test_FTL_detect_armv7l_no_errors(Pihole):
    '''
    # mock uname to return armv7l platform
    mock_command('uname', {'-m': ('armv7l', '0')}, Pihole)
-    # mock ldd to respond with aarch64 shared library
+    # mock ldd to respond with ld-linux-armhf shared library
    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
    detectPlatform = Pihole.run('''
    source /opt/pihole/basic-install.sh
@@ -236,7 +284,32 @@ def test_FTL_detect_armv7l_no_errors(Pihole):
    ''')
    expected_stdout = info_box + ' FTL Checks...'
    assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + ' Detected ARM-hf architecture (armv7+)'
+    expected_stdout = tick_box + (' Detected ARMv7 processor '
+                                  '(with hard-float support)')
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Downloading and Installing FTL'
+    assert expected_stdout in detectPlatform.stdout
+
+
+def test_FTL_detect_armv8a_no_errors(Pihole):
+    '''
+    confirms only armv8a package is downloaded for FTL engine
+    '''
+    # mock uname to return armv8a platform
+    mock_command('uname', {'-m': ('armv8a', '0')}, Pihole)
+    # mock ldd to respond with ld-linux-armhf shared library
+    mock_command('ldd', {'/bin/ls': ('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    create_pihole_user
+    funcOutput=$(get_binary_name)
+    binary="pihole-FTL${funcOutput##*pihole-FTL}"
+    theRest="${funcOutput%pihole-FTL*}"
+    FTLdetect "${binary}" "${theRest}"
+    ''')
+    expected_stdout = info_box + ' FTL Checks...'
+    assert expected_stdout in detectPlatform.stdout
+    expected_stdout = tick_box + ' Detected ARMv8 (or newer) processor'
    assert expected_stdout in detectPlatform.stdout
    expected_stdout = tick_box + ' Downloading and Installing FTL'
    assert expected_stdout in detectPlatform.stdout
@@ -256,7 +329,7 @@ def test_FTL_detect_x86_64_no_errors(Pihole):
    ''')
    expected_stdout = info_box + ' FTL Checks...'
    assert expected_stdout in detectPlatform.stdout
-    expected_stdout = tick_box + ' Detected x86_64 architecture'
+    expected_stdout = tick_box + ' Detected x86_64 processor'
    assert expected_stdout in detectPlatform.stdout
    expected_stdout = tick_box + ' Downloading and Installing FTL'
    assert expected_stdout in detectPlatform.stdout
@@ -274,7 +347,7 @@ def test_FTL_detect_unknown_no_errors(Pihole):
    theRest="${funcOutput%pihole-FTL*}"
    FTLdetect "${binary}" "${theRest}"
    ''')
-    expected_stdout = 'Not able to detect architecture (unknown: mips)'
+    expected_stdout = 'Not able to detect processor (unknown: mips)'
    assert expected_stdout in detectPlatform.stdout


@@ -451,40 +524,76 @@ def test_IPv6_ULA_GUA_test(Pihole):
    assert expected_stdout in detectPlatform.stdout


-def test_validate_ip_valid(Pihole):
+def test_validate_ip(Pihole):
    '''
-    Given a valid IP address, valid_ip returns success
+    Tests valid_ip for various IP addresses
    '''

-    output = Pihole.run('''
+    def test_address(addr, success=True):
+        output = Pihole.run('''
+        source /opt/pihole/basic-install.sh
+        valid_ip "{addr}"
+        '''.format(addr=addr))
+
+        assert output.rc == 0 if success else 1
+
+    test_address('192.168.1.1')
+    test_address('127.0.0.1')
+    test_address('255.255.255.255')
+    test_address('255.255.255.256', False)
+    test_address('255.255.256.255', False)
+    test_address('255.256.255.255', False)
+    test_address('256.255.255.255', False)
+    test_address('1092.168.1.1', False)
+    test_address('not an IP', False)
+    test_address('8.8.8.8#', False)
+    test_address('8.8.8.8#0')
+    test_address('8.8.8.8#1')
+    test_address('8.8.8.8#42')
+    test_address('8.8.8.8#888')
+    test_address('8.8.8.8#1337')
+    test_address('8.8.8.8#65535')
+    test_address('8.8.8.8#65536', False)
+    test_address('8.8.8.8#-1', False)
+    test_address('00.0.0.0', False)
+    test_address('010.0.0.0', False)
+    test_address('001.0.0.0', False)
+    test_address('0.0.0.0#00', False)
+    test_address('0.0.0.0#01', False)
+    test_address('0.0.0.0#001', False)
+    test_address('0.0.0.0#0001', False)
+    test_address('0.0.0.0#00001', False)
+
+
+def test_os_check_fails(Pihole):
+    ''' Confirms install fails on unsupported OS '''
+    Pihole.run('''
    source /opt/pihole/basic-install.sh
-    valid_ip "192.168.1.1"
+    distro_check
+    install_dependent_packages ${INSTALLER_DEPS[@]}
+    cat <<EOT > /etc/os-release
+    ID=UnsupportedOS
+    VERSION_ID="2"
+    EOT
    ''')
-
-    assert output.rc == 0
-
-
-def test_validate_ip_invalid_octet(Pihole):
-    '''
-    Given an invalid IP address (large octet), valid_ip returns an error
-    '''
-
-    output = Pihole.run('''
+    detectOS = Pihole.run('''t
    source /opt/pihole/basic-install.sh
-    valid_ip "1092.168.1.1"
+    os_check
    ''')
-
-    assert output.rc == 1
+    expected_stdout = 'Unsupported OS detected: UnsupportedOS'
+    assert expected_stdout in detectOS.stdout


-def test_validate_ip_invalid_letters(Pihole):
-    '''
-    Given an invalid IP address (contains letters), valid_ip returns an error
-    '''
-
-    output = Pihole.run('''
+def test_os_check_passes(Pihole):
+    ''' Confirms OS meets the requirements '''
+    Pihole.run('''
    source /opt/pihole/basic-install.sh
-    valid_ip "not an IP"
+    distro_check
+    install_dependent_packages ${INSTALLER_DEPS[@]}
    ''')
-
-    assert output.rc == 1
+    detectOS = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    os_check
+    ''')
+    expected_stdout = 'Supported OS detected'
+    assert expected_stdout in detectOS.stdout
--- a/test/test_centos_7_support.py
+++ b/test/test_centos_7_support.py
@@ -0,0 +1,60 @@
+from .conftest import (
+    tick_box,
+    info_box,
+    mock_command,
+)
+
+
+def test_php_upgrade_default_optout_centos_eq_7(Pihole):
+    '''
+    confirms the default behavior to opt-out of installing PHP7 from REMI
+    '''
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in distro_check.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optout_centos_eq_7(Pihole):
+    '''
+    confirms installer behavior when user opt-out of installing PHP7 from REMI
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*': ('', '1')}, Pihole)
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in distro_check.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optin_centos_eq_7(Pihole):
+    '''
+    confirms installer behavior when user opt-in to installing PHP7 from REMI
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Continue for user prompt
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    assert 'opt-out' not in distro_check.stdout
+    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
+                                  '(https://rpms.remirepo.net)')
+    assert expected_stdout in distro_check.stdout
+    expected_stdout = tick_box + (' Remi\'s RPM repository has '
+                                  'been enabled for PHP7')
+    assert expected_stdout in distro_check.stdout
+    remi_package = Pihole.package('remi-release')
+    assert remi_package.is_installed
--- a/test/test_centos_8_support.py
+++ b/test/test_centos_8_support.py
@@ -0,0 +1,65 @@
+from .conftest import (
+    tick_box,
+    info_box,
+    mock_command,
+)
+
+
+def test_php_upgrade_default_continue_centos_gte_8(Pihole):
+    '''
+    confirms the latest version of CentOS continues / does not optout
+    (should trigger on CentOS7 only)
+    '''
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    unexpected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS.'
+                                    ' Deprecated PHP may be in use.')
+    assert unexpected_stdout not in distro_check.stdout
+    # ensure remi was not installed on latest CentOS
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optout_skipped_centos_gte_8(Pihole):
+    '''
+    confirms installer skips user opt-out of installing PHP7 from REMI on
+    latest CentOS (should trigger on CentOS7 only)
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*': ('', '1')}, Pihole)
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    unexpected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS.'
+                                    ' Deprecated PHP may be in use.')
+    assert unexpected_stdout not in distro_check.stdout
+    # ensure remi was not installed on latest CentOS
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_upgrade_user_optin_skipped_centos_gte_8(Pihole):
+    '''
+    confirms installer skips user opt-in to installing PHP7 from REMI on
+    latest CentOS (should trigger on CentOS7 only)
+    (php not currently installed)
+    '''
+    # Whiptail dialog returns Continue for user prompt
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    assert 'opt-out' not in distro_check.stdout
+    unexpected_stdout = info_box + (' Enabling Remi\'s RPM repository '
+                                    '(https://rpms.remirepo.net)')
+    assert unexpected_stdout not in distro_check.stdout
+    unexpected_stdout = tick_box + (' Remi\'s RPM repository has '
+                                    'been enabled for PHP7')
+    assert unexpected_stdout not in distro_check.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
--- a/test/test_centos_common_support.py
+++ b/test/test_centos_common_support.py
@@ -0,0 +1,120 @@
+import pytest
+from .conftest import (
+    tick_box,
+    info_box,
+    cross_box,
+    mock_command,
+)
+
+
+def test_release_supported_version_check_centos(Pihole):
+    '''
+    confirms installer exits on unsupported releases of CentOS
+    '''
+    # modify /etc/redhat-release to mock an unsupported CentOS release
+    Pihole.run('echo "CentOS Linux release 6.9" > /etc/redhat-release')
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    expected_stdout = cross_box + (' CentOS 6 is not supported.')
+    assert expected_stdout in distro_check.stdout
+    expected_stdout = 'Please update to CentOS release 7 or later'
+    assert expected_stdout in distro_check.stdout
+
+
+def test_enable_epel_repository_centos(Pihole):
+    '''
+    confirms the EPEL package repository is enabled when installed on CentOS
+    '''
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    expected_stdout = info_box + (' Enabling EPEL package repository '
+                                  '(https://fedoraproject.org/wiki/EPEL)')
+    assert expected_stdout in distro_check.stdout
+    expected_stdout = tick_box + ' Installed epel-release'
+    assert expected_stdout in distro_check.stdout
+    epel_package = Pihole.package('epel-release')
+    assert epel_package.is_installed
+
+
+def test_php_version_lt_7_detected_upgrade_default_optout_centos(Pihole):
+    '''
+    confirms the default behavior to opt-out of upgrading to PHP7 from REMI
+    '''
+    # first we will install the default php version to test installer behavior
+    php_install = Pihole.run('yum install -y php')
+    assert php_install.rc == 0
+    php_package = Pihole.package('php')
+    default_centos_php_version = php_package.version.split('.')[0]
+    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
+        pytest.skip("Test deprecated . Detected default PHP version >= 7")
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in distro_check.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_version_lt_7_detected_upgrade_user_optout_centos(Pihole):
+    '''
+    confirms installer behavior when user opt-out to upgrade to PHP7 via REMI
+    '''
+    # first we will install the default php version to test installer behavior
+    php_install = Pihole.run('yum install -y php')
+    assert php_install.rc == 0
+    php_package = Pihole.package('php')
+    default_centos_php_version = php_package.version.split('.')[0]
+    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
+        pytest.skip("Test deprecated . Detected default PHP version >= 7")
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*': ('', '1')}, Pihole)
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout in distro_check.stdout
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
+
+
+def test_php_version_lt_7_detected_upgrade_user_optin_centos(Pihole):
+    '''
+    confirms installer behavior when user opt-in to upgrade to PHP7 via REMI
+    '''
+    # first we will install the default php version to test installer behavior
+    php_install = Pihole.run('yum install -y php')
+    assert php_install.rc == 0
+    php_package = Pihole.package('php')
+    default_centos_php_version = php_package.version.split('.')[0]
+    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
+        pytest.skip("Test deprecated . Detected default PHP version >= 7")
+    # Whiptail dialog returns Continue for user prompt
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    install_dependent_packages PIHOLE_WEB_DEPS[@]
+    ''')
+    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
+                                  'Deprecated PHP may be in use.')
+    assert expected_stdout not in distro_check.stdout
+    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
+                                  '(https://rpms.remirepo.net)')
+    assert expected_stdout in distro_check.stdout
+    expected_stdout = tick_box + (' Remi\'s RPM repository has '
+                                  'been enabled for PHP7')
+    assert expected_stdout in distro_check.stdout
+    remi_package = Pihole.package('remi-release')
+    assert remi_package.is_installed
+    updated_php_package = Pihole.package('php')
+    updated_php_version = updated_php_package.version.split('.')[0]
+    assert int(updated_php_version) == 7
--- a/test/test_centos_fedora_common_support.py
+++ b/test/test_centos_fedora_common_support.py
@@ -0,0 +1,65 @@
+from .conftest import (
+    tick_box,
+    cross_box,
+    mock_command,
+)
+
+
+def mock_selinux_config(state, Pihole):
+    '''
+    Creates a mock SELinux config file with expected content
+    '''
+    # validate state string
+    valid_states = ['enforcing', 'permissive', 'disabled']
+    assert state in valid_states
+    # getenforce returns the running state of SELinux
+    mock_command('getenforce', {'*': (state.capitalize(), '0')}, Pihole)
+    # create mock configuration with desired content
+    Pihole.run('''
+    mkdir /etc/selinux
+    echo "SELINUX={state}" > /etc/selinux/config
+    '''.format(state=state.lower()))
+
+
+def test_selinux_enforcing_exit(Pihole):
+    '''
+    confirms installer prompts to exit when SELinux is Enforcing by default
+    '''
+    mock_selinux_config("enforcing", Pihole)
+    check_selinux = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    checkSelinux
+    ''')
+    expected_stdout = cross_box + ' Current SELinux: Enforcing'
+    assert expected_stdout in check_selinux.stdout
+    expected_stdout = 'SELinux Enforcing detected, exiting installer'
+    assert expected_stdout in check_selinux.stdout
+    assert check_selinux.rc == 1
+
+
+def test_selinux_permissive(Pihole):
+    '''
+    confirms installer continues when SELinux is Permissive
+    '''
+    mock_selinux_config("permissive", Pihole)
+    check_selinux = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    checkSelinux
+    ''')
+    expected_stdout = tick_box + ' Current SELinux: Permissive'
+    assert expected_stdout in check_selinux.stdout
+    assert check_selinux.rc == 0
+
+
+def test_selinux_disabled(Pihole):
+    '''
+    confirms installer continues when SELinux is Disabled
+    '''
+    mock_selinux_config("disabled", Pihole)
+    check_selinux = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    checkSelinux
+    ''')
+    expected_stdout = tick_box + ' Current SELinux: Disabled'
+    assert expected_stdout in check_selinux.stdout
+    assert check_selinux.rc == 0
--- a/test/test_centos_fedora_support.py
+++ b/test/test_centos_fedora_support.py
@@ -1,264 +0,0 @@
-import pytest
-from .conftest import (
-    tick_box,
-    info_box,
-    cross_box,
-    mock_command,
-)
-
-
-def mock_selinux_config(state, Pihole):
-    '''
-    Creates a mock SELinux config file with expected content
-    '''
-    # validate state string
-    valid_states = ['enforcing', 'permissive', 'disabled']
-    assert state in valid_states
-    # getenforce returns the running state of SELinux
-    mock_command('getenforce', {'*': (state.capitalize(), '0')}, Pihole)
-    # create mock configuration with desired content
-    Pihole.run('''
-    mkdir /etc/selinux
-    echo "SELINUX={state}" > /etc/selinux/config
-    '''.format(state=state.lower()))
-
-
-@pytest.mark.parametrize("tag", [('centos'), ('fedora'), ])
-def test_selinux_enforcing_exit(Pihole):
-    '''
-    confirms installer prompts to exit when SELinux is Enforcing by default
-    '''
-    mock_selinux_config("enforcing", Pihole)
-    check_selinux = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    checkSelinux
-    ''')
-    expected_stdout = cross_box + ' Current SELinux: Enforcing'
-    assert expected_stdout in check_selinux.stdout
-    expected_stdout = 'SELinux Enforcing detected, exiting installer'
-    assert expected_stdout in check_selinux.stdout
-    assert check_selinux.rc == 1
-
-
-@pytest.mark.parametrize("tag", [('centos'), ('fedora'), ])
-def test_selinux_permissive(Pihole):
-    '''
-    confirms installer continues when SELinux is Permissive
-    '''
-    mock_selinux_config("permissive", Pihole)
-    check_selinux = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    checkSelinux
-    ''')
-    expected_stdout = tick_box + ' Current SELinux: Permissive'
-    assert expected_stdout in check_selinux.stdout
-    assert check_selinux.rc == 0
-
-
-@pytest.mark.parametrize("tag", [('centos'), ('fedora'), ])
-def test_selinux_disabled(Pihole):
-    '''
-    confirms installer continues when SELinux is Disabled
-    '''
-    mock_selinux_config("disabled", Pihole)
-    check_selinux = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    checkSelinux
-    ''')
-    expected_stdout = tick_box + ' Current SELinux: Disabled'
-    assert expected_stdout in check_selinux.stdout
-    assert check_selinux.rc == 0
-
-
-@pytest.mark.parametrize("tag", [('fedora'), ])
-def test_epel_and_remi_not_installed_fedora(Pihole):
-    '''
-    confirms installer does not attempt to install EPEL/REMI repositories
-    on Fedora
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    assert distro_check.stdout == ''
-
-    epel_package = Pihole.package('epel-release')
-    assert not epel_package.is_installed
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_release_supported_version_check_centos(Pihole):
-    '''
-    confirms installer exits on unsupported releases of CentOS
-    '''
-    # modify /etc/redhat-release to mock an unsupported CentOS release
-    Pihole.run('echo "CentOS Linux release 6.9" > /etc/redhat-release')
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = cross_box + (' CentOS 6 is not supported.')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = 'Please update to CentOS release 7 or later'
-    assert expected_stdout in distro_check.stdout
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_enable_epel_repository_centos(Pihole):
-    '''
-    confirms the EPEL package repository is enabled when installed on CentOS
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' Enabling EPEL package repository '
-                                  '(https://fedoraproject.org/wiki/EPEL)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + ' Installed epel-release'
-    assert expected_stdout in distro_check.stdout
-    epel_package = Pihole.package('epel-release')
-    assert epel_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_upgrade_default_optout_centos(Pihole):
-    '''
-    confirms the default behavior to opt-out of installing PHP7 from REMI
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_upgrade_user_optout_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-out of installing PHP7 from REMI
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_upgrade_user_optin_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-in to installing PHP7 from REMI
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Continue for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    assert 'opt-out' not in distro_check.stdout
-    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
-                                  '(https://rpms.remirepo.net)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + (' Remi\'s RPM repository has '
-                                  'been enabled for PHP7')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_version_lt_7_detected_upgrade_default_optout_centos(Pihole):
-    '''
-    confirms the default behavior to opt-out of upgrading to PHP7 from REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_version_lt_7_detected_upgrade_user_optout_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-out to upgrade to PHP7 via REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-@pytest.mark.parametrize("tag", [('centos'), ])
-def test_php_version_lt_7_detected_upgrade_user_optin_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-in to upgrade to PHP7 via REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    # Whiptail dialog returns Continue for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    install_dependent_packages PIHOLE_WEB_DEPS[@]
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout not in distro_check.stdout
-    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
-                                  '(https://rpms.remirepo.net)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + (' Remi\'s RPM repository has '
-                                  'been enabled for PHP7')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert remi_package.is_installed
-    updated_php_package = Pihole.package('php')
-    updated_php_version = updated_php_package.version.split('.')[0]
-    assert int(updated_php_version) == 7
--- a/test/test_fedora_support.py
+++ b/test/test_fedora_support.py
@@ -0,0 +1,15 @@
+def test_epel_and_remi_not_installed_fedora(Pihole):
+    '''
+    confirms installer does not attempt to install EPEL/REMI repositories
+    on Fedora
+    '''
+    distro_check = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    ''')
+    assert distro_check.stdout == ''
+
+    epel_package = Pihole.package('epel-release')
+    assert not epel_package.is_installed
+    remi_package = Pihole.package('remi-release')
+    assert not remi_package.is_installed
--- a/test/test_shellcheck.py
+++ b/test/test_shellcheck.py
@@ -1,18 +0,0 @@
-import testinfra
-
-run_local = testinfra.get_backend(
-    "local://"
-).get_module("Command").run
-
-
-def test_scripts_pass_shellcheck():
-    '''
-    Make sure shellcheck does not find anything wrong with our shell scripts
-    '''
-    shellcheck = ("find . -type f -name 'update.sh' "
-                  "| while read file; do "
-                  "shellcheck -x \"$file\" -e SC1090,SC1091; "
-                  "done;")
-    results = run_local(shellcheck)
-    print(results.stdout)
-    assert '' == results.stdout
--- a/test/tox.centos_7.ini
+++ b/test/tox.centos_7.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _centos_7.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_common_support.py ./test_centos_7_support.py
--- a/test/tox.centos_8.ini
+++ b/test/tox.centos_8.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _centos_8.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_common_support.py ./test_centos_8_support.py
--- a/test/tox.debian_10.ini
+++ b/test/tox.debian_10.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _debian_10.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py
--- a/test/tox.debian_9.ini
+++ b/test/tox.debian_9.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _debian_9.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py
--- a/test/tox.fedora_32.ini
+++ b/test/tox.fedora_32.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _fedora_32.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py
--- a/test/tox.fedora_33.ini
+++ b/test/tox.fedora_33.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _fedora_33.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py
--- a/test/tox.ubuntu_16.ini
+++ b/test/tox.ubuntu_16.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _ubuntu_16.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py
--- a/test/tox.ubuntu_18.ini
+++ b/test/tox.ubuntu_18.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _ubuntu_18.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py
--- a/test/tox.ubuntu_20.ini
+++ b/test/tox.ubuntu_20.ini
@@ -0,0 +1,8 @@
+[tox]
+envlist = py37
+
+[testenv]
+whitelist_externals = docker
+deps = -rrequirements.txt
+commands = docker build -f _ubuntu_20.Dockerfile -t pytest_pihole:test_container ../
+           pytest {posargs:-vv -n auto} ./test_automated_install.py
--- a/tox.ini
+++ b/tox.ini
@@ -1,10 +0,0 @@
-[tox]
-envlist = py36
-
-[testenv]
-whitelist_externals = docker
-deps = -rrequirements.txt
-commands = docker build -f test/debian.Dockerfile -t pytest_pihole:debian .
-           docker build -f test/centos.Dockerfile -t pytest_pihole:centos .
-           docker build -f test/fedora.Dockerfile -t pytest_pihole:fedora .
-           pytest {posargs:-vv -n auto} -m "not build_stage" ./test/