Merge pull request #4114 from MichaIng/patch-3

Remove obsolete iproute APT package check

.github/workflows/test.yml

@@ -10,7 +10,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        distro: [debian_9, debian_10, ubuntu_16, ubuntu_18, ubuntu_20, centos_7, centos_8, fedora_31, fedora_32]
+        distro: [debian_9, debian_10, ubuntu_16, ubuntu_18, ubuntu_20, centos_7, centos_8, fedora_32, fedora_33]
     env:
       DISTRO: ${{matrix.distro}}
     steps:

README.md

@@ -1,5 +1,7 @@
 <!-- markdownlint-configure-file { "MD004": { "style": "consistent" } } -->
 <!-- markdownlint-disable MD033 -->
+#
+
 <p align="center">
     <a href="https://pi-hole.net/">
         <img src="https://pi-hole.github.io/graphics/Vortex/Vortex_with_Wordmark.svg" width="150" height="260" alt="Pi-hole">
@@ -9,8 +11,6 @@
 </p>
 <!-- markdownlint-enable MD033 -->
 
-#
-
 The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) that protects your devices from unwanted content, without installing any client-side software.
 
 - **Easy-to-install**: our versatile installer walks you through the process, and takes less than ten minutes
@@ -26,8 +26,6 @@ The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) th
 
 -----
 
-Master [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=master)](https://travis-ci.com/pi-hole/pi-hole) Development [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=development)](https://travis-ci.com/pi-hole/pi-hole)
-
 ## One-Step Automated Install
 
 Those who want to get started quickly and conveniently may install Pi-hole using the following command:
@@ -52,6 +50,8 @@ sudo bash basic-install.sh
 wget -O basic-install.sh https://install.pi-hole.net
 sudo bash basic-install.sh
 ```
+### Method 3: Using Docker to deploy Pi-hole
+Please refer to the [Pi-hole docker repo](https://github.com/pi-hole/docker-pi-hole) to use the Official Docker Images.
 
 ## [Post-install: Make your network take advantage of Pi-hole](https://docs.pi-hole.net/main/post-install/)
 
@@ -71,16 +71,18 @@ Make no mistake: **your support is absolutely vital to help keep us innovating!*
 
 ### [Donations](https://pi-hole.net/donate)
 
-Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses:
+Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses and rewarding our dedicated development team:
 
 ### Alternative support
 
 If you'd rather not donate (_which is okay!_), there are other ways you can help support us:
 
-- [Patreon](https://patreon.com/pihole) _Become a patron for rewards_
+- [GitHub Sponsors](https://github.com/sponsors/pi-hole/)
+- [Patreon](https://patreon.com/pihole)
+- [Hetzner Cloud](https://hetzner.cloud/?ref=7aceisRX3AzA) _affiliate link_
 - [Digital Ocean](https://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
 - [Stickermule](https://www.stickermule.com/unlock?ref_id=9127301701&utm_medium=link&utm_source=invite) _earn a $10 credit after your first purchase_
-- [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
+- [Amazon US](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
 - Spreading the word about our software, and how you have benefited from it
 
 ### Contributing via GitHub
@@ -95,7 +97,9 @@ You'll find that the [install script](https://github.com/pi-hole/pi-hole/blob/ma
 
 ## Getting in touch with us
 
-While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets. **Please be sure to check the FAQ's** before starting a new discussion, as we do not have the spare time to reply to every request for assistance.
+While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets.
+
+**Please be sure to check the FAQ's** before starting a new discussion. Many user questions already have answers and can be solved without any additional assistance.
 
 - [Frequently Asked Questions](https://discourse.pi-hole.net/c/faqs)
 - [Feature Requests](https://discourse.pi-hole.net/c/feature-requests?order=votes)
@@ -106,12 +110,27 @@ While we are primarily reachable on our [Discourse User Forum](https://discourse
 
 ## Breakdown of Features
 
+### [Faster-than-light Engine](https://github.com/pi-hole/ftl)
+
+[FTLDNS](https://github.com/pi-hole/ftl) is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
+
+Some of the statistics you can integrate include:
+
+- Total number of domains being blocked
+- Total number of DNS queries today
+- Total number of ads blocked today
+- Percentage of ads blocked
+- Unique domains
+- Queries forwarded (to your chosen upstream DNS server)
+- Queries cached
+- Unique clients
+
+The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).
+
 ### The Command Line Interface
 
 The [pihole](https://docs.pi-hole.net/core/pihole-command/) command has all the functionality necessary to be able to fully administer the Pi-hole, without the need of the Web Interface. It's fast, user-friendly, and auditable by anyone with an understanding of `bash`.
 
-![Pi-hole Blacklist Demo](https://pi-hole.github.io/graphics/Screenshots/blacklist-cli.gif)
-
 Some notable features include:
 
 - [Whitelisting, Blacklisting and Regex](https://docs.pi-hole.net/core/pihole-command/#whitelisting-blacklisting-and-regex)
@@ -128,8 +147,6 @@ You can read our [Core Feature Breakdown](https://docs.pi-hole.net/core/pihole-c
 
 This [optional dashboard](https://github.com/pi-hole/AdminLTE) allows you to view stats, change settings, and configure your Pi-hole. It's the power of the Command Line Interface, with none of the learning curve!
 
-![Pi-hole Dashboard](https://pi-hole.github.io/graphics/Screenshots/pihole-dashboard.png)
-
 Some notable features include:
 
 - Mobile friendly interface
@@ -145,21 +162,3 @@ There are several ways to [access the dashboard](https://discourse.pi-hole.net/t
 
 1. `http://pi.hole/admin/` (when using Pi-hole as your DNS server)
 2. `http://<IP_ADDPRESS_OF_YOUR_PI_HOLE>/admin/`
-3. `http://pi.hole/` (when using Pi-hole as your DNS server)
-
-## Faster-than-light Engine
-
-FTLDNS is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
-
-Some of the statistics you can integrate include:
-
-- Total number of domains being blocked
-- Total number of DNS queries today
-- Total number of ads blocked today
-- Percentage of ads blocked
-- Unique domains
-- Queries forwarded (to your chosen upstream DNS server)
-- Queries cached
-- Unique clients
-
-The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).

advanced/Scripts/chronometer.sh

@@ -498,10 +498,6 @@ chronoFunc() {
         printFunc " RAM usage: " "$ram_perc%" "$ram_info"
         printFunc " HDD usage: " "$disk_perc" "$disk_info"
 
-        if [[ "$scr_lines" -gt 17 ]] && [[ "$chrono_width" != "small" ]]; then
-            printFunc "  LAN addr: " "${IPV4_ADDRESS/\/*/}" "$lan_info"
-        fi
-
         if [[ "$DHCP_ACTIVE" == "true" ]]; then
             printFunc "DHCP usage: " "$ph_dhcp_percent%" "$dhcp_info"
         fi
@@ -559,7 +555,7 @@ Calculates stats and displays to an LCD
 Options:
   -j, --json          Output stats as JSON formatted string
   -r, --refresh       Set update frequency (in seconds)
-  -e, --exit          Output stats and exit witout refreshing
+  -e, --exit          Output stats and exit without refreshing
   -h, --help          Display this help text"
   fi
 

advanced/Scripts/database_migration/gravity-db.sh

@@ -111,9 +111,15 @@ upgrade_gravityDB(){
 		version=12
 	fi
 	if [[ "$version" == "12" ]]; then
-		# Add column date_updated to alist table
+		# Add column date_updated to adlist table
 		echo -e "  ${INFO} Upgrading gravity database from version 12 to 13"
 		sqlite3 "${database}" < "${scriptPath}/12_to_13.sql"
 		version=13
 	fi
+	if [[ "$version" == "13" ]]; then
+		# Add columns number and status to adlist table
+		echo -e "  ${INFO} Upgrading gravity database from version 13 to 14"
+		sqlite3 "${database}" < "${scriptPath}/13_to_14.sql"
+		version=14
+	fi
 }

advanced/Scripts/database_migration/gravity/13_to_14.sql

@@ -0,0 +1,13 @@
+.timeout 30000
+
+PRAGMA FOREIGN_KEYS=OFF;
+
+BEGIN TRANSACTION;
+
+ALTER TABLE adlist ADD COLUMN number INTEGER NOT NULL DEFAULT 0;
+ALTER TABLE adlist ADD COLUMN invalid_domains INTEGER NOT NULL DEFAULT 0;
+ALTER TABLE adlist ADD COLUMN status INTEGER NOT NULL DEFAULT 0;
+
+UPDATE info SET value = 14 WHERE property = 'version';
+
+COMMIT;

advanced/Scripts/list.sh

@@ -1,4 +1,6 @@
 #!/usr/bin/env bash
+# shellcheck disable=SC1090
+
 # Pi-hole: A black hole for Internet advertisements
 # (c) 2017 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
@@ -9,9 +11,17 @@
 # Please see LICENSE file for your rights under this license.
 
 # Globals
-basename=pihole
-piholeDir=/etc/"${basename}"
-gravityDBfile="${piholeDir}/gravity.db"
+piholeDir="/etc/pihole"
+GRAVITYDB="${piholeDir}/gravity.db"
+# Source pihole-FTL from install script
+pihole_FTL="${piholeDir}/pihole-FTL.conf"
+if [[ -f "${pihole_FTL}" ]]; then
+  source "${pihole_FTL}"
+fi
+
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"
 
 reload=false
 addmode=true
@@ -112,7 +122,7 @@ ProcessDomainList() {
     for dom in "${domList[@]}"; do
         # Format domain into regex filter if requested
         if [[ "${wildcard}" == true ]]; then
-            dom="(^|\\.)${dom//\./\\.}$"
+            dom="(\\.|^)${dom//\./\\.}$"
         fi
 
         # Logic: If addmode then add to desired list and remove from the other;

advanced/Scripts/piholeARPTable.sh

@@ -38,7 +38,7 @@ flushARP(){
 
     # Truncate network_addresses table in pihole-FTL.db
     # This needs to be done before we can truncate the network table due to
-    # foreign key contraints
+    # foreign key constraints
     if ! output=$(sqlite3 "${DBFILE}" "DELETE FROM network_addresses" 2>&1); then
         echo -e "${OVER}  ${CROSS} Failed to truncate network_addresses table"
         echo "  Database location: ${DBFILE}"

advanced/Scripts/piholeDebug.sh

@@ -72,8 +72,8 @@ PIHOLE_SCRIPTS_DIRECTORY="/opt/pihole"
 BIN_DIRECTORY="/usr/local/bin"
 RUN_DIRECTORY="/run"
 LOG_DIRECTORY="/var/log"
-WEB_SERVER_LOG_DIRECTORY="${LOG_DIRECTORY}/lighttpd"
-WEB_SERVER_CONFIG_DIRECTORY="/etc/lighttpd"
+#WEB_SERVER_LOG_DIRECTORY="${LOG_DIRECTORY}/lighttpd" #TODO: FTL access log?
+#WEB_SERVER_CONFIG_DIRECTORY="/etc/lighttpd" #TODO: web server config?
 HTML_DIRECTORY="/var/www/html"
 WEB_GIT_DIRECTORY="${HTML_DIRECTORY}/admin"
 #BLOCK_PAGE_DIRECTORY="${HTML_DIRECTORY}/pihole"
@@ -87,8 +87,8 @@ PIHOLE_DNS_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/01-pihole.conf"
 PIHOLE_DHCP_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/02-pihole-dhcp.conf"
 PIHOLE_WILDCARD_CONFIG_FILE="${DNSMASQ_D_DIRECTORY}/03-wildcard.conf"
 
-WEB_SERVER_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/lighttpd.conf"
-WEB_SERVER_CUSTOM_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/external.conf"
+#WEB_SERVER_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/lighttpd.conf"
+#WEB_SERVER_CUSTOM_CONFIG_FILE="${WEB_SERVER_CONFIG_DIRECTORY}/external.conf"
 
 PIHOLE_INSTALL_LOG_FILE="${PIHOLE_DIRECTORY}/install.log"
 PIHOLE_RAW_BLOCKLIST_FILES="${PIHOLE_DIRECTORY}/list.*"
@@ -138,15 +138,15 @@ PIHOLE_LOG_GZIPS="${LOG_DIRECTORY}/pihole.log.[0-9].*"
 PIHOLE_DEBUG_LOG="${LOG_DIRECTORY}/pihole_debug.log"
 PIHOLE_FTL_LOG="$(get_ftl_conf_value "LOGFILE" "${LOG_DIRECTORY}/pihole-FTL.log")"
 
-PIHOLE_WEB_SERVER_ACCESS_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/access.log"
-PIHOLE_WEB_SERVER_ERROR_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/error.log"
+# PIHOLE_WEB_SERVER_ACCESS_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/access.log" #TODO: FTL access log?
+# PIHOLE_WEB_SERVER_ERROR_LOG_FILE="${WEB_SERVER_LOG_DIRECTORY}/error.log" #TODO: FTL Error log?
 
 # An array of operating system "pretty names" that we officially support
 # We can loop through the array at any time to see if it matches a value
 #SUPPORTED_OS=("Raspbian" "Ubuntu" "Fedora" "Debian" "CentOS")
 
 # Store Pi-hole's processes in an array for easy use and parsing
-PIHOLE_PROCESSES=( "lighttpd" "pihole-FTL" )
+PIHOLE_PROCESSES=( "pihole-FTL" )
 
 # Store the required directories in an array so it can be parsed through
 #REQUIRED_DIRECTORIES=("${CORE_GIT_DIRECTORY}"
@@ -168,8 +168,8 @@ REQUIRED_FILES=("${PIHOLE_CRON_FILE}"
 "${PIHOLE_DNS_CONFIG_FILE}"
 "${PIHOLE_DHCP_CONFIG_FILE}"
 "${PIHOLE_WILDCARD_CONFIG_FILE}"
-"${WEB_SERVER_CONFIG_FILE}"
-"${WEB_SERVER_CUSTOM_CONFIG_FILE}"
+#"${WEB_SERVER_CONFIG_FILE}"
+#"${WEB_SERVER_CUSTOM_CONFIG_FILE}"
 "${PIHOLE_INSTALL_LOG_FILE}"
 "${PIHOLE_RAW_BLOCKLIST_FILES}"
 "${PIHOLE_LOCAL_HOSTS_FILE}"
@@ -244,7 +244,7 @@ initialize_debug() {
     log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initialized."
 }
 
-# This is a function for visually displaying the curent test that is being run.
+# This is a function for visually displaying the current test that is being run.
 # Accepts one variable: the name of what is being diagnosed
 # Colors do not show in the dasboard, but the icons do: [i], [✓], and [✗]
 echo_current_diagnostic() {
@@ -373,39 +373,6 @@ check_component_versions() {
     check_ftl_version
 }
 
-
-get_program_version() {
-    local program_name="${1}"
-    # Create a local variable so this function can be safely reused
-    local program_version
-    echo_current_diagnostic "${program_name} version"
-    # Evalutate the program we are checking, if it is any of the ones below, show the version
-    case "${program_name}" in
-        "lighttpd") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)"
-                    ;;
-        "php") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '-' -f1 | cut -d ' ' -f2)"
-                ;;
-        # If a match is not found, show an error
-        *) echo "Unrecognized program";
-    esac
-    # If the program does not have a version (the variable is empty)
-    if [[ -z "${program_version}" ]]; then
-        # Display and error
-        log_write "${CROSS} ${COL_RED}${program_name} version could not be detected.${COL_NC}"
-    else
-        # Otherwise, display the version
-        log_write "${INFO} ${program_version}"
-    fi
-}
-
-# These are the most critical dependencies of Pi-hole, so we check for them
-# and their versions, using the functions above.
-check_critical_program_versions() {
-    # Use the function created earlier and bundle them into one function that checks all the version numbers
-    get_program_version "lighttpd"
-    get_program_version "php"
-}
-
 os_check() {
     # This function gets a list of supported OS versions from a TXT record at versions.pi-hole.net
     # and determines whether or not the script is running on one of those systems
@@ -641,7 +608,7 @@ detect_ip_addresses() {
     # First argument should be a 4 or a 6
     local protocol=${1}
     # Use ip to show the addresses for the chosen protocol
-    # Store the values in an arry so they can be looped through
+    # Store the values in an array so they can be looped through
     # Get the lines that are in the file(s) and store them in an array for parsing later
     mapfile -t ip_addr_list < <(ip -"${protocol}" addr show dev "${PIHOLE_INTERFACE}" | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }')
 
@@ -695,7 +662,7 @@ ping_gateway() {
     # Check if we are using IPv4 or IPv6
     # Find the default gateway using IPv4 or IPv6
     local gateway
-    gateway="$(ip -"${protocol}" route | grep default | cut -d ' ' -f 3)"
+    gateway="$(ip -"${protocol}" route | grep default | grep "${PIHOLE_INTERFACE}" | cut -d ' ' -f 3)"
 
     # If the gateway variable has a value (meaning a gateway was found),
     if [[ -n "${gateway}" ]]; then
@@ -758,10 +725,10 @@ compare_port_to_service_assigned() {
 
 check_required_ports() {
     echo_current_diagnostic "Ports in use"
-    # Since Pi-hole needs 53, 80, and 4711, check what they are being used by
+    # Since Pi-hole needs 53 and 4711, check what they are being used by
     # so we can detect any issues
     local resolver="pihole-FTL"
-    local web_server="lighttpd"
+    local web_server="pihole-FTL"
     local ftl="pihole-FTL"
     # Create an array for these ports in use
     ports_in_use=()
@@ -823,7 +790,7 @@ check_x_headers() {
     # Do it for the dashboard as well, as the header is different than above
     local dashboard
     dashboard=$(curl -Is localhost/admin/ | awk '/X-Pi-hole/' | tr -d '\r')
-    # Store what the X-Header shoud be in variables for comparison later
+    # Store what the X-Header should be in variables for comparison later
     local block_page_working
     block_page_working="X-Pi-hole: A black hole for Internet advertisements."
     local dashboard_working
@@ -842,12 +809,12 @@ check_x_headers() {
         log_write "${COL_RED}${full_curl_output_block_page}${COL_NC}"
     fi
 
-    # Same logic applies to the dashbord as above, if the X-Header matches what a working system shoud have,
+    # Same logic applies to the dashboard as above, if the X-Header matches what a working system should have,
     if [[ $dashboard == "$dashboard_working" ]]; then
         # then we can show a success
         log_write "$TICK Web interface X-Header: ${COL_GREEN}${dashboard}${COL_NC}"
     else
-        # Othewise, it's a failure since the X-Headers either don't exist or have been modified in some way
+        # Otherwise, it's a failure since the X-Headers either don't exist or have been modified in some way
         log_write "$CROSS Web interface X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
         log_write "${COL_RED}${full_curl_output_dashboard}${COL_NC}"
     fi
@@ -859,13 +826,13 @@ dig_at() {
 
     # Store the arguments as variables with names
     local protocol="${1}"
-    local IP="${2}"
     echo_current_diagnostic "Name resolution (IPv${protocol}) using a random blocked domain and a known ad-serving domain"
     # Set more local variables
     # We need to test name resolution locally, via Pi-hole, and via a public resolver
     local local_dig
-    local pihole_dig
     local remote_dig
+    local interfaces
+    local addresses
     # Use a static domain that we know has IPv4 and IPv6 to avoid false positives
     # Sometimes the randomly chosen domains don't use IPv6, or something else is wrong with them
     local remote_url="doubleclick.com"
@@ -874,15 +841,15 @@ dig_at() {
     if [[ ${protocol} == "6" ]]; then
         # Set the IPv6 variables and record type
         local local_address="::1"
-        local pihole_address="${IP}"
         local remote_address="2001:4860:4860::8888"
+        local sed_selector="inet6"
         local record_type="AAAA"
-    # Othwerwise, it should be 4
+    # Otherwise, it should be 4
     else
         # so use the IPv4 values
         local local_address="127.0.0.1"
-        local pihole_address="${IP}"
         local remote_address="8.8.8.8"
+        local sed_selector="inet"
         local record_type="A"
     fi
 
@@ -892,32 +859,51 @@ dig_at() {
     local random_url
     random_url=$(sqlite3 "${PIHOLE_GRAVITY_DB_FILE}" "SELECT domain FROM vw_gravity ORDER BY RANDOM() LIMIT 1")
 
-    # First, do a dig on localhost to see if Pi-hole can use itself to block a domain
-    if local_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @${local_address} +short "${record_type}"); then
-        # If it can, show success
-        log_write "${TICK} ${random_url} ${COL_GREEN}is ${local_dig}${COL_NC} via ${COL_CYAN}localhost$COL_NC (${local_address})"
-    else
-        # Otherwise, show a failure
-        log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_RED}localhost${COL_NC} (${local_address})"
-    fi
-
     # Next we need to check if Pi-hole can resolve a domain when the query is sent to it's IP address
     # This better emulates how clients will interact with Pi-hole as opposed to above where Pi-hole is
     # just asing itself locally
-    # The default timeouts and tries are reduced in case the DNS server isn't working, so the user isn't waiting for too long
+    # The default timeouts and tries are reduced in case the DNS server isn't working, so the user isn't
+    # waiting for too long
+    #
+    # Turn off history expansion such that the "!" in the sed command cannot do silly things
+    set +H
+    # Get interfaces
+    # sed logic breakdown:
+    #     / master /d;
+    #          Removes all interfaces that are slaves of others (e.g. virtual docker interfaces)
+    #     /UP/!d;
+    #          Removes all interfaces which are not UP
+    #     s/^[0-9]*: //g;
+    #          Removes interface index
+    #     s/: <.*//g;
+    #          Removes everything after the interface name
+    interfaces="$(ip link show | sed "/ master /d;/UP/!d;s/^[0-9]*: //g;s/: <.*//g;")"
 
-    # If Pi-hole can dig itself from it's IP (not the loopback address)
-    if pihole_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @"${pihole_address}" +short "${record_type}"); then
-        # show a success
-        log_write "${TICK} ${random_url} ${COL_GREEN}is ${pihole_dig}${COL_NC} via ${COL_CYAN}Pi-hole${COL_NC} (${pihole_address})"
+    while IFS= read -r iface ; do
+        # Get addresses of current interface
+        # sed logic breakdown:
+        #     /inet(|6) /!d;
+        #          Removes all lines from ip a that do not contain either "inet " or "inet6 "
+        #     s/^.*inet(|6) //g;
+        #          Removes all leading whitespace as well as the "inet " or "inet6 " string
+        #     s/\/.*$//g;
+        #          Removes CIDR and everything thereafter (e.g., scope properties)
+        addresses="$(ip address show dev "${iface}" | sed "/${sed_selector} /!d;s/^.*${sed_selector} //g;s/\/.*$//g;")"
+        while IFS= read -r local_address ; do
+            # Check if Pi-hole can use itself to block a domain
+            if local_dig=$(dig +tries=1 +time=2 -"${protocol}" "${random_url}" @"${local_address}" +short "${record_type}"); then
+                # If it can, show success
+                log_write "${TICK} ${random_url} ${COL_GREEN}is ${local_dig}${COL_NC} on ${COL_CYAN}${iface}${COL_NC} (${COL_CYAN}${local_address}${COL_NC})"
             else
-        # Othewise, show a failure
-        log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_RED}Pi-hole${COL_NC} (${pihole_address})"
+                # Otherwise, show a failure
+                log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} on ${COL_RED}${iface}${COL_NC} (${COL_RED}${local_address}${COL_NC})"
             fi
+        done <<< "${addresses}"
+    done <<< "${interfaces}"
 
     # Finally, we need to make sure legitimate queries can out to the Internet using an external, public DNS server
     # We are using the static remote_url here instead of a random one because we know it works with IPv4 and IPv6
-    if remote_dig=$(dig +tries=1 +time=2 -"${protocol}" "${remote_url}" @${remote_address} +short "${record_type}" | head -n1); then
+    if remote_dig=$(dig +tries=1 +time=2 -"${protocol}" "${remote_url}" @"${remote_address}" +short "${record_type}" | head -n1); then
         # If successful, the real IP of the domain will be returned instead of Pi-hole's IP
         log_write "${TICK} ${remote_url} ${COL_GREEN}is ${remote_dig}${COL_NC} via ${COL_CYAN}a remote, public DNS server${COL_NC} (${remote_address})"
     else
@@ -1032,7 +1018,7 @@ parse_file() {
     local file_lines
     # For each line in the file,
     for file_lines in "${file_info[@]}"; do
-        if [[ ! -z "${file_lines}" ]]; then
+        if [[ -n "${file_lines}" ]]; then
             # don't include the Web password hash
             [[ "${file_lines}" =~ ^\#.*$  || ! "${file_lines}" || "${file_lines}" == "WEBPASSWORD="* ]] && continue
             # otherwise, display the lines of the file
@@ -1044,14 +1030,10 @@ parse_file() {
 }
 
 check_name_resolution() {
-    # Check name resoltion from localhost, Pi-hole's IP, and Google's name severs
+    # Check name resolution from localhost, Pi-hole's IP, and Google's name severs
     # using the function we created earlier
-    dig_at 4 "${IPV4_ADDRESS%/*}"
-    # If IPv6 enabled,
-    if [[ "${IPV6_ADDRESS}" ]]; then
-        # check resolution
-        dig_at 6 "${IPV6_ADDRESS%/*}"
-    fi
+    dig_at 4
+    dig_at 6
 }
 
 # This function can check a directory exists
@@ -1281,39 +1263,29 @@ analyze_gravity_list() {
     IFS="$OLD_IFS"
 }
 
-analyze_pihole_log() {
-    echo_current_diagnostic "Pi-hole log"
-    local head_line
-    # Put the current Internal Field Separator into another variable so it can be restored later
-    OLD_IFS="$IFS"
-    # Get the lines that are in the file(s) and store them in an array for parsing later
-    IFS=$'\r\n'
-    local pihole_log_permissions
-    pihole_log_permissions=$(ls -ld "${PIHOLE_LOG}")
-    log_write "${COL_GREEN}${pihole_log_permissions}${COL_NC}"
-    local pihole_log_head=()
-    mapfile -t pihole_log_head < <(head -n 20 ${PIHOLE_LOG})
-    log_write "   ${COL_CYAN}-----head of $(basename ${PIHOLE_LOG})------${COL_NC}"
+obfuscated_pihole_log() {
+  local pihole_log=("$@")
+  local line
   local error_to_check_for
   local line_to_obfuscate
   local obfuscated_line
-    for head_line in "${pihole_log_head[@]}"; do
+  for line in "${pihole_log[@]}"; do
       # A common error in the pihole.log is when there is a non-hosts formatted file
       # that the DNS server is attempting to read.  Since it's not formatted
       # correctly, there will be an entry for "bad address at line n"
       # So we can check for that here and highlight it in red so the user can see it easily
-        error_to_check_for=$(echo "${head_line}" | grep 'bad address at')
+      error_to_check_for=$(echo "${line}" | grep 'bad address at')
       # Some users may not want to have the domains they visit sent to us
       # To that end, we check for lines in the log that would contain a domain name
-        line_to_obfuscate=$(echo "${head_line}" | grep ': query\|: forwarded\|: reply')
+      line_to_obfuscate=$(echo "${line}" | grep ': query\|: forwarded\|: reply')
       # If the variable contains a value, it found an error in the log
       if [[ -n ${error_to_check_for} ]]; then
           # So we can print it in red to make it visible to the user
-            log_write "   ${CROSS} ${COL_RED}${head_line}${COL_NC} (${FAQ_BAD_ADDRESS})"
+          log_write "   ${CROSS} ${COL_RED}${line}${COL_NC} (${FAQ_BAD_ADDRESS})"
       else
           # If the variable does not a value (the current default behavior), so do not obfuscate anything
           if [[ -z ${OBFUSCATE} ]]; then
-                log_write "   ${head_line}"
+              log_write "   ${line}"
           # Othwerise, a flag was passed to this command to obfuscate domains in the log
           else
               # So first check if there are domains in the log that should be obfuscated
@@ -1323,11 +1295,39 @@ analyze_pihole_log() {
                   obfuscated_line=$(echo "${line_to_obfuscate}" | awk -v placeholder="${OBFUSCATED_PLACEHOLDER}" '{sub($6,placeholder); print $0}')
                   log_write "   ${obfuscated_line}"
               else
-                    log_write "   ${head_line}"
+                  log_write "   ${line}"
               fi
           fi
       fi
   done
+}
+
+analyze_pihole_log() {
+  echo_current_diagnostic "Pi-hole log"
+  local pihole_log_head=()
+  local pihole_log_tail=()
+  local pihole_log_permissions
+  local logging_enabled
+
+  logging_enabled=$(grep -c "^log-queries" /etc/dnsmasq.d/01-pihole.conf)
+  if [[ "${logging_enabled}" == "0" ]]; then
+      # Inform user that logging has been disabled and pihole.log does not contain queries
+      log_write "${INFO} Query logging is disabled"
+      log_write ""
+  fi
+  # Put the current Internal Field Separator into another variable so it can be restored later
+  OLD_IFS="$IFS"
+  # Get the lines that are in the file(s) and store them in an array for parsing later
+  IFS=$'\r\n'
+  pihole_log_permissions=$(ls -ld "${PIHOLE_LOG}")
+  log_write "${COL_GREEN}${pihole_log_permissions}${COL_NC}"
+  mapfile -t pihole_log_head < <(head -n 20 ${PIHOLE_LOG})
+  log_write "   ${COL_CYAN}-----head of $(basename ${PIHOLE_LOG})------${COL_NC}"
+  obfuscated_pihole_log "${pihole_log_head[@]}"
+  log_write ""
+  mapfile -t pihole_log_tail < <(tail -n 20 ${PIHOLE_LOG})
+  log_write "   ${COL_CYAN}-----tail of $(basename ${PIHOLE_LOG})------${COL_NC}"
+  obfuscated_pihole_log "${pihole_log_tail[@]}"
   log_write ""
   # Set the IFS back to what it was
   IFS="$OLD_IFS"
@@ -1423,7 +1423,6 @@ initialize_debug
 # available to the other functions
 source_setup_variables
 check_component_versions
-check_critical_program_versions
 diagnose_operating_system
 check_selinux
 check_firewalld

advanced/Scripts/query.sh

@@ -11,12 +11,21 @@
 
 # Globals
 piholeDir="/etc/pihole"
-gravityDBfile="${piholeDir}/gravity.db"
+GRAVITYDB="${piholeDir}/gravity.db"
 options="$*"
 all=""
 exact=""
 blockpage=""
 matchType="match"
+# Source pihole-FTL from install script
+pihole_FTL="${piholeDir}/pihole-FTL.conf"
+if [[ -f "${pihole_FTL}" ]]; then
+  source "${pihole_FTL}"
+fi
+
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"
 
 colfile="/opt/pihole/COL_TABLE"
 source "${colfile}"

advanced/Scripts/version.sh

@@ -153,7 +153,7 @@ versionOutput() {
     if [[ -n "$current" ]] && [[ -n "$latest" ]]; then
         output="${1^} version is $branch$current (Latest: $latest)"
     elif [[ -n "$current" ]] && [[ -z "$latest" ]]; then
-        output="Current ${1^} version is $branch$current."
+        output="Current ${1^} version is $branch$current"
     elif [[ -z "$current" ]] && [[ -n "$latest" ]]; then
         output="Latest ${1^} version is $latest"
     elif [[ "$curHash" == "N/A" ]] || [[ "$latHash" == "N/A" ]]; then

advanced/Scripts/webpage.sh

@@ -273,11 +273,6 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423
         fi
     fi
 
-    # Prevent Firefox from automatically switching over to DNS-over-HTTPS
-    # This follows https://support.mozilla.org/en-US/kb/configuring-networks-disable-dns-over-https
-    # (sourced 7th September 2019)
-    add_dnsmasq_setting "server=/use-application-dns.net/"
-
     # We need to process DHCP settings here as well to account for possible
     # changes in the non-FQDN forwarding. This cannot be done in 01-pihole.conf
     # as we don't want to delete all local=/.../ lines so it's much safer to
@@ -426,7 +421,7 @@ dhcp-leasefile=/etc/pihole/dhcp.leases
         echo "#quiet-dhcp6
 #enable-ra
 dhcp-option=option6:dns-server,[::]
-dhcp-range=::100,::1ff,constructor:${interface},ra-names,slaac,${leasetime}
+dhcp-range=::100,::1ff,constructor:${interface},ra-names,slaac,64,3600
 ra-param=*,0,0
 " >> "${dhcpconfig}"
     fi
@@ -564,7 +559,13 @@ AddDHCPStaticAddress() {
 
 RemoveDHCPStaticAddress() {
     mac="${args[2]}"
+    if [[ "$mac" =~ ^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$ ]]; then
         sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
+    else
+        echo "  ${CROSS} Invalid Mac Passed!"
+        exit 1
+    fi
+
 }
 
 SetAdminEmail() {
@@ -636,8 +637,11 @@ Interfaces:
 
 Teleporter() {
     local datetimestamp
+    local host
     datetimestamp=$(date "+%Y-%m-%d_%H-%M-%S")
-    php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-teleporter_${datetimestamp}.tar.gz"
+    host=$(hostname)
+    host="${host//./_}"
+    php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-${host:-noname}-teleporter_${datetimestamp}.tar.gz"
 }
 
 checkDomain()
@@ -705,7 +709,13 @@ RemoveCustomDNSAddress() {
 
     ip="${args[2]}"
     host="${args[3]}"
-    sed -i "/${ip} ${host}/d" "${dnscustomfile}"
+
+    if valid_ip "${ip}" || valid_ip6 "${ip}" ; then
+        sed -i "/^${ip} ${host}$/d" "${dnscustomfile}"
+    else
+        echo -e "  ${CROSS} Invalid IP has been passed"
+        exit 1
+    fi
 
     # Restart dnsmasq to update removed custom DNS entries
     RestartDNS
@@ -727,7 +737,20 @@ RemoveCustomCNAMERecord() {
 
     domain="${args[2]}"
     target="${args[3]}"
-    sed -i "/cname=${domain},${target}/d" "${dnscustomcnamefile}"
+
+    validDomain="$(checkDomain "${domain}")"
+    if [[ -n "${validDomain}" ]]; then
+        validTarget="$(checkDomain "${target}")"
+        if [[ -n "${validDomain}" ]]; then
+            sed -i "/cname=${validDomain},${validTarget}$/d" "${dnscustomcnamefile}"
+        else
+            echo "  ${CROSS} Invalid Target Passed!"
+            exit 1
+        fi
+    else
+        echo "  ${CROSS} Invalid Domain passed!"
+        exit 1
+    fi
 
     # Restart dnsmasq to update removed custom CNAME records
     RestartDNS

advanced/Templates/gravity.db.sql

@@ -32,7 +32,10 @@ CREATE TABLE adlist
 	date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
 	comment TEXT,
-	date_updated INTEGER
+	date_updated INTEGER,
+	number INTEGER NOT NULL DEFAULT 0,
+	invalid_domains INTEGER NOT NULL DEFAULT 0,
+	status INTEGER NOT NULL DEFAULT 0
 );
 
 CREATE TABLE adlist_by_group
@@ -54,7 +57,7 @@ CREATE TABLE info
 	value TEXT NOT NULL
 );
 
-INSERT INTO "info" VALUES('version','13');
+INSERT INTO "info" VALUES('version','14');
 
 CREATE TABLE domain_audit
 (

advanced/blockingpage.css

@@ -145,7 +145,17 @@ body {
 }
 
 /* User is greeted with a splash page when browsing to Pi-hole IP address */
-#splashpage { background: #222; color: rgba(255, 255, 255, 0.7); text-align: center; }
+#splashpage {
+  background: #222;
+  color: rgba(255, 255, 255, 0.7);
+  text-align: center;
+  width: 100%;
+  height: 100%;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+}
+
 #splashpage img { margin: 5px; width: 256px; }
 #splashpage b { color: inherit; }
 
@@ -196,6 +206,26 @@ header #bpAlt label {
   display: block;
 }
 
+html, body {
+    height: 100%;
+}
+
+#pihole_card {
+    width: 400px;
+    height: auto;
+    max-width: 400px;
+}
+
+    #pihole_card p, #pihole_card a {
+        font-size: 13pt;
+        text-align: center;
+    }
+
+#pihole_logo_splash {
+    height: auto;
+    width: 100%;
+}
+
 /* Click anywhere else on screen to hide #bpAbout */
 #bpAboutToggle:checked {
   display: block;
@@ -382,12 +412,44 @@ footer {
 
 /* Responsive Content */
 @media only screen and (max-width: 500px) {
-  h1 a { font-size: 1.8rem; min-width: 170px; }
-  footer span::before { content: "Generated "; }
-  footer span { display: block; }
+  h1 a {
+        font-size: 1.8rem;
+        min-width: 170px;
+    }
+
+    footer span::before {
+        content: "Generated ";
+    }
+
+    footer span {
+        display: block;
+    }
 }
 
 @media only screen and (min-width: 1251px) {
-  #bpWrapper, footer { border-radius: 0 0 5px 5px; }
-  #bpAbout { border-right-width: 1px; }
+  #bpWrapper, footer {
+        border-radius: 0 0 5px 5px;
+    }
+
+    #bpAbout {
+        border-right-width: 1px;
+    }
+}
+
+@media only screen and (max-width: 400px) {
+    #pihole_card {
+        width: 100%;
+        height: auto;
+    }
+
+        #pihole_card p, #pihole_card a {
+            font-size: 100%;
+        }
+}
+
+@media only screen and (max-width: 256px) {
+    #pihole_logo_splash {
+        width: 90% !important;
+        height: auto;
+    }
 }

advanced/index.php

@@ -58,14 +58,15 @@ if ($serverName === "pi.hole"
     // When directly browsing via IP or authorized hostname
     // Render splash/landing page based off presence of $landPage file
     // Unset variables so as to not be included in $landPage or $splashPage
-    unset($serverName, $svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt, $viewPort);
+    unset($svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt);
     // If $landPage file is present
     if (is_file(getcwd()."/$landPage")) {
+        unset($serverName, $viewPort); // unset extra variables not to be included in $landpage
         include $landPage;
         exit();
     }
     // If $landPage file was not present, Set Splash Page output
-    $splashPage = "
+    $splashPage = <<<EOT
     <!doctype html>
     <html lang='en'>
         <head>
@@ -76,13 +77,14 @@ if ($serverName === "pi.hole"
             <link rel='shortcut icon' href='admin/img/favicons/favicon.ico' type='image/x-icon'>
         </head>
         <body id='splashpage'>
-            <img src='admin/img/logo.svg' alt='Pi-hole logo' width='256' height='377'>
-            <br>
+            <div id="pihole_card">
+              <img src='admin/img/logo.svg' alt='Pi-hole logo' id="pihole_logo_splash" />
               <p>Pi-<strong>hole</strong>: Your black hole for Internet advertisements</p>
               <a href='/admin'>Did you mean to go to the admin panel?</a>
+            </div>
         </body>
     </html>
-    ";
+EOT;
     exit($splashPage);
 } elseif ($currentUrlExt === "js") {
     // Serve Pi-hole JavaScript for blocked domains requesting JS

advanced/lighttpd.conf.debian

@@ -81,6 +81,10 @@ mimetype.assign = (
     ".woff2" => "font/woff2"
 )
 
+# Add user chosen options held in external file
+# This uses include_shell instead of an include wildcard for compatibility
+include_shell "cat external.conf 2>/dev/null"
+
 # default listening port for IPv6 falls back to the IPv4 port
 include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
 
@@ -109,7 +113,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {
 
 # Default expire header
 expire.url = ( "" => "access plus 0 seconds" )
-
-# Add user chosen options held in external file
-# This uses include_shell instead of an include wildcard for compatibility
-include_shell "cat external.conf 2>/dev/null"

advanced/lighttpd.conf.fedora

@@ -82,6 +82,10 @@ mimetype.assign = (
     ".woff2" => "font/woff2"
 )
 
+# Add user chosen options held in external file
+# This uses include_shell instead of an include wildcard for compatibility
+include_shell "cat external.conf 2>/dev/null"
+
 # default listening port for IPv6 falls back to the IPv4 port
 #include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
 #include_shell "/usr/share/lighttpd/create-mime.assign.pl"
@@ -117,7 +121,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {
 
 # Default expire header
 expire.url = ( "" => "access plus 0 seconds" )
-
-# Add user chosen options held in external file
-# This uses include_shell instead of an include wildcard for compatibility
-include_shell "cat external.conf 2>/dev/null"

automated install/uninstall.sh

@@ -31,7 +31,7 @@ else
     else
         echo -e "  ${CROSS} ${str}
             Script called with non-root privileges
-            The Pi-hole requires elevated privleges to uninstall"
+            The Pi-hole requires elevated privileges to uninstall"
         exit 1
     fi
 fi
@@ -206,11 +206,7 @@ removeNoPurge() {
 }
 
 ######### SCRIPT ###########
-if command -v vcgencmd &> /dev/null; then
-    echo -e "  ${INFO} All dependencies are safe to remove on Raspbian"
-else
 echo -e "  ${INFO} Be sure to confirm if any dependencies should not be removed"
-fi
 while true; do
     echo -e "  ${INFO} ${COL_YELLOW}The following dependencies may have been added by the Pi-hole install:"
     echo -n "    "

gravity.sh

@@ -35,8 +35,9 @@ localList="${piholeDir}/local.list"
 VPNList="/etc/openvpn/ipp.txt"
 
 piholeGitDir="/etc/.pihole"
-gravityDBfile="${piholeDir}/gravity.db"
-gravityTEMPfile="${piholeDir}/gravity_temp.db"
+gravityDBfile_default="${piholeDir}/gravity.db"
+# GRAVITYDB may be overwritten by source pihole-FTL.conf below
+GRAVITYDB="${gravityDBfile_default}"
 gravityDBschema="${piholeGitDir}/advanced/Templates/gravity.db.sql"
 gravityDBcopy="${piholeGitDir}/advanced/Templates/gravity_copy.sql"
 
@@ -46,16 +47,6 @@ domainsExtension="domains"
 setupVars="${piholeDir}/setupVars.conf"
 if [[ -f "${setupVars}" ]];then
   source "${setupVars}"
-
-  # Remove CIDR mask from IPv4/6 addresses
-  IPV4_ADDRESS="${IPV4_ADDRESS%/*}"
-  IPV6_ADDRESS="${IPV6_ADDRESS%/*}"
-
-  # Determine if IPv4/6 addresses exist
-  if [[ -z "${IPV4_ADDRESS}" ]] && [[ -z "${IPV6_ADDRESS}" ]]; then
-    echo -e "  ${COL_LIGHT_RED}No IP addresses found! Please run 'pihole -r' to reconfigure${COL_NC}"
-    exit 1
-  fi
 else
   echo -e "  ${COL_LIGHT_RED}Installation Failure: ${setupVars} does not exist! ${COL_NC}
   Please run 'pihole -r', and choose the 'reconfigure' option to fix."
@@ -68,6 +59,13 @@ if [[ -f "${pihole_FTL}" ]]; then
   source "${pihole_FTL}"
 fi
 
+# Set this only after sourcing pihole-FTL.conf as the gravity database path may
+# have changed
+gravityDBfile="${GRAVITYDB}"
+gravityTEMPfile="${GRAVITYDB}_temp"
+gravityDIR="$(dirname -- "${gravityDBfile}")"
+gravityOLDfile="${gravityDIR}/gravity_old.db"
+
 if [[ -z "${BLOCKINGMODE}" ]] ; then
   BLOCKINGMODE="NULL"
 fi
@@ -84,11 +82,11 @@ generate_gravity_database() {
 
 # Copy data from old to new database file and swap them
 gravity_swap_databases() {
-  local str
+  local str copyGravity
   str="Building tree"
   echo -ne "  ${INFO} ${str}..."
 
-  # The index is intentionally not UNIQUE as prro quality adlists may contain domains more than once
+  # The index is intentionally not UNIQUE as poor quality adlists may contain domains more than once
   output=$( { sqlite3 "${gravityTEMPfile}" "CREATE INDEX idx_gravity ON gravity (domain, adlist_id);"; } 2>&1 )
   status="$?"
 
@@ -101,7 +99,14 @@ gravity_swap_databases() {
   str="Swapping databases"
   echo -ne "  ${INFO} ${str}..."
 
-  output=$( { sqlite3 "${gravityTEMPfile}" < "${gravityDBcopy}"; } 2>&1 )
+  # Gravity copying SQL script
+  copyGravity="$(cat "${gravityDBcopy}")"
+  if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
+    # Replace default gravity script location by custom location
+    copyGravity="${copyGravity//"${gravityDBfile_default}"/"${gravityDBfile}"}"
+  fi
+
+  output=$( { sqlite3 "${gravityTEMPfile}" <<< "${copyGravity}"; } 2>&1 )
   status="$?"
 
   if [[ "${status}" -ne 0 ]]; then
@@ -110,8 +115,19 @@ gravity_swap_databases() {
   fi
   echo -e "${OVER}  ${TICK} ${str}"
 
-  # Swap databases and remove old database
+  # Swap databases and remove or conditionally rename old database
+  # Number of available blocks on disk
+  availableBlocks=$(stat -f --format "%a" "${gravityDIR}")
+  # Number of blocks, used by gravity.db
+  gravityBlocks=$(stat --format "%b" ${gravityDBfile})
+  # Only keep the old database if available disk space is at least twice the size of the existing gravity.db.
+  # Better be safe than sorry...
+  if [ "${availableBlocks}" -gt "$(("${gravityBlocks}" * 2))" ] && [ -f "${gravityDBfile}" ]; then
+    echo -e "  ${TICK} The old database remains available."
+    mv "${gravityDBfile}" "${gravityOLDfile}"
+  else
     rm "${gravityDBfile}"
+  fi
   mv "${gravityTEMPfile}" "${gravityDBfile}"
 }
 
@@ -176,7 +192,7 @@ database_table_from_file() {
         echo "${rowid},\"${domain}\",${timestamp}" >> "${tmpFile}"
       elif [[ "${table}" == "adlist" ]]; then
         # Adlist table format
-        echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"," >> "${tmpFile}"
+        echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\",,0,0,0" >> "${tmpFile}"
       else
         # White-, black-, and regexlist table format
         echo "${rowid},${type},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"" >> "${tmpFile}"
@@ -217,6 +233,48 @@ database_adlist_updated() {
   fi
 }
 
+# Check if a column with name ${2} exists in gravity table with name ${1}
+gravity_column_exists() {
+  output=$( { printf ".timeout 30000\\nSELECT EXISTS(SELECT * FROM pragma_table_info('%s') WHERE name='%s');\\n" "${1}" "${2}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  if [[ "${output}" == "1" ]]; then
+    return 0 # Bash 0 is success
+  fi
+
+  return 1 # Bash non-0 is failure
+}
+
+# Update number of domain on this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_number() {
+  # Only try to set number of domains when this field exists in the gravity database
+  if ! gravity_column_exists "adlist" "number"; then
+    return;
+  fi
+
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET number = %i, invalid_domains = %i WHERE id = %i;\\n" "${num_lines}" "${num_invalid}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update number of domains in adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
+# Update status of this list. We store this in the "old" database as all values in the new database will later be overwritten
+database_adlist_status() {
+  # Only try to set the status when this field exists in the gravity database
+  if ! gravity_column_exists "adlist" "status"; then
+    return;
+  fi
+
+  output=$( { printf ".timeout 30000\\nUPDATE adlist SET status = %i WHERE id = %i;\\n" "${2}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
+  status="$?"
+
+  if [[ "${status}" -ne 0 ]]; then
+    echo -e "\\n  ${CROSS} Unable to update status of adlist with ID ${1} in database ${gravityDBfile}\\n  ${output}"
+    gravity_Cleanup "error"
+  fi
+}
+
 # Migrate pre-v5.0 list files to database-based Pi-hole versions
 migrate_to_database() {
   # Create database file only if not present
@@ -317,6 +375,10 @@ gravity_CheckDNSResolutionAvailable() {
 gravity_DownloadBlocklists() {
   echo -e "  ${INFO} ${COL_BOLD}Neutrino emissions detected${COL_NC}..."
 
+  if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
+    echo -e "  ${INFO} Storing gravity database in ${COL_BOLD}${gravityDBfile}${COL_NC}"
+  fi
+
   # Retrieve source URLs from gravity database
   # We source only enabled adlists, sqlite3 stores boolean values as 0 (false) or 1 (true)
   mapfile -t sources <<< "$(sqlite3 "${gravityDBfile}" "SELECT address FROM vw_adlist;" 2> /dev/null)"
@@ -363,7 +425,7 @@ gravity_DownloadBlocklists() {
 
   target="$(mktemp -p "/tmp" --suffix=".gravity")"
 
-  # Use compression to reduce the amount of data that is transfered
+  # Use compression to reduce the amount of data that is transferred
   # between the Pi-hole and the ad list provider. Use this feature
   # only if it is supported by the locally available version of curl
   if curl -V | grep -q "Features:.* libz"; then
@@ -444,6 +506,8 @@ gravity_DownloadBlocklists() {
 }
 
 total_num=0
+num_lines=0
+num_invalid=0
 parseList() {
   local adlistID="${1}" src="${2}" target="${3}" incorrect_lines
   # This sed does the following things:
@@ -454,7 +518,7 @@ parseList() {
   # Find (up to) five domains containing invalid characters (see above)
   incorrect_lines="$(sed -e "/[^a-zA-Z0-9.\_-]/!d" "${src}" | head -n 5)"
 
-  local num_lines num_target_lines num_correct_lines num_invalid
+  local num_target_lines num_correct_lines num_invalid
   # Get number of lines in source file
   num_lines="$(grep -c "^" "${src}")"
   # Get number of lines in destination file
@@ -463,9 +527,9 @@ parseList() {
   total_num="$num_target_lines"
   num_invalid="$(( num_lines-num_correct_lines ))"
   if [[ "${num_invalid}" -eq 0 ]]; then
-    echo "  ${INFO} Received ${num_lines} domains"
+    echo "  ${INFO} Analyzed ${num_lines} domains"
   else
-    echo "  ${INFO} Received ${num_lines} domains, ${num_invalid} domains invalid!"
+    echo "  ${INFO} Analyzed ${num_lines} domains, ${num_invalid} domains invalid!"
   fi
 
   # Display sample of invalid lines if we found some
@@ -476,11 +540,34 @@ parseList() {
     done <<< "${incorrect_lines}"
   fi
 }
+compareLists() {
+  local adlistID="${1}" target="${2}"
+
+  # Verify checksum when an older checksum exists
+  if [[ -s "${target}.sha1" ]]; then
+    if ! sha1sum --check --status --strict "${target}.sha1"; then
+      # The list changed upstream, we need to update the checksum
+      sha1sum "${target}" > "${target}.sha1"
+      echo "  ${INFO} List has been updated"
+      database_adlist_status "${adlistID}" "1"
+      database_adlist_updated "${adlistID}"
+    else
+      echo "  ${INFO} List stayed unchanged"
+      database_adlist_status "${adlistID}" "2"
+    fi
+  else
+    # No checksum available, create one for comparing on the next run
+    sha1sum "${target}" > "${target}.sha1"
+    # We assume here it was changed upstream
+    database_adlist_status "${adlistID}" "1"
+    database_adlist_updated "${adlistID}"
+  fi
+}
 
 # Download specified URL and perform checks on HTTP status and file content
 gravity_DownloadBlocklistFromUrl() {
   local url="${1}" cmd_ext="${2}" agent="${3}" adlistID="${4}" saveLocation="${5}" target="${6}" compression="${7}"
-  local heisenbergCompensator="" patternBuffer str httpCode success=""
+  local heisenbergCompensator="" patternBuffer str httpCode success="" ip
 
   # Create temp file to store content on disk instead of RAM
   patternBuffer=$(mktemp -p "/tmp" --suffix=".phgpb")
@@ -498,7 +585,10 @@ gravity_DownloadBlocklistFromUrl() {
   blocked=false
   case $BLOCKINGMODE in
     "IP-NODATA-AAAA"|"IP")
-        if [[ $(dig "${domain}" +short | grep "${IPV4_ADDRESS}" -c) -ge 1 ]]; then
+        # Get IP address of this domain
+        ip="$(dig "${domain}" +short)"
+        # Check if this IP matches any IP of the system
+        if [[ -n "${ip}" && $(grep -Ec "inet(|6) ${ip}" <<< "$(ip a)") -gt 0 ]]; then
           blocked=true
         fi;;
     "NXDOMAIN")
@@ -559,31 +649,49 @@ gravity_DownloadBlocklistFromUrl() {
       esac;;
   esac
 
+  local done="false"
   # Determine if the blocklist was downloaded and saved correctly
   if [[ "${success}" == true ]]; then
     if [[ "${httpCode}" == "304" ]]; then
       # Add domains to database table file
       parseList "${adlistID}" "${saveLocation}" "${target}"
+      database_adlist_status "${adlistID}" "2"
+      database_adlist_number "${adlistID}"
+      done="true"
     # Check if $patternbuffer is a non-zero length file
     elif [[ -s "${patternBuffer}" ]]; then
       # Determine if blocklist is non-standard and parse as appropriate
       gravity_ParseFileIntoDomains "${patternBuffer}" "${saveLocation}"
       # Add domains to database table file
       parseList "${adlistID}" "${saveLocation}" "${target}"
-      # Update date_updated field in gravity database table
-      database_adlist_updated "${adlistID}"
+      # Compare lists, are they identical?
+      compareLists "${adlistID}" "${saveLocation}"
+      # Update gravity database table (status and updated timestamp are set in
+      # compareLists)
+      database_adlist_number "${adlistID}"
+      done="true"
     else
       # Fall back to previously cached list if $patternBuffer is empty
-      echo -e "  ${INFO} Received empty file: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
+      echo -e "  ${INFO} Received empty file"
     fi
-  else
+  fi
+
+  # Do we need to fall back to a cached list (if available)?
+  if [[ "${done}" != "true" ]]; then
     # Determine if cached list has read permission
     if [[ -r "${saveLocation}" ]]; then
       echo -e "  ${CROSS} List download failed: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
       # Add domains to database table file
       parseList "${adlistID}" "${saveLocation}" "${target}"
+      database_adlist_number "${adlistID}"
+      database_adlist_status "${adlistID}" "3"
     else
       echo -e "  ${CROSS} List download failed: ${COL_LIGHT_RED}no cached list available${COL_NC}"
+      # Manually reset these two numbers because we do not call parseList here
+      num_lines=0
+      num_invalid=0
+      database_adlist_number "${adlistID}"
+      database_adlist_status "${adlistID}" "4"
     fi
   fi
 }
@@ -595,7 +703,7 @@ gravity_ParseFileIntoDomains() {
   # Determine if we are parsing a consolidated list
   #if [[ "${source}" == "${piholeDir}/${matterAndLight}" ]]; then
     # Remove comments and print only the domain name
-    # Most of the lists downloaded are already in hosts file format but the spacing/formating is not contiguous
+    # Most of the lists downloaded are already in hosts file format but the spacing/formatting is not contiguous
     # This helps with that and makes it easier to read
     # It also helps with debugging so each stage of the script can be researched more in depth
     # 1) Remove carriage returns
@@ -683,43 +791,12 @@ gravity_ShowCount() {
   gravity_Table_Count "vw_regex_whitelist" "regex whitelist filters"
 }
 
-# Parse list of domains into hosts format
-gravity_ParseDomainsIntoHosts() {
-  awk -v ipv4="$IPV4_ADDRESS" -v ipv6="$IPV6_ADDRESS" '{
-    # Remove windows CR line endings
-    sub(/\r$/, "")
-    # Parse each line as "ipaddr domain"
-    if(ipv6 && ipv4) {
-      print ipv4" "$0"\n"ipv6" "$0
-    } else if(!ipv6) {
-      print ipv4" "$0
-    } else {
-      print ipv6" "$0
-    }
-  }' >> "${2}" < "${1}"
-}
-
 # Create "localhost" entries into hosts format
 gravity_generateLocalList() {
-  local hostname
-
-  if [[ -s "/etc/hostname" ]]; then
-    hostname=$(< "/etc/hostname")
-  elif command -v hostname &> /dev/null; then
-    hostname=$(hostname -f)
-  else
-    echo -e "  ${CROSS} Unable to determine fully qualified domain name of host"
-    return 0
-  fi
-
-  echo -e "${hostname}\\npi.hole" > "${localList}.tmp"
-
   # Empty $localList if it already exists, otherwise, create it
-  : > "${localList}"
+  echo "### Do not modify this file, it will be overwritten by pihole -g" > "${localList}"
   chmod 644 "${localList}"
 
-  gravity_ParseDomainsIntoHosts "${localList}.tmp" "${localList}"
-
   # Add additional LAN hosts provided by OpenVPN (if available)
   if [[ -f "${VPNList}" ]]; then
     awk -F, '{printf $2"\t"$1".vpn\n"}' "${VPNList}" >> "${localList}"
@@ -788,6 +865,11 @@ for var in "$@"; do
   esac
 done
 
+# Remove OLD (backup) gravity file, if it exists
+if [[ -f "${gravityOLDfile}" ]]; then
+  rm "${gravityOLDfile}"
+fi
+
 # Trap Ctrl-C
 gravity_Trap
 

manpages/pihole.8

@@ -56,7 +56,7 @@ Available commands and options:
 
 \fB-w, whitelist\fR [options] [<domain1> <domain2 ...>]
 .br
-    Adds or removes specified domain or domains tho the Whitelist
+    Adds or removes specified domain or domains to the Whitelist
 .br
 
 \fB-b, blacklist\fR [options] [<domain1> <domain2 ...>]
@@ -153,7 +153,7 @@ Available commands and options:
 .br
       -r, --refresh     Set update frequency (in seconds)
 .br
-      -e, --exit        Output stats and exit witout refreshing
+      -e, --exit        Output stats and exit without refreshing
 .br
 
 \fB-g, updateGravity\fR

pihole

@@ -16,6 +16,7 @@ readonly PI_HOLE_SCRIPT_DIR="/opt/pihole"
 # error due to modifying a readonly variable.
 setupVars="/etc/pihole/setupVars.conf"
 PI_HOLE_BIN_DIR="/usr/local/bin"
+readonly FTL_PID_FILE="/run/pihole-FTL.pid"
 
 readonly colfile="${PI_HOLE_SCRIPT_DIR}/COL_TABLE"
 source "${colfile}"
@@ -98,8 +99,25 @@ versionFunc() {
   exit 0
 }
 
+# Get PID of main pihole-FTL process
+getFTLPID() {
+  local pid
+
+  if [ -s "${FTL_PID_FILE}" ]; then
+    # -s: FILE exists and has a size greater than zero
+    pid="$(<"$FTL_PID_FILE")"
+    # Exploit prevention: unset the variable if there is malicious content
+    # Verify that the value read from the file is numeric
+    [[ "$pid" =~ [^[:digit:]] ]] && unset pid
+  fi
+
+  # If FTL is not running, or the PID file contains malicious stuff, substitute
+  # negative PID to signal this to the caller
+  echo "${pid:=-1}"
+}
+
 restartDNS() {
-  local svcOption svc str output status
+  local svcOption svc str output status pid icon
   svcOption="${1:-restart}"
 
   # Determine if we should reload or restart
@@ -108,17 +126,34 @@ restartDNS() {
     # Note 1: This will NOT re-read any *.conf files
     # Note 2: We cannot use killall here as it does
     #         not know about real-time signals
-    svc="pkill -RTMIN pihole-FTL"
+    pid="$(getFTLPID)"
+    if [[ "$pid" -eq "-1" ]]; then
+      svc="true"
+      str="FTL is not running"
+      icon="${INFO}"
+    else
+      svc="kill -RTMIN ${pid}"
       str="Reloading DNS lists"
+      icon="${TICK}"
+    fi
   elif [[ "${svcOption}" =~ "reload" ]]; then
     # Reloading of the DNS cache has been requested
     # Note: This will NOT re-read any *.conf files
-    svc="pkill -HUP pihole-FTL"
+    pid="$(getFTLPID)"
+    if [[ "$pid" -eq "-1" ]]; then
+      svc="true"
+      str="FTL is not running"
+      icon="${INFO}"
+    else
+      svc="kill -HUP ${pid}"
       str="Flushing DNS cache"
+      icon="${TICK}"
+    fi
   else
     # A full restart has been requested
     svc="service pihole-FTL restart"
     str="Restarting DNS server"
+    icon="${TICK}"
   fi
 
   # Print output to Terminal, but not to Web Admin
@@ -128,7 +163,7 @@ restartDNS() {
   status="$?"
 
   if [[ "${status}" -eq 0 ]]; then
-    [[ -t 1 ]] && echo -e "${OVER}  ${TICK} ${str}"
+    [[ -t 1 ]] && echo -e "${OVER}  ${icon} ${str}"
     return 0
   else
     [[ ! -t 1 ]] && local OVER=""
@@ -328,16 +363,13 @@ tailFunc() {
   fi
   echo -e "  ${INFO} Press Ctrl-C to exit"
 
-  # Retrieve IPv4/6 addresses
-  source /etc/pihole/setupVars.conf
-
   # Strip date from each line
   # Color blocklist/blacklist/wildcard entries as red
   # Color A/AAAA/DHCP strings as white
   # Color everything else as gray
-  tail -f /var/log/pihole.log | sed -E \
+  tail -f /var/log/pihole.log | grep --line-buffered "${1}" | sed -E \
     -e "s,($(date +'%b %d ')| dnsmasq\[[0-9]*\]),,g" \
-    -e "s,(.*(blacklisted |gravity blocked ).* is (0.0.0.0|::|NXDOMAIN|${IPV4_ADDRESS%/*}|${IPV6_ADDRESS:-NULL}).*),${COL_RED}&${COL_NC}," \
+    -e "s,(.*(blacklisted |gravity blocked ).* is (0.0.0.0|::|NXDOMAIN).*),${COL_RED}&${COL_NC}," \
     -e "s,.*(query\\[A|DHCP).*,${COL_NC}&${COL_NC}," \
     -e "s,.*,${COL_GRAY}&${COL_NC},"
   exit 0
@@ -421,7 +453,10 @@ Debugging Options:
                         Add '-a' to automatically upload the log to tricorder.pi-hole.net
   -f, flush           Flush the Pi-hole log
   -r, reconfigure     Reconfigure or Repair Pi-hole subsystems
-  -t, tail            View the live output of the Pi-hole log
+  -t, tail [arg]      View the live output of the Pi-hole log.
+                      Add an optional argument to filter the log
+                      (regular expressions are supported)
+
 
 Options:
   -a, admin           Web interface options
@@ -495,7 +530,7 @@ case "${1}" in
   "status"                      ) statusFunc "$2";;
   "restartdns"                  ) restartDNS "$2";;
   "-a" | "admin"                ) webpageFunc "$@";;
-  "-t" | "tail"                 ) tailFunc;;
+  "-t" | "tail"                 ) tailFunc "$2";;
   "checkout"                    ) piholeCheckoutFunc "$@";;
   "tricorder"                   ) tricorderFunc;;
   "updatechecker"               ) updateCheckFunc "$@";;

supportedos.txt

@@ -1,5 +1,5 @@
 Raspbian=9,10
 Ubuntu=16,18,20
 Debian=9,10
-Fedora=31,32
+Fedora=32,33
 CentOS=7,8

test/_fedora_33.Dockerfile

@@ -1,4 +1,4 @@
-FROM fedora:31
+FROM fedora:33
 
 ENV GITDIR /etc/.pihole
 ENV SCRIPTDIR /opt/pihole

test/conftest.py

@@ -8,8 +8,6 @@ check_output = testinfra.get_backend(
 
 SETUPVARS = {
     'PIHOLE_INTERFACE': 'eth99',
-    'IPV4_ADDRESS': '1.1.1.1',
-    'IPV6_ADDRESS': 'FE80::240:D0FF:FE48:4672',
     'PIHOLE_DNS_1': '4.2.2.1',
     'PIHOLE_DNS_2': '4.2.2.2'
 }

test/test_automated_install.py

@@ -45,8 +45,6 @@ def test_setupVars_are_sourced_to_global_scope(Pihole):
         # Currently debug test function only
         echo "Outputting sourced variables"
         echo "PIHOLE_INTERFACE=${PIHOLE_INTERFACE}"
-        echo "IPV4_ADDRESS=${IPV4_ADDRESS}"
-        echo "IPV6_ADDRESS=${IPV6_ADDRESS}"
         echo "PIHOLE_DNS_1=${PIHOLE_DNS_1}"
         echo "PIHOLE_DNS_2=${PIHOLE_DNS_2}"
     }
@@ -106,31 +104,6 @@ def test_selinux_not_detected(Pihole):
     assert check_selinux.rc == 0
 
 
-def test_installPiholeWeb_fresh_install_no_errors(Pihole):
-    '''
-    confirms all web page assets from Core repo are installed on a fresh build
-    '''
-    installWeb = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    installPiholeWeb
-    ''')
-    expected_stdout = info_box + ' Installing blocking page...'
-    assert expected_stdout in installWeb.stdout
-    expected_stdout = tick_box + (' Creating directory for blocking page, '
-                                  'and copying files')
-    assert expected_stdout in installWeb.stdout
-    expected_stdout = info_box + ' Backing up index.lighttpd.html'
-    assert expected_stdout in installWeb.stdout
-    expected_stdout = ('No default index.lighttpd.html file found... '
-                       'not backing up')
-    assert expected_stdout in installWeb.stdout
-    expected_stdout = tick_box + ' Installing sudoer file'
-    assert expected_stdout in installWeb.stdout
-    web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
-    assert 'index.php' in web_directory
-    assert 'blockingpage.css' in web_directory
-
-
 def test_update_package_cache_success_no_errors(Pihole):
     '''
     confirms package cache was updated without any errors
@@ -524,43 +497,45 @@ def test_IPv6_ULA_GUA_test(Pihole):
     assert expected_stdout in detectPlatform.stdout
 
 
-def test_validate_ip_valid(Pihole):
+def test_validate_ip(Pihole):
     '''
-    Given a valid IP address, valid_ip returns success
+    Tests valid_ip for various IP addresses
     '''
 
+    def test_address(addr, success=True):
         output = Pihole.run('''
         source /opt/pihole/basic-install.sh
-    valid_ip "192.168.1.1"
-    ''')
+        valid_ip "{addr}"
+        '''.format(addr=addr))
 
-    assert output.rc == 0
+        assert output.rc == 0 if success else 1
 
-
-def test_validate_ip_invalid_octet(Pihole):
-    '''
-    Given an invalid IP address (large octet), valid_ip returns an error
-    '''
-
-    output = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    valid_ip "1092.168.1.1"
-    ''')
-
-    assert output.rc == 1
-
-
-def test_validate_ip_invalid_letters(Pihole):
-    '''
-    Given an invalid IP address (contains letters), valid_ip returns an error
-    '''
-
-    output = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    valid_ip "not an IP"
-    ''')
-
-    assert output.rc == 1
+    test_address('192.168.1.1')
+    test_address('127.0.0.1')
+    test_address('255.255.255.255')
+    test_address('255.255.255.256', False)
+    test_address('255.255.256.255', False)
+    test_address('255.256.255.255', False)
+    test_address('256.255.255.255', False)
+    test_address('1092.168.1.1', False)
+    test_address('not an IP', False)
+    test_address('8.8.8.8#', False)
+    test_address('8.8.8.8#0')
+    test_address('8.8.8.8#1')
+    test_address('8.8.8.8#42')
+    test_address('8.8.8.8#888')
+    test_address('8.8.8.8#1337')
+    test_address('8.8.8.8#65535')
+    test_address('8.8.8.8#65536', False)
+    test_address('8.8.8.8#-1', False)
+    test_address('00.0.0.0', False)
+    test_address('010.0.0.0', False)
+    test_address('001.0.0.0', False)
+    test_address('0.0.0.0#00', False)
+    test_address('0.0.0.0#01', False)
+    test_address('0.0.0.0#001', False)
+    test_address('0.0.0.0#0001', False)
+    test_address('0.0.0.0#00001', False)
 
 
 def test_os_check_fails(Pihole):
@@ -595,3 +570,42 @@ def test_os_check_passes(Pihole):
     ''')
     expected_stdout = 'Supported OS detected'
     assert expected_stdout in detectOS.stdout
+
+
+def test_package_manager_has_installer_deps(Pihole):
+    ''' Confirms OS is able to install the required packages for the installer'''
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    output = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    install_dependent_packages ${INSTALLER_DEPS[@]}
+    ''')
+
+    assert 'No package' not in output.stdout  # centos7 still exits 0...
+    assert output.rc == 0
+
+
+def test_package_manager_has_pihole_deps(Pihole):
+    ''' Confirms OS is able to install the required packages for Pi-hole '''
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    output = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    install_dependent_packages ${PIHOLE_DEPS[@]}
+    ''')
+
+    assert 'No package' not in output.stdout  # centos7 still exits 0...
+    assert output.rc == 0
+
+
+def test_package_manager_has_web_deps(Pihole):
+    ''' Confirms OS is able to install the required packages for web '''
+    mock_command('whiptail', {'*': ('', '0')}, Pihole)
+    output = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    distro_check
+    install_dependent_packages ${PIHOLE_WEB_DEPS[@]}
+    ''')
+
+    assert 'No package' not in output.stdout  # centos7 still exits 0...
+    assert output.rc == 0

test/test_centos_7_support.py

@@ -5,56 +5,18 @@ from .conftest import (
 )
 
 
-def test_php_upgrade_default_optout_centos_eq_7(Pihole):
+def test_epel_installed_centos_7(Pihole):
     '''
-    confirms the default behavior to opt-out of installing PHP7 from REMI
+    confirms the EPEL package repository is enabled when installed on CentOS
     '''
     distro_check = Pihole.run('''
     source /opt/pihole/basic-install.sh
     distro_check
     ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
+    expected_stdout = info_box + (' Enabling EPEL package repository '
+                                  '(https://fedoraproject.org/wiki/EPEL)')
     assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-def test_php_upgrade_user_optout_centos_eq_7(Pihole):
-    '''
-    confirms installer behavior when user opt-out of installing PHP7 from REMI
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
+    expected_stdout = tick_box + ' Installed epel-release'
     assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-def test_php_upgrade_user_optin_centos_eq_7(Pihole):
-    '''
-    confirms installer behavior when user opt-in to installing PHP7 from REMI
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Continue for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    assert 'opt-out' not in distro_check.stdout
-    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
-                                  '(https://rpms.remirepo.net)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + (' Remi\'s RPM repository has '
-                                  'been enabled for PHP7')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert remi_package.is_installed
+    epel_package = Pihole.package('epel-release')
+    assert epel_package.is_installed

test/test_centos_8_support.py

@@ -5,61 +5,15 @@ from .conftest import (
 )
 
 
-def test_php_upgrade_default_continue_centos_gte_8(Pihole):
+def test_epel_not_installed_centos_gt7(Pihole):
     '''
-    confirms the latest version of CentOS continues / does not optout
-    (should trigger on CentOS7 only)
+    confirms installer does not attempt to install EPEL repository on CentOS 8+
     '''
     distro_check = Pihole.run('''
     source /opt/pihole/basic-install.sh
     distro_check
     ''')
-    unexpected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS.'
-                                    ' Deprecated PHP may be in use.')
-    assert unexpected_stdout not in distro_check.stdout
-    # ensure remi was not installed on latest CentOS
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
+    assert distro_check.stdout == ''
 
-
-def test_php_upgrade_user_optout_skipped_centos_gte_8(Pihole):
-    '''
-    confirms installer skips user opt-out of installing PHP7 from REMI on
-    latest CentOS (should trigger on CentOS7 only)
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    unexpected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS.'
-                                    ' Deprecated PHP may be in use.')
-    assert unexpected_stdout not in distro_check.stdout
-    # ensure remi was not installed on latest CentOS
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-def test_php_upgrade_user_optin_skipped_centos_gte_8(Pihole):
-    '''
-    confirms installer skips user opt-in to installing PHP7 from REMI on
-    latest CentOS (should trigger on CentOS7 only)
-    (php not currently installed)
-    '''
-    # Whiptail dialog returns Continue for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    assert 'opt-out' not in distro_check.stdout
-    unexpected_stdout = info_box + (' Enabling Remi\'s RPM repository '
-                                    '(https://rpms.remirepo.net)')
-    assert unexpected_stdout not in distro_check.stdout
-    unexpected_stdout = tick_box + (' Remi\'s RPM repository has '
-                                    'been enabled for PHP7')
-    assert unexpected_stdout not in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
+    epel_package = Pihole.package('epel-release')
+    assert not epel_package.is_installed

test/test_centos_common_support.py

@@ -1,120 +0,0 @@
-import pytest
-from .conftest import (
-    tick_box,
-    info_box,
-    cross_box,
-    mock_command,
-)
-
-
-def test_release_supported_version_check_centos(Pihole):
-    '''
-    confirms installer exits on unsupported releases of CentOS
-    '''
-    # modify /etc/redhat-release to mock an unsupported CentOS release
-    Pihole.run('echo "CentOS Linux release 6.9" > /etc/redhat-release')
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = cross_box + (' CentOS 6 is not supported.')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = 'Please update to CentOS release 7 or later'
-    assert expected_stdout in distro_check.stdout
-
-
-def test_enable_epel_repository_centos(Pihole):
-    '''
-    confirms the EPEL package repository is enabled when installed on CentOS
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' Enabling EPEL package repository '
-                                  '(https://fedoraproject.org/wiki/EPEL)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + ' Installed epel-release'
-    assert expected_stdout in distro_check.stdout
-    epel_package = Pihole.package('epel-release')
-    assert epel_package.is_installed
-
-
-def test_php_version_lt_7_detected_upgrade_default_optout_centos(Pihole):
-    '''
-    confirms the default behavior to opt-out of upgrading to PHP7 from REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-def test_php_version_lt_7_detected_upgrade_user_optout_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-out to upgrade to PHP7 via REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    # Whiptail dialog returns Cancel for user prompt
-    mock_command('whiptail', {'*': ('', '1')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed
-
-
-def test_php_version_lt_7_detected_upgrade_user_optin_centos(Pihole):
-    '''
-    confirms installer behavior when user opt-in to upgrade to PHP7 via REMI
-    '''
-    # first we will install the default php version to test installer behavior
-    php_install = Pihole.run('yum install -y php')
-    assert php_install.rc == 0
-    php_package = Pihole.package('php')
-    default_centos_php_version = php_package.version.split('.')[0]
-    if int(default_centos_php_version) >= 7:  # PHP7 is supported/recommended
-        pytest.skip("Test deprecated . Detected default PHP version >= 7")
-    # Whiptail dialog returns Continue for user prompt
-    mock_command('whiptail', {'*': ('', '0')}, Pihole)
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    install_dependent_packages PIHOLE_WEB_DEPS[@]
-    ''')
-    expected_stdout = info_box + (' User opt-out of PHP 7 upgrade on CentOS. '
-                                  'Deprecated PHP may be in use.')
-    assert expected_stdout not in distro_check.stdout
-    expected_stdout = info_box + (' Enabling Remi\'s RPM repository '
-                                  '(https://rpms.remirepo.net)')
-    assert expected_stdout in distro_check.stdout
-    expected_stdout = tick_box + (' Remi\'s RPM repository has '
-                                  'been enabled for PHP7')
-    assert expected_stdout in distro_check.stdout
-    remi_package = Pihole.package('remi-release')
-    assert remi_package.is_installed
-    updated_php_package = Pihole.package('php')
-    updated_php_version = updated_php_package.version.split('.')[0]
-    assert int(updated_php_version) == 7

test/test_fedora_support.py

@@ -1,15 +0,0 @@
-def test_epel_and_remi_not_installed_fedora(Pihole):
-    '''
-    confirms installer does not attempt to install EPEL/REMI repositories
-    on Fedora
-    '''
-    distro_check = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    distro_check
-    ''')
-    assert distro_check.stdout == ''
-
-    epel_package = Pihole.package('epel-release')
-    assert not epel_package.is_installed
-    remi_package = Pihole.package('remi-release')
-    assert not remi_package.is_installed

test/tox.centos_7.ini

@@ -5,4 +5,4 @@ envlist = py37
 whitelist_externals = docker
 deps = -rrequirements.txt
 commands = docker build -f _centos_7.Dockerfile -t pytest_pihole:test_container ../
-           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_common_support.py ./test_centos_7_support.py
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_7_support.py

test/tox.centos_8.ini

@@ -5,4 +5,4 @@ envlist = py37
 whitelist_externals = docker
 deps = -rrequirements.txt
 commands = docker build -f _centos_8.Dockerfile -t pytest_pihole:test_container ../
-           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_common_support.py ./test_centos_8_support.py
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_centos_8_support.py

test/tox.fedora_32.ini

@@ -5,4 +5,4 @@ envlist = py37
 whitelist_externals = docker
 deps = -rrequirements.txt
 commands = docker build -f _fedora_32.Dockerfile -t pytest_pihole:test_container ../
-           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py
+           pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py

test/tox.fedora_33.ini

@@ -4,5 +4,5 @@ envlist = py37
 [testenv]
 whitelist_externals = docker
 deps = -rrequirements.txt
-commands = docker build -f _fedora_31.Dockerfile -t pytest_pihole:test_container ../
+commands = docker build -f _fedora_33.Dockerfile -t pytest_pihole:test_container ../
            pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py