Use runuser command instead of su in init.d script (the runuser command is part of the util-linux package and always available)

Signed-off-by: DL6ER <dl6er@dl6er.de>

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,32 +1,31 @@
-**By submitting this pull request, I confirm the following:** `{please fill any appropriate checkboxes, e.g: [X]}`
+**By submitting this pull request, I confirm the following:** 
+*please fill any appropriate checkboxes, e.g: [X]*
 
-`{Please ensure that your pull request is for the 'development' branch!}`
+- [ ] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md), as well as this entire template.
+- [ ] I have made only one major change in my proposed changes.
+- [ ] I have commented my proposed changes within the code.
+- [ ] I have tested my proposed changes, and have included unit tests where possible.
+- [ ] I am willing to help maintain this change if there are issues with it later.
+- [ ] I give this submission freely and claim no ownership.
+- [ ] It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1)
+- [ ] I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
 
-- [] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md), as well as this entire template.
-- [] I have made only one major change in my proposed changes.
-- [] I have commented my proposed changes within the code.
-- [] I have tested my proposed changes, and have included unit tests where possible.
-- [] I am willing to help maintain this change if there are issues with it later.
-- [] I give this submission freely and claim no ownership.
-- [] It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1)
-- [] I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html))
-- [] I have Signed Off all commits. ([`git commit --signoff`](https://git-scm.com/docs/git-commit#git-commit---signoff))
+Please make sure you [Sign Off](https://github.com/pi-hole/pi-hole/wiki/How-to-signoff-your-commits.) all commits. Pi-hole enforces the [DCO](https://github.com/pi-hole/pi-hole/wiki/Contributing-to-the-project).
 
 ---
-
 **What does this PR aim to accomplish?:**
+*A detailed description, screenshots (if necessary), as well as links to any relevant GitHub issues*
 
-`{A detailed description, screenshots (if necessary), as well as links to any relevant GitHub issues}`
 
 **How does this PR accomplish the above?:**
+*A detailed description (such as a changelog) and screenshots (if necessary) of the implemented fix*
 
-`{A detailed description (such as a changelog) and screenshots (if necessary) of the implemented fix}`
 
 **What documentation changes (if any) are needed to support this PR?:**
+*A detailed list of any necessary changes*
 
-`{A detailed list of any necessary changes}`
 
-> * `{Please delete this quoted section when opening your pull request}`
-> * You must follow the template instructions. Failure to do so will result in your issue being closed.
-> * Please respect that Pi-hole is developed by volunteers, who can only reply in their spare time.
-> * Detail helps us understand an issue quicker, but please ensure it's relevant.
+---
+* You must follow the template instructions. Failure to do so will result in your pull request being closed.
+* Please respect that Pi-hole is developed by volunteers, who can only reply in their spare time.
+

.github/dco.yml

@@ -0,0 +1,2 @@
+require:
+  members: false

.gitignore

@@ -3,3 +3,69 @@
 *.swp
 __pycache__
 .cache
+
+# Created by https://www.gitignore.io/api/jetbrains+iml
+
+### JetBrains+iml ###
+# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
+# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
+
+# All idea files, with execptions
+.idea
+!.idea/codeStyles/*
+!.idea/codeStyleSettings.xml
+
+
+# Sensitive or high-churn files:
+.idea/**/dataSources/
+.idea/**/dataSources.ids
+.idea/**/dataSources.xml
+.idea/**/dataSources.local.xml
+.idea/**/sqlDataSources.xml
+.idea/**/dynamic.xml
+.idea/**/uiDesigner.xml
+
+# Gradle:
+.idea/**/gradle.xml
+.idea/**/libraries
+
+# CMake
+cmake-build-debug/
+
+# Mongo Explorer plugin:
+.idea/**/mongoSettings.xml
+
+## File-based project format:
+*.iws
+
+## Plugin-specific files:
+
+# IntelliJ
+/out/
+
+# mpeltonen/sbt-idea plugin
+.idea_modules/
+
+# JIRA plugin
+atlassian-ide-plugin.xml
+
+# Cursive Clojure plugin
+.idea/replstate.xml
+
+# Ruby plugin and RubyMine
+/.rakeTasks
+
+# Crashlytics plugin (for Android Studio and IntelliJ)
+com_crashlytics_export_strings.xml
+crashlytics.properties
+crashlytics-build.properties
+fabric.properties
+
+### JetBrains+iml Patch ###
+# Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-249601023
+
+*.iml
+.idea/misc.xml
+*.ipr
+
+# End of https://www.gitignore.io/api/jetbrains+iml

.idea/codeStyles/Project.xml

@@ -0,0 +1,13 @@
+<component name="ProjectCodeStyleConfiguration">
+  <code_scheme name="Project" version="173">
+    <option name="OTHER_INDENT_OPTIONS">
+      <value>
+        <option name="INDENT_SIZE" value="2" />
+        <option name="TAB_SIZE" value="2" />
+      </value>
+    </option>
+    <MarkdownNavigatorCodeStyleSettings>
+      <option name="RIGHT_MARGIN" value="72" />
+    </MarkdownNavigatorCodeStyleSettings>
+  </code_scheme>
+</component>

.idea/codeStyles/codeStyleConfig.xml

@@ -0,0 +1,5 @@
+<component name="ProjectCodeStyleConfiguration">
+  <state>
+    <option name="USE_PER_PROJECT_SETTINGS" value="true" />
+  </state>
+</component>

.pullapprove.yml

@@ -1,38 +0,0 @@
-version: 2
-
-always_pending:
-  title_regex: '(WIP|wip)'
-  labels:
-    - wip
-  explanation: 'This PR is a work in progress...'
-
-group_defaults:
-  reset_on_push:
-    enabled: true
-  reject_value: -2
-  approve_regex: '^(Approved|:shipit:|:\+1:|Engage|:taco:)'
-  reject_regex: '^(Rejected|:-1:|Borg)'
-  author_approval:
-    auto: true
-
-
-groups:
-  development:
-    approve_by_comment:
-      enabled: true
-    conditions:
-      branches:
-        - development
-    required: 2
-    teams:
-      - approvers
-
-  master:
-    approve_by_comment:
-      enabled: true
-    conditions:
-      branches:
-        - master
-    required: 4
-    teams:
-      - approvers

.stickler.yml

@@ -0,0 +1,3 @@
+linters:
+  shellcheck:
+    shell: bash

CONTRIBUTING.md

@@ -1,5 +1,3 @@
-_This template was created based on the work of [`udemy-dl`](https://github.com/nishad/udemy-dl/blob/master/LICENSE)._
-
 # Contributors Guide
 
 Please read and understand the contribution guide before creating an issue or pull request.
@@ -28,6 +26,7 @@ When requesting or submitting new features, first consider whether it might be u
 
 - Check the codebase to ensure that your feature doesn't already exist.
 - Check the pull requests to ensure that another person hasn't already submitted the feature or fix.
+- Read and understand the [DCO guidelines](https://github.com/pi-hole/pi-hole/wiki/Contributing-to-the-project) for the project.
 
 ## Technical Requirements
 

README.md

@@ -185,7 +185,7 @@ While quite outdated at this point, [this original blog post about Pi-hole](http
 - [CHiP-hole: Network-wide Ad-blocker](https://www.hackster.io/jacobsalmela/chip-hole-network-wide-ad-blocker-98e037)
 - [Chrome Extension: Pi-Hole List Editor](https://chrome.google.com/webstore/detail/pi-hole-list-editor/hlnoeoejkllgkjbnnnhfolapllcnaglh) ([Source Code](https://github.com/packtloss/pihole-extension))
 - [Splunk: Pi-hole Visualiser](https://splunkbase.splunk.com/app/3023/)
-- [Adblocking with P-hole and Ubuntu 14.04 on VirtualBox](https://hbalagtas.blogspot.com.au/2016/02/adblocking-with-pi-hole-and-ubuntu-1404.html)
+- [Adblocking with Pi-hole and Ubuntu 14.04 on VirtualBox](https://hbalagtas.blogspot.com.au/2016/02/adblocking-with-pi-hole-and-ubuntu-1404.html)
 - [Pi-hole stats in your Mac's menu bar](https://getbitbar.com/plugins/Network/pi-hole.1m.py)
 - [Pi-hole unRAID Template](https://forums.lime-technology.com/topic/36810-support-spants-nodered-mqtt-dashing-couchdb/)
 - [Copernicus: Windows Tray Application](https://github.com/goldbattle/copernicus)
@@ -193,7 +193,7 @@ While quite outdated at this point, [this original blog post about Pi-hole](http
 - [Pi-hole metrics](https://github.com/nlamirault/pihole_exporter) exporter for [Prometheus](https://prometheus.io/)
 - [Magic Mirror with DNS Filtering](https://zonksec.com/blog/magic-mirror-dns-filtering/#dnssoftware)
 - [Pi-hole Droid: Android client](https://github.com/friimaind/pi-hole-droid)
-
+- [Windows DNS Swapper](https://github.com/roots84/DNS-Swapper), see [#1400](https://github.com/pi-hole/pi-hole/issues/1400)
 -----
 
 ## Coverage

advanced/01-pihole.conf

@@ -39,7 +39,7 @@ interface=@INT@
 
 cache-size=10000
 
-log-queries
+log-queries=extra
 log-facility=/var/log/pihole.log
 
 local-ttl=2

advanced/Scripts/chronometer.sh

@@ -15,7 +15,7 @@ pihole-FTL() {
   ftl_port=$(cat /var/run/pihole-FTL.port 2> /dev/null)
   if [[ -n "$ftl_port" ]]; then
     # Open connection to FTL
-    exec 3<>"/dev/tcp/localhost/$ftl_port"
+    exec 3<>"/dev/tcp/127.0.0.1/$ftl_port"
 
     # Test if connection is open
     if { "true" >&3; } 2> /dev/null; then
@@ -122,13 +122,13 @@ get_init_stats() {
   }
 
   # Convert seconds to human-readable format
-  hrSecs() { 
+  hrSecs() {
     day=$(( $1/60/60/24 )); hrs=$(( $1/3600%24 ))
     mins=$(( ($1%3600)/60 )); secs=$(( $1%60 ))
     [[ "$day" -ge "2" ]] && plu="s"
     [[ "$day" -ge "1" ]] && days="$day day${plu}, " || days=""
     printf "%s%02d:%02d:%02d\\n" "$days" "$hrs" "$mins" "$secs"
-  } 
+  }
 
   # Set Colour Codes
   coltable="/opt/pihole/COL_TABLE"
@@ -179,6 +179,7 @@ get_init_stats() {
       90009[2-3]|920093) sys_model=" Zero";; # 512MB
       9000c1) sys_model=" Zero W";; # 512MB
       a02082|a[2-3]2082) sys_model=" 3, Model B";; # 1GB
+      a020d3) sys_model=" 3, Model B+";; # 1GB
       *) sys_model="";;
     esac
     sys_type="Raspberry Pi$sys_model"
@@ -199,7 +200,7 @@ get_init_stats() {
   # Test existence of temperature file
   if [[ -f "/sys/class/thermal/thermal_zone0/temp" ]]; then
     temp_file="/sys/class/thermal/thermal_zone0/temp"
-  elif [[ -f "/sys/class/hwmon/hwmon0/temp1_input" ]]; then 
+  elif [[ -f "/sys/class/hwmon/hwmon0/temp1_input" ]]; then
     temp_file="/sys/class/hwmon/hwmon0/temp1_input"
   else
     temp_file=""
@@ -531,7 +532,7 @@ chronoFunc() {
         sleep 5
       fi
     fi
-    
+
   done
 }
 

advanced/Scripts/list.sh

@@ -10,9 +10,9 @@
 
 # Globals
 basename=pihole
-piholeDir=/etc/${basename}
-whitelist=${piholeDir}/whitelist.txt
-blacklist=${piholeDir}/blacklist.txt
+piholeDir=/etc/"${basename}"
+whitelist="${piholeDir}"/whitelist.txt
+blacklist="${piholeDir}"/blacklist.txt
 readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
 reload=false
 addmode=true
@@ -80,8 +80,13 @@ HandleOther() {
 
 PoplistFile() {
   # Check whitelist file exists, and if not, create it
-  if [[ ! -f ${whitelist} ]]; then
-    touch ${whitelist}
+  if [[ ! -f "${whitelist}" ]]; then
+    touch "${whitelist}"
+  fi
+
+  # Check blacklist file exists, and if not, create it
+  if [[ ! -f "${blacklist}" ]]; then
+    touch "${blacklist}"
   fi
 
   for dom in "${domList[@]}"; do

advanced/Scripts/piholeCheckout.sh

@@ -19,7 +19,6 @@ source "${PI_HOLE_FILES_DIR}/automated install/basic-install.sh"
 # setupVars set in basic-install.sh
 
 source "${setupVars}"
-update="false"
 
 coltable="/opt/pihole/COL_TABLE"
 source ${coltable}
@@ -33,89 +32,6 @@ check_download_exists() {
   fi
 }
 
-FTLinstall() {
-  # Download and install FTL binary
-  local binary
-  binary="${1}"
-  local path
-  path="${2}"
-  local str
-  str="Installing FTL"
-  echo -ne "  ${INFO} ${str}..."
-
-  if curl -sSL --fail "https://ftl.pi-hole.net/${path}" -o "/tmp/${binary}"; then
-    # Get sha1 of the binary we just downloaded for verification.
-    curl -sSL --fail "https://ftl.pi-hole.net/${path}.sha1" -o "/tmp/${binary}.sha1"
-    # Check if we just downloaded text, or a binary file.
-    cd /tmp || return 1
-    if sha1sum --status --quiet -c "${binary}".sha1; then
-      echo -n "transferred... "
-      stop_service pihole-FTL &> /dev/null
-      install -T -m 0755 "/tmp/${binary}" "/usr/bin/pihole-FTL"
-      rm "/tmp/${binary}" "/tmp/${binary}.sha1"
-      start_service pihole-FTL &> /dev/null
-      echo -e "${OVER}  ${TICK} ${str}"
-      return 0
-    else
-      echo -e "${OVER}  ${CROSS} ${str}"
-      echo -e "  ${COL_LIGHT_RED}Error: Download of binary from ftl.pi-hole.net failed${COL_NC}"
-      return 1
-    fi
-  else
-    echo -e "${OVER}  ${CROSS} ${str}"
-    echo -e "  ${COL_LIGHT_RED}Error: URL not found${COL_NC}"
-  fi
-}
-
-get_binary_name() {
-  local machine
-  machine=$(uname -m)
-
-  local str
-  str="Detecting architecture"
-  echo -ne "  ${INFO} ${str}..."
-  if [[ "${machine}" == "arm"* || "${machine}" == *"aarch"* ]]; then
-    # ARM
-    local rev
-    rev=$(uname -m | sed "s/[^0-9]//g;")
-    local lib
-    lib=$(ldd /bin/ls | grep -E '^\s*/lib' | awk '{ print $1 }')
-    if [[ "${lib}" == "/lib/ld-linux-aarch64.so.1" ]]; then
-      echo -e "${OVER}  ${TICK} Detected ARM-aarch64 architecture"
-      binary="pihole-FTL-aarch64-linux-gnu"
-    elif [[ "${lib}" == "/lib/ld-linux-armhf.so.3" ]]; then
-      if [[ "$rev" -gt "6" ]]; then
-        echo -e "${OVER}  ${TICK} Detected ARM-hf architecture (armv7+)"
-        binary="pihole-FTL-arm-linux-gnueabihf"
-      else
-        echo -e "${OVER}  ${TICK} Detected ARM-hf architecture (armv6 or lower) Using ARM binary"
-        binary="pihole-FTL-arm-linux-gnueabi"
-      fi
-    else
-      echo -e "${OVER}  ${TICK} Detected ARM architecture"
-      binary="pihole-FTL-arm-linux-gnueabi"
-    fi
-  elif [[ "${machine}" == "ppc" ]]; then
-    # PowerPC
-    echo -e "${OVER}  ${TICK} Detected PowerPC architecture"
-    binary="pihole-FTL-powerpc-linux-gnu"
-  elif [[ "${machine}" == "x86_64" ]]; then
-    # 64bit
-    echo -e "${OVER}  ${TICK} Detected x86_64 architecture"
-    binary="pihole-FTL-linux-x86_64"
-  else
-    # Something else - we try to use 32bit executable and warn the user
-    if [[ ! "${machine}" == "i686" ]]; then
-      echo -e "${OVER}  ${CROSS} ${str}...
-      ${COL_LIGHT_RED}Not able to detect architecture (unknown: ${machine}), trying 32bit executable
-      Contact support if you experience issues (e.g: FTL not running)${COL_NC}"
-    else
-      echo -e "${OVER}  ${TICK} Detected 32bit (i686) architecture"
-    fi
-    binary="pihole-FTL-linux-x86_32"
-  fi
-}
-
 fully_fetch_repo() {
   # Add upstream branches to shallow clone
   local directory="${1}"
@@ -176,11 +92,6 @@ checkout_pull_branch() {
   git checkout "${branch}" --quiet || return 1
   echo -e "${OVER}  ${TICK} $str"
 
-
-  if [[ "$(git diff "${oldbranch}" | grep -c "^")" -gt "0" ]]; then
-    update="true"
-  fi
-
   git_pull=$(git pull || return 1)
 
   if [[ "$git_pull" == *"up-to-date"* ]]; then
@@ -256,7 +167,7 @@ checkout() {
     get_binary_name
     local path
     path="development/${binary}"
-    FTLinstall "${binary}" "${path}"
+    echo "development" > /etc/pihole/ftlbranch
   elif [[ "${1}" == "master" ]] ; then
     # Shortcut to check out master branches
     echo -e "  ${INFO} Shortcut \"master\" detected - checking out master branches..."
@@ -270,7 +181,7 @@ checkout() {
     get_binary_name
     local path
     path="master/${binary}"
-    FTLinstall "${binary}" "${path}"
+    echo "master" > /etc/pihole/ftlbranch
   elif [[ "${1}" == "core" ]] ; then
     str="Fetching branches from ${piholeGitUrl}"
     echo -ne "  ${INFO} $str"
@@ -332,7 +243,7 @@ checkout() {
 
     if check_download_exists "$path"; then
         echo "  ${TICK} Branch ${2} exists"
-        FTLinstall "${binary}" "${path}"
+        echo "${2}" > /etc/pihole/ftlbranch
     else
         echo "  ${CROSS} Requested branch \"${2}\" is not available"
         ftlbranches=( $(git ls-remote https://github.com/pi-hole/ftl | grep 'heads' | sed 's/refs\/heads\///;s/ //g' | awk '{print $2}') )
@@ -347,7 +258,7 @@ checkout() {
   fi
 
   # Force updating everything
-  if [[ ( ! "${1}" == "web" && ! "${1}" == "ftl" ) && "${update}" == "true" ]]; then
+  if [[  ! "${1}" == "web"  ]]; then
     echo -e "  ${INFO} Running installer to upgrade your installation"
     if "${PI_HOLE_FILES_DIR}/automated install/basic-install.sh" --unattended; then
       exit 0

advanced/Scripts/piholeDebug.sh

@@ -215,14 +215,14 @@ copy_to_debug_log() {
   sed 's/\[[0-9;]\{1,5\}m//g' > "${PIHOLE_DEBUG_LOG_SANITIZED}" <<< cat "${PIHOLE_DEBUG_LOG}"
 }
 
-initiate_debug() {
+initialize_debug() {
   # Clear the screen so the debug log is readable
   clear
   show_disclaimer
   # Display that the debug process is beginning
   log_write "${COL_PURPLE}*** [ INITIALIZING ]${COL_NC}"
   # Timestamp the start of the log
-  log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initiated."
+  log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initialized."
 }
 
 # This is a function for visually displaying the curent test that is being run.
@@ -465,15 +465,15 @@ processor_check() {
   else
     # Check if the architecture is currently supported for FTL
     case "${PROCESSOR}" in
-      "amd64") "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
+      "amd64") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
       ;;
-      "armv6l") "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
+      "armv6l") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
       ;;
-      "armv6") "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
+      "armv6") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
       ;;
-      "armv7l") "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
+      "armv7l") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
       ;;
-      "aarch64") "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
+      "aarch64") log_write "${TICK} ${COL_GREEN}${PROCESSOR}${COL_NC}"
       ;;
     # Otherwise, show the processor type
     *) log_write "${INFO} ${PROCESSOR}";
@@ -547,7 +547,7 @@ detect_ip_addresses() {
     log_write ""
   else
     # If there are no IPs detected, explain that the protocol is not configured
-    log_write "${CROSS} ${COL_RED}No IPv${protocol} address(es) found on the ${PIHOLE_INTERFACE}${COL_NC} interace.\n"
+    log_write "${CROSS} ${COL_RED}No IPv${protocol} address(es) found on the ${PIHOLE_INTERFACE}${COL_NC} interface.\n"
     return 1
   fi
   # If the protocol is v6
@@ -712,20 +712,20 @@ check_x_headers() {
   # If the X-header found by curl matches what is should be,
   if [[ $block_page == "$block_page_working" ]]; then
     # display a success message
-    log_write "$TICK ${COL_GREEN}${block_page}${COL_NC}"
+    log_write "$TICK Block page X-Header: ${COL_GREEN}${block_page}${COL_NC}"
   else
     # Otherwise, show an error
-    log_write "$CROSS ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
+    log_write "$CROSS Block page X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
     log_write "${COL_RED}${full_curl_output_block_page}${COL_NC}"
   fi
 
   # Same logic applies to the dashbord as above, if the X-Header matches what a working system shoud have,
   if [[ $dashboard == "$dashboard_working" ]]; then
     # then we can show a success
-    log_write "$TICK ${COL_GREEN}${dashboard}${COL_NC}"
+    log_write "$TICK Web interface X-Header: ${COL_GREEN}${dashboard}${COL_NC}"
   else
     # Othewise, it's a failure since the X-Headers either don't exist or have been modified in some way
-    log_write "$CROSS ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
+    log_write "$CROSS Web interface X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
     log_write "${COL_RED}${full_curl_output_dashboard}${COL_NC}"
   fi
 }
@@ -809,8 +809,14 @@ process_status(){
   local i
   # For each process,
   for i in "${PIHOLE_PROCESSES[@]}"; do
-    # get its status via systemctl
-    local status_of_process=$(systemctl is-active "${i}")
+    # If systemd
+    if command -v systemctl &> /dev/null; then
+      # get its status via systemctl
+      local status_of_process=$(systemctl is-active "${i}")
+    else
+      # Otherwise, use the service command
+      local status_of_process=$(service "${i}" status | awk '/Active:/ {print $2}') &> /dev/null
+    fi
     # and print it out to the user
     if [[ "${status_of_process}" == "active" ]]; then
       # If it's active, show it in green
@@ -1143,7 +1149,7 @@ upload_to_tricorder() {
 
 # Run through all the functions we made
 make_temporary_log
-initiate_debug
+initialize_debug
 # setupVars.conf needs to be sourced before the networking so the values are
 # available to the other functions
 source_setup_variables

advanced/Scripts/piholeLogFlush.sh

@@ -11,6 +11,20 @@
 colfile="/opt/pihole/COL_TABLE"
 source ${colfile}
 
+# Determine database location
+# Obtain DBFILE=... setting from pihole-FTL.db
+# Constructed to return nothing when
+# a) the setting is not present in the config file, or
+# b) the setting is commented out (e.g. "#DBFILE=...")
+FTLconf="/etc/pihole/pihole-FTL.conf"
+if [ -e "$FTLconf" ]; then
+  DBFILE="$(sed -n -e 's/^\s*DBFILE\s*=\s*//p' ${FTLconf})"
+fi
+# Test for empty string. Use standard path in this case.
+if [ -z "$DBFILE" ]; then
+  DBFILE="/etc/pihole/pihole-FTL.db"
+fi
+
 if [[ "$@" != *"quiet"* ]]; then
   echo -ne "  ${INFO} Flushing /var/log/pihole.log ..."
 fi
@@ -41,8 +55,12 @@ else
       echo " " > /var/log/pihole.log.1
     fi
   fi
+  # Delete most recent 24 hours from FTL's database, leave even older data intact (don't wipe out all history)
+  deleted=$(sqlite3 "${DBFILE}" "DELETE FROM queries WHERE timestamp >= strftime('%s','now')-86400; select changes() from queries limit 1")
+
 fi
 
 if [[ "$@" != *"quiet"* ]]; then
   echo -e "${OVER}  ${TICK} Flushed /var/log/pihole.log"
+  echo -e "  ${TICK} Deleted ${deleted} queries from database"
 fi

advanced/Scripts/update.sh

@@ -28,9 +28,12 @@ source "/opt/pihole/COL_TABLE"
 # make_repo() sourced from basic-install.sh
 # update_repo() source from basic-install.sh
 # getGitFiles() sourced from basic-install.sh
+# get_binary_name() sourced from basic-install.sh
+# FTLcheckUpdate() sourced from basic-install.sh
 
 GitCheckUpdateAvail() {
-  local directory="${1}"
+  local directory
+  directory="${1}"
   curdir=$PWD
   cd "${directory}" || return
 
@@ -77,24 +80,16 @@ GitCheckUpdateAvail() {
   fi
 }
 
-FTLcheckUpdate() {
-	local FTLversion
-	FTLversion=$(/usr/bin/pihole-FTL tag)
-	local FTLlatesttag
-	FTLlatesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep 'Location' | awk -F '/' '{print $NF}' | tr -d '\r\n')
-
-	if [[ "${FTLversion}" != "${FTLlatesttag}" ]]; then
-		return 0
-	else
-		return 1
-	fi
-}
-
 main() {
-  local pihole_version_current
-  local web_version_current
   local basicError="\\n  ${COL_LIGHT_RED}Unable to complete update, please contact Pi-hole Support${COL_NC}"
-  
+  local core_update
+  local web_update
+  local FTL_update
+
+  core_update=false
+  web_update=false
+  FTL_update=false
+
   # shellcheck disable=1090,2154
   source "${setupVars}"
 
@@ -115,24 +110,6 @@ main() {
     echo -e "  ${INFO} Pi-hole Core:\\t${COL_LIGHT_GREEN}up to date${COL_NC}"
   fi
 
-  if FTLcheckUpdate ; then
-    FTL_update=true
-    echo -e "  ${INFO} FTL:\\t\\t${COL_YELLOW}update available${COL_NC}"
-  else
-    FTL_update=false
-    echo -e "  ${INFO} FTL:\\t\\t${COL_LIGHT_GREEN}up to date${COL_NC}"
-  fi
-
-  # Logic: Don't update FTL when there is a core update available
-  # since the core update will run the installer which will itself
-  # re-install (i.e. update) FTL
-  if ${FTL_update} && ! ${core_update}; then
-    echo ""
-    echo -e "  ${INFO} FTL out of date"
-    FTLdetect
-    echo ""
-  fi
-
   if [[ "${INSTALL_WEB}" == true ]]; then
     if ! is_repo "${ADMIN_INTERFACE_DIR}" ; then
       echo -e "\\n  ${COL_LIGHT_RED}Error: Web Admin repo is missing from system!
@@ -147,80 +124,45 @@ main() {
       web_update=false
       echo -e "  ${INFO} Web Interface:\\t${COL_LIGHT_GREEN}up to date${COL_NC}"
     fi
-
-    # Logic
-    # If Core up to date AND web up to date:
-    #            Do nothing
-    # If Core up to date AND web NOT up to date:
-    #            Pull web repo
-    # If Core NOT up to date AND web up to date:
-    #            pull pihole repo, run install --unattended -- reconfigure
-    # if Core NOT up to date AND web NOT up to date:
-    #            pull pihole repo run install --unattended
-
-    if ! ${core_update} && ! ${web_update} ; then
-      if ! ${FTL_update} ; then
-        echo ""
-        echo -e "  ${TICK} Everything is up to date!"
-        exit 0
-      fi
-    elif ! ${core_update} && ${web_update} ; then
-      echo ""
-      echo -e "  ${INFO} Pi-hole Web Admin files out of date"
-      getGitFiles "${ADMIN_INTERFACE_DIR}" "${ADMIN_INTERFACE_GIT_URL}"
-    elif ${core_update} && ! ${web_update} ; then
-      echo ""
-      echo -e "  ${INFO} Pi-hole core files out of date"
-      getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
-      ${PI_HOLE_FILES_DIR}/automated\ install/basic-install.sh --reconfigure --unattended || \
-        echo -e "${basicError}" && exit 1
-    elif ${core_update} && ${web_update} ; then
-      echo ""
-      echo -e "  ${INFO} Updating Pi-hole core and web admin files"
-      getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
-      ${PI_HOLE_FILES_DIR}/automated\ install/basic-install.sh --unattended || \
-        echo -e "${basicError}" && exit 1
-    else
-      echo -e "  ${COL_LIGHT_RED}Update script has malfunctioned, please contact Pi-hole Support${COL_NC}"
-      exit 1
-    fi
-  else # Web Admin not installed, so only verify if core is up to date
-    if ! ${core_update}; then
-      if ! ${FTL_update} ; then
-        echo ""
-        echo -e "  ${INFO} Everything is up to date!"
-        exit 0
-      fi
-    else
-      echo ""
-      echo -e "  ${INFO} Pi-hole Core files out of date"
-      getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
-      ${PI_HOLE_FILES_DIR}/automated\ install/basic-install.sh --reconfigure --unattended || \
-        echo -e "${basicError}" && exit 1
-    fi
   fi
 
-  if [[ "${web_update}" == true ]]; then
-    web_version_current="$(/usr/local/bin/pihole version --admin --current)"
+  if FTLcheckUpdate > /dev/null; then
+    FTL_update=true
+    echo -e "  ${INFO} FTL:\\t\\t${COL_YELLOW}update available${COL_NC}"
+  else
+    FTL_update=false
+    echo -e "  ${INFO} FTL:\\t\\t${COL_LIGHT_GREEN}up to date${COL_NC}"
+  fi
+
+  if [[ "${core_update}" == false && "${web_update}" == false && "${FTL_update}" == false ]]; then
     echo ""
-    echo -e "  ${INFO} Web Admin version is now at ${web_version_current/* v/v}
-  ${INFO} If you had made any changes in '/var/www/html/admin/', they have been stashed using 'git stash'"
+    echo -e "  ${TICK} Everything is up to date!"
+    exit 0
   fi
 
   if [[ "${core_update}" == true ]]; then
-    pihole_version_current="$(/usr/local/bin/pihole version --pihole --current)"
     echo ""
-    echo -e "  ${INFO} Pi-hole version is now at ${pihole_version_current/* v/v}
-  ${INFO} If you had made any changes in '/etc/.pihole/', they have been stashed using 'git stash'"
+    echo -e "  ${INFO} Pi-hole core files out of date, updating local repo."
+    getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
+    echo -e "  ${INFO} If you had made any changes in '/etc/.pihole/', they have been stashed using 'git stash'"
+  fi
+
+  if [[ "${web_update}" == true ]]; then
+    echo ""
+    echo -e "  ${INFO} Pi-hole Web Admin files out of date, updating local repo."
+    getGitFiles "${ADMIN_INTERFACE_DIR}" "${ADMIN_INTERFACE_GIT_URL}"
+    echo -e "  ${INFO} If you had made any changes in '/var/www/html/admin/', they have been stashed using 'git stash'"
   fi
 
   if [[ "${FTL_update}" == true ]]; then
-    FTL_version_current="$(/usr/bin/pihole-FTL tag)"
-    echo -e "\\n  ${INFO} FTL version is now at ${FTL_version_current/* v/v}"
-    start_service pihole-FTL
-    enable_service pihole-FTL
+    echo ""
+    echo -e "  ${INFO} FTL out of date, it will be updated by the installer."
   fi
 
+  if [[ "${FTL_update}" == true || "${core_update}" == true || "${web_update}" == true ]]; then
+    ${PI_HOLE_FILES_DIR}/automated\ install/basic-install.sh --reconfigure --unattended || \
+         echo -e "${basicError}" && exit 1
+  fi
   echo ""
   exit 0
 }

advanced/Scripts/updatecheck.sh

@@ -3,7 +3,7 @@
 # (c) 2017 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
 #
-# Checks for updates via GitHub
+# Checks for local or remote versions and branches
 #
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.
@@ -25,35 +25,42 @@ function json_extract() {
   fi
 }
 
-GITHUB_CORE_VERSION="$(json_extract tag_name "$(curl -q 'https://api.github.com/repos/pi-hole/pi-hole/releases/latest' 2> /dev/null)")"
-GITHUB_WEB_VERSION="$(json_extract tag_name "$(curl -q 'https://api.github.com/repos/pi-hole/AdminLTE/releases/latest' 2> /dev/null)")"
-GITHUB_FTL_VERSION="$(json_extract tag_name "$(curl -q 'https://api.github.com/repos/pi-hole/FTL/releases/latest' 2> /dev/null)")"
-
-echo "${GITHUB_CORE_VERSION} ${GITHUB_WEB_VERSION} ${GITHUB_FTL_VERSION}" > "/etc/pihole/GitHubVersions"
-
 function get_local_branch() {
   # Return active branch
   cd "${1}" 2> /dev/null || return 1
   git rev-parse --abbrev-ref HEAD || return 1
 }
 
-CORE_BRANCH="$(get_local_branch /etc/.pihole)"
-WEB_BRANCH="$(get_local_branch /var/www/html/admin)"
-#FTL_BRANCH="$(pihole-FTL branch)"
-# Don't store FTL branch until the next release of FTL which
-# supports returning the branch in an easy way
-FTL_BRANCH="XXX"
-
-echo "${CORE_BRANCH} ${WEB_BRANCH} ${FTL_BRANCH}" > "/etc/pihole/localbranches"
-
 function get_local_version() {
-  # Return active branch
-  cd "${1}" 2> /dev/null || return 1
-  git describe --long --dirty --tags || return 1
+# Return active branch
+cd "${1}" 2> /dev/null || return 1
+git describe --long --dirty --tags || return 1
 }
 
-CORE_VERSION="$(get_local_version /etc/.pihole)"
-WEB_VERSION="$(get_local_version /var/www/html/admin)"
-FTL_VERSION="$(pihole-FTL version)"
+if [[ "$2" == "remote" ]]; then
 
-echo "${CORE_VERSION} ${WEB_VERSION} ${FTL_VERSION}" > "/etc/pihole/localversions"
+  if [[ "$3" == "reboot" ]]; then
+    sleep 30
+  fi
+
+  GITHUB_CORE_VERSION="$(json_extract tag_name "$(curl -q 'https://api.github.com/repos/pi-hole/pi-hole/releases/latest' 2> /dev/null)")"
+  GITHUB_WEB_VERSION="$(json_extract tag_name "$(curl -q 'https://api.github.com/repos/pi-hole/AdminLTE/releases/latest' 2> /dev/null)")"
+  GITHUB_FTL_VERSION="$(json_extract tag_name "$(curl -q 'https://api.github.com/repos/pi-hole/FTL/releases/latest' 2> /dev/null)")"
+
+  echo -n "${GITHUB_CORE_VERSION} ${GITHUB_WEB_VERSION} ${GITHUB_FTL_VERSION}" > "/etc/pihole/GitHubVersions"
+
+else
+
+  CORE_BRANCH="$(get_local_branch /etc/.pihole)"
+  WEB_BRANCH="$(get_local_branch /var/www/html/admin)"
+  FTL_BRANCH="$(pihole-FTL branch)"
+
+  echo -n "${CORE_BRANCH} ${WEB_BRANCH} ${FTL_BRANCH}" > "/etc/pihole/localbranches"
+
+  CORE_VERSION="$(get_local_version /etc/.pihole)"
+  WEB_VERSION="$(get_local_version /var/www/html/admin)"
+  FTL_VERSION="$(pihole-FTL version)"
+
+  echo -n "${CORE_VERSION} ${WEB_VERSION} ${FTL_VERSION}" > "/etc/pihole/localversions"
+
+fi

advanced/Scripts/webpage.sh

@@ -13,6 +13,7 @@
 readonly setupVars="/etc/pihole/setupVars.conf"
 readonly dnsmasqconfig="/etc/dnsmasq.d/01-pihole.conf"
 readonly dhcpconfig="/etc/dnsmasq.d/02-pihole-dhcp.conf"
+readonly FTLconf="/etc/pihole/pihole-FTL.conf"
 # 03 -> wildcards
 readonly dhcpstaticconfig="/etc/dnsmasq.d/04-pihole-static-dhcp.conf"
 
@@ -35,7 +36,7 @@ Options:
   -e, email           Set an administrative contact address for the Block Page
   -h, --help          Show this help dialog
   -i, interface       Specify dnsmasq's interface listening behavior
-                        Add '-h' for more info on interface usage"
+  -l, privacylevel    Set privacy level (0 = lowest, 3 = highest)"
 	exit 0
 }
 
@@ -52,6 +53,19 @@ change_setting() {
 	add_setting "${1}" "${2}"
 }
 
+addFTLsetting() {
+	echo "${1}=${2}" >> "${FTLconf}"
+}
+
+deleteFTLsetting() {
+	sed -i "/${1}/d" "${FTLconf}"
+}
+
+changeFTLsetting() {
+	deleteFTLsetting "${1}"
+	addFTLsetting "${1}" "${2}"
+}
+
 add_dnsmasq_setting() {
 	if [[ "${2}" != "" ]]; then
 		echo "${1}=${2}" >> "${dnsmasqconfig}"
@@ -153,6 +167,7 @@ ProcessDNSSettings() {
 	if [[ "${DNSSEC}" == true ]]; then
 		echo "dnssec
 trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5
+trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC683457104237C7F8EC8D
 " >> "${dnsmasqconfig}"
 	fi
 
@@ -174,8 +189,17 @@ trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE3
 		add_dnsmasq_setting "local-service"
 	else
 		# Listen only on one interface
+		# Use eth0 as fallback interface if interface is missing in setupVars.conf
+		if [ -z "${PIHOLE_INTERFACE}" ]; then
+			PIHOLE_INTERFACE="eth0"
+		fi
+
 		add_dnsmasq_setting "interface" "${PIHOLE_INTERFACE}"
 	fi
+	if [[ "${CONDITIONAL_FORWARDING}" == true ]]; then
+		add_dnsmasq_setting "server=/${CONDITIONAL_FORWARDING_DOMAIN}/${CONDITIONAL_FORWARDING_IP}"
+		add_dnsmasq_setting "server=/${CONDITIONAL_FORWARDING_REVERSE}/${CONDITIONAL_FORWARDING_IP}"
+	fi
 
 }
 
@@ -205,6 +229,17 @@ SetDNSServers() {
 	else
 		change_setting "DNSSEC" "false"
 	fi
+	if [[ "${args[6]}" == "conditional_forwarding" ]]; then
+		change_setting "CONDITIONAL_FORWARDING" "true"
+		change_setting "CONDITIONAL_FORWARDING_IP" "${args[7]}"
+		change_setting "CONDITIONAL_FORWARDING_DOMAIN" "${args[8]}"
+		change_setting "CONDITIONAL_FORWARDING_REVERSE" "${args[9]}"
+	else
+		change_setting "CONDITIONAL_FORWARDING" "false"
+		delete_setting "CONDITIONAL_FORWARDING_IP"
+		delete_setting "CONDITIONAL_FORWARDING_DOMAIN"
+		delete_setting "CONDITIONAL_FORWARDING_REVERSE"
+	fi
 
 	ProcessDNSSettings
 
@@ -240,7 +275,7 @@ ProcessDHCPSettings() {
 	source "${setupVars}"
 
 	if [[ "${DHCP_ACTIVE}" == "true" ]]; then
-    interface=$(grep 'PIHOLE_INTERFACE=' /etc/pihole/setupVars.conf | sed "s/.*=//")
+    interface="${PIHOLE_INTERFACE}"
 
     # Use eth0 as fallback interface
     if [ -z ${interface} ]; then
@@ -248,7 +283,7 @@ ProcessDHCPSettings() {
     fi
 
     if [[ "${PIHOLE_DOMAIN}" == "" ]]; then
-      PIHOLE_DOMAIN="local"
+      PIHOLE_DOMAIN="lan"
       change_setting "PIHOLE_DOMAIN" "${PIHOLE_DOMAIN}"
     fi
 
@@ -484,36 +519,44 @@ audit()
 	echo "${args[2]}" >> /etc/pihole/auditlog.list
 }
 
+SetPrivacyLevel() {
+	# Set privacy level. Minimum is 0, maximum is 3
+	if [ "${args[2]}" -ge 0 ] && [ "${args[2]}" -le 3 ]; then
+		changeFTLsetting "PRIVACYLEVEL" "${args[2]}"
+	fi
+}
+
 main() {
 	args=("$@")
 
 	case "${args[1]}" in
-		"-p" | "password"   ) SetWebPassword;;
-		"-c" | "celsius"    ) unit="C"; SetTemperatureUnit;;
-		"-f" | "fahrenheit" ) unit="F"; SetTemperatureUnit;;
-		"-k" | "kelvin"     ) unit="K"; SetTemperatureUnit;;
-		"setdns"            ) SetDNSServers;;
-		"setexcludedomains" ) SetExcludeDomains;;
-		"setexcludeclients" ) SetExcludeClients;;
-		"poweroff"          ) Poweroff;;
-		"reboot"            ) Reboot;;
-		"restartdns"        ) RestartDNS;;
-		"setquerylog"       ) SetQueryLogOptions;;
-		"enabledhcp"        ) EnableDHCP;;
-		"disabledhcp"       ) DisableDHCP;;
-		"layout"            ) SetWebUILayout;;
-		"-h" | "--help"     ) helpFunc;;
-		"privacymode"       ) SetPrivacyMode;;
-		"resolve"           ) ResolutionSettings;;
-		"addstaticdhcp"     ) AddDHCPStaticAddress;;
-		"removestaticdhcp"  ) RemoveDHCPStaticAddress;;
-		"-r" | "hostrecord" ) SetHostRecord "$3";;
-		"-e" | "email"      ) SetAdminEmail "$3";;
-		"-i" | "interface"  ) SetListeningMode "$@";;
-		"-t" | "teleporter" ) Teleporter;;
-		"adlist"            ) CustomizeAdLists;;
-		"audit"             ) audit;;
-		*                   ) helpFunc;;
+		"-p" | "password"     ) SetWebPassword;;
+		"-c" | "celsius"      ) unit="C"; SetTemperatureUnit;;
+		"-f" | "fahrenheit"   ) unit="F"; SetTemperatureUnit;;
+		"-k" | "kelvin"       ) unit="K"; SetTemperatureUnit;;
+		"setdns"              ) SetDNSServers;;
+		"setexcludedomains"   ) SetExcludeDomains;;
+		"setexcludeclients"   ) SetExcludeClients;;
+		"poweroff"            ) Poweroff;;
+		"reboot"              ) Reboot;;
+		"restartdns"          ) RestartDNS;;
+		"setquerylog"         ) SetQueryLogOptions;;
+		"enabledhcp"          ) EnableDHCP;;
+		"disabledhcp"         ) DisableDHCP;;
+		"layout"              ) SetWebUILayout;;
+		"-h" | "--help"       ) helpFunc;;
+		"privacymode"         ) SetPrivacyMode;;
+		"resolve"             ) ResolutionSettings;;
+		"addstaticdhcp"       ) AddDHCPStaticAddress;;
+		"removestaticdhcp"    ) RemoveDHCPStaticAddress;;
+		"-r" | "hostrecord"   ) SetHostRecord "$3";;
+		"-e" | "email"        ) SetAdminEmail "$3";;
+		"-i" | "interface"    ) SetListeningMode "$@";;
+		"-t" | "teleporter"   ) Teleporter;;
+		"adlist"              ) CustomizeAdLists;;
+		"audit"               ) audit;;
+		"-l" | "privacylevel" ) SetPrivacyLevel;;
+		*                     ) helpFunc;;
 	esac
 
 	shift

advanced/index.php

@@ -64,7 +64,7 @@ if ($serverName === "pi.hole") {
     <html><head>
         $viewPort
         <link rel='stylesheet' href='/pihole/blockingpage.css' type='text/css'/>
-    </head><body id='splashpage'><img src='/admin/img/logo.svg'/><br/>Pi-<b>hole</b>: Your black hole for Internet advertisements</body></html>
+    </head><body id='splashpage'><img src='/admin/img/logo.svg'/><br/>Pi-<b>hole</b>: Your black hole for Internet advertisements<br><a href='/admin'>Did you mean to go to the admin panel?</a></body></html>
     ";
 
     // Set splash/landing page based off presence of $landPage
@@ -98,9 +98,6 @@ if ($serverName === "pi.hole") {
 
 /* Start processing Block Page from here */
 
-// Determine placeholder text based off $svPasswd presence
-$wlPlaceHolder = empty($svPasswd) ? "No admin password set" : "Javascript disabled";
-
 // Define admin email address text based off $svEmail presence
 $bpAskAdmin = !empty($svEmail) ? '<a href="mailto:'.$svEmail.'?subject=Site Blocked: '.$serverName.'"></a>' : "<span/>";
 
@@ -213,6 +210,8 @@ if (explode("-", $phVersion)[1] != "0")
 
 // Please Note: Text is added via CSS to allow an admin to provide a localised
 // language without the need to edit this file
+
+setHeader();
 ?>
 <!DOCTYPE html>
 <!-- Pi-hole: A black hole for Internet advertisements
@@ -224,7 +223,6 @@ if (explode("-", $phVersion)[1] != "0")
 <head>
   <meta charset="UTF-8">
   <?=$viewPort ?>
-  <?=setHeader() ?>
   <meta name="robots" content="noindex,nofollow"/>
   <meta http-equiv="x-dns-prefetch-control" content="off">
   <link rel="shortcut icon" href="<?=$proto ?>://pi.hole/admin/img/favicon.png" type="image/x-icon"/>
@@ -235,11 +233,21 @@ if (explode("-", $phVersion)[1] != "0")
     window.onload = function () {
       <?php
       // Remove href fallback from "Back to safety" button
-      if ($featuredTotal > 0) echo '$("#bpBack").removeAttr("href");';
-      // Enable whitelisting if $svPasswd is present & JS is available
-      if (!empty($svPasswd) && $featuredTotal > 0) {
-          echo '$("#bpWLPassword, #bpWhitelist").prop("disabled", false);';
+      if ($featuredTotal > 0) {
+        echo '$("#bpBack").removeAttr("href");';
+
+        // Enable whitelisting if JS is available
+        echo '$("#bpWhitelist").prop("disabled", false);';
+
+        // Enable password input if necessary
+        if (!empty($svPasswd)) {
           echo '$("#bpWLPassword").attr("placeholder", "Password");';
+          echo '$("#bpWLPassword").prop("disabled", false);';
+        }
+        // Otherwise hide the input
+        else {
+          echo '$("#bpWLPassword").hide();';
+        }
       }
       ?>
     }
@@ -293,7 +301,7 @@ if (explode("-", $phVersion)[1] != "0")
 
     <form id="bpWLButtons" class="buttons">
       <input id="bpWLDomain" type="text" value="<?=$serverName ?>" disabled/>
-      <input id="bpWLPassword" type="password" placeholder="<?=$wlPlaceHolder ?>" disabled/><button id="bpWhitelist" type="button" disabled></button>
+      <input id="bpWLPassword" type="password" placeholder="Javascript disabled" disabled/><button id="bpWhitelist" type="button" disabled></button>
     </form>
   </div>
 </main>

advanced/pihole-FTL.initd

@@ -20,15 +20,23 @@ is_running() {
     ps "$(get_pid)" > /dev/null 2>&1
 }
 
+
 # Start the service
 start() {
   if is_running; then
     echo "pihole-FTL is already running"
   else
-    touch /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
-    chown pihole:pihole /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /etc/pihole
-    chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
-    su -s /bin/sh -c "/usr/bin/pihole-FTL" "$FTLUSER"
+    touch /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole.log
+    mkdir -p /var/run/pihole
+    mkdir -p /var/log/pihole
+    chown pihole:pihole /var/run/pihole /var/log/pihole
+    rm /var/run/pihole/FTL.sock 2> /dev/null
+    chown pihole:pihole /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
+    chown pihole:pihole /etc/pihole /etc/pihole/dhcp.leases /var/log/pihole.log
+    chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole.log
+    setcap CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_NET_ADMIN+eip "$(which pihole-FTL)"
+    echo "nameserver 127.0.0.1" | /sbin/resolvconf -a lo.piholeFTL
+    runuser -u "$FTLUSER" "/usr/bin/pihole-FTL"
     echo
   fi
 }
@@ -36,6 +44,7 @@ start() {
 # Stop the service
 stop() {
   if is_running; then
+    /sbin/resolvconf -d lo.piholeFTL
     kill "$(get_pid)"
     for i in {1..5}; do
       if ! is_running; then

advanced/pihole-FTL.systemd

@@ -0,0 +1,45 @@
+[Unit]
+Description=Pi-hole FTLDNS
+; This unit is supposed to indicate when network functionality is available, but it is only
+; very weakly defined what that is supposed to mean, with one exception: at shutdown, a unit
+; that is ordered after network.target will be stopped before the network
+After=network.target
+; A target that should be used as synchronization point for all host/network name service lookups.
+; All services for which the availability of full host/network name resolution is essential should
+; be ordered after this target, but not pull it in.
+Wants=nss-lookup.target
+Before=nss-lookup.target
+
+[Service]
+Restart=on-abnormal
+User=pihole
+Group=pihole
+PermissionsStartOnly=true
+
+Type=forking
+PIDFile=/run/pihole-FTL.pid
+
+ExecStartPre=/bin/touch /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole.log
+ExecStartPre=/bin/mkdir -p /var/run/pihole /var/log/pihole
+ExecStartPre=/bin/chown pihole:pihole /var/run/pihole /var/log/pihole
+ExecStartPre=-/bin/rm /var/run/pihole/FTL.sock
+ExecStartPre=/bin/chown pihole:pihole /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /etc/pihole /etc/pihole/dhcp.leases /var/log/pihole.log
+ExecStartPre=/bin/chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port /var/log/pihole.log
+ExecStartPre=/bin/echo "nameserver 127.0.0.1" | /sbin/resolvconf -a lo.piholeFTL
+
+ExecStart=/usr/bin/pihole-FTL
+RestartSec=30s
+ExecReload=/bin/kill -HUP $MAINPID
+
+; Use graceful shutdown with a reasonable timeout
+TimeoutStopSec=10s
+
+; Make /usr, /boot, /etc and possibly some more folders read-only...
+ProtectSystem=full
+; ... except /etc/pihole
+; This merely retains r/w access rights, it does not add any new.
+; Must still be writable on the host!
+ReadWriteDirectories=/etc/pihole
+
+[Install]
+WantedBy=multi-user.target

advanced/pihole.cron

@@ -18,9 +18,6 @@
 #          early morning. Download any updates from the adlists
 59 1    * * 7   root    PATH="$PATH:/usr/local/bin/" pihole updateGravity
 
-# Pi-hole: Update Pi-hole! Uncomment to enable auto update
-#30 2    * * 7   root    PATH="$PATH:/usr/local/bin/" pihole updatePihole
-
 # Pi-hole: Flush the log daily at 00:00
 #          The flush script will use logrotate if available
 #          parameter "once": logrotate only once (default is twice)
@@ -29,5 +26,9 @@
 
 @reboot root /usr/sbin/logrotate /etc/pihole/logrotate
 
-# Pi-hole: Grab remote version and branch every 10 minutes
-*/10 *  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker
+# Pi-hole: Grab local version and branch every 10 minutes
+*/10 *  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker local
+
+# Pi-hole: Grab remote version every 24 hours
+59 17  * * *   root    PATH="$PATH:/usr/local/bin/" pihole updatechecker remote
+@reboot root    PATH="$PATH:/usr/local/bin/" pihole updatechecker remote reboot

automated install/basic-install.sh

@@ -2,7 +2,7 @@
 # shellcheck disable=SC1090
 
 # Pi-hole: A black hole for Internet advertisements
-# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
+# (c) 2017-2018 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
 #
 # Installs and Updates Pi-hole
@@ -14,7 +14,7 @@
 #
 # Install with this command (from your Linux machine):
 #
-# curl -L install.pi-hole.net | bash
+# curl -sSL https://install.pi-hole.net | bash
 
 # -e option instructs bash to immediately exit if any command [1] has a non-zero exit status
 # We do not want users to end up with a partially working install, so we exit the script
@@ -23,14 +23,13 @@ set -e
 
 ######## VARIABLES #########
 # For better maintainability, we store as much information that can change in variables
-# This allows us to make a change in one place that can propogate to all instances of the variable
+# This allows us to make a change in one place that can propagate to all instances of the variable
 # These variables should all be GLOBAL variables, written in CAPS
 # Local variables will be in lowercase and will exist only within functions
 # It's still a work in progress, so you may see some variance in this guideline until it is complete
 
-# We write to a temporary file before moving the log to the pihole folder
-tmpLog=/tmp/pihole-install.log
-instalLogLoc=/etc/pihole/install.log
+# Location for final installation log storage
+installLogLoc=/etc/pihole/install.log
 # This is an important file as it contains information specific to the machine it's being installed on
 setupVars=/etc/pihole/setupVars.conf
 # Pi-hole uses lighttpd as a Web server, and this is the config file for it
@@ -44,7 +43,7 @@ webInterfaceGitUrl="https://github.com/pi-hole/AdminLTE.git"
 webInterfaceDir="/var/www/html/admin"
 piholeGitUrl="https://github.com/pi-hole/pi-hole.git"
 PI_HOLE_LOCAL_REPO="/etc/.pihole"
-# These are the names of piholes files, stored in an array
+# These are the names of pi-holes files, stored in an array
 PI_HOLE_FILES=(chronometer list piholeDebug piholeLogFlush setupLCD update version gravity uninstall webpage)
 # This folder is where the Pi-hole scripts will be installed
 PI_HOLE_INSTALL_DIR="/opt/pihole"
@@ -82,7 +81,7 @@ runUnattended=false
 if [[ -f "${coltable}" ]]; then
   # source it
   source ${coltable}
-# Othwerise,
+# Otherwise,
 else
   # Set these values so the installer can still run in color
   COL_NC='\e[0m' # No Color
@@ -164,7 +163,7 @@ if command -v apt-get &> /dev/null; then
   # These programs are stored in an array so they can be looped through later
   INSTALLER_DEPS=(apt-utils dialog debconf dhcpcd5 git ${iproute_pkg} whiptail)
   # Pi-hole itself has several dependencies that also need to be installed
-  PIHOLE_DEPS=(bc cron curl dnsmasq dnsutils iputils-ping lsof netcat sudo unzip wget idn2)
+  PIHOLE_DEPS=(bc cron curl dnsutils iputils-ping lsof netcat sudo unzip wget idn2 sqlite3 libcap2-bin dns-root-data resolvconf)
   # The Web dashboard has some that also need to be installed
   # It's useful to separate the two since our repos are also setup as "Core" code and "Web" code
   PIHOLE_WEB_DEPS=(lighttpd ${phpVer}-common ${phpVer}-cgi ${phpVer}-${phpSqlite})
@@ -174,8 +173,6 @@ if command -v apt-get &> /dev/null; then
   LIGHTTPD_GROUP="www-data"
   # and config file
   LIGHTTPD_CFG="lighttpd.conf.debian"
-  # The DNS server user
-  DNSMASQ_USER="dnsmasq"
 
   # A function to check...
   test_dpkg_lock() {
@@ -208,15 +205,15 @@ elif command -v rpm &> /dev/null; then
   PKG_INSTALL=(${PKG_MANAGER} install -y)
   PKG_COUNT="${PKG_MANAGER} check-update | egrep '(.i686|.x86|.noarch|.arm|.src)' | wc -l"
   INSTALLER_DEPS=(dialog git iproute net-tools newt procps-ng)
-  PIHOLE_DEPS=(bc bind-utils cronie curl dnsmasq findutils nmap-ncat sudo unzip wget libidn2 psmisc)
+  PIHOLE_DEPS=(bc bind-utils cronie curl findutils nmap-ncat sudo unzip wget libidn2 psmisc)
   PIHOLE_WEB_DEPS=(lighttpd lighttpd-fastcgi php php-common php-cli php-pdo)
-  if ! grep -q 'Fedora' /etc/redhat-release; then
+  # EPEL (https://fedoraproject.org/wiki/EPEL) is required for lighttpd on CentOS
+  if grep -qi 'centos' /etc/redhat-release; then
     INSTALLER_DEPS=("${INSTALLER_DEPS[@]}" "epel-release");
   fi
     LIGHTTPD_USER="lighttpd"
     LIGHTTPD_GROUP="lighttpd"
     LIGHTTPD_CFG="lighttpd.conf.fedora"
-    DNSMASQ_USER="nobody"
 
 # If neither apt-get or rmp/dnf are found
 else
@@ -690,13 +687,13 @@ setStaticIPv4() {
   elif [[ -f "/etc/sysconfig/network-scripts/ifcfg-${PIHOLE_INTERFACE}" ]];then
     # If it exists,
     IFCFG_FILE=/etc/sysconfig/network-scripts/ifcfg-${PIHOLE_INTERFACE}
+    IPADDR=$(echo "${IPV4_ADDRESS}" | cut -f1 -d/)
     # check if the desired IP is already set
-    if grep -q "${IPV4_ADDRESS}" "${IFCFG_FILE}"; then
+    if grep -q "${IPADDR}" "${IFCFG_FILE}"; then
       echo -e "  ${INFO} Static IP already configured"
     # Otherwise,
     else
       # Put the IP in variables without the CIDR notation
-      IPADDR=$(echo "${IPV4_ADDRESS}" | cut -f1 -d/)
       CIDR=$(echo "${IPV4_ADDRESS}" | cut -f2 -d/)
       # Backup existing interface configuration:
       cp "${IFCFG_FILE}" "${IFCFG_FILE}".pihole.orig
@@ -715,8 +712,8 @@ setStaticIPv4() {
       }> "${IFCFG_FILE}"
       # Use ip to immediately set the new address
       ip addr replace dev "${PIHOLE_INTERFACE}" "${IPV4_ADDRESS}"
-      # If NetworkMangler command line interface exists,
-      if command -v nmcli &> /dev/null;then
+      # If NetworkMangler command line interface exists and ready to mangle,
+      if command -v nmcli &> /dev/null && nmcli general status &> /dev/null; then
         # Tell NetworkManagler to read our new sysconfig file
         nmcli con load "${IFCFG_FILE}" > /dev/null
       fi
@@ -771,6 +768,8 @@ setDNS() {
       Comodo ""
       DNSWatch ""
       Quad9 ""
+      FamilyShield ""
+      Cloudflare ""
       Custom "")
   # In a whiptail dialog, show the options
   DNSchoices=$(whiptail --separate-output --menu "Select Upstream DNS Provider. To use your own, select Custom." ${r} ${c} 7 \
@@ -815,6 +814,17 @@ setDNS() {
     Quad9)
       echo "Quad9 servers"
       PIHOLE_DNS_1="9.9.9.9"
+      PIHOLE_DNS_2="149.112.112.112"
+      ;;
+    FamilyShield)
+      echo "FamilyShield servers"
+      PIHOLE_DNS_1="208.67.222.123"
+      PIHOLE_DNS_2="208.67.220.123"
+      ;;
+    Cloudflare)
+      echo "Cloudflare servers"
+      PIHOLE_DNS_1="1.1.1.1"
+      PIHOLE_DNS_2="1.0.0.1"
       ;;
     Custom)
       # Until the DNS settings are selected,
@@ -917,7 +927,7 @@ setLogging() {
     esac
 }
 
-# Funtion to ask the user if they want to install the dashboard
+# Function to ask the user if they want to install the dashboard
 setAdminFlag() {
   # Local, named variables
   local WebToggleCommand
@@ -945,7 +955,7 @@ setAdminFlag() {
     esac
 }
 
-# Check if /etc/dnsmasq.conf is from pihole.  If so replace with an original and install new in .d directory
+# Check if /etc/dnsmasq.conf is from pi-hole.  If so replace with an original and install new in .d directory
 version_check_dnsmasq() {
   # Local, named variables
   local dnsmasq_conf="/etc/dnsmasq.conf"
@@ -983,6 +993,10 @@ version_check_dnsmasq() {
   fi
 
   echo -en "  ${INFO} Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf..."
+  # Check to see if dnsmasq directory exists (it may not due to being a fresh install and dnsmasq no longer being a dependency)
+  if [[ ! -d "/etc/dnsmasq.d"  ]];then
+    mkdir "/etc/dnsmasq.d"
+  fi
   # Copy the new Pi-hole DNS config file into the dnsmasq.d directory
   cp ${dnsmasq_pihole_01_snippet} ${dnsmasq_pihole_01_location}
   echo -e "${OVER}  ${TICK} Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf"
@@ -1111,7 +1125,6 @@ stop_service() {
   # Stop service passed in as argument.
   # Can softfail, as process may not be installed when this is called
   local str="Stopping ${1} service"
-  echo ""
   echo -ne "  ${INFO} ${str}..."
   if command -v systemctl &> /dev/null; then
     systemctl stop "${1}" &> /dev/null || true
@@ -1125,7 +1138,6 @@ stop_service() {
 start_service() {
   # Local, named variables
   local str="Starting ${1} service"
-  echo ""
   echo -ne "  ${INFO} ${str}..."
   # If systemctl exists,
   if command -v systemctl &> /dev/null; then
@@ -1143,13 +1155,12 @@ start_service() {
 enable_service() {
   # Local, named variables
   local str="Enabling ${1} service to start on reboot"
-  echo ""
   echo -ne "  ${INFO} ${str}..."
   # If systemctl exists,
   if command -v systemctl &> /dev/null; then
     # use that to enable the service
     systemctl enable "${1}" &> /dev/null
-  # Othwerwise,
+  # Otherwise,
   else
     # use update-rc.d to accomplish this
     update-rc.d "${1}" defaults &> /dev/null
@@ -1157,6 +1168,35 @@ enable_service() {
   echo -e "${OVER}  ${TICK} ${str}"
 }
 
+# Disable service so that it will not with next reboot
+disable_service() {
+  # Local, named variables
+  local str="Disabling ${1} service"
+  echo -ne "  ${INFO} ${str}..."
+  # If systemctl exists,
+  if command -v systemctl &> /dev/null; then
+    # use that to disable the service
+    systemctl disable "${1}" &> /dev/null
+  # Otherwise,
+  else
+    # use update-rc.d to accomplish this
+    update-rc.d "${1}" disable &> /dev/null
+  fi
+  echo -e "${OVER}  ${TICK} ${str}"
+}
+
+check_service_active() {
+    # If systemctl exists,
+  if command -v systemctl &> /dev/null; then
+    # use that to check the status of the service
+    systemctl is-enabled "${1}" > /dev/null
+  # Otherwise,
+  else
+    # fall back to service command
+    service "${1}" status > /dev/null
+  fi
+}
+
 update_package_cache() {
   # Running apt-get update/upgrade with minimal output can cause some issues with
   # requiring user input (e.g password for phpmyadmin see #218)
@@ -1243,7 +1283,7 @@ install_dependent_packages() {
         echo -e "${OVER}  ${TICK} Checking for $i"
       else
         #
-        echo -e "${OVER}  ${CROSS} Checking for $i (will be installed)"
+        echo -e "${OVER}  ${INFO} Checking for $i (will be installed)"
         #
         installArray+=("${i}")
       fi
@@ -1268,7 +1308,7 @@ install_dependent_packages() {
     if ${PKG_MANAGER} -q list installed "${i}" &> /dev/null; then
       echo -e "${OVER}  ${TICK} Checking for $i"
     else
-      echo -e "${OVER}  ${CROSS} Checking for $i (will be installed)"
+      echo -e "${OVER}  ${INFO} Checking for $i (will be installed)"
       #
       installArray+=("${i}")
     fi
@@ -1283,27 +1323,6 @@ install_dependent_packages() {
   return 0
 }
 
-# Create logfiles if necessary
-CreateLogFile() {
-  local str="Creating log and changing owner to dnsmasq"
-  echo ""
-  echo -ne "  ${INFO} ${str}..."
-  # If the pihole log does not exist,
-  if [[ ! -f "/var/log/pihole.log" ]]; then
-    # Make it,
-    touch /var/log/pihole.log
-    # set the permissions,
-    chmod 644 /var/log/pihole.log
-    # and owners
-    chown "${DNSMASQ_USER}":root /var/log/pihole.log
-    echo -e "${OVER}  ${TICK} ${str}"
-  # Otherwise,
-  else
-    # the file should already exist
-    echo -e " ${COL_LIGHT_GREEN}log already exists!${COL_NC}"
-  fi
-}
-
 # Install the Web interface dashboard
 installPiholeWeb() {
   echo ""
@@ -1330,7 +1349,7 @@ installPiholeWeb() {
     # back it up
     mv /var/www/html/index.lighttpd.html /var/www/html/index.lighttpd.orig
     echo -e "${OVER}  ${TICK} ${str}"
-  # Othwerwise,
+  # Otherwise,
   else
     # don't do anything
     echo -e "${OVER}  ${CROSS} ${str}
@@ -1368,29 +1387,17 @@ installCron() {
   # Copy the cron file over from the local repo
   cp ${PI_HOLE_LOCAL_REPO}/advanced/pihole.cron /etc/cron.d/pihole
   # Randomize gravity update time
-  sed -i "s/59 1/$((1 + RANDOM % 58)) $((3 + RANDOM % 2))/" /etc/cron.d/pihole
+  sed -i "s/59 1 /$((1 + RANDOM % 58)) $((3 + RANDOM % 2))/" /etc/cron.d/pihole
+  # Randomize update checker time
+  sed -i "s/59 17/$((1 + RANDOM % 58)) $((12 + RANDOM % 8))/" /etc/cron.d/pihole
   echo -e "${OVER}  ${TICK} ${str}"
 }
 
 # Gravity is a very important script as it aggregates all of the domains into a single HOSTS formatted list,
 # which is what Pi-hole needs to begin blocking ads
 runGravity() {
-  echo ""
-  echo -e "  ${INFO} Preparing to run gravity.sh to refresh hosts..."
-  # If cached lists exist,
-  if ls /etc/pihole/list* 1> /dev/null 2>&1; then
-    echo -e "  ${INFO} Cleaning up previous install (preserving whitelist/blacklist)"
-    # remove them
-    rm /etc/pihole/list.*
-  fi
-  # If the default ad lists file exists,
-  if [[ ! -e /etc/pihole/adlists.default ]]; then
-    # copy it over from the local repo
-    cp ${PI_HOLE_LOCAL_REPO}/adlists.default /etc/pihole/adlists.default
-  fi
-  echo -e "  ${INFO} Running gravity.sh"
   # Run gravity in the current shell
-  { /opt/pihole/gravity.sh; }
+  { /opt/pihole/gravity.sh --force; }
 }
 
 # Check if the pihole user exists and create if it does not
@@ -1401,7 +1408,7 @@ create_pihole_user() {
   if id -u pihole &> /dev/null; then
     # just show a success
     echo -ne "${OVER}  ${TICK} ${str}"
-  # Othwerwise,
+  # Otherwise,
   else
     echo -ne "${OVER}  ${CROSS} ${str}"
     local str="Creating user 'pihole'"
@@ -1420,7 +1427,7 @@ configureFirewall() {
     # ask if the user wants to install Pi-hole's default firwall rules
     whiptail --title "Firewall in use" --yesno "We have detected a running firewall\\n\\nPi-hole currently requires HTTP and DNS port access.\\n\\n\\n\\nInstall Pi-hole default firewall rules?" ${r} ${c} || \
     { echo -e "  ${INFO} Not installing firewall rulesets."; return 0; }
-    echo -e "  ${TICK} Configuring FirewallD for httpd and dnsmasq"
+    echo -e "  ${TICK} Configuring FirewallD for httpd and pihole-FTL"
     # Allow HTTP and DNS traffice
     firewall-cmd --permanent --add-service=http --add-service=dns
     # Reload the firewall to apply these changes
@@ -1441,7 +1448,7 @@ configureFirewall() {
       iptables -C INPUT -p tcp -m tcp --dport 4711:4720 -i lo -j ACCEPT &> /dev/null || iptables -I INPUT 1 -p tcp -m tcp --dport 4711:4720 -i lo -j ACCEPT
       return 0
     fi
-  # Othwerwise,
+  # Otherwise,
   else
     # no firewall is running
     echo -e "  ${INFO} No active firewall detected.. skipping firewall configuration"
@@ -1547,8 +1554,6 @@ installPihole() {
   installScripts
   # configs,
   installConfigs
-  # and create the log file
-  CreateLogFile
   # If the user wants to install the dashboard,
   if [[ "${INSTALL_WEB}" == true ]]; then
     # do so
@@ -1585,8 +1590,6 @@ updatePihole() {
   installScripts
   # Install config files
   installConfigs
-  # Create the log file
-  CreateLogFile
   # If the user wants to install the dasboard,
   if [[ "${INSTALL_WEB}" == true ]]; then
     # do so
@@ -1731,17 +1734,14 @@ clone_or_update_repos() {
   fi
 }
 
-# Download and install FTL binary
+# Download FTL binary to random temp directory and install FTL binary
 FTLinstall() {
   # Local, named variables
   local binary="${1}"
   local latesttag
-  local orig_dir
   local str="Downloading and Installing FTL"
   echo -ne "  ${INFO} ${str}..."
 
-  # Get the current working directory
-  orig_dir="${PWD}"
   # Find the latest version tag for FTL
   latesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep "Location" | awk -F '/' '{print $NF}')
   # Tags should always start with v, check for that.
@@ -1751,54 +1751,104 @@ FTLinstall() {
     return 1
   fi
 
-  # If the download worked,
-  if curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${latesttag%$'\r'}/${binary}" -o "/tmp/${binary}"; then
-    # get sha1 of the binary we just downloaded for verification.
-    curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${latesttag%$'\r'}/${binary}.sha1" -o "/tmp/${binary}.sha1"
+  # Move into the temp ftl directory
+  pushd "$(mktemp -d)" > /dev/null || { echo "Unable to make temporary directory for FTL binary download"; return 1; }
+
+  # Determine if systemd is used on this system
+  if file "$(which init)" | grep "systemd" &> /dev/null; then
+    # Use systemd unit
+    # Always replace pihole-FTL.service (systemd unit)
+    install -T -m 0644 "${PI_HOLE_LOCAL_REPO}/advanced/pihole-FTL.systemd" "/etc/systemd/system/pihole-FTL.service"
+    install -T -m 0755 "${PI_HOLE_LOCAL_REPO}/advanced/pihole-FTL-prestart.sh" "/opt/pihole/pihole-FTL-prestart.sh"
+
+    # Remove old init.d script if present as it cannot coexist with the systemd unit we are installing here
+    if [ -e "/etc/init.d/pihole-FTL" ]; then
+      rm "/etc/init.d/pihole-FTL"
+      update-rc.d pihole-FTL remove
+    fi
+
+    # Enable service script (we have to do this after replacing the service unit)
+    systemctl enable pihole-FTL.service
+  else
+    # Use old init.d script
+    # Always replace pihole-FTL.service (init.d script)
+    install -T -m 0644 "${PI_HOLE_LOCAL_REPO}/advanced/pihole-FTL.initd" "/etc/init.d/pihole-FTL"
+  fi
+
+  local ftlBranch
+  local url
+  local ftlBranch
+
+  if [[ -f "/etc/pihole/ftlbranch" ]];then
+    ftlBranch=$(</etc/pihole/ftlbranch)
+  else
+    ftlBranch="master"
+  fi
+
+  # Determine which version of FTL to download
+  if [[ "${ftlBranch}" == "master" ]];then
+    url="https://github.com/pi-hole/FTL/releases/download/${latesttag%$'\r'}"
+  else
+    url="https://ftl.pi-hole.net/${ftlBranch}"
+  fi
+
+  # If the download worked,
+  if curl -sSL --fail "${url}/${binary}" -o "${binary}"; then
+    # get sha1 of the binary we just downloaded for verification.
+    curl -sSL --fail "${url}/${binary}.sha1" -o "${binary}.sha1"
 
-    # Move into the temp directory
-    cd /tmp
     # If we downloaded binary file (as opposed to text),
     if sha1sum --status --quiet -c "${binary}".sha1; then
       echo -n "transferred... "
       # Stop FTL
       stop_service pihole-FTL &> /dev/null
       # Install the new version with the correct permissions
-      install -T -m 0755 /tmp/${binary} /usr/bin/pihole-FTL
-      # Remove the tempoary file
-      rm /tmp/${binary} /tmp/${binary}.sha1
+      install -T -m 0755 "${binary}" /usr/bin/pihole-FTL
+      # Set net admin permissions so that FTL can serve DNS, DHCP and IMAP (for DHCPv6)
+      setcap CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_NET_ADMIN+eip "/usr/bin/pihole-FTL"
       # Move back into the original directory the user was in
-      cd "${orig_dir}"
+      popd > /dev/null || { echo "Unable to return to original directory after FTL binary download."; return 1; }
       # Install the FTL service
-      install -T -m 0755 "${PI_HOLE_LOCAL_REPO}/advanced/pihole-FTL.service" "/etc/init.d/pihole-FTL"
       echo -e "${OVER}  ${TICK} ${str}"
+      # If the --resolver flag returns True (exit code 0), then we can safely stop & disable dnsmasq
+      if pihole-FTL --resolver > /dev/null; then
+        if which dnsmasq > /dev/null; then
+          if check_service_active "dnsmasq";then
+            echo "  ${INFO} FTL can now resolve DNS Queries without dnsmasq running separately"
+            stop_service dnsmasq
+            disable_service dnsmasq
+          fi
+        fi
+
+        #ensure /etc/dnsmasq.conf contains `conf-dir=/etc/dnsmasq.d`
+        confdir="conf-dir=/etc/dnsmasq.d"
+        conffile="/etc/dnsmasq.conf"
+        if ! grep -q "$confdir" "$conffile"; then
+            echo "$confdir" >> "$conffile"
+        fi
+      fi
       return 0
     # Otherise,
     else
+      # the download failed, so just go back to the original directory
+      popd > /dev/null || { echo "Unable to return to original directory after FTL binary download."; return 1; }
       echo -e "${OVER}  ${CROSS} ${str}"
       echo -e "  ${COL_LIGHT_RED}Error: Download of binary from Github failed${COL_NC}"
-      # the download failed, so just go back to the original directory
-      cd "${orig_dir}"
       return 1
     fi
   # Otherwise,
   else
-    cd "${orig_dir}"
+    popd > /dev/null || { echo "Unable to return to original directory after FTL binary download."; return 1; }
     echo -e "${OVER}  ${CROSS} ${str}"
     # The URL could not be found
     echo -e "  ${COL_LIGHT_RED}Error: URL not found${COL_NC}"
+    return 1
   fi
 }
 
-# Detect suitable FTL binary platform
-FTLdetect() {
-  echo ""
-  echo -e "  ${INFO} FTL Checks..."
-
-  # Local, named variables
+get_binary_name() {
+# Local, named variables
   local machine
-  local binary
-
   # Store architecture in a variable
   machine=$(uname -m)
 
@@ -1857,45 +1907,108 @@ FTLdetect() {
     fi
     binary="pihole-FTL-linux-x86_32"
   fi
+}
+
+FTLcheckUpdate()
+{
+  get_binary_name
 
   #In the next section we check to see if FTL is already installed (in case of pihole -r).
   #If the installed version matches the latest version, then check the installed sha1sum of the binary vs the remote sha1sum. If they do not match, then download
   echo -e "  ${INFO} Checking for existing FTL binary..."
 
-  local ftlLoc=$(which pihole-FTL 2>/dev/null)
+  local ftlLoc
+  ftlLoc=$(which pihole-FTL 2>/dev/null)
 
-  if [[ ${ftlLoc} ]]; then
-    local FTLversion=$(/usr/bin/pihole-FTL tag)
-	  local FTLlatesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep 'Location' | awk -F '/' '{print $NF}' | tr -d '\r\n')
+  local ftlBranch
 
-	  if [[ "${FTLversion}" != "${FTLlatesttag}" ]]; then
-		  # Install FTL
-      FTLinstall "${binary}" || return 1
-	  else
-	    echo -e "  ${INFO} Latest FTL Binary already installed (${FTLlatesttag}). Confirming Checksum..."
+  if [[ -f "/etc/pihole/ftlbranch" ]];then
+    ftlBranch=$(</etc/pihole/ftlbranch)
+  else
+    ftlBranch="master"
+  fi
 
-	    local remoteSha1=$(curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${FTLversion%$'\r'}/${binary}.sha1" | cut -d ' ' -f 1)
-	    local localSha1=$(sha1sum "$(which pihole-FTL)" | cut -d ' ' -f 1)
+  local remoteSha1
+  local localSha1
 
-	    if [[ "${remoteSha1}" != "${localSha1}" ]]; then
-	      echo -e "  ${INFO} Corruption detected..."
-	      FTLinstall "${binary}" || return 1
-	    else
-	      echo -e "  ${INFO} Checksum correct. No need to download!"
-	    fi
-	  fi
-	else
-	  # Install FTL
+  if [[ ! "${ftlBranch}" == "master" ]]; then
+    if [[ ${ftlLoc} ]]; then
+      # We already have a pihole-FTL binary downloaded.
+      # Alt branches don't have a tagged version against them, so just confirm the checksum of the local vs remote to decide whether we download or not
+      remoteSha1=$(curl -sSL --fail "https://ftl.pi-hole.net/${ftlBranch}/${binary}.sha1" | cut -d ' ' -f 1)
+      localSha1=$(sha1sum "$(which pihole-FTL)" | cut -d ' ' -f 1)
+
+      if [[ "${remoteSha1}" != "${localSha1}" ]]; then
+        echo -e "  ${INFO} Checksums do not match, downloading from ftl.pi-hole.net."
+        return 0
+      else
+        echo -e "  ${INFO} Checksum of installed binary matches remote. No need to download!"
+        return 1
+      fi
+    else
+      return 0
+    fi
+  else
+    if [[ ${ftlLoc} ]]; then
+      local FTLversion
+      FTLversion=$(/usr/bin/pihole-FTL tag)
+      local FTLlatesttag
+      FTLlatesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep 'Location' | awk -F '/' '{print $NF}' | tr -d '\r\n')
+
+      if [[ "${FTLversion}" != "${FTLlatesttag}" ]]; then
+        return 0
+      else
+        echo -e "  ${INFO} Latest FTL Binary already installed (${FTLlatesttag}). Confirming Checksum..."
+
+        remoteSha1=$(curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${FTLversion%$'\r'}/${binary}.sha1" | cut -d ' ' -f 1)
+        localSha1=$(sha1sum "$(which pihole-FTL)" | cut -d ' ' -f 1)
+
+        if [[ "${remoteSha1}" != "${localSha1}" ]]; then
+          echo -e "  ${INFO} Corruption detected..."
+          return 0
+        else
+          echo -e "  ${INFO} Checksum correct. No need to download!"
+          return 1
+        fi
+      fi
+    else
+      return 0
+    fi
+  fi
+}
+
+# Detect suitable FTL binary platform
+FTLdetect() {
+  echo ""
+  echo -e "  ${INFO} FTL Checks..."
+
+  if FTLcheckUpdate ; then
     FTLinstall "${binary}" || return 1
   fi
 
+  echo ""
+}
 
+make_temporary_log() {
+  # Create a random temporary file for the log
+  TEMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX)
+  # Open handle 3 for templog
+  # https://stackoverflow.com/questions/18460186/writing-outputs-to-log-file-and-console
+  exec 3>"$TEMPLOG"
+  # Delete templog, but allow for addressing via file handle
+  # This lets us write to the log without having a temporary file on the drive, which
+  # is meant to be a security measure so there is not a lingering file on the drive during the install process
+  rm "$TEMPLOG"
+}
+
+copy_to_install_log() {
+  # Copy the contents of file descriptor 3 into the install log
+  # Since we use color codes such as '\e[1;33m', they should be removed
+  sed 's/\[[0-9;]\{1,5\}m//g' < /proc/$$/fd/3 > "${installLogLoc}"
 }
 
 main() {
   ######## FIRST CHECK ########
-  # Show the Pi-hole logo so people know it's genuine since the logo and name are trademarked
-  show_ascii_berry
   # Must be root to install
   local str="Root user check"
   echo ""
@@ -1904,12 +2017,15 @@ main() {
   if [[ "${EUID}" -eq 0 ]]; then
     # they are root and all is good
     echo -e "  ${TICK} ${str}"
+    # Show the Pi-hole logo so people know it's genuine since the logo and name are trademarked
+    show_ascii_berry
+    make_temporary_log
   # Otherwise,
   else
     # They do not have enough privileges, so let the user know
     echo -e "  ${CROSS} ${str}
       ${COL_LIGHT_RED}Script called with non-root privileges${COL_NC}
-      The Pi-hole requires elevated privleges to install and run
+      The Pi-hole requires elevated privileges to install and run
       Please check the installer for any concerns regarding this requirement
       Make sure to download this script from a trusted source\\n"
     echo -ne "  ${INFO} Sudo utility check"
@@ -1937,7 +2053,7 @@ main() {
   for var in "$@"; do
     case "$var" in
       "--reconfigure" ) reconfigure=true;;
-      "--i_do_not_follow_recommendations" ) skipSpaceCheck=false;;
+      "--i_do_not_follow_recommendations" ) skipSpaceCheck=true;;
       "--unattended" ) runUnattended=true;;
     esac
   done
@@ -1982,7 +2098,19 @@ main() {
     # Create directory for Pi-hole storage
     mkdir -p /etc/pihole/
 
-    stop_service dnsmasq
+    #Do we need to stop pihole-FTL or dnsmasq(if coming from an old install)?
+    if [[ $(which pihole-FTL 2>/dev/null) ]]; then
+      if pihole-FTL --resolver > /dev/null; then
+        stop_service pihole-FTL
+      else
+        stop_service dnsmasq
+      fi
+    else
+      if [[ $(which dnsmasq 2>/dev/null) ]]; then
+        stop_service dnsmasq
+      fi
+    fi
+
     if [[ "${INSTALL_WEB}" == true ]]; then
       stop_service lighttpd
     fi
@@ -2028,14 +2156,14 @@ main() {
     fi
 
     # Install and log everything to a file
-    installPihole | tee ${tmpLog}
+    installPihole | tee -a /proc/$$/fd/3
   else
+    # Source ${setupVars} to use predefined user variables in the functions
+    source ${setupVars}
+
     # Clone/Update the repos
     clone_or_update_repos
 
-    # Source ${setupVars} for use in the rest of the functions
-    source ${setupVars}
-
     # Install packages used by the Pi-hole
     if [[ "${INSTALL_WEB}" == true ]]; then
       # Install the Web dependencies
@@ -2054,12 +2182,11 @@ main() {
       # Value will either be 1, if true, or 0
       LIGHTTPD_ENABLED=$(service lighttpd status | awk '/Loaded:/ {print $0}' | grep -c 'enabled' || true)
     fi
-
-    updatePihole | tee ${tmpLog}
+    updatePihole | tee -a /proc/$$/fd/3
   fi
 
-  # Move the log file into /etc/pihole for storage
-  mv ${tmpLog} ${instalLogLoc}
+  # Copy the temp log file into final log location for storage
+  copy_to_install_log
 
   if [[ "${INSTALL_WEB}" == true ]]; then
     # Add password to web UI if there is none
@@ -2076,8 +2203,11 @@ main() {
 
   echo -e "  ${INFO} Restarting services..."
   # Start services
-  start_service dnsmasq
-  enable_service dnsmasq
+  # Only start and enable dnsmasq if FTL does not have the --resolver switch
+  if ! pihole-FTL --resolver > /dev/null; then
+    start_service dnsmasq
+    enable_service dnsmasq
+  fi
 
   # If the Web server was installed,
   if [[ "${INSTALL_WEB}" == true ]]; then
@@ -2099,6 +2229,7 @@ main() {
 
   # Force an update of the updatechecker
   . /opt/pihole/updatecheck.sh
+  . /opt/pihole/updatecheck.sh x remote
 
   #
   if [[ "${useUpdateVars}" == false ]]; then
@@ -2135,9 +2266,13 @@ main() {
   fi
 
   # Display where the log file is
-  echo -e "\\n  ${INFO} The install log is located at: /etc/pihole/install.log
+  echo -e "\\n  ${INFO} The install log is located at: ${installLogLoc}
   ${COL_LIGHT_GREEN}${INSTALL_TYPE} Complete! ${COL_NC}"
 
+  if [[ "${INSTALL_TYPE}" == "Update" ]]; then
+    echo ""
+    /usr/local/bin/pihole version --current
+  fi
 }
 
 #

gravity.sh

@@ -11,6 +11,8 @@
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.
 
+export LC_ALL=C
+
 coltable="/opt/pihole/COL_TABLE"
 source "${coltable}"
 
@@ -42,6 +44,8 @@ preEventHorizon="list.preEventHorizon"
 
 skipDownload="false"
 
+resolver="pihole-FTL"
+
 # Source setupVars from install script
 setupVars="${piholeDir}/setupVars.conf"
 if [[ -f "${setupVars}" ]];then
@@ -68,8 +72,8 @@ if [[ -r "${piholeDir}/pihole.conf" ]]; then
 fi
 
 # Determine if DNS resolution is available before proceeding
-gravity_DNSLookup() {
-  local lookupDomain="pi.hole" plural=""
+gravity_CheckDNSResolutionAvailable() {
+  local lookupDomain="pi.hole"
 
   # Determine if $localList does not exist
   if [[ ! -e "${localList}" ]]; then
@@ -88,8 +92,21 @@ gravity_DNSLookup() {
     exit 1
   fi
 
+  # If the /etc/resolv.conf contains resolvers other than 127.0.0.1 then the local dnsmasq will not be queried and pi.hole is NXDOMAIN.
+  # This means that even though name resolution is working, the getent hosts check fails and the holddown timer keeps ticking and eventualy fails
+  # So we check the output of the last command and if it failed, attempt to use dig +short as a fallback
+  if timeout 1 dig +short "${lookupDomain}" &> /dev/null; then
+    if [[ -n "${secs:-}" ]]; then
+      echo -e "${OVER}  ${TICK} DNS resolution is now available\\n"
+    fi
+    return 0
+  elif [[ -n "${secs:-}" ]]; then
+    echo -e "${OVER}  ${CROSS} DNS resolution is not available"
+    exit 1
+  fi
+
   # Determine error output message
-  if pidof dnsmasq &> /dev/null; then
+  if pidof ${resolver} &> /dev/null; then
     echo -e "  ${CROSS} DNS resolution is currently unavailable"
   else
     echo -e "  ${CROSS} DNS service is not running"
@@ -98,21 +115,20 @@ gravity_DNSLookup() {
 
   # Ensure DNS server is given time to be resolvable
   secs="120"
-  echo -ne "  ${INFO} Waiting up to ${secs} seconds before continuing..."
+  echo -ne "  ${INFO} Time until retry: ${secs}"
   until timeout 1 getent hosts "${lookupDomain}" &> /dev/null; do
     [[ "${secs:-}" -eq 0 ]] && break
-    [[ "${secs:-}" -ne 1 ]] && plural="s"
-    echo -ne "${OVER}  ${INFO} Waiting up to ${secs} second${plural} before continuing..."
+    echo -ne "${OVER}  ${INFO} Time until retry: ${secs}"
     : $((secs--))
     sleep 1
   done
 
   # Try again
-  gravity_DNSLookup
+  gravity_CheckDNSResolutionAvailable
 }
 
 # Retrieve blocklist URLs and parse domains from adlists.list
-gravity_Collapse() {
+gravity_GetBlocklistUrls() {
   echo -e "  ${INFO} ${COL_BOLD}Neutrino emissions detected${COL_NC}..."
 
   # Determine if adlists file needs handling
@@ -139,7 +155,8 @@ gravity_Collapse() {
     awk -F '[/:]' '{
       # Remove URL protocol & optional username:password@
       gsub(/(.*:\/\/|.*:.*@)/, "", $0)
-      print $1
+      if(length($1)>0){print $1}
+      else {print "local"}
     }' <<< "$(printf '%s\n' "${sources[@]}")" 2> /dev/null
   )"
 
@@ -152,7 +169,7 @@ gravity_Collapse() {
 }
 
 # Define options for when retrieving blocklists
-gravity_Supernova() {
+gravity_SetDownloadOptions() {
   local url domain agent cmd_ext str
 
   echo ""
@@ -177,7 +194,7 @@ gravity_Supernova() {
 
     if [[ "${skipDownload}" == false ]]; then
       echo -e "  ${INFO} Target: ${domain} (${url##*/})"
-      gravity_Pull "${url}" "${cmd_ext}" "${agent}"
+      gravity_DownloadBlocklistFromUrl "${url}" "${cmd_ext}" "${agent}"
       echo ""
     fi
   done
@@ -185,16 +202,17 @@ gravity_Supernova() {
 }
 
 # Download specified URL and perform checks on HTTP status and file content
-gravity_Pull() {
+gravity_DownloadBlocklistFromUrl() {
   local url="${1}" cmd_ext="${2}" agent="${3}" heisenbergCompensator="" patternBuffer str httpCode success=""
 
   # Create temp file to store content on disk instead of RAM
   patternBuffer=$(mktemp -p "/tmp" --suffix=".phgpb")
 
   # Determine if $saveLocation has read permission
-  if [[ -r "${saveLocation}" ]]; then
+  if [[ -r "${saveLocation}" && $url != "file"* ]]; then
     # Have curl determine if a remote file has been modified since last retrieval
     # Uses "Last-Modified" header, which certain web servers do not provide (e.g: raw github urls)
+    # Note: Don't do this for local files, always download them
     heisenbergCompensator="-z ${saveLocation}"
   fi
 
@@ -203,20 +221,32 @@ gravity_Pull() {
   # shellcheck disable=SC2086
   httpCode=$(curl -s -L ${cmd_ext} ${heisenbergCompensator} -w "%{http_code}" -A "${agent}" "${url}" -o "${patternBuffer}" 2> /dev/null)
 
-  # Determine "Status:" output based on HTTP response
-  case "${httpCode}" in
-    "200") echo -e "${OVER}  ${TICK} ${str} Retrieval successful"; success=true;;
-    "304") echo -e "${OVER}  ${TICK} ${str} No changes detected"; success=true;;
-    "000") echo -e "${OVER}  ${CROSS} ${str} Connection Refused";;
-    "403") echo -e "${OVER}  ${CROSS} ${str} Forbidden";;
-    "404") echo -e "${OVER}  ${CROSS} ${str} Not found";;
-    "408") echo -e "${OVER}  ${CROSS} ${str} Time-out";;
-    "451") echo -e "${OVER}  ${CROSS} ${str} Unavailable For Legal Reasons";;
-    "500") echo -e "${OVER}  ${CROSS} ${str} Internal Server Error";;
-    "504") echo -e "${OVER}  ${CROSS} ${str} Connection Timed Out (Gateway)";;
-    "521") echo -e "${OVER}  ${CROSS} ${str} Web Server Is Down (Cloudflare)";;
-    "522") echo -e "${OVER}  ${CROSS} ${str} Connection Timed Out (Cloudflare)";;
-    *    ) echo -e "${OVER}  ${CROSS} ${str} ${httpCode}";;
+  case $url in
+    # Did we "download" a remote file?
+    "http"*)
+      # Determine "Status:" output based on HTTP response
+      case "${httpCode}" in
+        "200") echo -e "${OVER}  ${TICK} ${str} Retrieval successful"; success=true;;
+        "304") echo -e "${OVER}  ${TICK} ${str} No changes detected"; success=true;;
+        "000") echo -e "${OVER}  ${CROSS} ${str} Connection Refused";;
+        "403") echo -e "${OVER}  ${CROSS} ${str} Forbidden";;
+        "404") echo -e "${OVER}  ${CROSS} ${str} Not found";;
+        "408") echo -e "${OVER}  ${CROSS} ${str} Time-out";;
+        "451") echo -e "${OVER}  ${CROSS} ${str} Unavailable For Legal Reasons";;
+        "500") echo -e "${OVER}  ${CROSS} ${str} Internal Server Error";;
+        "504") echo -e "${OVER}  ${CROSS} ${str} Connection Timed Out (Gateway)";;
+        "521") echo -e "${OVER}  ${CROSS} ${str} Web Server Is Down (Cloudflare)";;
+        "522") echo -e "${OVER}  ${CROSS} ${str} Connection Timed Out (Cloudflare)";;
+        *    ) echo -e "${OVER}  ${CROSS} ${str} ${httpCode}";;
+      esac;;
+    # Did we "download" a local file?
+    "file"*)
+        if [[ -s "${patternBuffer}" ]]; then
+          echo -e "${OVER}  ${TICK} ${str} Retrieval successful"; success=true
+        else
+          echo -e "${OVER}  ${CROSS} ${str} Not found / empty list"
+        fi;;
+    *) echo -e "${OVER}  ${CROSS} ${str} ${url} ${httpCode}";;
   esac
 
   # Determine if the blocklist was downloaded and saved correctly
@@ -243,36 +273,22 @@ gravity_Pull() {
 
 # Parse source files into domains format
 gravity_ParseFileIntoDomains() {
-  local source="${1}" destination="${2}" commentPattern firstLine abpFilter
+  local source="${1}" destination="${2}" firstLine abpFilter
 
   # Determine if we are parsing a consolidated list
   if [[ "${source}" == "${piholeDir}/${matterAndLight}" ]]; then
-    # Define symbols used as comments: #;@![/
-    commentPattern="[#;@![\\/]"
+    # Remove comments and print only the domain name
+    # Most of the lists downloaded are already in hosts file format but the spacing/formating is not contigious
+    # This helps with that and makes it easier to read
+    # It also helps with debugging so each stage of the script can be researched more in depth
+    #Awk -F splits on given IFS, we grab the right hand side (chops trailing #coments and /'s to grab the domain only.
+    #Last awk command takes non-commented lines and if they have 2 fields, take the left field (the domain) and leave
+    #+ the right (IP address), otherwise grab the single field.
 
-    # Parse Domains/Hosts files by removing comments & host IPs
-    # Logic: Ignore lines which begin with comments
-    awk '!/^'"${commentPattern}"'/ {
-      # Determine if there are multiple words seperated by a space
-      if(NF>1) {
-        # Remove comments (including prefixed spaces/tabs)
-        if($0 ~ /'"${commentPattern}"'/) { gsub("( |\t)'"${commentPattern}"'.*", "", $0) }
-        # Determine if there are aliased domains
-        if($3) {
-          # Remove IP address
-          $1=""
-          # Remove space which is left in $0 when removing $1
-          gsub("^ ", "", $0)
-          print $0
-        } else if($2) {
-          # Print single domain without IP
-          print $2
-        }
-      # If there are no words seperated by space
-      } else if($1) {
-        print $1
-      }
-    }' "${source}" 2> /dev/null > "${destination}"
+    < ${source} awk -F '#' '{print $1}' | \
+    awk -F '/' '{print $1}' | \
+    awk '($1 !~ /^#/) { if (NF>1) {print $2} else {print $1}}' | \
+    sed -nr -e 's/\.{2,}/./g' -e '/\./p' >  ${destination}
     return 0
   fi
 
@@ -318,7 +334,7 @@ gravity_ParseFileIntoDomains() {
       }' "${source}" > "${destination}.exceptionsFile.tmp"
 
       # Remove exceptions
-      grep -F -x -v -f "${destination}.exceptionsFile.tmp" "${destination}" > "${source}"
+      comm -23 "${destination}" <(sort "${destination}.exceptionsFile.tmp") > "${source}"
       mv "${source}" "${destination}"
     fi
 
@@ -353,7 +369,7 @@ gravity_ParseFileIntoDomains() {
 }
 
 # Create (unfiltered) "Matter and Light" consolidated list
-gravity_Schwarzschild() {
+gravity_ConsolidateDownloadedBlocklists() {
   local str lastLine
 
   str="Consolidating blocklists"
@@ -381,7 +397,7 @@ gravity_Schwarzschild() {
 }
 
 # Parse consolidated list into (filtered, unique) domains-only format
-gravity_Filter() {
+gravity_SortAndFilterConsolidatedList() {
   local str num
 
   str="Extracting domains from blocklists"
@@ -393,7 +409,7 @@ gravity_Filter() {
   # Format $parsedMatter line total as currency
   num=$(printf "%'.0f" "$(wc -l < "${piholeDir}/${parsedMatter}")")
   echo -e "${OVER}  ${TICK} ${str}
-  ${INFO} ${COL_BLUE}${num}${COL_NC} domains being pulled in by gravity"
+  ${INFO} Number of domains being pulled in by gravity: ${COL_BLUE}${num}${COL_NC}"
 
   str="Removing duplicate domains"
   echo -ne "  ${INFO} ${str}..."
@@ -402,31 +418,12 @@ gravity_Filter() {
 
   # Format $preEventHorizon line total as currency
   num=$(printf "%'.0f" "$(wc -l < "${piholeDir}/${preEventHorizon}")")
-  echo -e "  ${INFO} ${COL_BLUE}${num}${COL_NC} unique domains trapped in the Event Horizon"
-}
-
-# Whitelist unique blocklist domain sources
-gravity_WhitelistBLD() {
-  local uniqDomains plural="" str 
-
-  echo ""
-
-  # Create array of unique $sourceDomains
-  mapfile -t uniqDomains <<< "$(awk '{ if(!a[$1]++) { print $1 } }' <<< "$(printf '%s\n' "${sourceDomains[@]}")")"
-  [[ "${#uniqDomains[@]}" -ne 1 ]] && plural="s"
-
-  str="Adding ${#uniqDomains[@]} blocklist source domain${plural} to the whitelist"
-  echo -ne "  ${INFO} ${str}..."
-
-  # Whitelist $uniqDomains
-  "${PIHOLE_COMMAND}" -w -nr -q "${uniqDomains[*]}" &> /dev/null
-
-  echo -e "${OVER}  ${TICK} ${str}"
+  echo -e "  ${INFO} Number of unique domains trapped in the Event Horizon: ${COL_BLUE}${num}${COL_NC}"
 }
 
 # Whitelist user-defined domains
 gravity_Whitelist() {
-  local num plural="" str
+  local num str
 
   if [[ ! -f "${whitelistFile}" ]]; then
     echo -e "  ${INFO} Nothing to whitelist!"
@@ -434,24 +431,22 @@ gravity_Whitelist() {
   fi
 
   num=$(wc -l < "${whitelistFile}")
-  [[ "${num}" -ne 1 ]] && plural="s"
-  str="Whitelisting ${num} domain${plural}"
+  str="Number of whitelisted domains: ${num}"
   echo -ne "  ${INFO} ${str}..."
 
   # Print everything from preEventHorizon into whitelistMatter EXCEPT domains in $whitelistFile
-  grep -F -x -v -f "${whitelistFile}" "${piholeDir}/${preEventHorizon}" > "${piholeDir}/${whitelistMatter}"
+  comm -23 "${piholeDir}/${preEventHorizon}" <(sort "${whitelistFile}") > "${piholeDir}/${whitelistMatter}"
 
-  echo -e "${OVER}  ${TICK} ${str}"
+  echo -e "${OVER}  ${INFO} ${str}"
 }
 
 # Output count of blacklisted domains and wildcards
 gravity_ShowBlockCount() {
-  local num plural
+  local num
 
   if [[ -f "${blacklistFile}" ]]; then
     num=$(printf "%'.0f" "$(wc -l < "${blacklistFile}")")
-    plural=; [[ "${num}" -ne 1 ]] && plural="s"
-    echo -e "  ${INFO} Blacklisted ${num} domain${plural}"
+    echo -e "  ${INFO} Number of blacklisted domains: ${num}"
   fi
 
   if [[ -f "${wildcardFile}" ]]; then
@@ -460,8 +455,7 @@ gravity_ShowBlockCount() {
     if [[ -n "${IPV4_ADDRESS}" ]] && [[ -n "${IPV6_ADDRESS}" ]];then
       num=$(( num/2 ))
     fi
-    plural=; [[ "${num}" -ne 1 ]] && plural="s"
-    echo -e "  ${INFO} Wildcard blocked ${num} domain${plural}"
+    echo -e "  ${INFO} Number of wildcard blocked domains: ${num}"
   fi
 }
 
@@ -513,8 +507,15 @@ gravity_ParseBlacklistDomains() {
 
   # Empty $accretionDisc if it already exists, otherwise, create it
   : > "${piholeDir}/${accretionDisc}"
-
-  gravity_ParseDomainsIntoHosts "${piholeDir}/${whitelistMatter}" "${piholeDir}/${accretionDisc}"
+  
+  if [[ -f "${piholeDir}/${whitelistMatter}" ]]; then
+    gravity_ParseDomainsIntoHosts "${piholeDir}/${whitelistMatter}" "${piholeDir}/${accretionDisc}"
+    grep -c "^" "${piholeDir}/${whitelistMatter}" > "${piholeDir}/numBlocked" 2> /dev/null
+  else
+    # There was no whitelist file, so use preEventHorizon instead of whitelistMatter.
+    gravity_ParseDomainsIntoHosts "${piholeDir}/${preEventHorizon}" "${piholeDir}/${accretionDisc}"
+    grep -c "^" "${piholeDir}/${preEventHorizon}" > "${piholeDir}/numBlocked" 2> /dev/null
+  fi
 
   # Move the file over as /etc/pihole/gravity.list so dnsmasq can use it
   output=$( { mv "${piholeDir}/${accretionDisc}" "${adList}"; } 2>&1 )
@@ -555,7 +556,7 @@ gravity_Cleanup() {
   rm ${piholeDir}/*.tmp 2> /dev/null
   rm /tmp/*.phgpb 2> /dev/null
 
-  # Ensure this function only runs when gravity_Supernova() has completed
+  # Ensure this function only runs when gravity_SetDownloadOptions() has completed
   if [[ "${gravity_Blackbody:-}" == true ]]; then
     # Remove any unused .domains files
     for file in ${piholeDir}/*.${domainsExtension}; do
@@ -570,7 +571,7 @@ gravity_Cleanup() {
   echo -e "${OVER}  ${TICK} ${str}"
 
   # Only restart DNS service if offline
-  if ! pidof dnsmasq &> /dev/null; then
+  if ! pidof ${resolver} &> /dev/null; then
     "${PIHOLE_COMMAND}" restartdns
     dnsWasOffline=true
   fi
@@ -617,12 +618,11 @@ fi
 # Determine which functions to run
 if [[ "${skipDownload}" == false ]]; then
   # Gravity needs to download blocklists
-  gravity_DNSLookup
-  gravity_Collapse
-  gravity_Supernova
-  gravity_Schwarzschild
-  gravity_Filter
-  gravity_WhitelistBLD
+  gravity_CheckDNSResolutionAvailable
+  gravity_GetBlocklistUrls
+  gravity_SetDownloadOptions
+  gravity_ConsolidateDownloadedBlocklists
+  gravity_SortAndFilterConsolidatedList
 else
   # Gravity needs to modify Blacklist/Whitelist/Wildcards
   echo -e "  ${INFO} Using cached Event Horizon list..."

pihole

@@ -14,6 +14,8 @@ readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
 readonly colfile="${PI_HOLE_SCRIPT_DIR}/COL_TABLE"
 source "${colfile}"
 
+resolver="pihole-FTL"
+
 # Must be root to use this tool
 if [[ ! $EUID -eq 0 ]];then
   if [[ -x "$(command -v sudo)" ]]; then
@@ -332,18 +334,18 @@ restartDNS() {
   local svcOption svc str output status
   svcOption="${1:-}"
 
-  # Determine if we should reload or restart dnsmasq
+  # Determine if we should reload or restart restart
   if [[ "${svcOption}" =~ "reload" ]]; then
     # Using SIGHUP will NOT re-read any *.conf files
-    svc="killall -s SIGHUP dnsmasq"
+    svc="killall -s SIGHUP ${resolver}"
   else
-    # Get PID of dnsmasq to determine if it needs to start or restart
-    if pidof dnsmasq &> /dev/null; then
+    # Get PID of resolver to determine if it needs to start or restart
+    if pidof pihole-FTL &> /dev/null; then
       svcOption="restart"
     else
       svcOption="start"
     fi
-    svc="service dnsmasq ${svcOption}"
+    svc="service ${resolver} ${svcOption}"
   fi
 
   # Print output to Terminal, but not to Web Admin
@@ -359,9 +361,6 @@ restartDNS() {
     [[ ! -t 1 ]] && local OVER=""
     echo -e "${OVER}  ${CROSS} ${output}"
   fi
-
-  # Send signal to FTL to have it re-parse the gravity files
-  killall -s SIGHUP pihole-FTL
 }
 
 piholeEnable() {
@@ -444,13 +443,17 @@ Specify whether the Pi-hole log should be used
 
 Options:
   on                  Enable the Pi-hole log at /var/log/pihole.log
-  off                 Disable the Pi-hole log at /var/log/pihole.log"
+  off                 Disable and flush the Pi-hole log at /var/log/pihole.log
+  off noflush         Disable the Pi-hole log at /var/log/pihole.log"
     exit 0
   elif [[ "${1}" == "off" ]]; then
     # Disable logging
     sed -i 's/^log-queries/#log-queries/' /etc/dnsmasq.d/01-pihole.conf
     sed -i 's/^QUERY_LOGGING=true/QUERY_LOGGING=false/' /etc/pihole/setupVars.conf
-    pihole -f
+    if [[ "${2}" != "noflush" ]]; then
+      # Flush logs
+      pihole -f
+    fi
     echo -e "  ${INFO} Disabling logging..."
     local str="Logging has been disabled!"
   elif [[ "${1}" == "on" ]]; then
@@ -472,7 +475,7 @@ statusFunc() {
   local addnConfigs
 
   # Determine if service is running on port 53 (Cr: https://superuser.com/a/806331)
-  if (echo > /dev/tcp/localhost/53) >/dev/null 2>&1; then
+  if (echo > /dev/tcp/127.0.0.1/53) >/dev/null 2>&1; then
     if [[ "${1}" != "web" ]]; then
       echo -e "  ${TICK} DNS service is running"
     fi
@@ -658,6 +661,6 @@ case "${1}" in
   "-t" | "tail"                 ) tailFunc;;
   "checkout"                    ) piholeCheckoutFunc "$@";;
   "tricorder"                   ) tricorderFunc;;
-  "updatechecker"               ) updateCheckFunc;;
+  "updatechecker"               ) updateCheckFunc "$@";;
   *                             ) helpFunc;;
 esac

test/test_automated_install.py

@@ -80,7 +80,7 @@ def test_configureFirewall_firewalld_running_no_errors(Pihole):
     source /opt/pihole/basic-install.sh
     configureFirewall
     ''')
-    expected_stdout = 'Configuring FirewallD for httpd and dnsmasq'
+    expected_stdout = 'Configuring FirewallD for httpd and pihole-FTL'
     assert expected_stdout in configureFirewall.stdout
     firewall_calls = Pihole.run('cat /var/log/firewall-cmd').stdout
     assert 'firewall-cmd --state' in firewall_calls
@@ -310,15 +310,16 @@ def test_FTL_download_unknown_fails_no_errors(Pihole):
     error = 'Error: URL not found'
     assert error in download_binary.stdout
 
-def test_FTL_binary_installed_and_responsive_no_errors(Pihole):
-    ''' confirms FTL binary is copied and functional in installed location '''
-    installed_binary = Pihole.run('''
-    source /opt/pihole/basic-install.sh
-    FTLdetect
-    pihole-FTL version
-    ''')
-    expected_stdout = 'v'
-    assert expected_stdout in installed_binary.stdout
+# Temporarily disabled as we cannot use setcap on Travis CI
+# def test_FTL_binary_installed_and_responsive_no_errors(Pihole):
+#     ''' confirms FTL binary is copied and functional in installed location '''
+#     installed_binary = Pihole.run('''
+#     source /opt/pihole/basic-install.sh
+#     FTLdetect
+#     pihole-FTL version
+#     ''')
+#     expected_stdout = 'v'
+#     assert expected_stdout in installed_binary.stdout
 
 # def test_FTL_support_files_installed(Pihole):
 #     ''' confirms FTL support files are installed '''