Merge pull request #1429 from pi-hole/development

Release v3.0.1

.editorconfig

@@ -0,0 +1,38 @@
+# EditorConfig is awesome: http://EditorConfig.org
+
+# top-most EditorConfig file
+root = true
+
+# Unix-style newlines with a newline ending every file
+[*]
+end_of_line = lf
+insert_final_newline = true
+indent_style = space
+indent_size = tab
+tab_width = 2
+charset = utf-8
+trim_trailing_whitespace = true
+
+# Matches multiple files with brace expansion notation
+# Set default charset
+[*.{js,py}]
+charset = utf-8
+
+# 4 space indentation
+[*.py]
+indent_style = space
+indent_size = 4
+
+# Tab indentation (no size specified)
+[Makefile]
+indent_style = tab
+
+# Indentation override for all JS under lib directory
+[scripts/**.js]
+indent_style = space
+indent_size = 2
+
+# Matches the exact files either package.json or .travis.yml
+[{package.json,.travis.yml}]
+indent_style = space
+indent_size = 2

.github/ISSUE_TEMPLATE.md

@@ -2,7 +2,7 @@
 
 - [] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md).
 - [] The issue I am reporting can be *replicated*
-- [] The issue I'm reporting isn't a duplicate (see [FAQs](https://github.com/pi-hole/pi-hole/wiki/FAQs), [closed issues](https://github.com/pi-hole/pi-hole/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), and [open issues](https://github.com/pi-hole/pi-hole/issues)).
+- [] The issue I am reporting isn't a duplicate (see [FAQs](https://github.com/pi-hole/pi-hole/wiki/FAQs), [closed issues](https://github.com/pi-hole/pi-hole/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), and [open issues](https://github.com/pi-hole/pi-hole/issues)).
 
 **How familiar are you with the codebase?:**
 

.gitignore

@@ -4,4 +4,3 @@
 __pycache__
 .cache
 .pullapprove.yml
-

.pullapprove.yml

@@ -1,7 +1,7 @@
 version: 2
 
 always_pending:
-  title_regex: '(WIP|wip)'  
+  title_regex: '(WIP|wip)'
   labels:
     - wip
   explanation: 'This PR is a work in progress...'
@@ -22,17 +22,17 @@ groups:
       enabled: true
     conditions:
       branches:
-        - development        
+        - development
     required: 2
     teams:
       - approvers
-  
+
   master:
     approve_by_comment:
       enabled: true
     conditions:
       branches:
         - master
-    required: -1
+    required: 4
     teams:
       - admin

CONTRIBUTING.md

@@ -35,4 +35,5 @@ When requesting or submitting new features, first consider whether it might be u
 - Before Submitting your Pull Request, merge `development` with your new branch and fix any conflicts. (Make sure you don't break anything in development!)
 - Please use the [Google Style Guide for Shell](https://google.github.io/styleguide/shell.xml) for your code submission styles. 
 - Commit Unix line endings.
+- Please use the Pi-hole brand: **Pi-hole** (Take a special look at the capitalized 'P' and a low 'h' with a hyphen)
 - (Optional fun) keep to the theme of Star Trek/black holes/gravity.

README.md

@@ -5,12 +5,12 @@
 </p>
 
 <p align="center">
-<a href=https://discourse.pi-hole.net><img src="https://assets.pi-hole.net/static/Vortex_text.png" width=210></a>
+<a href=https://discourse.pi-hole.net><img src="https://assets.pi-hole.net/static/Vortex_with_text_and_TM.png" width=210></a>
 </p>
 
 ## The multi-platform, network-wide ad blocker
 
-Block ads for **all** your devices _without_ the need to install client-side software.  The Pi-hole blocks ads at the DNS-level, so all your devices are protected.
+Block ads for **all** your devices _without_ the need to install client-side software.  The Pi-hole™ blocks ads at the DNS-level, so all your devices are protected.
 
 - Web Browsers
 - Cell Phones
@@ -53,9 +53,9 @@ wget -O basic-install.sh https://install.pi-hole.net
 bash basic-install.sh
 ```
 
-Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to use Pi-hole as their DNS server.
+Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to use Pi-hole™ as their DNS server.
 
-## What is Pi-hole and how do I install it?
+## What is Pi-hole™ and how do I install it?
 <p align="center">
 <a href=https://www.youtube.com/watch?v=vKWjx1AQYgs><img src="https://assets.pi-hole.net/static/video-explainer.png"></a>
 </p>
@@ -73,7 +73,7 @@ Once installed, [configure your router to have **DHCP clients use the Pi as thei
 
 ## Technical Details
 
-The Pi-hole is an **advertising-aware DNS/Web server**. If an ad domain is queried, a small Web page or GIF is delivered in place of the advertisement.
+The Pi-hole™ is an **advertising-aware DNS/Web server**. If an ad domain is queried, a small Web page or GIF is delivered in place of the advertisement.
 
 ### Gravity
 
@@ -83,7 +83,7 @@ The [gravity.sh](https://github.com/pi-hole/pi-hole/blob/master/gravity.sh) does
 
 #### Other Operating Systems
 
-The automated install is only for a clean install of a Debian family or Fedora based system, such as the Raspberry Pi. However, this script will work for most UNIX-like systems, some with some slight **modifications** that we can help you work through. If you can install `dnsmasq` and a Webserver, it should work OK. If there are other platforms you'd like supported, let us know.
+The automated install is only for a clean install of a Debian family or Fedora based system, such as the Raspberry Pi. However, this script will work for most UNIX-like systems, some with some slight **modifications** that we can help you work through. If you can install `dnsmasq` and a web server, it should work OK. If there are other platforms you'd like supported, let us know.
 
 ### Web Interface
 
@@ -102,7 +102,7 @@ Domains can be whitelisted and blacklisted using either the web interface or the
 
 ### Settings
 
-The settings page lets you control and configure your Pi-hole.  You can do things like:
+The settings page lets you control and configure your Pi-hole™.  You can do things like:
 
 - enable Pi-hole's built-in DHCP server
 - exclude domains from the graphs
@@ -113,7 +113,7 @@ The settings page lets you control and configure your Pi-hole.  You can do thing
 
 #### Built-in DHCP Server
 
-Pi-hole ships with a built-in DHCP server.  This allows you to let your network devices use Pi-hole as their DNS server if your router does not let you adjust the DHCP options.
+Pi-hole™ ships with a built-in DHCP server.  This allows you to let your network devices use Pi-hole™ as their DNS server if your router does not let you adjust the DHCP options.
 <p align="center">
 <a href=hhttps://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245><img src="https://assets.pi-hole.net/static/piholedhcpserver.png"></a>
 </p>
@@ -137,14 +137,14 @@ The same output can be achieved on the CLI by running `chronometer.sh -j`
 
 You can view [real-time stats](https://discourse.pi-hole.net/t/how-do-i-view-my-pi-holes-stats-over-ssh-or-on-an-lcd-using-chronometer/240) via `ssh` or on an [2.8" LCD screen](http://amzn.to/1P0q1Fj). This is accomplished via [`chronometer.sh`](https://github.com/pi-hole/pi-hole/blob/master/advanced/Scripts/chronometer.sh). ![Pi-hole LCD](http://i.imgur.com/nBEqycp.jpg)
 
-## Pi-hole Projects
+## Pi-hole™ Projects
 
 -   [An ad blocking Magic Mirror](https://zonksec.com/blog/magic-mirror-dns-filtering/#dnssoftware)
 -   [Pi-hole stats in your Mac's menu bar](https://getbitbar.com/plugins/Network/pi-hole.1m.py)
 -   [Get LED alerts for each blocked ad](http://thetimmy.silvernight.org/pages/endisbutton/)
 -   [Pi-hole on Ubuntu 14.04 on VirtualBox](http://hbalagtas.blogspot.com/2016/02/adblocking-with-pi-hole-and-ubuntu-1404.html)
 -   [Docker Pi-hole container (x86 and ARM)](https://hub.docker.com/r/diginc/pi-hole/)
--   [Splunk: Pi-hole Visualizser](https://splunkbase.splunk.com/app/3023/)
+-   [Splunk: Pi-hole Visualiser](https://splunkbase.splunk.com/app/3023/)
 -   [Pi-hole Chrome extension](https://chrome.google.com/webstore/detail/pi-hole-list-editor/hlnoeoejkllgkjbnnnhfolapllcnaglh) ([open source](https://github.com/packtloss/pihole-extension))
 -   [Go Bananas for CHiP-hole ad blocking](https://www.hackster.io/jacobsalmela/chip-hole-network-wide-ad-blocker-98e037)
 -   [Sky-Hole](http://dlaa.me/blog/post/skyhole)
@@ -154,7 +154,7 @@ You can view [real-time stats](https://discourse.pi-hole.net/t/how-do-i-view-my-
 -   [Minibian Pi-hole](http://munkjensen.net/wiki/index.php/See_my_Pi-Hole#Minibian_Pi-hole)
 -   [Windows Tray Stat Application](https://github.com/goldbattle/copernicus)
 -   [Let your blink1 device blink when Pi-hole filters ads](https://gist.github.com/elpatron68/ec0b4c582e5abf604885ac1e068d233f)
--   [Pi-Hole Prometheus exporter](https://github.com/nlamirault/pihole_exporter) : a [Prometheus](https://prometheus.io/) exporter for Pi-Hole
+-   [Pi-hole Prometheus exporter](https://github.com/nlamirault/pihole_exporter): a [Prometheus](https://prometheus.io/) exporter for Pi-hole
 -   [Pi-hole Droid - open source Android client](https://github.com/friimaind/pi-hole-droid)
 
 ## Coverage

adlists.default

@@ -1,53 +1,23 @@
-## Pi-hole ad-list default sources. Updated 29/10/2016 #########################
-#                                                                              #
-#  To make changes to this file:                                               #
-#    1. run `cp /etc/pihole/adlists.default /etc/pihole/adlists.list`          #
-#    2. run `nano /etc/pihole/adlists.list`                                    #
-#    3. Uncomment or comment any of the below lists                            #
-#                                                                              #
-#  Know of any other lists? Feel free to let us know about them, or add them   #
-#  to this file!                                                               #
-################################################################################
-
 # The below list amalgamates several lists we used previously.
 # See `https://github.com/StevenBlack/hosts` for details
+##StevenBlack's list
 https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
 
-# Other lists we consider safe:
+##MalwareDomains
 https://mirror1.malwaredomains.com/files/justdomains
+
+##Cameleon
 http://sysctl.org/cameleon/hosts
+
+##Zeustracker
 https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
+
+##Disconnect.me Tracking
 https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
+
+##Disconnect.me Ads
 https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
 
-# hosts-file.net list. Updated frequently, but has been known to block legitimate sites.
+##Hosts-file.net
 https://hosts-file.net/ad_servers.txt
 
-# Mahakala list. Has been known to block legitimate domains including the entire .com range.
-# Warning: Due to the sheer size of this list, the web admin console will be unresponsive.
-#https://adblock.mahakala.is/
-
-# ADZHOSTS list. Has been known to block legitimate domains
-#http://pilotfiber.dl.sourceforge.net/project/adzhosts/HOSTS.txt
-
-# Windows 10 telemetry list
-#https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/win10/spy.txt
-
-# Securemecca.com list - Also blocks "adult" sites (pornography/gambling etc)
-#http://securemecca.com/Downloads/hosts.txt
-
-# Quidsup's tracker list
-#https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
-
-# Block the BBC News website Breaking News banner
-#https://raw.githubusercontent.com/BreakingTheNews/BreakingTheNews.github.io/master/hosts
-
-# Untested Lists:
-#https://raw.githubusercontent.com/reek/anti-adblock-killer/master/anti-adblock-killer-filters.txt
-#https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt
-#http://malwaredomains.lehigh.edu/files/domains.txt
-# Following two lists should be used simultaneously: (readme https://github.com/notracking/hosts-blocklists/)
-#https://raw.github.com/notracking/hosts-blocklists/master/hostnames.txt
-#https://raw.github.com/notracking/hosts-blocklists/master/domains.txt
-# Combination of several host files on the internet (warning some facebook domains are also blocked but you can go to facebook.com). See https://github.com/mat1th/Dns-add-block for more information.
-#https://raw.githubusercontent.com/mat1th/Dns-add-block/master/hosts

advanced/01-pihole.conf

@@ -25,6 +25,8 @@ addn-hosts=/etc/pihole/local.list
 
 domain-needed
 
+localise-queries
+
 bogus-priv
 
 no-resolv

advanced/Scripts/chronometer.sh

@@ -8,34 +8,55 @@
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.
 
-
-
-
 #Functions##############################################################################################################
 piLog="/var/log/pihole.log"
 gravity="/etc/pihole/gravity.list"
 
 . /etc/pihole/setupVars.conf
 
-# Borrowed/modified from https://gist.github.com/cjus/1047794
+function GetFTLData {
-function GetJSONValue {
+    # Open connection to FTL
-    retVal=$(echo $1 | sed 's/\\\\\//\//g' | \
+    exec 3<>/dev/tcp/localhost/"$(cat /var/run/pihole-FTL.port)"
-                       sed 's/[{}]//g' | \
+
-                       awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | \
+    # Test if connection is open
-                       sed 's/\"\:/\|/g' | \
+    if { >&3; } 2> /dev/null; then
-                       sed 's/[\,]/ /g' | \
+       # Send command to FTL
-                       sed 's/\"//g' | \
+       echo -e ">$1" >&3
-                       grep -w $2)
+
-    echo ${retVal##*|}
+       # Read input
+       read -r -t 1 LINE <&3
+       until [ ! $? ] || [[ "$LINE" == *"EOM"* ]]; do
+           echo "$LINE" >&1
+           read -r -t 1 LINE <&3
+       done
+
+       # Close connection
+       exec 3>&-
+       exec 3<&-
+   fi
 }
 
 outputJSON() {
-	json=$(curl -s -X GET http://127.0.0.1/admin/api.php?summaryRaw)
+	get_summary_data
-	echo ${json}
+	echo "{\"domains_being_blocked\":${domains_being_blocked_raw},\"dns_queries_today\":${dns_queries_today_raw},\"ads_blocked_today\":${ads_blocked_today_raw},\"ads_percentage_today\":${ads_percentage_today_raw}}"
+}
+
+get_summary_data() {
+	local summary=$(GetFTLData "stats")
+	domains_being_blocked_raw=$(grep "domains_being_blocked" <<< "${summary}" | grep -Eo "[0-9]+$")
+	domains_being_blocked=$(printf "%'.f" ${domains_being_blocked_raw})
+	dns_queries_today_raw=$(grep "dns_queries_today" <<< "$summary" | grep -Eo "[0-9]+$")
+	dns_queries_today=$(printf "%'.f" ${dns_queries_today_raw})
+	ads_blocked_today_raw=$(grep "ads_blocked_today" <<< "$summary" | grep -Eo "[0-9]+$")
+	ads_blocked_today=$(printf "%'.f" ${ads_blocked_today_raw})
+	ads_percentage_today_raw=$(grep "ads_percentage_today" <<< "$summary" | grep -Eo "[0-9.]+$")
+	LC_NUMERIC=C ads_percentage_today=$(printf "%'.f" ${ads_percentage_today_raw})
 }
 
 normalChrono() {
 	for (( ; ; )); do
+		get_summary_data
+		domain=$(GetFTLData recentBlocked)
 		clear
 		# Displays a colorful Pi-hole logo
 		echo " ___ _     _        _"
@@ -49,20 +70,12 @@ normalChrono() {
 		#uptime -p	#Doesn't work on all versions of uptime
 		uptime | awk -F'( |,|:)+' '{if ($7=="min") m=$6; else {if ($7~/^day/) {d=$6;h=$8;m=$9} else {h=$6;m=$7}}} {print d+0,"days,",h+0,"hours,",m+0,"minutes."}'
 		echo "-------------------------------"
-		# Uncomment to continually read the log file and display the current domain being blocked
+		echo "Recently blocked:"
-		#tail -f /var/log/pihole.log | awk '/\/etc\/pihole\/gravity.list/ {if ($7 != "address" && $7 != "name" && $7 != "/etc/pihole/gravity.list") print $7; else;}'
+		echo "  $domain"
 
-		json=$(curl -s -X GET http://127.0.0.1/admin/api.php?summaryRaw)
+		echo "Blocking:      ${domains_being_blocked}"
-
+		echo "Queries:       ${dns_queries_today}"
-    domains=$(printf "%'.f" $(GetJSONValue ${json} "domains_being_blocked")) #add commas in
+		echo "Pi-holed:      ${ads_blocked_today} (${ads_percentage_today}%)"
-    queries=$(printf "%'.f" $(GetJSONValue ${json} "dns_queries_today"))
-    blocked=$(printf "%'.f" $(GetJSONValue ${json} "ads_blocked_today"))
-    LC_NUMERIC=C percentage=$(printf "%0.2f\n" $(GetJSONValue ${json} "ads_percentage_today")) #2 decimal places
-
-		echo "Blocking:      ${domains}"
-		echo "Queries:       ${queries}"
-
-		echo "Pi-holed:      ${blocked} (${percentage}%)"
 
 		sleep 5
 	done

advanced/Scripts/list.sh

@@ -84,6 +84,9 @@ PoplistFile() {
 		if ${addmode}; then
 			AddDomain "${dom}" "${listMain}"
 			RemoveDomain "${dom}" "${listAlt}"
+			if [[ "${listMain}" == "${whitelist}" || "${listMain}" == "${blacklist}" ]]; then
+			  RemoveDomain "${dom}" "${wildcardlist}"
+			fi
 		else
 			RemoveDomain "${dom}" "${listMain}"
 		fi

advanced/Scripts/piholeCheckout.sh

@@ -0,0 +1,202 @@
+#!/usr/bin/env bash
+# Pi-hole: A black hole for Internet advertisements
+# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
+# Network-wide ad blocking via your own hardware.
+#
+# Checkout other branches than master
+#
+# This file is copyright under the latest version of the EUPL.
+# Please see LICENSE file for your rights under this license.
+
+readonly PI_HOLE_FILES_DIR="/etc/.pihole"
+PH_TEST="true" source "${PI_HOLE_FILES_DIR}/automated install/basic-install.sh"
+
+# webInterfaceGitUrl set in basic-install.sh
+# webInterfaceDir set in basic-install.sh
+# piholeGitURL set in basic-install.sh
+# is_repo() sourced from basic-install.sh
+# setupVars set in basic-install.sh
+
+source "${setupVars}"
+
+update="false"
+
+fully_fetch_repo() {
+  # Add upstream branches to shallow clone
+  local directory="${1}"
+
+  cd "${directory}" || return 1
+  if is_repo "${directory}"; then
+    git remote set-branches origin '*' || return 1
+    git fetch --quiet || return 1
+  else
+    return 1
+  fi
+  return 0
+}
+
+get_available_branches(){
+  # Return available branches
+  local directory="${1}"
+
+  cd "${directory}" || return 1
+  # Get reachable remote branches
+  git remote show origin | grep 'tracked' | sed 's/tracked//;s/ //g'
+  return
+}
+
+
+fetch_checkout_pull_branch() {
+  # Check out specified branch
+  local directory="${1}"
+  local branch="${2}"
+
+  # Set the reference for the requested branch, fetch, check it put and pull it
+  cd "${directory}"
+  git remote set-branches origin "${branch}" || return 1
+  git fetch --quiet || return 1
+  checkout_pull_branch "${directory}" "${branch}" || return 1
+}
+
+checkout_pull_branch() {
+  # Check out specified branch
+  local directory="${1}"
+  local branch="${2}"
+  local oldbranch
+
+  cd "${directory}" || return 1
+
+  oldbranch="$(git symbolic-ref HEAD)"
+
+  git checkout "${branch}" || return 1
+
+  if [ "$(git diff "${oldbranch}" | grep -c "^")" -gt "0" ]; then
+    update="true"
+  fi
+
+  git pull || return 1
+  return 0
+}
+
+warning1() {
+  echo "::: Note that changing the branch is a severe change of your Pi-hole system."
+  echo "::: This is not supported unless one of the developers explicitly asks you to do this!"
+  read -r -p "::: Have you read and understood this? [y/N] " response
+  case ${response} in
+  [yY][eE][sS]|[yY])
+    echo "::: Continuing."
+    return 0
+    ;;
+  *)
+    echo "::: Aborting."
+    return 1
+    ;;
+  esac
+}
+
+checkout()
+{
+  local corebranches
+  local webbranches
+
+  # Avoid globbing
+  set -f
+
+  #This is unlikely
+  if ! is_repo "${PI_HOLE_FILES_DIR}" ; then
+    echo "::: Critical Error: Core Pi-hole repo is missing from system!"
+    echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
+    exit 1;
+  fi
+  if [[ ${INSTALL_WEB} == "true" ]]; then
+    if ! is_repo "${webInterfaceDir}" ; then
+      echo "::: Critical Error: Web Admin repo is missing from system!"
+      echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
+      exit 1;
+    fi
+  fi
+
+  if [[ -z "${1}" ]]; then
+    echo "::: No option detected. Please use 'pihole checkout <master|dev>'."
+    echo "::: Or enter the repository and branch you would like to check out:"
+    echo "::: 'pihole checkout <web|core> <branchname>'"
+    exit 1
+  fi
+
+  if ! warning1 ; then
+    exit 1
+  fi
+
+  if [[ "${1}" == "dev" ]] ; then
+    # Shortcut to check out development branches
+    echo "::: Shortcut \"dev\" detected - checking out development / devel branches ..."
+    echo "::: Pi-hole core"
+    fetch_checkout_pull_branch "${PI_HOLE_FILES_DIR}" "development" || { echo "Unable to pull Core developement branch"; exit 1; }
+    if [[ ${INSTALL_WEB} == "true" ]]; then
+      echo "::: Web interface"
+      fetch_checkout_pull_branch "${webInterfaceDir}" "devel" || { echo "Unable to pull Web development branch"; exit 1; }
+    fi
+    echo "::: done!"
+  elif [[ "${1}" == "master" ]] ; then
+    # Shortcut to check out master branches
+    echo "::: Shortcut \"master\" detected - checking out master branches ..."
+    echo "::: Pi-hole core"
+    fetch_checkout_pull_branch "${PI_HOLE_FILES_DIR}" "master" || { echo "Unable to pull Core master branch"; exit 1; }
+    if [[ ${INSTALL_WEB} == "true" ]]; then
+      echo "::: Web interface"
+      fetch_checkout_pull_branch "${webInterfaceDir}" "master" || { echo "Unable to pull web master branch"; exit 1; }
+    fi
+    echo "::: done!"
+  elif [[ "${1}" == "core" ]] ; then
+    echo -n "::: Fetching remote branches for Pi-hole core from ${piholeGitUrl} ... "
+    if ! fully_fetch_repo "${PI_HOLE_FILES_DIR}" ; then
+      echo "::: Fetching all branches for Pi-hole core repo failed!"
+      exit 1
+    fi
+    corebranches=($(get_available_branches "${PI_HOLE_FILES_DIR}"))
+    echo " done!"
+    echo "::: ${#corebranches[@]} branches available"
+    echo ":::"
+    # Have to user chosing the branch he wants
+    if ! (for e in "${corebranches[@]}"; do [[ "$e" == "${2}" ]] && exit 0; done); then
+      echo "::: Requested branch \"${2}\" is not available!"
+      echo "::: Available branches for core are:"
+      for e in "${corebranches[@]}"; do echo ":::   $e"; done
+      exit 1
+    fi
+    checkout_pull_branch "${PI_HOLE_FILES_DIR}" "${2}"
+  elif [[ "${1}" == "web" && "${INSTALL_WEB}" == "true" ]] ; then
+    echo -n "::: Fetching remote branches for the web interface from ${webInterfaceGitUrl} ... "
+    if ! fully_fetch_repo "${webInterfaceDir}" ; then
+      echo "::: Fetching all branches for Pi-hole web interface repo failed!"
+      exit 1
+    fi
+    webbranches=($(get_available_branches "${webInterfaceDir}"))
+    echo " done!"
+    echo "::: ${#webbranches[@]} branches available"
+    echo ":::"
+    # Have to user chosing the branch he wants
+    if ! (for e in "${webbranches[@]}"; do [[ "$e" == "${2}" ]] && exit 0; done); then
+      echo "::: Requested branch \"${2}\" is not available!"
+      echo "::: Available branches for web are:"
+      for e in "${webbranches[@]}"; do echo ":::   $e"; done
+      exit 1
+    fi
+    checkout_pull_branch "${webInterfaceDir}" "${2}"
+  else
+    echo "::: Requested option \"${1}\" is not available!"
+    exit 1
+  fi
+
+  # Force updating everything
+  if [[ ! "${1}" == "web" && "${update}" == "true" ]]; then
+    echo "::: Running installer to upgrade your installation"
+    if "${PI_HOLE_FILES_DIR}/automated install/basic-install.sh" --unattended; then
+     exit 0
+    else
+     echo "Unable to complete update, contact Pi-hole"
+     exit 1
+    fi
+  fi
+}
+

advanced/Scripts/piholeDebug.sh

@@ -24,7 +24,10 @@ WHITELISTFILE="/etc/pihole/whitelist.txt"
 BLACKLISTFILE="/etc/pihole/blacklist.txt"
 ADLISTFILE="/etc/pihole/adlists.list"
 PIHOLELOG="/var/log/pihole.log"
+PIHOLEGITDIR="/etc/.pihole/"
+ADMINGITDIR="/var/www/html/admin/"
 WHITELISTMATCHES="/tmp/whitelistmatches.list"
+readonly FTLLOG="/var/log/pihole-FTL.log"
 
 TIMEOUT=60
 # Header info and introduction
@@ -41,16 +44,11 @@ cat << EOM
 ::: Please read and note any issues, and follow any directions advised during this process.
 EOM
 
-# Ensure the file exists, create if not, clear if exists.
-truncate --size=0 "${DEBUG_LOG}"
-chmod 644 ${DEBUG_LOG}
-chown "$USER":pihole ${DEBUG_LOG}
-
 source ${VARSFILE}
 
 ### Private functions exist here ###
 log_write() {
-    echo "${1}" >> "${DEBUG_LOG}"
+    echo "${@}" >&3
 }
 
 log_echo() {
@@ -75,14 +73,14 @@ log_echo() {
 
 header_write() {
   log_echo ""
-  log_echo "${1}"
+  log_echo "---= ${1}"
   log_write ""
 }
 
 file_parse() {
     while read -r line; do
 		  if [ ! -z "${line}" ]; then
-			  [[ "${line}" =~ ^#.*$  || ! "${line}" ]] && continue
+			  [[ "${line}" =~ ^#.*$  || ! "${line}" || "${line}" == "WEBPASSWORD="* ]] && continue
 				log_write "${line}"
 			fi
 		done < "${1}"
@@ -111,22 +109,61 @@ version_check() {
   header_write "Detecting Installed Package Versions:"
 
   local error_found
+  local pi_hole_ver
+  local pi_hole_branch
+  local pi_hole_commit
+  local admin_ver
+  local admin_branch
+  local admin_commit
+  local light_ver
+  local php_ver
+  local status
   error_found=0
 
-	local pi_hole_ver="$(cd /etc/.pihole/ && git describe --tags --abbrev=0)" \
+  cd "${PIHOLEGITDIR}" &> /dev/null || \
-	&& log_echo -r "Pi-hole: $pi_hole_ver" || (log_echo "Pi-hole git repository not detected." && error_found=1)
+    { status="Pi-hole git directory not found."; error_found=1; }
-	local admin_ver="$(cd /var/www/html/admin && git describe --tags --abbrev=0)" \
+  if git status &> /dev/null; then
-	&& log_echo -r "WebUI: $admin_ver" || (log_echo "Pi-hole Admin Pages git repository not detected." && error_found=1)
+    pi_hole_ver=$(git describe --tags --abbrev=0)
-	local light_ver="$(lighttpd -v |& head -n1 | cut -d " " -f1)" \
+    pi_hole_branch=$(git rev-parse --abbrev-ref HEAD)
-	&& log_echo -r "${light_ver}" || (log_echo "lighttpd not installed." && error_found=1)
+    pi_hole_commit=$(git describe --long --dirty --tags --always)
-	local php_ver="$(php -v |& head -n1)" \
+    log_echo -r "Pi-hole: ${pi_hole_ver:-Untagged} (${pi_hole_branch:-Detached}:${pi_hole_commit})"
-	&& log_echo -r "${php_ver}" || (log_echo "PHP not installed." && error_found=1)
+  else
+    status=${status:-"Pi-hole repository damaged."}
+    error_found=1
+  fi
+  if [[ "${status}" ]]; then
+    log_echo "${status}"
+    unset status
+  fi
 
-	(local pi_hole_branch="$(cd /etc/.pihole/ && git rev-parse --abbrev-ref HEAD)" && log_echo -r "Pi-hole branch:  ${pi_hole_branch}") || log_echo "Unable to obtain Pi-hole branch"
+  cd "${ADMINGITDIR}" || \
-	(local pi_hole_rev="$(cd /etc/.pihole/ && git describe --long --dirty --tags)" && log_echo -r "Pi-hole rev:     ${pi_hole_rev}") || log_echo "Unable to obtain Pi-hole revision"
+    { status="Pi-hole Dashboard git directory not found."; error_found=1; }
+  if git status &> /dev/null; then
+    admin_ver=$(git describe --tags --abbrev=0)
+    admin_branch=$(git rev-parse --abbrev-ref HEAD)
+    admin_commit=$(git describe --long --dirty --tags --always)
+    log_echo -r "Pi-hole Dashboard: ${admin_ver:-Untagged} (${admin_branch:-Detached}:${admin_commit})"
+  else
+    status=${status:-"Pi-hole Dashboard repository damaged."}
+    error_found=1
+  fi
+  if [[ "${status}" ]]; then
+    log_echo "${status}"
+    unset status
+  fi
 
-	(local admin_branch="$(cd /var/www/html/admin && git rev-parse --abbrev-ref HEAD)" && log_echo -r "AdminLTE branch: ${admin_branch}") || log_echo "Unable to obtain AdminLTE branch"
+	if light_ver=$(lighttpd -v |& head -n1 | cut -d " " -f1); then
-	(local admin_rev="$(cd /var/www/html/admin && git describe --long --dirty --tags)" && log_echo -r "AdminLTE rev:    ${admin_rev}") || log_echo "Unable to obtain AdminLTE revision"
+	  log_echo -r "${light_ver}"
+	else
+	  log_echo "lighttpd not installed."
+	  error_found=1
+	fi
+	if php_ver=$(php -v |& head -n1); then
+	  log_echo -r "${php_ver}"
+	else
+	  log_echo "PHP not installed."
+	  error_found=1
+	fi
 
 	return "${error_found}"
 }
@@ -190,6 +227,7 @@ ipv6_check() {
 ip_check() {
   local protocol=${1}
   local gravity=${2}
+  header_write "Checking IPv${protocol} Stack"
 
   local ip_addr_list="$(ip -${protocol} addr show dev ${PIHOLE_INTERFACE} | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }')"
   if [[ -n ${ip_addr_list} ]]; then
@@ -264,62 +302,78 @@ daemon_check() {
 }
 
 testResolver() {
-	header_write "Resolver Functions Check"
+  local protocol="${1}"
+  header_write "Resolver Functions Check (IPv${protocol})"
+  local IP="${2}"
+  local g_addr
+  local l_addr
+  local url
+  local testurl
+  local localdig
+  local piholedig
+  local remotedig
+
+  if [[ ${protocol} == "6" ]]; then
+    g_addr="2001:4860:4860::8888"
+    l_addr="::1"
+    r_type="AAAA"
+  else
+    g_addr="8.8.8.8"
+    l_addr="127.0.0.1"
+    r_type="A"
+  fi
 
 	# Find a blocked url that has not been whitelisted.
-	TESTURL="doubleclick.com"
+	url=$(shuf -n 1 "${GRAVITYFILE}" | awk -F ' ' '{ print $2 }')
-	if [ -s "${WHITELISTMATCHES}" ]; then
-		while read -r line; do
-			CUTURL=${line#*" "}
-			if [ "${CUTURL}" != "Pi-Hole.IsWorking.OK" ]; then
-				while read -r line2; do
-					CUTURL2=${line2#*" "}
-					if [ "${CUTURL}" != "${CUTURL2}" ]; then
-						TESTURL="${CUTURL}"
-						break 2
-					fi
-				done < "${WHITELISTMATCHES}"
-			fi
-		done < "${GRAVITYFILE}"
-	fi
 
-	log_write "Resolution of ${TESTURL} from Pi-hole:"
+	testurl="${url:-doubleclick.com}"
-	LOCALDIG=$(dig "${TESTURL}" @127.0.0.1)
+
-	if [[ $? = 0 ]]; then
+
-		log_write "${LOCALDIG}"
+	log_write "Resolution of ${testurl} from Pi-hole (${l_addr}):"
+	if localdig=$(dig -"${protocol}" "${testurl}" @${l_addr} +short "${r_type}"); then
+		log_write "${localdig}"
 	else
-		log_write "Failed to resolve ${TESTURL} on Pi-hole"
+		log_write "Failed to resolve ${testurl} on Pi-hole (${l_addr})"
+	fi
+	log_write ""
+
+	log_write "Resolution of ${testurl} from Pi-hole (${IP}):"
+	if piholedig=$(dig -"${protocol}" "${testurl}" @"${IP}" +short "${r_type}"); then
+		log_write "${piholedig}"
+	else
+		log_write "Failed to resolve ${testurl} on Pi-hole (${IP})"
 	fi
 	log_write ""
 
 
-	log_write "Resolution of ${TESTURL} from 8.8.8.8:"
+	log_write "Resolution of ${testurl} from ${g_addr}:"
-	REMOTEDIG=$(dig "${TESTURL}" @8.8.8.8)
+	if remotedig=$(dig -"${protocol}" "${testurl}" @${g_addr} +short "${r_type}"); then
-	if [[ $? = 0 ]]; then
+		log_write "${remotedig:-NXDOMAIN}"
-		log_write "${REMOTEDIG}"
 	else
-		log_write "Failed to resolve ${TESTURL} on 8.8.8.8"
+		log_write "Failed to resolve ${testurl} on upstream server ${g_addr}"
 	fi
 	log_write ""
-
-	log_write "Pi-hole dnsmasq specific records lookups"
-	log_write "Cache Size:"
-	dig +short chaos txt cachesize.bind >> ${DEBUG_LOG}
-	log_write "Upstream Servers:"
-	dig +short chaos txt servers.bind >> ${DEBUG_LOG}
-	log_write ""
 }
 
+testChaos(){
+  # Check Pi-hole specific records
+
+	log_write "Pi-hole dnsmasq specific records lookups"
+	log_write "Cache Size:"
+	log_write $(dig +short chaos txt cachesize.bind)
+	log_write "Upstream Servers:"
+	log_write $(dig +short chaos txt servers.bind)
+	log_write ""
+
+}
 checkProcesses() {
 	header_write "Processes Check"
 
-	echo ":::     Logging status of lighttpd and dnsmasq..."
+	echo ":::     Logging status of lighttpd, dnsmasq and pihole-FTL..."
-	PROCESSES=( lighttpd dnsmasq )
+	PROCESSES=( lighttpd dnsmasq pihole-FTL )
 	for i in "${PROCESSES[@]}"; do
-		log_write ""
+		log_write "Status for ${i} daemon:"
-		log_write "${i}"
+		log_write $(systemctl is-active "${i}")
-		log_write " processes status:"
-		systemctl -l status "${i}" >> "${DEBUG_LOG}"
 	done
 	log_write ""
 }
@@ -347,49 +401,6 @@ countdown() {
     tuvix=$(( tuvix - 5 ))
   done
 }
-### END FUNCTIONS ###
-
-# Gather version of required packages / repositories
-version_check || echo "REQUIRED FILES MISSING"
-# Check for newer setupVars storage file
-source_file "/etc/pihole/setupVars.conf"
-# Gather information about the running distribution
-distro_check || echo "Distro Check soft fail"
-# Gather processor type
-processor_check || echo "Processor Check soft fail"
-
-ip_check 6 ${IPV6_ADDRESS}
-ip_check 4 ${IPV4_ADDRESS}
-
-daemon_check lighttpd http
-daemon_check dnsmasq domain
-checkProcesses
-testResolver
-debugLighttpd
-
-files_check "${DNSMASQFILE}"
-dir_check "${DNSMASQCONFDIR}"
-files_check "${WHITELISTFILE}"
-files_check "${BLACKLISTFILE}"
-files_check "${ADLISTFILE}"
-
-
-header_write "Analyzing gravity.list"
-
-	gravity_length=$(grep -c ^ "${GRAVITYFILE}") \
-	&& log_write "${GRAVITYFILE} is ${gravity_length} lines long." \
-	|| log_echo "Warning: No gravity.list file found!"
-
-header_write "Analyzing pihole.log"
-
-  pihole_length=$(grep -c ^ "${PIHOLELOG}") \
-  && log_write "${PIHOLELOG} is ${pihole_length} lines long." \
-  || log_echo "Warning: No pihole.log file found!"
-
-  pihole_size=$(du -h "${PIHOLELOG}" | awk '{ print $1 }') \
-  && log_write "${PIHOLELOG} is ${pihole_size}." \
-  || log_echo "Warning: No pihole.log file found!"
-
 
 # Continuously append the pihole.log file to the pihole_debug.log file
 dumpPiHoleLog() {
@@ -401,7 +412,7 @@ dumpPiHoleLog() {
 	if [ -e "${PIHOLELOG}" ]; then
 	# Dummy process to use for flagging down tail to terminate
 	  countdown &
-		tail -n0 -f --pid=$! "${PIHOLELOG}" >> ${DEBUG_LOG}
+		tail -n0 -f --pid=$! "${PIHOLELOG}" >&4
 	else
 		log_write "No pihole.log file found!"
 		printf ":::\tNo pihole.log file found!\n"
@@ -412,6 +423,16 @@ dumpPiHoleLog() {
 finalWork() {
   local tricorder
 	echo "::: Finshed debugging!"
+
+  # Ensure the file exists, create if not, clear if exists.
+  truncate --size=0 "${DEBUG_LOG}"
+  chmod 644 ${DEBUG_LOG}
+  chown "$USER":pihole ${DEBUG_LOG}
+  # copy working temp file to final log location
+  cat /proc/$$/fd/3 >> "${DEBUG_LOG}"
+  # Straight dump of tailing the logs, can sanitize later if needed.
+  cat /proc/$$/fd/4 >> "${DEBUG_LOG}"
+
 	echo "::: The debug log can be uploaded to tricorder.pi-hole.net for sharing with developers only."
 	if [[ "${AUTOMATED}" ]]; then
 	  echo "::: Debug script running in automated mode, uploading log to tricorder..."
@@ -439,6 +460,82 @@ finalWork() {
 		echo "::: A local copy of the Debug log can be found at : /var/log/pihole_debug.log"
 }
 
+### END FUNCTIONS ###
+# Create temporary file for log
+TEMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX)
+# Open handle 3 for templog
+exec 3>"$TEMPLOG"
+# Delete templog, but allow for addressing via file handle.
+rm "$TEMPLOG"
+
+# Create temporary file for logdump using file handle 4
+DUMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX)
+exec 4>"$DUMPLOG"
+rm "$DUMPLOG"
+
+# Gather version of required packages / repositories
+version_check || echo "REQUIRED FILES MISSING"
+# Check for newer setupVars storage file
+source_file "/etc/pihole/setupVars.conf"
+# Gather information about the running distribution
+distro_check || echo "Distro Check soft fail"
+# Gather processor type
+processor_check || echo "Processor Check soft fail"
+
+ip_check 6 ${IPV6_ADDRESS}
+ip_check 4 ${IPV4_ADDRESS}
+
+daemon_check lighttpd http
+daemon_check dnsmasq domain
+daemon_check pihole-FTL 4711
+checkProcesses
+
+# Check local/IP/Google for IPv4 Resolution
+testResolver 4 "${IPV4_ADDRESS%/*}"
+# If IPv6 enabled, check resolution
+if [[ "${IPV6_ADDRESS}" ]]; then
+  testResolver 6 "${IPV6_ADDRESS%/*}"
+fi
+# Poll dnsmasq Pi-hole specific queries
+testChaos
+
+debugLighttpd
+
+files_check "${DNSMASQFILE}"
+dir_check "${DNSMASQCONFDIR}"
+files_check "${WHITELISTFILE}"
+files_check "${BLACKLISTFILE}"
+files_check "${ADLISTFILE}"
+
+
+header_write "Analyzing gravity.list"
+
+	gravity_length=$(grep -c ^ "${GRAVITYFILE}") \
+	&& log_write "${GRAVITYFILE} is ${gravity_length} lines long." \
+	|| log_echo "Warning: No gravity.list file found!"
+
+header_write "Analyzing pihole.log"
+
+  pihole_length=$(grep -c ^ "${PIHOLELOG}") \
+  && log_write "${PIHOLELOG} is ${pihole_length} lines long." \
+  || log_echo "Warning: No pihole.log file found!"
+
+  pihole_size=$(du -h "${PIHOLELOG}" | awk '{ print $1 }') \
+  && log_write "${PIHOLELOG} is ${pihole_size}." \
+  || log_echo "Warning: No pihole.log file found!"
+
+header_write "Analyzing pihole-FTL.log"
+
+  FTL_length=$(grep -c ^ "${FTLLOG}") \
+  && log_write "${FTLLOG} is ${FTL_length} lines long." \
+  || log_echo "Warning: No pihole-FTL.log file found!"
+
+  FTL_size=$(du -h "${FTLLOG}" | awk '{ print $1 }') \
+  && log_write "${FTLLOG} is ${FTL_size}." \
+  || log_echo "Warning: No pihole-FTL.log file found!"
+
+tail -n50 "${FTLLOG}" >&3
+
 trap finalWork EXIT
 
 ### Method calls for additional logging ###

advanced/Scripts/piholeLogFlush.sh

@@ -3,18 +3,22 @@
 # (c) 2017 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
 #
-# Flushes /var/log/pihole.log
+# Flushes Pi-hole's log file
 #
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.
 
-
-
 echo -n "::: Flushing /var/log/pihole.log ..."
 # Test if logrotate is available on this system
 if command -v /usr/sbin/logrotate &> /dev/null; then
+  # Flush twice to move all data out of sight of FTL
+  /usr/sbin/logrotate --force /etc/pihole/logrotate
   /usr/sbin/logrotate --force /etc/pihole/logrotate
 else
+  # Flush both pihole.log and pihole.log.1 (if existing)
   echo " " > /var/log/pihole.log
+  if [ -f /var/log/pihole.log.1 ]; then
+    echo " " > /var/log/pihole.log.1
+  fi
 fi
 echo "... done!"

advanced/Scripts/update.sh

@@ -75,13 +75,26 @@ GitCheckUpdateAvail() {
   fi
 }
 
+FTLcheckUpdate() {
+
+	local FTLversion=$(/usr/bin/pihole-FTL tag)
+	local FTLlatesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep 'Location' | awk -F '/' '{print $NF}' | tr -d '\r\n')
+
+	if [[ "${FTLversion}" != "${FTLlatesttag}" ]]; then
+		return 0
+	else
+		return 1
+	fi
+}
+
 main() {
   local pihole_version_current
   local web_version_current
+  source "${setupVars}"
 
   #This is unlikely
   if ! is_repo "${PI_HOLE_FILES_DIR}" ; then
-    echo "::: Critical Error: Core Pi-Hole repo is missing from system!"
+    echo "::: Critical Error: Core Pi-hole repo is missing from system!"
     echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
     exit 1;
   fi
@@ -96,6 +109,21 @@ main() {
     echo "::: Pi-hole Core:   up to date"
   fi
 
+  if FTLcheckUpdate ; then
+    FTL_update=true
+    echo "::: FTL:            update available"
+  else
+    FTL_update=false
+    echo "::: FTL:            up to date"
+  fi
+
+  if ${FTL_update}; then
+    echo ":::"
+    echo "::: FTL out of date"
+    FTLdetect
+    echo ":::"
+  fi
+
   if [[ ${INSTALL_WEB} == true ]]; then
     if ! is_repo "${ADMIN_INTERFACE_DIR}" ; then
       echo "::: Critical Error: Web Admin repo is missing from system!"
@@ -122,9 +150,11 @@ main() {
     #            pull pihole repo run install --unattended
 
     if ! ${core_update} && ! ${web_update} ; then
-      echo ":::"
+      if ! ${FTL_update} ; then
-      echo "::: Everything is up to date!"
+        echo ":::"
-      exit 0
+        echo "::: Everything is up to date!"
+        exit 0
+      fi
 
     elif ! ${core_update} && ${web_update} ; then
       echo ":::"
@@ -139,7 +169,7 @@ main() {
 
     elif ${core_update} && ${web_update} ; then
       echo ":::"
-      echo "::: Updating Everything"
+      echo "::: Updating Pi-hole core and web admin files"
       getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
       ${PI_HOLE_FILES_DIR}/automated\ install/basic-install.sh --unattended || echo "Unable to complete update, contact Pi-hole" && exit 1
     else
@@ -148,9 +178,11 @@ main() {
     fi
   else # Web Admin not installed, so only verify if core is up to date
     if ! ${core_update}; then
-      echo ":::"
+      if ! ${FTL_update} ; then
-      echo "::: Everything is up to date!"
+        echo ":::"
-      exit 0
+        echo "::: Everything is up to date!"
+        exit 0
+      fi
     else
       echo ":::"
       echo "::: Pi-hole core files out of date"
@@ -173,6 +205,15 @@ main() {
     echo "::: If you had made any changes in '/etc/.pihole/', they have been stashed using 'git stash'"
   fi
 
+  if [[ ${FTL_update} == true ]]; then
+    FTL_version_current="$(/usr/bin/pihole-FTL tag)"
+    echo ":::"
+    echo "::: FTL version is now at ${FTL_version_current}"
+    start_service pihole-FTL
+    enable_service pihole-FTL
+  fi
+
+
   echo ""
   exit 0
 

advanced/Scripts/version.sh

@@ -8,67 +8,104 @@
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.
 
-
+# Variables
-
-# Flags:
-latest=false
-current=false
-
 DEFAULT="-1"
+PHGITDIR="/etc/.pihole/"
+WEBGITDIR="/var/www/html/admin/"
+
+getLocalVersion() {
+  # Get the tagged version of the local repository
+  local directory="${1}"
+  local version
+
+  cd "${directory}" || { echo "${DEFAULT}"; return 1; }
+  version=$(git describe --tags --always || \
+            echo "${DEFAULT}")
+  if [[ "${version}" =~ ^v ]]; then
+    echo "${version}"
+  elif [[ "${version}" == "${DEFAULT}" ]]; then
+    echo "ERROR"
+    return 1
+  else
+    echo "Untagged"
+  fi
+  return 0
+}
+
+getLocalHash() {
+  # Get the short hash of the local repository
+  local directory="${1}"
+  local hash
+
+  cd "${directory}" || { echo "${DEFAULT}"; return 1; }
+  hash=$(git rev-parse --short HEAD || \
+         echo "${DEFAULT}")
+  if [[ "${hash}" == "${DEFAULT}" ]]; then
+    echo "ERROR"
+    return 1
+  else
+    echo "${hash}"
+  fi
+  return 0
+}
+
+getRemoteVersion(){
+  # Get the version from the remote origin
+  local daemon="${1}"
+  local version
+
+  version=$(curl --silent --fail https://api.github.com/repos/pi-hole/${daemon}/releases/latest | \
+            awk -F: '$1 ~/tag_name/ { print $2 }' | \
+            tr -cd '[[:alnum:]]._-')
+  if [[ "${version}" =~ ^v ]]; then
+    echo "${version}"
+  else
+    echo "ERROR"
+    return 1
+  fi
+  return 0
+}
+
+#PHHASHLATEST=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/commits/master | \
+#                   grep sha | \
+#                   head -n1 | \
+#                   awk -F ' ' '{ print $2 }' | \
+#                   tr -cd '[[:alnum:]]._-')
+
+#WEBHASHLATEST=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/commits/master | \
+#                   grep sha | \
+#                   head -n1 | \
+#                   awk -F ' ' '{ print $2 }' | \
+#                   tr -cd '[[:alnum:]]._-')
+
 
 normalOutput() {
-	piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
+	echo "::: Pi-hole version is $(getLocalVersion "${PHGITDIR}") (Latest version is $(getRemoteVersion pi-hole))"
-	webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
+	if [ -d "${WEBGITDIR}" ]; then
-
+		echo "::: Web-Admin version is $(getLocalVersion "${WEBGITDIR}") (Latest version is $(getRemoteVersion AdminLTE))"
-	piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
+	fi
-	webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-
-	echo "::: Pi-hole version is ${piholeVersion} (Latest version is ${piholeVersionLatest:-${DEFAULT}})"
-	echo "::: Web-Admin version is ${webVersion} (Latest version is ${webVersionLatest:-${DEFAULT}})"
 }
 
 webOutput() {
-	for var in "$@"; do
+  if [ -d "${WEBGITDIR}" ]; then
-		case "${var}" in
+    case "${1}" in
-			"-l" | "--latest"    ) latest=true;;
+      "-l" | "--latest"    ) echo $(getRemoteVersion AdminLTE);;
-			"-c" | "--current"   ) current=true;;
+      "-c" | "--current"   ) echo $(getLocalVersion "${WEBGITDIR}");;
-			*                    ) echo "::: Invalid Option!"; exit 1;
+      "-h" | "--hash"      ) echo $(getLocalHash "${WEBGITDIR}");;
-		esac
+      *                    ) echo "::: Invalid Option!"; exit 1;
-	done
+    esac
-
+  else
-	if [[ "${latest}" == true && "${current}" == false ]]; then
+    echo "::: Web interface not installed!"; exit 1;
-		webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
+  fi
-		echo "${webVersionLatest:--1}"
-	elif [[ "${latest}" == false && "${current}" == true ]]; then
-		webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
-		echo "${webVersion}"
-	else
-		webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
-		webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "::: Web-Admin version is ${webVersion} (Latest version is ${webVersionLatest:-${DEFAULT}})"
-	fi
 }
 
 coreOutput() {
-	for var in "$@"; do
+  case "${1}" in
-		case "${var}" in
+    "-l" | "--latest"    ) echo $(getRemoteVersion pi-hole);;
-			"-l" | "--latest"    ) latest=true;;
+    "-c" | "--current"   ) echo $(getLocalVersion "${PHGITDIR}");;
-			"-c" | "--current"   ) current=true;;
+    "-h" | "--hash"      ) echo $(getLocalHash "${PHGITDIR}");;
-			*                    ) echo "::: Invalid Option!"; exit 1;
+    *                    ) echo "::: Invalid Option!"; exit 1;
-		esac
+  esac
-	done
-
-	if [[ "${latest}" == true && "${current}" == false ]]; then
-		piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "${piholeVersionLatest:--1}"
-	elif [[ "${latest}" == false && "${current}" == true ]]; then
-		piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
-		echo "${piholeVersion}"
-	else
-		piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
-		piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "::: Pi-hole version is ${piholeVersion} (Latest version is ${piholeVersionLatest:-${DEFAULT}})"
-	fi
 }
 
 helpFunc() {
@@ -93,10 +130,8 @@ if [[ $# = 0 ]]; then
 	normalOutput
 fi
 
-for var in "$@"; do
+case "${1}" in
-	case "${var}" in
+  "-a" | "--admin"     ) shift; webOutput "$@";;
-	"-a" | "--admin"     ) shift; webOutput "$@";;
+  "-p" | "--pihole"    ) shift; coreOutput "$@" ;;
-	"-p" | "--pihole"    ) shift; coreOutput "$@" ;;
+  "-h" | "--help"      ) helpFunc;;
-	"-h" | "--help"      ) helpFunc;;
+esac
-	esac
-done

advanced/Scripts/webpage.sh

@@ -67,6 +67,13 @@ SetTemperatureUnit(){
 
 }
 
+HashPassword(){
+    # Compute password hash twice to avoid rainbow table vulnerability
+    return=$(echo -n ${1} | sha256sum | sed 's/\s.*$//')
+		return=$(echo -n ${return} | sha256sum | sed 's/\s.*$//')
+		echo ${return}
+}
+
 SetWebPassword(){
 
 	if [ "${SUDO_USER}" == "www-data" ]; then
@@ -81,19 +88,32 @@ SetWebPassword(){
 		exit 1
 	fi
 
-	# Set password only if there is one to be set
+  if (( ${#args[2]} > 0 )) ; then
-	if (( ${#args[2]} > 0 )) ; then
+    readonly PASSWORD="${args[2]}"
-		# Compute password hash twice to avoid rainbow table vulnerability
+    readonly CONFIRM="${PASSWORD}"
-		hash=$(echo -n ${args[2]} | sha256sum | sed 's/\s.*$//')
+  else
-		hash=$(echo -n ${hash} | sha256sum | sed 's/\s.*$//')
+    read -s -p "Enter New Password (Blank for no password): " PASSWORD
+    echo ""
+
+    if [ "${PASSWORD}" == "" ]; then
+      change_setting "WEBPASSWORD" ""
+      echo "Password Removed"
+      exit 0
+    fi
+
+    read -s -p "Confirm Password: " CONFIRM
+    echo ""
+  fi
+
+	if [ "${PASSWORD}" == "${CONFIRM}" ] ; then
+		hash=$(HashPassword ${PASSWORD})
 		# Save hash to file
 		change_setting "WEBPASSWORD" "${hash}"
 		echo "New password set"
 	else
-		change_setting "WEBPASSWORD" ""
+		echo "Passwords don't match. Your password has not been changed"
-		echo "Password removed"
+		exit 1
 	fi
-
 }
 
 ProcessDNSSettings() {
@@ -134,7 +154,7 @@ trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE3
 
 	delete_dnsmasq_setting "host-record"
 
-	if [ ! -z "${HOSTRECORD+x}" ]; then
+	if [ ! -z "${HOSTRECORD}" ]; then
 		add_dnsmasq_setting "host-record" "${HOSTRECORD}"
 	fi
 
@@ -319,6 +339,25 @@ SetWebUILayout(){
 
 }
 
+CustomizeAdLists() {
+
+  list="/etc/pihole/adlists.list"
+
+	if [[ "${args[2]}" == "enable" ]] ; then
+		sed -i "\\@${args[3]}@s/^#http/http/g" "${list}"
+	elif [[ "${args[2]}" == "disable" ]] ; then
+		sed -i "\\@${args[3]}@s/^http/#http/g" "${list}"
+	elif [[ "${args[2]}" == "add" ]] ; then
+		echo "${args[3]}" >> ${list}
+	elif [[ "${args[2]}" == "del" ]] ; then
+	  var=$(echo "${args[3]}" | sed 's/\//\\\//g')
+	  sed -i "/${var}/Id" "${list}"
+	else
+		echo "Not permitted"
+		return 1
+  fi
+}
+
 SetPrivacyMode(){
 
 	if [[ "${args[2]}" == "true" ]] ; then
@@ -410,6 +449,12 @@ SetListeningMode(){
 
 }
 
+Teleporter()
+{
+	local datetimestamp=$(date "+%Y-%m-%d_%H-%M-%S")
+	php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-teleporter_${datetimestamp}.zip"
+}
+
 main() {
 
 	args=("$@")
@@ -435,6 +480,8 @@ main() {
 		"removestaticdhcp"  ) RemoveDHCPStaticAddress;;
 		"hostrecord"        ) SetHostRecord;;
 		"-i" | "interface"  ) SetListeningMode;;
+		"-t" | "teleporter" ) Teleporter;;
+		"adlist"            ) CustomizeAdLists;;
 		*                   ) helpFunc;;
 	esac
 

advanced/bash-completion/pihole

@@ -3,7 +3,7 @@ _pihole() {
 	COMPREPLY=()
 	cur="${COMP_WORDS[COMP_CWORD]}"
 	prev="${COMP_WORDS[COMP_CWORD-1]}"
-	opts="admin blacklist chronometer debug disable enable flush help logging query reconfigure restartdns setupLCD status tail uninstall updateGravity updatePihole version whitelist"
+	opts="admin blacklist chronometer debug disable enable flush help logging query reconfigure restartdns setupLCD status tail uninstall updateGravity updatePihole version whitelist checkout"
 
 	COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
 	return 0

advanced/index.php

@@ -1,15 +1,55 @@
 <?php
-/* Detailed Pi-Hole Block Page: Show "Website Blocked" if user browses to site, but not to image/file requests based on the work of WaLLy3K for DietPi & Pi-Hole */
+/* Detailed Pi-hole Block Page: Show "Website Blocked" if user browses to site, but not to image/file requests based on the work of WaLLy3K for DietPi & Pi-Hole */
+
+function validIP($address){
+	if (preg_match('/[.:0]/', $address) && !preg_match('/[1-9a-f]/', $address)) {
+		// Test if address contains either `:` or `0` but not 1-9 or a-f
+		return false;
+	}
+	return !filter_var($address, FILTER_VALIDATE_IP) === false;
+}
 
 $uri = escapeshellcmd($_SERVER['REQUEST_URI']);
 $serverName = escapeshellcmd($_SERVER['SERVER_NAME']);
 
+// If the server name is 'pi.hole', it's likely a user trying to get to the admin panel.
+// Let's be nice and redirect them.
+if ($serverName === 'pi.hole')
+{
+    header('HTTP/1.1 301 Moved Permanently');
+    header("Location: /admin/");
+}
+
 // Retrieve server URI extension (EG: jpg, exe, php)
+ini_set('pcre.recursion_limit',100);
 $uriExt = pathinfo($uri, PATHINFO_EXTENSION);
 
 // Define which URL extensions get rendered as "Website Blocked"
 $webExt = array('asp', 'htm', 'html', 'php', 'rss', 'xml');
 
+// Get IPv4 and IPv6 addresses from setupVars.conf (if available)
+$setupVars = parse_ini_file("/etc/pihole/setupVars.conf");
+$ipv4 = isset($setupVars["IPV4_ADDRESS"]) ? explode("/", $setupVars["IPV4_ADDRESS"])[0] : $_SERVER['SERVER_ADDR'];
+$ipv6 = isset($setupVars["IPV6_ADDRESS"]) ? explode("/", $setupVars["IPV6_ADDRESS"])[0] : $_SERVER['SERVER_ADDR'];
+
+$AUTHORIZED_HOSTNAMES = array(
+	$ipv4,
+	$ipv6,
+	str_replace(array("[","]"), array("",""), $_SERVER["SERVER_ADDR"]),
+	"pi.hole",
+	"localhost");
+// Allow user set virtual hostnames
+$virtual_host = getenv('VIRTUAL_HOST');
+if (!empty($virtual_host))
+	array_push($AUTHORIZED_HOSTNAMES, $virtual_host);
+
+// Immediately quit since we didn't block this page (the IP address or pi.hole is explicitly requested)
+if(validIP($serverName) || in_array($serverName,$AUTHORIZED_HOSTNAMES))
+{
+	http_response_code(404);
+	die();
+}
+
 if(in_array($uriExt, $webExt) || empty($uriExt))
 {
 	// Requested resource has an extension listed in $webExt
@@ -37,7 +77,7 @@ if (!$showPage)
 	die();
 }
 
-// Get Pi-Hole version
+// Get Pi-hole version
 $piHoleVersion = exec('cd /etc/.pihole/ && git describe --tags --abbrev=0');
 
 // Don't show the URI if it is the root directory
@@ -48,6 +88,7 @@ if($uri == "/")
 
 ?>
 <!DOCTYPE html>
+<html>
 <head>
 	<meta charset='UTF-8'/>
 	<title>Website Blocked</title>

advanced/pihole-FTL.service

@@ -0,0 +1,80 @@
+#!/bin/bash
+### BEGIN INIT INFO
+# Provides:          pihole-FTL
+# Required-Start:    $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: pihole-FTL daemon
+# Description:       Enable service provided by pihole-FTL daemon
+### END INIT INFO
+
+FTLUSER=pihole
+PIDFILE=/var/run/pihole-FTL.pid
+
+get_pid() {
+    pidof "pihole-FTL"
+}
+
+is_running() {
+    ps "$(get_pid)" > /dev/null 2>&1
+}
+
+# Start the service
+start() {
+  if is_running; then
+    echo "pihole-FTL is already running"
+  else
+    touch /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
+    chown pihole:pihole /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
+    chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
+    su -s /bin/sh -c "/usr/bin/pihole-FTL" "$FTLUSER"
+    echo
+  fi
+}
+
+# Stop the service
+stop() {
+  if is_running; then
+    kill "$(get_pid)"
+    for i in {1..5}; do
+      if ! is_running; then
+        break
+      fi
+
+      echo -n "."
+      sleep 1
+    done
+    echo
+
+    if is_running; then
+      echo "Not stopped; may still be shutting down or shutdown may have failed, killing now"
+      kill -9 "$(get_pid)"
+      exit 1
+    else
+      echo "Stopped"
+    fi
+  else
+    echo "Not running"
+  fi
+  echo
+}
+
+### main logic ###
+case "$1" in
+  stop)
+        stop
+        ;;
+  status)
+        status pihole-FTL
+        ;;
+  start|restart|reload|condrestart)
+        stop
+        start
+        ;;
+  *)
+        echo $"Usage: $0 {start|stop|restart|reload|status}"
+        exit 1
+esac
+
+exit 0

automated install/basic-install.sh

@@ -54,13 +54,39 @@ skipSpaceCheck=false
 reconfigure=false
 runUnattended=false
 
+show_ascii_berry() {
+  echo "
+        .;;,.
+        .ccccc:,.
+         :cccclll:.      ..,,
+          :ccccclll.   ;ooodc
+           'ccll:;ll .oooodc
+             .;cll.;;looo:.
+                 .. ','.
+                .',,,,,,'.
+              .',,,,,,,,,,.
+            .',,,,,,,,,,,,....
+          ....''',,,,,,,'.......
+        .........  ....  .........
+        ..........      ..........
+        ..........      ..........
+        .........  ....  .........
+          ........,,,,,,,'......
+            ....',,,,,,,,,,,,.
+               .',,,,,,,,,'.
+                .',,,,,,'.
+                  ..'''.
+"
+}
+
+
 # Compatibility
 distro_check() {
 if command -v apt-get &> /dev/null; then
   #Debian Family
   #############################################
   PKG_MANAGER="apt-get"
-  UPDATE_PKG_CACHE="${PKG_MANAGER} update"
+  UPDATE_PKG_CACHE="test_dpkg_lock; ${PKG_MANAGER} update"
   PKG_INSTALL=(${PKG_MANAGER} --yes --no-install-recommends install)
   # grep -c will return 1 retVal on 0 matches, block this throwing the set -e with an OR TRUE
   PKG_COUNT="${PKG_MANAGER} -s -o Debug::NoLocking=true upgrade | grep -c ^Inst || true"
@@ -79,7 +105,7 @@ if command -v apt-get &> /dev/null; then
     phpVer="php5"
   fi
   # #########################################
-  INSTALLER_DEPS=(apt-utils debconf dhcpcd5 git ${iproute_pkg} whiptail)
+  INSTALLER_DEPS=(apt-utils dialog debconf dhcpcd5 git ${iproute_pkg} whiptail)
   PIHOLE_DEPS=(bc cron curl dnsmasq dnsutils iputils-ping lsof netcat sudo unzip wget)
   PIHOLE_WEB_DEPS=(lighttpd ${phpVer}-common ${phpVer}-cgi)
   LIGHTTPD_USER="www-data"
@@ -87,6 +113,17 @@ if command -v apt-get &> /dev/null; then
   LIGHTTPD_CFG="lighttpd.conf.debian"
   DNSMASQ_USER="dnsmasq"
 
+  test_dpkg_lock() {
+    i=0
+    while fuser /var/lib/dpkg/lock >/dev/null 2>&1 ; do
+      sleep 0.5
+      ((i=i+1))
+    done
+    # Always return success, since we only return if there is no
+    # lock (anymore)
+    return 0
+  }
+
 elif command -v rpm &> /dev/null; then
   # Fedora Family
   if command -v dnf &> /dev/null; then
@@ -99,7 +136,7 @@ elif command -v rpm &> /dev/null; then
   UPDATE_PKG_CACHE=":"
   PKG_INSTALL=(${PKG_MANAGER} install -y)
   PKG_COUNT="${PKG_MANAGER} check-update | egrep '(.i686|.x86|.noarch|.arm|.src)' | wc -l"
-  INSTALLER_DEPS=(git iproute net-tools newt procps-ng)
+  INSTALLER_DEPS=(dialog git iproute net-tools newt procps-ng)
   PIHOLE_DEPS=(bc bind-utils cronie curl dnsmasq findutils nmap-ncat sudo unzip wget)
   PIHOLE_WEB_DEPS=(lighttpd lighttpd-fastcgi php php-common php-cli)
   if ! grep -q 'Fedora' /etc/redhat-release; then
@@ -196,7 +233,7 @@ find_IPv4_information() {
 
 get_available_interfaces() {
   # Get available UP interfaces.
-  availableInterfaces=$(ip -o link | grep -v "state DOWN\|lo" | awk '{print $2}' | cut -d':' -f1 | cut -d'@' -f1)
+  availableInterfaces=$(ip --oneline link show up | grep -v "lo" | awk '{print $2}' | cut -d':' -f1 | cut -d'@' -f1)
 }
 
 welcomeDialogs() {
@@ -231,7 +268,7 @@ verifyFreeDiskSpace() {
   # - Insufficient free disk space
   elif [[ ${existing_free_kilobytes} -lt ${required_free_kilobytes} ]]; then
     echo "::: Insufficient Disk Space!"
-    echo "::: Your system appears to be low on disk space. pi-hole recommends a minimum of $required_free_kilobytes KiloBytes."
+    echo "::: Your system appears to be low on disk space. Pi-hole recommends a minimum of $required_free_kilobytes KiloBytes."
     echo "::: You only have ${existing_free_kilobytes} KiloBytes free."
     echo "::: If this is a new install you may need to expand your disk."
     echo "::: Try running 'sudo raspi-config', and choose the 'expand file system option'"
@@ -401,7 +438,7 @@ setStaticIPv4() {
       cp "${IFCFG_FILE}" "${IFCFG_FILE}".pihole.orig
       # Build Interface configuration file:
       {
-        echo "# Configured via Pi-Hole installer"
+        echo "# Configured via Pi-hole installer"
         echo "DEVICE=$PIHOLE_INTERFACE"
         echo "BOOTPROTO=none"
         echo "ONBOOT=yes"
@@ -582,14 +619,14 @@ version_check_dnsmasq() {
   local dnsmasq_conf="/etc/dnsmasq.conf"
   local dnsmasq_conf_orig="/etc/dnsmasq.conf.orig"
   local dnsmasq_pihole_id_string="addn-hosts=/etc/pihole/gravity.list"
-  local dnsmasq_original_config="/etc/.pihole/advanced/dnsmasq.conf.original"
+  local dnsmasq_original_config="${PI_HOLE_LOCAL_REPO}/advanced/dnsmasq.conf.original"
-  local dnsmasq_pihole_01_snippet="/etc/.pihole/advanced/01-pihole.conf"
+  local dnsmasq_pihole_01_snippet="${PI_HOLE_LOCAL_REPO}/advanced/01-pihole.conf"
   local dnsmasq_pihole_01_location="/etc/dnsmasq.d/01-pihole.conf"
 
   if [ -f ${dnsmasq_conf} ]; then
     echo -n ":::    Existing dnsmasq.conf found..."
     if grep -q ${dnsmasq_pihole_id_string} ${dnsmasq_conf}; then
-      echo " it is from a previous pi-hole install."
+      echo " it is from a previous Pi-hole install."
       echo -n ":::    Backing up dnsmasq.conf to dnsmasq.conf.orig..."
       mv -f ${dnsmasq_conf} ${dnsmasq_conf_orig}
       echo " done."
@@ -597,7 +634,7 @@ version_check_dnsmasq() {
       cp ${dnsmasq_original_config} ${dnsmasq_conf}
       echo " done."
     else
-      echo " it is not a pi-hole file, leaving alone!"
+      echo " it is not a Pi-hole file, leaving alone!"
     fi
   else
     echo -n ":::    No dnsmasq.conf found.. restoring default dnsmasq.conf..."
@@ -669,9 +706,9 @@ installScripts() {
 }
 
 installConfigs() {
-  # Install the configs from /etc/.pihole to their various locations
+  # Install the configs from PI_HOLE_LOCAL_REPO to their various locations
   echo ":::"
-  echo "::: Installing configs..."
+  echo "::: Installing configs from ${PI_HOLE_LOCAL_REPO}..."
   version_check_dnsmasq
 
   #Only mess with lighttpd configs if user has chosen to install web interface
@@ -682,7 +719,7 @@ installConfigs() {
     elif [ -f "/etc/lighttpd/lighttpd.conf" ]; then
       mv /etc/lighttpd/lighttpd.conf /etc/lighttpd/lighttpd.conf.orig
     fi
-    cp /etc/.pihole/advanced/${LIGHTTPD_CFG} /etc/lighttpd/lighttpd.conf
+    cp ${PI_HOLE_LOCAL_REPO}/advanced/${LIGHTTPD_CFG} /etc/lighttpd/lighttpd.conf
     mkdir -p /var/run/lighttpd
     chown ${LIGHTTPD_USER}:${LIGHTTPD_GROUP} /var/run/lighttpd
     mkdir -p /var/cache/lighttpd/compress
@@ -739,10 +776,11 @@ update_package_cache() {
 
   echo ":::"
   echo -n "::: Updating local cache of available packages..."
-  if eval ${UPDATE_PKG_CACHE} &> /dev/null; then
+  if eval "${UPDATE_PKG_CACHE}" &> /dev/null; then
     echo " done!"
   else
-    echo -n "\n!!! ERROR - Unable to update package cache. Please try \"${UPDATE_PKG_CACHE}\""
+    echo -en "\n!!! ERROR - Unable to update package cache. Please try \"${UPDATE_PKG_CACHE}\""
+    return 1
   fi
 }
 
@@ -759,7 +797,7 @@ notify_package_updates_available() {
       echo "::: Your system is up to date! Continuing with Pi-hole installation..."
     else
       echo "::: There are ${updatesToInstall} updates available for your system!"
-      echo "::: We recommend you update your OS after installing Pi-Hole! "
+      echo "::: We recommend you update your OS after installing Pi-hole! "
       echo ":::"
     fi
   else
@@ -789,6 +827,7 @@ install_dependent_packages() {
       fi
     done
     if [[ ${#installArray[@]} -gt 0 ]]; then
+      test_dpkg_lock
       debconf-apt-progress -- "${PKG_INSTALL[@]}" "${installArray[@]}"
       return
     fi
@@ -835,7 +874,7 @@ installPiholeWeb() {
       echo ":::     Existing index.php detected, not overwriting"
     else
       echo -n ":::     index.php missing, replacing... "
-      cp /etc/.pihole/advanced/index.php /var/www/html/pihole/
+      cp ${PI_HOLE_LOCAL_REPO}/advanced/index.php /var/www/html/pihole/
       echo " done!"
     fi
 
@@ -843,7 +882,7 @@ installPiholeWeb() {
       echo ":::     Existing index.js detected, not overwriting"
     else
       echo -n ":::     index.js missing, replacing... "
-      cp /etc/.pihole/advanced/index.js /var/www/html/pihole/
+      cp ${PI_HOLE_LOCAL_REPO}/advanced/index.js /var/www/html/pihole/
       echo " done!"
     fi
 
@@ -851,14 +890,14 @@ installPiholeWeb() {
       echo ":::     Existing blockingpage.css detected, not overwriting"
     else
       echo -n ":::     blockingpage.css missing, replacing... "
-      cp /etc/.pihole/advanced/blockingpage.css /var/www/html/pihole
+      cp ${PI_HOLE_LOCAL_REPO}/advanced/blockingpage.css /var/www/html/pihole
       echo " done!"
     fi
 
   else
     echo ":::     Creating directory for blocking page"
     install -d /var/www/html/pihole
-    install -D /etc/.pihole/advanced/{index,blockingpage}.* /var/www/html/pihole/
+    install -D ${PI_HOLE_LOCAL_REPO}/advanced/{index,blockingpage}.* /var/www/html/pihole/
     if [ -f /var/www/html/index.lighttpd.html ]; then
       mv /var/www/html/index.lighttpd.html /var/www/html/index.lighttpd.orig
     else
@@ -871,7 +910,7 @@ installPiholeWeb() {
   echo ":::"
   echo -n "::: Installing sudoer file..."
   mkdir -p /etc/sudoers.d/
-  cp /etc/.pihole/advanced/pihole.sudo /etc/sudoers.d/pihole
+  cp ${PI_HOLE_LOCAL_REPO}/advanced/pihole.sudo /etc/sudoers.d/pihole
   # Add lighttpd user (OS dependent) to sudoers file
   echo "${LIGHTTPD_USER} ALL=NOPASSWD: /usr/local/bin/pihole" >> /etc/sudoers.d/pihole
 
@@ -889,7 +928,7 @@ installCron() {
   # Install the cron job
   echo ":::"
   echo -n "::: Installing latest Cron script..."
-  cp /etc/.pihole/advanced/pihole.cron /etc/cron.d/pihole
+  cp ${PI_HOLE_LOCAL_REPO}/advanced/pihole.cron /etc/cron.d/pihole
   echo " done!"
 }
 
@@ -903,7 +942,7 @@ runGravity() {
   fi
   # Test if /etc/pihole/adlists.default exists
   if [[ ! -e /etc/pihole/adlists.default ]]; then
-    cp /etc/.pihole/adlists.default /etc/pihole/adlists.default
+    cp ${PI_HOLE_LOCAL_REPO}/adlists.default /etc/pihole/adlists.default
   fi
   echo "::: Running gravity.sh"
   { /opt/pihole/gravity.sh; }
@@ -926,7 +965,7 @@ configureFirewall() {
     whiptail --title "Firewall in use" --yesno "We have detected a running firewall\n\nPi-hole currently requires HTTP and DNS port access.\n\n\n\nInstall Pi-hole default firewall rules?" ${r} ${c} || \
     { echo -e ":::\n::: Not installing firewall rulesets."; return 0; }
     echo -e ":::\n:::\n Configuring FirewallD for httpd and dnsmasq."
-    firewall-cmd --permanent --add-port=80/tcp --add-port=53/tcp --add-port=53/udp
+    firewall-cmd --permanent --add-service=http --add-service=dns
     firewall-cmd --reload
     return 0
   # Check for proper kernel modules to prevent failure
@@ -978,7 +1017,7 @@ finalExports() {
 
   # Look for DNS server settings which would have to be reapplied
   source "${setupVars}"
-  source "/etc/.pihole/advanced/Scripts/webpage.sh"
+  source "${PI_HOLE_LOCAL_REPO}/advanced/Scripts/webpage.sh"
 
   if [[ "${DNS_FQDN_REQUIRED}" != "" ]] ; then
     ProcessDNSSettings
@@ -993,7 +1032,7 @@ installLogrotate() {
   # Install the logrotate script
   echo ":::"
   echo -n "::: Installing latest logrotate script..."
-  cp /etc/.pihole/advanced/logrotate /etc/pihole/logrotate
+  cp ${PI_HOLE_LOCAL_REPO}/advanced/logrotate /etc/pihole/logrotate
   # Different operating systems have different user / group
   # settings for logrotate that makes it impossible to create
   # a static logrotate file that will work with e.g.
@@ -1032,6 +1071,7 @@ installPihole() {
   fi
   installCron
   installLogrotate
+  FTLdetect || echo "::: FTL Engine not installed."
   configureFirewall
   finalExports
   #runGravity
@@ -1063,6 +1103,7 @@ updatePihole() {
   fi
   installCron
   installLogrotate
+  FTLdetect || echo "::: FTL Engine not installed."
   finalExports #re-export setupVars.conf to account for any new vars added in new versions
   #runGravity
 }
@@ -1156,10 +1197,98 @@ if [[ "${reconfigure}" == true ]]; then
     fi
 }
 
+FTLinstall() {
+  # Download and install FTL binary
+  local binary="${1}"
+  local latesttag
+  local orig_dir
+  echo -n ":::  Installing FTL... "
+
+  orig_dir="${PWD}"
+  latesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep "Location" | awk -F '/' '{print $NF}')
+  # Tags should always start with v, check for that.
+  if [[ ! "${latesttag}" == v* ]]; then
+    echo "failed (error in getting latest release location from GitHub)"
+    return 1
+  fi
+  if curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${latesttag%$'\r'}/${binary}" -o "/tmp/${binary}"; then
+    # Get sha1 of the binary we just downloaded for verification.
+    curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${latesttag%$'\r'}/${binary}.sha1" -o "/tmp/${binary}.sha1"
+    # Check if we just downloaded text, or a binary file.
+    cd /tmp
+    if sha1sum --status --quiet -c "${binary}".sha1; then
+      echo -n "transferred... "
+      stop_service pihole-FTL &> /dev/null
+      install -T -m 0755 /tmp/${binary} /usr/bin/pihole-FTL
+      cd "${orig_dir}"
+      install -T -m 0755 "${PI_HOLE_LOCAL_REPO}/advanced/pihole-FTL.service" "/etc/init.d/pihole-FTL"
+      echo "done."
+      return 0
+    else
+      echo "failed (download of binary from Github failed)"
+      cd "${orig_dir}"
+      return 1
+    fi
+  else
+    cd "${orig_dir}"
+    echo "failed (URL not found.)"
+  fi
+}
+
+FTLdetect() {
+  # Detect suitable FTL binary platform
+  echo ":::"
+  echo "::: Downloading latest version of FTL..."
+
+  local machine
+  local binary
+
+  machine=$(uname -m)
+
+  if [[ $machine == arm* || $machine == *aarch* ]]; then
+    # ARM
+    local rev=$(uname -m | sed "s/[^0-9]//g;")
+    local lib=$(ldd /bin/ls | grep -E '^\s*/lib' | awk '{ print $1 }')
+    if [[ "$lib" == "/lib/ld-linux-aarch64.so.1" ]]; then
+      echo ":::  Detected ARM-aarch64 architecture"
+      binary="pihole-FTL-aarch64-linux-gnu"
+    elif [[ "$lib" == "/lib/ld-linux-armhf.so.3" ]]; then
+      if [ "$rev" -gt "6" ]; then
+        echo ":::  Detected ARM-hf architecture (armv7+)"
+        binary="pihole-FTL-arm-linux-gnueabihf"
+      else
+        echo ":::  Detected ARM-hf architecture (armv6 or lower)"
+        echo ":::  Using ARM binary"
+        binary="pihole-FTL-arm-linux-gnueabi"
+      fi
+    else
+      echo ":::  Detected ARM architecture"
+      binary="pihole-FTL-arm-linux-gnueabi"
+    fi
+  elif [[ $machine == x86_64 ]]; then
+    # 64bit
+    echo ":::  Detected x86_64 architecture"
+    binary="pihole-FTL-linux-x86_64"
+  else
+    # Something else - we try to use 32bit executable and warn the user
+    if [[ ! $machine == i686 ]]; then
+      echo ":::  Not able to detect architecture (unknown: ${machine}), trying 32bit executable"
+      echo ":::  Contact Pi-hole support if you experience problems (like FTL not running)"
+    else
+      echo ":::  Detected 32bit (i686) architecture"
+    fi
+    binary="pihole-FTL-linux-x86_32"
+  fi
+
+  FTLinstall "${binary}" || return 1
+
+}
+
 main() {
 
   ######## FIRST CHECK ########
   # Must be root to install
+  show_ascii_berry
   echo ":::"
   if [[ ${EUID} -eq 0 ]]; then
     echo "::: You are root."
@@ -1210,7 +1339,7 @@ main() {
   fi
 
   # Update package cache
-  update_package_cache
+  update_package_cache || exit 1
 
   # Notify user of package availability
   notify_package_updates_available
@@ -1284,7 +1413,8 @@ main() {
     pw=""
     if [[ $(grep 'WEBPASSWORD' -c /etc/pihole/setupVars.conf) == 0 ]] ; then
         pw=$(tr -dc _A-Z-a-z-0-9 < /dev/urandom | head -c 8)
-        /usr/local/bin/pihole -a -p "${pw}"
+        . /opt/pihole/webpage.sh
+        echo "WEBPASSWORD=$(HashPassword ${pw})" >> ${setupVars}
     fi
   fi
 
@@ -1300,6 +1430,9 @@ main() {
 
   runGravity
 
+  start_service pihole-FTL
+  enable_service pihole-FTL
+
   echo "::: done."
 
   if [[ "${useUpdateVars}" == false ]]; then
@@ -1328,7 +1461,7 @@ main() {
       echo ":::                                ${pw}"
       echo ":::"
       echo "::: You can always change it using"
-      echo ":::                                pihole -a -p new_password"
+      echo ":::                                pihole -a -p"
     fi
   fi
 

gravity.sh

@@ -26,10 +26,12 @@ EOM
 	exit 0
 }
 
+PIHOLE_COMMAND="/usr/local/bin/pihole"
 
 adListFile=/etc/pihole/adlists.list
-adListDefault=/etc/pihole/adlists.default
+adListDefault=/etc/pihole/adlists.default #being deprecated
-whitelistScript="pihole -w"
+adListRepoDefault=/etc/.pihole/adlists.default
+whitelistScript="${PIHOLE_COMMAND} -w"
 whitelistFile=/etc/pihole/whitelist.txt
 blacklistFile=/etc/pihole/blacklist.txt
 readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
@@ -70,36 +72,34 @@ fi
 ###########################
 # collapse - begin formation of pihole
 gravity_collapse() {
+
+  #New Logic:
+  # Does /etc/pihole/adlists.list exist? If so leave it alone
+  #                                      If not, cp /etc/.pihole/adlists.default /etc/pihole/adlists.list
+  # Read from adlists.list
+
+  #The following two blocks will sort out any missing adlists in the /etc/pihole directory, and remove legacy adlists.default
+  if [ -f ${adListDefault} ] && [ -f ${adListFile} ]; then
+    rm ${adListDefault}
+  fi
+
+  if [ ! -f ${adListFile} ]; then
+    cp ${adListRepoDefault} ${adListFile}
+  fi
+
 	echo "::: Neutrino emissions detected..."
 	echo ":::"
-	#Decide if we're using a custom ad block list, or defaults.
+  echo -n "::: Pulling source lists into range..."
-	if [ -f ${adListFile} ]; then
+  sources=()
-		#custom file found, use this instead of default
+  while IFS= read -r line || [[ -n "$line" ]]; do
-		echo -n "::: Custom adList file detected. Reading..."
+    #Do not read commented out or blank lines
-		sources=()
+    if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
-		while IFS= read -r line || [[ -n "$line" ]]; do
+      echo "" > /dev/null
-			#Do not read commented out or blank lines
+    else
-			if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
+      sources+=(${line})
-				echo "" > /dev/null
+    fi
-			else
+  done < ${adListFile}
-				sources+=(${line})
+  echo " done!"
-			fi
-		done < ${adListFile}
-		echo " done!"
-	else
-		#no custom file found, use defaults!
-		echo -n "::: No custom adlist file detected, reading from default file..."
-		sources=()
-		while IFS= read -r line || [[ -n "$line" ]]; do
-			#Do not read commented out or blank lines
-			if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
-				echo "" > /dev/null
-			else
-				sources+=(${line})
-			fi
-		done < ${adListDefault}
-		echo " done!"
-	fi
 }
 
 # patternCheck - check to see if curl downloaded any new files.
@@ -168,6 +168,10 @@ gravity_transport() {
 	# Process result
 	gravity_patternCheck "${patternBuffer}" ${success} "${err}"
 
+        # Delete temp file if it hasn't been moved
+        if [[ -f "${patternBuffer}" ]]; then
+                rm "${patternBuffer}"
+        fi
 }
 
 # spinup - main gravity function
@@ -183,22 +187,26 @@ gravity_spinup() {
 		saveLocation=${piholeDir}/list.${i}.${domain}.${justDomainsExtension}
 		activeDomains[$i]=${saveLocation}
 
-		agent="Mozilla/10.0"
+		agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36"
 
 		# Use a case statement to download lists that need special cURL commands
 		# to complete properly and reset the user agent when required
 		case "${domain}" in
-			"adblock.mahakala.is")
+		    "adblock.mahakala.is")
 			agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36'
 			cmd_ext="-e http://forum.xda-developers.com/"
 		    ;;
 
+		    "adaway.org")
+			agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36'
+		    ;;
+
 		    "pgl.yoyo.org")
 			cmd_ext="-d mimetype=plaintext -d hostformat=hosts"
 		    ;;
 
-            # Default is a simple request
+		    # Default is a simple request
-            *) cmd_ext=""
+		    *) cmd_ext=""
         esac
         if [[ "${skipDownload}" == false ]]; then
             echo -n "::: Getting $domain list..."
@@ -385,7 +393,7 @@ gravity_reload() {
 	#Now replace the line in dnsmasq file
 #	sed -i "s/^addn-hosts.*/addn-hosts=$adList/" /etc/dnsmasq.d/01-pihole.conf
 
-	pihole restartdns
+	"${PIHOLE_COMMAND}" restartdns
 	echo " done!"
 }
 
@@ -403,8 +411,6 @@ if [[ "${forceGrav}" == true ]]; then
 	echo " done!"
 fi
 
-#Overwrite adlists.default from /etc/.pihole in case any changes have been made. Changes should be saved in /etc/adlists.list
-cp /etc/.pihole/adlists.default /etc/pihole/adlists.default
 gravity_collapse
 gravity_spinup
 if [[ "${skipDownload}" == false ]]; then
@@ -423,4 +429,4 @@ gravity_hostFormat
 gravity_blackbody
 
 gravity_reload
-pihole status
+"${PIHOLE_COMMAND}" status

pihole

@@ -8,9 +8,8 @@
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.
 
+readonly PI_HOLE_SCRIPT_DIR="/opt/pihole"
 
-
-PI_HOLE_SCRIPT_DIR="/opt/pihole"
 readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
 # Must be root to use this tool
 if [[ ! $EUID -eq 0 ]];then
@@ -128,17 +127,19 @@ queryFunc() {
   done
 
   # Scan for possible wildcard matches
-  local wildcards=($(processWildcards "${domain}"))
+  if [ -e "${wildcardlist}" ]; then
-  for domain in ${wildcards[@]}; do
+    local wildcards=($(processWildcards "${domain}"))
-    result=$(scanList "\/${domain}\/" ${wildcardlist})
+    for domain in ${wildcards[@]}; do
-    # Remove empty lines before couting number of results
+      result=$(scanList "\/${domain}\/" ${wildcardlist})
-    count=$(sed '/^\s*$/d' <<< "$result" | wc -l)
+      # Remove empty lines before couting number of results
-    if [[ ${count} > 0 ]]; then
+      count=$(sed '/^\s*$/d' <<< "$result" | wc -l)
-      echo "::: Wildcard blocking ${domain} (${count} results)"
+      if [[ ${count} > 0 ]]; then
-      echo "${result}"
+        echo "::: Wildcard blocking ${domain} (${count} results)"
-      echo ""
+        echo "${result}"
-    fi
+        echo ""
-  done
+      fi
+    done
+  fi
   exit 0
 }
 
@@ -278,19 +279,25 @@ tailFunc() {
   exit 0
 }
 
+piholeCheckoutFunc() {
+  source "${PI_HOLE_SCRIPT_DIR}"/piholeCheckout.sh
+  shift
+  checkout "$@"
+}
+
 helpFunc() {
 	cat << EOM
-::: Control all PiHole specific functions!
+::: Control all Pi-hole specific functions
 :::
 ::: Usage: pihole [options]
-:::		Add -h after -w (whitelist), -b (blacklist), -c (chronometer), or -a (admin)  for more information on usage
+:::		Add -h after -w (whitelist), -b (blacklist), -c (chronometer), or -a (admin) for more information on usage
 :::
 ::: Options:
 :::  -w, whitelist            Whitelist domain(s)
 :::  -b, blacklist            Blacklist domain(s) (exact match)
 :::  -wild, wildcard          Blacklist whole domain(s) (wildcard)
 :::  -d, debug                Start a debugging session
-:::                             Automated debugging can be enabled with `-a`.
+:::                             Automated debugging can be enabled with '-a'.
 :::                             'pihole -d -a'
 :::  -f, flush                Flush the 'pihole.log' file
 :::  -t, tail                 Output the last lines of the 'pihole.log' file. Lines are appended as the file grows
@@ -299,18 +306,19 @@ helpFunc() {
 :::  -g, updateGravity        Update the list of ad-serving domains
 :::  -c, chronometer          Calculates stats and displays to an LCD
 :::  -h, help                 Show this help dialog
-:::  -v, version              Show installed versions of Pi-Hole and Web-Admin
+:::  -v, version              Show installed versions of Pi-hole and Web-Admin
 :::  -q, query                Query the adlists for a specific domain
 :::                             'pihole -q domain -exact' shows exact matches only
 :::  -l, logging              Enable or Disable logging (pass 'on' or 'off')
 :::  -a, admin                Admin webpage options
-:::  uninstall                Uninstall Pi-Hole from your system :(!
+:::  uninstall                Uninstall Pi-hole from your system! :(
-:::  status                   Is Pi-Hole Enabled or Disabled
+:::  status                   Display if Pi-hole is Enabled or Disabled
-:::  enable                   Enable Pi-Hole DNS Blocking
+:::  enable                   Enable Pi-hole DNS Blocking
-:::  disable                  Disable Pi-Hole DNS Blocking
+:::  disable                  Disable Pi-hole DNS Blocking
 :::                             Blocking can also be disabled only temporarily, e.g.,
 :::                             'pihole disable 5m' - will disable blocking for 5 minutes
 :::  restartdns               Restart dnsmasq
+:::  checkout                 Check out different branches
 EOM
   exit 0
 }
@@ -341,5 +349,6 @@ case "${1}" in
   "restartdns"                  ) restartDNS;;
   "-a" | "admin"                ) webpageFunc "$@";;
   "-t" | "tail"                 ) tailFunc;;
+  "checkout"                    ) piholeCheckoutFunc "$@";;
   *                             ) helpFunc;;
 esac

test/debian.Dockerfile

@@ -1,4 +1,4 @@
-FROM debian:jessie
+FROM buildpack-deps:jessie-scm
 
 ENV GITDIR /etc/.pihole
 ENV SCRIPTDIR /opt/pihole

test/test_automated_install.py

@@ -78,7 +78,7 @@ def test_configureFirewall_firewalld_running_no_errors(Pihole):
     assert expected_stdout in configureFirewall.stdout
     firewall_calls = Pihole.run('cat /var/log/firewall-cmd').stdout
     assert 'firewall-cmd --state' in firewall_calls
-    assert 'firewall-cmd --permanent --add-port=80/tcp --add-port=53/tcp --add-port=53/udp' in firewall_calls
+    assert 'firewall-cmd --permanent --add-service=http --add-service=dns' in firewall_calls
     assert 'firewall-cmd --reload' in firewall_calls
 
 def test_configureFirewall_firewalld_disabled_no_errors(Pihole):
@@ -297,6 +297,111 @@ def test_update_package_cache_failure_no_errors(Pihole):
     assert 'ERROR' in updateCache.stdout
     assert 'done!' not in updateCache.stdout
 
+def test_FTL_detect_aarch64_no_errors(Pihole):
+    ''' confirms only aarch64 package is downloaded for FTL engine '''
+    # mock uname to return aarch64 platform
+    mock_command('uname', {'-m':('aarch64', '0')}, Pihole)
+    # mock ldd to respond with aarch64 shared library
+    mock_command('ldd', {'/bin/ls':('/lib/ld-linux-aarch64.so.1', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected ARM-aarch64 architecture'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_armv6l_no_errors(Pihole):
+    ''' confirms only armv6l package is downloaded for FTL engine '''
+    # mock uname to return armv6l platform
+    mock_command('uname', {'-m':('armv6l', '0')}, Pihole)
+    # mock ldd to respond with aarch64 shared library
+    mock_command('ldd', {'/bin/ls':('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected ARM-hf architecture (armv6 or lower)'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_armv7l_no_errors(Pihole):
+    ''' confirms only armv7l package is downloaded for FTL engine '''
+    # mock uname to return armv7l platform
+    mock_command('uname', {'-m':('armv7l', '0')}, Pihole)
+    # mock ldd to respond with aarch64 shared library
+    mock_command('ldd', {'/bin/ls':('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected ARM-hf architecture (armv7+)'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_x86_64_no_errors(Pihole):
+    ''' confirms only x86_64 package is downloaded for FTL engine '''
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected x86_64 architecture'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_unknown_no_errors(Pihole):
+    ''' confirms only generic package is downloaded for FTL engine '''
+    # mock uname to return generic platform
+    mock_command('uname', {'-m':('mips', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Not able to detect architecture (unknown: mips)'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_download_aarch64_no_errors(Pihole):
+    ''' confirms only aarch64 package is downloaded for FTL engine '''
+    # mock uname to return generic platform
+    download_binary = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLinstall pihole-FTL-aarch64-linux-gnu
+    ''')
+    expected_stdout = 'done'
+    assert expected_stdout in download_binary.stdout
+    assert 'failed' not in download_binary.stdout
+
+def test_FTL_download_unknown_fails_no_errors(Pihole):
+    ''' confirms unknown binary is not downloaded for FTL engine '''
+    # mock uname to return generic platform
+    download_binary = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLinstall pihole-FTL-mips
+    ''')
+    expected_stdout = 'failed'
+    assert expected_stdout in download_binary.stdout
+    assert 'done' not in download_binary.stdout
+
+def test_FTL_binary_installed_and_responsive_no_errors(Pihole):
+    ''' confirms FTL binary is copied and functional in installed location '''
+    installed_binary = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    pihole-FTL version
+    ''')
+    expected_stdout = 'v'
+    assert expected_stdout in installed_binary.stdout
+
+# def test_FTL_support_files_installed(Pihole):
+#     ''' confirms FTL support files are installed '''
+#     support_files = Pihole.run('''
+#     source /opt/pihole/basic-install.sh
+#     FTLdetect
+#     stat -c '%a %n' /var/log/pihole-FTL.log
+#     stat -c '%a %n' /run/pihole-FTL.port
+#     stat -c '%a %n' /run/pihole-FTL.pid
+#     ls -lac /run
+#     ''')
+#     assert '644 /run/pihole-FTL.port' in support_files.stdout
+#     assert '644 /run/pihole-FTL.pid' in support_files.stdout
+#     assert '644 /var/log/pihole-FTL.log' in support_files.stdout
+
 # Helper functions
 def mock_command(script, args, container):
     ''' Allows for setup of commands we don't really want to have to run for real in unit tests '''