Merge pull request #1429 from pi-hole/development

Release v3.0.1
Merge branch 'master' into development
2017-05-04 15:12:26 +01:00 · 2017-05-04 12:12:44 +02:00 · 2017-05-04 03:11:54 -07:00 · 2017-05-04 11:43:48 +02:00 · 2017-05-03 12:07:38 -07:00 · 2017-05-02 22:37:38 +01:00
24 changed files with 1198 additions and 371 deletions
--- a/.editorconfig
+++ b/.editorconfig
@@ -0,0 +1,38 @@
+# EditorConfig is awesome: http://EditorConfig.org
+
+# top-most EditorConfig file
+root = true
+
+# Unix-style newlines with a newline ending every file
+[*]
+end_of_line = lf
+insert_final_newline = true
+indent_style = space
+indent_size = tab
+tab_width = 2
+charset = utf-8
+trim_trailing_whitespace = true
+
+# Matches multiple files with brace expansion notation
+# Set default charset
+[*.{js,py}]
+charset = utf-8
+
+# 4 space indentation
+[*.py]
+indent_style = space
+indent_size = 4
+
+# Tab indentation (no size specified)
+[Makefile]
+indent_style = tab
+
+# Indentation override for all JS under lib directory
+[scripts/**.js]
+indent_style = space
+indent_size = 2
+
+# Matches the exact files either package.json or .travis.yml
+[{package.json,.travis.yml}]
+indent_style = space
+indent_size = 2
--- a/.github/ISSUE_TEMPLATE.md
+++ b/.github/ISSUE_TEMPLATE.md
@@ -2,7 +2,7 @@

 - [] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md).
 - [] The issue I am reporting can be *replicated*
- [] The issue I'm reporting isn't a duplicate (see [FAQs](https://github.com/pi-hole/pi-hole/wiki/FAQs), [closed issues](https://github.com/pi-hole/pi-hole/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), and [open issues](https://github.com/pi-hole/pi-hole/issues)).
+- [] The issue I am reporting isn't a duplicate (see [FAQs](https://github.com/pi-hole/pi-hole/wiki/FAQs), [closed issues](https://github.com/pi-hole/pi-hole/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), and [open issues](https://github.com/pi-hole/pi-hole/issues)).

 **How familiar are you with the codebase?:**

--- a/.gitignore
+++ b/.gitignore
@@ -4,4 +4,3 @@
 __pycache__
 .cache
 .pullapprove.yml
-
--- a/.pullapprove.yml
+++ b/.pullapprove.yml
@@ -1,7 +1,7 @@
 version: 2

 always_pending:
-  title_regex: '(WIP|wip)'  
+  title_regex: '(WIP|wip)'
  labels:
    - wip
  explanation: 'This PR is a work in progress...'
@@ -22,17 +22,17 @@ groups:
      enabled: true
    conditions:
      branches:
-        - development        
+        - development
    required: 2
    teams:
      - approvers
-  
+
  master:
    approve_by_comment:
      enabled: true
    conditions:
      branches:
        - master
-    required: -1
+    required: 4
    teams:
      - admin
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -35,4 +35,5 @@ When requesting or submitting new features, first consider whether it might be u
 - Before Submitting your Pull Request, merge `development` with your new branch and fix any conflicts. (Make sure you don't break anything in development!)
 - Please use the [Google Style Guide for Shell](https://google.github.io/styleguide/shell.xml) for your code submission styles. 
 - Commit Unix line endings.
+- Please use the Pi-hole brand: **Pi-hole** (Take a special look at the capitalized 'P' and a low 'h' with a hyphen)
 - (Optional fun) keep to the theme of Star Trek/black holes/gravity.
--- a/README.md
+++ b/README.md
@@ -5,12 +5,12 @@
 </p>

 <p align="center">
-<a href=https://discourse.pi-hole.net><img src="https://assets.pi-hole.net/static/Vortex_text.png" width=210></a>
+<a href=https://discourse.pi-hole.net><img src="https://assets.pi-hole.net/static/Vortex_with_text_and_TM.png" width=210></a>
 </p>

 ## The multi-platform, network-wide ad blocker

-Block ads for **all** your devices _without_ the need to install client-side software.  The Pi-hole blocks ads at the DNS-level, so all your devices are protected.
+Block ads for **all** your devices _without_ the need to install client-side software.  The Pi-hole™ blocks ads at the DNS-level, so all your devices are protected.

 - Web Browsers
 - Cell Phones
@@ -53,9 +53,9 @@ wget -O basic-install.sh https://install.pi-hole.net
 bash basic-install.sh
 ```

-Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to use Pi-hole as their DNS server.
+Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to use Pi-hole™ as their DNS server.

-## What is Pi-hole and how do I install it?
+## What is Pi-hole™ and how do I install it?
 <p align="center">
 <a href=https://www.youtube.com/watch?v=vKWjx1AQYgs><img src="https://assets.pi-hole.net/static/video-explainer.png"></a>
 </p>
@@ -73,7 +73,7 @@ Once installed, [configure your router to have **DHCP clients use the Pi as thei

 ## Technical Details

-The Pi-hole is an **advertising-aware DNS/Web server**. If an ad domain is queried, a small Web page or GIF is delivered in place of the advertisement.
+The Pi-hole™ is an **advertising-aware DNS/Web server**. If an ad domain is queried, a small Web page or GIF is delivered in place of the advertisement.

 ### Gravity

@@ -83,7 +83,7 @@ The [gravity.sh](https://github.com/pi-hole/pi-hole/blob/master/gravity.sh) does

 #### Other Operating Systems

-The automated install is only for a clean install of a Debian family or Fedora based system, such as the Raspberry Pi. However, this script will work for most UNIX-like systems, some with some slight **modifications** that we can help you work through. If you can install `dnsmasq` and a Webserver, it should work OK. If there are other platforms you'd like supported, let us know.
+The automated install is only for a clean install of a Debian family or Fedora based system, such as the Raspberry Pi. However, this script will work for most UNIX-like systems, some with some slight **modifications** that we can help you work through. If you can install `dnsmasq` and a web server, it should work OK. If there are other platforms you'd like supported, let us know.

 ### Web Interface

@@ -102,7 +102,7 @@ Domains can be whitelisted and blacklisted using either the web interface or the

 ### Settings

-The settings page lets you control and configure your Pi-hole.  You can do things like:
+The settings page lets you control and configure your Pi-hole™.  You can do things like:

 - enable Pi-hole's built-in DHCP server
 - exclude domains from the graphs
@@ -113,7 +113,7 @@ The settings page lets you control and configure your Pi-hole.  You can do thing

 #### Built-in DHCP Server

-Pi-hole ships with a built-in DHCP server.  This allows you to let your network devices use Pi-hole as their DNS server if your router does not let you adjust the DHCP options.
+Pi-hole™ ships with a built-in DHCP server.  This allows you to let your network devices use Pi-hole™ as their DNS server if your router does not let you adjust the DHCP options.
 <p align="center">
 <a href=hhttps://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245><img src="https://assets.pi-hole.net/static/piholedhcpserver.png"></a>
 </p>
@@ -137,14 +137,14 @@ The same output can be achieved on the CLI by running `chronometer.sh -j`

 You can view [real-time stats](https://discourse.pi-hole.net/t/how-do-i-view-my-pi-holes-stats-over-ssh-or-on-an-lcd-using-chronometer/240) via `ssh` or on an [2.8" LCD screen](http://amzn.to/1P0q1Fj). This is accomplished via [`chronometer.sh`](https://github.com/pi-hole/pi-hole/blob/master/advanced/Scripts/chronometer.sh). ![Pi-hole LCD](http://i.imgur.com/nBEqycp.jpg)

-## Pi-hole Projects
+## Pi-hole™ Projects

 -   [An ad blocking Magic Mirror](https://zonksec.com/blog/magic-mirror-dns-filtering/#dnssoftware)
 -   [Pi-hole stats in your Mac's menu bar](https://getbitbar.com/plugins/Network/pi-hole.1m.py)
 -   [Get LED alerts for each blocked ad](http://thetimmy.silvernight.org/pages/endisbutton/)
 -   [Pi-hole on Ubuntu 14.04 on VirtualBox](http://hbalagtas.blogspot.com/2016/02/adblocking-with-pi-hole-and-ubuntu-1404.html)
 -   [Docker Pi-hole container (x86 and ARM)](https://hub.docker.com/r/diginc/pi-hole/)
-   [Splunk: Pi-hole Visualizser](https://splunkbase.splunk.com/app/3023/)
+-   [Splunk: Pi-hole Visualiser](https://splunkbase.splunk.com/app/3023/)
 -   [Pi-hole Chrome extension](https://chrome.google.com/webstore/detail/pi-hole-list-editor/hlnoeoejkllgkjbnnnhfolapllcnaglh) ([open source](https://github.com/packtloss/pihole-extension))
 -   [Go Bananas for CHiP-hole ad blocking](https://www.hackster.io/jacobsalmela/chip-hole-network-wide-ad-blocker-98e037)
 -   [Sky-Hole](http://dlaa.me/blog/post/skyhole)
@@ -154,7 +154,7 @@ You can view [real-time stats](https://discourse.pi-hole.net/t/how-do-i-view-my-
 -   [Minibian Pi-hole](http://munkjensen.net/wiki/index.php/See_my_Pi-Hole#Minibian_Pi-hole)
 -   [Windows Tray Stat Application](https://github.com/goldbattle/copernicus)
 -   [Let your blink1 device blink when Pi-hole filters ads](https://gist.github.com/elpatron68/ec0b4c582e5abf604885ac1e068d233f)
-   [Pi-Hole Prometheus exporter](https://github.com/nlamirault/pihole_exporter) : a [Prometheus](https://prometheus.io/) exporter for Pi-Hole
+-   [Pi-hole Prometheus exporter](https://github.com/nlamirault/pihole_exporter): a [Prometheus](https://prometheus.io/) exporter for Pi-hole
 -   [Pi-hole Droid - open source Android client](https://github.com/friimaind/pi-hole-droid)

 ## Coverage
--- a/adlists.default
+++ b/adlists.default
@@ -1,53 +1,23 @@
-## Pi-hole ad-list default sources. Updated 29/10/2016 #########################
-#                                                                              #
-#  To make changes to this file:                                               #
-#    1. run `cp /etc/pihole/adlists.default /etc/pihole/adlists.list`          #
-#    2. run `nano /etc/pihole/adlists.list`                                    #
-#    3. Uncomment or comment any of the below lists                            #
-#                                                                              #
-#  Know of any other lists? Feel free to let us know about them, or add them   #
-#  to this file!                                                               #
-################################################################################
-
 # The below list amalgamates several lists we used previously.
 # See `https://github.com/StevenBlack/hosts` for details
+##StevenBlack's list
 https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

-# Other lists we consider safe:
+##MalwareDomains
 https://mirror1.malwaredomains.com/files/justdomains
+
+##Cameleon
 http://sysctl.org/cameleon/hosts
+
+##Zeustracker
 https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
+
+##Disconnect.me Tracking
 https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
+
+##Disconnect.me Ads
 https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt

-# hosts-file.net list. Updated frequently, but has been known to block legitimate sites.
+##Hosts-file.net
 https://hosts-file.net/ad_servers.txt

-# Mahakala list. Has been known to block legitimate domains including the entire .com range.
-# Warning: Due to the sheer size of this list, the web admin console will be unresponsive.
-#https://adblock.mahakala.is/
-
-# ADZHOSTS list. Has been known to block legitimate domains
-#http://pilotfiber.dl.sourceforge.net/project/adzhosts/HOSTS.txt
-
-# Windows 10 telemetry list
-#https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/win10/spy.txt
-
-# Securemecca.com list - Also blocks "adult" sites (pornography/gambling etc)
-#http://securemecca.com/Downloads/hosts.txt
-
-# Quidsup's tracker list
-#https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
-
-# Block the BBC News website Breaking News banner
-#https://raw.githubusercontent.com/BreakingTheNews/BreakingTheNews.github.io/master/hosts
-
-# Untested Lists:
-#https://raw.githubusercontent.com/reek/anti-adblock-killer/master/anti-adblock-killer-filters.txt
-#https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt
-#http://malwaredomains.lehigh.edu/files/domains.txt
-# Following two lists should be used simultaneously: (readme https://github.com/notracking/hosts-blocklists/)
-#https://raw.github.com/notracking/hosts-blocklists/master/hostnames.txt
-#https://raw.github.com/notracking/hosts-blocklists/master/domains.txt
-# Combination of several host files on the internet (warning some facebook domains are also blocked but you can go to facebook.com). See https://github.com/mat1th/Dns-add-block for more information.
-#https://raw.githubusercontent.com/mat1th/Dns-add-block/master/hosts
--- a/advanced/01-pihole.conf
+++ b/advanced/01-pihole.conf
@@ -25,6 +25,8 @@ addn-hosts=/etc/pihole/local.list

 domain-needed

+localise-queries
+
 bogus-priv

 no-resolv
@@ -32,6 +34,8 @@ no-resolv
 server=@DNS1@
 server=@DNS2@

+interface=@INT@
+
 cache-size=10000

 log-queries
--- a/advanced/Scripts/chronometer.sh
+++ b/advanced/Scripts/chronometer.sh
@@ -8,34 +8,55 @@
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.

-
-
-
 #Functions##############################################################################################################
 piLog="/var/log/pihole.log"
 gravity="/etc/pihole/gravity.list"

 . /etc/pihole/setupVars.conf

-# Borrowed/modified from https://gist.github.com/cjus/1047794
-function GetJSONValue {
-    retVal=$(echo $1 | sed 's/\\\\\//\//g' | \
-                       sed 's/[{}]//g' | \
-                       awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | \
-                       sed 's/\"\:/\|/g' | \
-                       sed 's/[\,]/ /g' | \
-                       sed 's/\"//g' | \
-                       grep -w $2)
-    echo ${retVal##*|}
+function GetFTLData {
+    # Open connection to FTL
+    exec 3<>/dev/tcp/localhost/"$(cat /var/run/pihole-FTL.port)"
+
+    # Test if connection is open
+    if { >&3; } 2> /dev/null; then
+       # Send command to FTL
+       echo -e ">$1" >&3
+
+       # Read input
+       read -r -t 1 LINE <&3
+       until [ ! $? ] || [[ "$LINE" == *"EOM"* ]]; do
+           echo "$LINE" >&1
+           read -r -t 1 LINE <&3
+       done
+
+       # Close connection
+       exec 3>&-
+       exec 3<&-
+   fi
 }

 outputJSON() {
-	json=$(curl -s -X GET http://127.0.0.1/admin/api.php?summaryRaw)
-	echo ${json}
+	get_summary_data
+	echo "{\"domains_being_blocked\":${domains_being_blocked_raw},\"dns_queries_today\":${dns_queries_today_raw},\"ads_blocked_today\":${ads_blocked_today_raw},\"ads_percentage_today\":${ads_percentage_today_raw}}"
+}
+
+get_summary_data() {
+	local summary=$(GetFTLData "stats")
+	domains_being_blocked_raw=$(grep "domains_being_blocked" <<< "${summary}" | grep -Eo "[0-9]+$")
+	domains_being_blocked=$(printf "%'.f" ${domains_being_blocked_raw})
+	dns_queries_today_raw=$(grep "dns_queries_today" <<< "$summary" | grep -Eo "[0-9]+$")
+	dns_queries_today=$(printf "%'.f" ${dns_queries_today_raw})
+	ads_blocked_today_raw=$(grep "ads_blocked_today" <<< "$summary" | grep -Eo "[0-9]+$")
+	ads_blocked_today=$(printf "%'.f" ${ads_blocked_today_raw})
+	ads_percentage_today_raw=$(grep "ads_percentage_today" <<< "$summary" | grep -Eo "[0-9.]+$")
+	LC_NUMERIC=C ads_percentage_today=$(printf "%'.f" ${ads_percentage_today_raw})
 }

 normalChrono() {
 	for (( ; ; )); do
+		get_summary_data
+		domain=$(GetFTLData recentBlocked)
 		clear
 		# Displays a colorful Pi-hole logo
 		echo " [0;1;35;95m_[0;1;31;91m__[0m [0;1;33;93m_[0m     [0;1;34;94m_[0m        [0;1;36;96m_[0m"
@@ -49,20 +70,12 @@ normalChrono() {
 		#uptime -p	#Doesn't work on all versions of uptime
 		uptime | awk -F'( |,|:)+' '{if ($7=="min") m=$6; else {if ($7~/^day/) {d=$6;h=$8;m=$9} else {h=$6;m=$7}}} {print d+0,"days,",h+0,"hours,",m+0,"minutes."}'
 		echo "-------------------------------"
-		# Uncomment to continually read the log file and display the current domain being blocked
-		#tail -f /var/log/pihole.log | awk '/\/etc\/pihole\/gravity.list/ {if ($7 != "address" && $7 != "name" && $7 != "/etc/pihole/gravity.list") print $7; else;}'
+		echo "Recently blocked:"
+		echo "  $domain"

-		json=$(curl -s -X GET http://127.0.0.1/admin/api.php?summaryRaw)
-
-    domains=$(printf "%'.f" $(GetJSONValue ${json} "domains_being_blocked")) #add commas in
-    queries=$(printf "%'.f" $(GetJSONValue ${json} "dns_queries_today"))
-    blocked=$(printf "%'.f" $(GetJSONValue ${json} "ads_blocked_today"))
-    LC_NUMERIC=C percentage=$(printf "%0.2f\n" $(GetJSONValue ${json} "ads_percentage_today")) #2 decimal places
-
-		echo "Blocking:      ${domains}"
-		echo "Queries:       ${queries}"
-
-		echo "Pi-holed:      ${blocked} (${percentage}%)"
+		echo "Blocking:      ${domains_being_blocked}"
+		echo "Queries:       ${dns_queries_today}"
+		echo "Pi-holed:      ${ads_blocked_today} (${ads_percentage_today}%)"

 		sleep 5
 	done
--- a/advanced/Scripts/list.sh
+++ b/advanced/Scripts/list.sh
@@ -84,6 +84,9 @@ PoplistFile() {
 		if ${addmode}; then
 			AddDomain "${dom}" "${listMain}"
 			RemoveDomain "${dom}" "${listAlt}"
+			if [[ "${listMain}" == "${whitelist}" || "${listMain}" == "${blacklist}" ]]; then
+			  RemoveDomain "${dom}" "${wildcardlist}"
+			fi
 		else
 			RemoveDomain "${dom}" "${listMain}"
 		fi
--- a/advanced/Scripts/piholeCheckout.sh
+++ b/advanced/Scripts/piholeCheckout.sh
@@ -0,0 +1,202 @@
+#!/usr/bin/env bash
+# Pi-hole: A black hole for Internet advertisements
+# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
+# Network-wide ad blocking via your own hardware.
+#
+# Checkout other branches than master
+#
+# This file is copyright under the latest version of the EUPL.
+# Please see LICENSE file for your rights under this license.
+
+readonly PI_HOLE_FILES_DIR="/etc/.pihole"
+PH_TEST="true" source "${PI_HOLE_FILES_DIR}/automated install/basic-install.sh"
+
+# webInterfaceGitUrl set in basic-install.sh
+# webInterfaceDir set in basic-install.sh
+# piholeGitURL set in basic-install.sh
+# is_repo() sourced from basic-install.sh
+# setupVars set in basic-install.sh
+
+source "${setupVars}"
+
+update="false"
+
+fully_fetch_repo() {
+  # Add upstream branches to shallow clone
+  local directory="${1}"
+
+  cd "${directory}" || return 1
+  if is_repo "${directory}"; then
+    git remote set-branches origin '*' || return 1
+    git fetch --quiet || return 1
+  else
+    return 1
+  fi
+  return 0
+}
+
+get_available_branches(){
+  # Return available branches
+  local directory="${1}"
+
+  cd "${directory}" || return 1
+  # Get reachable remote branches
+  git remote show origin | grep 'tracked' | sed 's/tracked//;s/ //g'
+  return
+}
+
+
+fetch_checkout_pull_branch() {
+  # Check out specified branch
+  local directory="${1}"
+  local branch="${2}"
+
+  # Set the reference for the requested branch, fetch, check it put and pull it
+  cd "${directory}"
+  git remote set-branches origin "${branch}" || return 1
+  git fetch --quiet || return 1
+  checkout_pull_branch "${directory}" "${branch}" || return 1
+}
+
+checkout_pull_branch() {
+  # Check out specified branch
+  local directory="${1}"
+  local branch="${2}"
+  local oldbranch
+
+  cd "${directory}" || return 1
+
+  oldbranch="$(git symbolic-ref HEAD)"
+
+  git checkout "${branch}" || return 1
+
+  if [ "$(git diff "${oldbranch}" | grep -c "^")" -gt "0" ]; then
+    update="true"
+  fi
+
+  git pull || return 1
+  return 0
+}
+
+warning1() {
+  echo "::: Note that changing the branch is a severe change of your Pi-hole system."
+  echo "::: This is not supported unless one of the developers explicitly asks you to do this!"
+  read -r -p "::: Have you read and understood this? [y/N] " response
+  case ${response} in
+  [yY][eE][sS]|[yY])
+    echo "::: Continuing."
+    return 0
+    ;;
+  *)
+    echo "::: Aborting."
+    return 1
+    ;;
+  esac
+}
+
+checkout()
+{
+  local corebranches
+  local webbranches
+
+  # Avoid globbing
+  set -f
+
+  #This is unlikely
+  if ! is_repo "${PI_HOLE_FILES_DIR}" ; then
+    echo "::: Critical Error: Core Pi-hole repo is missing from system!"
+    echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
+    exit 1;
+  fi
+  if [[ ${INSTALL_WEB} == "true" ]]; then
+    if ! is_repo "${webInterfaceDir}" ; then
+      echo "::: Critical Error: Web Admin repo is missing from system!"
+      echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
+      exit 1;
+    fi
+  fi
+
+  if [[ -z "${1}" ]]; then
+    echo "::: No option detected. Please use 'pihole checkout <master|dev>'."
+    echo "::: Or enter the repository and branch you would like to check out:"
+    echo "::: 'pihole checkout <web|core> <branchname>'"
+    exit 1
+  fi
+
+  if ! warning1 ; then
+    exit 1
+  fi
+
+  if [[ "${1}" == "dev" ]] ; then
+    # Shortcut to check out development branches
+    echo "::: Shortcut \"dev\" detected - checking out development / devel branches ..."
+    echo "::: Pi-hole core"
+    fetch_checkout_pull_branch "${PI_HOLE_FILES_DIR}" "development" || { echo "Unable to pull Core developement branch"; exit 1; }
+    if [[ ${INSTALL_WEB} == "true" ]]; then
+      echo "::: Web interface"
+      fetch_checkout_pull_branch "${webInterfaceDir}" "devel" || { echo "Unable to pull Web development branch"; exit 1; }
+    fi
+    echo "::: done!"
+  elif [[ "${1}" == "master" ]] ; then
+    # Shortcut to check out master branches
+    echo "::: Shortcut \"master\" detected - checking out master branches ..."
+    echo "::: Pi-hole core"
+    fetch_checkout_pull_branch "${PI_HOLE_FILES_DIR}" "master" || { echo "Unable to pull Core master branch"; exit 1; }
+    if [[ ${INSTALL_WEB} == "true" ]]; then
+      echo "::: Web interface"
+      fetch_checkout_pull_branch "${webInterfaceDir}" "master" || { echo "Unable to pull web master branch"; exit 1; }
+    fi
+    echo "::: done!"
+  elif [[ "${1}" == "core" ]] ; then
+    echo -n "::: Fetching remote branches for Pi-hole core from ${piholeGitUrl} ... "
+    if ! fully_fetch_repo "${PI_HOLE_FILES_DIR}" ; then
+      echo "::: Fetching all branches for Pi-hole core repo failed!"
+      exit 1
+    fi
+    corebranches=($(get_available_branches "${PI_HOLE_FILES_DIR}"))
+    echo " done!"
+    echo "::: ${#corebranches[@]} branches available"
+    echo ":::"
+    # Have to user chosing the branch he wants
+    if ! (for e in "${corebranches[@]}"; do [[ "$e" == "${2}" ]] && exit 0; done); then
+      echo "::: Requested branch \"${2}\" is not available!"
+      echo "::: Available branches for core are:"
+      for e in "${corebranches[@]}"; do echo ":::   $e"; done
+      exit 1
+    fi
+    checkout_pull_branch "${PI_HOLE_FILES_DIR}" "${2}"
+  elif [[ "${1}" == "web" && "${INSTALL_WEB}" == "true" ]] ; then
+    echo -n "::: Fetching remote branches for the web interface from ${webInterfaceGitUrl} ... "
+    if ! fully_fetch_repo "${webInterfaceDir}" ; then
+      echo "::: Fetching all branches for Pi-hole web interface repo failed!"
+      exit 1
+    fi
+    webbranches=($(get_available_branches "${webInterfaceDir}"))
+    echo " done!"
+    echo "::: ${#webbranches[@]} branches available"
+    echo ":::"
+    # Have to user chosing the branch he wants
+    if ! (for e in "${webbranches[@]}"; do [[ "$e" == "${2}" ]] && exit 0; done); then
+      echo "::: Requested branch \"${2}\" is not available!"
+      echo "::: Available branches for web are:"
+      for e in "${webbranches[@]}"; do echo ":::   $e"; done
+      exit 1
+    fi
+    checkout_pull_branch "${webInterfaceDir}" "${2}"
+  else
+    echo "::: Requested option \"${1}\" is not available!"
+    exit 1
+  fi
+
+  # Force updating everything
+  if [[ ! "${1}" == "web" && "${update}" == "true" ]]; then
+    echo "::: Running installer to upgrade your installation"
+    if "${PI_HOLE_FILES_DIR}/automated install/basic-install.sh" --unattended; then
+     exit 0
+    else
+     echo "Unable to complete update, contact Pi-hole"
+     exit 1
+    fi
+  fi
+}
+
--- a/advanced/Scripts/piholeDebug.sh
+++ b/advanced/Scripts/piholeDebug.sh
@@ -24,7 +24,10 @@ WHITELISTFILE="/etc/pihole/whitelist.txt"
 BLACKLISTFILE="/etc/pihole/blacklist.txt"
 ADLISTFILE="/etc/pihole/adlists.list"
 PIHOLELOG="/var/log/pihole.log"
+PIHOLEGITDIR="/etc/.pihole/"
+ADMINGITDIR="/var/www/html/admin/"
 WHITELISTMATCHES="/tmp/whitelistmatches.list"
+readonly FTLLOG="/var/log/pihole-FTL.log"

 TIMEOUT=60
 # Header info and introduction
@@ -41,16 +44,11 @@ cat << EOM
 ::: Please read and note any issues, and follow any directions advised during this process.
 EOM

-# Ensure the file exists, create if not, clear if exists.
-truncate --size=0 "${DEBUG_LOG}"
-chmod 644 ${DEBUG_LOG}
-chown "$USER":pihole ${DEBUG_LOG}
-
 source ${VARSFILE}

 ### Private functions exist here ###
 log_write() {
-    echo "${1}" >> "${DEBUG_LOG}"
+    echo "${@}" >&3
 }

 log_echo() {
@@ -75,14 +73,14 @@ log_echo() {

 header_write() {
  log_echo ""
-  log_echo "${1}"
+  log_echo "---= ${1}"
  log_write ""
 }

 file_parse() {
    while read -r line; do
 		  if [ ! -z "${line}" ]; then
-			  [[ "${line}" =~ ^#.*$  || ! "${line}" ]] && continue
+			  [[ "${line}" =~ ^#.*$  || ! "${line}" || "${line}" == "WEBPASSWORD="* ]] && continue
 				log_write "${line}"
 			fi
 		done < "${1}"
@@ -111,22 +109,61 @@ version_check() {
  header_write "Detecting Installed Package Versions:"

  local error_found
+  local pi_hole_ver
+  local pi_hole_branch
+  local pi_hole_commit
+  local admin_ver
+  local admin_branch
+  local admin_commit
+  local light_ver
+  local php_ver
+  local status
  error_found=0

-	local pi_hole_ver="$(cd /etc/.pihole/ && git describe --tags --abbrev=0)" \
-	&& log_echo -r "Pi-hole: $pi_hole_ver" || (log_echo "Pi-hole git repository not detected." && error_found=1)
-	local admin_ver="$(cd /var/www/html/admin && git describe --tags --abbrev=0)" \
-	&& log_echo -r "WebUI: $admin_ver" || (log_echo "Pi-hole Admin Pages git repository not detected." && error_found=1)
-	local light_ver="$(lighttpd -v |& head -n1 | cut -d " " -f1)" \
-	&& log_echo -r "${light_ver}" || (log_echo "lighttpd not installed." && error_found=1)
-	local php_ver="$(php -v |& head -n1)" \
-	&& log_echo -r "${php_ver}" || (log_echo "PHP not installed." && error_found=1)
+  cd "${PIHOLEGITDIR}" &> /dev/null || \
+    { status="Pi-hole git directory not found."; error_found=1; }
+  if git status &> /dev/null; then
+    pi_hole_ver=$(git describe --tags --abbrev=0)
+    pi_hole_branch=$(git rev-parse --abbrev-ref HEAD)
+    pi_hole_commit=$(git describe --long --dirty --tags --always)
+    log_echo -r "Pi-hole: ${pi_hole_ver:-Untagged} (${pi_hole_branch:-Detached}:${pi_hole_commit})"
+  else
+    status=${status:-"Pi-hole repository damaged."}
+    error_found=1
+  fi
+  if [[ "${status}" ]]; then
+    log_echo "${status}"
+    unset status
+  fi

-	(local pi_hole_branch="$(cd /etc/.pihole/ && git rev-parse --abbrev-ref HEAD)" && log_echo -r "Pi-hole branch:  ${pi_hole_branch}") || log_echo "Unable to obtain Pi-hole branch"
-	(local pi_hole_rev="$(cd /etc/.pihole/ && git describe --long --dirty --tags)" && log_echo -r "Pi-hole rev:     ${pi_hole_rev}") || log_echo "Unable to obtain Pi-hole revision"
+  cd "${ADMINGITDIR}" || \
+    { status="Pi-hole Dashboard git directory not found."; error_found=1; }
+  if git status &> /dev/null; then
+    admin_ver=$(git describe --tags --abbrev=0)
+    admin_branch=$(git rev-parse --abbrev-ref HEAD)
+    admin_commit=$(git describe --long --dirty --tags --always)
+    log_echo -r "Pi-hole Dashboard: ${admin_ver:-Untagged} (${admin_branch:-Detached}:${admin_commit})"
+  else
+    status=${status:-"Pi-hole Dashboard repository damaged."}
+    error_found=1
+  fi
+  if [[ "${status}" ]]; then
+    log_echo "${status}"
+    unset status
+  fi

-	(local admin_branch="$(cd /var/www/html/admin && git rev-parse --abbrev-ref HEAD)" && log_echo -r "AdminLTE branch: ${admin_branch}") || log_echo "Unable to obtain AdminLTE branch"
-	(local admin_rev="$(cd /var/www/html/admin && git describe --long --dirty --tags)" && log_echo -r "AdminLTE rev:    ${admin_rev}") || log_echo "Unable to obtain AdminLTE revision"
+	if light_ver=$(lighttpd -v |& head -n1 | cut -d " " -f1); then
+	  log_echo -r "${light_ver}"
+	else
+	  log_echo "lighttpd not installed."
+	  error_found=1
+	fi
+	if php_ver=$(php -v |& head -n1); then
+	  log_echo -r "${php_ver}"
+	else
+	  log_echo "PHP not installed."
+	  error_found=1
+	fi

 	return "${error_found}"
 }
@@ -190,6 +227,7 @@ ipv6_check() {
 ip_check() {
  local protocol=${1}
  local gravity=${2}
+  header_write "Checking IPv${protocol} Stack"

  local ip_addr_list="$(ip -${protocol} addr show dev ${PIHOLE_INTERFACE} | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }')"
  if [[ -n ${ip_addr_list} ]]; then
@@ -264,62 +302,78 @@ daemon_check() {
 }

 testResolver() {
-	header_write "Resolver Functions Check"
+  local protocol="${1}"
+  header_write "Resolver Functions Check (IPv${protocol})"
+  local IP="${2}"
+  local g_addr
+  local l_addr
+  local url
+  local testurl
+  local localdig
+  local piholedig
+  local remotedig
+
+  if [[ ${protocol} == "6" ]]; then
+    g_addr="2001:4860:4860::8888"
+    l_addr="::1"
+    r_type="AAAA"
+  else
+    g_addr="8.8.8.8"
+    l_addr="127.0.0.1"
+    r_type="A"
+  fi

 	# Find a blocked url that has not been whitelisted.
-	TESTURL="doubleclick.com"
-	if [ -s "${WHITELISTMATCHES}" ]; then
-		while read -r line; do
-			CUTURL=${line#*" "}
-			if [ "${CUTURL}" != "Pi-Hole.IsWorking.OK" ]; then
-				while read -r line2; do
-					CUTURL2=${line2#*" "}
-					if [ "${CUTURL}" != "${CUTURL2}" ]; then
-						TESTURL="${CUTURL}"
-						break 2
-					fi
-				done < "${WHITELISTMATCHES}"
-			fi
-		done < "${GRAVITYFILE}"
-	fi
+	url=$(shuf -n 1 "${GRAVITYFILE}" | awk -F ' ' '{ print $2 }')

-	log_write "Resolution of ${TESTURL} from Pi-hole:"
-	LOCALDIG=$(dig "${TESTURL}" @127.0.0.1)
-	if [[ $? = 0 ]]; then
-		log_write "${LOCALDIG}"
+	testurl="${url:-doubleclick.com}"
+
+
+	log_write "Resolution of ${testurl} from Pi-hole (${l_addr}):"
+	if localdig=$(dig -"${protocol}" "${testurl}" @${l_addr} +short "${r_type}"); then
+		log_write "${localdig}"
 	else
-		log_write "Failed to resolve ${TESTURL} on Pi-hole"
+		log_write "Failed to resolve ${testurl} on Pi-hole (${l_addr})"
+	fi
+	log_write ""
+
+	log_write "Resolution of ${testurl} from Pi-hole (${IP}):"
+	if piholedig=$(dig -"${protocol}" "${testurl}" @"${IP}" +short "${r_type}"); then
+		log_write "${piholedig}"
+	else
+		log_write "Failed to resolve ${testurl} on Pi-hole (${IP})"
 	fi
 	log_write ""


-	log_write "Resolution of ${TESTURL} from 8.8.8.8:"
-	REMOTEDIG=$(dig "${TESTURL}" @8.8.8.8)
-	if [[ $? = 0 ]]; then
-		log_write "${REMOTEDIG}"
+	log_write "Resolution of ${testurl} from ${g_addr}:"
+	if remotedig=$(dig -"${protocol}" "${testurl}" @${g_addr} +short "${r_type}"); then
+		log_write "${remotedig:-NXDOMAIN}"
 	else
-		log_write "Failed to resolve ${TESTURL} on 8.8.8.8"
+		log_write "Failed to resolve ${testurl} on upstream server ${g_addr}"
 	fi
 	log_write ""
-
-	log_write "Pi-hole dnsmasq specific records lookups"
-	log_write "Cache Size:"
-	dig +short chaos txt cachesize.bind >> ${DEBUG_LOG}
-	log_write "Upstream Servers:"
-	dig +short chaos txt servers.bind >> ${DEBUG_LOG}
-	log_write ""
 }

+testChaos(){
+  # Check Pi-hole specific records
+
+	log_write "Pi-hole dnsmasq specific records lookups"
+	log_write "Cache Size:"
+	log_write $(dig +short chaos txt cachesize.bind)
+	log_write "Upstream Servers:"
+	log_write $(dig +short chaos txt servers.bind)
+	log_write ""
+
+}
 checkProcesses() {
 	header_write "Processes Check"

-	echo ":::     Logging status of lighttpd and dnsmasq..."
-	PROCESSES=( lighttpd dnsmasq )
+	echo ":::     Logging status of lighttpd, dnsmasq and pihole-FTL..."
+	PROCESSES=( lighttpd dnsmasq pihole-FTL )
 	for i in "${PROCESSES[@]}"; do
-		log_write ""
-		log_write "${i}"
-		log_write " processes status:"
-		systemctl -l status "${i}" >> "${DEBUG_LOG}"
+		log_write "Status for ${i} daemon:"
+		log_write $(systemctl is-active "${i}")
 	done
 	log_write ""
 }
@@ -347,49 +401,6 @@ countdown() {
    tuvix=$(( tuvix - 5 ))
  done
 }
-### END FUNCTIONS ###
-
-# Gather version of required packages / repositories
-version_check || echo "REQUIRED FILES MISSING"
-# Check for newer setupVars storage file
-source_file "/etc/pihole/setupVars.conf"
-# Gather information about the running distribution
-distro_check || echo "Distro Check soft fail"
-# Gather processor type
-processor_check || echo "Processor Check soft fail"
-
-ip_check 6 ${IPV6_ADDRESS}
-ip_check 4 ${IPV4_ADDRESS}
-
-daemon_check lighttpd http
-daemon_check dnsmasq domain
-checkProcesses
-testResolver
-debugLighttpd
-
-files_check "${DNSMASQFILE}"
-dir_check "${DNSMASQCONFDIR}"
-files_check "${WHITELISTFILE}"
-files_check "${BLACKLISTFILE}"
-files_check "${ADLISTFILE}"
-
-
-header_write "Analyzing gravity.list"
-
-	gravity_length=$(grep -c ^ "${GRAVITYFILE}") \
-	&& log_write "${GRAVITYFILE} is ${gravity_length} lines long." \
-	|| log_echo "Warning: No gravity.list file found!"
-
-header_write "Analyzing pihole.log"
-
-  pihole_length=$(grep -c ^ "${PIHOLELOG}") \
-  && log_write "${PIHOLELOG} is ${pihole_length} lines long." \
-  || log_echo "Warning: No pihole.log file found!"
-
-  pihole_size=$(du -h "${PIHOLELOG}" | awk '{ print $1 }') \
-  && log_write "${PIHOLELOG} is ${pihole_size}." \
-  || log_echo "Warning: No pihole.log file found!"
-

 # Continuously append the pihole.log file to the pihole_debug.log file
 dumpPiHoleLog() {
@@ -401,7 +412,7 @@ dumpPiHoleLog() {
 	if [ -e "${PIHOLELOG}" ]; then
 	# Dummy process to use for flagging down tail to terminate
 	  countdown &
-		tail -n0 -f --pid=$! "${PIHOLELOG}" >> ${DEBUG_LOG}
+		tail -n0 -f --pid=$! "${PIHOLELOG}" >&4
 	else
 		log_write "No pihole.log file found!"
 		printf ":::\tNo pihole.log file found!\n"
@@ -412,6 +423,16 @@ dumpPiHoleLog() {
 finalWork() {
  local tricorder
 	echo "::: Finshed debugging!"
+
+  # Ensure the file exists, create if not, clear if exists.
+  truncate --size=0 "${DEBUG_LOG}"
+  chmod 644 ${DEBUG_LOG}
+  chown "$USER":pihole ${DEBUG_LOG}
+  # copy working temp file to final log location
+  cat /proc/$$/fd/3 >> "${DEBUG_LOG}"
+  # Straight dump of tailing the logs, can sanitize later if needed.
+  cat /proc/$$/fd/4 >> "${DEBUG_LOG}"
+
 	echo "::: The debug log can be uploaded to tricorder.pi-hole.net for sharing with developers only."
 	if [[ "${AUTOMATED}" ]]; then
 	  echo "::: Debug script running in automated mode, uploading log to tricorder..."
@@ -439,6 +460,82 @@ finalWork() {
 		echo "::: A local copy of the Debug log can be found at : /var/log/pihole_debug.log"
 }

+### END FUNCTIONS ###
+# Create temporary file for log
+TEMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX)
+# Open handle 3 for templog
+exec 3>"$TEMPLOG"
+# Delete templog, but allow for addressing via file handle.
+rm "$TEMPLOG"
+
+# Create temporary file for logdump using file handle 4
+DUMPLOG=$(mktemp /tmp/pihole_temp.XXXXXX)
+exec 4>"$DUMPLOG"
+rm "$DUMPLOG"
+
+# Gather version of required packages / repositories
+version_check || echo "REQUIRED FILES MISSING"
+# Check for newer setupVars storage file
+source_file "/etc/pihole/setupVars.conf"
+# Gather information about the running distribution
+distro_check || echo "Distro Check soft fail"
+# Gather processor type
+processor_check || echo "Processor Check soft fail"
+
+ip_check 6 ${IPV6_ADDRESS}
+ip_check 4 ${IPV4_ADDRESS}
+
+daemon_check lighttpd http
+daemon_check dnsmasq domain
+daemon_check pihole-FTL 4711
+checkProcesses
+
+# Check local/IP/Google for IPv4 Resolution
+testResolver 4 "${IPV4_ADDRESS%/*}"
+# If IPv6 enabled, check resolution
+if [[ "${IPV6_ADDRESS}" ]]; then
+  testResolver 6 "${IPV6_ADDRESS%/*}"
+fi
+# Poll dnsmasq Pi-hole specific queries
+testChaos
+
+debugLighttpd
+
+files_check "${DNSMASQFILE}"
+dir_check "${DNSMASQCONFDIR}"
+files_check "${WHITELISTFILE}"
+files_check "${BLACKLISTFILE}"
+files_check "${ADLISTFILE}"
+
+
+header_write "Analyzing gravity.list"
+
+	gravity_length=$(grep -c ^ "${GRAVITYFILE}") \
+	&& log_write "${GRAVITYFILE} is ${gravity_length} lines long." \
+	|| log_echo "Warning: No gravity.list file found!"
+
+header_write "Analyzing pihole.log"
+
+  pihole_length=$(grep -c ^ "${PIHOLELOG}") \
+  && log_write "${PIHOLELOG} is ${pihole_length} lines long." \
+  || log_echo "Warning: No pihole.log file found!"
+
+  pihole_size=$(du -h "${PIHOLELOG}" | awk '{ print $1 }') \
+  && log_write "${PIHOLELOG} is ${pihole_size}." \
+  || log_echo "Warning: No pihole.log file found!"
+
+header_write "Analyzing pihole-FTL.log"
+
+  FTL_length=$(grep -c ^ "${FTLLOG}") \
+  && log_write "${FTLLOG} is ${FTL_length} lines long." \
+  || log_echo "Warning: No pihole-FTL.log file found!"
+
+  FTL_size=$(du -h "${FTLLOG}" | awk '{ print $1 }') \
+  && log_write "${FTLLOG} is ${FTL_size}." \
+  || log_echo "Warning: No pihole-FTL.log file found!"
+
+tail -n50 "${FTLLOG}" >&3
+
 trap finalWork EXIT

 ### Method calls for additional logging ###
--- a/advanced/Scripts/piholeLogFlush.sh
+++ b/advanced/Scripts/piholeLogFlush.sh
@@ -3,18 +3,22 @@
 # (c) 2017 Pi-hole, LLC (https://pi-hole.net)
 # Network-wide ad blocking via your own hardware.
 #
-# Flushes /var/log/pihole.log
+# Flushes Pi-hole's log file
 #
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.

-
-
 echo -n "::: Flushing /var/log/pihole.log ..."
 # Test if logrotate is available on this system
 if command -v /usr/sbin/logrotate &> /dev/null; then
+  # Flush twice to move all data out of sight of FTL
+  /usr/sbin/logrotate --force /etc/pihole/logrotate
  /usr/sbin/logrotate --force /etc/pihole/logrotate
 else
+  # Flush both pihole.log and pihole.log.1 (if existing)
  echo " " > /var/log/pihole.log
+  if [ -f /var/log/pihole.log.1 ]; then
+    echo " " > /var/log/pihole.log.1
+  fi
 fi
 echo "... done!"
--- a/advanced/Scripts/update.sh
+++ b/advanced/Scripts/update.sh
@@ -75,13 +75,26 @@ GitCheckUpdateAvail() {
  fi
 }

+FTLcheckUpdate() {
+
+	local FTLversion=$(/usr/bin/pihole-FTL tag)
+	local FTLlatesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep 'Location' | awk -F '/' '{print $NF}' | tr -d '\r\n')
+
+	if [[ "${FTLversion}" != "${FTLlatesttag}" ]]; then
+		return 0
+	else
+		return 1
+	fi
+}
+
 main() {
  local pihole_version_current
  local web_version_current
+  source "${setupVars}"

  #This is unlikely
  if ! is_repo "${PI_HOLE_FILES_DIR}" ; then
-    echo "::: Critical Error: Core Pi-Hole repo is missing from system!"
+    echo "::: Critical Error: Core Pi-hole repo is missing from system!"
    echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
    exit 1;
  fi
@@ -96,6 +109,21 @@ main() {
    echo "::: Pi-hole Core:   up to date"
  fi

+  if FTLcheckUpdate ; then
+    FTL_update=true
+    echo "::: FTL:            update available"
+  else
+    FTL_update=false
+    echo "::: FTL:            up to date"
+  fi
+
+  if ${FTL_update}; then
+    echo ":::"
+    echo "::: FTL out of date"
+    FTLdetect
+    echo ":::"
+  fi
+
  if [[ ${INSTALL_WEB} == true ]]; then
    if ! is_repo "${ADMIN_INTERFACE_DIR}" ; then
      echo "::: Critical Error: Web Admin repo is missing from system!"
@@ -122,9 +150,11 @@ main() {
    #            pull pihole repo run install --unattended

    if ! ${core_update} && ! ${web_update} ; then
-      echo ":::"
-      echo "::: Everything is up to date!"
-      exit 0
+      if ! ${FTL_update} ; then
+        echo ":::"
+        echo "::: Everything is up to date!"
+        exit 0
+      fi

    elif ! ${core_update} && ${web_update} ; then
      echo ":::"
@@ -139,7 +169,7 @@ main() {

    elif ${core_update} && ${web_update} ; then
      echo ":::"
-      echo "::: Updating Everything"
+      echo "::: Updating Pi-hole core and web admin files"
      getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
      ${PI_HOLE_FILES_DIR}/automated\ install/basic-install.sh --unattended || echo "Unable to complete update, contact Pi-hole" && exit 1
    else
@@ -148,9 +178,11 @@ main() {
    fi
  else # Web Admin not installed, so only verify if core is up to date
    if ! ${core_update}; then
-      echo ":::"
-      echo "::: Everything is up to date!"
-      exit 0
+      if ! ${FTL_update} ; then
+        echo ":::"
+        echo "::: Everything is up to date!"
+        exit 0
+      fi
    else
      echo ":::"
      echo "::: Pi-hole core files out of date"
@@ -173,6 +205,15 @@ main() {
    echo "::: If you had made any changes in '/etc/.pihole/', they have been stashed using 'git stash'"
  fi

+  if [[ ${FTL_update} == true ]]; then
+    FTL_version_current="$(/usr/bin/pihole-FTL tag)"
+    echo ":::"
+    echo "::: FTL version is now at ${FTL_version_current}"
+    start_service pihole-FTL
+    enable_service pihole-FTL
+  fi
+
+
  echo ""
  exit 0

--- a/advanced/Scripts/version.sh
+++ b/advanced/Scripts/version.sh
@@ -8,67 +8,104 @@
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.

-
-
-# Flags:
-latest=false
-current=false
-
+# Variables
 DEFAULT="-1"
+PHGITDIR="/etc/.pihole/"
+WEBGITDIR="/var/www/html/admin/"
+
+getLocalVersion() {
+  # Get the tagged version of the local repository
+  local directory="${1}"
+  local version
+
+  cd "${directory}" || { echo "${DEFAULT}"; return 1; }
+  version=$(git describe --tags --always || \
+            echo "${DEFAULT}")
+  if [[ "${version}" =~ ^v ]]; then
+    echo "${version}"
+  elif [[ "${version}" == "${DEFAULT}" ]]; then
+    echo "ERROR"
+    return 1
+  else
+    echo "Untagged"
+  fi
+  return 0
+}
+
+getLocalHash() {
+  # Get the short hash of the local repository
+  local directory="${1}"
+  local hash
+
+  cd "${directory}" || { echo "${DEFAULT}"; return 1; }
+  hash=$(git rev-parse --short HEAD || \
+         echo "${DEFAULT}")
+  if [[ "${hash}" == "${DEFAULT}" ]]; then
+    echo "ERROR"
+    return 1
+  else
+    echo "${hash}"
+  fi
+  return 0
+}
+
+getRemoteVersion(){
+  # Get the version from the remote origin
+  local daemon="${1}"
+  local version
+
+  version=$(curl --silent --fail https://api.github.com/repos/pi-hole/${daemon}/releases/latest | \
+            awk -F: '$1 ~/tag_name/ { print $2 }' | \
+            tr -cd '[[:alnum:]]._-')
+  if [[ "${version}" =~ ^v ]]; then
+    echo "${version}"
+  else
+    echo "ERROR"
+    return 1
+  fi
+  return 0
+}
+
+#PHHASHLATEST=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/commits/master | \
+#                   grep sha | \
+#                   head -n1 | \
+#                   awk -F ' ' '{ print $2 }' | \
+#                   tr -cd '[[:alnum:]]._-')
+
+#WEBHASHLATEST=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/commits/master | \
+#                   grep sha | \
+#                   head -n1 | \
+#                   awk -F ' ' '{ print $2 }' | \
+#                   tr -cd '[[:alnum:]]._-')
+

 normalOutput() {
-	piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
-	webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
-
-	piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-	webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-
-	echo "::: Pi-hole version is ${piholeVersion} (Latest version is ${piholeVersionLatest:-${DEFAULT}})"
-	echo "::: Web-Admin version is ${webVersion} (Latest version is ${webVersionLatest:-${DEFAULT}})"
+	echo "::: Pi-hole version is $(getLocalVersion "${PHGITDIR}") (Latest version is $(getRemoteVersion pi-hole))"
+	if [ -d "${WEBGITDIR}" ]; then
+		echo "::: Web-Admin version is $(getLocalVersion "${WEBGITDIR}") (Latest version is $(getRemoteVersion AdminLTE))"
+	fi
 }

 webOutput() {
-	for var in "$@"; do
-		case "${var}" in
-			"-l" | "--latest"    ) latest=true;;
-			"-c" | "--current"   ) current=true;;
-			*                    ) echo "::: Invalid Option!"; exit 1;
-		esac
-	done
-
-	if [[ "${latest}" == true && "${current}" == false ]]; then
-		webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "${webVersionLatest:--1}"
-	elif [[ "${latest}" == false && "${current}" == true ]]; then
-		webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
-		echo "${webVersion}"
-	else
-		webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
-		webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "::: Web-Admin version is ${webVersion} (Latest version is ${webVersionLatest:-${DEFAULT}})"
-	fi
+  if [ -d "${WEBGITDIR}" ]; then
+    case "${1}" in
+      "-l" | "--latest"    ) echo $(getRemoteVersion AdminLTE);;
+      "-c" | "--current"   ) echo $(getLocalVersion "${WEBGITDIR}");;
+      "-h" | "--hash"      ) echo $(getLocalHash "${WEBGITDIR}");;
+      *                    ) echo "::: Invalid Option!"; exit 1;
+    esac
+  else
+    echo "::: Web interface not installed!"; exit 1;
+  fi
 }

 coreOutput() {
-	for var in "$@"; do
-		case "${var}" in
-			"-l" | "--latest"    ) latest=true;;
-			"-c" | "--current"   ) current=true;;
-			*                    ) echo "::: Invalid Option!"; exit 1;
-		esac
-	done
-
-	if [[ "${latest}" == true && "${current}" == false ]]; then
-		piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "${piholeVersionLatest:--1}"
-	elif [[ "${latest}" == false && "${current}" == true ]]; then
-		piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
-		echo "${piholeVersion}"
-	else
-		piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
-		piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "::: Pi-hole version is ${piholeVersion} (Latest version is ${piholeVersionLatest:-${DEFAULT}})"
-	fi
+  case "${1}" in
+    "-l" | "--latest"    ) echo $(getRemoteVersion pi-hole);;
+    "-c" | "--current"   ) echo $(getLocalVersion "${PHGITDIR}");;
+    "-h" | "--hash"      ) echo $(getLocalHash "${PHGITDIR}");;
+    *                    ) echo "::: Invalid Option!"; exit 1;
+  esac
 }

 helpFunc() {
@@ -93,10 +130,8 @@ if [[ $# = 0 ]]; then
 	normalOutput
 fi

-for var in "$@"; do
-	case "${var}" in
-	"-a" | "--admin"     ) shift; webOutput "$@";;
-	"-p" | "--pihole"    ) shift; coreOutput "$@" ;;
-	"-h" | "--help"      ) helpFunc;;
-	esac
-done
+case "${1}" in
+  "-a" | "--admin"     ) shift; webOutput "$@";;
+  "-p" | "--pihole"    ) shift; coreOutput "$@" ;;
+  "-h" | "--help"      ) helpFunc;;
+esac
--- a/advanced/Scripts/webpage.sh
+++ b/advanced/Scripts/webpage.sh
@@ -67,6 +67,13 @@ SetTemperatureUnit(){

 }

+HashPassword(){
+    # Compute password hash twice to avoid rainbow table vulnerability
+    return=$(echo -n ${1} | sha256sum | sed 's/\s.*$//')
+		return=$(echo -n ${return} | sha256sum | sed 's/\s.*$//')
+		echo ${return}
+}
+
 SetWebPassword(){

 	if [ "${SUDO_USER}" == "www-data" ]; then
@@ -81,19 +88,32 @@ SetWebPassword(){
 		exit 1
 	fi

-	# Set password only if there is one to be set
-	if (( ${#args[2]} > 0 )) ; then
-		# Compute password hash twice to avoid rainbow table vulnerability
-		hash=$(echo -n ${args[2]} | sha256sum | sed 's/\s.*$//')
-		hash=$(echo -n ${hash} | sha256sum | sed 's/\s.*$//')
+  if (( ${#args[2]} > 0 )) ; then
+    readonly PASSWORD="${args[2]}"
+    readonly CONFIRM="${PASSWORD}"
+  else
+    read -s -p "Enter New Password (Blank for no password): " PASSWORD
+    echo ""
+
+    if [ "${PASSWORD}" == "" ]; then
+      change_setting "WEBPASSWORD" ""
+      echo "Password Removed"
+      exit 0
+    fi
+
+    read -s -p "Confirm Password: " CONFIRM
+    echo ""
+  fi
+
+	if [ "${PASSWORD}" == "${CONFIRM}" ] ; then
+		hash=$(HashPassword ${PASSWORD})
 		# Save hash to file
 		change_setting "WEBPASSWORD" "${hash}"
 		echo "New password set"
 	else
-		change_setting "WEBPASSWORD" ""
-		echo "Password removed"
+		echo "Passwords don't match. Your password has not been changed"
+		exit 1
 	fi
-
 }

 ProcessDNSSettings() {
@@ -134,7 +154,7 @@ trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE3

 	delete_dnsmasq_setting "host-record"

-	if [ ! -z "${HOSTRECORD+x}" ]; then
+	if [ ! -z "${HOSTRECORD}" ]; then
 		add_dnsmasq_setting "host-record" "${HOSTRECORD}"
 	fi

@@ -144,15 +164,13 @@ trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE3

 	if [[ "${DNSMASQ_LISTENING}" == "all" ]]; then
 		# Listen on all interfaces, permit all origins
-		# Leave a comment in 01-pihole.conf
-		add_dnsmasq_setting "# Listening on all interfaces"
 		add_dnsmasq_setting "except-interface" "nonexisting"
-	elif [[ "${DNSMASQ_LISTENING}" == "single" ]]; then
-		# Listen only on one interface
-		add_dnsmasq_setting "interface" "${PIHOLE_INTERFACE}"
-	else
+	elif [[ "${DNSMASQ_LISTENING}" == "local" ]]; then
 		# Listen only on all interfaces, but only local subnets
 		add_dnsmasq_setting "local-service"
+	else
+		# Listen only on one interface
+		add_dnsmasq_setting "interface" "${PIHOLE_INTERFACE}"
 	fi

 }
@@ -321,6 +339,25 @@ SetWebUILayout(){

 }

+CustomizeAdLists() {
+
+  list="/etc/pihole/adlists.list"
+
+	if [[ "${args[2]}" == "enable" ]] ; then
+		sed -i "\\@${args[3]}@s/^#http/http/g" "${list}"
+	elif [[ "${args[2]}" == "disable" ]] ; then
+		sed -i "\\@${args[3]}@s/^http/#http/g" "${list}"
+	elif [[ "${args[2]}" == "add" ]] ; then
+		echo "${args[3]}" >> ${list}
+	elif [[ "${args[2]}" == "del" ]] ; then
+	  var=$(echo "${args[3]}" | sed 's/\//\\\//g')
+	  sed -i "/${var}/Id" "${list}"
+	else
+		echo "Not permitted"
+		return 1
+  fi
+}
+
 SetPrivacyMode(){

 	if [[ "${args[2]}" == "true" ]] ; then
@@ -394,12 +431,12 @@ SetListeningMode(){
 	if [[ "${args[2]}" == "all" ]] ; then
 		echo "Listening on all interfaces, permiting all origins, hope you have a firewall!"
 		change_setting "DNSMASQ_LISTENING" "all"
-	elif [[ "${args[2]}" == "single" ]] ; then
-		echo "Listening only on interface ${PIHOLE_INTERFACE}"
-		change_setting "DNSMASQ_LISTENING" "single"
-	else
+	elif [[ "${args[2]}" == "local" ]] ; then
 		echo "Listening on all interfaces, permitting only origins that are at most one hop away (local devices)"
 		change_setting "DNSMASQ_LISTENING" "local"
+	else
+		echo "Listening only on interface ${PIHOLE_INTERFACE}"
+		change_setting "DNSMASQ_LISTENING" "single"
 	fi

 	# Don't restart DNS server yet because other settings
@@ -412,6 +449,12 @@ SetListeningMode(){

 }

+Teleporter()
+{
+	local datetimestamp=$(date "+%Y-%m-%d_%H-%M-%S")
+	php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-teleporter_${datetimestamp}.zip"
+}
+
 main() {

 	args=("$@")
@@ -437,6 +480,8 @@ main() {
 		"removestaticdhcp"  ) RemoveDHCPStaticAddress;;
 		"hostrecord"        ) SetHostRecord;;
 		"-i" | "interface"  ) SetListeningMode;;
+		"-t" | "teleporter" ) Teleporter;;
+		"adlist"            ) CustomizeAdLists;;
 		*                   ) helpFunc;;
 	esac

--- a/advanced/bash-completion/pihole
+++ b/advanced/bash-completion/pihole
@@ -3,7 +3,7 @@ _pihole() {
 	COMPREPLY=()
 	cur="${COMP_WORDS[COMP_CWORD]}"
 	prev="${COMP_WORDS[COMP_CWORD-1]}"
-	opts="admin blacklist chronometer debug disable enable flush help logging query reconfigure restartdns setupLCD status tail uninstall updateGravity updatePihole version whitelist"
+	opts="admin blacklist chronometer debug disable enable flush help logging query reconfigure restartdns setupLCD status tail uninstall updateGravity updatePihole version whitelist checkout"

 	COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
 	return 0
--- a/advanced/index.php
+++ b/advanced/index.php
@@ -1,15 +1,55 @@
 <?php
-/* Detailed Pi-Hole Block Page: Show "Website Blocked" if user browses to site, but not to image/file requests based on the work of WaLLy3K for DietPi & Pi-Hole */
+/* Detailed Pi-hole Block Page: Show "Website Blocked" if user browses to site, but not to image/file requests based on the work of WaLLy3K for DietPi & Pi-Hole */
+
+function validIP($address){
+	if (preg_match('/[.:0]/', $address) && !preg_match('/[1-9a-f]/', $address)) {
+		// Test if address contains either `:` or `0` but not 1-9 or a-f
+		return false;
+	}
+	return !filter_var($address, FILTER_VALIDATE_IP) === false;
+}

 $uri = escapeshellcmd($_SERVER['REQUEST_URI']);
 $serverName = escapeshellcmd($_SERVER['SERVER_NAME']);

+// If the server name is 'pi.hole', it's likely a user trying to get to the admin panel.
+// Let's be nice and redirect them.
+if ($serverName === 'pi.hole')
+{
+    header('HTTP/1.1 301 Moved Permanently');
+    header("Location: /admin/");
+}
+
 // Retrieve server URI extension (EG: jpg, exe, php)
+ini_set('pcre.recursion_limit',100);
 $uriExt = pathinfo($uri, PATHINFO_EXTENSION);

 // Define which URL extensions get rendered as "Website Blocked"
 $webExt = array('asp', 'htm', 'html', 'php', 'rss', 'xml');

+// Get IPv4 and IPv6 addresses from setupVars.conf (if available)
+$setupVars = parse_ini_file("/etc/pihole/setupVars.conf");
+$ipv4 = isset($setupVars["IPV4_ADDRESS"]) ? explode("/", $setupVars["IPV4_ADDRESS"])[0] : $_SERVER['SERVER_ADDR'];
+$ipv6 = isset($setupVars["IPV6_ADDRESS"]) ? explode("/", $setupVars["IPV6_ADDRESS"])[0] : $_SERVER['SERVER_ADDR'];
+
+$AUTHORIZED_HOSTNAMES = array(
+	$ipv4,
+	$ipv6,
+	str_replace(array("[","]"), array("",""), $_SERVER["SERVER_ADDR"]),
+	"pi.hole",
+	"localhost");
+// Allow user set virtual hostnames
+$virtual_host = getenv('VIRTUAL_HOST');
+if (!empty($virtual_host))
+	array_push($AUTHORIZED_HOSTNAMES, $virtual_host);
+
+// Immediately quit since we didn't block this page (the IP address or pi.hole is explicitly requested)
+if(validIP($serverName) || in_array($serverName,$AUTHORIZED_HOSTNAMES))
+{
+	http_response_code(404);
+	die();
+}
+
 if(in_array($uriExt, $webExt) || empty($uriExt))
 {
 	// Requested resource has an extension listed in $webExt
@@ -37,7 +77,7 @@ if (!$showPage)
 	die();
 }

-// Get Pi-Hole version
+// Get Pi-hole version
 $piHoleVersion = exec('cd /etc/.pihole/ && git describe --tags --abbrev=0');

 // Don't show the URI if it is the root directory
@@ -48,6 +88,7 @@ if($uri == "/")

 ?>
 <!DOCTYPE html>
+<html>
 <head>
 	<meta charset='UTF-8'/>
 	<title>Website Blocked</title>
--- a/advanced/pihole-FTL.service
+++ b/advanced/pihole-FTL.service
@@ -0,0 +1,80 @@
+#!/bin/bash
+### BEGIN INIT INFO
+# Provides:          pihole-FTL
+# Required-Start:    $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: pihole-FTL daemon
+# Description:       Enable service provided by pihole-FTL daemon
+### END INIT INFO
+
+FTLUSER=pihole
+PIDFILE=/var/run/pihole-FTL.pid
+
+get_pid() {
+    pidof "pihole-FTL"
+}
+
+is_running() {
+    ps "$(get_pid)" > /dev/null 2>&1
+}
+
+# Start the service
+start() {
+  if is_running; then
+    echo "pihole-FTL is already running"
+  else
+    touch /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
+    chown pihole:pihole /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
+    chmod 0644 /var/log/pihole-FTL.log /run/pihole-FTL.pid /run/pihole-FTL.port
+    su -s /bin/sh -c "/usr/bin/pihole-FTL" "$FTLUSER"
+    echo
+  fi
+}
+
+# Stop the service
+stop() {
+  if is_running; then
+    kill "$(get_pid)"
+    for i in {1..5}; do
+      if ! is_running; then
+        break
+      fi
+
+      echo -n "."
+      sleep 1
+    done
+    echo
+
+    if is_running; then
+      echo "Not stopped; may still be shutting down or shutdown may have failed, killing now"
+      kill -9 "$(get_pid)"
+      exit 1
+    else
+      echo "Stopped"
+    fi
+  else
+    echo "Not running"
+  fi
+  echo
+}
+
+### main logic ###
+case "$1" in
+  stop)
+        stop
+        ;;
+  status)
+        status pihole-FTL
+        ;;
+  start|restart|reload|condrestart)
+        stop
+        start
+        ;;
+  *)
+        echo $"Usage: $0 {start|stop|restart|reload|status}"
+        exit 1
+esac
+
+exit 0
--- a/install/basic-install.sh
+++ b/install/basic-install.sh
@@ -54,13 +54,39 @@ skipSpaceCheck=false
 reconfigure=false
 runUnattended=false

+show_ascii_berry() {
+  echo "
+        .;;,.
+        .ccccc:,.
+         :cccclll:.      ..,,
+          :ccccclll.   ;ooodc
+           'ccll:;ll .oooodc
+             .;cll.;;looo:.
+                 .. ','.
+                .',,,,,,'.
+              .',,,,,,,,,,.
+            .',,,,,,,,,,,,....
+          ....''',,,,,,,'.......
+        .........  ....  .........
+        ..........      ..........
+        ..........      ..........
+        .........  ....  .........
+          ........,,,,,,,'......
+            ....',,,,,,,,,,,,.
+               .',,,,,,,,,'.
+                .',,,,,,'.
+                  ..'''.
+"
+}
+
+
 # Compatibility
 distro_check() {
 if command -v apt-get &> /dev/null; then
  #Debian Family
  #############################################
  PKG_MANAGER="apt-get"
-  UPDATE_PKG_CACHE="${PKG_MANAGER} update"
+  UPDATE_PKG_CACHE="test_dpkg_lock; ${PKG_MANAGER} update"
  PKG_INSTALL=(${PKG_MANAGER} --yes --no-install-recommends install)
  # grep -c will return 1 retVal on 0 matches, block this throwing the set -e with an OR TRUE
  PKG_COUNT="${PKG_MANAGER} -s -o Debug::NoLocking=true upgrade | grep -c ^Inst || true"
@@ -79,7 +105,7 @@ if command -v apt-get &> /dev/null; then
    phpVer="php5"
  fi
  # #########################################
-  INSTALLER_DEPS=(apt-utils debconf dhcpcd5 git ${iproute_pkg} whiptail)
+  INSTALLER_DEPS=(apt-utils dialog debconf dhcpcd5 git ${iproute_pkg} whiptail)
  PIHOLE_DEPS=(bc cron curl dnsmasq dnsutils iputils-ping lsof netcat sudo unzip wget)
  PIHOLE_WEB_DEPS=(lighttpd ${phpVer}-common ${phpVer}-cgi)
  LIGHTTPD_USER="www-data"
@@ -87,6 +113,17 @@ if command -v apt-get &> /dev/null; then
  LIGHTTPD_CFG="lighttpd.conf.debian"
  DNSMASQ_USER="dnsmasq"

+  test_dpkg_lock() {
+    i=0
+    while fuser /var/lib/dpkg/lock >/dev/null 2>&1 ; do
+      sleep 0.5
+      ((i=i+1))
+    done
+    # Always return success, since we only return if there is no
+    # lock (anymore)
+    return 0
+  }
+
 elif command -v rpm &> /dev/null; then
  # Fedora Family
  if command -v dnf &> /dev/null; then
@@ -99,7 +136,7 @@ elif command -v rpm &> /dev/null; then
  UPDATE_PKG_CACHE=":"
  PKG_INSTALL=(${PKG_MANAGER} install -y)
  PKG_COUNT="${PKG_MANAGER} check-update | egrep '(.i686|.x86|.noarch|.arm|.src)' | wc -l"
-  INSTALLER_DEPS=(git iproute net-tools newt procps-ng)
+  INSTALLER_DEPS=(dialog git iproute net-tools newt procps-ng)
  PIHOLE_DEPS=(bc bind-utils cronie curl dnsmasq findutils nmap-ncat sudo unzip wget)
  PIHOLE_WEB_DEPS=(lighttpd lighttpd-fastcgi php php-common php-cli)
  if ! grep -q 'Fedora' /etc/redhat-release; then
@@ -196,7 +233,7 @@ find_IPv4_information() {

 get_available_interfaces() {
  # Get available UP interfaces.
-  availableInterfaces=$(ip -o link | grep -v "state DOWN\|lo" | awk '{print $2}' | cut -d':' -f1 | cut -d'@' -f1)
+  availableInterfaces=$(ip --oneline link show up | grep -v "lo" | awk '{print $2}' | cut -d':' -f1 | cut -d'@' -f1)
 }

 welcomeDialogs() {
@@ -231,7 +268,7 @@ verifyFreeDiskSpace() {
  # - Insufficient free disk space
  elif [[ ${existing_free_kilobytes} -lt ${required_free_kilobytes} ]]; then
    echo "::: Insufficient Disk Space!"
-    echo "::: Your system appears to be low on disk space. pi-hole recommends a minimum of $required_free_kilobytes KiloBytes."
+    echo "::: Your system appears to be low on disk space. Pi-hole recommends a minimum of $required_free_kilobytes KiloBytes."
    echo "::: You only have ${existing_free_kilobytes} KiloBytes free."
    echo "::: If this is a new install you may need to expand your disk."
    echo "::: Try running 'sudo raspi-config', and choose the 'expand file system option'"
@@ -401,7 +438,7 @@ setStaticIPv4() {
      cp "${IFCFG_FILE}" "${IFCFG_FILE}".pihole.orig
      # Build Interface configuration file:
      {
-        echo "# Configured via Pi-Hole installer"
+        echo "# Configured via Pi-hole installer"
        echo "DEVICE=$PIHOLE_INTERFACE"
        echo "BOOTPROTO=none"
        echo "ONBOOT=yes"
@@ -582,14 +619,14 @@ version_check_dnsmasq() {
  local dnsmasq_conf="/etc/dnsmasq.conf"
  local dnsmasq_conf_orig="/etc/dnsmasq.conf.orig"
  local dnsmasq_pihole_id_string="addn-hosts=/etc/pihole/gravity.list"
-  local dnsmasq_original_config="/etc/.pihole/advanced/dnsmasq.conf.original"
-  local dnsmasq_pihole_01_snippet="/etc/.pihole/advanced/01-pihole.conf"
+  local dnsmasq_original_config="${PI_HOLE_LOCAL_REPO}/advanced/dnsmasq.conf.original"
+  local dnsmasq_pihole_01_snippet="${PI_HOLE_LOCAL_REPO}/advanced/01-pihole.conf"
  local dnsmasq_pihole_01_location="/etc/dnsmasq.d/01-pihole.conf"

  if [ -f ${dnsmasq_conf} ]; then
    echo -n ":::    Existing dnsmasq.conf found..."
    if grep -q ${dnsmasq_pihole_id_string} ${dnsmasq_conf}; then
-      echo " it is from a previous pi-hole install."
+      echo " it is from a previous Pi-hole install."
      echo -n ":::    Backing up dnsmasq.conf to dnsmasq.conf.orig..."
      mv -f ${dnsmasq_conf} ${dnsmasq_conf_orig}
      echo " done."
@@ -597,7 +634,7 @@ version_check_dnsmasq() {
      cp ${dnsmasq_original_config} ${dnsmasq_conf}
      echo " done."
    else
-      echo " it is not a pi-hole file, leaving alone!"
+      echo " it is not a Pi-hole file, leaving alone!"
    fi
  else
    echo -n ":::    No dnsmasq.conf found.. restoring default dnsmasq.conf..."
@@ -608,6 +645,7 @@ version_check_dnsmasq() {
  echo -n ":::    Copying 01-pihole.conf to /etc/dnsmasq.d/01-pihole.conf..."
  cp ${dnsmasq_pihole_01_snippet} ${dnsmasq_pihole_01_location}
  echo " done."
+  sed -i "s/@INT@/$PIHOLE_INTERFACE/" ${dnsmasq_pihole_01_location}
  if [[ "${PIHOLE_DNS_1}" != "" ]]; then
    sed -i "s/@DNS1@/$PIHOLE_DNS_1/" ${dnsmasq_pihole_01_location}
  else
@@ -668,9 +706,9 @@ installScripts() {
 }

 installConfigs() {
-  # Install the configs from /etc/.pihole to their various locations
+  # Install the configs from PI_HOLE_LOCAL_REPO to their various locations
  echo ":::"
-  echo "::: Installing configs..."
+  echo "::: Installing configs from ${PI_HOLE_LOCAL_REPO}..."
  version_check_dnsmasq

  #Only mess with lighttpd configs if user has chosen to install web interface
@@ -681,7 +719,7 @@ installConfigs() {
    elif [ -f "/etc/lighttpd/lighttpd.conf" ]; then
      mv /etc/lighttpd/lighttpd.conf /etc/lighttpd/lighttpd.conf.orig
    fi
-    cp /etc/.pihole/advanced/${LIGHTTPD_CFG} /etc/lighttpd/lighttpd.conf
+    cp ${PI_HOLE_LOCAL_REPO}/advanced/${LIGHTTPD_CFG} /etc/lighttpd/lighttpd.conf
    mkdir -p /var/run/lighttpd
    chown ${LIGHTTPD_USER}:${LIGHTTPD_GROUP} /var/run/lighttpd
    mkdir -p /var/cache/lighttpd/compress
@@ -738,10 +776,11 @@ update_package_cache() {

  echo ":::"
  echo -n "::: Updating local cache of available packages..."
-  if eval ${UPDATE_PKG_CACHE} &> /dev/null; then
+  if eval "${UPDATE_PKG_CACHE}" &> /dev/null; then
    echo " done!"
  else
-    echo -n "\n!!! ERROR - Unable to update package cache. Please try \"${UPDATE_PKG_CACHE}\""
+    echo -en "\n!!! ERROR - Unable to update package cache. Please try \"${UPDATE_PKG_CACHE}\""
+    return 1
  fi
 }

@@ -758,7 +797,7 @@ notify_package_updates_available() {
      echo "::: Your system is up to date! Continuing with Pi-hole installation..."
    else
      echo "::: There are ${updatesToInstall} updates available for your system!"
-      echo "::: We recommend you update your OS after installing Pi-Hole! "
+      echo "::: We recommend you update your OS after installing Pi-hole! "
      echo ":::"
    fi
  else
@@ -788,6 +827,7 @@ install_dependent_packages() {
      fi
    done
    if [[ ${#installArray[@]} -gt 0 ]]; then
+      test_dpkg_lock
      debconf-apt-progress -- "${PKG_INSTALL[@]}" "${installArray[@]}"
      return
    fi
@@ -834,7 +874,7 @@ installPiholeWeb() {
      echo ":::     Existing index.php detected, not overwriting"
    else
      echo -n ":::     index.php missing, replacing... "
-      cp /etc/.pihole/advanced/index.php /var/www/html/pihole/
+      cp ${PI_HOLE_LOCAL_REPO}/advanced/index.php /var/www/html/pihole/
      echo " done!"
    fi

@@ -842,7 +882,7 @@ installPiholeWeb() {
      echo ":::     Existing index.js detected, not overwriting"
    else
      echo -n ":::     index.js missing, replacing... "
-      cp /etc/.pihole/advanced/index.js /var/www/html/pihole/
+      cp ${PI_HOLE_LOCAL_REPO}/advanced/index.js /var/www/html/pihole/
      echo " done!"
    fi

@@ -850,14 +890,14 @@ installPiholeWeb() {
      echo ":::     Existing blockingpage.css detected, not overwriting"
    else
      echo -n ":::     blockingpage.css missing, replacing... "
-      cp /etc/.pihole/advanced/blockingpage.css /var/www/html/pihole
+      cp ${PI_HOLE_LOCAL_REPO}/advanced/blockingpage.css /var/www/html/pihole
      echo " done!"
    fi

  else
    echo ":::     Creating directory for blocking page"
    install -d /var/www/html/pihole
-    install -D /etc/.pihole/advanced/{index,blockingpage}.* /var/www/html/pihole/
+    install -D ${PI_HOLE_LOCAL_REPO}/advanced/{index,blockingpage}.* /var/www/html/pihole/
    if [ -f /var/www/html/index.lighttpd.html ]; then
      mv /var/www/html/index.lighttpd.html /var/www/html/index.lighttpd.orig
    else
@@ -870,7 +910,7 @@ installPiholeWeb() {
  echo ":::"
  echo -n "::: Installing sudoer file..."
  mkdir -p /etc/sudoers.d/
-  cp /etc/.pihole/advanced/pihole.sudo /etc/sudoers.d/pihole
+  cp ${PI_HOLE_LOCAL_REPO}/advanced/pihole.sudo /etc/sudoers.d/pihole
  # Add lighttpd user (OS dependent) to sudoers file
  echo "${LIGHTTPD_USER} ALL=NOPASSWD: /usr/local/bin/pihole" >> /etc/sudoers.d/pihole

@@ -888,7 +928,7 @@ installCron() {
  # Install the cron job
  echo ":::"
  echo -n "::: Installing latest Cron script..."
-  cp /etc/.pihole/advanced/pihole.cron /etc/cron.d/pihole
+  cp ${PI_HOLE_LOCAL_REPO}/advanced/pihole.cron /etc/cron.d/pihole
  echo " done!"
 }

@@ -902,7 +942,7 @@ runGravity() {
  fi
  # Test if /etc/pihole/adlists.default exists
  if [[ ! -e /etc/pihole/adlists.default ]]; then
-    cp /etc/.pihole/adlists.default /etc/pihole/adlists.default
+    cp ${PI_HOLE_LOCAL_REPO}/adlists.default /etc/pihole/adlists.default
  fi
  echo "::: Running gravity.sh"
  { /opt/pihole/gravity.sh; }
@@ -925,7 +965,7 @@ configureFirewall() {
    whiptail --title "Firewall in use" --yesno "We have detected a running firewall\n\nPi-hole currently requires HTTP and DNS port access.\n\n\n\nInstall Pi-hole default firewall rules?" ${r} ${c} || \
    { echo -e ":::\n::: Not installing firewall rulesets."; return 0; }
    echo -e ":::\n:::\n Configuring FirewallD for httpd and dnsmasq."
-    firewall-cmd --permanent --add-port=80/tcp --add-port=53/tcp --add-port=53/udp
+    firewall-cmd --permanent --add-service=http --add-service=dns
    firewall-cmd --reload
    return 0
  # Check for proper kernel modules to prevent failure
@@ -977,7 +1017,7 @@ finalExports() {

  # Look for DNS server settings which would have to be reapplied
  source "${setupVars}"
-  source "/etc/.pihole/advanced/Scripts/webpage.sh"
+  source "${PI_HOLE_LOCAL_REPO}/advanced/Scripts/webpage.sh"

  if [[ "${DNS_FQDN_REQUIRED}" != "" ]] ; then
    ProcessDNSSettings
@@ -992,7 +1032,7 @@ installLogrotate() {
  # Install the logrotate script
  echo ":::"
  echo -n "::: Installing latest logrotate script..."
-  cp /etc/.pihole/advanced/logrotate /etc/pihole/logrotate
+  cp ${PI_HOLE_LOCAL_REPO}/advanced/logrotate /etc/pihole/logrotate
  # Different operating systems have different user / group
  # settings for logrotate that makes it impossible to create
  # a static logrotate file that will work with e.g.
@@ -1031,6 +1071,7 @@ installPihole() {
  fi
  installCron
  installLogrotate
+  FTLdetect || echo "::: FTL Engine not installed."
  configureFirewall
  finalExports
  #runGravity
@@ -1062,6 +1103,7 @@ updatePihole() {
  fi
  installCron
  installLogrotate
+  FTLdetect || echo "::: FTL Engine not installed."
  finalExports #re-export setupVars.conf to account for any new vars added in new versions
  #runGravity
 }
@@ -1155,10 +1197,98 @@ if [[ "${reconfigure}" == true ]]; then
    fi
 }

+FTLinstall() {
+  # Download and install FTL binary
+  local binary="${1}"
+  local latesttag
+  local orig_dir
+  echo -n ":::  Installing FTL... "
+
+  orig_dir="${PWD}"
+  latesttag=$(curl -sI https://github.com/pi-hole/FTL/releases/latest | grep "Location" | awk -F '/' '{print $NF}')
+  # Tags should always start with v, check for that.
+  if [[ ! "${latesttag}" == v* ]]; then
+    echo "failed (error in getting latest release location from GitHub)"
+    return 1
+  fi
+  if curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${latesttag%$'\r'}/${binary}" -o "/tmp/${binary}"; then
+    # Get sha1 of the binary we just downloaded for verification.
+    curl -sSL --fail "https://github.com/pi-hole/FTL/releases/download/${latesttag%$'\r'}/${binary}.sha1" -o "/tmp/${binary}.sha1"
+    # Check if we just downloaded text, or a binary file.
+    cd /tmp
+    if sha1sum --status --quiet -c "${binary}".sha1; then
+      echo -n "transferred... "
+      stop_service pihole-FTL &> /dev/null
+      install -T -m 0755 /tmp/${binary} /usr/bin/pihole-FTL
+      cd "${orig_dir}"
+      install -T -m 0755 "${PI_HOLE_LOCAL_REPO}/advanced/pihole-FTL.service" "/etc/init.d/pihole-FTL"
+      echo "done."
+      return 0
+    else
+      echo "failed (download of binary from Github failed)"
+      cd "${orig_dir}"
+      return 1
+    fi
+  else
+    cd "${orig_dir}"
+    echo "failed (URL not found.)"
+  fi
+}
+
+FTLdetect() {
+  # Detect suitable FTL binary platform
+  echo ":::"
+  echo "::: Downloading latest version of FTL..."
+
+  local machine
+  local binary
+
+  machine=$(uname -m)
+
+  if [[ $machine == arm* || $machine == *aarch* ]]; then
+    # ARM
+    local rev=$(uname -m | sed "s/[^0-9]//g;")
+    local lib=$(ldd /bin/ls | grep -E '^\s*/lib' | awk '{ print $1 }')
+    if [[ "$lib" == "/lib/ld-linux-aarch64.so.1" ]]; then
+      echo ":::  Detected ARM-aarch64 architecture"
+      binary="pihole-FTL-aarch64-linux-gnu"
+    elif [[ "$lib" == "/lib/ld-linux-armhf.so.3" ]]; then
+      if [ "$rev" -gt "6" ]; then
+        echo ":::  Detected ARM-hf architecture (armv7+)"
+        binary="pihole-FTL-arm-linux-gnueabihf"
+      else
+        echo ":::  Detected ARM-hf architecture (armv6 or lower)"
+        echo ":::  Using ARM binary"
+        binary="pihole-FTL-arm-linux-gnueabi"
+      fi
+    else
+      echo ":::  Detected ARM architecture"
+      binary="pihole-FTL-arm-linux-gnueabi"
+    fi
+  elif [[ $machine == x86_64 ]]; then
+    # 64bit
+    echo ":::  Detected x86_64 architecture"
+    binary="pihole-FTL-linux-x86_64"
+  else
+    # Something else - we try to use 32bit executable and warn the user
+    if [[ ! $machine == i686 ]]; then
+      echo ":::  Not able to detect architecture (unknown: ${machine}), trying 32bit executable"
+      echo ":::  Contact Pi-hole support if you experience problems (like FTL not running)"
+    else
+      echo ":::  Detected 32bit (i686) architecture"
+    fi
+    binary="pihole-FTL-linux-x86_32"
+  fi
+
+  FTLinstall "${binary}" || return 1
+
+}
+
 main() {

  ######## FIRST CHECK ########
  # Must be root to install
+  show_ascii_berry
  echo ":::"
  if [[ ${EUID} -eq 0 ]]; then
    echo "::: You are root."
@@ -1209,7 +1339,7 @@ main() {
  fi

  # Update package cache
-  update_package_cache
+  update_package_cache || exit 1

  # Notify user of package availability
  notify_package_updates_available
@@ -1283,7 +1413,8 @@ main() {
    pw=""
    if [[ $(grep 'WEBPASSWORD' -c /etc/pihole/setupVars.conf) == 0 ]] ; then
        pw=$(tr -dc _A-Z-a-z-0-9 < /dev/urandom | head -c 8)
-        /usr/local/bin/pihole -a -p "${pw}"
+        . /opt/pihole/webpage.sh
+        echo "WEBPASSWORD=$(HashPassword ${pw})" >> ${setupVars}
    fi
  fi

@@ -1299,6 +1430,9 @@ main() {

  runGravity

+  start_service pihole-FTL
+  enable_service pihole-FTL
+
  echo "::: done."

  if [[ "${useUpdateVars}" == false ]]; then
@@ -1327,7 +1461,7 @@ main() {
      echo ":::                                ${pw}"
      echo ":::"
      echo "::: You can always change it using"
-      echo ":::                                pihole -a -p new_password"
+      echo ":::                                pihole -a -p"
    fi
  fi

--- a/gravity.sh
+++ b/gravity.sh
@@ -26,10 +26,12 @@ EOM
 	exit 0
 }

+PIHOLE_COMMAND="/usr/local/bin/pihole"

 adListFile=/etc/pihole/adlists.list
-adListDefault=/etc/pihole/adlists.default
-whitelistScript="pihole -w"
+adListDefault=/etc/pihole/adlists.default #being deprecated
+adListRepoDefault=/etc/.pihole/adlists.default
+whitelistScript="${PIHOLE_COMMAND} -w"
 whitelistFile=/etc/pihole/whitelist.txt
 blacklistFile=/etc/pihole/blacklist.txt
 readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
@@ -70,36 +72,34 @@ fi
 ###########################
 # collapse - begin formation of pihole
 gravity_collapse() {
+
+  #New Logic:
+  # Does /etc/pihole/adlists.list exist? If so leave it alone
+  #                                      If not, cp /etc/.pihole/adlists.default /etc/pihole/adlists.list
+  # Read from adlists.list
+
+  #The following two blocks will sort out any missing adlists in the /etc/pihole directory, and remove legacy adlists.default
+  if [ -f ${adListDefault} ] && [ -f ${adListFile} ]; then
+    rm ${adListDefault}
+  fi
+
+  if [ ! -f ${adListFile} ]; then
+    cp ${adListRepoDefault} ${adListFile}
+  fi
+
 	echo "::: Neutrino emissions detected..."
 	echo ":::"
-	#Decide if we're using a custom ad block list, or defaults.
-	if [ -f ${adListFile} ]; then
-		#custom file found, use this instead of default
-		echo -n "::: Custom adList file detected. Reading..."
-		sources=()
-		while IFS= read -r line || [[ -n "$line" ]]; do
-			#Do not read commented out or blank lines
-			if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
-				echo "" > /dev/null
-			else
-				sources+=(${line})
-			fi
-		done < ${adListFile}
-		echo " done!"
-	else
-		#no custom file found, use defaults!
-		echo -n "::: No custom adlist file detected, reading from default file..."
-		sources=()
-		while IFS= read -r line || [[ -n "$line" ]]; do
-			#Do not read commented out or blank lines
-			if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
-				echo "" > /dev/null
-			else
-				sources+=(${line})
-			fi
-		done < ${adListDefault}
-		echo " done!"
-	fi
+  echo -n "::: Pulling source lists into range..."
+  sources=()
+  while IFS= read -r line || [[ -n "$line" ]]; do
+    #Do not read commented out or blank lines
+    if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
+      echo "" > /dev/null
+    else
+      sources+=(${line})
+    fi
+  done < ${adListFile}
+  echo " done!"
 }

 # patternCheck - check to see if curl downloaded any new files.
@@ -168,6 +168,10 @@ gravity_transport() {
 	# Process result
 	gravity_patternCheck "${patternBuffer}" ${success} "${err}"

+        # Delete temp file if it hasn't been moved
+        if [[ -f "${patternBuffer}" ]]; then
+                rm "${patternBuffer}"
+        fi
 }

 # spinup - main gravity function
@@ -183,22 +187,26 @@ gravity_spinup() {
 		saveLocation=${piholeDir}/list.${i}.${domain}.${justDomainsExtension}
 		activeDomains[$i]=${saveLocation}

-		agent="Mozilla/10.0"
+		agent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36"

 		# Use a case statement to download lists that need special cURL commands
 		# to complete properly and reset the user agent when required
 		case "${domain}" in
-			"adblock.mahakala.is")
+		    "adblock.mahakala.is")
 			agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36'
 			cmd_ext="-e http://forum.xda-developers.com/"
 		    ;;

+		    "adaway.org")
+			agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36'
+		    ;;
+
 		    "pgl.yoyo.org")
 			cmd_ext="-d mimetype=plaintext -d hostformat=hosts"
 		    ;;

-            # Default is a simple request
-            *) cmd_ext=""
+		    # Default is a simple request
+		    *) cmd_ext=""
        esac
        if [[ "${skipDownload}" == false ]]; then
            echo -n "::: Getting $domain list..."
@@ -385,7 +393,7 @@ gravity_reload() {
 	#Now replace the line in dnsmasq file
 #	sed -i "s/^addn-hosts.*/addn-hosts=$adList/" /etc/dnsmasq.d/01-pihole.conf

-	pihole restartdns
+	"${PIHOLE_COMMAND}" restartdns
 	echo " done!"
 }

@@ -403,8 +411,6 @@ if [[ "${forceGrav}" == true ]]; then
 	echo " done!"
 fi

-#Overwrite adlists.default from /etc/.pihole in case any changes have been made. Changes should be saved in /etc/adlists.list
-cp /etc/.pihole/adlists.default /etc/pihole/adlists.default
 gravity_collapse
 gravity_spinup
 if [[ "${skipDownload}" == false ]]; then
@@ -423,4 +429,4 @@ gravity_hostFormat
 gravity_blackbody

 gravity_reload
-pihole status
+"${PIHOLE_COMMAND}" status
--- a/51
+++ b/51
@@ -8,9 +8,8 @@
 # This file is copyright under the latest version of the EUPL.
 # Please see LICENSE file for your rights under this license.

+readonly PI_HOLE_SCRIPT_DIR="/opt/pihole"

-
-PI_HOLE_SCRIPT_DIR="/opt/pihole"
 readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
 # Must be root to use this tool
 if [[ ! $EUID -eq 0 ]];then
@@ -128,17 +127,19 @@ queryFunc() {
  done

  # Scan for possible wildcard matches
-  local wildcards=($(processWildcards "${domain}"))
-  for domain in ${wildcards[@]}; do
-    result=$(scanList "\/${domain}\/" ${wildcardlist})
-    # Remove empty lines before couting number of results
-    count=$(sed '/^\s*$/d' <<< "$result" | wc -l)
-    if [[ ${count} > 0 ]]; then
-      echo "::: Wildcard blocking ${domain} (${count} results)"
-      echo "${result}"
-      echo ""
-    fi
-  done
+  if [ -e "${wildcardlist}" ]; then
+    local wildcards=($(processWildcards "${domain}"))
+    for domain in ${wildcards[@]}; do
+      result=$(scanList "\/${domain}\/" ${wildcardlist})
+      # Remove empty lines before couting number of results
+      count=$(sed '/^\s*$/d' <<< "$result" | wc -l)
+      if [[ ${count} > 0 ]]; then
+        echo "::: Wildcard blocking ${domain} (${count} results)"
+        echo "${result}"
+        echo ""
+      fi
+    done
+  fi
  exit 0
 }

@@ -278,19 +279,25 @@ tailFunc() {
  exit 0
 }

+piholeCheckoutFunc() {
+  source "${PI_HOLE_SCRIPT_DIR}"/piholeCheckout.sh
+  shift
+  checkout "$@"
+}
+
 helpFunc() {
 	cat << EOM
-::: Control all PiHole specific functions!
+::: Control all Pi-hole specific functions
 :::
 ::: Usage: pihole [options]
-:::		Add -h after -w (whitelist), -b (blacklist), -c (chronometer), or -a (admin)  for more information on usage
+:::		Add -h after -w (whitelist), -b (blacklist), -c (chronometer), or -a (admin) for more information on usage
 :::
 ::: Options:
 :::  -w, whitelist            Whitelist domain(s)
 :::  -b, blacklist            Blacklist domain(s) (exact match)
 :::  -wild, wildcard          Blacklist whole domain(s) (wildcard)
 :::  -d, debug                Start a debugging session
-:::                             Automated debugging can be enabled with `-a`.
+:::                             Automated debugging can be enabled with '-a'.
 :::                             'pihole -d -a'
 :::  -f, flush                Flush the 'pihole.log' file
 :::  -t, tail                 Output the last lines of the 'pihole.log' file. Lines are appended as the file grows
@@ -299,18 +306,19 @@ helpFunc() {
 :::  -g, updateGravity        Update the list of ad-serving domains
 :::  -c, chronometer          Calculates stats and displays to an LCD
 :::  -h, help                 Show this help dialog
-:::  -v, version              Show installed versions of Pi-Hole and Web-Admin
+:::  -v, version              Show installed versions of Pi-hole and Web-Admin
 :::  -q, query                Query the adlists for a specific domain
 :::                             'pihole -q domain -exact' shows exact matches only
 :::  -l, logging              Enable or Disable logging (pass 'on' or 'off')
 :::  -a, admin                Admin webpage options
-:::  uninstall                Uninstall Pi-Hole from your system :(!
-:::  status                   Is Pi-Hole Enabled or Disabled
-:::  enable                   Enable Pi-Hole DNS Blocking
-:::  disable                  Disable Pi-Hole DNS Blocking
+:::  uninstall                Uninstall Pi-hole from your system! :(
+:::  status                   Display if Pi-hole is Enabled or Disabled
+:::  enable                   Enable Pi-hole DNS Blocking
+:::  disable                  Disable Pi-hole DNS Blocking
 :::                             Blocking can also be disabled only temporarily, e.g.,
 :::                             'pihole disable 5m' - will disable blocking for 5 minutes
 :::  restartdns               Restart dnsmasq
+:::  checkout                 Check out different branches
 EOM
  exit 0
 }
@@ -341,5 +349,6 @@ case "${1}" in
  "restartdns"                  ) restartDNS;;
  "-a" | "admin"                ) webpageFunc "$@";;
  "-t" | "tail"                 ) tailFunc;;
+  "checkout"                    ) piholeCheckoutFunc "$@";;
  *                             ) helpFunc;;
 esac
--- a/test/debian.Dockerfile
+++ b/test/debian.Dockerfile
@@ -1,4 +1,4 @@
-FROM debian:jessie
+FROM buildpack-deps:jessie-scm

 ENV GITDIR /etc/.pihole
 ENV SCRIPTDIR /opt/pihole
--- a/test/test_automated_install.py
+++ b/test/test_automated_install.py
@@ -78,7 +78,7 @@ def test_configureFirewall_firewalld_running_no_errors(Pihole):
    assert expected_stdout in configureFirewall.stdout
    firewall_calls = Pihole.run('cat /var/log/firewall-cmd').stdout
    assert 'firewall-cmd --state' in firewall_calls
-    assert 'firewall-cmd --permanent --add-port=80/tcp --add-port=53/tcp --add-port=53/udp' in firewall_calls
+    assert 'firewall-cmd --permanent --add-service=http --add-service=dns' in firewall_calls
    assert 'firewall-cmd --reload' in firewall_calls

 def test_configureFirewall_firewalld_disabled_no_errors(Pihole):
@@ -297,6 +297,111 @@ def test_update_package_cache_failure_no_errors(Pihole):
    assert 'ERROR' in updateCache.stdout
    assert 'done!' not in updateCache.stdout

+def test_FTL_detect_aarch64_no_errors(Pihole):
+    ''' confirms only aarch64 package is downloaded for FTL engine '''
+    # mock uname to return aarch64 platform
+    mock_command('uname', {'-m':('aarch64', '0')}, Pihole)
+    # mock ldd to respond with aarch64 shared library
+    mock_command('ldd', {'/bin/ls':('/lib/ld-linux-aarch64.so.1', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected ARM-aarch64 architecture'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_armv6l_no_errors(Pihole):
+    ''' confirms only armv6l package is downloaded for FTL engine '''
+    # mock uname to return armv6l platform
+    mock_command('uname', {'-m':('armv6l', '0')}, Pihole)
+    # mock ldd to respond with aarch64 shared library
+    mock_command('ldd', {'/bin/ls':('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected ARM-hf architecture (armv6 or lower)'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_armv7l_no_errors(Pihole):
+    ''' confirms only armv7l package is downloaded for FTL engine '''
+    # mock uname to return armv7l platform
+    mock_command('uname', {'-m':('armv7l', '0')}, Pihole)
+    # mock ldd to respond with aarch64 shared library
+    mock_command('ldd', {'/bin/ls':('/lib/ld-linux-armhf.so.3', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected ARM-hf architecture (armv7+)'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_x86_64_no_errors(Pihole):
+    ''' confirms only x86_64 package is downloaded for FTL engine '''
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Detected x86_64 architecture'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_detect_unknown_no_errors(Pihole):
+    ''' confirms only generic package is downloaded for FTL engine '''
+    # mock uname to return generic platform
+    mock_command('uname', {'-m':('mips', '0')}, Pihole)
+    detectPlatform = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    ''')
+    expected_stdout = 'Not able to detect architecture (unknown: mips)'
+    assert expected_stdout in detectPlatform.stdout
+
+def test_FTL_download_aarch64_no_errors(Pihole):
+    ''' confirms only aarch64 package is downloaded for FTL engine '''
+    # mock uname to return generic platform
+    download_binary = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLinstall pihole-FTL-aarch64-linux-gnu
+    ''')
+    expected_stdout = 'done'
+    assert expected_stdout in download_binary.stdout
+    assert 'failed' not in download_binary.stdout
+
+def test_FTL_download_unknown_fails_no_errors(Pihole):
+    ''' confirms unknown binary is not downloaded for FTL engine '''
+    # mock uname to return generic platform
+    download_binary = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLinstall pihole-FTL-mips
+    ''')
+    expected_stdout = 'failed'
+    assert expected_stdout in download_binary.stdout
+    assert 'done' not in download_binary.stdout
+
+def test_FTL_binary_installed_and_responsive_no_errors(Pihole):
+    ''' confirms FTL binary is copied and functional in installed location '''
+    installed_binary = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    FTLdetect
+    pihole-FTL version
+    ''')
+    expected_stdout = 'v'
+    assert expected_stdout in installed_binary.stdout
+
+# def test_FTL_support_files_installed(Pihole):
+#     ''' confirms FTL support files are installed '''
+#     support_files = Pihole.run('''
+#     source /opt/pihole/basic-install.sh
+#     FTLdetect
+#     stat -c '%a %n' /var/log/pihole-FTL.log
+#     stat -c '%a %n' /run/pihole-FTL.port
+#     stat -c '%a %n' /run/pihole-FTL.pid
+#     ls -lac /run
+#     ''')
+#     assert '644 /run/pihole-FTL.port' in support_files.stdout
+#     assert '644 /run/pihole-FTL.pid' in support_files.stdout
+#     assert '644 /var/log/pihole-FTL.log' in support_files.stdout
+
 # Helper functions
 def mock_command(script, args, container):
    ''' Allows for setup of commands we don't really want to have to run for real in unit tests '''