Compare commits

...

74 Commits

Author SHA1 Message Date
Dan Schaper
6ce527010b Initial work on stubbing for lua.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-06-21 10:49:46 -07:00
Adam Warner
761c1cd305 Merge pull request #4175 from ashuio/development
Added docker to README
2021-05-20 18:49:01 +01:00
Ashutosh Verma
6f60555f79 Added install Method 3
Signed-off-by: Ashutosh Verma <mail@ashu.io>
2021-05-20 22:34:19 +05:30
Adam Warner
a51be7d498 Merge pull request #4153 from MichaIng/patch-2
Re-remove wget from dependencies
2021-05-01 00:09:19 +01:00
MichaIng
f3ceebbe06 Re-remove wget from dependencies
The dependency has been removed here: https://github.com/pi-hole/pi-hole/pull/3185
But accidentally re-added here: 982c1b0059 (diff-595630a29a855f6d667a84ca0662042e826bf3ec56322ef61d4a6ef149147d23)

wget is still not used in any Pi-hole script, hence it can be removed safely from dependencies.

Signed-off-by: MichaIng <micha@dietpi.com>
2021-04-30 20:48:12 +02:00
Adam Warner
57d9b96b53 Merge pull request #4143 from pi-hole/version-fullstops
Cosmetic change to pihole -v -c output
2021-04-25 17:47:40 +01:00
Adam Warner
5895690b58 this has been annoying me for months
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2021-04-24 15:38:27 +01:00
DL6ER
d26f8dd665 Merge pull request #4135 from MichaIng/patch-1
Enable PHP8 support for Debian/Ubuntu
2021-04-21 20:19:25 +02:00
MichaIng
2ba350984f Enable PHP8 support for Debian/Ubuntu
Since PHP8.0, the JSON extension is a core PHP extension and hence the php8.0-json package does not exist and is not required:
- https://www.php.net/manual/json.installation.php
- https://packages.debian.org/php8.0-json

Solves: https://discourse.pi-hole.net/t/php-8-packages-not-found/46286

Signed-off-by: MichaIng <micha@dietpi.com>
2021-04-18 15:33:05 +02:00
Dan Schaper
1fb6366c41 Merge pull request #4123 from pi-hole/master
Master v5.3.1 to development
2021-04-14 14:30:30 -07:00
Dan Schaper
4736e03108 Merge pull request #4122 from pi-hole/release/v5.3.1
HotFix release v5.3.1
2021-04-14 14:10:21 -07:00
Dan Schaper
18bedc0493 Merge pull request #4121 from pi-hole/fix/port_hash
Fix port separator in IPv4 address validation code
2021-04-14 13:59:52 -07:00
DL6ER
74948ae5c1 The correct port separator is '#' not ':' in dnsmasq style. This should not have been changed in #4083 in the first place.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-04-14 22:53:12 +02:00
Dan Schaper
9b57f7fa33 Merge pull request #4120 from pi-hole/fix/splashPage
Add missing closing single quote
2021-04-14 13:50:45 -07:00
Dan Schaper
866045968d Add missing closing single quote
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-04-14 12:27:36 -07:00
Dan Schaper
f2e0cf8a92 Merge pull request #4117 from pi-hole/master
v5.8 master to dev
2021-04-14 12:24:08 -07:00
Adam Warner
4c39edbeb9 Merge pull request #4113 from pi-hole/release/v5.3
Pi-hole Core v5.3
2021-04-14 19:04:03 +01:00
Adam Warner
b388ac5bb5 Merge pull request from GHSA-3597-244c-wrpj
Add some validation to passed arguments before running the values through `sed`
2021-04-14 18:16:46 +01:00
Adam Warner
7855859726 Merge pull request #4116 from pi-hole/fix/update-readme
Update README
2021-04-14 17:57:47 +01:00
DL6ER
5df7ed2f32 Obtain FTL's PID from the PID file (#4103)
* Try to obtain FTL's PID from the PID file. If this fails, try to identify the main process using pgrep --oldest (instead of relying on pkill finding the right one by itself). This allows the script to work in even when FTL is running inside the memory checker valgrind.

* Rename FTL_PID -> FTL_PID_FILE

* Remove the pgrep fallback after discussions about that it should be more obvious to users if something strange happened to their PID file. Also, simplify the routine using a bashism in the end.

* Shorten if [[ regex ]]

* Use unset instead of emptying the PID variable

Signed-off-by: DL6ER <dl6er@dl6er.de>

Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-04-14 09:33:10 -07:00
Dan Schaper
25ded79190 Update README 2021-04-14 09:26:52 -07:00
Adam Warner
d3e00f038b Merge pull request #4004 from pi-hole/fedora_33_support
Fedora 33 Support
2021-04-13 08:41:56 +01:00
bcambl
d5f13018a7 Merge branch 'release/v5.3' into fedora_33_support
Signed-off-by: bcambl <blayne@blaynecampbell.com>
2021-04-12 20:06:11 -06:00
Adam Warner
c6e3805bbb Merge pull request #4083 from jbzdarkid/patch-2
Improve regexes for IPv4 matching
2021-04-12 21:12:20 +01:00
Adam Warner
ea451e3c22 Merge pull request #4095 from jbaez/ft-3896
Fix error trying to access undefined variables on splash page.
2021-04-12 20:41:25 +01:00
DL6ER
b97f890a8c Merge pull request #4086 from pi-hole/fix/GRAVITDB_in_pihole-g
pihole -g should respose GRAVITYDB in pihole-FTL.conf
2021-04-11 05:18:09 +02:00
Jaime Baez
6198165df8 Fix error trying to access undefined variables on splash page.
Signed-off-by: Jaime Baez <j.baezcastillo@gmail.com>
2021-03-26 23:42:11 +01:00
jbzdarkid
8090071eff Update regex to disallow leading zeros
Also updated a comment to point to a non-experimental RFC.

Signed-off-by: jbzdarkid <jbzdarkid@gmail.com>
2021-03-21 16:27:08 -07:00
Joseph Blackman
89c80947df Improve regexes for ipv4 matching
Signed-off-by: jbzdarkid <jbzdarkid@gmail.com>
2021-03-21 16:24:35 -07:00
Adam Warner
b51751e892 Merge pull request #4084 from jbzdarkid/patch-1
Improve comments in basic-install.sh
2021-03-21 23:14:59 +00:00
Adam Warner
45d4ab4c22 Add some validation to passed arguments before running the values through sed
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2021-03-19 18:39:43 +00:00
DL6ER
f536718aaa Use possibly custom gravity.db location in the copying SQL script
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-03-18 16:46:59 +01:00
DL6ER
f1740da9ff Update gravity path in query.sh
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-03-18 09:58:19 +01:00
DL6ER
9bdbe88a66 Update gravity path in list.sh
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-03-18 09:58:19 +01:00
DL6ER
d25240fe9f pihole -g should respose GRAVITYDB in pihole-FTL.conf
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-03-18 09:00:27 +01:00
jbzdarkid
20b6f9cceb Improve comments in basic-install.sh
Signed-off-by: jbzdarkid <jbzdarkid@gmail.com>
2021-03-17 18:37:18 -07:00
Dan Schaper
f27fd9f899 Merge pull request #3982 from andras-tim/list-fix-same-wildcard-regex
Fix list: User same wildcard regex that the UI use
2021-03-17 11:20:12 -07:00
Dany
13cf0c1288 Aligned pihole logo to center, Fixed responsive issue (#4036)
* Aligned pihole logo to center, Fixed responsive issue

* removed duplicate instance of #splashpage

Signed-off-by: Dany Gauthier <danygauthier57@yahoo.ca>
2021-03-17 11:18:50 -07:00
Adam Warner
ae762574e5 Merge pull request #4070 from pi-hole/fix/pihole-g-r
Fix `pihole -g -r`
2021-03-03 14:43:42 +00:00
DL6ER
126b9ae381 Add missing three new columns to pihole -g -r
Signed-off-by: DL6ER <dl6er@dl6er.de>
2021-03-02 20:08:09 +01:00
DL6ER
57fe3b6621 Merge pull request #3951 from pi-hole/new/gravity_adlist_infos
Store gravity details in gravity.db adlist table
2021-02-28 17:01:19 +01:00
Adam Warner
56fa9714b5 Merge pull request #4010 from mtzfederico/master
Updates lighttpd config to allow a persistent port change on IPv6
2021-02-28 14:42:21 +00:00
Dan Schaper
03a8ca5c1b Merge pull request #4049 from yubiuser/tewak/teleporter_cli
Add hostname to teleporter backup file if called from cli
2021-02-16 08:02:49 -08:00
Christian König
78027bd2bf Add default noname if hostname is empty
Signed-off-by: Christian König <ckoenig@posteo.de>
2021-02-12 20:25:01 +01:00
Christian König
7dc7cbb80e Add hostname to teleporter backup file if called from cli
Signed-off-by: Christian König <ckoenig@posteo.de>
2021-02-11 19:45:32 +01:00
Dan Schaper
debc9a69c5 Merge pull request #4021 from Flashbang007/patch-3
Fix issue with multipe default gateways when running the debug
2021-01-27 08:45:22 -08:00
Alexander Schmitz
1ab193fa9d Update piholeDebug.sh
Signed-off-by: Alexander Schmitz <Flashbang007@gmail.com>
2021-01-27 11:00:43 +01:00
Adam Warner
9ec012289f Merge pull request #4017 from mwoolweaver/patch-2
fix suspected typo in comment
2021-01-26 08:48:05 +00:00
Michael Woolweaver
c42b97ddb2 fix typo in comment
Signed-off-by: Michael Woolweaver <m.woolweaver@icloud.com>
2021-01-26 01:04:37 -06:00
mtzfederico
e5bfafefb9 Moved where external.conf is included
Signed-off-by: mtzfederico <fm@mtzfederico.com>
2021-01-22 15:53:40 -06:00
bcambl
b62495d89e update tests to fedora 33
Signed-off-by: bcambl <blayne@blaynecampbell.com>
2021-01-21 20:20:07 -06:00
bcambl
c6810a0124 touch sysconfig network script when absent
sysconfig network scripts are missing on pure NetworkManager installs.
This commit touches a placeholder to allow the network configuration to
be generated and loaded via NetworkManager.

Signed-off-by: bcambl <blayne@blaynecampbell.com>
2021-01-19 23:06:39 -06:00
Adam Warner
8e7670db41 Merge pull request #4002 from freddii/master
fixed typos
2021-01-19 18:46:55 +00:00
freddii
7c0c30fb0b fixed typos 2021-01-19 19:33:38 +01:00
Adam Warner
cbfb58f7a2 Merge pull request #4000 from pi-hole/release/v5.2.4
Pi-hole Core v5.2.4
2021-01-19 12:29:01 +00:00
Adam Warner
0c45d23c22 Merge pull request #3997 from pi-hole/revert-3537-patch-2
Revert "Further failsafe check for available APT packages"
2021-01-19 08:19:46 +00:00
Dan Schaper
3f8fedfb16 Revert "Further failsafe check for available APT packages" 2021-01-17 11:10:24 -08:00
Adam Warner
cbc3fbdfe6 Merge pull request #3989 from pi-hole/release/v5.2.3
Pi-hole Core v5.2.3
2021-01-15 19:45:55 +00:00
Dan Schaper
020b6b8064 Merge pull request #3987 from pi-hole/fix/GitHubAPI
Use the 'Location:' header only for latest FTL tag API check.
2021-01-13 09:38:09 -08:00
Dan Schaper
523f650157 Use the 'Location:' header only.
Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
2021-01-12 13:42:51 -08:00
Andras Tim
bfee230c79 scripts/list: User same wildcard regex that the UI use
The CLI and the UI should do the same to be able to add/remove the same
records via any supported interface.

Signed-off-by: Andras Tim <andras.tim@gmail.com>
2021-01-11 17:05:17 +01:00
DL6ER
c5ed8f8bed Merge pull request #3537 from MichaIng/patch-2
Further failsafe check for available APT packages
2021-01-07 11:56:52 +01:00
Dan Schaper
66dfa5fc1e Merge pull request #3958 from pi-hole/Fix/centos_stream
Fix release check for centos stream
2021-01-06 10:59:27 -08:00
DL6ER
1791fe22f6 Merge pull request #3912 from mdujava/basic_auth
Fix validation of adlist url
2020-12-31 12:19:32 +01:00
bcambl
a5422dbdf6 fix release check for centos stream
fixes #3947 #3953

Signed-off-by: bcambl <blayne@blaynecampbell.com>
2020-12-30 19:45:51 -06:00
DL6ER
a2625df5e2 Remove "The list is new" comment because it is superfluous
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-12-30 11:27:34 +01:00
DL6ER
7507d533ac Merge pull request #3954 from pi-hole/fix/chronometer_connection_reset
Fix connection mechanism between Chronometer and FTL
2020-12-30 11:13:05 +01:00
DL6ER
0944807491 Actually store correct number of domains for the individual lists (and not the sum of the so far collected number of domains)
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-12-29 20:36:49 +01:00
DL6ER
a216848c1d Only update time of last list change when we see a list for the first time or when it really changed content
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-12-29 20:28:09 +01:00
DL6ER
e033ee6664 Fix connection mechanism between Chronometer and FTL
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-12-29 10:29:02 +01:00
DL6ER
8c56f54a1e Compare checksum of downloaded list against older checksums to see if the list content changed since the last download
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-12-29 09:54:25 +01:00
DL6ER
6b33b8b4c0 Store status of downloaded list (downloaded, using cache, some error, ...) and number of (in-)valid domains on this list in the gravity database. This updates the gravity databaes to version 14.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2020-12-28 08:08:49 +01:00
Matej Dujava
ef0bdf6470 Fix validation of adlist url
Already existing regex validation will be used on url after removing @ (in case
its in separating userinfo and host).

Signed-off-by: Matej Dujava <mdujava@kocurkovo.cz>
Fixes: https://github.com/pi-hole/pi-hole/issues/3911
Fixes: 7d19ee1b: validate blocklist URL before adding to the database (#3237)
2020-12-21 12:50:02 +01:00
MichaIng
eb5661b553 Further failsafe check for available APT packages
- "apt-cache show package" succeeds as well if package is listed as (optional) dependency or conflict by another package, hence is not a 100% reliable measure.
- There is no command which explicitly checks which package/name can be selected by apt-get for install. An install simulation/dry-run is possible as it was before Pi-hole v5.1, or the whole package cache can be scraped, which is still the less time consuming solution.
- Allow to succeed if another package "provides" it, like "php7.3-apcu" provided by "php-apcu" or "awk" provided by "mawk" and "gawk", in which case the non-virtual package is selected automatically by apt-get.

For reference: 066b89fa41

Signed-off-by: MichaIng <micha@dietpi.com>
2020-12-18 23:20:24 +01:00
25 changed files with 663 additions and 354 deletions

View File

@@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
distro: [debian_9, debian_10, ubuntu_16, ubuntu_18, ubuntu_20, centos_7, centos_8, fedora_31, fedora_32]
distro: [debian_9, debian_10, ubuntu_16, ubuntu_18, ubuntu_20, centos_7, centos_8, fedora_32, fedora_33]
env:
DISTRO: ${{matrix.distro}}
steps:

View File

@@ -1,5 +1,7 @@
<!-- markdownlint-configure-file { "MD004": { "style": "consistent" } } -->
<!-- markdownlint-disable MD033 -->
#
<p align="center">
<a href="https://pi-hole.net/">
<img src="https://pi-hole.github.io/graphics/Vortex/Vortex_with_Wordmark.svg" width="150" height="260" alt="Pi-hole">
@@ -9,8 +11,6 @@
</p>
<!-- markdownlint-enable MD033 -->
#
The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) that protects your devices from unwanted content, without installing any client-side software.
- **Easy-to-install**: our versatile installer walks you through the process, and takes less than ten minutes
@@ -26,8 +26,6 @@ The Pi-hole® is a [DNS sinkhole](https://en.wikipedia.org/wiki/DNS_Sinkhole) th
-----
Master [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=master)](https://travis-ci.com/pi-hole/pi-hole) Development [![Build Status](https://travis-ci.com/pi-hole/pi-hole.svg?branch=development)](https://travis-ci.com/pi-hole/pi-hole)
## One-Step Automated Install
Those who want to get started quickly and conveniently may install Pi-hole using the following command:
@@ -52,6 +50,8 @@ sudo bash basic-install.sh
wget -O basic-install.sh https://install.pi-hole.net
sudo bash basic-install.sh
```
### Method 3: Using Docker to deploy Pi-hole
Please refer to the [Pi-hole docker repo](https://github.com/pi-hole/docker-pi-hole) to use the Official Docker Images.
## [Post-install: Make your network take advantage of Pi-hole](https://docs.pi-hole.net/main/post-install/)
@@ -71,16 +71,18 @@ Make no mistake: **your support is absolutely vital to help keep us innovating!*
### [Donations](https://pi-hole.net/donate)
Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses:
Sending a donation using our Sponsor Button is **extremely helpful** in offsetting a portion of our monthly expenses and rewarding our dedicated development team:
### Alternative support
If you'd rather not donate (_which is okay!_), there are other ways you can help support us:
- [Patreon](https://patreon.com/pihole) _Become a patron for rewards_
- [GitHub Sponsors](https://github.com/sponsors/pi-hole/)
- [Patreon](https://patreon.com/pihole)
- [Hetzner Cloud](https://hetzner.cloud/?ref=7aceisRX3AzA) _affiliate link_
- [Digital Ocean](https://www.digitalocean.com/?refcode=344d234950e1) _affiliate link_
- [Stickermule](https://www.stickermule.com/unlock?ref_id=9127301701&utm_medium=link&utm_source=invite) _earn a $10 credit after your first purchase_
- [Amazon](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
- [Amazon US](http://www.amazon.com/exec/obidos/redirect-home/pihole09-20) _affiliate link_
- Spreading the word about our software, and how you have benefited from it
### Contributing via GitHub
@@ -95,7 +97,9 @@ You'll find that the [install script](https://github.com/pi-hole/pi-hole/blob/ma
## Getting in touch with us
While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets. **Please be sure to check the FAQ's** before starting a new discussion, as we do not have the spare time to reply to every request for assistance.
While we are primarily reachable on our [Discourse User Forum](https://discourse.pi-hole.net/), we can also be found on a variety of social media outlets.
**Please be sure to check the FAQ's** before starting a new discussion. Many user questions already have answers and can be solved without any additional assistance.
- [Frequently Asked Questions](https://discourse.pi-hole.net/c/faqs)
- [Feature Requests](https://discourse.pi-hole.net/c/feature-requests?order=votes)
@@ -106,12 +110,27 @@ While we are primarily reachable on our [Discourse User Forum](https://discourse
## Breakdown of Features
### [Faster-than-light Engine](https://github.com/pi-hole/ftl)
[FTLDNS](https://github.com/pi-hole/ftl) is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
Some of the statistics you can integrate include:
- Total number of domains being blocked
- Total number of DNS queries today
- Total number of ads blocked today
- Percentage of ads blocked
- Unique domains
- Queries forwarded (to your chosen upstream DNS server)
- Queries cached
- Unique clients
The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).
### The Command Line Interface
The [pihole](https://docs.pi-hole.net/core/pihole-command/) command has all the functionality necessary to be able to fully administer the Pi-hole, without the need of the Web Interface. It's fast, user-friendly, and auditable by anyone with an understanding of `bash`.
![Pi-hole Blacklist Demo](https://pi-hole.github.io/graphics/Screenshots/blacklist-cli.gif)
Some notable features include:
- [Whitelisting, Blacklisting and Regex](https://docs.pi-hole.net/core/pihole-command/#whitelisting-blacklisting-and-regex)
@@ -128,8 +147,6 @@ You can read our [Core Feature Breakdown](https://docs.pi-hole.net/core/pihole-c
This [optional dashboard](https://github.com/pi-hole/AdminLTE) allows you to view stats, change settings, and configure your Pi-hole. It's the power of the Command Line Interface, with none of the learning curve!
![Pi-hole Dashboard](https://pi-hole.github.io/graphics/Screenshots/pihole-dashboard.png)
Some notable features include:
- Mobile friendly interface
@@ -145,21 +162,3 @@ There are several ways to [access the dashboard](https://discourse.pi-hole.net/t
1. `http://pi.hole/admin/` (when using Pi-hole as your DNS server)
2. `http://<IP_ADDPRESS_OF_YOUR_PI_HOLE>/admin/`
3. `http://pi.hole/` (when using Pi-hole as your DNS server)
## Faster-than-light Engine
FTLDNS is a lightweight, purpose-built daemon used to provide statistics needed for the Web Interface, and its API can be easily integrated into your own projects. As the name implies, FTLDNS does this all *very quickly*!
Some of the statistics you can integrate include:
- Total number of domains being blocked
- Total number of DNS queries today
- Total number of ads blocked today
- Percentage of ads blocked
- Unique domains
- Queries forwarded (to your chosen upstream DNS server)
- Queries cached
- Unique clients
The API can be accessed via [`telnet`](https://github.com/pi-hole/FTL), the Web (`admin/api.php`) and Command Line (`pihole -c -j`). You can find out [more details over here](https://discourse.pi-hole.net/t/pi-hole-api/1863).

View File

@@ -13,6 +13,7 @@ LC_NUMERIC=C
# Retrieve stats from FTL engine
pihole-FTL() {
local ftl_port LINE
ftl_port=$(cat /run/pihole-FTL.port 2> /dev/null)
if [[ -n "$ftl_port" ]]; then
# Open connection to FTL
@@ -20,12 +21,13 @@ pihole-FTL() {
# Test if connection is open
if { "true" >&3; } 2> /dev/null; then
# Send command to FTL
echo -e ">$1" >&3
# Send command to FTL and ask to quit when finished
echo -e ">$1 >quit" >&3
# Read input
# Read input until we received an empty string and the connection is
# closed
read -r -t 1 LINE <&3
until [[ ! $? ]] || [[ "$LINE" == *"EOM"* ]]; do
until [[ -z "${LINE}" ]] && [[ ! -t 3 ]]; do
echo "$LINE" >&1
read -r -t 1 LINE <&3
done
@@ -557,7 +559,7 @@ Calculates stats and displays to an LCD
Options:
-j, --json Output stats as JSON formatted string
-r, --refresh Set update frequency (in seconds)
-e, --exit Output stats and exit witout refreshing
-e, --exit Output stats and exit without refreshing
-h, --help Display this help text"
fi

View File

@@ -111,9 +111,15 @@ upgrade_gravityDB(){
version=12
fi
if [[ "$version" == "12" ]]; then
# Add column date_updated to alist table
# Add column date_updated to adlist table
echo -e " ${INFO} Upgrading gravity database from version 12 to 13"
sqlite3 "${database}" < "${scriptPath}/12_to_13.sql"
version=13
fi
if [[ "$version" == "13" ]]; then
# Add columns number and status to adlist table
echo -e " ${INFO} Upgrading gravity database from version 13 to 14"
sqlite3 "${database}" < "${scriptPath}/13_to_14.sql"
version=14
fi
}

View File

@@ -0,0 +1,13 @@
.timeout 30000
PRAGMA FOREIGN_KEYS=OFF;
BEGIN TRANSACTION;
ALTER TABLE adlist ADD COLUMN number INTEGER NOT NULL DEFAULT 0;
ALTER TABLE adlist ADD COLUMN invalid_domains INTEGER NOT NULL DEFAULT 0;
ALTER TABLE adlist ADD COLUMN status INTEGER NOT NULL DEFAULT 0;
UPDATE info SET value = 14 WHERE property = 'version';
COMMIT;

View File

@@ -1,4 +1,6 @@
#!/usr/bin/env bash
# shellcheck disable=SC1090
# Pi-hole: A black hole for Internet advertisements
# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
# Network-wide ad blocking via your own hardware.
@@ -9,9 +11,17 @@
# Please see LICENSE file for your rights under this license.
# Globals
basename=pihole
piholeDir=/etc/"${basename}"
gravityDBfile="${piholeDir}/gravity.db"
piholeDir="/etc/pihole"
GRAVITYDB="${piholeDir}/gravity.db"
# Source pihole-FTL from install script
pihole_FTL="${piholeDir}/pihole-FTL.conf"
if [[ -f "${pihole_FTL}" ]]; then
source "${pihole_FTL}"
fi
# Set this only after sourcing pihole-FTL.conf as the gravity database path may
# have changed
gravityDBfile="${GRAVITYDB}"
reload=false
addmode=true
@@ -112,7 +122,7 @@ ProcessDomainList() {
for dom in "${domList[@]}"; do
# Format domain into regex filter if requested
if [[ "${wildcard}" == true ]]; then
dom="(^|\\.)${dom//\./\\.}$"
dom="(\\.|^)${dom//\./\\.}$"
fi
# Logic: If addmode then add to desired list and remove from the other;

View File

@@ -38,7 +38,7 @@ flushARP(){
# Truncate network_addresses table in pihole-FTL.db
# This needs to be done before we can truncate the network table due to
# foreign key contraints
# foreign key constraints
if ! output=$(sqlite3 "${DBFILE}" "DELETE FROM network_addresses" 2>&1); then
echo -e "${OVER} ${CROSS} Failed to truncate network_addresses table"
echo " Database location: ${DBFILE}"

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Pi-hole: A black hole for Internet advertisements
# (c) 2017 Pi-hole, LLC (https://pi-hole.net)
# (c) 2021 Pi-hole (https://pi-hole.net)
# Network-wide ad blocking via your own hardware.
#
# Generates pihole_debug.log to be used for troubleshooting.
@@ -198,6 +198,33 @@ show_disclaimer(){
log_write "${DISCLAIMER}"
}
check_for_ftl(){
echo_current_diagnostic "Checking for pihole-FTL binary"
declare -g FTL_PATH
read -r FTL_PATH < <(which pihole-FTL)
if [ -z "${FTL_PATH}" ]; then
log_write "${CROSS} ${COL_RED} Unable to find pihole-FTL binary.${COL_NC}"
# Non-zero return value
return 2
else
log_write "${TICK} pihole-FTL: ${COL_GREEN}${FTL_PATH}${COL_NC}"
fi
}
check_for_lua(){
echo_current_diagnostic "Checking for lua capabilities"
if ! (${FTL_PATH} lua -v &>/dev/null); then
log_write "${CROSS} ${COL_RED} pihole-FTL binary does not have lua capabilites.${COL_NC}"
# Non-zero return value
return
else
log_write "${TICK} pihole-FTL: ${COL_GREEN}lua found!${COL_NC}"
fi
}
source_setup_variables() {
# Display the current test that is running
log_write "\\n${COL_PURPLE}*** [ INITIALIZING ]${COL_NC} Sourcing setup variables"
@@ -244,7 +271,7 @@ initialize_debug() {
log_write "${INFO} $(date "+%Y-%m-%d:%H:%M:%S") debug log has been initialized."
}
# This is a function for visually displaying the curent test that is being run.
# This is a function for visually displaying the current test that is being run.
# Accepts one variable: the name of what is being diagnosed
# Colors do not show in the dasboard, but the icons do: [i], [✓], and [✗]
echo_current_diagnostic() {
@@ -379,7 +406,7 @@ get_program_version() {
# Create a local variable so this function can be safely reused
local program_version
echo_current_diagnostic "${program_name} version"
# Evalutate the program we are checking, if it is any of the ones below, show the version
# Evaluate the program we are checking, if it is any of the ones below, show the version
case "${program_name}" in
"lighttpd") program_version="$(${program_name} -v 2> /dev/null | head -n1 | cut -d '/' -f2 | cut -d ' ' -f1)"
;;
@@ -641,7 +668,7 @@ detect_ip_addresses() {
# First argument should be a 4 or a 6
local protocol=${1}
# Use ip to show the addresses for the chosen protocol
# Store the values in an arry so they can be looped through
# Store the values in an array so they can be looped through
# Get the lines that are in the file(s) and store them in an array for parsing later
mapfile -t ip_addr_list < <(ip -"${protocol}" addr show dev "${PIHOLE_INTERFACE}" | awk -F ' ' '{ for(i=1;i<=NF;i++) if ($i ~ '/^inet/') print $(i+1) }')
@@ -695,7 +722,7 @@ ping_gateway() {
# Check if we are using IPv4 or IPv6
# Find the default gateway using IPv4 or IPv6
local gateway
gateway="$(ip -"${protocol}" route | grep default | cut -d ' ' -f 3)"
gateway="$(ip -"${protocol}" route | grep default | grep "${PIHOLE_INTERFACE}" | cut -d ' ' -f 3)"
# If the gateway variable has a value (meaning a gateway was found),
if [[ -n "${gateway}" ]]; then
@@ -823,7 +850,7 @@ check_x_headers() {
# Do it for the dashboard as well, as the header is different than above
local dashboard
dashboard=$(curl -Is localhost/admin/ | awk '/X-Pi-hole/' | tr -d '\r')
# Store what the X-Header shoud be in variables for comparison later
# Store what the X-Header should be in variables for comparison later
local block_page_working
block_page_working="X-Pi-hole: A black hole for Internet advertisements."
local dashboard_working
@@ -842,12 +869,12 @@ check_x_headers() {
log_write "${COL_RED}${full_curl_output_block_page}${COL_NC}"
fi
# Same logic applies to the dashbord as above, if the X-Header matches what a working system shoud have,
# Same logic applies to the dashboard as above, if the X-Header matches what a working system should have,
if [[ $dashboard == "$dashboard_working" ]]; then
# then we can show a success
log_write "$TICK Web interface X-Header: ${COL_GREEN}${dashboard}${COL_NC}"
else
# Othewise, it's a failure since the X-Headers either don't exist or have been modified in some way
# Otherwise, it's a failure since the X-Headers either don't exist or have been modified in some way
log_write "$CROSS Web interface X-Header: ${COL_RED}X-Header does not match or could not be retrieved.${COL_NC}"
log_write "${COL_RED}${full_curl_output_dashboard}${COL_NC}"
fi
@@ -877,7 +904,7 @@ dig_at() {
local pihole_address="${IP}"
local remote_address="2001:4860:4860::8888"
local record_type="AAAA"
# Othwerwise, it should be 4
# Otherwise, it should be 4
else
# so use the IPv4 values
local local_address="127.0.0.1"
@@ -911,7 +938,7 @@ dig_at() {
# show a success
log_write "${TICK} ${random_url} ${COL_GREEN}is ${pihole_dig}${COL_NC} via ${COL_CYAN}Pi-hole${COL_NC} (${pihole_address})"
else
# Othewise, show a failure
# Otherwise, show a failure
log_write "${CROSS} ${COL_RED}Failed to resolve${COL_NC} ${random_url} via ${COL_RED}Pi-hole${COL_NC} (${pihole_address})"
fi
@@ -1044,7 +1071,7 @@ parse_file() {
}
check_name_resolution() {
# Check name resoltion from localhost, Pi-hole's IP, and Google's name severs
# Check name resolution from localhost, Pi-hole's IP, and Google's name severs
# using the function we created earlier
dig_at 4 "${IPV4_ADDRESS%/*}"
# If IPv6 enabled,
@@ -1421,28 +1448,40 @@ make_temporary_log
initialize_debug
# setupVars.conf needs to be sourced before the networking so the values are
# available to the other functions
source_setup_variables
check_component_versions
check_critical_program_versions
diagnose_operating_system
check_selinux
check_firewalld
processor_check
check_networking
check_name_resolution
check_dhcp_servers
process_status
ftl_full_status
parse_setup_vars
check_x_headers
analyze_gravity_list
show_groups
show_domainlist
show_clients
show_adlists
show_content_of_pihole_files
show_messages
parse_locale
analyze_pihole_log
copy_to_debug_log
upload_to_tricorder
if ! check_for_ftl; then
log_write "${COL_RED}Unable to complete debug run. Please contact support for assistance."
log_write "Please note the error that is displayed above.${COL_NC}"
#Non-zero return value
exit 2
fi
if ! check_for_lua; then
log_write "${COL_RED}Unable to complete debug run. Please contact support for assistance."
log_write "Please note the error that is displayed above.${COL_NC}"
#Non-zero return value
exit 2
fi
# source_setup_variables
# check_component_versions
# check_critical_program_versions
# diagnose_operating_system
# check_selinux
# check_firewalld
# processor_check
# check_networking
# check_name_resolution
# check_dhcp_servers
# process_status
# ftl_full_status
# parse_setup_vars
# check_x_headers
# analyze_gravity_list
# show_groups
# show_domainlist
# show_clients
# show_adlists
# show_content_of_pihole_files
# show_messages
# parse_locale
# analyze_pihole_log
# copy_to_debug_log
# upload_to_tricorder

View File

@@ -1,5 +1,6 @@
#!/usr/bin/env bash
# shellcheck disable=SC1090
# Pi-hole: A black hole for Internet advertisements
# (c) 2018 Pi-hole, LLC (https://pi-hole.net)
# Network-wide ad blocking via your own hardware.
@@ -11,12 +12,21 @@
# Globals
piholeDir="/etc/pihole"
gravityDBfile="${piholeDir}/gravity.db"
GRAVITYDB="${piholeDir}/gravity.db"
options="$*"
all=""
exact=""
blockpage=""
matchType="match"
# Source pihole-FTL from install script
pihole_FTL="${piholeDir}/pihole-FTL.conf"
if [[ -f "${pihole_FTL}" ]]; then
source "${pihole_FTL}"
fi
# Set this only after sourcing pihole-FTL.conf as the gravity database path may
# have changed
gravityDBfile="${GRAVITYDB}"
colfile="/opt/pihole/COL_TABLE"
source "${colfile}"

View File

@@ -153,7 +153,7 @@ versionOutput() {
if [[ -n "$current" ]] && [[ -n "$latest" ]]; then
output="${1^} version is $branch$current (Latest: $latest)"
elif [[ -n "$current" ]] && [[ -z "$latest" ]]; then
output="Current ${1^} version is $branch$current."
output="Current ${1^} version is $branch$current"
elif [[ -z "$current" ]] && [[ -n "$latest" ]]; then
output="Latest ${1^} version is $latest"
elif [[ "$curHash" == "N/A" ]] || [[ "$latHash" == "N/A" ]]; then

View File

@@ -238,18 +238,18 @@ trust-anchor=.,20326,8,2,E06D44B80B8F1D39A95C0B0D7C65D08458E880409BBC68345710423
# 168.192.in-addr.arpa to 192.168.0.0/16
# 192.in-addr.arpa to 192.0.0.0/8
if [[ "${CONDITIONAL_FORWARDING_REVERSE}" == *"in-addr.arpa" ]];then
arrRev=("${CONDITIONAL_FORWARDING_REVERSE//./ }")
case ${#arrRev[@]} in
arrRev=("${CONDITIONAL_FORWARDING_REVERSE//./ }")
case ${#arrRev[@]} in
6 ) REV_SERVER_CIDR="${arrRev[3]}.${arrRev[2]}.${arrRev[1]}.${arrRev[0]}/32";;
5 ) REV_SERVER_CIDR="${arrRev[2]}.${arrRev[1]}.${arrRev[0]}.0/24";;
4 ) REV_SERVER_CIDR="${arrRev[1]}.${arrRev[0]}.0.0/16";;
3 ) REV_SERVER_CIDR="${arrRev[0]}.0.0.0/8";;
3 ) REV_SERVER_CIDR="${arrRev[0]}.0.0.0/8";;
esac
else
# Set REV_SERVER_CIDR to whatever value it was set to
REV_SERVER_CIDR="${CONDITIONAL_FORWARDING_REVERSE}"
fi
# If REV_SERVER_CIDR is not converted by the above, then use the REV_SERVER_TARGET variable to derive it
if [ -z "${REV_SERVER_CIDR}" ]; then
# Convert existing input to /24 subnet (preserves legacy behavior)
@@ -486,10 +486,15 @@ SetWebUITheme() {
}
CheckUrl(){
local regex
local regex check_url
# Check for characters NOT allowed in URLs
regex="[^a-zA-Z0-9:/?&%=~._-]"
if [[ "${1}" =~ ${regex} ]]; then
regex="[^a-zA-Z0-9:/?&%=~._()-;]"
# this will remove first @ that is after schema and before domain
# \1 is optional schema, \2 is userinfo
check_url="$( sed -re 's#([^:/]*://)?([^/]+)@#\1\2#' <<< "$1" )"
if [[ "${check_url}" =~ ${regex} ]]; then
return 1
else
return 0
@@ -559,7 +564,13 @@ AddDHCPStaticAddress() {
RemoveDHCPStaticAddress() {
mac="${args[2]}"
sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
if [[ "$mac" =~ ^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$ ]]; then
sed -i "/dhcp-host=${mac}.*/d" "${dhcpstaticconfig}"
else
echo " ${CROSS} Invalid Mac Passed!"
exit 1
fi
}
SetAdminEmail() {
@@ -631,8 +642,11 @@ Interfaces:
Teleporter() {
local datetimestamp
local host
datetimestamp=$(date "+%Y-%m-%d_%H-%M-%S")
php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-teleporter_${datetimestamp}.tar.gz"
host=$(hostname)
host="${host//./_}"
php /var/www/html/admin/scripts/pi-hole/php/teleporter.php > "pi-hole-${host:-noname}-teleporter_${datetimestamp}.tar.gz"
}
checkDomain()
@@ -700,7 +714,13 @@ RemoveCustomDNSAddress() {
ip="${args[2]}"
host="${args[3]}"
sed -i "/${ip} ${host}/d" "${dnscustomfile}"
if valid_ip "${ip}" || valid_ip6 "${ip}" ; then
sed -i "/${ip} ${host}/d" "${dnscustomfile}"
else
echo -e " ${CROSS} Invalid IP has been passed"
exit 1
fi
# Restart dnsmasq to update removed custom DNS entries
RestartDNS
@@ -711,6 +731,7 @@ AddCustomCNAMERecord() {
domain="${args[2]}"
target="${args[3]}"
echo "cname=${domain},${target}" >> "${dnscustomcnamefile}"
# Restart dnsmasq to load new custom CNAME records
@@ -722,7 +743,20 @@ RemoveCustomCNAMERecord() {
domain="${args[2]}"
target="${args[3]}"
sed -i "/cname=${domain},${target}/d" "${dnscustomcnamefile}"
validDomain="$(checkDomain "${domain}")"
if [[ -n "${validDomain}" ]]; then
validTarget="$(checkDomain "${target}")"
if [[ -n "${validDomain}" ]]; then
sed -i "/cname=${validDomain},${validTarget}/d" "${dnscustomcnamefile}"
else
echo " ${CROSS} Invalid Target Passed!"
exit 1
fi
else
echo " ${CROSS} Invalid Domain passed!"
exit 1
fi
# Restart dnsmasq to update removed custom CNAME records
RestartDNS

View File

@@ -32,7 +32,10 @@ CREATE TABLE adlist
date_added INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
date_modified INTEGER NOT NULL DEFAULT (cast(strftime('%s', 'now') as int)),
comment TEXT,
date_updated INTEGER
date_updated INTEGER,
number INTEGER NOT NULL DEFAULT 0,
invalid_domains INTEGER NOT NULL DEFAULT 0,
status INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE adlist_by_group
@@ -54,7 +57,7 @@ CREATE TABLE info
value TEXT NOT NULL
);
INSERT INTO "info" VALUES('version','13');
INSERT INTO "info" VALUES('version','14');
CREATE TABLE domain_audit
(

View File

@@ -145,7 +145,17 @@ body {
}
/* User is greeted with a splash page when browsing to Pi-hole IP address */
#splashpage { background: #222; color: rgba(255, 255, 255, 0.7); text-align: center; }
#splashpage {
background: #222;
color: rgba(255, 255, 255, 0.7);
text-align: center;
width: 100%;
height: 100%;
display: flex;
align-items: center;
justify-content: center;
}
#splashpage img { margin: 5px; width: 256px; }
#splashpage b { color: inherit; }
@@ -196,6 +206,26 @@ header #bpAlt label {
display: block;
}
html, body {
height: 100%;
}
#pihole_card {
width: 400px;
height: auto;
max-width: 400px;
}
#pihole_card p, #pihole_card a {
font-size: 13pt;
text-align: center;
}
#pihole_logo_splash {
height: auto;
width: 100%;
}
/* Click anywhere else on screen to hide #bpAbout */
#bpAboutToggle:checked {
display: block;
@@ -382,12 +412,44 @@ footer {
/* Responsive Content */
@media only screen and (max-width: 500px) {
h1 a { font-size: 1.8rem; min-width: 170px; }
footer span::before { content: "Generated "; }
footer span { display: block; }
h1 a {
font-size: 1.8rem;
min-width: 170px;
}
footer span::before {
content: "Generated ";
}
footer span {
display: block;
}
}
@media only screen and (min-width: 1251px) {
#bpWrapper, footer { border-radius: 0 0 5px 5px; }
#bpAbout { border-right-width: 1px; }
#bpWrapper, footer {
border-radius: 0 0 5px 5px;
}
#bpAbout {
border-right-width: 1px;
}
}
@media only screen and (max-width: 400px) {
#pihole_card {
width: 100%;
height: auto;
}
#pihole_card p, #pihole_card a {
font-size: 100%;
}
}
@media only screen and (max-width: 256px) {
#pihole_logo_splash {
width: 90% !important;
height: auto;
}
}

View File

@@ -58,14 +58,15 @@ if ($serverName === "pi.hole"
// When directly browsing via IP or authorized hostname
// Render splash/landing page based off presence of $landPage file
// Unset variables so as to not be included in $landPage or $splashPage
unset($serverName, $svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt, $viewPort);
unset($svPasswd, $svEmail, $authorizedHosts, $validExtTypes, $currentUrlExt);
// If $landPage file is present
if (is_file(getcwd()."/$landPage")) {
unset($serverName, $viewPort); // unset extra variables not to be included in $landpage
include $landPage;
exit();
}
// If $landPage file was not present, Set Splash Page output
$splashPage = "
$splashPage = <<<EOT
<!doctype html>
<html lang='en'>
<head>
@@ -76,13 +77,14 @@ if ($serverName === "pi.hole"
<link rel='shortcut icon' href='admin/img/favicons/favicon.ico' type='image/x-icon'>
</head>
<body id='splashpage'>
<img src='admin/img/logo.svg' alt='Pi-hole logo' width='256' height='377'>
<br>
<p>Pi-<strong>hole</strong>: Your black hole for Internet advertisements</p>
<a href='/admin'>Did you mean to go to the admin panel?</a>
<div id="pihole_card">
<img src='admin/img/logo.svg' alt='Pi-hole logo' id="pihole_logo_splash" />
<p>Pi-<strong>hole</strong>: Your black hole for Internet advertisements</p>
<a href='/admin'>Did you mean to go to the admin panel?</a>
</div>
</body>
</html>
";
EOT;
exit($splashPage);
} elseif ($currentUrlExt === "js") {
// Serve Pi-hole JavaScript for blocked domains requesting JS

View File

@@ -81,6 +81,10 @@ mimetype.assign = (
".woff2" => "font/woff2"
)
# Add user chosen options held in external file
# This uses include_shell instead of an include wildcard for compatibility
include_shell "cat external.conf 2>/dev/null"
# default listening port for IPv6 falls back to the IPv4 port
include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
@@ -109,7 +113,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {
# Default expire header
expire.url = ( "" => "access plus 0 seconds" )
# Add user chosen options held in external file
# This uses include_shell instead of an include wildcard for compatibility
include_shell "cat external.conf 2>/dev/null"

View File

@@ -82,6 +82,10 @@ mimetype.assign = (
".woff2" => "font/woff2"
)
# Add user chosen options held in external file
# This uses include_shell instead of an include wildcard for compatibility
include_shell "cat external.conf 2>/dev/null"
# default listening port for IPv6 falls back to the IPv4 port
#include_shell "/usr/share/lighttpd/use-ipv6.pl " + server.port
#include_shell "/usr/share/lighttpd/create-mime.assign.pl"
@@ -117,7 +121,3 @@ $HTTP["url"] =~ "^/admin/\.(.*)" {
# Default expire header
expire.url = ( "" => "access plus 0 seconds" )
# Add user chosen options held in external file
# This uses include_shell instead of an include wildcard for compatibility
include_shell "cat external.conf 2>/dev/null"

File diff suppressed because it is too large Load Diff

View File

@@ -31,7 +31,7 @@ else
else
echo -e " ${CROSS} ${str}
Script called with non-root privileges
The Pi-hole requires elevated privleges to uninstall"
The Pi-hole requires elevated privileges to uninstall"
exit 1
fi
fi

View File

@@ -35,8 +35,9 @@ localList="${piholeDir}/local.list"
VPNList="/etc/openvpn/ipp.txt"
piholeGitDir="/etc/.pihole"
gravityDBfile="${piholeDir}/gravity.db"
gravityTEMPfile="${piholeDir}/gravity_temp.db"
gravityDBfile_default="${piholeDir}/gravity.db"
# GRAVITYDB may be overwritten by source pihole-FTL.conf below
GRAVITYDB="${gravityDBfile_default}"
gravityDBschema="${piholeGitDir}/advanced/Templates/gravity.db.sql"
gravityDBcopy="${piholeGitDir}/advanced/Templates/gravity_copy.sql"
@@ -68,6 +69,11 @@ if [[ -f "${pihole_FTL}" ]]; then
source "${pihole_FTL}"
fi
# Set this only after sourcing pihole-FTL.conf as the gravity database path may
# have changed
gravityDBfile="${GRAVITYDB}"
gravityTEMPfile="${GRAVITYDB}_temp"
if [[ -z "${BLOCKINGMODE}" ]] ; then
BLOCKINGMODE="NULL"
fi
@@ -84,11 +90,11 @@ generate_gravity_database() {
# Copy data from old to new database file and swap them
gravity_swap_databases() {
local str
local str copyGravity
str="Building tree"
echo -ne " ${INFO} ${str}..."
# The index is intentionally not UNIQUE as prro quality adlists may contain domains more than once
# The index is intentionally not UNIQUE as poor quality adlists may contain domains more than once
output=$( { sqlite3 "${gravityTEMPfile}" "CREATE INDEX idx_gravity ON gravity (domain, adlist_id);"; } 2>&1 )
status="$?"
@@ -101,7 +107,14 @@ gravity_swap_databases() {
str="Swapping databases"
echo -ne " ${INFO} ${str}..."
output=$( { sqlite3 "${gravityTEMPfile}" < "${gravityDBcopy}"; } 2>&1 )
# Gravity copying SQL script
copyGravity="$(cat "${gravityDBcopy}")"
if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
# Replace default gravity script location by custom location
copyGravity="${copyGravity//"${gravityDBfile_default}"/"${gravityDBfile}"}"
fi
output=$( { sqlite3 "${gravityTEMPfile}" <<< "${copyGravity}"; } 2>&1 )
status="$?"
if [[ "${status}" -ne 0 ]]; then
@@ -176,7 +189,7 @@ database_table_from_file() {
echo "${rowid},\"${domain}\",${timestamp}" >> "${tmpFile}"
elif [[ "${table}" == "adlist" ]]; then
# Adlist table format
echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"," >> "${tmpFile}"
echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\",,0,0,0" >> "${tmpFile}"
else
# White-, black-, and regexlist table format
echo "${rowid},${type},\"${domain}\",1,${timestamp},${timestamp},\"Migrated from ${source}\"" >> "${tmpFile}"
@@ -217,6 +230,48 @@ database_adlist_updated() {
fi
}
# Check if a column with name ${2} exists in gravity table with name ${1}
gravity_column_exists() {
output=$( { printf ".timeout 30000\\nSELECT EXISTS(SELECT * FROM pragma_table_info('%s') WHERE name='%s');\\n" "${1}" "${2}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
if [[ "${output}" == "1" ]]; then
return 0 # Bash 0 is success
fi
return 1 # Bash non-0 is failure
}
# Update number of domain on this list. We store this in the "old" database as all values in the new database will later be overwritten
database_adlist_number() {
# Only try to set number of domains when this field exists in the gravity database
if ! gravity_column_exists "adlist" "number"; then
return;
fi
output=$( { printf ".timeout 30000\\nUPDATE adlist SET number = %i, invalid_domains = %i WHERE id = %i;\\n" "${num_lines}" "${num_invalid}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
status="$?"
if [[ "${status}" -ne 0 ]]; then
echo -e "\\n ${CROSS} Unable to update number of domains in adlist with ID ${1} in database ${gravityDBfile}\\n ${output}"
gravity_Cleanup "error"
fi
}
# Update status of this list. We store this in the "old" database as all values in the new database will later be overwritten
database_adlist_status() {
# Only try to set the status when this field exists in the gravity database
if ! gravity_column_exists "adlist" "status"; then
return;
fi
output=$( { printf ".timeout 30000\\nUPDATE adlist SET status = %i WHERE id = %i;\\n" "${2}" "${1}" | sqlite3 "${gravityDBfile}"; } 2>&1 )
status="$?"
if [[ "${status}" -ne 0 ]]; then
echo -e "\\n ${CROSS} Unable to update status of adlist with ID ${1} in database ${gravityDBfile}\\n ${output}"
gravity_Cleanup "error"
fi
}
# Migrate pre-v5.0 list files to database-based Pi-hole versions
migrate_to_database() {
# Create database file only if not present
@@ -317,6 +372,10 @@ gravity_CheckDNSResolutionAvailable() {
gravity_DownloadBlocklists() {
echo -e " ${INFO} ${COL_BOLD}Neutrino emissions detected${COL_NC}..."
if [[ "${gravityDBfile}" != "${gravityDBfile_default}" ]]; then
echo -e " ${INFO} Storing gravity database in ${COL_BOLD}${gravityDBfile}${COL_NC}"
fi
# Retrieve source URLs from gravity database
# We source only enabled adlists, sqlite3 stores boolean values as 0 (false) or 1 (true)
mapfile -t sources <<< "$(sqlite3 "${gravityDBfile}" "SELECT address FROM vw_adlist;" 2> /dev/null)"
@@ -363,7 +422,7 @@ gravity_DownloadBlocklists() {
target="$(mktemp -p "/tmp" --suffix=".gravity")"
# Use compression to reduce the amount of data that is transfered
# Use compression to reduce the amount of data that is transferred
# between the Pi-hole and the ad list provider. Use this feature
# only if it is supported by the locally available version of curl
if curl -V | grep -q "Features:.* libz"; then
@@ -393,10 +452,15 @@ gravity_DownloadBlocklists() {
esac
echo -e " ${INFO} Target: ${url}"
local regex
local regex check_url
# Check for characters NOT allowed in URLs
regex="[^a-zA-Z0-9:/?&%=~._()-;]"
if [[ "${url}" =~ ${regex} ]]; then
# this will remove first @ that is after schema and before domain
# \1 is optional schema, \2 is userinfo
check_url="$( sed -re 's#([^:/]*://)?([^/]+)@#\1\2#' <<< "$url" )"
if [[ "${check_url}" =~ ${regex} ]]; then
echo -e " ${CROSS} Invalid Target"
else
gravity_DownloadBlocklistFromUrl "${url}" "${cmd_ext}" "${agent}" "${sourceIDs[$i]}" "${saveLocation}" "${target}" "${compression}"
@@ -439,6 +503,8 @@ gravity_DownloadBlocklists() {
}
total_num=0
num_lines=0
num_invalid=0
parseList() {
local adlistID="${1}" src="${2}" target="${3}" incorrect_lines
# This sed does the following things:
@@ -449,7 +515,7 @@ parseList() {
# Find (up to) five domains containing invalid characters (see above)
incorrect_lines="$(sed -e "/[^a-zA-Z0-9.\_-]/!d" "${src}" | head -n 5)"
local num_lines num_target_lines num_correct_lines num_invalid
local num_target_lines num_correct_lines num_invalid
# Get number of lines in source file
num_lines="$(grep -c "^" "${src}")"
# Get number of lines in destination file
@@ -458,9 +524,9 @@ parseList() {
total_num="$num_target_lines"
num_invalid="$(( num_lines-num_correct_lines ))"
if [[ "${num_invalid}" -eq 0 ]]; then
echo " ${INFO} Received ${num_lines} domains"
echo " ${INFO} Analyzed ${num_lines} domains"
else
echo " ${INFO} Received ${num_lines} domains, ${num_invalid} domains invalid!"
echo " ${INFO} Analyzed ${num_lines} domains, ${num_invalid} domains invalid!"
fi
# Display sample of invalid lines if we found some
@@ -471,6 +537,29 @@ parseList() {
done <<< "${incorrect_lines}"
fi
}
compareLists() {
local adlistID="${1}" target="${2}"
# Verify checksum when an older checksum exists
if [[ -s "${target}.sha1" ]]; then
if ! sha1sum --check --status --strict "${target}.sha1"; then
# The list changed upstream, we need to update the checksum
sha1sum "${target}" > "${target}.sha1"
echo " ${INFO} List has been updated"
database_adlist_status "${adlistID}" "1"
database_adlist_updated "${adlistID}"
else
echo " ${INFO} List stayed unchanged"
database_adlist_status "${adlistID}" "2"
fi
else
# No checksum available, create one for comparing on the next run
sha1sum "${target}" > "${target}.sha1"
# We assume here it was changed upstream
database_adlist_status "${adlistID}" "1"
database_adlist_updated "${adlistID}"
fi
}
# Download specified URL and perform checks on HTTP status and file content
gravity_DownloadBlocklistFromUrl() {
@@ -554,31 +643,49 @@ gravity_DownloadBlocklistFromUrl() {
esac;;
esac
local done="false"
# Determine if the blocklist was downloaded and saved correctly
if [[ "${success}" == true ]]; then
if [[ "${httpCode}" == "304" ]]; then
# Add domains to database table file
parseList "${adlistID}" "${saveLocation}" "${target}"
database_adlist_status "${adlistID}" "2"
database_adlist_number "${adlistID}"
done="true"
# Check if $patternbuffer is a non-zero length file
elif [[ -s "${patternBuffer}" ]]; then
# Determine if blocklist is non-standard and parse as appropriate
gravity_ParseFileIntoDomains "${patternBuffer}" "${saveLocation}"
# Add domains to database table file
parseList "${adlistID}" "${saveLocation}" "${target}"
# Update date_updated field in gravity database table
database_adlist_updated "${adlistID}"
# Compare lists, are they identical?
compareLists "${adlistID}" "${saveLocation}"
# Update gravity database table (status and updated timestamp are set in
# compareLists)
database_adlist_number "${adlistID}"
done="true"
else
# Fall back to previously cached list if $patternBuffer is empty
echo -e " ${INFO} Received empty file: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
echo -e " ${INFO} Received empty file"
fi
else
fi
# Do we need to fall back to a cached list (if available)?
if [[ "${done}" != "true" ]]; then
# Determine if cached list has read permission
if [[ -r "${saveLocation}" ]]; then
echo -e " ${CROSS} List download failed: ${COL_LIGHT_GREEN}using previously cached list${COL_NC}"
# Add domains to database table file
parseList "${adlistID}" "${saveLocation}" "${target}"
database_adlist_number "${adlistID}"
database_adlist_status "${adlistID}" "3"
else
echo -e " ${CROSS} List download failed: ${COL_LIGHT_RED}no cached list available${COL_NC}"
# Manually reset these two numbers because we do not call parseList here
num_lines=0
num_invalid=0
database_adlist_number "${adlistID}"
database_adlist_status "${adlistID}" "4"
fi
fi
}
@@ -590,7 +697,7 @@ gravity_ParseFileIntoDomains() {
# Determine if we are parsing a consolidated list
#if [[ "${source}" == "${piholeDir}/${matterAndLight}" ]]; then
# Remove comments and print only the domain name
# Most of the lists downloaded are already in hosts file format but the spacing/formating is not contiguous
# Most of the lists downloaded are already in hosts file format but the spacing/formatting is not contiguous
# This helps with that and makes it easier to read
# It also helps with debugging so each stage of the script can be researched more in depth
# 1) Remove carriage returns

View File

@@ -153,7 +153,7 @@ Available commands and options:
.br
-r, --refresh Set update frequency (in seconds)
.br
-e, --exit Output stats and exit witout refreshing
-e, --exit Output stats and exit without refreshing
.br
\fB-g, updateGravity\fR

47
pihole
View File

@@ -16,6 +16,7 @@ readonly PI_HOLE_SCRIPT_DIR="/opt/pihole"
# error due to modifying a readonly variable.
setupVars="/etc/pihole/setupVars.conf"
PI_HOLE_BIN_DIR="/usr/local/bin"
readonly FTL_PID_FILE="/run/pihole-FTL.pid"
readonly colfile="${PI_HOLE_SCRIPT_DIR}/COL_TABLE"
source "${colfile}"
@@ -98,8 +99,25 @@ versionFunc() {
exit 0
}
# Get PID of main pihole-FTL process
getFTLPID() {
local pid
if [ -s "${FTL_PID_FILE}" ]; then
# -s: FILE exists and has a size greater than zero
pid="$(<"$FTL_PID_FILE")"
# Exploit prevention: unset the variable if there is malicious content
# Verify that the value read from the file is numeric
[[ "$pid" =~ [^[:digit:]] ]] && unset pid
fi
# If FTL is not running, or the PID file contains malicious stuff, substitute
# negative PID to signal this to the caller
echo "${pid:=-1}"
}
restartDNS() {
local svcOption svc str output status
local svcOption svc str output status pid icon
svcOption="${1:-restart}"
# Determine if we should reload or restart
@@ -108,17 +126,34 @@ restartDNS() {
# Note 1: This will NOT re-read any *.conf files
# Note 2: We cannot use killall here as it does
# not know about real-time signals
svc="pkill -RTMIN pihole-FTL"
str="Reloading DNS lists"
pid="$(getFTLPID)"
if [[ "$pid" -eq "-1" ]]; then
svc="true"
str="FTL is not running"
icon="${INFO}"
else
svc="kill -RTMIN ${pid}"
str="Reloading DNS lists"
icon="${TICK}"
fi
elif [[ "${svcOption}" =~ "reload" ]]; then
# Reloading of the DNS cache has been requested
# Note: This will NOT re-read any *.conf files
svc="pkill -HUP pihole-FTL"
str="Flushing DNS cache"
pid="$(getFTLPID)"
if [[ "$pid" -eq "-1" ]]; then
svc="true"
str="FTL is not running"
icon="${INFO}"
else
svc="kill -HUP ${pid}"
str="Flushing DNS cache"
icon="${TICK}"
fi
else
# A full restart has been requested
svc="service pihole-FTL restart"
str="Restarting DNS server"
icon="${TICK}"
fi
# Print output to Terminal, but not to Web Admin
@@ -128,7 +163,7 @@ restartDNS() {
status="$?"
if [[ "${status}" -eq 0 ]]; then
[[ -t 1 ]] && echo -e "${OVER} ${TICK} ${str}"
[[ -t 1 ]] && echo -e "${OVER} ${icon} ${str}"
return 0
else
[[ ! -t 1 ]] && local OVER=""

View File

@@ -1,5 +1,5 @@
Raspbian=9,10
Ubuntu=16,18,20
Debian=9,10
Fedora=31,32
Fedora=32,33
CentOS=7,8

View File

@@ -1,4 +1,4 @@
FROM fedora:31
FROM fedora:33
ENV GITDIR /etc/.pihole
ENV SCRIPTDIR /opt/pihole

View File

@@ -524,43 +524,45 @@ def test_IPv6_ULA_GUA_test(Pihole):
assert expected_stdout in detectPlatform.stdout
def test_validate_ip_valid(Pihole):
def test_validate_ip(Pihole):
'''
Given a valid IP address, valid_ip returns success
Tests valid_ip for various IP addresses
'''
output = Pihole.run('''
source /opt/pihole/basic-install.sh
valid_ip "192.168.1.1"
''')
def test_address(addr, success=True):
output = Pihole.run('''
source /opt/pihole/basic-install.sh
valid_ip "{addr}"
'''.format(addr=addr))
assert output.rc == 0
assert output.rc == 0 if success else 1
def test_validate_ip_invalid_octet(Pihole):
'''
Given an invalid IP address (large octet), valid_ip returns an error
'''
output = Pihole.run('''
source /opt/pihole/basic-install.sh
valid_ip "1092.168.1.1"
''')
assert output.rc == 1
def test_validate_ip_invalid_letters(Pihole):
'''
Given an invalid IP address (contains letters), valid_ip returns an error
'''
output = Pihole.run('''
source /opt/pihole/basic-install.sh
valid_ip "not an IP"
''')
assert output.rc == 1
test_address('192.168.1.1')
test_address('127.0.0.1')
test_address('255.255.255.255')
test_address('255.255.255.256', False)
test_address('255.255.256.255', False)
test_address('255.256.255.255', False)
test_address('256.255.255.255', False)
test_address('1092.168.1.1', False)
test_address('not an IP', False)
test_address('8.8.8.8#', False)
test_address('8.8.8.8#0')
test_address('8.8.8.8#1')
test_address('8.8.8.8#42')
test_address('8.8.8.8#888')
test_address('8.8.8.8#1337')
test_address('8.8.8.8#65535')
test_address('8.8.8.8#65536', False)
test_address('8.8.8.8#-1', False)
test_address('00.0.0.0', False)
test_address('010.0.0.0', False)
test_address('001.0.0.0', False)
test_address('0.0.0.0#00', False)
test_address('0.0.0.0#01', False)
test_address('0.0.0.0#001', False)
test_address('0.0.0.0#0001', False)
test_address('0.0.0.0#00001', False)
def test_os_check_fails(Pihole):

View File

@@ -4,5 +4,5 @@ envlist = py37
[testenv]
whitelist_externals = docker
deps = -rrequirements.txt
commands = docker build -f _fedora_31.Dockerfile -t pytest_pihole:test_container ../
commands = docker build -f _fedora_33.Dockerfile -t pytest_pihole:test_container ../
pytest {posargs:-vv -n auto} ./test_automated_install.py ./test_centos_fedora_common_support.py ./test_fedora_support.py