Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Portainer Agent #123

Open
wants to merge 103 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
a8d02a2
Added Portainer Agent
Willem-Dekker Jan 2, 2020
48b67d9
Update directoryfix.sh
drkauffman Jan 9, 2020
09351d6
Update directoryfix.sh
drkauffman Jan 9, 2020
a39d01c
Adds a Gitea container to IOTstack
Paraphraser Jan 12, 2020
eb9b982
Added filter.acl file to mosquitto/config
Tomasmerca Jan 15, 2020
c4adb55
Created empty file whit example filter
Tomasmerca Jan 15, 2020
89f4647
Added options for password and filter config
Tomasmerca Jan 15, 2020
72e6345
Update menu.sh
RavHilton Feb 6, 2020
818063f
adding deconz docker compose setup
DIYtechie Feb 28, 2020
6461ed1
deconz added to menu
DIYtechie Feb 28, 2020
413c0ae
Update service.yml
DIYtechie Mar 2, 2020
050fdb1
deconz added
DIYtechie Mar 2, 2020
614e56e
added node-red-contrib-deconz
DIYtechie Mar 2, 2020
cb3e1b0
added transmission container
gpongelli Mar 22, 2020
7751ed0
add volumes management from template
gpongelli Mar 22, 2020
da5a549
update menu.sh
peyanski Apr 4, 2020
d5b991b
small update to container name
Willem-Dekker Apr 19, 2020
b068114
first import
Willem-Dekker Apr 19, 2020
d4a9cd1
Merge pull request #1 from Willem-Dekker/mkdocks
Willem-Dekker Apr 19, 2020
f6571ae
added index.md
Willem-Dekker Apr 19, 2020
55542d4
added index.md (#2)
Willem-Dekker Apr 19, 2020
96454c9
Updated styling of existing pages
Willem-Dekker Apr 19, 2020
171283a
Merge pull request #3 from Willem-Dekker/mkdocks
Willem-Dekker Apr 19, 2020
7bf68e7
Updated styling of existing pages
Willem-Dekker Apr 19, 2020
1646a7a
Merge pull request #20 from Willem-Dekker/mkdocks
SensorsIot Apr 19, 2020
a9afc96
Update README.md
SensorsIot Apr 19, 2020
50cbf52
Update README.md
SensorsIot Apr 19, 2020
0b2385b
Merge pull request #16 from SensorsIot/pr/161
Willem-Dekker Apr 19, 2020
5d0c85a
Changed each menu item to be on a separate line to make merges easier.
Apr 19, 2020
4f8a47d
Merge branch 'armhf_keys_menu_lines' into pr/126
Apr 20, 2020
52ddbcb
Changed docker-compose yml file version to 3.6.
Apr 20, 2020
d464d0f
Merge pull request #26 from SensorsIot/armhf_keys_menu_lines
Willem-Dekker Apr 20, 2020
65e48e0
Merge pull request #7 from SensorsIot/pr/139
Willem-Dekker Apr 20, 2020
67e0056
Merge pull request #3 from SensorsIot/pr/126
Willem-Dekker Apr 20, 2020
15d840b
Update README.md
Slyke Apr 20, 2020
a3078f2
Merge pull request #27 from SensorsIot/readme-update
Willem-Dekker Apr 20, 2020
d86a5cd
Merge pull request #4 from SensorsIot/pr/130
Willem-Dekker Apr 20, 2020
a7d37c0
Fixes some inconsistencies and makes other tweaks to index page
Paraphraser Apr 21, 2020
e82dac8
Fixed typo in readme (PRi --> RPi)
Apr 21, 2020
7889207
Added docker version check
Apr 21, 2020
93b10ad
Merge pull request #31 from Paraphraser/20200421-doco-index
Willem-Dekker Apr 21, 2020
3e3b8fe
Merge pull request #32 from SensorsIot/docker-version-check
Slyke Apr 21, 2020
fec3260
Merge pull request #8 from SensorsIot/pr/143
Slyke Apr 21, 2020
c133968
Merge pull request #10 from SensorsIot/pr/151
Slyke Apr 21, 2020
c23d787
Merge pull request #17 from SensorsIot/pr/169
Slyke Apr 21, 2020
dc1c366
Merge branch 'master' into PR_add_volumes_to_script
gpongelli Apr 21, 2020
47fde77
Update README.md
Slyke Apr 22, 2020
b36b7b3
Added a check to see if docker is installed before attempting to get …
Apr 22, 2020
e95298f
Removed bookmark from transmission volumes.yml
gpongelli Apr 22, 2020
1936989
Create default directories on build
Apr 25, 2020
7689228
Merge pull request #23 from gpongelli/PR_add_volumes_to_script
Slyke Apr 25, 2020
4976473
Added requirements.txt for Python 3
Apr 25, 2020
694778f
Added YAML merging Python script, with menu script modifications. Add…
Apr 26, 2020
18010cd
Removed testing override file and useless comment
Apr 26, 2020
186c2c0
Added loader option to avoid errors on some systems
Apr 26, 2020
ba686ad
added timescaledb
gpongelli Apr 26, 2020
1ce3b57
Add timescaledb in menu.sh
gpongelli Apr 26, 2020
78ae80f
Added in Python, pip and PyYaml version checking
Apr 27, 2020
038be7d
Added simplified version checking. Added whiptail for docker version
Apr 27, 2020
35bf6a3
Added better version messaging for python and deps
Apr 27, 2020
f75df7a
Merge pull request #39 from gpongelli/PR_add_timescaledb
Slyke Apr 27, 2020
0dc9c53
Merge remote-tracking branch 'origin/fix-python-requirements.txt' int…
Apr 28, 2020
0a490e7
Merge branch 'check_docker_b4_version' into experimental
Apr 28, 2020
eccff23
Merge remote-tracking branch 'origin/migration-readme' into experimental
Apr 28, 2020
53d715e
Updated readme with instructions on how to select experimental features
Apr 28, 2020
bcb42c3
Fixed merge conflict issue
Apr 28, 2020
0b30604
Added some more nodered modules. Removed override file. Added branch …
Apr 28, 2020
c6390f3
Merge branch 'master' into patch-1
DIYtechie May 3, 2020
18bdefe
port changes and devices commented
DIYtechie May 3, 2020
f3be535
devices commented out
DIYtechie May 3, 2020
07837f7
Merge pull request #24 from DIYtechie/patch-1
Slyke May 3, 2020
1a0c310
Create deconz.md
DIYtechie May 5, 2020
fc30dee
Update deconz.md
DIYtechie May 6, 2020
c3f39e8
Update deconz.md
DIYtechie May 6, 2020
609c6c9
:pencil2: fixing typo
lepirlouit May 7, 2020
ab1d590
Merge pull request #43 from DIYtechie/patch-2
Slyke May 7, 2020
153543e
Merge pull request #45 from lepirlouit/patch-1
Slyke May 7, 2020
8cbdba6
Update deconz.md
DIYtechie May 9, 2020
c584129
make empty
DIYtechie May 9, 2020
b80b21e
create deCONZ device config
DIYtechie May 9, 2020
9fc4d67
Create service_conbee.yml
DIYtechie May 9, 2020
c2d231c
Create service_conbee_II.yml
DIYtechie May 9, 2020
d09497e
Create service_raspbee.yml
DIYtechie May 9, 2020
a483b43
fixing typos and simplifying with gif
DIYtechie May 9, 2020
9fadb35
Add Dozzle container.
robertcsakany May 10, 2020
e102a7e
Fixing tablulator
robertcsakany May 10, 2020
bbb5766
Merge pull request #48 from DIYtechie/deconz
Slyke May 10, 2020
ae04776
Expansion of Grafana documentation page.
Paraphraser May 16, 2020
d3d676b
Merge pull request #53 from Paraphraser/20200515-doco-grafana
Slyke May 16, 2020
45af02c
Merge pull request #49 from robertcsakany/feature/AddDozzleContainer
Slyke May 16, 2020
0b71c43
Removes pin of Grafana to version 6.3.6.
Paraphraser May 18, 2020
993e358
Merge pull request #54 from Paraphraser/20200518-template-grafana
Slyke May 18, 2020
64f755f
Added troubleshooting text and other amendments
DIYtechie May 21, 2020
d39ae55
Update deconz.md
DIYtechie May 21, 2020
f4ad7b9
Update deconz.md
DIYtechie May 21, 2020
ebe29db
Update deconz.md
DIYtechie May 21, 2020
1cc95db
Clean up
DIYtechie May 21, 2020
576c88f
Merge pull request #57 from DIYtechie/deconz
Slyke May 21, 2020
78d90b6
Merge pull request #40 from SensorsIot/experimental
Slyke May 29, 2020
01eb79a
Fixed bug where yaml merging was overwriting deconz services
May 29, 2020
1b6ea53
Merge pull request #62 from SensorsIot/deconz-fix
Slyke May 31, 2020
4324578
Merge branch 'master' into Portainer-agent
Willem-Dekker Jun 15, 2020
2955f36
Merge remote-tracking branch 'SensorsIot/master' into Portainer-agent
Willem-Dekker Jun 15, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: Publish docs via GitHub Pages
on:
push:
branches:
- master

jobs:
build:
name: Deploy docs
runs-on: ubuntu-latest
steps:
- name: Checkout master
uses: actions/checkout@v1

- name: Deploy docs
uses: mhausenblas/mkdocs-deploy-gh-pages@master
env:
PERSONAL_TOKEN: ${{ secrets.PERSONAL_TOKEN }}
44 changes: 44 additions & 0 deletions docs/Accessing-your-Device-from-the-internet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# accessing your device from the internet
The challenge most of us face with remotely accessing your home network is that you don't have a static IP. From time to time the IP that your ISP assigns to you changes and it's difficult to keep up. Fortunately, there is a solution, a DynamicDNS. The section below shows you how to set up an easy to remember address that follows your public IP no matter when it changes.

Secondly, how do you get into your home network? Your router has a firewall that is designed to keep the rest of the internet out of your network to protect you. Here we install a VPN and configure the firewall to only allow very secure VPN traffic in.

## DuckDNS
If you want to have a dynamic DNS point to your Public IP I added a helper script.
Register with duckdns.org and create a subdomain name. Then edit the `nano ~/IOTstack/duck/duck.sh` file and add your

```bash
DOMAINS="YOUR_DOMAINS"
DUCKDNS_TOKEN="YOUR_DUCKDNS_TOKEN"
```

first test the script to make sure it works `sudo ~/IOTstack/duck/duck.sh` then `cat /var/log/duck.log`. If you get KO then something has gone wrong and you should check out your settings in the script. If you get an OK then you can do the next step.

Create a cron job by running the following command `crontab -e`

You will be asked to use an editor option 1 for nano should be fine
paste the following in the editor `*/5 * * * * sudo ~/IOTstack/duck/duck.sh >/dev/null 2>&1` then ctrl+s and ctrl+x to save

Your Public IP should be updated every five minutes

## PiVPN
pimylifeup.com has an excellent tutorial on how to install [PiVPN](https://pimylifeup.com/raspberry-pi-vpn-server/)

In point 17 and 18 they mention using noip for their dynamic DNS. Here you can use the DuckDNS address if you created one.

Don't forget you need to open the port 1194 on your firewall. Most people won't be able to VPN from inside their network so download OpenVPN client for your mobile phone and try to connect over mobile data. ([More info.](https://en.wikipedia.org/wiki/Hairpinning))

Once you activate your VPN (from your phone/laptop/work computer) you will effectively be on your home network and you can access your devices as if you were on the wifi at home.

I personally use the VPN any time I'm on public wifi, all your traffic is secure.

## Zerotier
https://www.zerotier.com/

Zerotier is an alternative to PiVPN that doesn't require port forwarding on your router. It does however require registering for their free tier service [here](https://my.zerotier.com/login).

Kevin Zhang has written a how to guide [here](https://iamkelv.in/blog/2017/06/zerotier.html). Just note that the install link is outdated and should be:
```
curl -s 'https://raw.githubusercontent.com/zerotier/ZeroTierOne/master/doc/contact%40zerotier.com.gpg' | gpg --import && \
if z=$(curl -s 'https://install.zerotier.com/' | gpg); then echo "$z" | sudo bash; fi
```
12 changes: 12 additions & 0 deletions docs/Adminer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Adminer
## References
- [Docker](https://hub.docker.com/_/adminer)
- [Website](https://www.adminer.org/)

## About

This is a nice tool for managing databases. Web interface has moved to port 9080. There was an issue where openHAB and Adminer were using the same ports. If you have an port conflict edit the docker-compose.yml and under the adminer service change the line to read:
```
ports:
- 9080:8080
```
113 changes: 113 additions & 0 deletions docs/Backups.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Backups
Because containers can easily be rebuilt from docker hub we only have to back up the data in the "volumes" directory.

## Cloud Backups
### Dropbox-Uploader
This a great utility to easily upload data from your Pi to the cloud. https://magpi.raspberrypi.org/articles/dropbox-raspberry-pi. It can be installed from the Menu under Backups.
### rclone (Google Drive)
This is a service to upload to Google Drive. The config is described [here]( https://medium.com/@artur.klauser/mounting-google-drive-on-raspberry-pi-f5002c7095c2). Install it from the menu then follow the link for these sections:
* Getting a Google Drive Client ID
* Setting up the Rclone Configuration

When naming the service in `rclone config` ensure to call it "gdrive"

**The Auto-mounting instructions for the drive in the link don't work on Rasbian**. Auto-mounting of the drive isn't necessary for the backup script.

If you want your Google Drive to mount on every boot then follow the instructions at the bottom of the wiki page


## Influxdb
`~/IOTstack/scripts/backup_influxdb.sh` does a database snapshot and stores it in ~/IOTstack/backups/influxdb/db . This can be restored with the help a script (that I still need to write)

## Docker backups
The script `~/IOTstack/scripts/docker_backup.sh` performs the master backup for the stack.

This script can be placed in a cron job to backup on a schedule.
Edit the crontab with`crontab -e`
Then add `0 23 * * * ~/IOTstack/scripts/docker_backup.sh >/dev/null 2>&1` to have a backup every night at 23:00.

This script cheats by copying the volume folder live. The correct way would be to stop the stack first then copy the volumes and restart. The cheating method shouldn't be a problem unless you have fast changing data like in influxdb. This is why the script makes a database export of influxdb and ignores its volume.

### Cloud integration
The docker_backup.sh script now no longer requires modification to enable cloud backups. It now tests for the presence of and enable file in the backups folder
#### Drobox-Uploader
The backup tests for a file called `~/IOTstack/backups/dropbox`, if it is present it will upload to dropbox. To disable dropbox upload delete the file. To enable run `sudo touch ~/IOTstack/backups/dropbox`
#### rclone
The backup tests for a file called `~/IOTstack/backups/rclone`, if it is present it will upload to google drive. To disable rclone upload delete the file. To enable run `sudo touch ~/IOTstack/backups/rclone`

#### Pruning online backups
@877dev has added functionality to prune both local and cloud backups. For dropbox make sure you dont have any files that contain spaces in your backup directory as the script cannot handle it at this time.

### Restoring a backup
The "volumes" directory contains all the persistent data necessary to recreate the container. The docker-compose.yml and the environment files are optional as they can be regenerated with the menu. Simply copy the volumes directory into the IOTstack directory, Rebuild the stack and start.

## Added your Dropbox token incorrectly or aborted the install at the token screen

Make sure you are running the latest version of the project [link](https://github.com/gcgarner/IOTstack/wiki/Updating-the-Project).

Run `~/Dropbox-Uploader/dropbox_uploader.sh unlink` and if you have added it key then it will prompt you to confirm its removal. If no key was found it will ask you for a new key.

Confirm by running `~/Dropbox-Uploader/dropbox_uploader.sh` it should ask you for your key if you removed it or show you the following prompt if it has the key:

```
$ ~/Dropbox-Uploader/dropbox_uploader.sh
Dropbox Uploader v1.0
Andrea Fabrizi - [email protected]

Usage: /home/pi/Dropbox-Uploader/dropbox_uploader.sh [PARAMETERS] COMMAND...

Commands:
upload <LOCAL_FILE/DIR ...> <REMOTE_FILE/DIR>
download <REMOTE_FILE/DIR> [LOCAL_FILE/DIR]
delete <REMOTE_FILE/DIR>
move <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
copy <REMOTE_FILE/DIR> <REMOTE_FILE/DIR>
mkdir <REMOTE_DIR>
....

```

Ensure you **are not** running as sudo as this will store your api in the /root directory as `/root/.dropbox_uploader`

If you ran the command with sudo the remove the old token file if it exists with either `sudo rm /root/.dropbox_uploader` or `sudo ~/Dropbox-Uploader/dropbox_uploader.sh unlink`

## Auto-mount Gdrive with rclone

To enable rclone to mount on boot you will need to make a user service. Run the following commands

```bash
mkdir -p ~/.config/systemd/user
nano ~/.config/systemd/user/gdrive.service
```
Copy the following code into the editor, save and exit

```
[Unit]
Description=rclone: Remote FUSE filesystem for cloud storage
Documentation=man:rclone(1)

[Service]
Type=notify
ExecStartPre=/bin/mkdir -p %h/mnt/gdrive
ExecStart= \
/usr/bin/rclone mount \
--fast-list \
--vfs-cache-mode writes \
gdrive: %h/mnt/gdrive

[Install]
WantedBy=default.target
```
enable it to start on boot with: (no sudo)
```bash
systemctl --user enable gdrive.service
```
start with
```bash
systemctl --user start gdrive.service
```
if you no longer want it to start on boot then type:
```bash
systemctl --user disable gdrive.service
```

82 changes: 82 additions & 0 deletions docs/Blynk_server.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Blynk server
This is a custom implementation of Blynk Server

```yml
blynk_server:
build: ./services/blynk_server/.
container_name: blynk_server
restart: unless-stopped
ports:
- 8180:8080
- 8441:8441
- 9443:9443
volumes:
- ./volumes/blynk_server/data:/data
```

To connect to the admin interface navigate to `<your pis IP>:9443/admin`

I don't know anything about this service so you will need to read though the setup on the [Project Homepage](https://github.com/blynkkk/blynk-server)

When setting up the application on your mobile be sure to select custom setup [here](https://github.com/blynkkk/blynk-server#app-and-sketch-changes)

Writeup From @877dev

## Getting started
Log into admin panel at https://youripaddress:9443/admin
(Use your Pi's IP address, and ignore Chrome warning).

Default credentials:
user:[email protected]
pass:admin

## Change username and password
Click on Users > "email address" and edit email, name and password.
Save changes
Restarting the container using Portainer may be required to take effect.

## Setup gmail
Optional step, useful for getting the auth token emailed to you.
(To be added once confirmed working....)

## iOS/Android app setup
Login the app as per the photos [HERE](https://github.com/blynkkk/blynk-server#app-and-sketch-changes)
Press "New Project"
Give it a name, choose device "Raspberry Pi 3 B" so you have plenty of [virtual pins](http://help.blynk.cc/en/articles/512061-what-is-virtual-pins) available, and lastly select WiFi.
Create project and the [auth token](https://docs.blynk.cc/#getting-started-getting-started-with-the-blynk-app-4-auth-token) will be emailed to you (if emails configured). You can also find the token in app under the phone app settings, or in the admin web interface by clicking Users>"email address" and scroll down to token.

## Quick usage guide for app
Press on the empty page, the widgets will appear from the right.
Select your widget, let's say a button.
It appears on the page, press on it to configure.
Give it a name and colour if you want.
Press on PIN, and select virtual. Choose any pin i.e. V0
Press ok.
To start the project running, press top right Play button.
You will get an offline message, because no devices are connected to your project via the token.
Enter node red.....

## Node red
Install node-red-contrib-blynk-ws from pallette manager
Drag a "write event" node into your flow, and connect to a debug node
Configure the Blynk node for the first time:
```URL: wss://youripaddress:9443/websockets``` more info [HERE](https://github.com/gablau/node-red-contrib-blynk-ws/blob/master/README.md#how-to-use)
Enter your [auth token](https://docs.blynk.cc/#getting-started-getting-started-with-the-blynk-app-4-auth-token) from before and save/exit.
When you deploy the flow, notice the app shows connected message, as does the Blynk node.
Press the button on the app, you will notice the payload is sent to the debug node.

## What next?
Further information and advanced setup:
https://github.com/blynkkk/blynk-server

Check the documentation:
https://docs.blynk.cc/

Visit the community forum pages:
https://community.blynk.cc/

Interesting post by Peter Knight on MQTT/Node Red flows:
https://community.blynk.cc/t/my-home-automation-projects-built-with-mqtt-and-node-red/29045

Some Blynk flow examples:
https://github.com/877dev/Node-Red-flow-examples
14 changes: 14 additions & 0 deletions docs/Custom.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Custom container
If you have a container that you want to stop and start with the stack you can now use the custom container option. This you can use for testing or in prep for a Pull Request.

You will need to create a directory for your container call `IOTstack/services/<container>`

Inside that container create a `service.yml` containing your service and configurations. Have a look at one of the other services for inspiration.

Create a file called `IOTstack/services/custom.txt` and and enter your container names, one per line

Run the menu.sh and build the stack. After you have made the selection you will be asked if you want to add the custom containers.

Now your container will be part of the docker-compose.yml file and will respond to the docker-compose up -d commands.

Docker creates volumes under root and your container may require different ownership on the volume directory. You can see an example of the workaround for this in the python template in `IOTstack/.templates/python/directoryfix.sh`
15 changes: 15 additions & 0 deletions docs/Docker-commands.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Docker commands
## Aliases

I've added bash aliases for stopping and starting the stack. They can be installed in the docker commands menu. These commands no longer need to be executed from the IOTstack directory and can be executed in any directory

```bash
alias iotstack_up="docker-compose -f ~/IOTstack/docker-compose.yml up -d"
alias iotstack_down="docker-compose -f ~/IOTstack/docker-compose.yml down"
alias iotstack_start="docker-compose -f ~/IOTstack/docker-compose.yml start"
alias iotstack_stop="docker-compose -f ~/IOTstack/docker-compose.yml stop"
alias iotstack_update="docker-compose -f ~/IOTstack/docker-compose.yml pull"
alias iotstack_build="docker-compose -f ~/IOTstack/docker-compose.yml build"
```

You can now type `iotstack_up`, they even accept additional parameters `iotstack_stop portainer`
12 changes: 12 additions & 0 deletions docs/EspruinoHub.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Espruinohub
This is a testing container

I tried it however the container keeps restarting `docker logs espruinohub` I get "BLE Broken?" but could just be i dont have any BLE devices nearby

web interface is on "{your_Pis_IP}:1888"

see https://github.com/espruino/EspruinoHub#status--websocket-mqtt--espruino-web-ide for other details

there were no recommendations for persistent data volumes. so `docker-compose down` may destroy all you configurations so use `docker-compose stop` in stead

please let me know about your success or issues [here](https://github.com/gcgarner/IOTstack/issues/84)
Loading