DIY Your Home Cloud with Docker / docker-compose ecosystem

"The Cloud" is a buzzword that just means "somebody else software and hardware", this is fine is you want to optimize your business and you don't know your requirements, but if you knows them you can also build it yourself. Current open source technologies are so much improved and if you know how to wire these Lego brick it isn't that hard to learn.

My video on topic in Italian (you can enable automatic english subs).

Why should I self host something?

Control and privacy: if you delegate to third part services, you don't really know what will happen to your data and who will access to it
Save money: Rent or buy? It isn't always true but 1 TB of space on Dropbox for 1 year costs $ 99, a Raspberry Pi plus 1 TB hard disk costs more or less the same amount (so if you need it for more than 1 year you pay your hardware investment in 1 year and if you need 2 TB your ROI is earlier )
Interesting job skill: the software is everywhere and knowing how it works and how to manage it is unavoidable to be competitive in the future.

What Will I Learn?

The basics for an Home Cloud o Test Deploy Server
How to wire together some open source components to be able to self-host personal projects or famous open source web software in the own network.

Requirements

To follow this tutorial you need this software

Ubuntu Linux LTS (or another operative system where Docker is supported)
docker https://docs.docker.com/install/linux/docker-ce/ubuntu/
docker-compose https://docs.docker.com/compose/install/

Difficulty

Intermediate

Tutorial Contents

Your Cloud Infrastructure
How to set up your domain *.home
Your first self hosted web app portainer.home

Your Cloud Infrastructure

You can put in your infrastructure what you want or what you need, the nice part is that many components aren't mandatory or you can activate them only when you need them

your docker cloud

Traefik (mandatory): Reverse proxy + HTTPS management (only if deployed with a Public IP with FQDN aka proper acquired domain name). It allows to have multiple web apps on the same IP, like portainer.home, yourwordpress.home.
Portainer (mandatory): Docker dashboard. You can avoid it but it's very convenient to monitor and manage the health of the containers.
Netdata (optional): Monitoring. How how RAM, CPU, NET does a web app consume? This is what you need

How to set up your domain http://portainer.home , *.home

We have enough theory, let's open a terminal to try it for real. Before running a container we need to check the environment.

If we have just a couple of computers in our LAN a DNS is overkill the easiest way to name a web app is by adding its name to

Open or edit /etc/hosts and add the line below

192.168.1.3 traefik.home portainer.home netdata.home filestash.home

You need the root / admin permission to edit it and every time you need a new app you need to modify it. The rules are simple 192.168.1.XXX is the computer IP which hosts the infrastructure with traefik, then every app domain is space separated.

Let's check you can reach the app

➜  ping -c 3 portainer.home
PING traefik.home (192.168.1.3) 56(84) bytes of data.
64 bytes from traefik.home (192.168.1.3): icmp_seq=1 ttl=64 time=0.527 ms
64 bytes from traefik.home (192.168.1.3): icmp_seq=2 ttl=64 time=0.508 ms
64 bytes from traefik.home (192.168.1.3): icmp_seq=3 ttl=64 time=0.460 ms

--- traefik.home ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 0.460/0.498/0.527/0.033 ms

OK portainer.home resolves to our wanted IP.

Your first self hosted web app portainer.home

We don't have our infrastructure yet, so we need to set up traefik once, and then our web apps. Portainer is just an example of dockerized app or service, all the other dockerized webapp starts more or less in the same way.

NOTE: you need one or more docker-compose.yml file to configure how to run the services, my advice is to keep them one per directory

# file: apps/traefik/docker-compose.yml
version: '2'

services:
  proxy:
    image: traefik:alpine
    hostname: traefik
    command: --web --docker --logLevel=INFO
    restart: unless-stopped
    networks:
      - web
    ports:
      - "80:80"
      - "443:443"
    labels:
      - "traefik.frontend.rule=Host:traefik.home"
      - "traefik.port=8080"
      - "traefik.frontend.entryPoints=http,https"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

networks:
  web:
    external: true

and type in the terminal

docker-compose up -d

It should download the docker image if missing and start the container. You can check the correctness with the command below traefik_reverse-proxy_1 is an automatically generated name

$ docker-compose ps
         Name                     Command           State                     Ports                    
------------------------------------------------------------------------------------------------------
traefik_reverse-proxy_1   /traefik --api --docker   Up      0.0.0.0:80->80/tcp, 0.0.0.0:8080->8080/tcp

Now the first useful app portainer.home Put the file below in a portainer directory.

# file: apps/portainer/docker-compose.yml
version: "3"
 
services:
  app:
    image: portainer/portainer
    hostname: portainer
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /persistent/portainer_data:/data
   
    networks:
      - web
    labels:
      - traefik.docker.network=web
      - traefik.frontend.rule=Host:portainer.home
      - traefik.port=9000
      - traefik.backend=portainer
    command: -H unix:///var/run/docker.sock
 
networks:
  web:
    external: true

/persistent/portainer_data is the path where you want to have the service persistent configuration, docker container by default have no persistent memory. Some containers need a persistent state some other don't, please have a look and destroy and restart a container to check that you configured it correctly. docker-compose.yml files are very powerful and can manage several services together but this is beyond the scope of this tutorial

docker-compose up -d

And now You should see Portainer web ui up and running. The first time it will require to set a password

Portainer

Congratulation! Now start a new app isn't what difficult is it? If you don't want to see that "Not Secure" label you have to accept that self signed SSL certificate and if you want to access to your services remotely you could add a further OpenVPN layer of protection.

Here some dockerized docker container that i raccomend:
https://traefik.io
https://www.portainer.io/
https://www.filestash.app/
https://github.com/kylemanna/docker-openvpn
https://github.com/MarvAmBass/docker-versatile-postfix

Curriculum

🖥 How to restore Monero private keys from Ledger Nano S wallet or Coinomi