setup_guide.md

# Pre-requisites

- Docker
- Python
- Aptly
- GPG
- Debian

# Current setup (Read carefully)

First run the aptly continer and note down the network name of the aptly container. That network name must be updated into the gitlab-runner container docker compose file like below:
```
networks:
  default:
    external:
      # Name of the aptly container's network
      name: aptly_default   # Verify this every time
```

The current setup allows to setup aptly with gitlab to create a self-sufficient
Debian repo hosting system.
As of writing this document, the setup is being done on an internal machine
called `node2` which can be accessed either via ssh or by using web GUI of
Portainer.

On Portainer, the left index shows something called `Stacks`. This is where
the containers for aptly and gitlab runner can be created with the use of
docker compose files. Current working setup uses `aptly_setup-compose.yaml` for
aptly container setup and `gitlab_w_aptly-compose.yaml` for gitlab runner setup.

In order to get things up and working on aptly container, refer to the setup
steps mentioned in `aptly-based-repository-setup.md`. This serves the purpose
of hosting the repos flawlessly. However, hosting custom packages alongwith
the packages from Debian requires a little extra effort.
Please note that you will need to import current GPG keys or create new keys
as per the situation. Do not forget to add them to apt keyring.

Currently, all the inhouse packages reside in https://git.hamaralinux.org/hamara-developers/hamara-packages, and
are modified, added and removed which triggers a pipeline to build all the
packages as per the configuration in `.gitlab-ci.yaml`. You will also see two
files named `watchdog-run.py` and `snapshot_from_repo.sh` which should be
copied to `/opt/watchdog` folder on the aptly container.

As of now, the watchdog script needs to be run either inside a tmux session or
as a background service. This script will ensure that on any successful build
there is a new repo of packages and a snapshot is created. You need to follow
steps 7 to 9 of `aptly-based-repository-setup.md` manually thereafter to make
sure that newly built packages are served on the repo.

## Manual interventions
Despite the effort to automate things, there are a few areas where manual
intervention is required. 

On Aptly container:
1. Install `python3-pip`, `ssh` and `watchdog`.
2. Create `/opt/watchdog` and then `/opt/watchdog/watches.
3. After any successful package build, publish the repos again.
4. ssh public key copied from gitlab-runner container 
   should be placed under /root/.ssh/authorized_keys
   make sure the permission of the authorized_hosts file by running 
   `$ chmod 600 authorized_keys`
5. Make sure the ssh service is running if not start it by
   `service ssh start`
6. Enable ssh-keybased access from ssh configuration file so that gitlab-runner can connect with aptly container via ssh.
7. If you are using external gpg key which are created on any other machine than we must import that keys via below method so we need to change the directory where we have these files and then need to run below comand and if promted for passowrd then enter the passphrase which we used to create the gpg key pair on the other machine:
   For private key import -  `gpg --import private.key`
   For public key import - `gpg --import public.key`
   For trustowner key import - `gpg --import-trustowner trustowner.key`
8. We must copy the public key file to /opt/aptly/public/aptly_repo_signing.key  file (if we are using an external gpg key pair with our setup) as this would let the client to download the file to install the key into their machine.

Note: If in case you face any issue while importing the keys then consider to update the version of gpg(GnuPG) tool.		


On GitLab container:
1. Register GitLab runner with **shell** as per the token from GitLab.
   ```
    $ gitlab-runner register \
      --non-interactive \
      --url "https://git.hamaralinux.org/" \
      --registration-token "PROJECT_REGISTRATION_TOKEN" \
      --executor "shell"    
   ```
2. Find the ip of aptly container and add it to /etc/hosts 
   eg: `10.0.1.14   aptly`
3. Install sudo and add the below line in sudoers file for scripts to execute the cmds with sudo
   `gitlab-runner ALL=(ALL) NOPASSWD:ALL`
4. switch to gitlab-runner user and create a sshkey by running
   `$ ssh-keygen`
5. copy the public key to aptly container 
6. Just to verify that ssh thing is working fine switch to gitlab-runner user and run below command:
   `ssh root@aptly`

After performing these essential steps on both the containers then we are good to go with aptly repo & snapshot create and publish.

## Things to take care of
1. Make sure there is only one GPG key on the system.
2. Be careful about old instances of watchdog which may be showing different
results than expected.
3. Running a single job on GitLab will **not** ensure creation of a new repo
and snapshot. All jobs shall be run, especially `trigger-watchdog`.
4. Restarting containers will take most of the things away unless configured
in the compose file.