setup_guide.md

# Pre-requisites

- Docker
- Python
- Aptly
- GPG
- Debian

# Current setup (Read carefully)
The current setup allows to setup aptly with gitlab to create a self-sufficient
Debian repo hosting system.
As of writing this document, the setup is being done on an internal machine
called `node2` which can be accessed either via ssh or by using web GUI of
Portainer.

On Portainer, the left index shows something called `Stacks`. This is where
the containers for aptly and gitlab runner can be created with the use of
docker compose files. Current working setup uses `aptly_setup-compose.yaml` for
aptly container setup and `gitlab_w_aptly-compose.yaml` for gitlab runner setup.

In order to get things up and working on aptly container, refer to the setup
steps mentioned in `aptly-based-repository-setup.md`. This serves the purpose
of hosting the repos flawlessly. However, hosting custom packages alongwith
the packages from Debian requires a little extra effort.
Please note that you will need to import current GPG keys or create new keys
as per the situation. Do not forget to add them to apt keyring.

Currently, all the inhouse packages reside in https://git.hamaralinux.org/hamara-developers/hamara-packages, and
are modified, added and removed which triggers a pipeline to build all the
packages as per the configuration in `.gitlab-ci.yaml`. You will also see two
files named `watchdog-run.py` and `snapshot_from_repo.sh` which should be
copied to `/opt/watchdog` folder on the aptly container.

As of now, the watchdog script needs to be run either inside a tmux session or
as a background service. This script will ensure that on any successful build
there is a new repo of packages and a snapshot is created. You need to follow
steps 7 to 9 of `aptly-based-repository-setup.md` manually thereafter to make
sure that newly built packages are served on the repo.

## Manual interventions
Despite the effort to automate things, there are a few areas where manual
intervention is required. 

On Aptly container:
1. Install `python3-pip`, `ssh` and `watchdog`.
2. Create `/opt/watchdog` and then `/opt/watchdog/watches.
3. After any successful package build, publish the repos again.
4. ssh public key copied from gitlab-runner container 
   should be placed under /root/.ssh/authorized_hosts
   make sure the permission of the authorized_hosts file by running 
   `$ chmod 600 authorized_hosts`
5. Make sure the ssh service is running if not start it by
   `service ssh start`


On GitLab container:
1. Register GitLab runner with **shell** as per the token from GitLab.
   ```
    $ gitlab-runner register \
      --non-interactive \
      --url "https://git.hamaralinux.org/" \
      --registration-token "PROJECT_REGISTRATION_TOKEN" \
      --executor "shell"    
   ```
2. Find the ip of aptly container and add it to /etc/hosts 
   eg: `10.0.1.14   aptly`
3. Install sudo and add the below line in sudoers file for scripts to execute the cmds with sudo
   `gitlab-runner ALL=(ALL) NOPASSWD:ALL`
4. switch to gitlab-runner user and create a sshkey by running
   `$ ssh-keygen`
5. copy the public key to aptly container 


## Things to take care of
1. Make sure there is only one GPG key on the system.
2. Be careful about old instances of watchdog which may be showing different
results than expected.
3. Running a single job on GitLab will **not** ensure creation of a new repo
and snapshot. All jobs shall be run, especially `trigger-watchdog`.
4. Restarting containers will take most of the things away unless configured
in the compose file.