aptly-based-repository-setup.md

# Aptly Setup Walkthorugh

### Terminology
#### mirror
Copy of a remote repository.
Mutable: Yes

#### repo
Repository that contains packages.
- Local (Testing)
  The local copy of the repository you’re trying to build. Should be accessible to the user testing
  and serving it.
  Server: Aptly HTTP Server
  Mutable: Yes
- Published (Production)
  The public and published version with GPG keys. Should be in a publicly accessible folder.
  Server: nginx
  Mutable: Shoud not be


#### snapshot
A locked version of a repo or mirror.
Mutable: No

### Things to take care of
 - You should only publish a snapshot. This will make sense if we have to rollback to a previous
   state.
 - Packages should only be replaced in the local repo.
 - Configuration of maximum open files should be highest else aptly would crash complaining
   that you have reached the maximum number of open files.
 - Always create a new snapshot after making any changes.
 - Avoid creating contents index unless necessary because it
   - takes long time
   - leads to several lock issues as new packages arrive to the server and cannot be added to the
     repository because of Aptly DB being locked 


### Procedure
1. Create Debian testing multi-component mirrors
2. Update the created mirrors
3. Create a snapshots of the mirrors
4. Create an empty repo
5. Add hamara packages with force replace option to the repo created in above step
6. Create a snapshot from this repo
7. Merge the new snapshot with the snapshot of main component
8. Publish merged snapshot, contrib snapshot and non-free snapshot
9. Add the appropriate sources to apt sources
10. Add your GPG key to APT with apt-key


On the side,
Add a hook in GitLab to repeat steps 4-7 above on every successful build of Hamara packages


### Useful commands
 ```
 $ aptly mirror create <mirror-name> http://deb.debian.org/debian <distribution> [<component>]
 
 $ aptly mirror update <mirror-name>
 
 $ aptly mirror list
 
 $ aptly mirror show <mirror-name>
 
 $ aptly snapshot create <snapshot-name> from mirror <mirror-name>
 
 # Note: Make sure to have successfully updated the mirror before running this command.
 
 $ aptly repo create <repo-name> from snapshot <some-snapshot>
 
 $ aptly repo add <repo-name> <hamara_package_directory>
 
 $ aptly snapshot create <snapshot-name> from repo <repo-name>
 
 $ aptly publish snapshot <snapshot-name>

 # Note: If the snapshot needs to be published with the source packages (`.dsc`, `*tar*` and so),
 use the following command

 $ aptly publish snapshot -architectures=source,<other-archs> <snapshot-name>

 $ aptly publish drop <distribution-name> <publish-name>
```

### Setup of a repository with multi components ( main, contrib, non-free)


[Multi component setup of repository](https://www.aptly.info/doc/feature/multi-component/)


### 1. Create mirrors from upstream repositories

```
# aptly mirror create testing-main http://deb.debian.org/debian testing main

# aptly mirror create testing-contrib http://deb.debian.org/debian testing contrib

# aptly mirror create testing-non-free http://deb.debian.org/debian testing non-free

```

### 2. Update the mirrors

```
# aptly mirror list -raw | xargs -n 1 aptly mirror update
```

### 3. Create snapshots from these mirrors

```
# aptly snapshot create <codename>-<version>-main-<date> from mirror testing-main

# aptly snapshot create  <codename>-<version>-contrib-<date> from mirror testing-contrib

# aptly snapshot create  <codename>-<version>-non-free-<date> from mirror testing-non-free
```

### 4. Create a repo for Hamara inhouse packages

```
# aptly repo create hamara-packages
```

### 5. Add Hamara inhouse packages to the newly created repo

```
# aptly repo add hamara-packages <packages-directory>

```

### 6. Create snapshot of the repo

```
# aptly create snapshot <codename>-<version>-inhouse-<date> from repo hamara-packages
```

### 7. Merge the snapshots

```
# aptly snapshot merge <codename>-<version>-main-w-inhouse-<date> <codename>-<version>-main-<date> <codename>-<version>-inhouse-<date>
```

### 8. Publish the snapshots

```
# aptly publish snapshot -skip-contents -skip-signing -component=,, <codename>-version-main-w-inhouse-<date> <codename>-<version>-contrib-<date> <codename>-<version>-non-free-<date>
```
**NOTE**: Use `skip-signing` only for testing purposes.


### 9. Serve the published snapshots

```
aptly serve -listen=":8000"
```
**NOTE**: Serve on a port that is accessible to you.


Check out the perfect multi-component published repo on `<server-address>:8000`


## HELP!

Following things are still left to be done and require help. Please note that you would require access to entire infrastructure in order to be able to do this.

1. Updations to .gitlab-ci.yml

Deployment procedure of the local packages from gitlab to the server is to be added and tested.

Added: No
Tested: No

TBD by: Shivani Bhardwaj


2. Dockerfile

A Dockerfile replicating all the procedure written above is required to be written.

Added: No
Tested: No

TBD by: -


3. nginx configuration for hosting

nginx.conf to serve the published repos is required to be added on the server.

Added: No
Tested: No

TBD by: Shivani Bhardwaj


4. GPG key signing of packages

Currently the published repo is not signed. Use the Hamara GPG key to sign the repos before publishing them.

Done: No
Tested: No

TBD by: -


5. Adding package source to the repos

Currently only `.deb` files have been added to the repo but the source files like `.dsc`, `*tar*` should also be there. Figure out how to do that.

Done: No
Tested: No

TBD by: -