# Overview of the task

The task was to implement a security-critical webshop that can be used to sell and buy animated images stored in a custom format. The webshop has to support CAFF (_CrySyS Animated File Format_). The system consists of a web service and a mobile or web client.

# Overview of the implementation

During the design phase, we decided to implement a web client.

Our implementation consists of the following modules:

- **CAFF previewer** / [caff-previewer](https://github.com/UnstableVortexSecurity/caff-previewer): A program written in C that is used to extract a preview (a single frame) from a CAFF file. The extracted preview is saved in .tga (TARGA) format, which is a simple bitmap image format.
- **CAFF previewer wrapper**  / [caff-previewer-wrapper](https://github.com/UnstableVortexSecurity/caff-previewer-wrapper): A wrapper written in Python/Flask that provides a simple HTTP interface between CAFF previewer and the web client. The wrapper also converts the extracted preview to PNG using ImageMagick.
- **Web application** / [webshop](https://github.com/UnstableVortexSecurity/webshop): A web client written in Python/Flask. This client implements the user functions.

Note: each module is hosted in its own repository.

# CAFF previewer
This module is responsible for extracting previews from CAFF files. The module can be built using CMake, detailed instructions are available in the module's [README file](https://github.com/UnstableVortexSecurity/caff-previewer/blob/master/README.md).

## Error handling

Functions that perform a complex task can fail in multiple ways. In order to be able to pinpoint the exact issue, these functions always have to return with a status code. The type of the status code is `uint8_t`.

Successful execution is indicated by the status code `0x00`. Any outputs of the function must be returned using pointers given in the parameters. The status code is almost always used as an exit code for the program, so users can easily identify the exact issue.

Each error type must have a status code that is unique in the whole program. The status codes are divided into two parts:

- First 4 bits: identifies a certain module of the previewer
- Last 4 bits: identifies the exact problem

## Memory allocation

The module provides its own memory allocation library called **MagicMemory**.

The library can be used to create MagicMemory contexts. All memory allocation calls must be performed through this library. If a function allocates memory, it has to take a MagicMemory context (that has the type `mm_ctx`) as its first parameter and perform the memory allocation inside that context.

Before the program exits, all of the contexts must be freed up.

## Data validation

There are two different kinds of functions that perform data validation:

- **Validator**: A validator only validates data according to its format specification.
- **Parser**: A parser validates data and then extracts some fields from the data.

It is not unusual for a parser function to call a validator function to validate its whole input before extracting the fields.

## Functions

This section gives an overview of the important source files and functions.

### caff_tools.c

This file contains functions that are used to validate and parse CAFF files.

`parse_caff_header` is used to parse the header of a CAFF file.

`validate_caff_credits` is used to validate the credits section of a CAFF file.

`validate_caff_animation` is used to validate a single CAFF animation section.

`validate_caff_file` is used to validate the whole CAFF file.

`parse_caff_get_first_ciff` is used to parse a CAFF file and extract the first CIFF image from it.

### ciff_tools.c

This file contains functions that are used to validate and parse CIFF images.

`validate_ciff` is used to validate a CIFF image.

`parse_ciff_get_pixel_data` is used to parse a CIFF image and to find the location of pixel data.

### magic_memory.c

This file contains our own MagicMemory library that provides memory allocation functions.

`magic_memory_begin` is used to create a MagicMemory context that can be used with other MagicMemory functions later.

`magic_malloc` is the same as the standard `malloc`, but the pointer is also added to a context. Contexts are used to keep track of pointers so that they won't be lost. A context is basically a linked list of pointers.

### main.c

`main` is the entrypoint of the previewer module.

`perform_extraction` takes care of the preview extraction process: it calls parser and validator functions and other helper functions. In theory, the size of a CAFF file can be really huge. In the case of our previewer, only files smaller than 512 megabytes are processed.

### utils.c

`read_file_to_mem` is used to read CAFF files into memory.

### libtarga.c

This file contains the [LibTarga](https://research.cs.wisc.edu/graphics/Gallery/LibTarga/) library by Alex Mohr.

# CAFF previewer wrapper
This module makes communication between the previewer and the web client possible.

## Endpoint

The module provides only one endpoint:

**POST** `/preview`

**Request:** The endpoint expects a single CAFF file to be sent. The content of the CAFF file is expected to be senr in a raw binary format without any extra encapsulation. The content type must be set to `application/octet-stream`.

**Response:** If the preview extraction is successful, the response contains a single PNG image. The content type is set to `image/png`.

## Integrity checking

CAFF previewer wrapper provides two headers in the response that can be used to check the integrity of transferred files. These are the following:

- `X-request-checksum` contains the md5 checksum of the received file.
- `X-response-checksum` contains the md5 checksum of the converted file.

These checksums can be used to detect errors during file transfer.

## Preview process

This section gives an overview of the preview creation process.

### Converting CAFF to TGA

After the wrapper receives a CAFF file, it uses the CAFF previewer written in C to extract a preview. The output of the previewer is a TGA file.

### Converting TGA to PNG

The wrapper uses ImageMagick to convert the TGA file (output of the previewer) to PNG. This step is done because PNG is a more suitable image format for the web application than TGA.

# Web application
This module provides the web UI for the users.

## Trying out our webshop
Since our webshop is a modern web application, it has quite a few dependencies. To make trying it out easier, we deployed the web application to our own secure Kubernetes cluster. It is available at [unstablevortex.kmlabz.com](https://unstablevortex.kmlabz.com).

The application connects to every dependency (Minio, CAFF previewer wrapper, SMTP and PostgreSQL server) over TLS and the configuration environment variables are stored in [Vault](https://www.vaultproject.io/docs/what-is-vault).

## Integrity checking

The function that is responsible for the preview creation (`create_caff_preview` in `src/utils/caff_previewer.py`) validates the response for the following:

- integrity (2-way)
- MIME type
- size