# User Image Stacks

## Overview

Our Docker image stack hierarchy is inspired by [jupyter/docker-stacks](https://jupyter-docker-stacks.readthedocs.io/en/latest/index.html), but customized for our specific needs. This document outlines our image hierarchy, key features, and customizations.

## Image Hierarchies

### jupyter/docker-stacks Reference Hierarchy

```{mermaid}
graph TD
    A[ubuntu LTS with point release] --> B[docker-stacks-foundation]
    B --> C[base-notebook]
    C --> D[minimal-notebook]
    D --> E[scipy-notebook]
    D --> F[r-notebook]
    D --> G[julia-notebook]
    E --> H[tensorflow-notebook]
    E --> I[pytorch-notebook]
    E --> J[datascience-notebook]
    E --> K[pyspark-notebook]
    K --> L[all-spark-notebook]
```

### Our Custom Hierarchy

```{mermaid}
graph TD
    A["ubuntu<br>(LTS with point release)"] --> B[docker-stacks-foundation]
    C["ubuntu<br>+ CUDA"] --> B["polusai/notebooks-hub-stacks-foundation<br><ul><li>add Lmod</li><li>use apt-fast</li></ul>"]
    B --> D["polusai/notebook<br>
<img src='../../img/app_logos/jupyter_logo.png' style='max-width:30px;min-height:0'/>"]
    B --> E["polusai/dashboard-base<br>(add jhsingle-native-proxy)"]
    E --> F["polusai/vscode<br>
<img src='../../img/app_logos/vscode_logo.png' style='max-width:30px;min-height:0'/>"]
    E --> G["polusai/rstudio<br>
<img src='../../img/app_logos/rstudio_logo.png' style='max-width:60px;min-height:0'/>"]
    E --> H["polusai/rshiny<br>
<img src='../../img/app_logos/rshiny_logo.png' style='max-width:60px;min-height:0'/>"]
    E --> I["polusai/pshiny<br>
<img src='../../img/app_logos/pshiny_logo.png' style='max-width:60px;min-height:0'/>"]
    E --> J["polusai/dash<br>
<img src='../../img/app_logos/dash_logo.png' style='max-width:60px;min-height:0'/>"]
    E --> K["polusai/solara<br>
<img src='../../img/app_logos/solara_logo.png' style='max-width:60px;min-height:0'/>"]
    E --> L["polusai/streamlit<br>
<img src='../../img/app_logos/streamlit_logo.png' style='max-width:60px;min-height:0'/>"]
    E --> M["polusai/voila<br>
<img src='../../img/app_logos/voila_logo.png' style='max-width:60px;min-height:0'/>"]
```

## Base Images

### polusai/notebooks-hub-stacks-foundation

This image is based on [`jupyter/docker-stacks-foundation`](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-docker-stacks-foundation) with additional customizations.

#### Features from jupyter/docker-stacks-foundation:

- Package managers:
  - [conda](https://github.com/conda/conda): Cross-platform, language-agnostic binary package manager
  - [mamba](https://github.com/mamba-org/mamba): Faster reimplementation of conda in C++ (default package manager)
- Unprivileged user `jovyan` (uid=1000, configurable) in group `users` (gid=100)
- `tini` and `start.sh` script as container entry point
- `run-hooks.sh` script for sourcing/running files in a given directory
- Options for passwordless sudo
- Common system libraries (bzip2, ca-certificates, locales)
- `wget` for downloading external files
- No preinstalled scientific computing packages

#### Additional customizations:

- Lmod for managing environment modules across Notebooks Hub
- `apt-fast` for faster .deb package installation
- Additional tools: `gdebi-core`, `vim`, `nano`, `jq`
- Symbolic link to shared directory: `/home/jovyan/shared` -> `/opt/shared`

### `polusai/notebook`

Based on the foundation image, this adds:

- JupyterLab
- Related packages and extensions

### `polusai/dashboard-base`

This image adds:

- `jhsingle-native-proxy` for making any web-based application compatible with JupyterHub by presenting itself as a Jupyter Server.
- A wrapper script for starting applications

## Application-specific Images

The following images are based on the `dashboard-base` image:

1. `polusai/vscode`: VS Code development environment based on [Coder](https://coder.com/)
2. `polusai/rstudio`: RStudio IDE for R development
3. `polusai/rshiny`: R Shiny for interactive web applications
4. `polusai/pshiny`: Python-based Shiny alternative
5. `polusai/dash`: Plotly Dash for analytical web applications
6. `polusai/solara`: Streamlit-like framework for data apps
7. `polusai/streamlit`: Streamlit for data applications
8. `polusai/voila`: Voilà for converting Jupyter notebooks to standalone applications

## Usage
   
### Building a Dashboard Image  
To build a custom dashboard image, navigate to the directory containing your Dockerfile and run the following command:  
   
```sh  
docker build . -t <dashboard_image>  
```  
   
This command will use the latest `dashboard-base` image by default to create your custom dashboard image.  
   
### Running the Dashboard Container  
To run the dashboard container, you need to set the `DASHBOARD_PORT` environment variable and ensure that the necessary port (default is 8888) is open. You can use the following `docker run` command to start your container:  
   
```sh  
docker run -d -p 8888:8888 -e DASHBOARD_PORT=8888 <dashboard_image>  
```  
   
Here's a breakdown of the command:  
- `docker run -d`: Run the container in detached mode.  
- `-p 8888:8888`: Map the container's port 8888 to the host's port 8888.  
- `-e DASHBOARD_PORT=8888`: Set the `DASHBOARD_PORT` environment variable to 8888.  
- `<dashboard_image>`: Replace this with the name of your custom dashboard image.  
   
After running this command, your dashboard application should be accessible via `http://localhost:8888`.  
   
### Example  
Suppose you have built an image named `my-custom-dashboard`. You would start it with:  
   
```sh  
docker run -d -p 8888:8888 -e DASHBOARD_PORT=8888 my-custom-dashboard  
```  
   
This will launch your custom dashboard and make it accessible at `http://localhost:8888`.  
   
By following these steps, you can easily build and run your custom dashboard images within the Notebooks Hub environment.

## Customization

The `polusai/dashboard-base` image is designed to simplify the adoption of new web-based applications. This flexibility allows you to easily add and integrate various applications into your Notebooks Hub environment.

### Prerequisites for New Applications

To successfully integrate a new application using the `dashboard-base` image, ensure that the application meets these two key requirements:

1. **Command-line Startup**: The application must be capable of starting from the command line in a Linux environment.
2. **Web Application Port**: The application should serve its web interface on a port, ideally one that is configurable.

### Steps to Integrate a New Application

1. **Create a New Dockerfile**:
   Start with the `polusai/dashboard-base` image as your base:

   ```dockerfile
   ARG ROOT_CONTAINER=polusai/dashboard-base:latest
   FROM $ROOT_CONTAINER

   USER root

   COPY start-dashboard.sh /usr/bin
   RUN chmod +x /usr/bin/start-dashboard.sh

   USER $NB_USER

   # Start the dashboard
   CMD ["/usr/bin/dashboard-wrapper.sh"]
   ```

2. **Create a start-dashboard.sh script**:
   This script will contain the specific logic to start your application. Here's an example based on the Streamlit implementation:

   ```bash
   #!/bin/bash

   # run dashboard based on type
   if [ "$DASHBOARD_TYPE" == "file" ]; then
     $PYTHON_EXEC_PATH -m pip install your-application==1.0.0 dependency1 dependency2
     $JHSINGLE_COMMAND --destport 8080 $PYTHON_EXEC_PATH {-}m your-application run $DASHBOARD_PATH {--}port 8080 {--}other-options
   elif [ "$DASHBOARD_TYPE" == "folder" ]; then
     echo "ERROR: Your application requires file path to be provided" >&2
     exit 1 
   elif [ "$DASHBOARD_TYPE" == "none" ]; then
     echo "ERROR: Your application requires file path to be provided" >&2
     exit 1 
   fi
   ```

3. **Understand the dashboard-wrapper.sh script**:
   The `dashboard-wrapper.sh` script in the `dashboard-base` image sets up the environment and calls your `start-dashboard.sh` script. Key points:
   - It activates Lmod and loads environment modules.
   - Sets up the `JHSINGLE_COMMAND` with appropriate options.
   - Sources your `start-dashboard.sh` script.

4. **Configure jhsingle-native-proxy**:
   The `JHSINGLE_COMMAND` in `dashboard-wrapper.sh` handles the proxying of your application. Ensure your `start-dashboard.sh` uses this command correctly.

5. **Port Configuration**:
   Use the `$DASHBOARD_PORT` environment variable (set in `dashboard-wrapper.sh`) for your application's port.

6. **Build and Test**:
   Build your new image and test it to ensure the application starts correctly and is accessible through the proxy.

### Example: Integrating a Hypothetical Web App

Let's say you want to integrate a hypothetical web application called "DataViz". Here's how you might set it up:

Dockerfile:
```dockerfile
ARG ROOT_CONTAINER=polusai/dashboard-base:latest
FROM $ROOT_CONTAINER

COPY start-dashboard.sh /usr/bin

CMD ["/usr/bin/dashboard-wrapper.sh"]
```

start-dashboard.sh:
```bash
#!/bin/bash

if [ "$DASHBOARD_TYPE" == "file" ]; then
  $PYTHON_EXEC_PATH -m pip install dataviz==1.0.0 required-dependency1 required-dependency2
  $JHSINGLE_COMMAND --destport 8080 $PYTHON_EXEC_PATH {-}m dataviz run $DASHBOARD_PATH {--}port 8080 {--}server.address=0.0.0.0 {--}server.headless True
elif [ "$DASHBOARD_TYPE" == "folder" ]; then
  echo "ERROR: DataViz requires a file path to be provided" >&2
  exit 1 
elif [ "$DASHBOARD_TYPE" == "none" ]; then
  echo "ERROR: DataViz requires a file path to be provided" >&2
  exit 1 
fi
```

This setup installs the DataViz application, sets its port, and uses `jhsingle-native-proxy` (via `$JHSINGLE_COMMAND`) to make it available through the standard Notebooks Hub interface.

### Separation of dashboarding tools from the environment
In Notebooks Hub, there are broadly two types of dashboarding tools: those that can be separated from the environment they use and those that cannot.

#### Tools that can run independently
Examples of tools that can run independently:

- Voila. It has a flag `--VoilaExecutor.kernel_name` which allows to point it to a separate environment.
- RStudio. It has a flag `--rsession-which-r` which allows to point it to R executable in a separate environment.
- RShiny. It implicitly uses `R` executable which can be remapped to a separate environment.

#### Tools That Need Runtime Installation of Dashboarding Library

These tools normally import Python packages from the same environment they are installed in and don't have any configuration flags to point them to a separate environment:

- PyShiny
- Streamlit
- Solara
- Dash

We install packages at runtime in the start-dashboard.sh script because we combine the user-provided environment with Shiny dependencies, and thus cannot install the packages during the Docker build.

### Best Practices

✅ Use environment variables like `$DASHBOARD_TYPE`, `$DASHBOARD_PATH`, and `$PYTHON_EXEC_PATH` provided by the `dashboard-base` image.

✅ Include necessary dependencies and their versions in your `start-dashboard.sh` script.

✅ Handle different `DASHBOARD_TYPE` scenarios appropriately.

✅ Use the `$JHSINGLE_COMMAND` for starting your application to ensure proper integration with the proxy.

✅ Test thoroughly to ensure compatibility with the Notebooks Hub environment.

✅ Consider adding health checks or additional error handling in your `start-dashboard.sh` script.

✅ Consider the [type of dashboarding application](#separation-of-dashboarding-tools-from-the-environment) when adding a new one to ensure proper configuration and integration.

By following these guidelines and the provided example, you can easily extend the functionality of your Notebooks Hub environment with new web-based applications, leveraging the flexibility provided by the `dashboard-base` image.