The introduction of massive sequencing (MS) in genomics facilities has meant an exponential growth in data generation, requiring a precise tracking system, from library preparation to fastq file generation, analysis and delivery to the researcher. Software designed to handle those tasks are called Laboratory Information Management Systems (LIMS), and its software has to be adapted to their own genomics laboratory particular needs. iSkyLIMS is born with the aim of helping with the wet laboratory tasks, and implementing a workflow that guides genomics labs on their activities from library preparation to data production, reducing potential errors associated to high throughput technology, and facilitating the quality control of the sequencing. Also, iSkyLIMS connects the wet lab with dry lab facilitating data analysis by bioinformaticians.
According to existent infrastructure sequencing is performed on an Illumina NextSeq instrument. Data is stored in NetApp mass storage device and fastq files are generated (bcl2fastq) on a Sun Grid Engine High Performance Computing cluster (SGE-HPC). Application servers run web applications for bioinformatics analysis (GALAXY), the iSkyLIMS app, and host the MySQL information tier. iSkyLIMS WetLab workflow deals with sequencing run tracking and statistics. Run tracking passes through five states: "recorded” genomics user record the new sequencing run into the system, the process will wait till run is completed by the machine and data is transferred to the mass storage device; “Sample sheet sent” sample sheet file with the sequencing run information will be copied to the run folder for bcl2fastq process; “Processing data” run parameters files are processed and data is stored in the database; “Running stats” demultiplexing data generated in bcl2fastq process is processed and stored into the database, “Completed” all data is processed and stored successfully. Statistics per sample, per project, per run and per investigation are provided, as well as annual and monthly reports. iSkyLIMS DryLab workflow deals with bioinformatics services request and statistics. User request services that can be associated with a sequencing run. Stats and services tracking is provided.
- iSkyLIMS
For any problems or bug reporting please post us an issue
All installation paths assume you already cloned the repository:
git clone https://github.com/BU-ISCIII/iskylims.git iskylims
cd iskylims- Docker (local test): spin up MySQL + Samba + iSkyLIMS with demo data to try the app quickly.
- Docker (production container): deploy only the application container, pointing to your existing DB/Samba.
- Bare-metal: install or upgrade directly on Ubuntu/CentOS hosts with
install.sh.
Container deployment requirements:
- Docker Engine + Docker Compose v2, or Podman +
podman-compose - git >= 2.34 to clone/update the repository
- Host MySQL/MariaDB, Apache, Python, and
lsb_releaseare not required for container deployment - For local test containers: MySQL and Samba are started as containers by
container_install.sh --test - For production containers: access to an external MySQL/MariaDB server and Samba share configured in the selected install config
- Host directories and permissions for logs, documents, and static files, as described in Persist logs/documents on the host
Bare-metal deployment requirements:
- sudo privileges for dependency installation
- MySQL >= 8.0 or MariaDB > 10.4
- Apache >= 2.4
- git >= 2.34
- Python >= 3.11
- Local email sender configured
- Access to the Samba share where run folders live
lsb_releasepackage:- RedHat/CentOS:
yum install redhat-lsb-core - Ubuntu:
apt install lsb-core lsb-release
- RedHat/CentOS:
Bring up a full test stack (database, Samba, app) plus fixtures and demo data:
bash container_install.sh --test 2>&1 | tee test.logUse --engine podman to run the same flow with Podman:
bash container_install.sh --test --engine podman 2>&1 | tee test.logThis uses docker-compose.test.yml by default.
Defaults can be customised:
--demo_data /path/to/iskylims_demo_data.tar.gzto reuse a local demo archive (otherwise it is downloaded).--skip_demo_dataor--skip_test_datato avoid loading extra data.--install_type(fullby default) and--git_revisionto control the build.--scriptto run one or more Django migration scripts throughinstall.sh(repeat the flag as needed).
Example running a migration script during Docker install:
bash container_install.sh --test --script migrate_optional_values 2>&1 | tee test.logWhen the script finishes, open http://localhost:8001 and follow the prompt to create the Django superuser.
The image now includes the staged application tree under ${APP_INSTALL_PATH}. Test containers can therefore be recreated or restarted without rerunning the file installation step; only DB/bootstrap tasks are executed by container_install.sh.
Deploy the iSkyLIMS container against external MySQL/Samba services:
-
Copy and edit the production settings template:
cp conf/docker_production_settings.txt conf/my_prod_settings.txt # edit conf/my_prod_settings.txt with your DB/Samba details -
Build and run in production mode (uses
docker-compose.prod.ymlby default):bash container_install.sh --install_conf conf/my_prod_settings.txt 2>&1 | tee ./iskylims_docker_install_$(date +%Y%m%d_%H%M%S).log
Use
--compose_fileto override the compose file or--install_type/--git_revisionto change the build. Add--engine podmanto use Podman instead of Docker. Tip: capture logs for troubleshooting:bash container_install.sh --install_conf conf/my_prod_settings.txt 2>&1 | tee ./iskylims_docker_install_$(date +%Y%m%d_%H%M%S).log
-
If this is a fresh install, create the Django superuser when prompted and complete the Samba configuration in the UI.
Production images now bake the staged iSkyLIMS application into the image itself. Host reboots or container recreation no longer require rerunning the app installation step; container_install.sh only performs runtime bootstrap tasks such as migrations, fixture refreshes, optional scripts, superuser creation on first install, and collectstatic.
Container build/runtime values are configured in the selected install config, not by exporting shell variables. Edit these fields in conf/my_prod_settings.txt before running container_install.sh:
APP_INSTALL_PATH: runtime install root used by the app container, static/documents mounts, and install scripts. Leave empty to reuseINSTALL_PATH.APACHE_CONF_PATH: host directory used for Apache bind-mounted config files. Leave empty to use${APP_INSTALL_PATH}/conf.DJANGO_SETTINGS_PATH: host path used for the bind-mounted Djangosettings.py. Leave empty to use${APP_INSTALL_PATH}/iskylims/settings.py. If the value is a directory or ends with/,container_install.shappendssettings.py.APP_UID/APP_GID: runtime UID/GID for theiskylimsuser inside the container. Default:1212:1212.APP_SHELL: shell assigned to the runtime user during image build. Default:/sbin/nologin.APP_PORT: internal Gunicorn bind port for theappservice. Default:8001.DJANGO_DEBUG: Django debug flag passed to the production app container. Default:false; keep it disabled in production.DB_CONN_MAX_AGE: Django persistent DB connection lifetime in seconds. Default:60.WEB_CONCURRENCY: Gunicorn worker count. Default:2.GUNICORN_THREADS: threads per Gunicorn worker. Default:2.GUNICORN_TIMEOUT: Gunicorn request timeout in seconds. Default:300.GUNICORN_KEEPALIVE: Gunicorn keep-alive in seconds. Default:5.
During production install/upgrade, container_install.sh writes .env.prod.file in the repository root. This file is ignored by git and is used by Compose for variable interpolation in docker-compose.prod.yml. It intentionally contains Compose/runtime metadata, not database or email passwords.
Host directory and ownership preparation is described in Persist logs/documents on the host.
The production compose file uses INSTALL_PATH from the selected install config, or APP_INSTALL_PATH if exported, as the runtime root for the app and for the Apache config/static mounts.
Persistence layout:
/var/log/local/iskylims/apps->${INSTALL_PATH}/logsinside theappcontainer/var/log/local/iskylims/apache->/var/log/httpdinside theapachecontainer${APACHE_CONF_PATH:-${INSTALL_PATH}/conf}/iskylims_apache_reverse_proxy.conf->/etc/httpd/conf.d/iskylims.confinside theapachecontainer${APACHE_CONF_PATH:-${INSTALL_PATH}/conf}/iskylims_apache_logs.conf->/etc/httpd/conf.d/logformat.confinside theapachecontainer${DJANGO_SETTINGS_PATH:-${INSTALL_PATH}/iskylims/settings.py}->${INSTALL_PATH}/iskylims/settings.pyinside theappcontaineriskylims_documentsnamed volume ->${INSTALL_PATH}/documentsiskylims_staticnamed volume ->${INSTALL_PATH}/static
If you override the compose file, ensure these mounts exist to keep logs and documents persistent.
Create host directories before the first deployment:
sudo mkdir -p /var/log/local/iskylims/apps
sudo mkdir -p /var/log/local/iskylims/apache
sudo mkdir -p <APP_INSTALL_PATH>/conf
sudo chown -R <APP_UID>:<APP_GID> /var/log/local/iskylims/apps <APP_INSTALL_PATH>For hardened/rootless Podman hosts, run the host preparation script as the same user that starts the containers. The script pre-creates Apache log files, fixes rootless Podman ownership for the UBI httpd user, and applies SELinux container labels when SELinux is enabled:
bash hardening.shIf an administrator runs it as root, set PODMAN_USER to the user that starts
the rootless containers:
PODMAN_USER=bioinfo bash hardening.shFor production, the app container runs gunicorn (not manage.py runserver) and the apache service in docker-compose.prod.yml acts as the reverse proxy.
Static files:
- The app collects static files into
${INSTALL_PATH}/static. docker-compose.prod.ymlshares that directory with theapacheservice through the named volumeiskylims_static.- The reverse proxy config serves
/staticdirectly from${INSTALL_PATH}/static.
During container_install.sh, conf/iskylims_apache_reverse_proxy.conf and conf/iskylims_apache_logs.conf are rendered and copied to ${APACHE_CONF_PATH} on the host. If APACHE_CONF_PATH is empty, they are copied to ${INSTALL_PATH}/conf. The reverse proxy ServerName, forwarded host, and access/error log file names are generated from DNS_URL in the selected install config. Edit those copied files for runtime Apache changes after deployment.
container_install.sh prepares a host-side Django settings.py bind source at ${DJANGO_SETTINGS_PATH}, or at ${INSTALL_PATH}/iskylims/settings.py when DJANGO_SETTINGS_PATH is empty. During the bootstrap step, install.sh updates that bind-mounted file from conf/template_settings.txt and the selected install config, preserving an existing SECRET_KEY. Runtime settings can then be edited and the container restarted without rebuilding the image.
If you need a different runtime root, set INSTALL_PATH in the install config file or export APP_INSTALL_PATH before running container_install.sh. If you need Apache configs or Django settings outside the app runtime root, set APACHE_CONF_PATH or DJANGO_SETTINGS_PATH in the install config file, or export them before running container_install.sh.
container_install.sh creates ${APP_INSTALL_PATH}/conf, ${APACHE_CONF_PATH:-${APP_INSTALL_PATH}/conf}, /var/log/local/iskylims/apps, and /var/log/local/iskylims/apache before compose up, copies both Apache config files there, prepares the bind-mounted Django settings file if it does not exist, passes APP_INSTALL_PATH, APACHE_CONF_PATH, and DJANGO_SETTINGS_PATH into Compose, and then runs install.sh --bootstrap ... inside the app container. The container image already contains the staged Django project and virtualenv under ${APP_INSTALL_PATH}; the bootstrap step updates settings, applies migrations, optional scripts/fixtures, and refreshes ${INSTALL_PATH}/static, while the Apache container keeps using the host log path /var/log/local/iskylims/apache.
SELinux note for pre-production and production:
- Ensure
/var/log/local/iskylims/apacheis writable by the container runtime and labeled for containers, for examplecontainer_file_t. - If the host path is already labeled
container_file_t, do not add:Zto the Apache log bind mount.:Zforces a relabel and may fail withlsetxattr(... container_file_t ...): operation not permitted. - A quick check is:
ls -ldZ /var/log/local/iskylims/apache- Expected example:
system_u:object_r:container_file_t:s0
- If Apache fails on startup with
ModSecurity: Failed to open debug log file: /var/log/httpd/modsec_debug.log, remove any stale host file and recreate/restart the container. In practice, deleting/var/log/local/iskylims/apache/modsec_debug.loghas been enough when the existing inode had bad permissions/label state.
Cron runs via supercronic, started by the container entrypoint script. The script writes the django-crontab entries to ${APP_INSTALL_PATH}/cron/iskylims and starts supercronic as the non-root app user.
If you change CRONJOBS, rebuild or restart the container to regenerate the cron file.
After a production install, use the generated .env.prod.file whenever you run Compose directly. This keeps paths, UID/GID, ports, and Gunicorn settings aligned with the install config.
Docker Compose examples:
docker compose --env-file .env.prod.file -f docker-compose.prod.yml ps
docker compose --env-file .env.prod.file -f docker-compose.prod.yml logs --tail 200 app
docker compose --env-file .env.prod.file -f docker-compose.prod.yml restart app
docker compose --env-file .env.prod.file -f docker-compose.prod.yml up -dPodman Compose examples:
podman compose --env-file .env.prod.file -f docker-compose.prod.yml ps
podman compose --env-file .env.prod.file -f docker-compose.prod.yml logs --tail 200 app
podman compose --env-file .env.prod.file -f docker-compose.prod.yml restart app
podman compose --env-file .env.prod.file -f docker-compose.prod.yml up -dIf you edit container runtime values in the install config, rerun container_install.sh --install_conf <file> so .env.prod.file and the running containers are regenerated consistently.
Keep the same APP_UID/APP_GID values in the selected install config before running an upgrade.
Re-deploy the application container against an existing production database:
bash container_install.sh --install_conf conf/my_prod_settings.txt --action upgrade 2>&1 | tee ./iskylims_docker_install_$(date +%Y%m%d_%H%M%S).logThe upgrade path rebuilds/restarts the container and runs install.sh --bootstrap upgrade --tables inside the app container. The app files are already baked into the rebuilt image; the bootstrap phase applies migrations with --fake-initial, refreshes conf/first_install_tables.json, refreshes static files, and skips superuser/demo/test data loading.
Run the backup steps in Backups first.
For 3.0.0 -> 3.1.0, export the LibraryPool mapping first, then run the upgrade with pre/post scripts:
mysql --user=<db_user> --password=<db_password> --host=<db_server_ip> --port=<db_port> iskylims \
-e "SELECT id, run_process_id_id FROM wetlab_library_pool" \
> /tmp/library_pool_run_process.tsvcd <your working directory>/iskylims
git pull
cp conf/docker_production_settings.txt myprod_settings.txt
sudo nano myprod_settings.txtEnsure the file uses Linux-friendly encoding (UTF-8/ASCII) if you edit it on Windows.
Keep the same APP_UID/APP_GID values in the selected install config before running the 3.0.0 -> 3.1.0 upgrade.
Run upgrade command:
bash container_install.sh --engine podman --install_conf my_prod_settings.txt --action upgrade \
--script_before convert_rawtop_counter_to_int \
--script_after library_pool_to_many_relation,/tmp/library_pool_run_process.tsv 2>&1 | tee ./iskylims_docker_install_$(date +%Y%m%d_%H%M%S).logcd <your working directory>
git clone https://github.com/BU-ISCIII/iskylims.git iskylims
cd iskylimsCreate the database and application user following Database creation, users and grants, then note DB host/port/user/password for install_settings.txt.
cp conf/template_install_settings.txt install_settings.txt
nano install_settings.txtSet your database, email, server IP/URL, and logging preferences in that file.
iSkyLIMS is installed to /opt/iskylims by default. The single install.sh script handles both dependencies and the app; choose what you need with --install:
dep: install system and Python dependencies (requires sudo).app: deploy iSkyLIMS code, update settings, run migrations, and collect static files (no sudo needed).full: run both stages in sequence.
The staged/bootstrap split added for container images is internal. Bare-metal commands do not change: --install and --upgrade still run the complete dependency, application, and database workflow documented here.
Examples:
# only software dependencies
sudo bash install.sh --install dep
# only iSkyLIMS application
bash install.sh --install app --git_revision main --tables
# dependencies + application
sudo bash install.sh --install full --git_revision main --tables-
Add
--tablesto load the initial fixtures on first-time installs, or--skip_tablesif you want to skip them. -
Capture logs for troubleshooting with
tee:sudo bash install.sh --install full --git_revision main --tables 2>&1 | tee ./iskylims_install_$(date +%Y%m%d_%H%M%S).log
-
If Apache is managed elsewhere, skip the automatic restart with
--skip_apache_restart.
Follow these steps to move from version 3.0.0 to the 3.1.x series.
Run the backup steps in Backups first.
-
Additionally, back up the full installation folder (for example
/opt/iskylims) for bare-metal rollback. -
If you use library pools, export them before upgrading:
mysql --user=<db_user> --password=<db_password> --host=<db_server_ip> --port=<db_port> iskylims
-e "SELECT id, run_process_id_id FROM wetlab_library_pool" \
/tmp/library_pool_run_process.tsv
#### Refresh code and settings
```bash
cd <your working directory>/iskylims
git pull
cp conf/template_install_settings.txt install_settings.txt
sudo nano install_settings.txt
Ensure the file uses Linux-friendly encoding (UTF-8/ASCII) if you edit it on Windows.
Update system and Python dependencies:
sudo bash install.sh --upgrade dep 2>&1 | tee install_full.logMake sure the installation directory permissions allow the non-root step to write to /opt/iskylims (adapt your hardening script if paths changed).
Upgrade the application code and database:
# with library pool restore
bash install.sh --upgrade app --git_revision main \
--script_before convert_rawtop_counter_to_int \
--script_after library_pool_to_many_relation,/tmp/library_pool_run_process.tsvUpgrades regenerate migrations and apply them with --fake-initial so existing tables remain intact, matching the Docker workflow.
Run as MySQL root:
CREATE DATABASE IF NOT EXISTS iskylims CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER IF NOT EXISTS 'iskylims'@'%' IDENTIFIED BY 'djangopass';
CREATE USER IF NOT EXISTS 'iskylims'@'localhost' IDENTIFIED BY 'djangopass';
GRANT ALL PRIVILEGES ON iskylims.* TO 'iskylims'@'%';
GRANT ALL PRIVILEGES ON iskylims.* TO 'iskylims'@'localhost';
FLUSH PRIVILEGES;Verification:
SHOW GRANTS FOR 'iskylims'@'%';Database dump:
mysqldump -h <db_host> -P <db_port> -u iskylims -p iskylims > iskylims_$(date +%Y%m%d_%H%M%S).sqlLogs archive:
tar -czf iskylims_app_logs_$(date +%Y%m%d_%H%M%S).tgz -C /var/log/local/iskylims/apps .
tar -czf iskylims_apache_logs_$(date +%Y%m%d_%H%M%S).tgz -C /var/log/local/iskylims/apache .Documents volume archive:
docker run --rm -v iskylims_documents:/from -v "$PWD":/to alpine \
tar -czf /to/iskylims_documents_$(date +%Y%m%d_%H%M%S).tgz -C /from .With Podman, use the same command replacing docker with podman.
Suggested order before upgrades:
- DB dump
- Documents volume archive
- Logs archive
Restore DB:
mysql -h <db_host> -P <db_port> -u iskylims -p iskylims < iskylims_YYYYMMDD_HHMMSS.sqlRestore documents volume:
docker run --rm -v iskylims_documents:/to -v "$PWD":/from alpine \
sh -lc "cd /to && tar -xzf /from/iskylims_documents_YYYYMMDD_HHMMSS.tgz"With Podman, use the same command replacing docker with podman.
Restore logs:
mkdir -p /var/log/local/iskylims/apps
tar -xzf iskylims_app_logs_YYYYMMDD_HHMMSS.tgz -C /var/log/local/iskylims/apps
mkdir -p /var/log/local/iskylims/apache
tar -xzf iskylims_apache_logs_YYYYMMDD_HHMMSS.tgz -C /var/log/local/iskylims/apacheBare-metal full rollback example:
sudo rm -rf /opt/iskylims
sudo cp -r /home/dadmin/backup_prod/iSkyLIMS/ /opt/
sudo /scripts/hardening.sh
mysql -u iskylims -p -h <db_host> iskylims < /home/dadmin/backup_prod/bk_iSkyLIMS_YYYYMMDDHHMM.sqlWhen install/upgrade fails, restore the previous state and retry with logs enabled.
Quick diagnostics:
# bare-metal
cd /opt/iskylims
python manage.py check
# docker
docker compose --env-file .env.prod.file -f docker-compose.prod.yml ps
docker compose --env-file .env.prod.file -f docker-compose.prod.yml logs --tail 200 app
# podman
podman compose --env-file .env.prod.file -f docker-compose.prod.yml ps
podman compose --env-file .env.prod.file -f docker-compose.prod.yml logs --tail 200 appIf you suspect a corrupted image/build cache in Docker:
docker compose --env-file .env.prod.file -f docker-compose.prod.yml build --no-cache app
docker compose --env-file .env.prod.file -f docker-compose.prod.yml up -d --force-recreate app- Login with admin account.
- Go to Massive sequencing
{width:50px} - Go to Configuration -> Samba configuration
- Fill the form with the appropiate params for the samba shared folder:

- Go to Massive sequencing
- Go to Configuration -> Email configuration
- Fill the form with the needed params for your email configuration and try to send a test email.
Migrations are committed to the repo. Do not run makemigrations during install/upgrade.
Baseline + upgrade flow for new releases:
- Generate baseline migrations from the last stable tag (example 3.0.0).
- Commit the baseline migrations.
- Generate new migrations on
developfor schema changes and commit them. - Upgrades run
migrate --fake-initialonce to align existing tables, thenmigrateto apply the new migration files.
See Persist logs/documents on the host in the production deployment section.
These steps apply to bare-metal Apache installations. Docker production deployments use the apache container described above and do not require copying configs into /etc/apache2 or /etc/httpd.
Copy the apache configuration file according to your distribution inside the apache configuration directory and rename it to iskylims.conf.
Typical config locations:
- Ubuntu/Debian:
/etc/apache2/sites-available/iskylims.conf(enable witha2ensite) - CentOS/RHEL:
/etc/httpd/conf.d/iskylims.conf
Suggested steps (host Apache as reverse proxy):
-
Copy the example config:
sudo cp conf/iskylims_apache_reverse_proxy.conf /etc/apache2/sites-available/iskylims.conf # CentOS/RHEL: # sudo cp conf/iskylims_apache_reverse_proxy.conf /etc/httpd/conf.d/iskylims.conf
-
Edit the config:
- Set
ServerName - Ensure
ProxyPasspoints tohttp://localhost:8001/ - Ensure
Alias /static/ /opt/iskylims/static/
- Set
-
Create the static folder on the host:
sudo mkdir -p /opt/iskylims/static
-
Enable required modules (Ubuntu/Debian):
sudo a2enmod proxy proxy_http headers sudo a2ensite iskylims.conf
-
Reload Apache:
sudo systemctl reload apache2 # CentOS/RHEL: # sudo systemctl reload httpd
Open the navigator and type "localhost" or the "server local IP" and check that iSkyLIMs is running.
You can also check some of the functionality, while also checking samba and database connections using:
- Go to configuration test
- Click submit
- Check all tabs so every connectin is successful.
- Run the 3 tests for each sequencing machine: MiSeq, NextSeq and NovaSeq.
iSkyLIMS documentation is available at https://iskylims.readthedocs.io/en/latest
