Skip to content

linlinw5/vlab

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

VLab - Virtual Lab Platform

An education and training virtual lab platform built on VMware vCenter, allowing users to access and operate assigned virtual machines directly from a web browser.

VLab 2.0 is a full-stack rewrite of the original 1.0 version. Version 1.0 was built with Express.js, EJS, and vanilla JavaScript, and relied on a Python-based Netmiko + Flask bridge for ASA communication. In 2.0, the backend was fully rewritten with Express.js, removing the Python dependency, adding a built-in VM console proxy, and introducing a highly automated integration layer so a single API call can trigger end-to-end workflows such as VM clone/delete and user/group create/delete operations. In development mode, the backend enables full debug logging to make troubleshooting easier. The frontend is built with Next.js, and complex background tasks are handled through streaming APIs that push progress updates from the backend to improve the user experience.


English

Project Goal

VLab is designed for enterprise training and vocational education scenarios. It enables administrators to quickly provision VM-based lab environments for users. Learners can connect to VM consoles directly in a browser without installing client software. The platform covers the full workflow: lab template definition, user group assignment, batch VM cloning, and VPN account provisioning.


Features

Administrator Features

  • User and group management: create user accounts and assign users to groups
  • Lab management: define labs, configure categories, and select template VMs
  • Lab assignment: assign labs to target groups and trigger batch cloning; cloned VMs are tagged by group for easier vCenter management
  • VM management: view VMs by group or by user, and perform power on/off/restart/delete operations
  • vCenter browser: browse datacenters, clusters, hosts, and datastore trees online
  • Scheduled tasks: configure cron tasks (for example, timed shutdown and expired VM cleanup)

User Features

  • My VMs: view all assigned VMs and control power state
  • Labs: browse available lab catalog
  • Console access: open fullscreen VM console and operate in real time via WMKS

Other Capabilities

  • JWT authentication with automatic access-token refresh
  • Long-running operations (clone, batch delete) streamed via NDJSON, with real-time toast updates in frontend
  • Cisco ASA integration: auto create/delete VPN accounts

Tech Stack

Layer Technology
Frontend Next.js 16, React 19, Tailwind CSS v4, shadcn/ui
Backend Node.js, Express 5, TypeScript
Database PostgreSQL 17
Virtualization VMware vCenter REST API, WMKS console protocol
Network Cisco ASA SSH, WebSocket proxy
Deployment Nginx (HTTPS reverse proxy), Docker Compose

Architecture

Browser
  |
  +- HTTPS (443) --> Nginx reverse proxy
  |                    |
  |                    +- /api/*  --> Backend :8800 (Express REST API)
  |                    +- /*      --> Frontend :3000 (Next.js)
  |                    +- /esxi/* --> ESXi host (WebSocket console pass-through)
  |
  +- WSS (8843) --> Backend built-in WebSocket proxy --> ESXi WMKS endpoint

Backend (backend/)

Runtime ports: 8800 (REST API) + 8843 (built-in WebSocket proxy)

src/
|- routes/          # API routes (auth / users / groups / labs / vms / assign / cron / vmware)
|- services/
|  |- vcenter/      # vCenter REST wrappers (clone, power, console ticket)
|  |- asa/          # Cisco ASA SSH client for VPN account management
|  |- proxy/        # Built-in HTTPS WebSocket proxy to ESXi console endpoint
|  |- cron/         # Scheduled task management
|- db/
|  |- queries/      # PostgreSQL query wrappers
|- middleware/      # JWT auth, error handling, etc.
|- config/          # Environment configuration loading

Frontend (frontend/)

Runtime port: 3000

app/
|- (dashboard)/     # Sidebar layout, authentication required
|  |- admin/        # Admin pages (users, groups, labs, assignments, VMs, vCenter, cron)
|  |- vms/          # User: my VMs
|  |- labs/         # User: lab list
|- console/[id]/    # Fullscreen VM console (WMKS loaded dynamically)
|- auth/            # Login page

components/
|- ui/              # shadcn/ui base components
|- vm-power-badge   # VM power badge, polling + action buttons
|- ...

lib/
|- api.ts           # Fetch wrapper, JWT refresh handling, streaming support
|- utils.ts

Key design points:

  • API proxy: next.config.ts rewrites /api/* to backend, no local CORS setup needed
  • Streaming toast: streamRequest consumes NDJSON line by line; each step shows a loading toast and then transitions to success/failure
  • VM console: WMKS is loaded dynamically on client side and connects through backend WSS proxy for fullscreen remote console

Deployment Layout

docker/
|- docker-compose.pg.yml     # PostgreSQL 17 container
|- docker-compose.nginx.yml  # Nginx HTTPS reverse proxy + certificate mount

Recommended deployment order in production:

  1. Start PostgreSQL container.
  2. Build backend and frontend artifacts:
  • cd backend && npm install && npm run build
  • cd frontend && npm install && npm run build
  1. Deploy service files:
  • Copy deploy/vlab-backend.service and deploy/vlab-frontend.service to /etc/systemd/system/
  1. Reload and enable services:
  • sudo systemctl daemon-reload
  • sudo systemctl enable --now vlab-backend
  • sudo systemctl enable --now vlab-frontend
  1. Start Nginx container (mount SSL certificates).

Notes:

  • The service files assume backend is deployed at /opt/vlab/backend and frontend at /opt/vlab/frontend.
  • node and npm are expected at /usr/bin/node and /usr/bin/npm.
  • Check runtime logs with journalctl -u vlab-backend -f and journalctl -u vlab-frontend -f.

Recommended production hardware/storage architecture:

  1. Server: VMware ESXi 7.0 or newer.
  2. Shared storage: SAN or NFS.
  3. VAAI (vStorage APIs for Array Integration): to improve VM clone performance, enable VAAI. For NFS storage, install vendor-provided VAAI-NAS plugin (vSphere Installation Bundle, VIB) on ESXi. For SAN storage, VMware usually provides native support, and storage-side firmware declaration is sufficient.
  4. Preferred setup: NetApp + NFS with VAAI plugin on ESXi to enable FlexClone. In this mode, VM clone on NFS datastore does not copy VMDK data blocks directly. ESXi sends a CLONE_FILE request (VAAI-NAS primitive) to the array, and the array completes clone by pointer remap or snapshot reference, typically finishing in 1-2 seconds.
  5. No shared storage: if centralized storage is not available (e.g. standalone ESXi hosts with local disks only), use Linked Clone mode instead. See the Linked Clone Mode section below.

Linked Clone Mode (No Shared Storage)

For deployments where centralized shared storage (SAN / NFS) is not available — such as standalone ESXi hosts with local disks only — VLab supports linked clone as a fast alternative to full VM cloning.

How it works

Linked clone is implemented via the vSphere SOAP API (the REST API does not expose this capability). Instead of copying the full VMDK, the clone gets a thin delta disk that references the source VM's snapshot as its read-only parent. Only changed blocks are written to the delta, so the clone is created almost instantly regardless of disk size.

Source VM snapshot (read-only parent)
        │
        ├── Clone A  (delta disk, a few MB)
        ├── Clone B  (delta disk, a few MB)
        └── Clone C  (delta disk, a few MB)

The clone is pinned to the same ESXi host and datastore as the source VM. The source VM must remain registered and its snapshot must not be deleted as long as any linked clone depends on it.

Enabling linked clone

Set the following environment variable in backend/.env:

VCENTER_LINKED_CLONE=true

When enabled, cloneVM() automatically:

  1. Checks whether the source VM has a snapshot.
  2. Creates a base snapshot automatically if none exists.
  3. Issues a CloneVM_Task SOAP call with diskMoveType: createNewChildDiskBacking.

No frontend changes are required.

Performance comparison

Mode Storage requirement Clone time
Full clone (REST, no VAAI) Any Minutes (copies all data blocks)
Full clone (REST, with VAAI) SAN / NFS + VAAI plugin 1–2 seconds (array-offloaded)
Linked clone (SOAP) Any (including local disk) ~3 seconds (delta descriptor only)

Limitations

  • All clones depend on the source VM's snapshot. Deleting the snapshot breaks all linked clones.
  • Delta disks grow over time as the student writes data; plan datastore capacity accordingly.
  • Linked clones are bound to a single ESXi host; live migration (vMotion) to another host is not supported without promoting to full copy first.
  • Requires vSphere 7.0 or newer with VMFS datastore.

Local Development

# Generate temporary certificates
cd backend/cert
openssl genrsa -out private.key 2048
openssl req -new -key private.key -out certificate.csr
openssl x509 -req -days 3650 -in certificate.csr -signkey private.key -out certificate.crt

# Backend
cd backend
cp .env.example .env   # Fill in vCenter / DB / ASA connection settings
npm install
npm run dev

# Frontend
cd frontend
npm install
npm run dev            # next dev, port 3000

Screenshots

Screenshot 1

Screenshot 2

Screenshot 3

Screenshot 4

Screenshot 5

Screenshot 6

Screenshot 7

License

MIT License, see LICENSE.


中文

项目目的

VLab 面向企业内训、职业教育等场景,让管理员能够快速为用户分配虚拟机实验环境,用户无需安装任何客户端,直接在浏览器中连接虚拟机控制台。平台管理从实验模板定义、用户组分配、虚拟机批量克隆,到 VPN 账号下发,全流程覆盖。

VLab 2.0 是对最初 1.0 版本的整体重写。1.0 采用 Express.js、EJS 和原生 JavaScript 构建,ASA 通信则依赖 Python 的 Netmiko + Flask 进行转接。2.0 中,后端使用 Express.js 全面重写,去掉了对 Python 的依赖,新增了内置的 VM console proxy,并引入高度自动化的集成能力,使得单个接口即可驱动后台一系列虚拟机克隆/删除、用户/组创建/删除等操作。开发环境下,后端会默认打开完整的 debug 日志,便于故障排查。前端则使用 Next.js 构建,对于复杂的后台任务,通过流式接口从后端持续获取消息,以提升用户体验。


功能概览

管理员功能

  • 用户与分组管理:创建用户账号、将用户分配到组
  • 实验室管理:定义实验室,设置实验分类,为实验室选择模板虚拟机
  • 实验室分配:将实验室分配给指定的组,触发批量克隆创建虚拟机,克隆的虚拟机会被打上组标记,以方便在 vCenter 中进行管理
  • 虚拟机管理:按组或按用户查看 VM 列表,执行开机/关机/重启/删除操作
  • vCenter 浏览器:在线浏览 vCenter 数据中心、集群、主机、数据存储树形结构
  • 定时任务:配置 cron 任务(如定时关机、清理过期 VM)

用户功能

  • 我的虚拟机:查看已分配给自己的全部 VM,控制电源状态
  • 实验列表:浏览可用的实验目录
  • 控制台访问:点击进入全屏 VM 控制台,通过 WMKS 协议实时操作虚拟机

其他特性

  • JWT 认证,支持 access token 自动刷新
  • 长操作(克隆、批量删除)通过 NDJSON 流式推送进度,前端以 Toast 实时展示
  • Cisco ASA 集成:自动创建/删除用户 VPN 账号

技术栈

层次 技术
前端 Next.js 16 · React 19 · Tailwind CSS v4 · shadcn/ui
后端 Node.js · Express 5 · TypeScript
数据库 PostgreSQL 17
虚拟化 VMware vCenter REST API · WMKS 控制台协议
网络 Cisco ASA SSH · WebSocket 代理
部署 Nginx(HTTPS 反向代理)· Docker Compose

架构说明

浏览器
  |
  +- HTTPS (443) --> Nginx 反向代理
  |                    |
  |                    +- /api/*  --> Backend :8800 (Express REST API)
  |                    +- /*      --> Frontend :3000 (Next.js)
  |                    +- /esxi/* --> ESXi 主机 (WebSocket 控制台直通)
  |
  +- WSS (8843) --> Backend 内置 WebSocket 代理 --> ESXi WMKS 端点

Backend(backend/)

运行端口:8800(REST API)+ 8843(内置 WebSocket 代理)

src/
|- routes/          # API 路由(auth / users / groups / labs / vms / assign / cron / vmware)
|- services/
|  |- vcenter/      # vCenter REST API 封装(克隆、电源、控制台票据)
|  |- asa/          # Cisco ASA SSH 连接,管理 VPN 账号
|  |- proxy/        # 内置 HTTPS WebSocket 代理,转发浏览器到 ESXi 控制台
|  |- cron/         # 定时任务调度
|- db/
|  |- queries/      # PostgreSQL 查询封装
|- middleware/      # JWT 鉴权、错误处理等中间件
|- config/          # 环境配置加载

Frontend(frontend/)

运行端口:3000

app/
|- (dashboard)/     # 带侧边栏布局,需登录
|  |- admin/        # 管理员页面(用户、组、实验室、作业、VM、vCenter、定时任务)
|  |- vms/          # 用户:我的虚拟机
|  |- labs/         # 用户:实验列表
|- console/[id]/    # 全屏 VM 控制台(动态加载 WMKS 库)
|- auth/            # 登录页

components/
|- ui/              # shadcn/ui 基础组件
|- vm-power-badge   # VM 电源状态徽标,轮询状态 + 操作按钮
|- ...

lib/
|- api.ts           # 封装 fetch,处理 JWT 自动刷新和流式响应
|- utils.ts

关键设计点:

  • API 代理:next.config.ts 将 /api/* 转发至后端,本地开发无需跨域配置
  • 流式 Toast:streamRequest 逐行消费 NDJSON,每一步显示 loading toast,全部完成后统一替换为成功/失败状态
  • VM 控制台:WMKS 库在客户端动态加载,连接后端 WSS 代理隧道,实现全屏远程桌面体验

软件部署结构及硬件架构

docker/
|- docker-compose.pg.yml     # PostgreSQL 17 容器
|- docker-compose.nginx.yml  # Nginx HTTPS 反向代理 + 证书挂载

生产环境推荐部署顺序:

  1. 启动 PostgreSQL 容器。
  2. 构建后端与前端产物:
  • cd backend && npm install && npm run build
  • cd frontend && npm install && npm run build
  1. 部署服务文件:
  • deploy/vlab-backend.servicedeploy/vlab-frontend.service 复制到 /etc/systemd/system/
  1. 重载并启用服务:
  • sudo systemctl daemon-reload
  • sudo systemctl enable --now vlab-backend
  • sudo systemctl enable --now vlab-frontend
  1. 启动 Nginx 容器(挂载 SSL 证书)。

说明:

  • 服务文件默认后端目录为 /opt/vlab/backend,前端目录为 /opt/vlab/frontend
  • 运行环境要求 nodenpm 分别位于 /usr/bin/node/usr/bin/npm
  • 可使用 journalctl -u vlab-backend -fjournalctl -u vlab-frontend -f 查看实时日志。

生产环境硬件部署架构:

  1. 服务器:建议使用 VMware ESXi 7.0 及以上版本。
  2. 集中存储:建议采用 SAN 或 NFS。
  3. VAAI(vStorage APIs for Array Integration)支持:为提升 VM 克隆速度,建议启用 VAAI。对于 NFS 存储,可安装厂商提供的 VAAI-NAS 插件(vSphere Installation Bundle,VIB)到 ESXi;对于 SAN 存储,VMware 通常已原生支持,只需存储阵列在固件层面声明支持即可,无需额外插件。
  4. 推荐方案:采用 NetApp + NFS,并在 ESXi 上安装 VAAI 插件以启用 FlexClone。这样在 NFS 数据存储上克隆 VM 时,ESXi 无需复制 VMDK 文件,而是向阵列发送 CLONE_FILE 请求(VAAI-NAS 原语);阵列在文件系统层通过指针重映射或快照引用完成”克隆”,几乎不搬移数据块,通常可在 1-2 秒内完成。
  5. 无共享存储:如果没有集中存储(例如仅使用本地磁盘的独立 ESXi 主机),请改用 Linked Clone 模式,详见下方 Linked Clone 模式 小节。

Linked Clone 模式(无共享存储)

对于没有集中共享存储(SAN / NFS)的部署场景——例如仅使用本地磁盘的独立 ESXi 主机——VLab 支持 Linked Clone(链接克隆) 作为全量克隆的快速替代方案。

工作原理

Linked Clone 通过 vSphere SOAP API 实现(REST API 不提供该能力)。克隆不复制完整 VMDK,而是为克隆机创建一个薄 delta 磁盘,以源 VM 的快照作为只读父盘;写入操作只发生在 delta 盘上。因此,无论磁盘多大,克隆几乎瞬间完成。

源 VM 快照(只读父盘)
        │
        ├── 克隆 A  (delta 磁盘,仅几 MB)
        ├── 克隆 B  (delta 磁盘,仅几 MB)
        └── 克隆 C  (delta 磁盘,仅几 MB)

克隆机会被固定在与源 VM 相同的 ESXi 主机和数据存储上。只要有链接克隆依赖某个快照,源 VM 必须保持注册状态且该快照不能被删除。

启用方式

backend/.env 中设置以下环境变量:

VCENTER_LINKED_CLONE=true

启用后,cloneVM() 会自动执行以下步骤:

  1. 检查源 VM 是否存在快照。
  2. 如不存在,自动创建一个名为 base 的快照。
  3. 发起 CloneVM_Task SOAP 调用,使用 diskMoveType: createNewChildDiskBacking

前端无需任何修改。

性能对比

模式 存储要求 克隆耗时
全量克隆(REST,无 VAAI) 任意 数分钟(完整复制数据块)
全量克隆(REST,有 VAAI) SAN / NFS + VAAI 插件 1–2 秒(存储阵列卸载)
Linked Clone(SOAP) 任意(含本地磁盘) 约 3 秒(仅创建 delta 描述符)

注意事项

  • 所有克隆机依赖源 VM 的快照,删除该快照会导致所有链接克隆无法启动。
  • Delta 磁盘会随学员写入操作持续增长,需合理规划数据存储容量。
  • 链接克隆绑定在单台 ESXi 主机上,在不升级为全量复制的前提下,不支持跨主机实时迁移(vMotion)。
  • 需要 vSphere 7.0 及以上版本,且使用 VMFS 数据存储。

本地开发

# 生成临时证书
cd backend/cert
openssl genrsa -out private.key 2048
openssl req -new -key private.key -out certificate.csr
openssl x509 -req -days 3650 -in certificate.csr -signkey private.key -out certificate.crt

# 后端
cd backend
cp .env.example .env   # 填写 vCenter / DB / ASA 连接信息
npm install
npm run dev

# 前端
cd frontend
npm install
npm run dev            # next dev,端口 3000

许可证

MIT 许可,见 LICENSE

About

A virtualization lab management system built with Express.js and NextJS

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors