diff --git a/custom-words.txt b/custom-words.txt index 65429e19b..e09278e75 100644 --- a/custom-words.txt +++ b/custom-words.txt @@ -114,6 +114,7 @@ ungated unsynchronized WCAG weweave +worktree Xcodes.app xcworkspace xmldoc diff --git a/docs/getting-started/server/advanced-setup.md b/docs/getting-started/server/advanced-setup.md index 628c7c066..cd648aafd 100644 --- a/docs/getting-started/server/advanced-setup.md +++ b/docs/getting-started/server/advanced-setup.md @@ -46,7 +46,7 @@ subscription. ## Emails -Docker compose will spin up a local smtp server, Mailcatcher, that can be used. See the +Aspire and Docker Compose both start a local SMTP server, Mailcatcher, that can be used. See the [Setup Guide](./guide.md#mailcatcher) for more information about Mailcatcher. It’s also possible to use other services such as Mailtrap, or Amazon to debug the amazon @@ -63,8 +63,8 @@ integration. File uploads are stored using one of two methods. - Azure Storage is used by our production cloud instance. - - Docker will create a local [Azurite](https://github.com/Azure/Azurite) instance which emulates - the Azure Storage API. And is used for the primary testing. + - Aspire or Docker will create a local [Azurite](https://github.com/Azure/Azurite) instance which + emulates the Azure Storage API. And is used for the primary testing. - We also have a test Azure Storage account for development use. The user secrets for this are attached to the the "Server User Secrets" shared vault item. You'll need to copy the `send` and `attachment` keys into your own user secrets. @@ -136,13 +136,16 @@ The steps for setting up your local server for YubiKey validation are: dotnet user-secrets set globalSettings:yubico:key [Key] dotnet user-secrets set globalSettings:yubico:clientid [ClientId] ``` + If you use Aspire, add these values to `dev/secrets.json` instead. Aspire clears and re-applies + user secrets from that file every time it starts. ## Reverse proxy setup Running a reverse proxy can be used to simulate running multiple server services in a distributed manner. The [Docker Compose](https://docs.docker.com/compose/) configuration in the `/dev` folder already has a configuration prepared for the Api and Identity services (can be expanded for other -services). +services). Aspire doesn't start the reverse proxy or extra service instances, so start them as +described below. 1. The reverse proxy container is setup to use an [nginx](https://nginx.org/en/docs/beginners_guide.html#conf_structure) config file located at diff --git a/docs/getting-started/server/database/ef/index.mdx b/docs/getting-started/server/database/ef/index.mdx index 69b975106..b3c8e4cf8 100644 --- a/docs/getting-started/server/database/ef/index.mdx +++ b/docs/getting-started/server/database/ef/index.mdx @@ -50,6 +50,13 @@ The workflow here is broadly the same as with the normal MSSQL implementation: s container, configure user secrets, and run migrations against their relating databases in chronological order. +:::note + +The Aspire `AppHost` only provides MSSQL. Use the [manual setup](../../guide.md#run-manually) and +the Docker Compose instructions below for Entity Framework databases. + +::: + ### Requirements - A working local development server ([see steps](../../guide.md)) diff --git a/docs/getting-started/server/database/mssql/index.md b/docs/getting-started/server/database/mssql/index.md index a757f5086..212291005 100644 --- a/docs/getting-started/server/database/mssql/index.md +++ b/docs/getting-started/server/database/mssql/index.md @@ -20,6 +20,10 @@ run migrations. You should run the helper script whenever you sync with the `mai a new migration script. Migrations that have already been run are tracked in the `Migration` table of your database. +If you run the server with [Aspire](../../guide.md#run-with-aspire), the `run-db-migrations` +resource runs this script every time the AppHost starts. To apply new migrations without restarting +the AppHost, restart `run-db-migrations` from the Aspire dashboard. + ## Modifying the database The process for modifying the database is described in diff --git a/docs/getting-started/server/events.md b/docs/getting-started/server/events.md index 7d30f0f8b..68b791476 100644 --- a/docs/getting-started/server/events.md +++ b/docs/getting-started/server/events.md @@ -47,6 +47,13 @@ For a detailed look at the architecture and technical details, see for writing events to Azure Table Storage). In addition, this assumes you're using the `mssql` default profile and have the `${MSSQL_PASSWORD}` set via `.env`. + :::note + + The emulator connects to the Docker Compose `mssql` container, so it doesn't work with the SQL + Server that Aspire starts. Use the [manual setup](./guide.md#run-manually) for Azure Service Bus. + + ::: + 2. Run Docker Compose to add/start the local emulator: ```bash @@ -204,7 +211,8 @@ To emulate this locally: default value of `UseDevelopmentStorage=true` 3. Start the Events and EventsProcessor projects using `dotnet run` or your IDE. (Also ensure you - have Api, Identity and your web vault running.) + have Api, Identity and your web vault running.) Aspire starts Events and EventsProcessor for + you. You should now observe that your enterprise organization is logging events (e.g. when creating an item or inviting a user). These should appear in the Event Logs section of the organization vault. diff --git a/docs/getting-started/server/guide.md b/docs/getting-started/server/guide.md index 0237bec51..c19346f1f 100644 --- a/docs/getting-started/server/guide.md +++ b/docs/getting-started/server/guide.md @@ -9,6 +9,19 @@ This page will show you how to set up a local Bitwarden server for development p The Bitwarden server is comprised of several services that can run independently. For a basic development setup, you will need the **Api** and **Identity** services. +There are two ways to run the server locally: + +- **[Aspire](#run-with-aspire) (recommended):** the `AppHost` project in the server repository uses + [.NET Aspire](https://aspire.dev) to start the supporting containers, apply your user secrets, + migrate the database, and run every server service from a single command. +- **[Manual setup](#run-manually):** you start the Docker Compose containers, run the helper + scripts, and launch each service yourself. Use this if you need an + [Entity Framework database](./database/ef/index.mdx) or another container that Aspire doesn't + manage. + +Both approaches share the same first steps and use the same `dev/secrets.json` file, so you can +switch between them. + :::info Before you start: make sure you’ve installed the recommended @@ -51,23 +64,42 @@ cd server you can run `dotnet format` from the command line when convenient (e.g. before requesting a PR review). -## Configure Docker +## Configure user secrets -We provide a [Docker Compose](https://docs.docker.com/compose/) configuration, which is used during -development to provide the required dependencies. This is split up into multiple service profiles to -facilitate easy customization. +[User secrets](https://docs.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-6.0) +are a method for managing application settings on a per-developer basis. They override the settings +in `appSettings.json` of each project. Your user secrets file should match the structure of the +`appSettings.json` file for the settings you intend to override. -1. Some Docker settings are configured in the environment file, `dev/.env`. Copy the example - environment file: +We provide a helper script which simplifies setting user secrets for all projects in the server +repository. + +1. Get a template `secrets.json`. We need to get an initial version of `secrets.json`, which you + will modify for your own secrets values. + + + + Navigate to the `dev` folder in your server repo and copy the example `secrets.json` file. ```bash cd dev - cp .env.example .env + cp secrets.json.example secrets.json ``` -2. Open `.env` with your preferred editor. + -3. Set the `MSSQL_PASSWORD` variable. This will be the password for your MSSQL database server. + + - Copy the user secrets file from the shared Development collection (Your Bitwarden Vault) into + the `dev` folder. + - If you don't have access to the Development collection, contact our IT Manager to arrange + access. Make sure you have first set up a Bitwarden account using your company email address. + - This `secrets.json` is configured to use the local Azurite and MailCatcher instances and is + recommended for this guide. + + + +2. Choose a password for your local SQL Server. You will use it for the SQL Server `sa` account in + either setup. :::caution @@ -83,188 +115,264 @@ facilitate easy customization. ::: -4. You can change the other variables or use their default values. Save and quit this file. -5. Start the Docker containers. - - Using PowerShell, navigate to the cloned server repo location, into the `dev` folder and run the - docker command below. +3. Update `secrets.json` with your own values: + - `sqlServer` > `connectionString`: insert your SQL Server password where indicated + - `installation` > `id` and `key`: + [request a hosting installation Id and Key](https://bitwarden.com/host/) and insert them here + - `licenseDirectory`: set this to an empty directory, this is where uploaded license files will + be stored. + + + + + - `licenseCertificatePath` and `licenseCertificatePassword`: to run your local server as a + licensed instance, download the `Licensing Certificate - Dev` from the shared Engineering + collection: + 1. Log in to your company-issued Bitwarden account + 2. On the "Vaults" page, scroll down to the "Licensing Certificate - Dev" item + 3. View attachments and only download `dev.pfx` + 4. Create a `~/.secrets/` folder and place `dev.pfx` inside + 5. Add the following under `globalSettings` in `secrets.json`, substituting your own path to + `dev.pfx` and the password from the "Licensing Certificate - Dev" vault item: + + ```json + "licenseCertificatePath": "/Users//.secrets/dev.pfx", + "licenseCertificatePassword": "" + ``` + + :::warning + + Do not import the "Licensing Certificate - Dev" into your keychain. Doing so may compromise all + TLS traffic on your development machine. + + ::: + + + +4. Once you have your `secrets.json` complete, run the below command from the `dev` folder to add + the secrets to each Bitwarden server project. ```bash - docker compose --profile mssql --profile mail up -d + pwsh setup_secrets.ps1 ``` - Which starts the MSSQL and local mail server containers, which should be suitable for most - community contributions. +The helper script also supports an optional `-clear` switch which removes all existing settings +before re-applying them: - +```bash +pwsh setup_secrets.ps1 -clear +``` - +:::note + +Aspire runs `setup_secrets.ps1 -clear` every time it starts. If you use Aspire, make your changes in +`dev/secrets.json` rather than with `dotnet user-secrets set` on an individual project, because +per-project changes are cleared on the next run. + +::: + +## Run with Aspire + +The `AppHost` project orchestrates the whole local environment. When you start it, Aspire: + +- starts SQL Server, Azurite, MailCatcher, and Redis containers +- applies `dev/secrets.json` to every project by running `setup_secrets.ps1 -clear` +- creates and migrates the database by running `migrate.ps1` +- bootstraps Azurite by running `setup_azurite.ps1` +- starts the Admin, Api, Billing, Events, EventsProcessor, Icons, Identity, Notifications, Scim, and + Sso services once their dependencies are ready + +The [AppHost README](https://github.com/bitwarden/server/blob/main/AppHost/README.md) contains the +full configuration reference. + +### Set up Aspire + +1. Stop any Docker Compose containers from a previous manual setup. Aspire uses the same host ports + (for example 1433 for SQL Server and 1080 for MailCatcher) and won't start while they are in + use. From the `dev` folder, run: ```bash - docker compose --profile cloud --profile mail up -d + docker compose --profile cloud --profile mail --profile idp down ``` - Which starts MSSQL, mail, Redis, and Azurite container. The additional Azurite container is - required to emulate Azure used by the Bitwarden cloud environment. + :::note - + Aspire creates its own SQL Server data volume. Data in your Docker Compose database is not + carried over. -After you’ve run the `docker compose` command, you can use the -[Docker Dashboard](https://docs.docker.com/desktop/dashboard/) to manage your containers. You should -see your containers running under the `bitwardenserver` group. + ::: -:::caution +2. Install the PowerShell `Az` module, which the Azurite setup script needs. This may take a few + minutes to complete without providing any user feedback (it may appear frozen). -Changing `MSSQL_PASSWORD` variable after first running docker compose will require a re-creation of -the storage volume. + ```bash + pwsh -Command "Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force" + ``` -**Warning: this will delete your development database.** +3. Trust the ASP.NET Core development certificate, which the Aspire dashboard uses for HTTPS: -To do this, run + ```bash + dotnet dev-certs https --trust + ``` -```bash -docker compose --profile mssql down -docker volume rm bitwardenserver_mssql_dev_data -``` +4. From the `dev` folder, navigate to the `AppHost` folder and store your SQL Server password in + the AppHost's user secrets. This must be the same password you put in the `sqlServer` connection + string in `secrets.json`. -After that, rerun the docker compose command from Step 5. + ```bash + cd ../AppHost + dotnet user-secrets set "Database:Password" "" + ``` -::: + :::warning -### SQL Server + Keep passwords and tokens in user secrets. Don't add them to + `AppHost/appsettings.Development.json`, which is checked in. -You can connect to the Microsoft SQL Server using your preferred database management tool with the -following credentials: + ::: -- Server: localhost -- Port: 1433 -- Username: sa -- Password: (the password you set in `dev/.env`) +### Start the server -### Mailcatcher +1. From the `AppHost` folder, run: -The server uses emails for many user interactions. We provide a pre-configured instance of -[MailCatcher](https://mailcatcher.me/), which catches any outbound email and prevents it from being -sent to real email addresses. You can open its web interface at -[http://localhost:1080](http://localhost:1080). + ```bash + dotnet run + ``` -### Redis + :::tip - + If you have the [Aspire CLI](https://aspire.dev) installed, you can run `aspire run` from the + root of the repository instead. -:::note + ::: -Redis is required for Email and Authenticator two-factor authentication flows, as well as OTP -validation for new device and user verification scenarios when developing for the `cloud` profile. +2. The Aspire dashboard opens in your browser at + [https://localhost:17271](https://localhost:17271). It shows the status, logs, traces, and + environment variables of every resource. +3. Wait for `setup-secrets` and `run-db-migrations` to finish. The services wait for both and then + start. +4. Test that the Identity service is alive by navigating to + [http://localhost:33656/.well-known/openid-configuration](http://localhost:33656/.well-known/openid-configuration) +5. Test that the Api service is alive by navigating to + [http://localhost:4000/alive](http://localhost:4000/alive) -::: +Aspire runs the database migrations and re-applies `dev/secrets.json` on every start. After you pull +changes or edit `secrets.json`, restart the AppHost, or restart the `setup-secrets` or +`run-db-migrations` resource from the dashboard. - +To stop the server, press `Ctrl+C` in the terminal running the AppHost. The SQL Server, Azurite, +MailCatcher, and Redis containers are persistent and keep running between sessions, so later starts +are faster. -Redis provides distributed caching capabilities. Available caching configurations have specific -fallbacks including process-scoped, in-memory caching, or database store for self-hosted -configurations. +### Optional resources -For details on which features can take advantage of Redis and available configurations, including -connection string setting in `secrets.json`, see -[CACHING.md](https://github.com/bitwarden/server/blob/main/src/Core/Utilities/CACHING.md). +Some resources don't start automatically. Start them from the Aspire dashboard when you need them: -(Optional) [Monitor](https://redis.io/docs/latest/commands/monitor/) Redis activity using the Redis -CLI: +- `web-frontend` runs the [web vault](../clients/web-vault/index.mdx) from a sibling `clients` + checkout. You still need to complete the web vault setup, including `npm ci`. +- `idp` runs a SAML identity provider for [SSO](./sso/index.md) development. + +To tunnel the Billing service through ngrok for Stripe webhooks, see +[Ingress Tunnels](./tunnel.md#ngrok-with-aspire). + +### Change Aspire settings + +Override AppHost settings with user secrets from the `AppHost` folder. For example, to move the Api +service to another port: ```bash -docker compose exec redis redis-cli +dotnet user-secrets set "Services:api:BasePort" "4001" ``` -### Azurite +Common settings include: -:::note +| Setting | Purpose | +| ------------------------------------------ | --------------------------------------------------------------------------- | +| `Services::BasePort` | Port for a service, if the default conflicts with something on your machine | +| `ClientsPath` | Path to the `clients` repository's `apps` folder for `web-frontend` | +| `SelfHost` | Run in a [self-hosted configuration](./self-hosted/index.mdx) | +| `AdditionalProjects::Path` | Add another project to the orchestration without changing code | +| `AdditionalProjects::ReferencedBy:0` | Give an existing service a reference to the additional project | -This section applies to Bitwarden developers only. +Restart the AppHost to apply changes. Path settings are relative to the `AppHost` folder, so use +absolute paths if you work from a Git worktree. -::: +## Run manually -[Azurite](https://github.com/Azure/Azurite) is an emulator for Azure Storage API and supports Blob, -Queues and Table storage. We use it to minimize the online dependencies required for developing in a -cloud environment. +If you aren't using Aspire, start the dependencies and services yourself. -To bootstrap the local Azurite instance, navigate to the `dev` directory in your server repo and run -the following commands: +### Configure Docker -1. Install the `Az` module. This may take a few minutes to complete without providing any user - feedback (it may appear frozen). - - ```bash - pwsh -Command "Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force" - ``` +We provide a [Docker Compose](https://docs.docker.com/compose/) configuration, which is used during +development to provide the required dependencies. This is split up into multiple service profiles to +facilitate easy customization. -2. Run the setup script: +1. Some Docker settings are configured in the environment file, `dev/.env`. Copy the example + environment file: ```bash - pwsh setup_azurite.ps1 + cd dev + cp .env.example .env ``` -## Configure user secrets +2. Open `.env` with your preferred editor. -[User secrets](https://docs.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-6.0) -are a method for managing application settings on a per-developer basis. They override the settings -in `appSettings.json` of each project. Your user secrets file should match the structure of the -`appSettings.json` file for the settings you intend to override. +3. Set the `MSSQL_PASSWORD` variable to the SQL Server password you chose when you + [configured user secrets](#configure-user-secrets). -We provide a helper script which simplifies setting user secrets for all projects in the server -repository. +4. You can change the other variables or use their default values. Save and quit this file. +5. Start the Docker containers. -1. Get a template `secrets.json`. We need to get an initial version of `secrets.json`, which you - will modify for your own secrets values. + Using PowerShell, navigate to the cloned server repo location, into the `dev` folder and run the + docker command below. - Navigate to the `dev` folder in your server repo and copy the example `secrets.json` file. - ```bash - cp secrets.json.example secrets.json + docker compose --profile mssql --profile mail up -d ``` + Which starts the MSSQL and local mail server containers, which should be suitable for most + community contributions. + - - Copy the user secrets file from the shared Development collection (Your Bitwarden Vault) into - the `dev` folder. - - If you don't have access to the Development collection, contact our IT Manager to arrange - access. Make sure you have first set up a Bitwarden account using your company email address. - - This `secrets.json` is configured to use the dockerized Azurite and MailCatcher instances and - is recommended for this guide. - + ```bash + docker compose --profile cloud --profile mail up -d + ``` -2. Update `secrets.json` with your own values: - - `sqlServer` > `connectionString`: insert your password where indicated + Which starts MSSQL, mail, Redis, and Azurite container. The additional Azurite container is + required to emulate Azure used by the Bitwarden cloud environment. - - - `installation` > `id` and `key`: - [request a hosting installation Id and Key](https://bitwarden.com/host/) and insert them here - - `licenseDirectory`: set this to an empty directory, this is where uploaded license files will - be stored. + - +After you’ve run the `docker compose` command, you can use the +[Docker Dashboard](https://docs.docker.com/desktop/dashboard/) to manage your containers. You should +see your containers running under the `bitwardenserver` group. -3. Once you have your `secrets.json` complete, run the below command to add the secrets to each - Bitwarden server project. +:::caution - ```bash - pwsh setup_secrets.ps1 - ``` +Changing `MSSQL_PASSWORD` variable after first running docker compose will require a re-creation of +the storage volume. -The helper script also supports an optional `-clear` switch which removes all existing settings -before re-applying them: +**Warning: this will delete your development database.** + +To do this, run ```bash -pwsh setup_secrets.ps1 -clear +docker compose --profile mssql down +docker volume rm bitwardenserver_mssql_dev_data ``` -## Create database +After that, rerun the docker compose command from Step 5. + +::: + +### Create database You now have the MSSQL server running in Docker. The next step is to create the database that will be used by the Bitwarden server. @@ -296,38 +404,7 @@ up-to-date. See [MSSQL Database](./database/mssql/index.md) for more information ::: - - -## Install licensing certificate - -To run your local server environment as a licensed instance, you will need to download the -`Licensing Certificate - Dev` from the shared Engineering collection. - -1. Log in to your company-issued Bitwarden account -2. On the "Vaults" page, scroll down to the "Licensing Certificate - Dev" item -3. View attachments and only download `dev.pfx` -4. Create a `~/.secrets/` folder and place `dev.pfx` inside -5. Add the following under `globalSettings` in `secrets.json`, substituting your own path to - `dev.pfx` and the password from the "Licensing Certificate - Dev" vault item: - - ```json - "licenseCertificatePath": "/Users//.secrets/dev.pfx", - "licenseCertificatePassword": "" - ``` - -6. Re-run `setup_secrets.ps1` to apply the new values to every server project — see - [Configure user secrets](#configure-user-secrets). - -:::warning - -Do not import the "Licensing Certificate - Dev" into your keychain. Doing so may compromise all TLS -traffic on your development machine. - -::: - - - -## Build and run the server +### Build and run the server You are now ready to build and run your development server. @@ -363,6 +440,87 @@ You are now ready to build and run your development server. 7. Test that the Api service is alive by navigating to [http://localhost:4000/alive](http://localhost:4000/alive) +## Local services + +Both setups provide the following local services. + +### SQL Server + +You can connect to the Microsoft SQL Server using your preferred database management tool with the +following credentials: + +- Server: localhost +- Port: 1433 +- Username: sa +- Password: the SQL Server password you chose when you + [configured user secrets](#configure-user-secrets) + +### Mailcatcher + +The server uses emails for many user interactions. We provide a pre-configured instance of +[MailCatcher](https://mailcatcher.me/), which catches any outbound email and prevents it from being +sent to real email addresses. You can open its web interface at +[http://localhost:1080](http://localhost:1080). + +### Redis + + + +:::note + +Redis is required for Email and Authenticator two-factor authentication flows, as well as OTP +validation for new device and user verification scenarios when developing for the `cloud` profile. + +::: + + + +Redis provides distributed caching capabilities. Available caching configurations have specific +fallbacks including process-scoped, in-memory caching, or database store for self-hosted +configurations. + +For details on which features can take advantage of Redis and available configurations, including +connection string setting in `secrets.json`, see +[CACHING.md](https://github.com/bitwarden/server/blob/main/src/Core/Utilities/CACHING.md). + +(Optional) [Monitor](https://redis.io/docs/latest/commands/monitor/) Redis activity using the Redis +CLI. With the manual setup, run this from the `dev` folder: + +```bash +docker compose exec redis redis-cli +``` + +With Aspire, open the `redis` resource in the dashboard to find its container, then use +`docker exec` with that container name. + +### Azurite + +:::note + +This section applies to Bitwarden developers only. + +::: + +[Azurite](https://github.com/Azure/Azurite) is an emulator for Azure Storage API and supports Blob, +Queues and Table storage. We use it to minimize the online dependencies required for developing in a +cloud environment. + +Aspire bootstraps Azurite for you with its `azurite-setup` resource. With the manual setup, navigate +to the `dev` directory in your server repo and run the following commands: + +1. Install the `Az` module. This may take a few minutes to complete without providing any user + feedback (it may appear frozen). + + ```bash + pwsh -Command "Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force" + ``` + +2. Run the setup script: + + ```bash + pwsh setup_azurite.ps1 + ``` + ## Client connection Connect a client to your local server by configuring the client’s Api and Identity endpoints. Refer @@ -375,8 +533,8 @@ client to create an account and validate your local server changes. :::info -If you cannot connect to the Api or Identity projects, check the terminal output to confirm the -ports they are running on. +If you cannot connect to the Api or Identity projects, check the terminal output or the Aspire +dashboard to confirm the ports they are running on. ::: @@ -396,6 +554,12 @@ debugger. ::: +### Aspire + +Debug the `AppHost` project from your IDE. Visual Studio and Rider attach the debugger to each +service that the AppHost starts, so breakpoints in Api, Identity, and the other services work +without launching them separately. Rider requires the Aspire plugin. + ### Visual Studio To debug: diff --git a/docs/getting-started/server/portal.md b/docs/getting-started/server/portal.md index f17aff45d..eae1a5320 100644 --- a/docs/getting-started/server/portal.md +++ b/docs/getting-started/server/portal.md @@ -16,6 +16,9 @@ To disambiguate this application from others in the Bitwarden landscape, we refe ## Setup +If you run the server with [Aspire](./guide.md#run-with-aspire), it starts the Admin project for +you. Complete steps 1 to 5 once to build the stylesheets and libraries, then skip step 6. + 1. Navigate to the `server/src/Admin` directory. 2. Restore nuget packages: @@ -74,10 +77,17 @@ See [User Secrets](../../contributing/user-secrets.md) for how to configure your ::: - - ### Authorization + + +Your local server runs as a self-hosted instance, which doesn't use role-based access control. No +further setup is needed. + + + + + The Bitwarden Portal uses role-based access control to restrict access to application functionality. In order to have access to the features within the Bitwarden Portal, you will need to assign your account to a role. This is in addition to the authentication setup above. diff --git a/docs/getting-started/server/scim.md b/docs/getting-started/server/scim.md index 1d5175964..75cd0c677 100644 --- a/docs/getting-started/server/scim.md +++ b/docs/getting-started/server/scim.md @@ -36,7 +36,8 @@ Bitwarden as they occur. ### Start the SCIM Project -3. Start the SCIM project in your local server repository: +3. Start the SCIM project in your local server repository. Skip this step if you run the server with + Aspire, which starts SCIM for you. ```bash cd bitwarden_license/src/Scim diff --git a/docs/getting-started/server/self-hosted/index.mdx b/docs/getting-started/server/self-hosted/index.mdx index 8470f2a03..f44f50ae5 100644 --- a/docs/getting-started/server/self-hosted/index.mdx +++ b/docs/getting-started/server/self-hosted/index.mdx @@ -121,6 +121,9 @@ it. You must call `migrate.ps1` with the `-selfhost` option in the future to keep your self-host database up-to-date with any migrations. +If you run the self-hosted server with [Aspire](#running), it runs `migrate.ps1 -selfhost` for you +on every start. + ### Define Installation Id and Key for Your Cloud Database You need to manually add the Installation key to your cloud-configured instance, so that it knows @@ -176,7 +179,45 @@ self-hosted launch configurations (e.g. "Api-SelfHost") will set the Environment `developSelfHosted` flag for you. - + + +The Aspire `AppHost` can run every service in the self-hosted configuration. It sets +`developSelfHosted` to `true` for each service and moves each service to its self-hosted port. + +1. Navigate to the `AppHost` folder and switch the AppHost to self-hosted mode: + + ```bash + cd AppHost + dotnet user-secrets set "SelfHost" "true" + ``` + +2. Set the SQL Server password for self-hosted mode. Aspire reuses the same SQL Server container + and data volume in both modes, so use the same value as `Database:Password`. The connection + string in the `Dev:SelfHostOverride:GlobalSettings` section of your `secrets.json` must use this + password too. + + ```bash + dotnet user-secrets set "Database:SelfHostPassword" "" + ``` + +3. Start the AppHost: + + ```bash + dotnet run + ``` + +4. Test that the Identity service is alive by navigating to + [http://localhost:33657/.well-known/openid-configuration](http://localhost:33657/.well-known/openid-configuration) +5. Test that the Api service is alive by navigating to + [http://localhost:4001/alive](http://localhost:4001/alive) + +To switch back to the cloud configuration, set `SelfHost` to `false` and restart the AppHost. + +The AppHost runs one configuration at a time. If you need cloud and self-hosted servers running +together, start one of them with the instructions in the other tabs. + + + We have a number of launch configurations as well as combined configurations to make launching the services easy. By default, the individual self-host launches are hidden. Navigate to `launch.json` @@ -252,6 +293,9 @@ to spin up a Bitwarden-licensed or OSS web server. The default port will be `808 both a cloud-configured and self-hosted-configured web client at once. It is also configured to point at the default `*-SelfHost` ports for the various server projects. +In self-hosted mode, the Aspire `web-frontend` resource runs `build:bit:selfhost:watch` on port +`8081` for you. Start it from the Aspire dashboard. +
How the configuration sausage is made diff --git a/docs/getting-started/server/sso/index.md b/docs/getting-started/server/sso/index.md index 0f7282618..70be42658 100644 --- a/docs/getting-started/server/sso/index.md +++ b/docs/getting-started/server/sso/index.md @@ -9,7 +9,9 @@ sidebar_custom_props: For local development, we use [Docker Test SAML 2.0 Identity Provider](https://github.com/kenchan0130/docker-simplesamlphp), which -we have pre-configured in an `idp` Docker container for easy setup. +we have pre-configured in an `idp` Docker container for easy setup. If you run the server with +[Aspire](../guide.md#run-with-aspire), the `idp` container is an Aspire resource. Otherwise, start +it with Docker Compose. ### Prerequisites @@ -32,21 +34,28 @@ we have pre-configured in an `idp` Docker container for easy setup. ```bash cd ~/Projects/server/dev ``` -6. Open your `.env` file and set the following environment variables using the "SP Entity ID" and - "Assertion Consumer Service (ACS) URL" values from the SSO configuration page opened in step #4 - above: +6. Configure the IdP with the "SP Entity ID" and "Assertion Consumer Service (ACS) URL" values from + the SSO configuration page opened in step #4 above. + - **Aspire:** the AppHost builds both values from your organization ID. Copy the organization ID + from the SP Entity ID (the last segment of the URL), then set it from the `AppHost` folder: - ```bash - IDP_SP_ENTITY_ID={SP Entity ID} - IDP_SP_ACS_URL={ACS URL} - ``` + ```bash + dotnet user-secrets set "Parameters:sso-org-id" "" + ``` + + - **Docker Compose:** open your `.env` file and set the following environment variables: + + ```bash + IDP_SP_ENTITY_ID={SP Entity ID} + IDP_SP_ACS_URL={ACS URL} + ``` - :::note + :::note - You should have created this `.env` file during your initial server setup. You can refer back to - the `.env.example` file if required. + You should have created this `.env` file during your initial server setup. You can refer back + to the `.env.example` file if required. - ::: + ::: 7. (Optional) You may generate a certificate to sign SSO requests. You can do this with a script made for your OS of choice. @@ -93,11 +102,14 @@ we have pre-configured in an `idp` Docker container for easy setup. [here](https://github.com/kenchan0130/docker-simplesamlphp#customize-sp-remote-metadata-reference) for more information about this file. -10. Start the docker container: +10. Start the IdP: + - **Aspire:** restart the AppHost so it picks up `sso-org-id`, then start the `idp` resource + from the Aspire dashboard. It doesn't start automatically. + - **Docker Compose:** start the docker container: - ```bash - docker compose --profile idp up -d - ``` + ```bash + docker compose --profile idp up -d + ``` 11. You can test your user configuration by navigating to [http://localhost:8090/simplesaml](http://localhost:8090/simplesaml) and clicking Authentication @@ -138,7 +150,10 @@ and click Logout. Alternatively, you can use a private browsing session. ### SAML configuration -To change the Entity ID or ACS URL, edit the `.env` file and then restart the Docker container: +With Aspire, set a new `Parameters:sso-org-id` user secret in the `AppHost` folder, then restart the +AppHost and start the `idp` resource again. + +With Docker Compose, edit the `.env` file and then restart the Docker container: ```bash docker compose --profile idp up -d @@ -152,9 +167,10 @@ You’re missing the `uid` claim for the user in `authsources.php`. ### IdP displays a "Metadata not found" error -- Your Entity ID and/or ACS URL in `.env` are incorrect. Make sure they match the values shown in - the SSO configuration page of the Admin Console. If you change the values in `.env`, run the - `docker compose` command above to restart the container with the updated variables. +- Your Entity ID and/or ACS URL are incorrect. Make sure they match the values shown in the SSO + configuration page of the Admin Console. With Aspire, check the `sso-org-id` parameter. With + Docker Compose, check `.env` and run the `docker compose` command above to restart the container + with the updated variables. - Your domain has been claimed by a different organization (e.g. due to old test data). Bitwarden is using that organization's SSO configuration, which doesn't match your local IDP configuration. Remove any claimed domains and try again. diff --git a/docs/getting-started/server/troubleshooting.md b/docs/getting-started/server/troubleshooting.md index 3692d86d6..06d465932 100644 --- a/docs/getting-started/server/troubleshooting.md +++ b/docs/getting-started/server/troubleshooting.md @@ -4,6 +4,21 @@ sidebar_position: 10 # Troubleshooting +## Aspire + +The Aspire dashboard shows the logs for every resource. Check the logs for `setup-secrets` and +`run-db-migrations` first, because the services wait for both to finish. + +| Symptom | Fix | +| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | +| Containers fail to start | Make sure Docker Desktop is running, and stop any Docker Compose containers that use the same ports | +| Services stay in a waiting state | Check the `setup-secrets` and `run-db-migrations` logs for errors | +| Migrations fail to connect | Make sure the `Database:Password` AppHost user secret matches the password in your `secrets.json` connection string | +| Migrations fail immediately | Make sure `pwsh` is on your `PATH` | +| `azurite-setup` fails | Install the `Az` PowerShell module, as described in the [Setup Guide](./guide.md#set-up-aspire) | +| A service fails with a port conflict | Set `Services::BasePort` to a free port with `dotnet user-secrets set` in the `AppHost` folder | +| User secrets changes disappear | Aspire runs `setup_secrets.ps1 -clear` on every start. Add the values to `dev/secrets.json` instead of setting them on individual projects | + ## macOS ### AppleCFErrorCryptographicException diff --git a/docs/getting-started/server/tunnel.md b/docs/getting-started/server/tunnel.md index fa299d48a..766f6a8bd 100644 --- a/docs/getting-started/server/tunnel.md +++ b/docs/getting-started/server/tunnel.md @@ -79,3 +79,29 @@ DNS to start resolving before trying to access it. Anyone with this URL can access the forwarded URL on your machine. ::: + +### Ngrok with Aspire + +The Aspire `AppHost` can tunnel the Billing service through ngrok, which is useful for testing +Stripe webhooks. The ngrok plugin is turned off by default. + +1. Create an `AppHost.csproj.user` file next to `AppHost/AppHost.csproj`. Git ignores this file. + + ```xml + + + true + + + ``` + +2. From the `AppHost` folder, store your ngrok auth token in user secrets: + + ```bash + dotnet user-secrets set "NgrokAuthToken" "" + ``` + +3. Start the AppHost, then start the `billing-webhook-ngrok-endpoint` resource from the Aspire + dashboard. It doesn't start automatically. + +To tunnel other services, use the standalone `ngrok` command above.