[johha]

Thanks for contributing to cloud_controller_ng. To speed up the process of reviewing your pull request please provide us with:

• A short explanation of the proposed change:

  Implements RFC-0046 behind the disabled-by-default diego_custom_stacks feature flag. Buildpack and CNB apps can use a docker:// image URI as their stack without requiring a Stack database record.

  Custom stacks are used for staging, runtime LRPs, and tasks. They support custom buildpacks or admin buildpacks without a stack association. Registry credentials are stored encrypted and only complete username/password pairs are passed to Diego. Embedded credentials are rejected.

  Public registry example:

  applications:
    - name: custom-stack-app
      stack: docker://cloudfoundry/cflinuxfs3:0.379.0

  The stack can also be provided directly to cf push:

  cf push custom-stack-app -s docker://cloudfoundry/cflinuxfs3:0.379.0 -b https://example.com/buildpack.zip

  Private registry example:

  applications:
    - name: private-stack-app
      stack: docker://ghcr.io/example/private-stack:latest
      credentials:
        ghcr.io:
          username: example
          password: ((registry-password))

  During staging, the final image repository component is exposed as CF_STACK or CNB_STACK_ID. Custom stack image layers count toward the app instance disk quota.

• An explanation of the use cases your change solves

  Operators can allow apps to use patched, specialized, or independently maintained root filesystems without registering platform stacks. This also supports private registries and apps that must continue using an image after its corresponding platform stack has been removed.

• Manual test coverage

  • Confirmed the existing system-stack workflow before and after the custom-stack tests.
  • Staged and ran classic buildpack apps with public and private custom stacks. Missing and incorrect credentials failed without exposing secrets; correct credentials were retained for later pushes, restages, restarts, scaling, and tasks.
  • Staged and ran CNB apps with public, private, and arbitrarily renamed custom stacks, including a private CNB buildpack. Restages, restarts, scaling, and tasks reused the stored credentials.
  • Used explicit custom buildpacks and auto-detected stackless admin buildpacks. Verified CF_STACK and CNB_STACK_ID derivation, a modified /etc/os-release, and the expected staging failure when image and buildpack ABIs were incompatible.
  • Disabled the feature flag and confirmed that pushes and restages were rejected while an already-running app remained healthy. Also verified operation after removal of the corresponding platform stack.
  • Verified early validation of malformed image URIs, embedded credentials, and stack-pinned admin buildpacks.
  • Confirmed that custom stack image layers count toward disk quota: undersized quotas failed, while app instances and tasks succeeded with sufficient quota.

• I have reviewed the contributing guide

• I have viewed, signed, and submitted the Contributor License Agreement

• I have made this pull request to the main branch

• I have run all the unit tests using bundle exec rake

• I have run CF Acceptance Tests

[jochenehret]

Tested on a bbl environment with a default CF deployment, works!

But I also tested some error scenarios and found a few confusing error messages and a few minor issues:

1. When pushing an app with a custom stack and without a buildpack, you get this error message:

For application 'custom-stack-app': Buildpack must specify a custom buildpack URL or an enabled unpinned admin buildpack when using a custom stack
  FAILED

Maybe better: "You must either specify a custom buildpack URL or your platform operator must provide an enabled buildpack not pinned to a stack."

2. Using a non-existent image like "docker://ghcr.io/example/cflinuxfs4:1.332.0" results in error message:

   Cell c57eb448-3919-452b-8783-13d4043f1850 failed to create container for instance 414f282c-bd44-4ca0-ae8a-a330b6531d45: running image plugin create: fetching image reference: creating image: creating image source: Requesting bearer token: received unexpected HTTP status: 403 Forbidden
   : exit status 1

Shouldn't the error say something like "404 not found"?

3. Using an invalid stack URL produces a confusing error message:

Updating with these attributes...
  ---
  applications:
  - name: custom-stack-app
    disk-quota: 2G
+   default-route: true
-   stack: docker://cloudfoundry/cflinuxfs5:0.63.0
+   stack: ocker://cloudfoundry/cflinuxfs5:0.63.0
    buildpacks:
      https://github.com/cloudfoundry/staticfile-buildpack
For application 'custom-stack-app': Stack must be an existing stack
FAILED

4. Specifying a non-existent tag produces a confusing error message:

-   stack: docker://cloudfoundry/cflinuxfs5:0.63.0
+   stack: docker://cloudfoundry/cflinuxfs5:0.63.777

   Cell c57eb448-3919-452b-8783-13d4043f1850 failed to create container for instance 47c2baeb-1174-48c3-994c-c8802f938539: running image plugin create: fetching image reference: creating image: creating image source: reading manifest 0.63.777 in docker.io/cloudfoundry/cflinuxfs5: manifest unknown
   : exit status 1

5. The environment variable CF_STACK is not set in the running container (only in staging). Is that intended?

[johha]

Thanks for the feedback @jochenehret . I addressed the points as follows:

1. Updated the buildpack validation message to: “You must either specify a custom buildpack URL or your platform operator must provide an enabled buildpack that is not pinned to a stack.”
2. The 403 is returned by GHCR while resolving the image. CAPI validates the URI syntax, while Diego reports the registry response, which is not necessarily a 404. Therefore nothing we can change on CC side.
3. Improved the validation message when custom stacks are enabled: “Stack must be an existing stack or a valid container image URI starting with docker://.”
4. Tags are resolved by the registry when Diego pulls the image. manifest unknown is the upstream registry response; CC cannot reliably validate tags beforehand, especially for private registries.
5. This is intended. CF_STACK and CNB_STACK_ID have historically only been set during staging. The documentation and PR description now state this explicitly.

[jochenehret]

Tested again, just one finding:

applications:
  - name: private-stack-app
    stack: ocker://jochenehret/cflinuxfs5:latest

This still produces the somewhat confusing error message:

For application 'private-stack-app': Stack must be an existing stack
FAILED

[johha]

Tested again, just one finding:

applications:
  - name: private-stack-app
    stack: ocker://jochenehret/cflinuxfs5:latest

This still produces the somewhat confusing error message:

For application 'private-stack-app': Stack must be an existing stack
FAILED

This was fixed a few days ago: https://github.com/cloudfoundry/cloud_controller_ng/pull/5492/changes#diff-c05e3cf7801009354b84e037205a5e1fca2fef85f667d318a4e7d5ffe8919cd8R36-R41