Upgrade Jackson dependency to a safe version

[Divyansh-db]

Summary

Upgrades Jackson from 2.15.2 to 2.18.6 to fix GHSA-72hv-8253-57qq (High severity, CVSS 8.7), a denial-of-service
vulnerability in jackson-core's async JSON parser.

Motivation

The async JSON parser in jackson-core versions prior to 2.18.6 does not enforce the maxNumberLength constraint
(default: 1000 characters), while the synchronous parser does. This allows an attacker to submit JSON with
arbitrarily long numeric values, causing memory and CPU exhaustion — particularly in reactive/non-blocking
applications (e.g. Spring WebFlux). This is classified as CWE-770 (Allocation of Resources Without Limits or
Throttling).

The SDK currently pins Jackson at 2.15.2, which falls in the vulnerable range (2.0.0–2.18.5). All consumers of the
SDK inherit this version, so the fix must be applied upstream here rather than overridden at each consumer.

Changes

• databricks-sdk-java/pom.xml: Bumps jackson.version from 2.15.2 to 2.18.6 (affects jackson-core, jackson-databind,
  jackson-annotations, jackson-datatype-jsr310, jackson-datatype-guava). Adds jackson-datatype-jdk8 as a new
  dependency.
• SerDeUtils.java: Registers Jdk8Module on the shared ObjectMapper. This is required because Jackson 2.18.x no
  longer serializes/deserializes java.util.Optional types by default — the SDK's Request class uses Optional
  for redirectionBehavior, which fails without this module.

Test plan

• All 1093 existing unit tests pass with no changes needed.
• The jackson-datatype-jdk8 module addition was validated by the ApiClientTest.errorDetails test, which exercises
  serialization of Request objects containing Optional fields.

NO_CHANGELOG=true

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-java

Inputs:

• PR number: 703
• Commit SHA: 2730fc3fc01c7bb8abae3b66cda1508a4d8e764c

Checks will be approved automatically on success.