RANGER-4076: Support Java 17 for build and runtime#872
RANGER-4076: Support Java 17 for build and runtime#872pradeepagrawal8184 wants to merge 14 commits intomasterfrom
Conversation
9755de9 to
68df023
Compare
8d5a3ce to
58be178
Compare
58be178 to
fc89780
Compare
|
Thanks @pradeepagrawal8184 for this patch, I think it is best to update the CI/CD build in this PR by removing builds for JDK8 and JDK11 and adding support for JDK17. The jobs |
fc89780 to
4732fb2
Compare
4732fb2 to
9467111
Compare
|
Current ranger master branch shipped jars vs jar shipped with this branch changes. |
e7cd2ea to
9f9247c
Compare
|
Hi @pradeepagrawal8184, |
@ramackri : Spring upgrade can be handled separately. We can discuss that in the spring upgrade ticket after this PR merge and decide. |
kumaab
left a comment
There was a problem hiding this comment.
I've reviewed the changes partially (since this is a big patch), some JWT related functionalities recently added seems to have been removed, please rebase and take a look. thanks.
890fc0a to
32469c3
Compare
32469c3 to
f3b4e48
Compare
53f814f to
49b1f03
Compare
|
@vyommani : I am unable to add you as reviewer. |
There was a problem hiding this comment.
Pull request overview
Updates Apache Ranger’s build/runtime stack to Java 17 by migrating off Jersey 1.x (and other JDK-internal/removed APIs), refreshing dependency sets, and adjusting tests/packaging/CI & docker assets accordingly.
Changes:
- Migrates multiple modules from Jersey 1.x client/server APIs to Jersey 2.x / JAX-RS APIs and updates related servlet configuration.
- Reworks JavaScript execution to use GraalJS where Nashorn is not available on newer JDKs; updates tests accordingly.
- Aligns build/test infrastructure with Java 17 (Maven plugin versions, CI workflow, docker configs, dependency exclusions).
Reviewed changes
Copilot reviewed 210 out of 212 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| unixauthservice/pom.xml | Replace Jersey 1.x deps and add Java 17/Jakarta-era deps. |
| unixauthnative/pom.xml | Use property-driven native plugin version. |
| unixauthclient/src/test/java/org/apache/ranger/authentication/unix/jaas/TestRemoteUnixLoginModule.java | Replace sun.security.* cert generation with BouncyCastle for JDK17. |
| unixauthclient/pom.xml | Swap test dependency from mockito-inline to BouncyCastle for cert test. |
| ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestRangerUgSyncRESTClient.java | Update Jersey client type import for tests. |
| ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestPolicyMgrUserGroupBuilder.java | Update REST response types + reset HA singletons for test stability. |
| ugsync/src/main/java/org/apache/ranger/unixusersync/process/RangerUgSyncRESTClient.java | Migrate REST client creation to Jersey 2 + anti-MOXy config. |
| ugsync/pom.xml | Replace Jersey 1.x bundle with Jersey 2/Jakarta-era deps; update surefire version. |
| ugsync-util/pom.xml | Drop mockito-inline; update JAXB dependency to Jakarta. |
| tagsync/src/test/java/org/apache/ranger/tagsync/process/TestTagSynchronizer.java | JUnit 5 visibility cleanup. |
| tagsync/src/main/resources/ranger-tagsync-site.xml | Add additional HA address + add missing/extra config keys. |
| tagsync/src/main/java/org/apache/ranger/tagsync/ha/TagSyncHAInitializerImpl.java | Singleton field adjustments during Java 17 migration. |
| tagsync/conf/templates/ranger-tagsync-template.xml | Add new service/SSL/ports template properties. |
| security-admin/src/test/java/org/apache/ranger/service/TestXUserService.java | Mockito/JUnit 5 wiring changes for injected fields. |
| security-admin/src/test/java/org/apache/ranger/service/TestXGroupUserService.java | Mockito/JUnit 5 wiring changes + BaseDao injection. |
| security-admin/src/test/java/org/apache/ranger/service/TestXAccessAuditService.java | Generic BaseDao + wiring changes. |
| security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceServiceBase.java | Mockito/JUnit 5 wiring changes; rename mock field. |
| security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceResourceService.java | Switch to MockitoExtension and spy/wiring approach. |
| security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceDefServiceBase.java | Mockito/JUnit 5 wiring changes; rename mock field. |
| security-admin/src/test/java/org/apache/ranger/service/TestRangerSecurityZoneServiceService.java | Switch to MockitoExtension and explicit field wiring. |
| security-admin/src/test/java/org/apache/ranger/service/TestRangerGdsServices.java | Replace deprecated initMocks usage with openMocks/spy wiring. |
| security-admin/src/test/java/org/apache/ranger/service/TestRangerGdsDataShareInDatasetService.java | Switch to MockitoExtension and explicit field wiring. |
| security-admin/src/test/java/org/apache/ranger/service/TestAuthSessionService.java | Generic BaseDao + wiring changes. |
| security-admin/src/test/java/org/apache/ranger/security/web/filter/TestRangerCSRFPreventionFilter.java | Mockito API updates + formatting cleanup. |
| security-admin/src/test/java/org/apache/ranger/rest/TestXKeyREST.java | Disable/inline-comment old Jersey 1.x exception test. |
| security-admin/src/test/java/org/apache/ranger/rest/TestUserREST.java | Update deprecated Mockito matchers usage. |
| security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java | Jersey multipart class import update + inject missing deps in test. |
| security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java | Update expected invocation count. |
| security-admin/src/test/java/org/apache/ranger/rest/TestRoleREST.java | Jersey multipart class import update. |
| security-admin/src/test/java/org/apache/ranger/common/db/TestBaseDao.java | Reset op-context ThreadLocal behavior for tests. |
| security-admin/src/test/java/org/apache/ranger/common/TestDateUtil.java | Re-enable & modernize date assertion logic. |
| security-admin/src/test/java/org/apache/ranger/biz/TestRangerBizUtil.java | Make bulk-mode test isolate ThreadLocal context. |
| security-admin/src/main/webapp/WEB-INF/web.xml | Switch Jersey servlet to Jersey 2 + provider config. |
| security-admin/src/main/java/org/apache/ranger/service/filter/RangerRESTAPIFilter.java | Migrate filter implementation from Jersey 1 to JAX-RS 2 filters. |
| security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java | Implement new ServletContext methods for newer servlet API. |
| security-admin/src/main/java/org/apache/ranger/security/context/RangerAPIList.java | Add API constant for new GDS endpoint. |
| security-admin/src/main/java/org/apache/ranger/rest/XKeyREST.java | Replace Jersey 1 exception type with JAX-RS exceptions. |
| security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java | Migrate multipart imports to Jersey 2 multipart. |
| security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java | Adjust role mutation endpoint parameter binding. |
| security-admin/src/main/java/org/apache/ranger/rest/PublicAPIsv2.java | Keep Public APIs aligned with RoleREST signature changes. |
| security-admin/src/main/java/org/apache/ranger/common/RangerJsonProvider.java | Ensure Jackson provider wins over MOXy; adjust mapper features. |
| security-admin/src/main/java/org/apache/ranger/common/RangerJAXBContextResolver.java | Replace Jersey JSONJAXBContext with standard JAXBContext. |
| ranger-yarn-plugin-shim/pom.xml | Exclude Jersey 1.x transitive deps. |
| ranger-util/pom.xml | Pin plugin version via property. |
| ranger-tools/pom.xml | Add Jackson JAX-RS deps; update plugin versions. |
| ranger-sqoop-plugin-shim/pom.xml | Exclude jettison to avoid conflicts. |
| ranger-presto-plugin-shim/pom.xml | Fix javax.annotation property name; add test log4j-over-slf4j. |
| ranger-knox-plugin-shim/pom.xml | Exclude Jersey 1.x transitive deps. |
| ranger-hive-plugin-shim/pom.xml | Exclude Jersey 1.x transitive deps. |
| ranger-hdfs-plugin-shim/pom.xml | Exclude Jersey 1.x transitive deps. |
| ranger-hbase-plugin-shim/src/main/test/org/apache/hadoop/hbase/security/access/RangerAccessControlListsTest.java | Convert JUnit 4 annotations to JUnit 5. |
| ranger-hbase-plugin-shim/pom.xml | Add servlet API; exclude Jersey 1.x + servlet duplicates. |
| ranger-examples/plugin-sampleapp/pom.xml | Replace jersey-bundle with Jakarta/Jersey2 runtime stack. |
| ranger-examples/distro/src/main/assembly/sample-client.xml | Remove jersey-bundle from packaged sample client libs. |
| ranger-examples/distro/src/main/assembly/plugin-sampleapp.xml | Remove jersey-bundle from packaged sample app libs. |
| ranger-common-ha/pom.xml | Use central javax.inject version property. |
| ranger-atlas-plugin-shim/pom.xml | Exclude Jersey 1.x transitive deps. |
| plugin-yarn/src/main/java/org/apache/ranger/services/yarn/client/YarnClient.java | Migrate Jersey 1 client calls to JAX-RS Client API + builder. |
| plugin-solr/pom.xml | Add Jackson/Jakarta/Jersey runtime deps for Java 17. |
| plugin-schema-registry/src/main/java/org/apache/ranger/services/schema/registry/client/connection/DefaultSchemaRegistryClient.java | Move to shaded JAX-RS/Jersey imports and register Jackson provider. |
| plugin-presto/pom.xml | Add provided Jersey/Jakarta/Jackson deps and exclude legacy deps. |
| plugin-ozone/pom.xml | Add Jackson JAX-RS runtime dep; remove mockito-inline. |
| plugin-nifi/pom.xml | Add Jakarta/Jersey runtime deps and exclude legacy Jersey transitive deps. |
| plugin-nifi-registry/pom.xml | Add Jakarta/Jersey runtime deps and exclude legacy Jersey transitive deps. |
| plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java | Update expectations around GraalVM behavior. |
| plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java | Replace Nashorn factory with GraalJS engine creation/config. |
| plugin-nestedstructure/pom.xml | Add Jakarta WS-RS provided dep; pin compiler plugin version. |
| plugin-kylin/pom.xml | Add Jakarta/Jersey runtime deps and exclude legacy Jersey transitive deps. |
| plugin-kafka/pom.xml | Add Jersey deps; align JUnit/platform versions; pin surefire version. |
| plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java | Migrate Jersey 1 client calls to JAX-RS Client API. |
| pdp/pom.xml | Remove Jersey 1 client deps; use Jakarta WS-RS version property. |
| knox-agent/src/test/java/org/apache/ranger/services/knox/KnoxRangerTest.java | Disable integration test by default. |
| knox-agent/src/test/java/org/apache/ranger/admin/client/TestRangerAdminJersey2RESTClient.java | Update spy behavior for kerberos-enabled method signature. |
| knox-agent/src/main/java/org/apache/ranger/admin/client/RangerAdminJersey2RESTClient.java | Use RangerJersey2ClientBuilder to avoid MOXy. |
| knox-agent/pom.xml | Align Jakarta WS-RS version; add commons-configuration2; exclude Jersey 1.x. |
| kms/src/test/resources/kms/dbks-site.xml | Whitespace/formatting cleanup. |
| kms/src/test/java/org/apache/hadoop/crypto/key/RangerMasterKeyTest.java | Replace Base64 import (currently to internal JDK class). |
| kms/src/main/webapp/WEB-INF/web.xml | Switch Jersey servlet class/config to Jersey 2. |
| kms/src/main/java/org/apache/ranger/common/RangerJsonProvider.java | Ensure Jackson provider wins over MOXy; adjust mapper features. |
| kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSExceptionsProvider.java | Update exception unwrapping for Jersey 2 / servlet exceptions. |
| kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java | Fix Location handling and add OPTIONS handler for SPNEGO handshake. |
| intg/src/main/java/org/apache/ranger/RangerServiceException.java | Replace ClientResponse with JAX-RS Response types. |
| intg/src/main/java/org/apache/ranger/RangerClient.java | Replace ClientResponse with JAX-RS Response and readEntity. |
| intg/pom.xml | Add provided Jakarta/Jersey deps; remove TestNG dependency. |
| hbase-agent/src/test/java/org/apache/ranger/services/hbase/client/TestHBaseResourceMgr.java | Relax stubbing with lenient matchers for Mockito/JDK17. |
| embeddedwebserver/pom.xml | Add tomcat-jasper-el for newer tomcat/JSP compatibility. |
| docs/pom.xml | Pin Maven plugin versions and set compiler source/target from properties. |
| distro/src/main/assembly/sample-client.xml | Update packaged client dependency set for Jersey2/Jackson/etc. |
| distro/src/main/assembly/ranger-tools.xml | Update packaged ranger-tools dependency set for Jersey2/Jackson/etc. |
| distro/src/main/assembly/plugin-sqoop.xml | Update plugin packaging includes for Jersey2/HK2/Jakarta stack. |
| distro/src/main/assembly/plugin-solr.xml | Update plugin packaging includes for Jersey2/HK2/Jakarta stack. |
| distro/src/main/assembly/plugin-schema-registry.xml | Minor XML formatting cleanup. |
| distro/src/main/assembly/plugin-ozone.xml | Update plugin packaging includes; minor formatting cleanup. |
| distro/src/main/assembly/plugin-kylin.xml | Update plugin packaging includes for Jersey2/HK2/Jakarta stack. |
| distro/src/main/assembly/plugin-kms.xml | Update plugin packaging includes for Jersey2/HK2/Jakarta stack. |
| distro/src/main/assembly/plugin-atlas.xml | Update plugin packaging includes; remove Jersey 1.x artifacts. |
| distro/src/main/assembly/pdp.xml | Remove Jersey 1.x client/core includes in packaging. |
| distro/src/main/assembly/knox-agent.xml | Update agent packaging includes; remove Jersey 1.x artifacts. |
| distro/src/main/assembly/hive-agent.xml | Update agent packaging includes; add GraalJS artifacts for JDK17. |
| distro/src/main/assembly/hbase-agent.xml | Update agent packaging includes; add GraalJS artifacts for JDK17. |
| distro/pom.xml | Disable a legacy dependency ban rule for this module. |
| dev-support/ranger-docker/scripts/usersync/ranger-usersync-install.properties | Update docker default Ranger URL hostname. |
| dev-support/ranger-docker/scripts/tagsync/ranger-tagsync-install.properties | Update docker default Ranger URL hostname. |
| dev-support/ranger-docker/scripts/solr/ranger-solr-plugin-install.properties | Update docker default Ranger/Solr hostnames. |
| dev-support/ranger-docker/scripts/kms/ranger-kms.sh | Persist JAVA_OPTS into KMS runtime config on startup. |
| dev-support/ranger-docker/scripts/kms/ranger-kms-install-sqlserver.properties | Update docker default Ranger URL hostname. |
| dev-support/ranger-docker/scripts/kms/ranger-kms-install-postgres.properties | Update docker default Ranger URL hostname. |
| dev-support/ranger-docker/scripts/kms/ranger-kms-install-oracle.properties | Update docker default Ranger URL hostname. |
| dev-support/ranger-docker/scripts/kms/ranger-kms-install-mysql.properties | Update docker default Ranger URL hostname. |
| dev-support/ranger-docker/scripts/hive/ranger-hive-setup.sh | Add --add-opens required for JDK17 runtime. |
| dev-support/ranger-docker/scripts/hive/ranger-hive-plugin-install.properties | Update docker default Ranger/Solr hostnames. |
| dev-support/ranger-docker/scripts/hbase/ranger-hbase-plugin-install.properties | Update docker default Ranger/Solr hostnames. |
| dev-support/ranger-docker/scripts/hadoop/ranger-yarn-plugin-install.properties | Update docker default Ranger/Solr hostnames. |
| dev-support/ranger-docker/scripts/hadoop/ranger-hdfs-plugin-install.properties | Update docker default Ranger/Solr hostnames. |
| dev-support/ranger-docker/docker-compose.ranger.yml | Pass JAVA_OPTS through to the Ranger container. |
| dev-support/ranger-docker/docker-compose.ranger-usersync.yml | Pass JAVA_OPTS through to the usersync container. |
| dev-support/ranger-docker/docker-compose.ranger-tagsync.yml | Pass JAVA_OPTS through to the tagsync container. |
| dev-support/ranger-docker/docker-compose.ranger-kms.yml | Add JAVA_OPTS module export for KMS container. |
| dev-support/ranger-docker/docker-compose.ranger-build.yml | Pass JAVA_OPTS through to build container. |
| dev-support/ranger-docker/Dockerfile.ranger-usersync | Bump base image version to JDK17 variant. |
| dev-support/ranger-docker/Dockerfile.ranger-tagsync | Bump base image version to JDK17 variant. |
| dev-support/ranger-docker/Dockerfile.ranger-solr | Install vim + krb5-user in solr container. |
| dev-support/ranger-docker/Dockerfile.ranger-kms | Bump base image version to JDK17 variant. |
| dev-support/ranger-docker/Dockerfile.ranger-build | Bump build base image version to JDK17 variant. |
| dev-support/ranger-docker/Dockerfile.ranger | Bump base image version to JDK17 variant. |
| dev-support/ranger-docker/.env | Default to JDK17 images + provide JAVA_OPTS for JDK17. |
| credentialbuilder/pom.xml | Replace commons-logging with activation/servlet deps; exclude Jersey 1.x from Hadoop auth. |
| authz-api/pom.xml | Use shared slf4j-api version property. |
| audit-server/server/src/main/java/org/apache/ranger/audit/security/NullServletContext.java | Add missing ServletContext methods for newer servlet API. |
| audit-server/server/pom.xml | Remove Jersey 1.x; add Jakarta WS-RS + exclude Jersey transitive deps. |
| audit-server/pom.xml | Remove Jersey 1.x from dependencyManagement; use compiler props. |
| audit-server/consumer-solr/pom.xml | Remove Jersey 1.x; add Jakarta WS-RS API dependency. |
| audit-server/consumer-hdfs/pom.xml | Remove Jersey 1.x; exclude Jersey 1.x transitives from Hadoop. |
| audit-server/common/pom.xml | Exclude Jersey 1.x transitives from Hadoop. |
| agents-cred/pom.xml | Remove mockito-inline. |
| agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestRangerRESTClientDeadlock.java | Fix latch countdown in 2nd server handler. |
| agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyEngine.java | Align test request preprocessing with production behavior. |
| agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerServiceDefHelper.java | Switch to MockitoExtension + lenient stubbing. |
| agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerPolicyValidator.java | Update deprecated Mockito matchers import. |
| agents-common/src/test/java/org/apache/ranger/plugin/conditionevaluator/RangerRequestScriptEvaluatorTest.java | Add GraalJS host-access coverage and engine creation helper. |
| agents-common/src/test/java/org/apache/ranger/admin/client/datatype/TestRESTResponse.java | Update tests for Response-based parsing paths. |
| agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java | Remove Nashorn creator from fallback list for JDK17. |
| agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java | Replace Jersey Base64 with java.util.Base64. |
| agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java | Remove Nashorn engine creator implementation. |
| agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerUserStoreRefresher.java | Replace ClientResponse usage with JAX-RS Response. |
| agents-common/src/main/java/org/apache/ranger/admin/client/datatype/RESTResponse.java | Replace ClientResponse usage with JAX-RS Response parsing. |
| agents-common/pom.xml | Remove Jersey 1.x; add Jersey 2 + GraalJS deps; align compiler plugin version. |
| agents-audit/orc-util/pom.xml | Replace TestNG test dep with log4j-over-slf4j; add provided Jersey/Jakarta deps. |
| agents-audit/dest-solr/pom.xml | Replace TestNG test dep with log4j-over-slf4j; add provided Jersey/Jakarta deps. |
| agents-audit/dest-log4j/pom.xml | Remove TestNG test dependency. |
| agents-audit/dest-kafka/pom.xml | Remove TestNG test dependency. |
| agents-audit/dest-es/pom.xml | Remove TestNG test dependency. |
| agents-audit/dest-cloudwatch/pom.xml | Remove TestNG test dependency. |
| agents-audit/dest-auditserver/src/main/java/org/apache/ranger/audit/destination/RangerAuditServerDestination.java | Migrate from Jersey 1 WebResource/ClientResponse to JAX-RS client API; add JWT support. |
| agents-audit/dest-auditserver/pom.xml | Add Jersey 2 + JAX-RS API deps; remove TestNG. |
| agents-audit/core/pom.xml | Remove TestNG test dependency. |
| README.md | Document JDK17 build requirements and JAVA_OPTS guidance. |
| .github/workflows/ci.yml | Switch CI build matrix to Java 17 only and update artifact naming. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Please also see (CI build): |
49b1f03 to
dc57f7c
Compare
|
|
||
| protected void setLastKnownActiveUrlIndex(int lastKnownActiveUrlIndex) { | ||
| this.lastKnownActiveUrlIndex = lastKnownActiveUrlIndex; | ||
| this.lastKnownActiveUrlIndex = lastKnownActiveUrlIndex % configuredURLs.size(); |
| import static org.mockito.Mockito.when; | ||
|
|
||
| /** | ||
| * @generated by Cursor |
There was a problem hiding this comment.
This is old test if i understood correctly , Do we need to put comment that it is generated by Curser ?
| HttpServer httpServer2 = HttpServer.create(new InetSocketAddress(0), 0); | ||
| httpServer2.createContext("/", exchange -> { | ||
| LOG.info("Server2: Received request, returning 503..."); | ||
| serverReceivedRequest.countDown(); // Count down the latch for server2 as well |
There was a problem hiding this comment.
Since we are migrating to JDK 17, we need to be careful with changes to test synchronization like this. Could you explain why this specific countDown() was added to TestRangerRESTClientDeadlock? If the migration to JDK 17 introduced a race condition or a hang, we should document that in the code so we don't accidentally re-introduce the deadlock later.
Was this test failing specifically when you switched the build to JDK 17?
| USER root | ||
|
|
||
| RUN apt update && DEBIAN_FRONTEND="noninteractive" apt-get install -y krb5-user | ||
| RUN apt update && DEBIAN_FRONTEND="noninteractive" apt-get install -y vim krb5-user |
There was a problem hiding this comment.
do we need vim in container ?
What changes were proposed in this pull request?
Support jdk17 build in Ranger. This change shall discontinue build support on jdk8/11
How was this patch tested?
Manual install, start and crud operations on the policy, group, users, service etc.