expose llm gateway callout for script

[atulikumwenayo]

Added two methods to allow customers to use their AI models to generate text in Script. In the public sdk, the methods use the public LLM Gateway API (the same used for function), so that the customer can test outside of DC. While running in DC, the methods will be overridden to call the registered spark UDF function.

Sample test and result

df = client.read_dlo("sample_data__dll")

print(client.llm_gateway_generate_text("Define Apache Spark in 20 words or less", "sfdc_ai__DefaultGPT52"))

df_upper1 = df.withColumn(
    "description__c",
    llm_gateway_generate_text_col(
        "Transform '{desc}' into upper case, and add '{id}' to the result as suffix separated by ' - ', then return the final result.",
        {"id": col("id__c"), "desc": col("description__c")},
        model_id="sfdc_ai__DefaultGPT52",
        max_tokens=1000,
    ),
)

dlo_name = "sample_data_copy__dll"
client.write_to_dlo(dlo_name, df_upper1, write_mode=WriteMode.APPEND)

datacustomcode run ./payload/entrypoint.py --sf-cli-org myorg
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
26/06/03 15:58:23 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2026-06-03 15:58:26.763 | INFO     | datacustomcode.einstein_platform_client:_get_einstein_platform_url:63 - Using Einstein Platform API endpoint: https://test.api.salesforce.com (env=test)
Apache Spark is a distributed computing framework for fast, in-memory big data processing, supporting batch, streaming, SQL, and machine learning.
+------------------------+--------------+-------------+-------------------+-----+-----------------------+
|cdp_sys_sourceversion__c|description__c|datasource__c|datasourceobject__c|id__c|internalorganization__c|
+------------------------+--------------+-------------+-------------------+-----+-----------------------+
|                    NULL|      TEST - 1|UploadedFiles|          data2.csv|    1|                       |
|                    NULL|   ANOTHER - 2|UploadedFiles|          data2.csv|    2|                       |
|                    NULL|     AGAIN - 3|UploadedFiles|          data2.csv|    3|                       |
|                    NULL|      FOUR - 4|UploadedFiles|          data2.csv|    4|                       |
|                    NULL|      FIVE - 5|UploadedFiles|          data2.csv|    5|                       |
+------------------------+--------------+-------------+-------------------+-----+-----------------------+

[atulikumwenayo]

I am going to move these comments from the code into README