BatchSpanProcessor error when running on Next.js instrumentation

[lapa182]

Hello, we are seeing a number of errors on the console (please see below):

2024-09-12T15:39:59.766848450Z {"stack":"Error: BatchSpanProcessor: span export failed\n    at /app/node_modules/.pnpm/@opentelemetry+sdk-trace-base@1.26.0_@opentelemetry+api@1.9.0/node_modules/@opentelemetry/sdk-trace-base/build/src/export/BatchSpanProcessorBase.js:154:85\n    at AzureMonitorTraceExporter.export (/app/node_modules/.pnpm/@azure+monitor-opentelemetry-exporter@1.0.0-beta.25/node_modules/@azure/monitor-opentelemetry-exporter/dist/index.js:3639:13)\n    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)","message":"BatchSpanProcessor: span export failed","name":"Error"} []

This is the latest applicationinsights package version (3.2.2) with Next.js 14.2.9 loaded in the instrumentation.

[hectorhdzg]

@lapa182 please follow instructions on how to create debug logs for this library here, we need the actual exporter error to determine the issue here.

[lapa182]

Thanks @hectorhdzg, I passed this to the team and asked them to enable the debug logs. Once we have the information me or someone from the team will post it here!

[lapa182]

Hi @hectorhdzg, sorry for taking too long, I'm attaching the CSV with the logs where the issue is happening (if you search for BatchSpanprocessor you can find it). From what I saw briefly, it's related to ContextTagKeys being too long.

query_data_v2_without_II.csv

Also, not sure if it's related to this, but when we did some load testing with the AppInsights turned on, our application memory usage sky-rocketed and it crashed after some time due to memory heap. We even bumped the memory to 28GB thinking it was because of the memory we needed and still continued to rump it up and it crashed, but as soon we turned AppInsights off the memory usage didn't go up at all.

[hectorhdzg]

@lapa182 error will be fixed next release, for performance issue, can you add more details about your app and initialization of the SDK?, @JacksonWeber we may need to add the scenario in our perf tests

[lapa182]

Sure.

We are using:

Node.js: 22.9.0
Next.js 14.2.3
applicationinsights: 3.3.0
Azure as host with App Service and Docker

Here's a screenshot of the memory usage example, which we run using Load Testing from Azure with 50 users accessing simultaneously 22 pages:

Example of the log of heap memory:

This was the pattern which triggered our investigation to do the load testing:

The way we currently load AppInsights is using the instrumentation.ts file following this link:

export async function register() {
	if (process.env.NEXT_RUNTIME === 'nodejs') {
		await import('./utils/appInsights/azure-monitor');
	}
}

And the azure-monitor.ts:

import * as appInsights from 'applicationinsights';

const connectionString = process.env.APPLICATIONINSIGHTS_CONNECTION_STRING;

if (connectionString) {
	appInsights
		.setup(connectionString)
		.setAutoCollectRequests(true)
		.setAutoCollectPerformance(true, false)
		.setAutoCollectExceptions(true)
		.setAutoCollectDependencies(true)
		.setAutoCollectConsole(true, true)
		.setAutoCollectPreAggregatedMetrics(true)
		.setSendLiveMetrics(false)
		.setInternalLogging(false, true)
		.enableWebInstrumentation(false)
		.start();
}

We then load inside AppInsightsProvider as the following:

import type { PropsWithChildren, ReactNode } from 'react';

'use client';

import { AppInsightsContext, ReactPlugin } from '@microsoft/applicationinsights-react-js';
import { ApplicationInsights } from '@microsoft/applicationinsights-web';
import type { PropsWithChildren } from 'react';

interface StateProps {
	appInsights: ApplicationInsights;
	reactPlugin: ReactPlugin;
}

// Store the AI state on `window` so we aren't constantly re-initializing it on every hot-reload during `npm run dev` (we can't use a `Symbol` for this, as we'd get a new symbol on every hot-reload)
const getState = (): StateProps | undefined =>
	typeof window !== 'undefined' ? (window.__AI_STATE__ as StateProps | undefined) : undefined;

const setState = (state: StateProps): void => {
	if (typeof window !== 'undefined') {
		window.__AI_STATE__ = state;
	}
};

function getOrInit(connectionString: string): StateProps {
	let state = getState();
	if (state) {
		return state;
	}

	const reactPlugin = new ReactPlugin();
	const appInsights = new ApplicationInsights({
		config: {
			connectionString,
			disableAjaxTracking: true,
			disableFetchTracking: true,
			extensions: [reactPlugin],
			extensionConfig: {
				[reactPlugin.identifier]: {},
			},
		},
	});

	appInsights.loadAppInsights();

	state = {
		appInsights,
		reactPlugin,
	};

	setState(state);

	return state;
}

interface AppInsightsContextProviderProps extends PropsWithChildren {
	connectionString: string;
}

function AppInsightsContextProvider({ connectionString, children }: AppInsightsContextProviderProps): JSX.Element {
	return (
		<AppInsightsContext.Provider value={getOrInit(connectionString).reactPlugin}>
			{children}
		</AppInsightsContext.Provider>
	);
}

export { AppInsightsContextProvider };

export type { AppInsightsContextProviderProps };


function AppInsightsProvider({ children }: PropsWithChildren): JSX.Element | ReactNode {
	const connectionString = process.env.APPLICATIONINSIGHTS_CONNECTION_STRING;

	if (!connectionString) {
		return children;
	}

	return <AppInsightsContextProvider connectionString={connectionString}>{children}</AppInsightsContextProvider>;
}

export { AppInsightsProvider };

And load in the layout.tsx:

export default function RootLayout({ children }: RootLayoutProps): JSX.Element {
	return (
		<html lang="en">
			<body>
				<AppInsightsProvider>
					{children}
				</AppInsightsProvider>
			</body>
		</html>
	);
}

For context, we were following this implementation: https://github.com/CMeeg/nextjs-aca

cc @Adonas-B

[ricardasjak]

Hello, same issue here. Any plans to release a fix?

[MathiasGruber]

We are also seeing this, quite annoying

[JacksonWeber]

@ricardasjak @MathiasGruber are you both seeing the BatchSpanProcessor error? Or are you referring to the increased load?