Fix FEM load-balanced layout reinitialization

[aaadelmann]

This PR fixes the FEM load-balancing failure, in collaboration with Hugo

closing #485.

The real problem was stale FEM field state after repartition. When Alpine triggered load balancing, the
solver called LagrangeSpace::initialize(mesh, layout) again, but the internal FEM workspace field resultField in src/FEM/LagrangeSpace.hpp was only passed through initialize(...). Since Field::initialize() is one-shot, that call became a no-op after the first setup, so resultField kept the old layout and old extents after repartition. The next evaluateAx() then operated on a workspace that no longer matched the redistributed domain decomposition, which explains the failed warmup solve, the huge t=0 field energy, and the later halo/exchange crashes.

The fix is to explicitly resize that workspace on every FEM reinitialization:

• in src/FEM/LagrangeSpace.hpp, resultField.updateLayout(layout) is now called in both layout-aware initialization paths, so the workspace is always rebuilt to the current repartitioned layout
• in alpine/LoadBalancer.hpp, rho_m->setFieldBC(rho_m->getFieldBC()) is called after rho_m.updateLayout(*fl) so periodic BC neighbor metadata is refreshed for the redistributed density field

Behavior After This Change

With this patch, the reduced FEM load-balanced runs no longer exhibit the old failure mode:

• no operator extent mismatch
• no 1000-iteration warmup stall
• no t=0 energy blow-up by orders of magnitude
• no follow-on halo corruption from stale FEM workspace state

This should not change the intended FEM discretization, conservation properties, or solver formulation. It is a state-management fix after repartition, not a numerical algorithm change. The floating-point path should follow the same correct code path as before load balancing; the difference is that the redistributed layout is now actually propagated into the FEM workspace and refreshed BC metadata.

Validation

I validated the fix in two stages.

Local reduced 2-rank CPU case:

• control FEM 1.0: t=0 Ex = 9.9526424907379614
• load-balanced FEM 0.01: t=0 Ex = 9.9532389888656407

A100 2-GPU reduced case on Merlin:

• control FEM 1.0: t=0 Ex = 9.3334714317865917, CG entries 0,23 then 0,12
• load-balanced FEM 0.01: t=0 Ex = 9.3018131240904989, CG entries 0,23 then 0,19

[s-mayani]

I am not sure I understand this updateLayout here, as there is not change of the layout between initialize and updateLayout, so the updateLayout does not need to be called.

[s-mayani]

Same as above.

[s-mayani]

I ran this, it caused an MPI Abort due to invalid count...

[s-mayani]

I ran it with both this change and the updateLayout change now, and it works. However, I do not understand still why the updateLayout is needed. In my opinion the updateLayout should be called before the solve, and not in the LagrangeSpace constructor...

[s-mayani]

I think it works because the loadBalancing only happens once in this Landau Damping case (at the beginning), and hence works with the call only in the constructor. However, if the load balancing changed the domain decomposition again during the simulation, I think it would crash again and cause incorrect results. The solution would be to call the updateLayout on the resultField every time the loadBalancer is called.

[s-mayani]

I have made a branch with the same changes proposed by Hugo but taking into account my comments.
Here it is: https://github.com/s-mayani/ippl/tree/fix_loadbalancing_fem

It seems to me that the addition of the line in alpine/LoadBalancer.hpp is not needed:
rho_m->setFieldBC(rho_m->getFieldBC());
Furthermore, now I have added an updateLayout function which updates the LagrangeSpace every time it is called.

Maybe Hugo can check it, and write a test where the load balancing changes during the simulation such that we can verify it.

[s-mayani]

Hugo is an AI tool

[aaadelmann]

Please push all you changes to and also add the messages of this PR to then lets see what Hugo can do :)

[s-mayani]

I have merged my branch to this one, and will look at the regression test soon.

[s-mayani]

Closing this pull request, clean version in PR #489.