PETSc problem when solving mixed 3D nonlinear variational problem

Change the underlying solver of your problem (as UMFPACK, the default LU solver is running out of memory). See for instance: UMFPACK error: out of memory despite system having free memory - #2 by plugged