Time series data in pyvista - XdmfReader

Version 0.39.0 of pyvista, released three days ago, now features an XdmfReader.

I’ve been using it quite a lot for post-processing time-series data created with fenicsx, so I thought I’d provide a simple example in case anyone else was interested.

import pyvista as pv
from dolfinx import fem, io, mesh 
from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.rank 

ncells = 100
domain = mesh.create_unit_square(comm, ncells, ncells)

V = fem.FunctionSpace(domain, ("CG", 1))
phi = fem.Function(V)
phi.name = "phi"  # This determines the name in reader.set_active_scalars("phi")

# Create xdmf file for saving time series
xdmf = io.XDMFFile(comm, "phi.xdmf", "w")
xdmf.write_mesh(domain)

# Set phi to an arbitrary function at three times and save to xdmf file
t = 0.0
phi.interpolate(lambda x: (t + 0.1) * x[0])
xdmf.write_function(phi, t)
t = 0.5
phi.interpolate(lambda x: (t + 0.1) * x[0])
xdmf.write_function(phi, t)
t = 1.0
phi.interpolate(lambda x: (t + 0.1) * x[0])
xdmf.write_function(phi, t)
xdmf.close()

# Plot with pyvista on one rank - most likely you would put the following in a 
# separate script and only run it with one core
if rank == 0:
    def plot_sol(func, min, max):
        """ Analyse 2d solution over time """
        cmap = "plasma"
        def load_time(value):
            """ Load solution at value specified using the slider """
            reader.set_active_time_value(value)
            grid = reader.read()[1]
            grid.set_active_scalars(func)
            p.add_mesh(grid, cmap=cmap, clim=[min, max], scalar_bar_args=sargs)

        # Load in data
        xdmf_file = "phi.xdmf"
        reader = pv.get_reader(xdmf_file)
        reader.set_active_time_value(0.0)
        grid = reader.read()[1]
        grid.set_active_scalars(func)
        # Create plot
        width = 0.6
        sargs = dict(width=width, height=0.1, position_x=(1-width)/2, position_y=0.01)
        p = pv.Plotter()
        p.add_mesh(grid, cmap=cmap, clim=[min, max], scalar_bar_args=sargs)
        p.show_bounds(xtitle="", ytitle="", ztitle="")
        p.view_xy()
        p.add_slider_widget(load_time, [0.0, 1.0], value=0.0, title="Time",
                            interaction_event="always", pointa=(0.25, 0.93), 
                            pointb=(0.75, 0.93))
        p.show()

    plot_sol("phi", 0.0, 1.1)

Some example screenshots:



4 Likes

Dear @nav, thanks for your post. I tried to run your code (in serial, i.e. by running python3 filename.py) and obtain the following error:

[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see https://petsc.org/release/faq/#valgrind and https://petsc.org/release/faq/
[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run 
[0]PETSC ERROR: to get more information on the crash.
[0]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.
Abort(59) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0

Do you know what could be the issue?

I have been aware of this xdmf-reader feature, and tried to implement a workflow like you have posted above before, but failed to do so.
One example of what I tried to do is visible in this post. Your code is supposed to do exactly what I was asking for, so it would provide an answer to the question I posted there.

Looking forward to your answer.


EDIT: Here is a minimal example that reproduces the error:

import pyvista as pv
from dolfinx import fem, io, mesh 
from mpi4py import MPI

domain = mesh.create_unit_square(MPI.COMM_WORLD, 10, 10)
V = fem.FunctionSpace(domain, ("CG", 1))
phi = fem.Function(V)
phi.name = "phi"  # This determines the name in reader.set_active_scalars("phi")

# Create xdmf file for saving time series
xdmf = io.XDMFFile(MPI.COMM_WORLD, "phi.xdmf", "w")
xdmf.write_mesh(domain)

# Set phi to an arbitrary function at three times and save to xdmf file
t = 0.0
phi.interpolate(lambda x: (t + 0.1) * x[0])
xdmf.write_function(phi, t)
t = 0.5
phi.interpolate(lambda x: (t + 0.1) * x[0])
xdmf.write_function(phi, t)
t = 1.0
phi.interpolate(lambda x: (t + 0.1) * x[0])
xdmf.write_function(phi, t)
xdmf.close()

# read file and plot
xdmf_file = "phi.xdmf"
reader = pv.get_reader(xdmf_file)
reader.set_active_time_value(0.0)
# grid = reader.read()[1]
# grid.set_active_scalars("phi")
# p = pv.Plotter()
# p.add_mesh(grid, cmap="plasma", clim=[0.0,1.0])
# p.show()

The line reader.set_active_time_value(0.0) seems to cause the problem.

Hi @renol-kth,

Just to check, does the xdmf file save with no errors when you run the script with python3?

Also, what version of pyvista are you using?

Hi, thank you for your response.
Yes, when I call python3 FILENAME.py the code produces two files:

  • phi.h5
  • phi.xdmf

Or in other words: if I call the script from my previous message and remove everything after the line

xdmf.close()

it runs without returning any errors.

The command print(pyvista.__version__) returns 0.42.3 which - to my knowledge - should be sufficient to support the XdmfReader.

This suggests to me that it is entirely a pyvista problem. I just installed pyvista 0.42.3. When I run the following with python

import pyvista as pv

def plot_sol(func, min, max):
    """ Analyse 2d solution over time """
    cmap = "plasma"
    def load_time(value):
        """ Load solution at value specified using the slider """
        reader.set_active_time_value(value)
        grid = reader.read()[1]
        grid.set_active_scalars(func)
        p.add_mesh(grid, cmap=cmap, clim=[min, max], scalar_bar_args=sargs)

    # Load in data
    xdmf_file = "phi.xdmf"
    reader = pv.get_reader(xdmf_file)
    reader.set_active_time_value(0.0)
    grid = reader.read()[1]
    grid.set_active_scalars(func)
    # Create plot
    width = 0.6
    sargs = dict(width=width, height=0.1, position_x=(1-width)/2, position_y=0.01)
    p = pv.Plotter()
    p.add_mesh(grid, cmap=cmap, clim=[min, max], scalar_bar_args=sargs)
    p.show_bounds(xtitle="", ytitle="", ztitle="")
    p.view_xy()
    p.add_slider_widget(load_time, [0.0, 1.0], value=0.0, title="Time",
                        interaction_event="always", pointa=(0.25, 0.93), 
                        pointb=(0.75, 0.93))
    p.show()

plot_sol("phi", 0.0, 1.1)

it all works fine with no errors. Is this the case with you, as well?

Please consider discussing this on the pyvista discussion forum pyvista/pyvista · Discussions · GitHub. This is not the support forum of pyvista, hence you can’t expect to get meaningful support here.

I arrived late to the party, and thus I’ll leave this open. If I had arrived on time, I would have closed this thread at post #1.

No, when I run your script i get the following

Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))
==== backtrace (tid: 697516) ====
 0  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../../././././libucs.so.0(ucs_handle_error+0x2fd) [0x7fbe26080c1d]
 1  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../../././././libucs.so.0(+0x2de11) [0x7fbe26080e11]
 2  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../../././././libucs.so.0(+0x2dfda) [0x7fbe26080fda]
 3  /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fbe55842520]
 4  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkxdmf2-9.3.so.1(_ZN5xdmf27XdmfDOM11FindElementEPKciP8_xmlNodei+0x19) [0x7fbe29bbc139]
 5  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../../libvtkIOXdmf2-9.3.so.1(_ZN16vtkXdmfHeavyData15ReadUniformDataEPN5xdmf28XdmfGridEi+0x265) [0x7fbe2a0c0125]
 6  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../../libvtkIOXdmf2-9.3.so.1(_ZN16vtkXdmfHeavyData22ReadTemporalCollectionEPN5xdmf28XdmfGridEi+0x2ce) [0x7fbe2a0c124e]
 7  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../../libvtkIOXdmf2-9.3.so.1(_ZN16vtkXdmfHeavyData8ReadDataEv+0x2ba) [0x7fbe2a0c0cca]
 8  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../../libvtkIOXdmf2-9.3.so.1(_ZN13vtkXdmfReader11RequestDataEP14vtkInformationPP20vtkInformationVectorS3_+0x278) [0x7fbe2a0a8d98]
 9  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN12vtkExecutive13CallAlgorithmEP14vtkInformationiPP20vtkInformationVectorS3_+0x54) [0x7fbe50cab974]
10  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN23vtkDemandDrivenPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x37) [0x7fbe50ca3d57]
11  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN24vtkCompositeDataPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x8a) [0x7fbe50ca115a]
12  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x58c) [0x7fbe50ca768c]
13  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x394) [0x7fbe50cec334]
14  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN32vtkStreamingDemandDrivenPipeline6UpdateEiP20vtkInformationVector+0x123) [0x7fbe50cedb83]
15  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN12vtkAlgorithm6UpdateEP14vtkInformation+0x33) [0x7fbe50c96523]
16  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/../../.././libvtkCommonExecutionModel-9.3.so.1(_ZN12vtkAlgorithm14UpdateTimeStepEdiiiPKi+0x78) [0x7fbe50c96738]
17  /home/renoldner/miniconda3/envs/fenicsx/lib/python3.10/site-packages/vtkmodules/vtkCommonExecutionModel.cpython-310-x86_64-linux-gnu.so(+0x30d1f) [0x7fbe4f8c1d1f]
18  python3(+0x144468) [0x5b9d7d57c468]
19  python3(_PyObject_MakeTpCall+0x26b) [0x5b9d7d57597b]
20  python3(_PyEval_EvalFrameDefault+0x54b6) [0x5b9d7d5718c6]
21  python3(_PyFunction_Vectorcall+0x6c) [0x5b9d7d57c8cc]
22  python3(_PyEval_EvalFrameDefault+0x72c) [0x5b9d7d56cb3c]
23  python3(_PyFunction_Vectorcall+0x6c) [0x5b9d7d57c8cc]
24  python3(_PyEval_EvalFrameDefault+0x72c) [0x5b9d7d56cb3c]
25  python3(_PyObject_FastCallDictTstate+0xd0) [0x5b9d7d574e60]
26  python3(+0x14df41) [0x5b9d7d585f41]
27  python3(_PyObject_MakeTpCall+0x27b) [0x5b9d7d57598b]
28  python3(_PyEval_EvalFrameDefault+0x4eba) [0x5b9d7d5712ca]
29  python3(_PyFunction_Vectorcall+0x6c) [0x5b9d7d57c8cc]
30  python3(_PyEval_EvalFrameDefault+0x4c12) [0x5b9d7d571022]
31  python3(_PyFunction_Vectorcall+0x6c) [0x5b9d7d57c8cc]
32  python3(_PyEval_EvalFrameDefault+0x320) [0x5b9d7d56c730]
33  python3(+0x1d7870) [0x5b9d7d60f870]
34  python3(PyEval_EvalCode+0x87) [0x5b9d7d60f7b7]
35  python3(+0x207d1a) [0x5b9d7d63fd1a]
36  python3(+0x203123) [0x5b9d7d63b123]
37  python3(+0x9a4d1) [0x5b9d7d4d24d1]
38  python3(_PyRun_SimpleFileObject+0x1ae) [0x5b9d7d63560e]
39  python3(_PyRun_AnyFileObject+0x44) [0x5b9d7d6351a4]
40  python3(Py_RunMain+0x38b) [0x5b9d7d63239b]
41  python3(Py_BytesMain+0x37) [0x5b9d7d602e17]
42  /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fbe55829d90]
43  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7fbe55829e40]
44  python3(+0x1cad11) [0x5b9d7d602d11]
=================================
Segmentation fault (core dumped)

Similar as before the xdmf reader seems to be the problem - to be precise, if I remove everything afer the line

reader = pv.get_reader(xdmf_file)

I still get the error message from above.

Dear @francesco-ballarin, thank you for the advice, I posted it on the pyvista github as well, see here.

I agree, that the problem is probably my pyvista installation. However, this Forum here contains many people that can in fact help with pyvista problems related to fenicsx (which is exactly what I need).

And moreover, the code snippet posted in the first message is (or would be, if it would work for me) extremely useful, as it adds a major new capability of fenicsx: exporting and importing time-dependent functions in an efficient way. I think, if it would have been posted on the pyvista github page, fenicsx users would not have found it.

The main issue with with XDMFFile is that it is a very generic format, and people rarely implement a generic enough reader to support all cases.

You can for instance see this with:

  1. Paraview having three XDMF standards implemented
  2. Meshio only supporting xdmf files with a single grid in them
  3. Legacy dolfin only supporting xdmf files with a single grid.
  4. DOLFINx supporting multiple grids in the XDMFFile’s but limited ability to read out multiple data collections per mesh, which is currently being extended by @Massimiliano_Leoni (Allow specifying attribute name when reading meshtag by mleoni-pf · Pull Request #3257 · FEniCS/dolfinx · GitHub).
  5. XDMF having both an ascii and binary output version, which means that one has to implement both.

For your issue at:

I would suggest that you:

  1. Use the ascii writer to write out the XDMF in DOLFINx (set encoding=XDMFFile.Encoding.ASCII) in XDMFFile, as done in: dolfinx/python/demo/demo_static-condensation.py at a547191a3c2765acd12aeadd0a3a66c0b99e83a9 · FEniCS/dolfinx · GitHub
  2. Remove the import of dolfinx and MPI4PY and dolfinx completely from the script, and just post the raw ASCII xdmf file, and only run pyvista import and commands with pyvista. That will likely give you a better error trace and make it way easier for the pyvista developers to pinpoint what is wrong.
1 Like

Dear Dokken, thank you for your response.

I understand why XDMF has its flaws. Similar points were also mentioned in other places here on the forum.

But - apart from the ADIOS4DOLFINx, which seems to be quite a complex software package - it seems to me there is nothing else that does the job; namely efficiently saving time-dependent PDE solutions from fenicsx and reading them to pyvista (although being only single-grid capable). Please let me know, if I am missing something here. And the script of @nav does exactly that in a very simple way.

Thank you very much for your advice on how to improve my post on the pyvista github discussion, I highly appreciate it.

As I’m the author of adios4dolfinx I am of course biased towards adios4dolfinx for storing data.could you elaborate on what makes you consider adios4dolfinx as a complex software package?

I have not yet managed to install it properly. Moreover, adios does much more than xdmf - namely it allows to store multiple meshes, e.g. if one adapts the mesh during a simulation. This is of course fantastic, but the little snippet posted by @nav seemed like the solution with the lowest entry barrier for me.

Adios4dolfinx is on conda, can be installed with pip if you have adios2 compiled with mpi (this is provided in the dolfinx docker images), and it is on Debian: Debian -- Details of package python3-adios4dolfinx in sid

What options have you tried and what are the issues you are facing?

Sure, if it works it is clearly the easiest route.
Note that it will only work for continuous P1 function spaces, as that is what is supported by XDMFFile.

1 Like

Could it be that your issue is related to this? Compare your vtk version with the version in that post. The code I posted works for me with pyvista 0.42.3 and vtk 9.3.1 My last suggestion would be to create a new conda environment and install just pyvista in that environment, ensuring that the versions are up to date, and see if the pyvista only code that I posted here works. If it does, that would suggest to me that it was simply a vtk issue, as suggested by tkoyama010.

1 Like

I used Conda, but it was on another machine I am currently working on, so I can not recall the exact versions and setup I had there anymore. I will give it another shot if necessary. Is the ADIOS package planned to be integrated into fenicsx at any point?

I do work pyvista 0.44.1 and vtk 9.3.1, so the versions should be up to date. In particular, they are newer than the one from this github post.
I will try run it in a clean environment.

It is a long term goal. But getting an abstraction and interface all the developers can agree on have proven challenging :wink:

1 Like