Os.makedirs(path) giving error in remote machine for a parallel run

1

Hi,

I have installed FEniCS 2017.1 in a remote machine. I am trying to run my code in parallel here.

In my code there is a line to create a folder to save some of my outputs, and since its remote machine they don’t allow me to write in HOME directory but in other directories. I am running the code in appropriate place, but FEniCS seems to trying to edit some .cache file while doing this which is only a read only filesystem according to remote machine filesystem. Any ideas, why this happens ?

code portion :

subpath = “…/data/Amats/subdom000”.rstrip(“0”)+str(ip+1)
if not os.path.exists(subpath):
os.makedirs(subpath)

Error :

Traceback (most recent call last):
File “/home/a/asarkar/sudhipv/software/dolfin_intel/lib/python3.6/site-packages/dolfin/compilemodules/jit.py”, line 142
, in jit
result = ffc.jit(ufl_object, parameters=p)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/ffc/jitcompiler.py”, line 204, in jit
module = jit_build(ufl_object, module_name, parameters)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/ffc/jitcompiler.py”, line 120, in jit_b
uild
generate=jit_generate)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/dijitso/jit.py”, line 160, in jit
header, source, dependencies = generate(jitable, name, signature, params[“generator”])
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/ffc/jitcompiler.py”, line 76, in jit_ge
nerate
dep_module_name = jit(dep, parameters, indirect=True)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/ffc/jitcompiler.py”, line 204, in jit
module = jit_build(ufl_object, module_name, parameters)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/ffc/jitcompiler.py”, line 120, in jit_build
generate=jit_generate)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/dijitso/jit.py”, line 173, in jit
params)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/dijitso/build.py”, line 143, in build_shared_library
dependencies, build_params, cache_params)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/dijitso/build.py”, line 60, in make_compile_command
inc_dir = make_inc_dir(cache_params)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/dijitso/cache.py”, line 248, in make_inc_dir
make_dirs(d)
File “/home/a/asarkar/sudhipv/packages/fenics_intel/lib/python3.6/site-packages/dijitso/system.py”, line 114, in make_dirs
os.makedirs(path)
File “/scinet/niagara/software/2018a/opt/base/anaconda3/5.1.0/lib/python3.6/os.py”, line 210, in makedirs
makedirs(head, mode, exist_ok)
File “/scinet/niagara/software/2018a/opt/base/anaconda3/5.1.0/lib/python3.6/os.py”, line 210, in makedirs
makedirs(head, mode, exist_ok)
File “/scinet/niagara/software/2018a/opt/base/anaconda3/5.1.0/lib/python3.6/os.py”, line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 30] Read-only file system: ‘/home/a/asarkar/sudhipv/packages/fenics_intel/.cache’

2

If you are able to set environment variables in your job submission script, you can set a custom location for the cache. Try setting DIJITSO_CACHE_DIR to a location where you have write permissions.

3

I have tried echo $DIJITSO_CACHE_DIR
after activating fenics and it doesn’t return anything for me.

I manually set DIJITSO_CACHE_DIR=./some/location and ran the code but it gives the same error.

Is it happening because of a particular module i use ? Because I have ran the code in serial mode with same version of fenics but installed with different compilers.

4

I don’t know. I’ve actually never had much luck with trying to directly build/install FEniCS on clusters. I would look into using the containerization system Singularity:

Based on some Google searching, it looks like this is available on the cluster you’re using. Basically, you can just convert the FEniCS Docker container (for whatever version you want) into a Singularity image, then run that on any resource that supports Singularity. Of course, in reality, you still have to do some troubleshooting, but, in my experience, the Singularity route is much easier than native installation on clusters.