Hi,
I am trying to build a singularity container to run FEniCS on HPC clusters. The singularity recipe I wrote is just as:
Bootstrap: docker From: quay.io/fenicsproject/stable:current
The building is successful on my Ubuntu-based virtual machine, but when I tried to run some FEniCS code by:
singularity exec my_fenics.simg python3 my_program.py
The error popped out as:
Thanks
Can you post a minimal code that you tried testing? Also how do you test if the build is “successful” ?
Are you able to shell-in the container and run
python3 -c "from dolfin import *"
I’ve built a couple of containers in the past, most recently here. You can download the container using
singularity pull --name my_fenics.simg shub://bhaveshshrimali/singularitFEniCS:recipe
The recipe files are available here in case if you wish to build locally. Note that the container has some additional libraries that you might not need, so, depending on your needs, you could tweak the recipe accordingly.
Hi,
Thanks for the reply!
The code I used for testing is just:
import fenics as fe
p = fe.Constant(pi/4)
That is what I get in shell mode:
Btw, When I ran normal python programs without FEniCS commands, this container just worked out without errors.
Thanks!
There doesn’t seem to be an installation issue, at least no major one. Could you open a python3 shell inside the container and test the following commands ?
from dolfin import *
p = Constant(pi/4)
Yeah, as you say, it works out.
Thus, Could it be something with the singularity exec command?
Thanks!
Shouldn’t be.
Can you try the same thing from outside the container using singularity exec. Specifically, try
singularity exec fenics.img python3 -c "from dolfin import *; p=Constant(pi/4)"
With your old code, though, you are not importing dolfin with wildcards; pi may not be defined per-se.
Unfortunately, it went back to the original permission denied error . But this time there is some string error shown in the beginning. I tried p = Constant(1), but error was the same.
Thanks
It seems to be file permissions issue that could have crept in when you built the container. Can you post the complete recipe file that you built your container from ? Also, can you try the other container that I posted above, and if you can reproduce this error there too ?
Alternatively this could be a minimal recipe
Bootstrap: docker
From: quay.io/fenicsproject/stable:current
%post
apt-get -y update
apt-get -y install python3-tk
ldconfig
%runscript
exec /bin/bash -i
Hi,
Sorry for the late reply.
Now, the thing is, directly pulling from your repo solves everything! But either this minimal recipe or the recipe from your repo cannot make my local building work(same permission denied error).
Because I do want to install more stuff inside the recipe, thus do you think it is my building command that leads to the problem?
I used sudo singularity build my_fenics.simg my_recipe.def
Thanks!
I see. Can you try building with the same recipe on SingularityHub ?
What happens if you do not run with super user privileges, i.e. without sudo ? As long as you have write-permissions to the directory you are building your image in, it should work. Also, just a check throw in
sudo singularity exec python3 -c "from dolfin import *"
Or see if you can change the file permissions the directory in the image that you posted (it is usually not a good idea to do that). Do you get the same error ?
Do you mean Bootstrap: shub in the first line of the recipe? (Sorry I am new to singularity). But in this way, image path doesn't exist
Hmm… I don’t think I can do that…
I am not sure how to do that…
Thanks!
No, I meant building the container on the cloud. You can do so by linking your GitHub with SingularityHub and create a GitHub repository with your recipe in it, e.g. here. Check this for a more detailed set of instructions. It is convenient this way since you can debug any errors through the log file if the build fails at any point. And once built, pull the container via
singularity pull --name <name>.simg shub://<path_to_container>
Yeah what I meant was to build remotely with the --builder option, much like above.
chmod -R 777 <path_to_directory>
Although I would highly advise against it. Try the above method. I am pretty sure it would work.
Building from remote seems solved my “permission denied problem”, but it still cannot finish the job…
The recipe I used is:
And after I pulled to my local machine, this error popped out
singularity exec my_fenics.simg python3 -c "from dolfin import *; p=Constant(pi/4)"mpi error like:
pip3 install mpi4py inside the recipe, but then the remote build met the failure.
Btw, on the remote, where should I add the --builder, when I choose to manually trigger the build.
Thanks!
Seems to be a problem with Singularity installation similar to as reported here. Which version of singularity are you running and on what version of Ubuntu? How did you install it ?
Shouldn’t have occured. mpi4py is installed and configured in the base docker layer that you are building upon. In any case pip-installing might not directly work for packages. As a workaround, what you could do is
python3 -m pip install <library>
Can you reproduce the first error with a simple Singularity container too ?
Thank you so much sir, problem solved!
A few points that I want to make them clear:
1:
Yes, singularity version < 2.25 will trigger this problem.
2:
This somehow was caused by the Ubuntu-based virtual machine, when I moved the remotely built image to the HPC, no problem.
Thanks again for the help all along the way!
Best,
Grayson
Perfect! Great to hear that!