I am restarting the running kernel: Runtime->Restart Runtime. This was working fine until yesterday. There seems that an update from python 3.6 to python 3.7 is causing the issue.
Hello, a student in my group is affected by this as well.
I can confirm that the issue seems to be that Colab has now switched to Ubuntu 18.04.5 LTS, which now uses python 3.7. I will send a message to Dr. Drew Parsons (who maintains the FEniCS PPA) to see if he has any suggestion.
Hi all, Google have indeed broken their Colab environment. What they need to do is stop running it on Ubuntu bionic 18.04, and start running it on Ubuntu groovy 20.04. Why they have not already done so is beyond me.
Here is a summary of the issues:
The default python on Ubuntu 18.04 is python 3.6, not python 3.7, controlled by the python3-minimal package (version 3.6.7-1~18.04)
Some Python packages (including dolfin) provide Python extensions (local .so libraries compiled from C code using cython). These need to be built against each supported python version separately. On Ubuntu 18.04 the dolfin extension is only build for python 3.6.
Debian provides an alternatives mechanisms (update-alternatives) allowing the system administrator to choose between alternative versions. But this mechanism is not used for python, essentially because of the very sort of problem reported here. The python3-minimal package directly sets the /usr/bin/python3 symlink without using alternatives (3.6.7-1~18.04 sets the symlink to python3.6).
Various pages on the internet instruct you to use the alternatives mechanism to choose your preferred python. If you do so, you will break your python extensions, if they are not built for that version.
Google has evidently chosen to manually apply the alternatives mechanism on the Colab instances to override the default python set by python3-minimal. By doing so they have broken all ubuntu packages providing python extensions. This does not just affect fenics, for instance mpi4py (via the MPI.cpython-36m-x86_64-linux-gnu.so extension):
[] !apt-get install python3-mpi4py
[] from mpi4py import MPI
-> ImportError Traceback (most recent call last)
<ipython-input-16-ac4b05198e5f> in <module>()
----> 1 from mpi4py import MPI
ImportError: cannot import name 'MPI' from 'mpi4py' (/usr/lib/python3/dist-packages/mpi4py/__init__.py)
Google has then manually (meaning outside the Ubuntu packaging system, probably using pip) installed many python modules in /usr/local/lib/python3.7. So if they haven’t installed the module you need (like mpi4py, or dolfin), then too bad. You have to do it yourself (using pip). Ubuntu can’t help you.
The problem is exacerbated by Ubuntu, who explicitly excluded python3.7 from the list of supported python versions known by python3-minimal on bionic 18.04, see 3.6.7-1~18.04 : python3-defaults package : Ubuntu . Otherwise, the python extensions could have been built for python3.7 even if it wasn’t the default version.
So Google appears to be expecting you to either not use additional python modules on Colab (which means no FEniCS on Colab), or else to install them manually using pip.
For a package the size of dolfin, reinstalling via pip is not a lot of fun, but it can be done. This recipe (from Chris Richardson) works: [edit: build dolfin first, mpi4py second]
# Add FEniCS PPA to Ubuntu and install (Python3.6)
!echo deb http://ppa.launchpad.net/fenics-packages/fenics/ubuntu bionic main >> /etc/apt/sources.list
!apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 2C5275D7EF63D9DE2D28D3702940F5212B746472
!apt-get -qq update
!apt-get -y install fenics > /dev/null
# Reinstall dolfin for Python3.7
!rm -r /usr/lib/petsc/lib/python3/dist-packages/dolfin
!git clone https://bitbucket.org/fenics-project/dolfin
!cd dolfin/python && python3 setup.py install
# Reinstall mpi4py for Python3.7
!rm -r /usr/lib/python3/dist-packages/mpi4py*
!pip3 install mpi4py --upgrade
[Edit] Intuitively you would expect that you should rebuild mpi4py first, but if you try that then the dolfin build loses track of petsc4py and fails. Build dolfin first as shown here, and it will work.
Note that dolfin itself (the C++ library) does not need to be rebuilt, only the python module needs rebuilding. So the procedure is not too painful (around 10 minutes).
Rebuilding petsc4py against python3.7, as done for mpi4py, fails. It means petsc4py will not be available for you. There is a reason why the Linux distributions exist, rather than installing everything ad hoc via pip. The distributions ensure self-consistency between the various package versions across the entire installation.
We are working on a further workaround which includes precompiled petsc4py, slepc4py, dolfin and mshr. Please join the #colab channel on the fenicsproject slack for further info, and report your feedback there. See Community – FEniCS Project for info on how to join slack.
For now I refrain from posting the workaround here because we might need to tweak it a little bit in the next few days, and it is more convenient to have the feedback, discussion and updates happening in one place (#colab channel on slack) rather than scattered in several different platforms.
Dear all,
in the last few weeks we have been working towards a better and more sustainable packaging of the former Colab workaround, that was posted a few months ago on the #colab channel on slack.
You may find the results of such effort at https://fem-on-colab.github.io/ . FEniCS, FEniCSx and firedrake are now supported. In case of technical issues feel free to leave your feedback on our github page.
The files associated to the former workaround will not be deleted, so it will continue working (at least until the next Colab upgrade )
)