When I was trying to remove the package using pip3 uninstall ufl I could not fix it either, since pip did not see it for some reason. I think that uninstalling the package using pip, leaves some stuff in the site-packages directory.
I had to remove the ufl package manually using:
cd /home/username/.local/lib/python3.6/site-packages/
sudo rm -r ufl/
I should have mentioned that earlier.