Why does PETSc krylov solver fail on a more resolved mesh on the same exact problem?

Hi,

I am trying to solve a steady state fluid dynamics simulation on a large 3D mesh. I ran the same simulation on two different mesh density, 5 millions elements, and 10 millions elements. The 10 millions elements simulation fails while the 5 millions elements converge to a reasonable solution. For these simulations, I use bicgstab solver with hypre_amg preconditioner.

The 5 millions element simulation took approximately 230 krylov iterations to solve a linear system, but the 10 millions element fail during the linear solve at iteration 172 with “PETSc reason DIVERGED_BREAKDOWN”. According to the convergence profile (attached below), it seems like the residue is decreasing, so I don’t understand how I get the DIVERGED_BREAKDOWN error.

I am wondering if anyone have any insights on why this problem is occuring since this is the exact same problem, but with a more refined mesh. My only guess is that since the linear system is far larger, there could be problems with converging to a solution, but the convergence profile seems like it is steadly, albeit slowly, converging so I’m quite sure if this is the case.

For the case of 10 millions elements, the simulation linear solver fail at step 172 with the convergence profile:

  Process 0: Newton iteration 0: r (abs) = 2.337e+03 (tol = 1.000e-08) r (rel) = 1.000e+00 (tol = 1.000e-06)
  0 KSP preconditioned resid norm 2.324400596072e+04 true resid norm 2.337075071461e+03 ||r(i)||/||b|| 1.000000000000e+00
  1 KSP preconditioned resid norm 7.298782511593e+03 true resid norm 2.207104758357e+03 ||r(i)||/||b|| 9.443876173724e-01
  2 KSP preconditioned resid norm 6.609120563398e+03 true resid norm 2.143726786754e+03 ||r(i)||/||b|| 9.172691168256e-01
  3 KSP preconditioned resid norm 1.629536115182e+04 true resid norm 4.429999327660e+03 ||r(i)||/||b|| 1.895531462278e+00
  4 KSP preconditioned resid norm 3.118857313069e+04 true resid norm 3.966718153232e+03 ||r(i)||/||b|| 1.697300271468e+00
  5 KSP preconditioned resid norm 1.852758121499e+04 true resid norm 4.900672588390e+03 ||r(i)||/||b|| 2.096925617938e+00
  6 KSP preconditioned resid norm 1.845366858139e+04 true resid norm 4.754890752197e+03 ||r(i)||/||b|| 2.034547717470e+00
  7 KSP preconditioned resid norm 1.948600192552e+04 true resid norm 4.677737472372e+03 ||r(i)||/||b|| 2.001534965433e+00
  8 KSP preconditioned resid norm 1.953187402797e+04 true resid norm 4.352042075995e+03 ||r(i)||/||b|| 1.862174702533e+00
  9 KSP preconditioned resid norm 1.929087745795e+04 true resid norm 4.930646500263e+03 ||r(i)||/||b|| 2.109751013339e+00
 10 KSP preconditioned resid norm 2.136486559517e+04 true resid norm 4.940849835525e+03 ||r(i)||/||b|| 2.114116870211e+00
 11 KSP preconditioned resid norm 2.123331514864e+04 true resid norm 3.898411829344e+03 ||r(i)||/||b|| 1.668073001569e+00
 12 KSP preconditioned resid norm 2.208954444421e+04 true resid norm 5.927731976665e+03 ||r(i)||/||b|| 2.536389202491e+00
 13 KSP preconditioned resid norm 4.090694668291e+04 true resid norm 8.896046599097e+03 ||r(i)||/||b|| 3.806487308744e+00
 14 KSP preconditioned resid norm 3.902795208411e+04 true resid norm 8.665571913444e+03 ||r(i)||/||b|| 3.707870585444e+00
 15 KSP preconditioned resid norm 1.109380319061e+04 true resid norm 3.507217545447e+03 ||r(i)||/||b|| 1.500686729440e+00
 16 KSP preconditioned resid norm 1.141769870038e+04 true resid norm 2.848511881715e+03 ||r(i)||/||b|| 1.218836278090e+00
 17 KSP preconditioned resid norm 1.119391859291e+04 true resid norm 2.938434864210e+03 ||r(i)||/||b|| 1.257312997812e+00
 18 KSP preconditioned resid norm 1.208272075095e+04 true resid norm 2.637574372069e+03 ||r(i)||/||b|| 1.128579224638e+00
 19 KSP preconditioned resid norm 1.194240013234e+04 true resid norm 2.521204390718e+03 ||r(i)||/||b|| 1.078786223646e+00
 20 KSP preconditioned resid norm 4.019557253284e+04 true resid norm 2.767666567206e+03 ||r(i)||/||b|| 1.184243758792e+00
 21 KSP preconditioned resid norm 3.698157476295e+04 true resid norm 6.343811459058e+03 ||r(i)||/||b|| 2.714423484519e+00
 22 KSP preconditioned resid norm 2.528262405370e+04 true resid norm 1.124804761394e+02 ||r(i)||/||b|| 4.812873900071e-02
 23 KSP preconditioned resid norm 2.178332811807e+04 true resid norm 3.742008536941e+02 ||r(i)||/||b|| 1.601150336434e-01
 24 KSP preconditioned resid norm 3.617194546025e+04 true resid norm 1.361137162729e+03 ||r(i)||/||b|| 5.824105435683e-01
 25 KSP preconditioned resid norm 3.568526268436e+04 true resid norm 1.208658272712e+03 ||r(i)||/||b|| 5.171670724108e-01
 26 KSP preconditioned resid norm 3.610837076675e+04 true resid norm 1.175354621391e+03 ||r(i)||/||b|| 5.029169305443e-01
 27 KSP preconditioned resid norm 3.492338489164e+04 true resid norm 9.950131089672e+02 ||r(i)||/||b|| 4.257514536515e-01
 28 KSP preconditioned resid norm 2.815365944851e+04 true resid norm 6.476111968135e+02 ||r(i)||/||b|| 2.771032923682e-01
 29 KSP preconditioned resid norm 1.032581514763e+05 true resid norm 6.152337801749e+03 ||r(i)||/||b|| 2.632494726797e+00
 30 KSP preconditioned resid norm 3.112114214388e+04 true resid norm 1.657348405937e+02 ||r(i)||/||b|| 7.091549716033e-02
 31 KSP preconditioned resid norm 3.027587725369e+04 true resid norm 4.189517857740e+02 ||r(i)||/||b|| 1.792632983382e-01
 32 KSP preconditioned resid norm 3.200180532201e+04 true resid norm 7.094088938094e+01 ||r(i)||/||b|| 3.035456166866e-02
 33 KSP preconditioned resid norm 8.848602582734e+04 true resid norm 3.247664766851e+03 ||r(i)||/||b|| 1.389627918465e+00
 34 KSP preconditioned resid norm 7.057601561743e+04 true resid norm 2.380062969450e+03 ||r(i)||/||b|| 1.018393888375e+00
 35 KSP preconditioned resid norm 1.810862941745e+05 true resid norm 5.922447628134e+03 ||r(i)||/||b|| 2.534128107589e+00
 36 KSP preconditioned resid norm 1.874351423311e+04 true resid norm 2.497136922277e+02 ||r(i)||/||b|| 1.068488108393e-01
 37 KSP preconditioned resid norm 4.934968830946e+04 true resid norm 1.398865433053e+03 ||r(i)||/||b|| 5.985539147352e-01
 38 KSP preconditioned resid norm 4.087405644345e+04 true resid norm 9.546600461497e+02 ||r(i)||/||b|| 4.084849724373e-01
 39 KSP preconditioned resid norm 5.179269120279e+04 true resid norm 1.409803818970e+03 ||r(i)||/||b|| 6.032342889563e-01
 40 KSP preconditioned resid norm 4.783751182393e+04 true resid norm 1.236611095802e+03 ||r(i)||/||b|| 5.291276736905e-01
 41 KSP preconditioned resid norm 9.652419143083e+03 true resid norm 1.415311955121e+02 ||r(i)||/||b|| 6.055911392852e-02
 42 KSP preconditioned resid norm 4.188990025982e+03 true resid norm 3.140955085273e+02 ||r(i)||/||b|| 1.343968417459e-01
 43 KSP preconditioned resid norm 1.557960518485e+03 true resid norm 3.372635632759e+02 ||r(i)||/||b|| 1.443101111275e-01
 44 KSP preconditioned resid norm 1.528723890478e+03 true resid norm 2.987658238884e+02 ||r(i)||/||b|| 1.278374954818e-01
 45 KSP preconditioned resid norm 5.322628989153e+03 true resid norm 5.815982998376e+02 ||r(i)||/||b|| 2.488573460647e-01
 46 KSP preconditioned resid norm 1.851583005958e+03 true resid norm 3.197324006769e+02 ||r(i)||/||b|| 1.368087848701e-01
 47 KSP preconditioned resid norm 1.724140965713e+03 true resid norm 2.911844663093e+02 ||r(i)||/||b|| 1.245935442404e-01
 48 KSP preconditioned resid norm 1.693270162308e+03 true resid norm 2.879613820059e+02 ||r(i)||/||b|| 1.232144339402e-01
 49 KSP preconditioned resid norm 1.575824519497e+03 true resid norm 2.674898276342e+02 ||r(i)||/||b|| 1.144549573527e-01
 50 KSP preconditioned resid norm 2.416072085131e+03 true resid norm 2.747866904117e+01 ||r(i)||/||b|| 1.175771774588e-02
 51 KSP preconditioned resid norm 1.662645645631e+03 true resid norm 1.868856805953e+02 ||r(i)||/||b|| 7.996563006358e-02
 52 KSP preconditioned resid norm 1.533435932388e+03 true resid norm 2.671136518022e+02 ||r(i)||/||b|| 1.142939972550e-01
 53 KSP preconditioned resid norm 1.548948753318e+03 true resid norm 2.521138840011e+02 ||r(i)||/||b|| 1.078758175464e-01
 54 KSP preconditioned resid norm 1.589916965325e+03 true resid norm 2.323190578694e+02 ||r(i)||/||b|| 9.940590300513e-02
 55 KSP preconditioned resid norm 1.586895992094e+03 true resid norm 2.391342022989e+02 ||r(i)||/||b|| 1.023220029254e-01
 56 KSP preconditioned resid norm 1.578194536857e+03 true resid norm 2.602791401439e+02 ||r(i)||/||b|| 1.113696103828e-01
 57 KSP preconditioned resid norm 1.575484126904e+03 true resid norm 2.528104948887e+02 ||r(i)||/||b|| 1.081738870847e-01
 58 KSP preconditioned resid norm 1.614403900909e+03 true resid norm 2.606777662836e+02 ||r(i)||/||b|| 1.115401766365e-01
 59 KSP preconditioned resid norm 1.604849073328e+03 true resid norm 2.481293616203e+02 ||r(i)||/||b|| 1.061708991082e-01
 60 KSP preconditioned resid norm 1.604915604873e+03 true resid norm 2.483579805372e+02 ||r(i)||/||b|| 1.062687217753e-01
 61 KSP preconditioned resid norm 1.611718093858e+03 true resid norm 2.505489226061e+02 ||r(i)||/||b|| 1.072061936160e-01
 62 KSP preconditioned resid norm 1.615566974881e+03 true resid norm 2.477224140121e+02 ||r(i)||/||b|| 1.059967722206e-01
 63 KSP preconditioned resid norm 2.464433778373e+03 true resid norm 1.255794416862e+02 ||r(i)||/||b|| 5.373359342186e-02
 64 KSP preconditioned resid norm 1.738189243288e+03 true resid norm 2.642664814899e+02 ||r(i)||/||b|| 1.130757350147e-01
 65 KSP preconditioned resid norm 3.236027921686e+03 true resid norm 3.245776519774e+02 ||r(i)||/||b|| 1.388819965353e-01
 66 KSP preconditioned resid norm 1.950173380086e+03 true resid norm 1.870197540947e+02 ||r(i)||/||b|| 8.002299813920e-02
 67 KSP preconditioned resid norm 3.027106868356e+03 true resid norm 1.066068292478e+02 ||r(i)||/||b|| 4.561549200951e-02
 68 KSP preconditioned resid norm 1.783294610653e+03 true resid norm 2.233519026829e+02 ||r(i)||/||b|| 9.556898937924e-02
 69 KSP preconditioned resid norm 1.923839924870e+03 true resid norm 2.374275783092e+02 ||r(i)||/||b|| 1.015917636573e-01
 70 KSP preconditioned resid norm 1.769000814405e+03 true resid norm 1.997897893757e+02 ||r(i)||/||b|| 8.548710814445e-02
 71 KSP preconditioned resid norm 1.769012445257e+03 true resid norm 1.980459668411e+02 ||r(i)||/||b|| 8.474095216689e-02
 72 KSP preconditioned resid norm 1.903304080909e+03 true resid norm 9.737652175182e+01 ||r(i)||/||b|| 4.166597938634e-02
 73 KSP preconditioned resid norm 1.805098136438e+03 true resid norm 1.149789233738e+02 ||r(i)||/||b|| 4.919778777234e-02
 74 KSP preconditioned resid norm 1.841253453700e+03 true resid norm 9.881834731882e+01 ||r(i)||/||b|| 4.228291530963e-02
 75 KSP preconditioned resid norm 1.795459510826e+03 true resid norm 1.145905413260e+02 ||r(i)||/||b|| 4.903160481464e-02
 76 KSP preconditioned resid norm 2.672302479492e+03 true resid norm 1.105706510008e+01 ||r(i)||/||b|| 4.731155295394e-03
 77 KSP preconditioned resid norm 2.894980011473e+03 true resid norm 3.237127709264e+01 ||r(i)||/||b|| 1.385119266725e-02
 78 KSP preconditioned resid norm 2.303744127923e+03 true resid norm 3.142623566023e+01 ||r(i)||/||b|| 1.344682335796e-02
 79 KSP preconditioned resid norm 1.340903705074e+04 true resid norm 5.635512175856e+02 ||r(i)||/||b|| 2.411352653868e-01
 80 KSP preconditioned resid norm 1.307893562248e+06 true resid norm 5.094282512796e+04 ||r(i)||/||b|| 2.179768452886e+01
 81 KSP preconditioned resid norm 2.419382152654e+04 true resid norm 7.762758648885e+02 ||r(i)||/||b|| 3.321570087191e-01
 82 KSP preconditioned resid norm 2.626782480611e+04 true resid norm 6.899886248156e+02 ||r(i)||/||b|| 2.952359696278e-01
 83 KSP preconditioned resid norm 2.575077646263e+04 true resid norm 5.884320624718e+02 ||r(i)||/||b|| 2.517814124404e-01
 84 KSP preconditioned resid norm 2.839490449482e+04 true resid norm 5.465046904847e+02 ||r(i)||/||b|| 2.338413075208e-01
 85 KSP preconditioned resid norm 2.855939320307e+04 true resid norm 4.874283672358e+02 ||r(i)||/||b|| 2.085634189453e-01
 86 KSP preconditioned resid norm 2.643145472291e+04 true resid norm 4.006903248653e+02 ||r(i)||/||b|| 1.714494881908e-01
 87 KSP preconditioned resid norm 2.076082169500e+04 true resid norm 3.136611443321e+02 ||r(i)||/||b|| 1.342109837045e-01
 88 KSP preconditioned resid norm 1.362688511775e+04 true resid norm 2.024765484063e+02 ||r(i)||/||b|| 8.663673276005e-02
 89 KSP preconditioned resid norm 1.365673894976e+04 true resid norm 2.161448429836e+02 ||r(i)||/||b|| 9.248519468758e-02
 90 KSP preconditioned resid norm 1.306639168557e+04 true resid norm 1.535587295018e+02 ||r(i)||/||b|| 6.570551856764e-02
 91 KSP preconditioned resid norm 1.656275169283e+04 true resid norm 3.350994686184e+02 ||r(i)||/||b|| 1.433841268988e-01
 92 KSP preconditioned resid norm 1.080908045366e+05 true resid norm 1.593663483300e+03 ||r(i)||/||b|| 6.819051312303e-01
 93 KSP preconditioned resid norm 8.574935417853e+04 true resid norm 1.249716804859e+03 ||r(i)||/||b|| 5.347354135603e-01
 94 KSP preconditioned resid norm 1.924705438157e+04 true resid norm 3.796812049064e+02 ||r(i)||/||b|| 1.624599952063e-01
 95 KSP preconditioned resid norm 2.557052946468e+04 true resid norm 5.254485002611e+02 ||r(i)||/||b|| 2.248316738635e-01
 96 KSP preconditioned resid norm 2.578308869810e+04 true resid norm 5.339661261193e+02 ||r(i)||/||b|| 2.284762405110e-01
 97 KSP preconditioned resid norm 4.678114455336e+04 true resid norm 9.021999205191e+02 ||r(i)||/||b|| 3.860380573719e-01
 98 KSP preconditioned resid norm 2.597927393083e+04 true resid norm 4.391805731031e+02 ||r(i)||/||b|| 1.879188984839e-01
 99 KSP preconditioned resid norm 2.719698672956e+04 true resid norm 5.430204617891e+02 ||r(i)||/||b|| 2.323504573816e-01
100 KSP preconditioned resid norm 2.908215535292e+04 true resid norm 3.620442789965e+02 ||r(i)||/||b|| 1.549134143860e-01
101 KSP preconditioned resid norm 3.083301233320e+04 true resid norm 4.327894971048e+02 ||r(i)||/||b|| 1.851842511992e-01
102 KSP preconditioned resid norm 8.330791278099e+04 true resid norm 1.967528569289e+03 ||r(i)||/||b|| 8.418764948186e-01
103 KSP preconditioned resid norm 8.754336094804e+04 true resid norm 1.965681853676e+03 ||r(i)||/||b|| 8.410863124081e-01
104 KSP preconditioned resid norm 2.088937834323e+05 true resid norm 4.132635944500e+03 ||r(i)||/||b|| 1.768294050527e+00
105 KSP preconditioned resid norm 3.246588922700e+04 true resid norm 2.350937513404e+02 ||r(i)||/||b|| 1.005931534726e-01
106 KSP preconditioned resid norm 2.333104330738e+06 true resid norm 4.108842348397e+04 ||r(i)||/||b|| 1.758113121214e+01
107 KSP preconditioned resid norm 8.436422374499e+04 true resid norm 1.084611064028e+03 ||r(i)||/||b|| 4.640890989222e-01
108 KSP preconditioned resid norm 8.243641456073e+04 true resid norm 9.992764285259e+02 ||r(i)||/||b|| 4.275756652957e-01
109 KSP preconditioned resid norm 7.065213121137e+05 true resid norm 1.046388930879e+04 ||r(i)||/||b|| 4.477344111266e+00
110 KSP preconditioned resid norm 4.006888955272e+04 true resid norm 5.881074911224e+02 ||r(i)||/||b|| 2.516425331407e-01
111 KSP preconditioned resid norm 2.556854734267e+04 true resid norm 3.117532546863e+01 ||r(i)||/||b|| 1.333946258266e-02
112 KSP preconditioned resid norm 3.603585859202e+04 true resid norm 6.702364240660e+02 ||r(i)||/||b|| 2.867842938597e-01
113 KSP preconditioned resid norm 2.281951510671e+04 true resid norm 4.080284654000e+02 ||r(i)||/||b|| 1.745893704411e-01
114 KSP preconditioned resid norm 2.462744966105e+04 true resid norm 4.613733252046e+02 ||r(i)||/||b|| 1.974148502282e-01
115 KSP preconditioned resid norm 1.007966387252e+04 true resid norm 1.945468820791e+02 ||r(i)||/||b|| 8.324374533572e-02
116 KSP preconditioned resid norm 2.109330692085e+03 true resid norm 3.776251639176e+01 ||r(i)||/||b|| 1.615802455509e-02
117 KSP preconditioned resid norm 6.831759586161e+03 true resid norm 1.335098483130e+02 ||r(i)||/||b|| 5.712689760949e-02
118 KSP preconditioned resid norm 2.469120077046e+03 true resid norm 4.998626121881e+01 ||r(i)||/||b|| 2.138838492148e-02
119 KSP preconditioned resid norm 1.211416271008e+03 true resid norm 3.362031478596e+00 ||r(i)||/||b|| 1.438563749899e-03
120 KSP preconditioned resid norm 1.224996393695e+03 true resid norm 6.103951168053e+00 ||r(i)||/||b|| 2.611790798931e-03
121 KSP preconditioned resid norm 1.209834665373e+03 true resid norm 5.508257872990e+00 ||r(i)||/||b|| 2.356902412017e-03
122 KSP preconditioned resid norm 1.176445798855e+03 true resid norm 3.376141817749e+00 ||r(i)||/||b|| 1.444601356190e-03
123 KSP preconditioned resid norm 2.388595874283e+03 true resid norm 2.025368406619e+01 ||r(i)||/||b|| 8.666253092813e-03
124 KSP preconditioned resid norm 1.835819816735e+03 true resid norm 1.368249587807e+01 ||r(i)||/||b|| 5.854538454992e-03
125 KSP preconditioned resid norm 1.535414710254e+03 true resid norm 8.502740142943e+00 ||r(i)||/||b|| 3.638197269216e-03
126 KSP preconditioned resid norm 1.080568878919e+03 true resid norm 2.390635645099e+00 ||r(i)||/||b|| 1.022917780559e-03
127 KSP preconditioned resid norm 1.094759817045e+03 true resid norm 7.613028249757e-01 ||r(i)||/||b|| 3.257502654803e-04
128 KSP preconditioned resid norm 1.095338168938e+03 true resid norm 7.145193335854e-01 ||r(i)||/||b|| 3.057322985944e-04
129 KSP preconditioned resid norm 1.064748229619e+03 true resid norm 2.343447117578e+00 ||r(i)||/||b|| 1.002726504679e-03
130 KSP preconditioned resid norm 1.255487935466e+03 true resid norm 5.431025002032e+00 ||r(i)||/||b|| 2.323855604106e-03
131 KSP preconditioned resid norm 4.498666605530e+03 true resid norm 2.726203940239e+01 ||r(i)||/||b|| 1.166502511421e-02
132 KSP preconditioned resid norm 2.266049132766e+04 true resid norm 1.636769902121e+02 ||r(i)||/||b|| 7.003497329241e-02
133 KSP preconditioned resid norm 1.599189591465e+03 true resid norm 6.850617715306e+00 ||r(i)||/||b|| 2.931278416753e-03
134 KSP preconditioned resid norm 1.359281497944e+03 true resid norm 2.120353404008e+00 ||r(i)||/||b|| 9.072679906181e-04
135 KSP preconditioned resid norm 1.398081045515e+03 true resid norm 3.178886423113e+00 ||r(i)||/||b|| 1.360198678225e-03
136 KSP preconditioned resid norm 1.406074847326e+03 true resid norm 2.782563532852e+00 ||r(i)||/||b|| 1.190617950973e-03
137 KSP preconditioned resid norm 1.560107927578e+03 true resid norm 1.618994686058e+01 ||r(i)||/||b|| 6.927439797840e-03
138 KSP preconditioned resid norm 1.222140874286e+03 true resid norm 9.620334312407e+00 ||r(i)||/||b|| 4.116399353143e-03
139 KSP preconditioned resid norm 1.213727165109e+03 true resid norm 9.052671174390e+00 ||r(i)||/||b|| 3.873504657568e-03
140 KSP preconditioned resid norm 1.511290053313e+03 true resid norm 1.284895777928e+00 ||r(i)||/||b|| 5.497879779810e-04
141 KSP preconditioned resid norm 1.526888451303e+03 true resid norm 1.136822432757e+00 ||r(i)||/||b|| 4.864295745733e-04
142 KSP preconditioned resid norm 1.572959289249e+03 true resid norm 7.493817732071e-01 ||r(i)||/||b|| 3.206494230152e-04
143 KSP preconditioned resid norm 1.577772419471e+03 true resid norm 7.195676859776e-01 ||r(i)||/||b|| 3.078924142252e-04
144 KSP preconditioned resid norm 3.608239859330e+03 true resid norm 6.227075471051e+00 ||r(i)||/||b|| 2.664473874670e-03
145 KSP preconditioned resid norm 1.504643118829e+03 true resid norm 1.621428801304e+00 ||r(i)||/||b|| 6.937855018455e-04
146 KSP preconditioned resid norm 1.493978112313e+03 true resid norm 2.686632199299e+00 ||r(i)||/||b|| 1.149570346330e-03
147 KSP preconditioned resid norm 1.493651958334e+03 true resid norm 2.650394605503e+00 ||r(i)||/||b|| 1.134064813693e-03
148 KSP preconditioned resid norm 1.472217504992e+03 true resid norm 2.091986246555e+00 ||r(i)||/||b|| 8.951301017675e-04
149 KSP preconditioned resid norm 1.512461491509e+03 true resid norm 2.994942887655e+00 ||r(i)||/||b|| 1.281491948730e-03
150 KSP preconditioned resid norm 1.515927982741e+03 true resid norm 3.139556111574e+00 ||r(i)||/||b|| 1.343369817218e-03
151 KSP preconditioned resid norm 1.580694738549e+03 true resid norm 4.332740630960e+00 ||r(i)||/||b|| 1.853915898496e-03
152 KSP preconditioned resid norm 1.592773253079e+03 true resid norm 4.655092959784e+00 ||r(i)||/||b|| 1.991845712031e-03
153 KSP preconditioned resid norm 1.597445562796e+03 true resid norm 4.784478995676e+00 ||r(i)||/||b|| 2.047208091046e-03
154 KSP preconditioned resid norm 1.825082479733e+03 true resid norm 8.956280495379e+00 ||r(i)||/||b|| 3.832260505770e-03
155 KSP preconditioned resid norm 1.531656881585e+03 true resid norm 3.345886675740e+00 ||r(i)||/||b|| 1.431655626556e-03
156 KSP preconditioned resid norm 1.530579043143e+03 true resid norm 3.310365269189e+00 ||r(i)||/||b|| 1.416456539892e-03
157 KSP preconditioned resid norm 1.456028581847e+03 true resid norm 3.987316433298e-01 ||r(i)||/||b|| 1.706113972114e-04
158 KSP preconditioned resid norm 1.588084202535e+03 true resid norm 4.336124379545e+00 ||r(i)||/||b|| 1.855363754676e-03
159 KSP preconditioned resid norm 1.553307024718e+03 true resid norm 3.640275487171e+00 ||r(i)||/||b|| 1.557620262876e-03
160 KSP preconditioned resid norm 1.546871397229e+03 true resid norm 3.489938953302e+00 ||r(i)||/||b|| 1.493293474360e-03
161 KSP preconditioned resid norm 1.548396148445e+03 true resid norm 3.540462748682e+00 ||r(i)||/||b|| 1.514911862232e-03
162 KSP preconditioned resid norm 1.546183359620e+03 true resid norm 3.491763219935e+00 ||r(i)||/||b|| 1.494074051182e-03
163 KSP preconditioned resid norm 1.541195908731e+03 true resid norm 3.385438574220e+00 ||r(i)||/||b|| 1.448579301350e-03
164 KSP preconditioned resid norm 1.541260232335e+03 true resid norm 3.388491683473e+00 ||r(i)||/||b|| 1.449885681830e-03
165 KSP preconditioned resid norm 1.546348250080e+03 true resid norm 3.499079043564e+00 ||r(i)||/||b|| 1.497204384358e-03
166 KSP preconditioned resid norm 1.561765957704e+03 true resid norm 3.832073113584e+00 ||r(i)||/||b|| 1.639687642207e-03
167 KSP preconditioned resid norm 1.567034486335e+03 true resid norm 3.954379815278e+00 ||r(i)||/||b|| 1.692020878390e-03
168 KSP preconditioned resid norm 1.558936731442e+03 true resid norm 3.789179430534e+00 ||r(i)||/||b|| 1.621334067016e-03
169 KSP preconditioned resid norm 1.556235308109e+03 true resid norm 3.733455196426e+00 ||r(i)||/||b|| 1.597490488011e-03
170 KSP preconditioned resid norm 1.558548245123e+03 true resid norm 3.781141913299e+00 ||r(i)||/||b|| 1.617894931777e-03
171 KSP preconditioned resid norm 1.559878101044e+03 true resid norm 3.810584646438e+00 ||r(i)||/||b|| 1.630493043621e-03
172 KSP preconditioned resid norm 1.559877957419e+03 true resid norm 3.811506647286e+00 ||r(i)||/||b|| 1.630887554204e-03

The solver fail with the following message:

*** -------------------------------------------------------------------------
*** Error:   Unable to solve linear system using PETSc Krylov solver.
*** Reason:  Solution failed to converge in 172 iterations (PETSc reason DIVERGED_BREAKDOWN, residual norm ||r|| = 1.559878e+03).
*** Where:   This error was encountered inside PETScKrylovSolver.cpp.
*** Process: 34
*** 
*** DOLFIN version: 2019.1.0
*** Git changeset:  74d7efe1e84d65e9433fd96c50f1d278fa3e3f3f
*** -------------------------------------------------------------------------

Since you don’t allude to the actual system you’re solving I’ll infer from “steady state fluid dynamics” that you’re approximating the incompressible Stokes or Navier-Stokes system with a small Reynolds number.

Iterative methods are rarely used as black boxes for solving linear systems. So there are many questions you should ask yourself. E.g. why bicgstab with hypre? Is your system indefinite? Why do you think you’re getting good rates of convergence with more than 100 iterations? Are you sure the solution the iterative solver gives you is actually an approximation of the true solution?

Good places to start are the elasticity and Stokes iterative demos.

1 Like