When I run this calculation without MPI, but using the same code compilation, the calculation runs just fine, but takes a very long time because there are so many particles.
When I begin using MPI and run with 2 nodes, the calculation also runs fine, but again is still rather slow.
Once I begin running with more MPI nodes, the code starts crashing after just a couple minutes. Each time the code exists with the error:
Code: Select all
slurmstepd: error: Exceeded step memory limit at some point. slurmstepd: error: Exceeded job memory limit at some point.
Perhaps I have a fundamental misunderstanding of what is going on, but I do not understand why this should happen. Any advice would be very much appreciated.
**Edit: I should also note that I have successfully run a similar job with 6 MPI nodes and 525,000 source particles without burnup. All other parameters should be the same.