This is the web page for the "FISH" assignment.
We implemented the N-body problem with 4 different programming languages and on two different architectures. We decided on a basic algorithm together (the obvious grid-based one, with ghost-cells if needed for the distrubuted memory architectures), then coded up the program in each of the languages separately.
Here is a description of each of our approaches:
Question 1: Speedup-load balance-other good ideas, etc
This question is answered separately for each of the approaches.
Question 2: Debugging the code:
Essentially, we used a combination of tricks to debug the code and verify that we had the right simulation. We calculated a correct expression for the energy of the system, and used it to analyze a simple two-body problem. From this analysis, we calculated the minimum approach distance for two particles starting at a certain distance from one another. We then verified that we obtained this predicted behavior with our simulations. As further validation, we checked to see if the x,y position of each particle after some number of timesteps was the same across different versions of the code (only intra-language, not inter). In other words, after each big change (no grid to having one, for example) we verified the exact results (for both parallel and serial runs). Also, performing the energy calculations discussed below flushed out a few bugs as we worked to get our simulations to come out as the energy model would predict.
Question 3: Error Metric.
We derived the expression for the energy and used it as an error metric. The expression we used is total_energy = kinetic_energy + 0.5*potential_energy.
To calculate kinetic energy, we simply inserted code to calculate 0.5v**2 in the position updating step of our calculations. To calculate potential energy, we inserted code into the CalculateForces component of the code to add the potential energy contributed by each particle-particle interaction. Because we updated this each time we updated a particle, we double counted the potentials, and thus have to divide by two. Our function for the potential was:
double potential(double r){
if(r > GRIDSIZE)
return 0;
if(r==0)
return 0;
if(r<=INV_VALUE)
return(0.1*(2*r-INV_VALUE*log(r)-
INV_VALUE*(9./4.-log(INV_VALUE))));
else
return(-0.1*pow(INV_VALUE,5)/(4*pow(r,4)));
}
where GRIDSIZE is the cutoff distance, and INV_VALUE is 1/30.
To verify that our energy equation (and our force dynamics) were correct, we utilized a simple two-particle system where the two particles start off with zero velocity 0.04 away from each other. They should then oscillate, returning to exactly their initial position each time. As the timestep is reduced, this effect is lessened, but it never goes away entirely.
As shown above, when the timestep is set much too high, the particles gain energy from the collisions, in this case propelling the particle past the range of the attractive force. The particles move out past the range at which they can interact, and then travel to the wall and back, where they again collide, gaining more energy each time. This is further shown in the energy graph for the same interaction.
Note that the time scales do NOT match up in the two figures. The energy goes up with each collision, as expected, but there is also a somewhat odd drop in energy. This, we argue, is due to the fact that the min_dist factor in our equations causes energy to be lost if it happens that particles become too close together. Also, in this case, the energy loss occurred after the particles were already travelling far faster than they should have been, causing them to nearly come in contact. With such a high timestep, the interaction comes to barely resemble the desired one.
Even with a smaller timestep, as shown above, drops in energy can still occur if the min_dist is too large.
We used progressively smaller timesteps with our two-particle system until we verified that the added energy was also decreasing as the min step size decreased.
With a timestep of 0.001, we still see that the interaction is unstable, but far less problematic than in the previous case. The energy curve is also in line with this.
When we go to a timestep of 0.0001, as shown below, the error introduced is much smaller.
We also tested the error in the energy as a
function of timestep in a system with 500 particles. We allowed
the system to run for the same amount of total simulated time in
each case, and measured the
(Final_Energy - Initial_Energy)/Initial_Energy as our error metric.
The results of this experiment are shown below.
Note that in this graph, the x_axis is not a perfect scale of time, but rather reports distinct tests at different timesteps. The shape of the curve can be explained by the fact that as the timestep size increases, the amount of energy added to the system from collisions increases. In an intuitive way, the larger the timestep, the more likely it is that particles will get "too close" to one another during a single timestep. As the timestep size is reduced, the particle interactions become more realistic, and the amount of added energy per collision decreases. We also tested for timesteps of 0.5, 0.1, and 0.05, but these caused behavior such as particles moving through one another and the like, so we decided that these values were where the simulation "goes to hell".
Question 4: Evalute the programming models
In addition to this section, more details on our experiences with each of the programming systems can be found in the separate write-ups found at:
In general, we found that the SMP machines were somewhat easier to program than the distributed memory machines, as the methodology for exchanging information could be simpler.
We found that threads were perhaps easiest to use and get started with, as the cost for sharing data between two processors on the shared memory machines (specifically the clumps) is comparatively very low. We actually found that we didn't need to resort to techniques such as "ghosting" to achieve good performance with the threads model. One difficulty was that it is somewhat difficult to cause a full 'read' of data using threads without forcing one by performing additional calculations. It was straightforward to use both the provided barrier implementation and the pthread_mutex functions to handle synchronization issues. Being able to use C++ was a win because of being able to build and utilize List, Grid, Processor, and Fish objects. Although we didn't get to the stage of dealing with handling load balance by exchanging grid cells between processors, it would have been straightforward to do so.
MPI was somewhat difficult to use because?
Titanium was ???? to use.
Split-C was ??? to use.