Friday, April 17 Link to previous meetings

Elliptic variables are done and have been tested for hydro + source terms. ~~I need to run an MHD test problem before I check in the code.~~ I've been running one of Sean's 3D MHD clump problems and it seems to work fine.
When this is checked in it will break bear.data. Putting in elliptic variables requires a change to the Info structure, meaning it's no longer the same length, meaning reading in old Info*.bin files won't work without one of these changes.

In bear.data, change
```
F             Restart  = ... 
```
to
```
F             Restart  = ... 
```
```
401           RestartRevision = the SVN revision to be restarted from (affects assumed Info structure size) 
```
Also, change (e.g. for 2d hydro with 3 tracers)
```
7                  NrVars      = number of primary fields
```
to (e.g. for 2d hydro with 3 tracers and 1 elliptic variable)
```
7                  NrHydroVars      = number of fields for hydro, protect, source steps
```
```
1                  NrEllipticVars   = number of fields for elliptic step
```
NrVars now will be, implicitly, NrHydroVars+NrEllipticVars. In terms of changes to the code, this requires not only figuring out whether what was NrVars now is NrHydroVars, but also whether any q(.....,:) should become q(.....,1:NrVars) or q(.....,1:NrHydroVars).

I'm not 100% sure I've found all the places where this would be a problem and fixed them, so I'll need people to check out the upcoming version of the code and try running it on a problem that works on the code revision right before, and let me know if/where it breaks. This is more feasible than me trying to think up & run every problem I can.
The name "elliptic variables" may be somewhat misleading: really, the variables are non-hyperbolic step, non-protect, non-source variables. Though they are coarsened/refined along with the other variables, their fixup fluxes are zero.
Here is an example using the new variables: Jonathan had suggested trying labelling each grid by its MPI_id. That way, at data dumps we could visualize where grids are physically located, and watch how this evolves.
The answer (as one would expect), is that boy do they move all over the place.

I ran a 2d+cylindrical shock/clump simulation at 384/rclump, with data dumps at every CFL=0.4 timestep. I ran the simulation out to 700 steps, which is still less than t_cc.

Here is an image, showing how the grids change between step #319 and step #320:

Different colors correspond to different processors (28 processors in this case).

Here is a GIF showing every 10 timesteps (2.3MB). Here's a GIF every five steps (4.5MB), and Here's every step (35MB; this is an AVI because the GIF (27MB) crashes my firefox).

Here is the same data in an alternative format, namely a histogram of processor on the bottom (sort of—convoluted with the number of colors in the image) versus how much area of the image they take up.(This image is broken. If you wait 15 seconds after clicking the link, you'll see part of it.)

We can describe the behavior more quantitatively by looking at the 2d correlation function of the above images, frame-by-frame. That gives us this:

A value of 1.0 would mean the images were identical; a value of 0.0 would mean all of the image pixels were different. The red line is a moving average.

There are two things going on: one is that the grids are moving (making the colors change and the correlation decrease). The other is that the simulation is running, so things are changing.

We can try estimate how much of an effect is "real" or unavoidable—that caused by the simple fact of the simulation running. I did this by taking two frames and manually coloring the latter to be as close as possible as the former. This would be like turning off grid relocation for one timestep:

I chose frames 319/320, which is the low point in the above plot, at rcorr=0.70. With this adjustment, the correlation coefficient increases to 0.85—this is the red cross on the above plot. The point is that the large variations in correlation coefficient we're seeing is coming from grids being moved around. The overall trend (the red line) is from the simulation.

Low correlation coefficient, therefore, is essentially a measure of a large amount of communication between two frames, because it implies that lots of grids got moved around. In the spread of values above, the top values are times when few grids moved around, and the bottom are when lots of grids moved.
I don't really have any recommendations. I just thought people would be interested to see this.