program into an MPI parallel program. At the start of the computation a copy of the program starts execution on each of the processors we have available. Te two first lines are calls provided by the MPI system and provide each of the executing programs with information on the total number of processors (the variable np) and a unique number in the range 0 to np identifying each processor. Let us assume we have a system where np equals 1,000 and that the number of rectangles is equal to one million. Te program then works by numbering the rectangles used in the computation from 1 to 1,000,000, and processor number 0 computes the sum of the areas for rectangles number 1, 1001, 2001… and so on up to one million. Processor number 1 does exactly the same, except it computes the sum of the areas for rectangles number 2, 1002, 2002… and so on. When each processor has finished computing its sum, a single MPI Reduce call will combine the sums from each of the 1,000 processors into a final sum. Te speedup for this kind of problem is directly proportional to the number of processors used. Running the program in Table 7.3 using 100 million rectangles on 12 processors gives a speedup of about 11.25, which is roughly what one would expect, allowing some time for overhead. However, if we increase the number of processors to 120 we will only get a speedup of about 40. Tis seems puzzling until one


} mypi = h * sum; MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD);


Table 7.3: The parallel version of the program shown in Table 7.2. The two first lines are calls to the MPI system returning the number of processors participating (the variable np) and a unique id (the variable myid) for each processor. The variable myid is a number between 0 and the number of processors. The loop in the fifth line is almost the same as for the program in Table 7.2, but each processor only computes a subset of all rectangles and stores the resulting sum in the variable mypi. The expression i+=np increments the loop variable i by np for each iteration of the loop. The MPI Reduce call in line number 10 adds together all results from each processor and returns this number to processor number 0. In the last three lines, processor number 0 prints the result.


realises that 120 processors involve 10 different nodes, since the particular machine used for the test runs is equipped with 12 processors per node. Te Reduce call then involves communication between 10 nodes. In general, communication between processors on different nodes takes more time than communication between nodes on the same processor.


7.2.3 Balancing Communication


Figure 7.8: NASA’s supercomputing resources have enabled them to exceed previous state- of-the-art magnetospheric models. The figure shows a snapshot from a 3D simulation of the Earth’s magnetosphere. The plasma density shows formation of helical structures in front of the magnetosphere. The structures lead to turbulence, which in turn amplifies the adverse effects of space weather on Earth and its technological systems. In this project, a typical simulation ran for over 400 hours across more than 4,000 cores on NASA’s Pleiades supercomputer. (Courtesy Homa Karimabadi, University of California, San Diego/SciberQuest; Burlen Loring, University of California, Berkeley.)


Not all problems can be split up into independent sub- problems. Often large simulation problems involving the solution of differential equations require large amounts of memory to hold intermediate results, exceeding the available memory on each node, or the number of numerical operations is so large that the problem has to be split into smaller parts, or domains, in such a way that each processor is able to deal with each of the domains. Tis is known as domain decomposition. Each of the smaller domains will usually depend on other domains, and an important issue then becomes the interchange of information at the domain boundaries. Each processor, working on a single domain, requires information from other processors. Hence, some form of communication


needs to take place. We have already seen how the MPI system can be used to pass information between processors via the Reduce call shown in Table 7.3. Tere are in addition a number of calls which can be used to send data from one processor to another. In practice these are relatively easy to use, but the logic and structure of programs rapidly become complicated if the pattern for information exchange is non-trivial. We also learnt that communication between processors on different nodes is expensive in terms of computer time. Hence an important issue is the balance between computing and communication.


250


MPI_Comm_size(MPI_COMM_WORLD,&np); MPI_Comm_rank(MPI_COMM_WORLD,&myid);


n=1000000; h=1/n;


for (i = myid+1; i <= n; i+=np){ x = h * ((double)i-0.5); sum += f(x);


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32  |  Page 33  |  Page 34  |  Page 35  |  Page 36  |  Page 37  |  Page 38  |  Page 39  |  Page 40  |  Page 41  |  Page 42  |  Page 43  |  Page 44  |  Page 45  |  Page 46  |  Page 47  |  Page 48  |  Page 49  |  Page 50  |  Page 51  |  Page 52  |  Page 53  |  Page 54  |  Page 55  |  Page 56  |  Page 57  |  Page 58  |  Page 59  |  Page 60  |  Page 61  |  Page 62  |  Page 63  |  Page 64  |  Page 65  |  Page 66  |  Page 67  |  Page 68  |  Page 69  |  Page 70  |  Page 71  |  Page 72  |  Page 73  |  Page 74  |  Page 75  |  Page 76  |  Page 77  |  Page 78  |  Page 79  |  Page 80  |  Page 81  |  Page 82  |  Page 83  |  Page 84  |  Page 85  |  Page 86  |  Page 87  |  Page 88  |  Page 89  |  Page 90  |  Page 91  |  Page 92  |  Page 93  |  Page 94  |  Page 95  |  Page 96  |  Page 97  |  Page 98  |  Page 99  |  Page 100  |  Page 101  |  Page 102  |  Page 103  |  Page 104  |  Page 105  |  Page 106  |  Page 107  |  Page 108  |  Page 109  |  Page 110  |  Page 111  |  Page 112  |  Page 113  |  Page 114  |  Page 115  |  Page 116  |  Page 117  |  Page 118  |  Page 119  |  Page 120  |  Page 121  |  Page 122  |  Page 123  |  Page 124  |  Page 125  |  Page 126  |  Page 127  |  Page 128  |  Page 129  |  Page 130  |  Page 131  |  Page 132  |  Page 133  |  Page 134  |  Page 135  |  Page 136  |  Page 137  |  Page 138  |  Page 139  |  Page 140  |  Page 141  |  Page 142  |  Page 143  |  Page 144  |  Page 145  |  Page 146  |  Page 147  |  Page 148  |  Page 149  |  Page 150  |  Page 151  |  Page 152  |  Page 153  |  Page 154  |  Page 155  |  Page 156  |  Page 157  |  Page 158  |  Page 159  |  Page 160  |  Page 161  |  Page 162  |  Page 163  |  Page 164  |  Page 165  |  Page 166  |  Page 167  |  Page 168  |  Page 169  |  Page 170  |  Page 171  |  Page 172  |  Page 173  |  Page 174  |  Page 175  |  Page 176  |  Page 177  |  Page 178  |  Page 179  |  Page 180  |  Page 181  |  Page 182  |  Page 183  |  Page 184  |  Page 185  |  Page 186  |  Page 187  |  Page 188  |  Page 189  |  Page 190  |  Page 191  |  Page 192  |  Page 193  |  Page 194  |  Page 195  |  Page 196  |  Page 197  |  Page 198  |  Page 199  |  Page 200  |  Page 201  |  Page 202  |  Page 203  |  Page 204  |  Page 205  |  Page 206  |  Page 207  |  Page 208  |  Page 209  |  Page 210  |  Page 211  |  Page 212  |  Page 213  |  Page 214  |  Page 215  |  Page 216  |  Page 217  |  Page 218  |  Page 219  |  Page 220  |  Page 221  |  Page 222  |  Page 223  |  Page 224  |  Page 225  |  Page 226  |  Page 227  |  Page 228  |  Page 229  |  Page 230  |  Page 231  |  Page 232  |  Page 233  |  Page 234  |  Page 235  |  Page 236  |  Page 237  |  Page 238  |  Page 239  |  Page 240  |  Page 241  |  Page 242  |  Page 243  |  Page 244  |  Page 245  |  Page 246  |  Page 247  |  Page 248  |  Page 249  |  Page 250  |  Page 251  |  Page 252  |  Page 253  |  Page 254  |  Page 255  |  Page 256  |  Page 257  |  Page 258  |  Page 259  |  Page 260  |  Page 261  |  Page 262  |  Page 263  |  Page 264  |  Page 265  |  Page 266  |  Page 267  |  Page 268  |  Page 269  |  Page 270  |  Page 271  |  Page 272  |  Page 273  |  Page 274  |  Page 275  |  Page 276  |  Page 277  |  Page 278  |  Page 279  |  Page 280  |  Page 281  |  Page 282  |  Page 283  |  Page 284  |  Page 285  |  Page 286  |  Page 287  |  Page 288  |  Page 289  |  Page 290  |  Page 291  |  Page 292  |  Page 293  |  Page 294  |  Page 295  |  Page 296  |  Page 297  |  Page 298  |  Page 299  |  Page 300  |  Page 301  |  Page 302  |  Page 303  |  Page 304  |  Page 305  |  Page 306  |  Page 307  |  Page 308  |  Page 309  |  Page 310  |  Page 311  |  Page 312  |  Page 313  |  Page 314  |  Page 315  |  Page 316  |  Page 317  |  Page 318  |  Page 319  |  Page 320  |  Page 321  |  Page 322  |  Page 323  |  Page 324  |  Page 325  |  Page 326  |  Page 327  |  Page 328  |  Page 329  |  Page 330  |  Page 331  |  Page 332  |  Page 333  |  Page 334  |  Page 335  |  Page 336  |  Page 337  |  Page 338  |  Page 339  |  Page 340  |  Page 341  |  Page 342  |  Page 343  |  Page 344  |  Page 345  |  Page 346  |  Page 347  |  Page 348  |  Page 349  |  Page 350  |  Page 351  |  Page 352