Parallel Programming Study
(Tutorial from Lawrence Livermore Labs)

Make up 5 multiple choice questions on the following topics.
- Put the questions on your website portfolio under "Parallel Programming Questions"
- List the answers at the bottom of the page, or on a separa te link.

1. Parallel vs serial computing.

2. von Neumann Architecture

3. Flynn's Classical Taxonomy, SISD, SIMD, MISD, MIMD

4. RISC/CISC

5. General Parallel Terminology
   • Task, Parallel Task, Serial Execution, Parallel Execution
   • Shared Memory, Distributed Memory, Communications, Synchronization
   • Granularity, Observed Speedup, Parallel Overhead, Massively Parallel, Scalability

6. Parallel Computer Memory Architectures
   • Shared memory - uniform memory access, non-uniform memory access
   • Distributed Memory
   • Hybrid Distributed-Shared Memory

7. Parallel Programming models
   - Shared Memory, Threads, Message Passing, Data Parallel, Hybrid

8. Automatic vs. Manual Parallelization

9. Designing Parallel Programs
   • Understand the Problem and the Program, Partitioning

10. Designing Parallel Programs
    • Communications - do/don't need communications
    • Factors to consider in communications
    • Synchronous vs. asynchronous communications, blocking, non-blocking
    • Scope of communications, point-to-point, collective
      broadcast, scatter, gather, reduction

11. Sychronization - Barrier, Lock / semaphore, Synchronous communication operations

12. Data Dependencies

13. Load balancing

14. Granularity

15. Amdahl's Law

16. Parallel examples
    - array processing, PI calculation, Heat Equation, 1-D wave equation