Lesson 233
High-Performance & Scientific Computing
Parallelism · Throughput · Amdahl's Law
1:00How to exploit parallel hardware — cores, clusters, and GPUs — to run scientific and data-intensive workloads faster, and how Amdahl's and Gustafson's laws govern the limits of that scaling.
By the end, you can
- Distinguish latency from throughput and explain which one parallelism improves.
- Classify a parallel workload as data parallel or task parallel and give an example of each.
- Explain why a vector add over independent elements is "embarrassingly parallel" — each output depends only on its own inputs, so the work splits across cores with no communication or synchronization.
- Apply Amdahl's law to calculate the speedup ceiling for a given serial fraction and core count.
- Describe the roofline model and identify when a kernel is memory-bound versus compute-bound.
- Explain what false sharing is, why it degrades performance, and how padding fixes it.
- Contrast Amdahl's and Gustafson's laws and explain why Gustafson is more optimistic.
- State the four recap rules for structuring work to scale on parallel hardware.
Up next in Graphics, DevOps & Cloud Systems
