Storage Informer
Storage Informer

Instruction Level Lock-step Parallelism on desert islands

by on May.28, 2009, under Storage

Instruction Level Lock-step Parallelism on desert islands

I told myself that all my posts here would be crosstaggable to both academic and multi-core, but Henry Neeman has got me all riled up.

Half this blog will NOT be about multi-core, but half will; it is thus a semi-multi-core post.

<![CDATA[Henry is Director of the OU Supercomputing Center for Education & Research (OSCER) at the University of Oklahoma; Henry has a highly regarded effective multi-format lecture series called ]]>Supercomputing in Plain English; and Henry is usually always right.

I still disagree with him, but that is what I frequently do, consistently since our first meeting as participants in the inaugural SC Parallel and Distributed workshop at Washington University in St Louis the summer of 2002. Henry is one of my dearest geek friends.

I was minding my own business on a conference call this morning, helping plan our Kean University based Parallel and Distributed workshop. I mentioned I wasn't completely comfortable with Henry's desert islands analogy guiding people to a deep conceptual understanding of what MPI is (it may be freedom)

Henry's desert islands analogy is theater, and we both agree that theater is great for teaching. We are also both hams, or at least Henry is.

      • a phone;
      • a pencil;
      • a calculator;
      • a piece of paper with instructions;
      • a piece of paper with numbers.
    • Inside the hut is a desk.
      On the desk is:

  • Imagine youˇ¦re on an island in a little hut.

This is how it starts (a lot like a Colossal Cave Adventure.) After looking over the analogy, I realize I'd organize the setup a little differently. I need to lead Desert Islands myself to really learn it. It is only director types who can read a script and "see" the finished product. Most of us need to watch the play to get it. I think that is one moral from for me. Desert Islands reminds me to continue to develop my director's eye when assembling curricula for others and trying to use curricula from others.

Anyway, for awhile now, another disagreement with Henry is what to call programming a GPU (Graphics Programming Unit), as opposed to a programming a CPU. When describing the three flavors of parallelism most prevalent today, I was calling it data parallelism, in the same breath that I mentioned shared memory parallelism and distributed memory parallelism. Henry points out data parallelism is more general, since it can also refer to techniques involving domain decomposition (sounds like we are composting mathematical results.)

OK, sez I, what is better term? SIMD (Single Instruction, Multiple Data), sez Henry. I groaned, and Henry quickly pointed out he hates it too. Neither of find that taxonomy helpful for fostering understanding. It is a correct term in the same sense that tendonitis is equivalent to saying "inflamed tendons", but it doesn't describe the why of the medical condition.

Henry asserts that vector processing is form of SIMD closest in spirit to what we are trying to describe. True, we both agree, but for that term to be of use, students, and probably most faculty, will not have seen, let alone used a vector processor.

Henry likes GPGPU (General Purpose GPU) programming to describe it, Paul Gray from the UNiversity of Northern Iowa likes it even better, especially when getting hands dirty up to the elbows in low level details doing performance programming. I view GPGPU as a short term description, and an unsatisfying FLA (Five Letter Acronym). We had floating point accelerators in the past, but that functionality got pulled into your now run of the mill, off the shelf CPUs. GPUs are being used in a similar way, and GPU functionality will likely be pulled into run of the mill, off the shelf future CPUs. I want a longer lasting term, that tastes great too.

Hence Instruction Level Lock-step Parallelism.

The trouble is, as soon as I wrote it down, I realize it basically says the same thing as SIMD.

Which may not be too bad. Shared memory parallelism and distributed memory parallelism are equally unrevealing of the joys, cautions, and pitfalls associated with their corresponding algorithmic and programing styles. They all require some teaching to flesh out those words, that flow so trippingly on the tongue; to be spoken whether directed to by the Dane or not. I think having a trippingly accessible phrase is important by providing a hook on which to hang a developing knowledge of the concept.

Oh, and though Henry didn't directly complain about the term Instruction Level Lock-step Parallelism, he did say it is not in common usage. Which almost immediately led me to start penning these words, words, words on the this blog, blog, blog. Boy, has it been a long semester, I am punchy, punchy, punchy.

Henry and I will helping lead three weeklong workshops this summer, so late at night you might hear the din of clashing swords and denting shields, but rest assured, when all the dust settles, the curricula I use will be the stronger for it, with the last Birnam oak remaining unharmed in the process.

Should we try to Colbert Instruction Level Lock-step Parallelism into the Webster's Dictionary? There is a indefinite truthiness to the prospect.

What words would/do you use to describe GPGPU SIMD programming?

    • … Tom

:, , , , , , , , , , ,

Leave a Reply

Powered by WP Hashcash

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...