Monday, January 23, 2012

Evil C++ #5: Member function pointer special

Function pointers are one of the most esoteric and misunderstood features of C, and little was done in C++ to improve that situation (until C++11). I recently came to appreciate the peculiar evil of member function pointers in C++.

Friend of mine has a class, call it Foo, with a member function, call it threadMe. When constructed, Foo should create a new pthreads-like thread running threadMe, which will have access to this Foo's members.

That's all I know. Maybe there's a way to refactor it, but this is inside a tangle of library GUI code, which is always a mess (especially when that library is ROOT), and it's academically interesting.

Phase 0: Introduction
Before thinking about this, I had to refresh my memory on function pointers. Here is a simple example:

Even the somewhat arcane (*pointer)(args,args) syntax isn't not so bad. You're clearly dereferencing a pointer to a function.

Phase 1: Denial
The extension from the above globally-scoped functions to class member functions seems obvious. If the member is static, I should be able to do this:

int (*pStaticMemberFunction)(int);
pStaticMemberFunction = &(Foo::threadMe);

(Full program here). In C++, static member functions behave exactly like non-member functions.

For a non-static member function, one would reasonably expect the following syntax (full program):

  Foo* o = new Foo;
  int (*pMemberFunction)(int);
  pMemberFunction = &(o->threadMe);

NOPE. This isn't valid:

  error: ISO C++ forbids taking the address of a bound
  member function to form a pointer to member function.
  Say ‘&Foo::threadMe’ 

But... but... compiler. I don't want any Foo's threadMe. I want o's threadMe. Of course, if you foolishly follow GCC's advice anyway, hoping for some kind of magic, no. Just no.

  error: invalid use of non-static member function
  ‘int Foo::threadMe(int)’

Saw that one coming.

It appears that the &(object->memberFunction) syntax was actually considered for C++ at one point, as described in this heartbreaking 1994 article in the "Callbacks Using Template Functors" section. But in the end, this feature was deemed too complicated for the language and the compilation thereof, and left for libraries to implement. Some have, notably Boost.

Phase 2: Bad Hacks
There appears to be no general solution to the above problem, which is more than a syntactic issue, but has deep roots in the language's representation of function pointers. To proceed, I clearly need a pointer to a function with external linkage. The goal is to use such a function as a proxy: somehow, I call it and it calls the real (non-static member) function for me.

Bad Hack #1: Inadvisable Casting: The C++ FAQ strongly discourages casting a function pointer to a void pointer, but such a cast could allow me to construct a static wrapper function which would take an instance and a cleverly disguised member function pointer as an argument, solving the problem. At least for me, this casting works. However, there is no way to cast the void* back to a member function pointer that you could actually use. If C++ supported rvalue references, you could do something like

(void*) fnPtr = voidPtrArgument;

and perhaps get away with it, but it would be really evil C++. At any rate, it doesn't work.

Bad Hack #2: The Class With Incredible Foresight / Sledgehammer Approach: Another bad option is to create a mysterious superclass with virtual member functions having a wide variety of prototypes. For each of these prototypes, you can use a typedef to create a type meaning "pointer to a function like this one," e.g.

typedef int (SuperClass::*pSuperClassMemFnTakesIntReturnsInt)(int);
typedef void (SuperClass::*pSuperClassMemFnTakesIntReturnsVoid)(int);

Then, for each of these types, create a global launcher function, like

void* launchMember(SuperClass* o, pSuperClassMemFnTakesIntReturnsInt p)

Rather unwieldy. But, with this vast library of types, you can do whatever you want. Subclass "SuperClass" and implement member functions of your choosing, then call

launchMember(superClassInstance, &SuperClass::whateverFunction)

with abandon. A example "solving" the original problem is here.

Bad Hack #3: One Member Function to Rule Them All: The least kludgy option, I think. Sacrifice some generality and pick one exact function prototype (name and all), which will the the thready one. Then you can at least use the launcher for any class that has that prototype. This is cheating: the global launcher function doesn't take an instance and a member function pointer, it just takes an instance and we "know" which function to call.

A working example can be found here. This approach was partly inspired by this thread.

Phase 3: The Solution, But You Won't Like It
The best, most general solution is to change the language, which they just did in the new C++11 standard. The biggest deal is the introduction of a proper native threading library. Lambda functions also provide the abstraction needed to make creating threads with ad-hoc functions pretty slick. In all its glory:


This is great if you're running GCC >= 4.6. Otherwise, I guess Bad Hack #3 is probably best. If anyone else has better ideas, let me know in the comments!

Wednesday, November 9, 2011

Hitachi Deskstar + RocketRAID 232x

Spent the morning tracking down a stupid RAID problem, so hopefully this helps someone...

I use RocketRAID 2320 cards as HBAs in a few Ubuntu servers. I use software RAID (mdadm), so no need for the RAID features.

The trick to using the rr232x as an HBA is to configure each drive as a separate single-disk JBOD "array" in the RocketRAID BIOS. In the servers I've set up before, I've used eight Western Digital (Green) 1.5 TB and 2.0 TB drives, did the JBOD thing, and they appeared to the OS without issue.

My new server has 12 3.0 TB Hitachi Deskstars (H3IK3000), and when I installed the rr232x driver, I saw no drives. The RAID card could see the drives, and the OS could see the RAID card, but the OS couldn't see the drives.

When the drives that had been attached to the RR2320 were plugged directly into the motherboard, I noticed that they didn't spin up. The OS saw them and tried to talk, but timed out and gave up. Identical drives not exposed to the RR2320 worked fine.

There turned out to be two problems:

  1. The staggered spin-up feature of the RR2320. Apparently not compatible with these Deskstars, when enabled it flips a bit telling the drives not to spin up at all. The solution is to enable-then-disable the staggered spin up setting (Settings menu in the BIOS utility).
  2. The RR2320 just doesn't seem to like H3IK3000s. Solution: run the drives in "Legacy" mode rather than as JBOD arrays, a tidbit I picked up from here.
To run in Legacy mode, put an empty partition on a drive (plug into mobo, "mklabel" and "mkpart" in parted). The RR2320 will ignore it and pass it through to the OS.

Recap:
  1. Enable-then-disable staggered spin-up
  2. Plug each drive into the motherboard and fire up parted
    1. mklabel gpt
    2. mkpart (enter: null, xfs, 1, 3tb)
    3. set 1 raid on
  3. Plug drives back into the RR2320 -- should see them on boot
  4. mdadm --create /dev/md0 --raid-level=6 --raid-devices=12 /dev/sdb1 /dev/sdc1 ... /dev/sdm1
  5. mkfs.xfs /dev/md0

Friday, August 26, 2011

Really Super Quick Start Guide to Setting Up SLURM

SLURM is the awesomely-named Simple Linux Utility for Resource Management written by the good people at LLNL. It's basically a smart task queuing system for clusters. My cluster has always run Sun Grid Engine, but it looks like SGE is more or less dead in the post-Oracle Sun software apocalypse. In light of this and since SGE recently looked at me the wrong way, I'm hoping to ditch it for SLURM. I like pop culture references and software that works.

The "Super Quick Start Guide" for LLNL SLURM has a lot of words, at least one of which is "make." If you're lazy like me, just do this:

0. Be using Ubuntu
1. Install: # apt-get install slurm-llnl
2. Create key for MUNGE authentication: /usr/sbin/create-munge-key
3a. Make config file: https://computing.llnl.gov/linux/slurm/configurator.html
3b. Put config file in: /etc/slurm-llnl/slurm.conf
4. Start master: # slurmctld
5. Start node: # slurmd
6. Test that fool: $ srun -N1 /bin/hostname

Bam.

(In my config file, I specified "localhost" as the master and the node. Probably a good place to start.)

Saturday, June 4, 2011

Evil C++ #2: Using GCC's -ftrapv flag to debug integer overflows

In C++, overflowing an integer type won't cause an exception and can result in weird numbers propagating through your program. GCC's ftrapv flag has your back.

Thursday, June 2, 2011

Evil C++ #1: Brackets and "at" for accessing STL vector elements

This is the first in a series of code snippets that demonstrate C/C++ pitfalls.

(For an thorough explanation of the many ways C++ is out to get you, see Yossi Kreinin's excellent C++ FQA).