I believe Q = A * B * A' is the dual quaternion equivalent of q = a * b / a for real numbers. The conjugate and the inverse of a unit dual quaternion is the same thing.
quaternions and the like do not behave like real numbers.
Suppose A = i and B = j. Then
Q = i * j * (-i)
= k * i * -1
= -jwhich is not the same as B.
Err... Please read up on quaternions and dual quaternions. If A is a unit quaternion, doing that will be the equivalent of first rotating B around A', and then rotate it around A, which effectively is the opposite rotation of A'. You basically rotate B an angle around an axis, and then back again to the starting position. Which means that Q should always be equal to B when the quaternion/dual quaternion is normalized.
Correct or not, I get the exact results I want when doing it this way. I multiply a bone with the conjugate of the parent bone to get the local orientation, then I multiply that with the frame's change in orientation, and then I multiply this with the parent's orientation again. If there is no local change in orientation, I always end up with the orientation I started with.
EDIT:
*Hits head*
Sorry, you're right. I had the order off in my post. What I'm actually doing is:
Q = B * A' * R * A
Where R is the change in local orientation. If R is identity, I end up with Q = B since AA' = 1 for unit quaternions