Hello, long time no see, wanna ask a question

[lucasjinreal]

Hi, recently I start tackling the retarget problem.

I have the output in SMPL model format, which is quaternion based on SMPL, then I want apply these animation to Mixamo model.

But Mxiamo skeleton is different than SMPL (not bone names, buthe the bone orientation also different).

Do u know how can I retarget the animation to mixamo? (by given the source animation and 2 model smpl & mixamo)

[Nor-s]

Yes, I've done something similar.

Bone names just map, The orientation needs to be recalculated with matrix calculations.

The formula:

New animation = (New Animation Binding)^-1 * (New global parent)^-1 * (global Parent) * (Animation Binding) * (animation) * (global bind offset) * (new global bind offset)^-1

This seems to work well for models with different orientations among mixamo models.

[Nor-s]

The new binding pose can be calculated as:

1. Get R, S of the binding pose of the target model

2. Take the coordinates relative to the parent bone as T in the existing global coordinates

3. new binding = TRS

[lucasjinreal]

@Nor-s hi, thank u for the reply, do u have an example for this? How should I get the R and S from a SMPL model and a Mixamo model in FBX format?

[Nor-s]

@jinfagang

see src/utility.h DecomposeTransform

T, R, S = DecomposeTransform(local bindpose)
"local bindpose" is "assimp aiNode::mTransformation"

And I wrote sample code for my editor.
(same mixamo name, different bone orientation)
( Here, PoseComponent holds source animation information and target character information.)

Sorry for the dirty code.

[Nor-s]

example

I haven't tested SMPL because I don't have an SMPL model,
but this is the result of retargeting different orientation Mixamo models.
(But for now, Mixamo has updated them, they all have the same bone orientation.)

before

after

[lucasjinreal]

@Nor-s thank, does there a way make your code buildable on macOS? I wnanna have a try, this seems exactly what I need

[Nor-s]

@jinfagang

sorry. I'm currently only trying the silicon version (m1),
so I don't know if it will work correctly. Also the animations can be weird like you did before.

To build embedded Python, you need to configure your environment according to this document

Setting up the embedded Python environment seems too difficult.

I configured the environment with this command.

• Clang 13.0.0 arm64-apple-darwin21.2.0
• Python 3.10 and newer
• Homebrew 3.5.9

xcode-select --install
brew install pkg-config [email protected] xz gdbm tcl-tk
git clone --recursive https://github.com/Nor-s/Anim.git
cd Anim

mkdir build
cd build
cmake ..
make

if u intel version
You will need to modify the following:
scripts/build_python_osx.sh

./pip3 install mediapipe-silicon

=>

./pip3 install mediapipe

[lucasjinreal]

@Nor-s hello, I am exactly Sillcon version M1 mac. But you needn't build arm python yourself. Using mini-forge can have a arm python in conda.

Have you tride that? this can be more universal since usally users have python in conda.