Use MicroXRCEDDSGen Binaries instead of making users compile it from source

[Ryanf55]

Currently, users have to compile microxrceddsgen from source. When it updates, users have to remember to pull the code in 4.5 due to an ABI break just before release.

Once this tool is available as binaries in Humble, with the memory usage improvements, we can have users install this as a binary.

The current workflow is a burden for users.

[seantfish]

Given microxrceddsgen is not yet available as a binary, I have a potential solution for making compilation easier:

• Add microxrceddsgen to the .repos file
• Use colcon-gradle to build microxrceddsgen
• Create an ament_cmake shim package that adds the microxrceddsgen bash script to the bin directory, such that the script is available on PATH automatically
• the shim package has a dependency on micro-xrce-dds-gen
• ardupilot_sitl then depends on the shim package, ensuring microxrceddsgen is built first

With this new workflow, all users need to do is:

• Install the correct version of Java
• Install colcon-gradle with pip

This approach seems to work with testing, would be happy to implement in a pull request if this seems reasonable.

---

The shim package isn't strictly necessary either, but it provides an easy way to add the build tool to path without modifying the micro-xrce-dds-gen package itself.

[Ryanf55]

Given microxrceddsgen is not yet available as a binary, I have a potential solution for making compilation easier:

• Add microxrceddsgen to the .repos file
• Use colcon-gradle to build microxrceddsgen
• Create an ament_cmake shim package that adds the microxrceddsgen bash script to the bin directory, such that the script is available on PATH automatically
• the shim package has a dependency on micro-xrce-dds-gen
• ardupilot_sitl then depends on the shim package, ensuring microxrceddsgen is built first

With this new workflow, all users need to do is:

• Install the correct version of Java
• Install colcon-gradle with pip

This approach seems to work with testing, would be happy to implement in a pull request if this seems reasonable.

The shim package isn't strictly necessary either, but it provides an easy way to add the build tool to path without modifying the micro-xrce-dds-gen package itself.

I tried colcon-gradle and it doesn't recognize the package. Excuse my ignorance, but can you show me how to get it to work?
According to the readme, it invokes gradle assemble. We have to run gradlew assemble.
If I run colcon build in Micro-XRCE-DDS_Gen`, this is what I see:

ryan@B650-970:~/Dev/ros2_ws/src/Micro-XRCE-DDS-Gen$ colcon build
[0.097s] ERROR:colcon.colcon_core.package_identification:Exception in package identification extension 'gradle' in '.': Failed to extract project name from 'build.gradle'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/colcon_core/package_identification/__init__.py", line 144, in _identify
    retval = extension.identify(_reused_descriptor_instance)
  File "/home/ryan/.local/lib/python3.10/site-packages/colcon_gradle/package_identification/gradle.py", line 32, in identify
    raise RuntimeError(
RuntimeError: Failed to extract project name from 'build.gradle'

Starting >>> IDL-Parser
Finished <<< IDL-Parser [3.20s]                

Summary: 1 package finished [3.27s]

[seantfish]

@Ryanf55 It looks like you're running colcon build from the Micro-XRCE-DDS-Gen folder, have you tried running it from ros2_ws? I looked through colcon_gradle source at the time and I believe it has gradlew support built in.

Also, colcon_gradle after building will take the jar and export it to JAVA_HOME so I modified the microxrceddsgen bash script's final line to:

exec $java_exec "com.eprosima.microxrcedds.microxrceddsgen" "$@"

[Ryanf55]

I tried both - colcon doesn't seem to recognize it as a package.

[seantfish]

Curious, are you on ROS Humble or ROS Jazzy? I am currently running Jazzy.

[seantfish]

Another note, I am having trouble building the upstream eProsima version but the ArduPilot fork does build fine (on Jazzy).

Edit: ArduPilot fork of Micro-XRCE-DDS-Gen also builds on my Humble dev container

Edit 2: I replicated the error you got when building from inside Micro-XRCE-DDS-Gen @Ryanf55 - what error do you get when building from the workspace?

[Ryanf55]

Another note, I am having trouble building the upstream eProsima version but the ArduPilot fork does build fine (on Jazzy).

Edit: ArduPilot fork of Micro-XRCE-DDS-Gen also builds on my Humble dev container

Edit 2: I replicated the error you got when building from inside Micro-XRCE-DDS-Gen @Ryanf55 - what error do you get when building from the workspace?

1. I'm using humble, we are slowly adding support for Jazzy. PR's welcome for issues you find.
2. If I build from a workspace it works, just not inside the repo. seems like a bug from the tool. It doesn't seem to add the executable to PATH though.

(venv-ardupilot) ryan@B650-970:~/Dev/ardu_ws/src/test_env$ colcon build
Starting >>> Micro-XRCE-DDS-Gen
Finished <<< Micro-XRCE-DDS-Gen [10.5s]                    

Summary: 1 package finished [10.6s]
(venv-ardupilot) ryan@B650-970:~/Dev/ardu_ws/src/test_env$ . install/setup.bash
w(venv-ardupilot) ryan@B650-970:~/Dev/ardu_ws/src/test_env$ which microxrceddsgen 
/home/ryan/Dev/ros2_ws/src/Micro-XRCE-DDS-Gen/scripts/microxrceddsgen

Do you know how we could automate that? It's still using the repo from my ~/.bashrc.

[seantfish]

So colcon-gradle doesn't install to PATH, but it does install to CLASSPATH (https://github.com/colcon/colcon-gradle/blob/main/colcon_gradle/task/gradle/build.py).

I've created a shim package that makes use of the CLASSPATH rather than a reference to a built jar.
Try this fork: https://github.com/ASDL-Robotics/ardupilot/tree/ament_shim
It fails building but looks like you're fixing it already:

../../libraries/AP_DDS/AP_DDS_Client.cpp: In member function ‘bool AP_DDS_Client::update_topic(ardupilot_msgs_msg_Status&)’:
../../libraries/AP_DDS/AP_DDS_Client.cpp:723:35: error: ‘FS_RADIO’ was not declared in this scope; did you mean ‘Status::FS_RADIO’?
  723 |         msg.failsafe[fs_iter++] = FS_RADIO;
      |                                   ^~~~~~~~
      |                                   Status::FS_RADIO
compilation terminated due to -Wfatal-errors.

Edit: Got the wrong URL first whoops

[seantfish]

Also, pexpect is not declared as a dependency in the package.xml files currently. I'm guessing this is a package that is common on some Ubuntu distributions but doesn't ship with my container environment.

[Ryanf55]

Also, pexpect is not declared as a dependency in the package.xml files currently. I'm guessing this is a package that is common on some Ubuntu distributions but doesn't ship with my container environment.

Not all ardupilot dependencies live in the package.xml. Many live here:

ardupilot/Tools/environment_install/install-prereqs-ubuntu.sh

Line 186 in dab81d0

BASE_PKGS="build-essential ccache g++ gawk git make wget valgrind screen python3-pexpect astyle"

This is part of our setup instructions, so they are assumed to be available.

[seantfish]

Also, pexpect is not declared as a dependency in the package.xml files currently. I'm guessing this is a package that is common on some Ubuntu distributions but doesn't ship with my container environment.

Not all ardupilot dependencies live in the package.xml. Many live here:

ardupilot/Tools/environment_install/install-prereqs-ubuntu.sh

Line 186 in dab81d0

BASE_PKGS="build-essential ccache g++ gawk git make wget valgrind screen python3-pexpect astyle"

This is part of our setup instructions, so they are assumed to be available.

Since python3-pexpect is listed in rosdistro, is there any harm in including it in the package.xml? I find that Ardupilot SITL builds when I do include it.

[Ryanf55]

Also, pexpect is not declared as a dependency in the package.xml files currently. I'm guessing this is a package that is common on some Ubuntu distributions but doesn't ship with my container environment.

Not all ardupilot dependencies live in the package.xml. Many live here:

  [ardupilot/Tools/environment_install/install-prereqs-ubuntu.sh](https://github.com/ArduPilot/ardupilot/blob/dab81d0f7d28b1a6ad71e49a94d5d6435f5f7036/Tools/environment_install/install-prereqs-ubuntu.sh#L186)


     Line 186
  in
  [dab81d0](/ArduPilot/ardupilot/commit/dab81d0f7d28b1a6ad71e49a94d5d6435f5f7036)





    
      
       BASE_PKGS="build-essential ccache g++ gawk git make wget valgrind screen python3-pexpect astyle" 

This is part of our setup instructions, so they are assumed to be available.

Since python3-pexpect is listed in rosdistro, is there any harm in including it in the package.xml? I find that Ardupilot SITL builds when I do include it.

No, that is helpful. Please feel free to contribute improvements.