hm2_eth: Create support library's / Xenomai3 support

[hdiethelm]

This PR has tree parts. If desired, I can split them also up in to separate PR's.

For now it is more about the concepts. No need to review the code from a style standpoint.

Part1: Create support library's
The issue is, that as soon as hm2_eth is linked to libevl, this module can not be used any more without libevl installed.
By creating liblinuxcnc-hm2_eth_net_evl.so and liblinuxcnc-hm2_eth_net_xenomai.so and loading them only when needed, the hard dependency is broken and the library's can be deployed in a linuxcnc-uspace-xenomai / linuxcnc-uspace-evl Debian package.

The Debian packages are not yet created but I am planning to do so if this approach is approved.

The firewall functions are moved out of the Ethernet support modules, so not so many exports are needed for the library's. The function order changes slightly but this should have no side effects.

Alternatives:

• hm2_eth / hm2_eth_xenomai / hm2_eth_evl
  • Three hm2_eth components, main core but different Ethernet code linked.
  • Disadvantages
    • You can not run one Ethernet card with EVL and the other with posix. Niche application if you have a low priority thread communicating with a second mesa card.
    • Three components that could have been a config flag.
• hm2_eth_xenomai / hm2_eth_evl linuxcnc components
  • Disadvantages:
    • More components
    • Empty rtapi_app_main
    • Needs entry in hal file
    • Makefile makes it hard to define custom C / LD flags for specific modules. I did not figure out how to do this.
• Just leave it as it is and depend on libevl
  • Disadvantage: Users would have to compile it from source or three different full linuxcnc-uspace packages are needed.

What do you think about the support library approach?

Part 2: Xenomai3 support
It was easy to do. Note that sendto(), recvfrom() and other syscalls are overwritten by the Xenomai C / LD flags, so even thoutght it looks like posix, behind are xenomai syscalls. This is just how xenomai3 posix skin works.

TBD: if anyone needs this. I just wanted to try it to see any differences to xenomai4. It needs more testing but on my hardware, performance is similar. However, setup is a bit more cumbersome. The two scripts rtnet_up.sh / rtnet_down.sh are just temporary and should go to the man page with some description how to set it up when done.

Part 3: Xenomai4 OOB improvements
The code is still in but there is an other PR only concerning this which should be merged first: #4503

[BsAtHome]

To me it certainly feels like a bad choice doing dlopen in a realtime component.

Is the target for compilation ever both uspace/posix, Xenomai3 and EVL at the same time? If the answer is no, then there should be only one hm2_eth component/driver, which adapts properly at compile time to the necessary layout(s) and format(s). That can still be supported by libraries, but whether they should be .so is doubtful. The compilation generates a (mostly) static driver. Only upwards are symbols matched (to the hostmot2 component). Requiring this also downwards is unnecessary when you know at build-time what your target system is.

So, what is the answer to the question, whether the build can only target one system at a time or multiple simultaneously?

[hdiethelm]

To me it certainly feels like a bad choice doing dlopen in a realtime component.

Why? It only changes from 2x to 3x dlopen in the rtapi_app main thread. But of course there is no dlopen in the realtime tasks which would be bad.
First two:

Load RT extension:

linuxcnc/src/rtapi/uspace_rtapi_main.cc

Line 1533 in fa910f1

dll = dlopen(dllName.c_str(), RTLD_NOW);

Load Component:

linuxcnc/src/rtapi/uspace_rtapi_main.cc

Line 576 in fa910f1

void *module = modules[name] = dlopen(what.c_str(), RTLD_GLOBAL | RTLD_NOW);

With this PR, there is a third dlopen in the same thread in the rtapi_app_main path called just after load:

linuxcnc/src/rtapi/uspace_rtapi_main.cc

Line 583 in fa910f1

int (*start)(void) = DLSYM<int (*)(void)>(module, "rtapi_app_main");

Is the target for compilation ever both uspace/posix, Xenomai3 and EVL at the same time? If the answer is no, then there should be only one hm2_eth component/driver, which adapts properly at compile time to the necessary layout(s) and format(s). That can still be supported by libraries, but whether they should be .so is doubtful. The compilation generates a (mostly) static driver. Only upwards are symbols matched (to the hostmot2 component). Requiring this also downwards is unnecessary when you know at build-time what your target system is.

At build time, if you have libxenomai / libevl installed, all three (+posix) variants are compiled in. But that also means that if you create a debian package, it will automatically depend on these libs.

I use this all day for testing, just boot a different kernel and it works out of the box with the correct back-end, no rebuild needed.

Looks like the reason liblinuxcnc-uspace-xenomai.so / liblinuxcnc-uspace-xenomai-evl.so for rtapi_app also exist to break this dependency, and there is even a control file to pack these in a different debian package: https://github.com/LinuxCNC/linuxcnc/blob/master/debian/control.uspace-xenomai.in But it looks like the rest was either removed or never finalized, so these packages are not built. If this is finalized, you would have:

linuxcnc-uspace: Posix, not depending on libxenomai / libevl
linuxcnc-uspace-xenomai: Xenomai support, depending on libxenomai
linuxcnc-uspace-evl: Xenomai EVL support, depending on libevl

So, what is the answer to the question, whether the build can only target one system at a time or multiple simultaneously?

It can target all of the existing uspace realtime systems at the same time but right now, linuxcnc-uspace will then depend on libxenomai / libevl which is unnecessary but only an issue if these libs are not available in the linuxcnc package repo.

[grandixximo]

This is best argument I heard so far, for splitting the deb...

[hdiethelm]

This is best argument I heard so far, for splitting the deb...

This is also the main intent besides being able to use linuxcnc built with libevl on a system withouth.

However, with a single deb, it will still depend on libevl.

Might be it can be changed to recommended, i could look into how debian packages are configured and if shlibdeps can be modifyed. But I think separate packages are nicer anyway.

[zz912]

Hello,

I have two questions from a user’s perspective:

1. What do you see as the main user-facing reason for supporting Xenomai 3 in hm2_eth? Is there a specific use case for it in LinuxCNC, or is it more about keeping the door open for future use?

2. If Xenomai 4/EVL is the direction LinuxCNC’s realtime support is heading, wouldn’t it be more beneficial for users to focus the effort on a ready-to-use Xenomai 4 + LinuxCNC image/Live CD rather than supporting multiple realtime backends?

[hdiethelm]

Hello,

I have two questions from a user’s perspective:

1. What do you see as the main user-facing reason for supporting Xenomai 3 in hm2_eth? Is there a specific use case for it in LinuxCNC, or is it more about keeping the door open for future use?

Basically, Xenomai3 support is already in since some time (2016). Now that the framework is in to have different backends for hm2_eth, it took only 140 lines of code for Xenomai3 support.

It might be of use for some people, so why not add it, especially due to it took only a few hours to create it.

All I have read so far about realtime performance, there is no way to predict which system works best on your PC. The only real way is to test it. And you can only do that if the support is there. It might well be that for some people, Xenomai3 works better. This was also the main reason for me to implement this: Being able to test how well it works.

2. If Xenomai 4/EVL is the direction LinuxCNC’s realtime support is heading, wouldn’t it be more beneficial for users to focus the effort on a ready-to-use Xenomai 4 + LinuxCNC image/Live CD rather than supporting multiple realtime backends?

This is a question for the maintainers creating live images. Basically, the actual framework would also support a live image with tree different kernels and depending on which you start, the matching RT framework is used.

However, to have generic Debian packages which can be installed with or without libevl / libxenomai, part 1 of this PR (or one of the alternatives) is needed.

[zz912]

Thanks for the explanation. I now understand why you want to keep the individual variants separate – mainly because of the dependencies and because, as a developer/tester, you can have a single build and test it against different kernels.

I’m looking at it a little differently, though, from the perspective of a beginner who is using LinuxCNC for the first time.

Someone coming from Windows or Mac is often also working with Debian for the first time. Such a user really doesn’t know what POSIX, Xenomai 3 or EVL mean. I don’t think it’s ideal to expect them to make this decision before they have even run LinuxCNC for the first time.

A Debian beginner also tends to have exactly the opposite approach from an experienced developer: they would rather have “everything” installed so that it works. For a developer this may be a nightmare, but for a beginner it is completely normal. :-)

So I could imagine something like a simple meta-package called linuxcnc (just a suggestion for the name), which would install all the required variants:

linuxcnc
   ├── linuxcnc-uspace
   ├── linuxcnc-uspace-xenomai3
   └── linuxcnc-uspace-xenomai4

An experienced user could of course still install only linuxcnc-uspace or only linuxcnc-uspace-xenomai4 if they know exactly what they need. But for a beginner there would be a simple path: install linuxcnc and they don’t need to understand the differences between the various realtime environments beforehand.

This is just a suggestion from a UX perspective. I’m not saying that it is technically a better solution than your package separation.

One more small point about the naming. Personally, I would prefer:

linuxcnc-uspace
linuxcnc-uspace-xenomai3
linuxcnc-uspace-xenomai4

instead of linuxcnc-uspace-xenomai and linuxcnc-uspace-evl.

Again, the main reason is the beginner. “EVL” doesn’t mean anything to them, whereas xenomai3 and xenomai4 immediately make it clear that these are two generations of Xenomai. I know that Xenomai 4 uses EVL and that technically it is a bit more complicated, but from a user’s perspective xenomai4 seems like a much more understandable name to me.

So perhaps it makes sense to keep the packages separate for clean dependencies, while also providing one simple “default” path for people who are just getting started with LinuxCNC and Debian.

[hdiethelm]

Basically, not keeping the add-ons separate would mean either:

• linuxcnc-uspace depends on libevl / libxenomai which are not available for install (for now) except from my github
• Build from source or deploy separately as it is the case right now

If you don't know what this means, either google it or read the doc. I still have to update the main section about real time tough. For beginners, it's probably the best to stay with PREEMPT_RT, at least until there is some experience from more advanced users.

linuxcnc -> this is already taken by the RTAI variant

Suggestion which (mostly) matches the naming already used in the source:
linuxcnc-uspace suggests: linuxcnc-uspace-xenomai, linuxcnc-uspace-xenomai-evl in the case these packages are built
linuxcnc-uspace-xenomai
linuxcnc-uspace-xenomai-evl

[hdiethelm]

@BsAtHome Would you prefer if I separate the Xenomai4 OOB improvements to a separate PR?

The library part seams to need some more discussion. But it is also needed for Xenomai3 due to Xenomai3 uses a many C and LD flags to replace posix functionality which would probably be a bad idea to apply to all components.

[hdiethelm]

To simplify review and testing, I decided to create a PR only concerning the Xenomai4 improvements: #4503 It should be merged first due to there will be some conflicts but they are easy to resolve.