profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/r7vme/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Roman Sokolkov r7vme @ApexAI Berlin https://r7v.me/about software / linux / networks / automotive / devops Currently @ApexAI , prev. @Reemote, @kopernikusauto, @giantswarm and @Mirantis

r7vme/learning-to-drive-in-a-day 83

Implementation of reinforcement learning approach to make a car learn to drive

r7vme/khmot 10

Multiple Object Tracker based on Hungarian algorithm (data association) and Kalman filter (motion estimation)

r7vme/donkey_gym 5

OpenAI gym environment for donkeycar simulator

r7vme/eagleMK4 5

The prototype of autonomous RC car based on MIT Racecar platform

r7vme/eaglemk4_nn_controller 3

Reinforsment learning (VAE + DDPG) controller for EagleMK4 car

r7vme/slam-ground-up 3

(WIP) "trial and error" notebook of implementing SLAM for monocular video input

r7vme/eagle2 2

(WIP) Self-driving gokart

r7vme/pyvmwareapi 2

Python module to work with VMware API.

r7vme/autoencoder 1

comma.ai autoencoder refactored for tensorflow 1.10+

PullRequestReviewEvent

Pull request review commenteclipse-iceoryx/iceoryx

iox-#924 Add icedocker example readme

+# Use Iceoryx In A Docker Environment++## Introduction++Let's assume we are working on a system in which `iox-roudi` runs in a docker+environment and it should orchestrate two applications which are running again+in two different docker containers so that we end up with a system of 3+different docker containers.++To demonstrate the setup we use the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++```+                         +-----------++                         | docker 1  |+                         |           |+                         | iox-roudi |+                         +-----------+++       +-------------------+        +--------------------++       | docker 2          | send   | docker 3           |+       |                   |------->|                    |+       | iox-cpp-publisher | data   | iox-cpp-subscriber |+       +-------------------+        +--------------------++```++## Requirements++### Shared Access to Unix Domain Sockets++Every iceoryx application is registering itself at our central broker RouDi+by sending a message to the unix domain socket located at+`IOX_UDS_SOCKET_PATH_PREFIX/roudi` which is defined in the corresponding+platform settings file `platform_settings.hpp`. In linux the socket file handle+can be found at `/tmp/roudi`. While registering it announces its unix+domain socket to roudi for the responses of application requests which were+send to roudi.+This socket is stored as well in `/tmp/IOX_RUNTIME_NAME`. The `iox-cpp-publisher`+runtime has the same name as the binary which leads to the socket+`/tmp/iox-cpp-publisher`.++### Shared Access to File Locks++Iceoryx applications ensure that every runtime name is unique in the system+by creating a file lock before creating the runtime. This is stored in +`IOX_LOCK_FILE_PATH_PREFIX/IOX_RUNTIME_NAME.lock` whereby +`IOX_LOCK_FILE_PATH_PREFIX` is defined in the platform settings file+`platform_settings.hpp`. When running the icedelivery example in a linux+environment one can observe+the lock files `/tmp/roudi.lock`, `/tmp/iox-cpp-subscriber.lock` and+`/tmp/iox-cpp-publisher.lock`.++### Shared Access to Semaphores and Shared Memory++One of the tasks of the central broker RouDi is to create and distribute shared+memory. When the `iox-cpp-publisher` would like to send data it acquires a+pointer to this shared memory, writes the data into it and sends the+pointer to the `iox-cpp-subscriber` which reads the memory at the received+memory position.+Additionally, it is possible to signal events across process boundaries via+semaphores. For instance to signal a subscriber that data has arrived.++## Implementation++To have shared access to the required resources we have to bind the host +filesystem:++ * `/tmp`+ * `/dev`++into every docker container.++### Example++We start in 3 separate terminals 3 docker instances. In this example we+use `archlinux:latest` but one is free to choose any other linux distribution.+The iceoryx repository which contains an already build iceoryx can be found at+`/home/user/iceoryx` which is bound to `/iceoryx`. The usage and building is +explained in detail in the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++#### Terminal 1 (iox-roudi)

FYI: docker-compose up

docker-compose.yml

services:
  roudi:
    image: archlinux:latest
    command: /iceoryx/build/iox-roudi
    volumes:
      - .:/iceoryx
      - /dev:/dev
      - /tmp:/tmp
  publisher:
    image: archlinux:latest
    command: /iceoryx/build/iceoryx_examples/icedelivery/iox-cpp-publisher
    volumes:
      - .:/iceoryx
      - /dev:/dev
      - /tmp:/tmp
  subscriber:
    image: archlinux:latest
    command: /iceoryx/build/iceoryx_examples/icedelivery/iox-cpp-subscriber
    volumes:
      - .:/iceoryx
      - /dev:/dev
      - /tmp:/tmp
elfenpiff

comment created time in 8 days

PullRequestReviewEvent

Pull request review commenteclipse-iceoryx/iceoryx

iox-#924 Add icedocker example readme

+# Use Iceoryx In A Docker Environment++## Introduction++Let's assume we are working on a system in which `iox-roudi` runs in a docker+environment and it should orchestrate two applications which are running again+in two different docker containers so that we end up with a system of 3+different docker containers.++To demonstrate the setup we use the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++```+                         +-----------++                         | docker 1  |+                         |           |+                         | iox-roudi |+                         +-----------+++       +-------------------+        +--------------------++       | docker 2          | send   | docker 3           |+       |                   |------->|                    |+       | iox-cpp-publisher | data   | iox-cpp-subscriber |+       +-------------------+        +--------------------++```++## Requirements++### Shared Access to Unix Domain Sockets++Every iceoryx application is registering itself at our central broker RouDi+by sending a message to the unix domain socket located at+`IOX_UDS_SOCKET_PATH_PREFIX/roudi` which is defined in the corresponding+platform settings file `platform_settings.hpp`. In linux the socket file handle+can be found at `/tmp/roudi`. While registering it announces its unix+domain socket to roudi for the responses of application requests which were+send to roudi.+This socket is stored as well in `/tmp/IOX_RUNTIME_NAME`. The `iox-cpp-publisher`+runtime has the same name as the binary which leads to the socket+`/tmp/iox-cpp-publisher`.++### Shared Access to File Locks++Iceoryx applications ensure that every runtime name is unique in the system+by creating a file lock before creating the runtime. This is stored in +`IOX_LOCK_FILE_PATH_PREFIX/IOX_RUNTIME_NAME.lock` whereby +`IOX_LOCK_FILE_PATH_PREFIX` is defined in the platform settings file+`platform_settings.hpp`. When running the icedelivery example in a linux+environment one can observe+the lock files `/tmp/roudi.lock`, `/tmp/iox-cpp-subscriber.lock` and+`/tmp/iox-cpp-publisher.lock`.++### Shared Access to Semaphores and Shared Memory++One of the tasks of the central broker RouDi is to create and distribute shared+memory. When the `iox-cpp-publisher` would like to send data it acquires a+pointer to this shared memory, writes the data into it and sends the+pointer to the `iox-cpp-subscriber` which reads the memory at the received+memory position.+Additionally, it is possible to signal events across process boundaries via+semaphores. For instance to signal a subscriber that data has arrived.++## Implementation++To have shared access to the required resources we have to bind the host +filesystem:++ * `/tmp`+ * `/dev`++into every docker container.++### Example++We start in 3 separate terminals 3 docker instances. In this example we+use `archlinux:latest` but one is free to choose any other linux distribution.+The iceoryx repository which contains an already build iceoryx can be found at+`/home/user/iceoryx` which is bound to `/iceoryx`. The usage and building is +explained in detail in the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++#### Terminal 1 (iox-roudi)+```+docker run --mount type=bind,source="/dev",target=/dev --mount type=bind,source=/home/user/iceoryx,target=/iceoryx --mount type=bind,source=/tmp,target=/tmp -it archlinux:latest++cd /iceoryx+./build/iox-roudi+```++#### Terminal 2 (iox-cpp-publisher)+```+docker run --mount type=bind,source="/dev",target=/dev --mount type=bind,source=/home/user/iceoryx,target=/iceoryx --mount type=bind,source=/tmp,target=/tmp -it archlinux:latest++cd /iceoryx+./build/iceoryx_examples/icedelivery/iox-cpp-publisher+```++#### Terminal 3 (iox-cpp-subscriber)++```+docker run --mount type=bind,source="/dev",target=/dev --mount type=bind,source=/home/user/iceoryx,target=/iceoryx --mount type=bind,source=/tmp,target=/tmp -it archlinux:latest

same as above

elfenpiff

comment created time in 8 days

Pull request review commenteclipse-iceoryx/iceoryx

iox-#924 Add icedocker example readme

+# Use Iceoryx In A Docker Environment++## Introduction++Let's assume we are working on a system in which `iox-roudi` runs in a docker+environment and it should orchestrate two applications which are running again+in two different docker containers so that we end up with a system of 3+different docker containers.++To demonstrate the setup we use the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++```+                         +-----------++                         | docker 1  |+                         |           |+                         | iox-roudi |+                         +-----------+++       +-------------------+        +--------------------++       | docker 2          | send   | docker 3           |+       |                   |------->|                    |+       | iox-cpp-publisher | data   | iox-cpp-subscriber |+       +-------------------+        +--------------------++```++## Requirements++### Shared Access to Unix Domain Sockets++Every iceoryx application is registering itself at our central broker RouDi+by sending a message to the unix domain socket located at+`IOX_UDS_SOCKET_PATH_PREFIX/roudi` which is defined in the corresponding+platform settings file `platform_settings.hpp`. In linux the socket file handle+can be found at `/tmp/roudi`. While registering it announces its unix+domain socket to roudi for the responses of application requests which were+send to roudi.+This socket is stored as well in `/tmp/IOX_RUNTIME_NAME`. The `iox-cpp-publisher`+runtime has the same name as the binary which leads to the socket+`/tmp/iox-cpp-publisher`.++### Shared Access to File Locks++Iceoryx applications ensure that every runtime name is unique in the system+by creating a file lock before creating the runtime. This is stored in +`IOX_LOCK_FILE_PATH_PREFIX/IOX_RUNTIME_NAME.lock` whereby +`IOX_LOCK_FILE_PATH_PREFIX` is defined in the platform settings file+`platform_settings.hpp`. When running the icedelivery example in a linux+environment one can observe+the lock files `/tmp/roudi.lock`, `/tmp/iox-cpp-subscriber.lock` and+`/tmp/iox-cpp-publisher.lock`.++### Shared Access to Semaphores and Shared Memory++One of the tasks of the central broker RouDi is to create and distribute shared+memory. When the `iox-cpp-publisher` would like to send data it acquires a+pointer to this shared memory, writes the data into it and sends the+pointer to the `iox-cpp-subscriber` which reads the memory at the received+memory position.+Additionally, it is possible to signal events across process boundaries via+semaphores. For instance to signal a subscriber that data has arrived.++## Implementation++To have shared access to the required resources we have to bind the host +filesystem:++ * `/tmp`+ * `/dev`++into every docker container.++### Example++We start in 3 separate terminals 3 docker instances. In this example we+use `archlinux:latest` but one is free to choose any other linux distribution.+The iceoryx repository which contains an already build iceoryx can be found at+`/home/user/iceoryx` which is bound to `/iceoryx`. The usage and building is +explained in detail in the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++#### Terminal 1 (iox-roudi)+```+docker run --mount type=bind,source="/dev",target=/dev --mount type=bind,source=/home/user/iceoryx,target=/iceoryx --mount type=bind,source=/tmp,target=/tmp -it archlinux:latest

nit: use -v /dev:/dev -v /home/user/iceoryx:/iceoryx -v /tmp:/tmp, this is less verbose, I agreed, but is widely used. nit2: you can run command directly, w/o cd /iceoryx; ./build/iox-roudi

Finally command can look like

docker run -v /dev:/dev -v /home/user/iceoryx:/iceoryx -v /tmp:/tmp archlinux:latest /iceoryx/build/iox-roudi
elfenpiff

comment created time in 8 days

Pull request review commenteclipse-iceoryx/iceoryx

iox-#924 Add icedocker example readme

+# Use Iceoryx In A Docker Environment++## Introduction++Let's assume we are working on a system in which `iox-roudi` runs in a docker+environment and it should orchestrate two applications which are running again+in two different docker containers so that we end up with a system of 3+different docker containers.++To demonstrate the setup we use the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++```+                         +-----------++                         | docker 1  |+                         |           |+                         | iox-roudi |+                         +-----------+++       +-------------------+        +--------------------++       | docker 2          | send   | docker 3           |+       |                   |------->|                    |+       | iox-cpp-publisher | data   | iox-cpp-subscriber |+       +-------------------+        +--------------------++```++## Requirements++### Shared Access to Unix Domain Sockets++Every iceoryx application is registering itself at our central broker RouDi+by sending a message to the unix domain socket located at+`IOX_UDS_SOCKET_PATH_PREFIX/roudi` which is defined in the corresponding+platform settings file `platform_settings.hpp`. In linux the socket file handle+can be found at `/tmp/roudi`. While registering it announces its unix+domain socket to roudi for the responses of application requests which were+send to roudi.+This socket is stored as well in `/tmp/IOX_RUNTIME_NAME`. The `iox-cpp-publisher`+runtime has the same name as the binary which leads to the socket+`/tmp/iox-cpp-publisher`.++### Shared Access to File Locks++Iceoryx applications ensure that every runtime name is unique in the system+by creating a file lock before creating the runtime. This is stored in +`IOX_LOCK_FILE_PATH_PREFIX/IOX_RUNTIME_NAME.lock` whereby +`IOX_LOCK_FILE_PATH_PREFIX` is defined in the platform settings file+`platform_settings.hpp`. When running the icedelivery example in a linux+environment one can observe+the lock files `/tmp/roudi.lock`, `/tmp/iox-cpp-subscriber.lock` and+`/tmp/iox-cpp-publisher.lock`.++### Shared Access to Semaphores and Shared Memory++One of the tasks of the central broker RouDi is to create and distribute shared+memory. When the `iox-cpp-publisher` would like to send data it acquires a+pointer to this shared memory, writes the data into it and sends the+pointer to the `iox-cpp-subscriber` which reads the memory at the received+memory position.+Additionally, it is possible to signal events across process boundaries via+semaphores. For instance to signal a subscriber that data has arrived.++## Implementation++To have shared access to the required resources we have to bind the host +filesystem:++ * `/tmp`+ * `/dev`++into every docker container.++### Example++We start in 3 separate terminals 3 docker instances. In this example we+use `archlinux:latest` but one is free to choose any other linux distribution.+The iceoryx repository which contains an already build iceoryx can be found at+`/home/user/iceoryx` which is bound to `/iceoryx`. The usage and building is +explained in detail in the +[icedelivery C++ example](https://github.com/eclipse-iceoryx/iceoryx/tree/master/iceoryx_examples/icedelivery).++#### Terminal 1 (iox-roudi)+```+docker run --mount type=bind,source="/dev",target=/dev --mount type=bind,source=/home/user/iceoryx,target=/iceoryx --mount type=bind,source=/tmp,target=/tmp -it archlinux:latest++cd /iceoryx+./build/iox-roudi+```++#### Terminal 2 (iox-cpp-publisher)+```+docker run --mount type=bind,source="/dev",target=/dev --mount type=bind,source=/home/user/iceoryx,target=/iceoryx --mount type=bind,source=/tmp,target=/tmp -it archlinux:latest

same comment as above

elfenpiff

comment created time in 8 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commenteclipse-iceoryx/iceoryx

Iox #911 design document dynamic size types

+# Dynamic Size Support for Zero Copy Types++## Summary++There are multiple obstacles when it comes to use true zero copy with types+whose size is not known at compile time. At the moment a developer can pursue+three approaches.++1. Send via untyped publisher and provide the size at runtime to the publisher.+   But defining message types with multiple dynamic sized containers would be+   a real challenge when they should be constructed inside of this sample.++2. One can perform a worst case estimation at compile time and declare+   a type with container members which fullfil those worst case estimations.+   This could lead to a lot of wasted memory and if the worst case estimations+   have to be adjusted all the applications which are using this type may have+   to be adjusted as well since the capacity is part of the type.++3. Another approach is to use the heap and copy/serialize the data into a sample+   before sending it with iceoryx. This would imply multiple copies since the+   publisher has to copy the data from the container into the sample and every+   subscriber may has to deserialize/copy the data out of the iceoryx sample.++Those obstacles can be overcome when iceoryx containers like `cxx::string`,+`cxx::vector` or `cxx::{forward}list` support custom allocators which allocate+memory in a part of the shared memory which is also in the process space of the+subscriber available.++To accomplish dynamic size types we require:++1. an allocator concept which supports states for allocators+2. have to adjust the containers like `cxx::string`, `cxx::vector` and +   `cxx::{forward}list`+3. the publisher has to provide access to the shared memory allocator via the+   sample++### Allocator Type Does Not Effect Container Type++One important restriction is for the allocator concept is that the specification+of the allocator does not change the type like the STL C++ allocator concept+does. Function developers for instance do not want to restrict their functions+to shared memory allocated types which would result in some implementation+overhead. One example could be that someone analyses an image with a function+like+`void analyseImage(cxx::vector<int, SharedMemoryAllocator> & a)`. This function+can now be only used with a vector which is using the `SharedMemoryAllocator`.+One could define the allocator type as a template but this would increase the+compile time and can make it nearly impossible to write proprietary code since+templates require the implementation in the header. Furthermore, the function+developer has to be aware of the implementation detail that there is something+like a `SharedMemoryAllocator`.++## Memory Layout++In the following section we would like to send the structure ++```cpp+struct MyData {+  int         value;+  vector<int> image;+  string      text;+};+```++and observe how this is stored currently in the shared memory of iceoryx and+compare it to the suggested solution.++An iceoryx containers memory structure looks mostly like this:+```+vector/string {+  mgmt; // contains internal variables to handle the logic and later the+        // suggested allocator++  data; // contains the user data which is stored in the container+}+```++### Static Layout++When we require a sample with our publisher and write `MyData` we gain have a+chunk where all the contents are stored inside one chunk. Hereby is the+Memory Manager the class which handles the distribution of shared memory chunks.+To support different sizes the Memory Manager uses bucket allocators with+preconfigured bucket sizes called Memory Pool.++```++--------------------- Memory Manager -------------------++|  MemPool with chunk size = 1000                        |+|  +----------------+----------------+----------------+  |+|  |  data          |                |                |  |+|  |  image::mgmt   |                |                |  |+|  |  image::data   |                |                |  |+|  |  text::mgmt    |                |                |  |+|  |  text::data    |                |                |  |+|  +----------------+----------------+----------------+  |+|                                                        |+|  MemPool with chunk size = 100                         |+|  +---+---+---+---+---+---+---+---+---+---+---+---+---+ |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  +---+---+---+---+---+---+---+---+---+---+---+---+---+ |++--------------------------------------------------------++```++### Layout With Dynamic Sized Types++With dynamic sized types we move the data part outside of the container and+add an allocator to the container which allows the container to acquire+memory for their data part. When we assume that `MyData::image` and+`MyData::text` require 100 bytes for their data part the memory layout could+look like this.++```++--------------------- Memory Manager -------------------++|  MemPool with chunk size = 1000                        |+|  +----------------+----------------+----------------+  |+|  |  data          |                |                |  |+|  |  image::mgmt   |                |                |  |+|  |  text::mgmt    |                |                |  |+|  +-------------+--+----------------+----------------+  |+|                |                                       |+|                +-------------------+-----------+       |+|  MemPool with chunk size = 100     |           |       |+|  +---+---+---+---+---+---+---+---+-+-+---+---+-+-+---+ |+|  |   |   |   |   |   |   |   |   |img|   |   |txt|   | |+|  |   |   |   |   |   |   |   |   |dat|   |   |dat|   | |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  +---+---+---+---+---+---+---+---+---+---+---+---+---+ |++--------------------------------------------------------++```++This does not require any adaptions on the subscriber side since the subscriber+already mapped this memory region which is handled by the memory manager into +its local process.++## Design Allocator++The allocator should be designed in a manner so that it can be used inside the+shared memory in an IPC context. This means the design has to consider that+vtables, virtual and inheritance are not allowed as well as function pointers+or the use of `cxx::function` or `cxx::function_ref`.++Therefore this allocator concept is less flexibel and more complex as it would
Therefore this allocator concept is less flexible and more complex as it would
elfenpiff

comment created time in 22 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commenteclipse-iceoryx/iceoryx

Iox #911 design document dynamic size types

+# Dynamic Size Support for Zero Copy Types++## Summary++There are multiple obstacles when it comes to use true zero copy with types+whose size is not known at compile time. At the moment a developer can pursue+three approaches.++1. Send via untyped publisher and provide the size at runtime to the publisher.+   But defining message types with multiple dynamic sized containers would be+   a real challenge when they should be constructed inside of this sample.++2. One can perform a worst case estimation at compile time and declare+   a type with container members which fullfil those worst case estimations.+   This could lead to a lot of wasted memory and if the worst case estimations+   have to be adjusted all the applications which are using this type may have+   to be adjusted as well since the capacity is part of the type.++3. Another approach is to use the heap and copy/serialize the data into a sample+   before sending it with iceoryx. This would imply multiple copies since the+   publisher has to copy the data from the container into the sample and every+   subscriber may has to deserialize/copy the data out of the iceoryx sample.++Those obstacles can be overcome when iceoryx containers like `cxx::string`,+`cxx::vector` or `cxx::{forward}list` support custom allocators which allocate+memory in a part of the shared memory which is also in the process space of the+subscriber available.++To accomplish dynamic size types we require:++1. an allocator concept which supports states for allocators+2. have to adjust the containers like `cxx::string`, `cxx::vector` and +   `cxx::{forward}list`+3. the publisher has to provide access to the shared memory allocator via the+   sample++### Allocator Type Does Not Effect Container Type++One important restriction is for the allocator concept is that the specification+of the allocator does not change the type like the STL C++ allocator concept+does. Function developers for instance do not want to restrict their functions+to shared memory allocated types which would result in some implementation+overhead. One example could be that someone analyses an image with a function+like+`void analyseImage(cxx::vector<int, SharedMemoryAllocator> & a)`. This function+can now be only used with a vector which is using the `SharedMemoryAllocator`.+One could define the allocator type as a template but this would increase the+compile time and can make it nearly impossible to write proprietary code since+templates require the implementation in the header. Furthermore, the function+developer has to be aware of the implementation detail that there is something+like a `SharedMemoryAllocator`.++## Memory Layout++In the following section we would like to send the structure ++```cpp+struct MyData {+  int         value;+  vector<int> image;+  string      text;+};+```++and observe how this is stored currently in the shared memory of iceoryx and+compare it to the suggested solution.++An iceoryx containers memory structure looks mostly like this:+```+vector/string {+  mgmt; // contains internal variables to handle the logic and later the+        // suggested allocator++  data; // contains the user data which is stored in the container+}+```++### Static Layout++When we require a sample with our publisher and write `MyData` we gain have a+chunk where all the contents are stored inside one chunk. Hereby is the+Memory Manager the class which handles the distribution of shared memory chunks.+To support different sizes the Memory Manager uses bucket allocators with+preconfigured bucket sizes called Memory Pool.++```++--------------------- Memory Manager -------------------++|  MemPool with chunk size = 1000                        |+|  +----------------+----------------+----------------+  |+|  |  data          |                |                |  |+|  |  image::mgmt   |                |                |  |+|  |  image::data   |                |                |  |+|  |  text::mgmt    |                |                |  |+|  |  text::data    |                |                |  |+|  +----------------+----------------+----------------+  |+|                                                        |+|  MemPool with chunk size = 100                         |+|  +---+---+---+---+---+---+---+---+---+---+---+---+---+ |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  +---+---+---+---+---+---+---+---+---+---+---+---+---+ |++--------------------------------------------------------++```++### Layout With Dynamic Sized Types++With dynamic sized types we move the data part outside of the container and+add an allocator to the container which allows the container to acquire+memory for their data part. When we assume that `MyData::image` and+`MyData::text` require 100 bytes for their data part the memory layout could+look like this.++```++--------------------- Memory Manager -------------------++|  MemPool with chunk size = 1000                        |+|  +----------------+----------------+----------------+  |+|  |  data          |                |                |  |+|  |  image::mgmt   |                |                |  |+|  |  text::mgmt    |                |                |  |+|  +-------------+--+----------------+----------------+  |+|                |                                       |+|                +-------------------+-----------+       |+|  MemPool with chunk size = 100     |           |       |+|  +---+---+---+---+---+---+---+---+-+-+---+---+-+-+---+ |+|  |   |   |   |   |   |   |   |   |img|   |   |txt|   | |+|  |   |   |   |   |   |   |   |   |dat|   |   |dat|   | |+|  |   |   |   |   |   |   |   |   |   |   |   |   |   | |+|  +---+---+---+---+---+---+---+---+---+---+---+---+---+ |++--------------------------------------------------------++```++This does not require any adaptions on the subscriber side since the subscriber+already mapped this memory region which is handled by the memory manager into +its local process.++## Design Allocator++The allocator should be designed in a manner so that it can be used inside the+shared memory in an IPC context. This means the design has to consider that+vtables, virtual and inheritance are not allowed as well as function pointers+or the use of `cxx::function` or `cxx::function_ref`.++Therefore this allocator concept is less flexibel and more complex as it would+be when everything would run in the same process since we have to use relative+pointer and have to allocate everything on the stack.++```+  +--------------------------------------------------------++  |  Allocator [Concept]                                   |+  |                                                        |+  |   - void* allocate(const uint64_t)                     |+  |   - void free(void* const chunk)                       |+  |   - template<AnotherAllocator>                         |+  |     bool isMoveCompatible(const AnotherAllocator& rhs) |+  +---+--------+-------------------------------------------++      |        |+      |  +-----+---------------------------------------------------++      |  | RangeAllocator                                          |+      |  |                                                         |+      |  |   - RangeAllocator(void* start, void* end)              |+      |  |   - void * allocate(const uint64_t)                     |+      |  |   - void free(void* chunk)                              |+      |  |   - template<AnotherAllocator>                          |+      |  |     bool isMoveCompatible(const AnotherAllocator & rhs) |+      |  |                                                         |+      |  |   # void * m_start                                      |+      |  |   # void * m_end                                        |+      |  +---------------------------------------------------------++      |+   +--+------------------------------------------------------++   | HeapAllocator                                           |+   |                                                         |+   |   - void * allocate(const uint64_t)                     |+   |   - void free(void* chunk)                              |+   |   - template<AnotherAllocator>                          |+   |     bool isMoveCompatible(const AnotherAllocator & rhs) |+   +---------------------------------------------------------++++--------------------------------++| template<typename Type>        |+| IsAllocatorTypeTrait           |   Used to verify that AllocatorTypes+|                                +---------------------------------------++|   static constexpr bool value; |  are compliant with Allocator Concept |++--------------------------------+                                       |+                                                                         |+                    +----------------------------------------------------+----++                    | template<AllocatorTypes...>                             |+                    | VariantAllocator                                        |+                    |                                                         |+                    |   - void * allocate(const uint64_t)                     |+                    |   - void free(void * chunk)                             |+                    |   - template<AnotherAllocator>                          |+                    |     bool isMoveCompatible(const AnotherAllocator & rhs) |+                    |                                                         |+                    |   # void* m_ptrToAllocator                              |+                    +---------------------------------------------------------++```+The `Allocator` described in this diagram is similar to a C++20 concept which can+be verified at compiletime without inheritance. The verification can be realized+with a `IsAllocatorTypeTrait` in C++14.+The shared memory and IPC restrictions are forcing us to implement a+VariantAllocator which provides us type independent access to the underlying+allocator like inheritance would provide. Hereby the implementation for+`allocate`, `free` and `isMoveCompatible` is always the same and can be+implemented similar to the `cxx::variant` move and copy operations. This would+make the `VariantAllocator` easier extendable since the user just has to add+any additional allocator as type in the variadic template list. An alternative+way of implementing this `VariantAllocator` would be with a switch statement+in every method which is selecting the correct type. This would lead to a less+extendable `VariantAllocator` since the implementation has to be adjusted with+every new allocator but would may provide the benefit that the `VariantAllocator`+does not require to be a template class anymore.++The `isMoveCompatible` method will be used by every container to see verify if+a real move of the underlying structure can be performed or not. This could be+the case for instance with two kinds of heap allocators are when the used+RangeAllocator is managing the same range. If this is not the case the move+operations will be replaced with an expensive copy operation (fake move).++Furthermore, the suggested zero copy types will only once acquire memory from an+allocator. The reason is that we would like to guarantee zero copy throughout the+usage of the container otherwise some intransparent copies may occur when the+container allocates memory multiple times and requires on one contiguous piece+of memory which the allocator may not provide. (See behavior of `realloc` in C).++### Implementation of a C++20 Concept in C++14+The type trait can be implemented via the C++ SFINAE (substition failure is not+an error) principle.+```cpp+template<typename T>+struct IsAllocatorTypeTrait {+    template <typename C, class = void>+    struct HasAllocateMethod : std::false_type {};++    template <typename C>+    struct HasAllocateMethod<+        C, std::void_t<std::enable_if_t<+               std::is_same<decltype(std::declval<C>().allocate(+                                std::declval<uint64_t>())),+                            void>::value,+               void>>> : std::true_type {};++    // test all other methods+    static_assert(HasAllocateMethod<T>::value, +              "Type is not an allocator, void* allocate(uint64_t) missing");+};+```++## Design of Zero Copy Containers++The design pursues the following goals.++1. The `cxx::vector<T, Capacity>` should not change in its API.+2. All static cxx containers are still owner of all their data. This means the+   memory which the stack allocator manages must be stored inside the cxx+   container class.+3. A function developer should not have to specify the allocator type of a+   container. This means the allocator type is independent of the container type.+   This has to work:+   ```cpp+   void myFunkyFunction(const cxx::vector<int> & bla) {}++   cxx::vector<int, 20> a;+   cxx::vector<int, 30> b;+   cxx::vector<int> c(Allocator::heap);++   myFunkyFunction(a);+   myFunkyFunction(b);+   myFunkyFunction(c);+   ```+4. All containers must be shared memory compatible, this means no vtables and+   virtual. Furthermore the usage of function pointers, `cxx::function` and+   `cxx::function_ref` is forbidden in all containers and their underlying +   constructs.+5. Before using any cxx container the user as to call `reserve()` once otherwise+   the container will have a default capacity of zero. It is not allowed to+   call `reserve()` multiple times since this could violate the zero copy+   guarantee.+   Stack based containers do not require a `reserve()` call beforehand but+   should provide this call as well for compatibility reasons.++The draft we provide here is using the `cxx::vector` but the techniques+described can be easily applied to the `cxx::string` or other cxx containers.++```++---------------------------------------------------++|template<T>                                        |+|vector                                             |+|                                                   |+|  - vector() = default                             |+|  - template<Allocator>                            |+|    vector(Allocator & allocator);                 |+|                                                   |+|  - void reserve(const uint64_t);                  |+|  - void release();                                |+|                                                   |+|  // remaining API is unchanged                    |++---------------------------------------------------++                          ^+                         /|\+                          |+       +------------------+--------------------------++       |template<T, Capacity>                        |+       |vector : public vector<Type>                 |+       |                                             |+       |  (#1)                                       |+       |  - vector(const vector<T> &)                |+       |  - vector(vector<T> &&)                     |+       |  - vector& operator=(const vector<T> &)     |+       |  - vector& operator=(vector<T> &&)          |+       |                                             |+       |                                             |+       |  // same API as current cxx::vector         |+       |                                             |+       |  # uint8_t m_data[Capacity]                 |+       |  # RangeAllocator m_allocator               |+       +---------------------------------------------++```+All non stack based version of cxx containers will not have constructors similar+to the STL. The only provided constructor has one argument which specifies the+type of allocator one would like to use for that container. If no argument is+provided the heap allocator will be used by default.++The constructors and operations defined in (#1) are added to support implicit+conversion from the generic `cxx::vector` which is required to the stack version+of a vector. It makes the following operations possible+```cpp+void f(const vector<int, 20> &a) {}++vector<int> v1;+vector<int, 20> v2;++// this works thanks to implicit conversions from vector<int>+f(v1);+v2 = v1;+```++Since every stack based vector is a child of the more generic vector we can with+the help of (#1) also assign and use stack based vectors of different sizes+to each other or use them in functions. This allows us to write code like this:+```cpp+void f(const vector<int, 20> &a) {}++vector<int, 10> v1;+vector<int, 20> v2;++// this works since vector<int, 10> is also a vector<int> and can be converted+v2 = v1;+v1 = v2; // this can fail when v1 capacity is smaller then v2.size, previously +         // it was only possible when v1.capacity < v2.capacity+f(v1);+f(v2);+```++The only restriction are functions which require a stack based vector of a +specific size as non const reference. This is usally the case when some function+would like to set the contents of a vector.+```cpp+void setVectorContents(vector<int, 20> &a) {}++vector<int, 10> v1;+vector<int> v2;++// both calls will result in a compile time warning+setVectorContents(v1);+setVectorContents(v2);+```++### Usage++```cpp+cxx::vector<int, 20> a;+a.emplace_back(123); // can be used without a preceding reserve call++cxx::vector<int> c(heapAllocator);+c.reserve(42); // reserve required

what should happen if no reserve was called?

elfenpiff

comment created time in 22 days

pull request commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

@elBoberido @elfenpiff @dkroenke @mossmaurice

Sorry for a huge PR. If you have time please take a look. And I welcome any suggestions.

There is also a new CI job that runs clang-tidy check for iceoryx_hoofs package.

r7vme

comment created time in 2 months

push eventApexAI/iceoryx

Roman Sokolkov

commit sha 6b73bcb24720dab22ed50f6ea69712100c87832b

iox#889 Fix clang-tidy violations for iceoryx_hoofs

view details

push time in 2 months

push eventApexAI/iceoryx

Manel Melaouhi

commit sha 2b1064eb1f326638fb57a1c28a43917f2607c225

iox-#496 add unit tests to improve coverage of toml_gateway_config_parser class Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Manel Melaouhi

commit sha 24b78d113cb7a0f6ac5ee2155c8bd7541d31ae73

iox-#496 add gateway_Base test file Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Manel Melaouhi

commit sha 568ed037f64b967c6fff413c6858e6162190e663

iox-#496 update copyright Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Manel Melaouhi

commit sha 4d1e4f54f9f0fe7264c51208558c59813647f52f

iox-#496 fix review findings Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Manel Melaouhi

commit sha cd788a306e60ae19e2a87142bc979e760428641c

iox-#496 fix review findings and add a test for gw base Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Manel Melaouhi

commit sha d7aa9ef70422a5f1b158de16a14a9167fee4189a

iox-#496 use parameterized tests for toml gateway config test file Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha 363c288a5b5329395307ff7cb0d4f9f1f7177e6c

Merge remote-tracking branch 'upstream/master' into iox-#496-Enhancing-coverage-for-folder-gateway

view details

Manel Melaouhi

commit sha c266c47bf9dd44d5bb6ab5b238ffa28aae602994

iox-#496 use parameterized tests for gateway base test file Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha 75971476075bffa1ef203894805a0dce07be3905

iox-#496 use parameterized tests for gateway base test file and add type tuple for toml gateway test file Signed-off-by: Chiranjeevi Maddi (RBEI/EBB1) <chiranjeevi.maddi@in.bosch.com>

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha 1baab7c808a6055dade98ffd068d0b9dac305b09

iox-#496 updated the test file with clang Signed-off-by: Chiranjeevi Maddi (RBEI/EBB1) <chiranjeevi.maddi@in.bosch.com>

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha af99cef135a3fda3a732fff4856f547b1d3e04b8

Merge branch 'master' of https://github.com/eclipse-iceoryx/iceoryx into iox-#496-Enhancing-coverage-for-folder-gateway

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha 1c83124d43628f2f1144aadae60c7609d30d2c35

iox-#496 fixing build issues Signed-off-by: Chiranjeevi Maddi (RBEI/EBB1) <chiranjeevi.maddi@in.bosch.com>

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha 7c4ac217908b7b3ca9a4dbd6c2ae4900779b86ec

iox-#496 fixing clang build issue Signed-off-by: Chiranjeevi Maddi (RBEI/EBB1) <chiranjeevi.maddi@in.bosch.com>

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha 3e060ab0b114c145c0d00a00edd17f092fda388f

iox-#496 fixing build issues Signed-off-by: Chiranjeevi Maddi (RBEI/EBB1) <chiranjeevi.maddi@in.bosch.com>

view details

Chiranjeevi Maddi (RBEI/EBB1)

commit sha a0b94a0d89a6de8669c6b125d075055ee7f7c891

iox-#496 trying to fix clang build issue Signed-off-by: Chiranjeevi Maddi (RBEI/EBB1) <chiranjeevi.maddi@in.bosch.com>

view details

Manel Melaouhi

commit sha 37f8415a7d40ea0dd790c16492c71dd500197ea4

iox-#496 add tests to check the maximum allowed number of configured services and to check duplicated services decription Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Chari Nehal Dattaram (RBEI/EBB1)

commit sha bdeada967df20866f2022e21791fa758129d4771

Merge remote-tracking branch 'origin/master' into iox-#496-Enhancing-coverage-for-folder-gateway Signed-off-by: Chari Nehal Dattaram (RBEI/EBB1) <NehalDattaram.Chari@in.bosch.com>

view details

Manel Melaouhi

commit sha 00c9100462a7adba3198adee944fc0d3f652ff74

iox-#496 add MAXIMUM_NUMBER_OF_ENTRIES_EXCEEDED into TomlGatewayConfigParseError enum and return it Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Manel Melaouhi

commit sha 3a1e11118c566a9750eb0cc9b78aca78927a03de

iox-#496 check contents of config Signed-off-by: Manel Melaouhi <external.Manel.Melaouhi@de.bosch.com>

view details

Chiranjeevi Maddi

commit sha 4d7698af276ab52633da0ac4c1d00ab015474108

iox-#496 removed test_gateway_gateway_base.cpp file, as there is a build problem in Iceory Windows variant. Signed-off-by: Chiranjeevi Maddi (RBEI/EBB1) <chiranjeevi.maddi@in.bosch.com>

view details

push time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 namespace iox { namespace log {-Logger& CreateLogger(std::string ctxId, std::string ctxDescription, LogLevel appDefLogLevel = LogLevel::kWarn) noexcept;+Logger& createLogger(const std::string& ctxId,+                     const std::string& ctxDescription,+                     LogLevel appDefLogLevel = LogLevel::kWarn) noexcept;

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 class SignalGuard    private:     Signal m_signal;-    struct sigaction m_previousAction;+    struct sigaction m_previousAction

Heh, here compiler complains with if I remove struct

       Must use 'struct' tag to refer to type 'sigaction' in this scope (fix                                                                                                                                                                                                                                        
       available)                                                                                                                                                                                                                                                                                                   
  };   [clang: use_of_tag_name_without_tag] 

I think the problem that there also a function defined with name sigaction

So I just disable clang format here to have struct sigaction m_previousAction{}

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 cxx::expected<std::string, IpcChannelError> UnixDomainSocket::receive() const no {     // we also support timedReceive. The setsockopt call sets the timeout for all further recvfrom calls, so we must set     // it to 0 to turn the timeout off-    struct timeval tv;+    struct timeval tv

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 cxx::expected<TimerError> Timer::OsTimer::restart(const units::Duration timeToWa  cxx::expected<units::Duration, TimerError> Timer::OsTimer::timeUntilExpiration() noexcept {-    struct itimerspec currentInterval;+    struct itimerspec currentInterval

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 struct timespec Duration::timespec(const TimeSpecReference& reference) const noe     }     else     {-        struct timespec referenceTime;+        struct timespec referenceTime

as above requires struct

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 void Timer::OsTimer::executeCallback() noexcept cxx::expected<TimerError> Timer::OsTimer::start(const RunMode runMode, const CatchUpPolicy catchUpPolicy) noexcept {     // Convert units::Duration to itimerspec-    struct itimerspec interval;+    struct itimerspec interval

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 void RouDi::processMessage(const runtime::IpcMessage& message,         }         else         {-            uint32_t pid;-            uid_t userId;-            int64_t transmissionTimestamp;+            uint32_t pid{0};

got it, done

cc @mossmaurice as expert in custom clang-tidy rules

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 cxx::expected<IpcChannelError> MessageQueue::timedSend(const std::string& msg, c  cxx::expected<bool, IpcChannelError> MessageQueue::isOutdated() {-    struct stat sb;+    struct stat sb

not really insired (in context of this PR :stuck_out_tongue: )

same as above, just disabled formatting as it requires to have struct stat (there are inline function defined in the header)

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 void SignalGuard::restorePreviousAction() noexcept  SignalGuard registerSignalHandler(const Signal signal, const SignalHandlerCallback_t callback) noexcept {-    struct sigaction action;+    struct sigaction action

as above disabled formatting

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 TimerError Timer::OsTimer::getError() const noexcept  cxx::expected<units::Duration, TimerError> Timer::now() noexcept {-    struct timespec value;+    struct timespec value

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 Timer::OsTimer::OsTimer(const units::Duration timeToWait, const std::function<vo     }      // Create the struct in order to configure the timer in the OS-    struct sigevent asyncCallNotification;+    struct sigevent asyncCallNotification

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 class PosixGroup {   public:     using string_t = cxx::string<100>;-    PosixGroup(gid_t f_id);-    PosixGroup(string_t f_name);+    explicit PosixGroup(gid_t f_id);

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 cxx::expected<TimerError> Timer::OsTimer::stop() noexcept         return cxx::success<void>();     } -    struct itimerspec interval;+    struct itimerspec interval

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 class PosixUser     using groupVector_t = cxx::vector<PosixGroup, MaxNumberOfGroups>;     using string_t = cxx::string<100>; -    PosixUser(uid_t f_id);-    PosixUser(const string_t& f_name);+    explicit PosixUser(uid_t f_id);

done

r7vme

comment created time in 2 months

Pull request review commenteclipse-iceoryx/iceoryx

iox-#889 Fix clang-tidy violations for iceoryx_hoofs package

 void Logger::Print(const LogEntry entry) const     auto sec = std::chrono::duration_cast<std::chrono::seconds>(entry.time);     std::time_t time = sec.count(); -    auto timeInfo = std::localtime(&time);+    // TODO: std::localtime is thread-unsafe, may be remove

https://en.cppreference.com/w/cpp/chrono/c/localtime This function may not be thread-safe.

#include <ctime>    
#include <iostream>    
#include <thread>    
    
void f1() {    
  std::time_t t{};    
  std::cout << std::localtime(&t) << '\n';    
}    
    
int main() {    
  std::thread t1(f1);    
  std::thread t2(f1);    
  t1.join();    
  t2.join();    
}    

gives

tmp|⇒ ./1
0x7f5bae678ec0
0x7f5bae678ec0

i.e. std::localtime updates the same internal struct

Not sure what we should do here :thinking:

r7vme

comment created time in 2 months