ROS 2 Component Intra-Process Communication Optimization

ROS 2 composition lets multiple nodes share a single process, and that architectural choice makes true zero-copy communication possible. Everything downstream of that decision, the rclcpp intra-process path, loaned messages, iceoryx, Agnocast, exists to answer one question: once nodes share memory, how much copying can actually be eliminated, and at what cost to safety and generality? The answer turns out to be layered, and each layer trades away something the last one assumed was free.

The Purpose and Impact of ROS 2 Composition on Node Communication

ROS 1 nodes almost always ran as separate operating system processes. Nodelets existed as a workaround, letting multiple nodes share a process, but they were bolted on rather than designed in. ROS 2 rebuilt this idea from the ground up with composition: nodes run inside a Component Container, and a container can host any number of them.

The shift matters because of what it changes about the data path. Nodes in separate processes have no choice but to serialize a message, send it across a process boundary, and deserialize it on the other side. Nodes co-located in the same container can skip all of that and pass a pointer instead. The message never leaves the address space it was created in.

That single change is what makes near-zero-latency communication between nodes possible, while leaving the publish/subscribe programming model untouched. A node written against the standard rclcpp API doesn't need to know where its subscriber lives, whether in the same process or on a machine three away. Composition, in other words, is the precondition for intra-process communication. It's the precondition. Without co-location, intra-process communication has nothing to optimize.

The built-in rclcpp IPC mechanism at the implementation level

Turning this on is a single flag: rclcpp::NodeOptions().use_intra_process_comms(true). Once set, and once a publisher and subscriber for the same topic land in the same process, the DDS middleware gets bypassed for the actual data transfer. Messages move through in-process ring buffers instead of getting serialized and deserialized, though meta-messages describing what's available still travel through the RMW layer.

The mechanism underneath is smart pointers pushed into shared ring buffers. No wire format, no memory copy of the payload itself, just a pointer handoff.

But whether that handoff is actually zero-copy depends on the pointer type chosen in application code. Publish or subscribe using std::unique_ptr, and ownership moves cleanly from publisher to subscriber: true zero-copy, one owner at a time. Publish or subscribe using const& or std::shared_ptr, and zero-copy does not occur. The message still gets copied, because the middleware has to assume multiple subscribers might hold onto that shared pointer at once, and it has no way to hand out one unique memory region to several owners simultaneously. This detail trips up a lot of otherwise careful engineers: enabling IPC without also auditing pointer types buys almost nothing.

The QoS and configuration constraints that limit where built-in IPC can be applied

Built-in IPC is compatible with a subset of QoS profiles, and that subset has changed over time. It's compatible with a subset, and that subset has changed over time. Transient Local durability, for instance, was unsupported by the IPC implementation for years; support only landed in ROS 2 Jazzy, tracked in rclcpp issue #2303.

A subtler problem appears in topics with mixed subscribers, some intra-process, some inter-process. Intuition suggests the intra-process ones should be unaffected by whatever the inter-process ones are doing. That intuition is wrong. Adding an inter-process subscriber to a topic raises the latency of the intra-process path too, a penalty that breaks the assumption that turning IPC on is a strictly neutral-or-beneficial move. Engineers who enable IPC as a blanket policy across a graph, expecting it to help everywhere and hurt nowhere, are working from an incorrect mental model.

A floor sets a limit on all of this too, and it produces the constraints described next. Even in a single process with IPC enabled, meta-messages still go through the RMW layer, and most RMW implementations do not optimize that path away. Practically, this means a single-process ROS 2 application with IPC turned on can still underperform a non-ROS application built directly on shared memory. The gap is architectural.

The scalability problem the research community identified in 2025–2026

For a long time, the working assumption in system design was that IPC overhead, whatever serialization cost you avoid by staying in-process, is either negligible or roughly constant. Treat it as a fixed tax and move on.

That assumption doesn't survive contact with careful measurement. A paper presented at the IEEE Real-Time Systems Symposium in Boston, "On the Scalability and Efficiency of Intra-Process Communication in ROS 2," found that IPC performance is highly sensitive to message configuration, workload structure, and the specific way messages get used in application code. Treating it as a flat constant produces performance models that are simply wrong.

The paper goes further and identifies a concrete scalability risk tied to misaligned communication configurations, situations where the way publishers and subscribers are set up interacts badly with how the underlying buffers and QoS settings behave under load. It proposes a configuration guideline specifically to help system designers avoid stepping into that trap. The finding matters less as a novelty and more as a correction: any capacity plan built on "IPC is basically free" needs revisiting.

The fault isolation trade-off that composition cannot solve on its own

Zero-copy in a ComponentContainer works because nodes share a process. That's also exactly the source of a cost nobody can configure away: the process boundary is the fault isolation boundary, and collapsing the first collapses the second along with it.

Autonomous driving stacks make the stakes concrete. Localization, object recognition, and a dozen other functions often run as separate nodes that collaborate constantly, and putting them in one container is the natural way to get the latency composition promises. But a single node failure in that container can take the entire co-located process down with it, according to findings on systems at that scale (arXiv:2506.16882). One faulty perception node doesn't just lose its own output; it can silence localization at the same moment.

Shared address space and pointer passing are precisely what makes composition efficient, and precisely what removes any fault containment between the nodes involved. It's a structural property of the in-process model, and any team adopting composition for performance needs to accept the fault-isolation trade as part of the deal, not an oversight to be patched.

What the loaned messages API offers

Loaned messages take a different approach: zero-copy across process boundaries, without requiring nodes to live in the same process. For teams that need fault isolation and low latency at the same time, this looks like the obvious answer.

It comes with real limits, though. Loaned messages are a separate API surface, not something that gets transparently substituted underneath existing publisher and subscriber code. A node has to be written in advance with the knowledge that it will use loaned messages; retrofitting it later isn't a drop-in change. Dynamic data types aren't supported either, which rules out a wide swath of real message traffic before it even gets a chance to work.

Benchmarks tell a two-sided story (arXiv:2305.09933). Composition using both intra-process communication and loaned messages cuts CPU use and latency by more than half compared to standard multi-process communication, and that holds across every message size tested. Plain multi-process communication without loaned messages, by contrast, sees latency exceed 30 percent of the publication period once messages grow past 1 MB, a serious problem for any sensor-heavy pipeline. Multi-process communication without loaned messages, by contrast, sees latency exceed 30 percent of the publication period once messages grow past 1 MB, and loaned messages themselves showed instability at scale: repeated crashes once message size passed 2 MB. It's a reliability ceiling that rules loaned messages out for exactly the large-payload workloads, camera frames, point clouds, where zero-copy matters most.

Iceoryx and CycloneDDS shared memory: what they provide and the fixed-size constraint

Iceoryx takes a related but distinct approach: an IPC middleware built specifically for zero-copy data transmission between processes on the same machine, again without requiring shared process space. It's available in ROS 2 through CycloneDDS integration, which allows a system to use shared-memory communication alongside conventional transports on the same machine.

The shared memory path through CycloneDDS comes with a specific list of constraints. The message has to have fixed length. QoS configuration has to be compatible with the shared memory transport. A topic can support at most 127 subscriptions. A publisher can hold at most 8 loaned messages at once. These aren't arbitrary numbers pulled from a spec sheet for their own sake; they reflect how the underlying shared memory pools are structured and sized ahead of time.

That fixed-length requirement is the crux of it. Zero-copy transmission means the middleware hands a subscriber a pointer directly into memory the publisher wrote to, and to do that safely, the middleware has to know in advance how large that memory region needs to be. A dynamically sized message, a string, a std::vector holding a point cloud, doesn't tell you its size until runtime. There's no way to pre-allocate a shared memory slot for something whose size isn't known ahead of time, so true zero-copy through this path is only available to data structures whose size is fixed.

Unsized message types as the central bottleneck in production ROS 2 systems

That fixed-size requirement collides directly with what production robotics systems actually send over the wire. Sensor data, images, point clouds, is variable in size almost by definition, and gets represented in C++ using unsized types like std::vector.

Autoware, one of the largest open-source ROS 2 applications running today, depends heavily on unsized message types throughout its own codebase and throughout the broader ecosystem of ROS 2 libraries it builds on. It's how the vast majority of real sensor data is naturally represented.

The consequence is that the messages carrying the most data, the big images and dense point clouds that dominate a perception pipeline's actual bandwidth, are exactly the messages existing zero-copy solutions can't touch. Current applications end up bottlenecked by serialization and copying overhead in DDS precisely at the points where that overhead would matter most to eliminate.

Earlier research efforts, TZC and LOT among them, did manage to achieve zero-copy transfer for unsized types. But they required significant code modifications at call sites throughout the application, which is a heavy lift to retrofit onto an existing, large codebase. A solution that demands every message-handling call site get rewritten to manage its own shared memory isn't a solution that scales to a project the size of Autoware.

Agnocast: how it handles unsized types in Autoware

Agnocast, introduced by T. Ishikawa-Aso and S. Kato in "ROS 2 Agnocast: Supporting Unsized Message Types for True Zero-Copy Publish/Subscribe IPC" at IEEE ISORC 2025, sets out to close that gap directly. A production version, maintained by the Autoware Foundation, was released on March 24, 2026, and the project was also presented at ROSCon 2025 in Singapore under the title "Agnocast: A ROS 2-Compatible Middleware Enabling True Zero-Copy IPC for Unsized Message Types."

The design rests on two ideas. First, Agnocast maps the publisher's heap directly into shared memory, so a subscriber accesses the message object exactly where it was constructed, with no copy step in between at all, sized or unsized. Second, it introduces a smart pointer mechanism built specifically to manage object lifetimes across process boundaries, without requiring the programmer to manage shared memory allocation by hand the way TZC and LOT did.

Put together, this lets Agnocast handle any ROS 2 message type, unsized ones included, while requiring minimal modification to existing code. That last part is what makes it applicable to a codebase the size of Autoware's, where rewriting call sites one by one was never going to be realistic.

The cross-process lifetime management problem and the two architectural responses to it

Solving the problem of copying directly produces a different problem: who's responsible for freeing the memory once nobody needs it anymore? In true zero-copy IPC, a subscriber holds a reference straight into memory the publisher owns, and the publisher can't safely reclaim that memory until every subscriber holding a reference has released it. Processes, though, don't always cooperate. They crash. They join late, particularly under Transient Local QoS, where a subscriber can show up and expect access to a message published before it existed. They can leave without warning at any point.

A process-local reference count, the kind std::shared_ptr keeps track of within a single process, can't answer this question on its own, because the count that matters spans every process holding a reference, not just one. This is the problem taken up in "ipc_shared_ptr: A Publish/Subscribe-Aware Smart Pointer for Cross-Process Object Lifetime Management" (IEEE ISORC 2026, arXiv:2605.04226, submitted May 5, 2026): reclamation decisions need visibility across the whole set of referencing processes, since the count that matters spans every process holding a reference.

Two architectural answers to this have emerged. Owner-driven reclaim, embodied in iceoryx2, offers greater scalability in principle by distributing the bookkeeping. But letting membership changes and reclamation decisions happen concurrently opens the door to races, and every race the design allows expands the state space that has to be verified for correctness. Single-writer designs, embodied in Agnocast's ipc_shared_ptr, take the opposite bet: centralize the reclamation decision in one place, and the race complexity disappears by construction. The cost is a centralized bottleneck, one component now responsible for a decision that used to be distributed.

Which trade-off wins depends on the workload, and at Autoware's scale, the benchmark in arXiv:2605.04226 gives a clear answer. At 200 topics, 2 subscribers each, running at 100 Hz, Agnocast's end-to-end p99.9 latency comes in substantially lower than iceoryx2's. Centralizing the bottleneck, at this scale, turns out to cost less than the race conditions it avoids. Whether that holds at a different scale, with a different topic count or subscriber fan-out, is the kind of thing the 2025 RTSS paper's warning about configuration sensitivity should make any engineer reluctant to assume an answer to without measuring it directly.