Attribute matching

An attribute condition could always say rpc.service = abc. Since v0.11.144 and v0.11.145 it can also say that the attribute is not something, or that it is not there at all - which is what you need to separate two flows through the same service, told apart by a field only one of them sets.

It can now also say “and everything that runs below that”. An attribute condition used to select only the spans carrying the attribute themselves; include child spans makes the condition choose an anchor and pulls the anchor’s whole subtree in with it. That is the shape most integrations actually have: the field is set once, on the step that receives the message, and none of the steps below it repeat the value. Include child spans is the second part of this page.

Both of those are about what a single condition selects. v0.11.149 adds a choice about an integration’s rules taken together: they have always been a union, where one rule matching was enough for a span to belong, and an integration can now ask instead that every rule match.

Surface	Negation and existence operators	What a row is about
Integration matchers	is not, doesn't contain, exists, is absent	One span
Metric selection	≠, !contains, exists, is absent	One series
Log attribute filters	≠, !contains, exists, is absent	One row
Trace alert rules	≠, doesn't contain, exists, is absent	One span
Saved views and message filters	never equals, never contains, is present, is never present	The whole message

Integration matchers got the full set in v0.11.144; metric selection, log filters and trace alert rules already had the negations and gained is absent beside them. Saved views and message filters, which had no negation at all, got theirs in v0.11.145 under a different vocabulary - and that difference is deliberate.

Two things about those operators behave in ways people reliably get wrong, and the next two sections are those two things.

A negation already matches rows that lack the attribute

An attribute a span does not carry reads as empty, and empty is not equal to anything. So is not and doesn't contain are already true of the rows that have no such attribute - before you do anything about it.

Take fifty spans through one service: thirty carry rpc.service, twenty do not.

Condition	Spans matched
rpc.service is absent	20
rpc.service doesn't contain "te"	40
rpc.service exists and doesn't contain "te"	20

Forty, not twenty: the twenty spans with no rpc.service at all, plus the twenty that carry one which happens not to contain te.

All three are legitimate, and which one you want depends on the question you are asking:

• “Everything that isn’t the te service” - the middle row. Traffic with no rpc.service belongs in that group.
• “The traffic that doesn’t set rpc.service” - the first row. That is what is absent is for.
• “Rows that set it, to something other than te” - the third row. The strict reading is a composition of two conditions in the same rule, not a separate operator.

The condition’s help text says this while you are writing it, so you find out before saving rather than from a count that looks wrong afterwards.

Why “is not” wasn’t just redefined

Making is not mean “has the attribute and differs” would have been the tidier operator - and it would have silently changed the meaning of every alert rule and integration matcher already using it in production. Rules nobody touched would start matching fewer spans, and nothing would have told anyone. Adding is absent alongside, and letting the strict form be composed, leaves existing configuration meaning exactly what it always meant.

In saved views and message filters, a negation is about the whole message

A message is a trace of many steps, and this is where the obvious reading is the wrong one.

Ask “rpc.service is not abc” step by step - the way a positive filter works, matching the message if some step qualifies - and it matches any message with at least one step that lacks the attribute. Root steps rarely carry rpc.service, so that is very nearly every message, for very nearly every negation. The filter would look applied and sift out almost nothing.

So a negated row in a saved view asks about the message as a whole: no step in this message satisfies it.

Take ten messages of three steps each. Five carry rpc.service = abc on two of their steps; the other five carry something else. Every one of the ten has at least one step with no rpc.service at all.

Filter	Messages matched
rpc.service equals abc	5
rpc.service is present	10
rpc.service never equals abc	5
rpc.service is never present	0

Zero, not ten. Under the step-by-step reading both negations would have returned all ten, which is exactly the useless result the whole-message reading exists to avoid.

This is why the filter row reads “never equals” rather than “is not”, and “is never present” rather than “is absent”. The sentence under the filters is the only place you can check what you actually asked for, and it has to make the claim it is really making.

The exclusion only looks at the steps you are already seeing

On a scoped page - one service, or one integration - the negation is evaluated over the steps that page is showing, not over the whole trace. A message never disappears from an integration’s Messages tab because of a step in some other service you weren’t looking at.

Include child spans

A condition matches the spans that carry the attribute. In practice the attribute is on exactly one of them: the step that receives the message sets messaging.destination.name, or tenant.id, or a correlation key, and the steps it then calls never mention it again. Matching the attribute gives you that one step and hides the work it caused.

Include child spans turns the condition into a starting point. The spans that satisfy the condition are the anchors; every span below an anchor in the same trace is included too, whether or not it satisfies the condition.

Take a trace of four spans:

message_received
├─ publish        messaging.destination.name = orders.canonical
│  └─ erp_ingest
└─ audit_log

Only publish carries the attribute. message_received above it and audit_log beside it do not, and erp_ingest sets nothing of its own - it is simply what publish went on to do.

Condition	Spans selected
messaging.destination.name = orders.canonical	publish
the same, with child spans	publish, erp_ingest

publish is the anchor, so erp_ingest joins it. message_received stays out because it is above the anchor, and audit_log stays out because it is beside it. Only what hangs below an anchor comes along - and all of it does: children, grandchildren, and however much deeper the subtree goes.

On an integration

Settings → Services. Open a service’s row with Edit: once it has an attribute condition, a child spans toggle sits beside + condition.

The rule as a whole picks the anchors: the service it names, plus its attribute conditions. Switch the toggle on and the descendants of those anchors join the integration’s slice of traffic.

That slice is what the whole integration is built from, so the setting shows up everywhere at once - message lists, trace and error counts, the flow graph, alerts, and the traffic that users granted the integration are allowed to see. The row says so in its summary - “Only where …, with child spans” - and counts the messages it takes in, so you can see the rule’s reach before you save.

Two boundaries do not move:

• Member services. A child span in a service that is not a member of the integration is still not part of it. Child spans widen the rule, not the membership.
• The time window. Child spans outside the selected window are outside the integration’s view of it, exactly as anchors are.

Logs and metrics

Logs ride along. A log line attached to an included span - matched on trace id and span id - is included with it, on top of the log lines that satisfy the conditions in their own right. That is usually the point: the failure you want is logged by the child step, not by the one carrying the attribute.

Metrics have no span tree to walk, so there is nothing below a data point to include. For metric conditions the checkbox changes nothing at all, and the conditions apply exactly as written.

The setting is part of the matcher, so it travels with it: a config export carries it, an import restores it, and cloning an integration clones it.

In a message view

Messages → the filter editor. A payload (attribute) row with a positive operator - equals, contains, matches, exists - grows a + child spans toggle. Switch it on and the row reads “with child spans”, and the sentence summary under the filters spells it out as “(and its child spans)”.

Negative rows do not offer it, and the reason follows from the section above: never equals, doesn't contain and is absent are claims about the whole message, and they exclude messages rather than pointing at a step. There is no anchor, so there is nothing below it to include.

As soon as one row has the toggle on, the anchor is chosen by everything else you asked for: the other filter rows, and the integration scope if the view is on an integration. The children then follow their anchors, and they count as steps of the match from that point on - a message’s status and its highlighted steps both cover them. That is what makes this combination work:

abc = 123 with child spans, status err only

which finds the messages where abc = 123 and the error happened somewhere below the step that set it. Without the toggle that search returns nothing, because the step carrying abc is fine; the one that broke is its child.

It stays inside what you are allowed to see

A user only ever sees steps in services they have access to, and a step they cannot see never gets to decide which of their steps match. An anchor in a service outside your access does not quietly pull its children into your results.

How an integration’s rules combine

An integration is defined by rules. Each rule pins a service, and can narrow it further with the attribute conditions the rest of this page is about; on the integration’s Services tab, each rule is one row - a service, or a pattern over service names, with the conditions that narrow it. Until v0.11.149 those rules were always a union: a span belonged to the integration if it satisfied any one of them.

The Services tab asks the question under Advanced matching, folded away while it has the usual answer and open whenever it does not:

Any service / One trace through every service

• Any service is the union, and the default. Every integration that existed before this keeps it, so nothing changed for anyone who does not go and change it.
• One trace through every service means a trace belongs to the integration only when every rule is satisfied by some span of that trace. That trace’s matching spans are then the integration’s, exactly as they would be under “Any service”.

The question is asked of the trace

A rule names a service, and one span is in one service, so no single span can ever satisfy two rules that name different services. “Service A where a = 1 and service B where b = 3” is not a question about a span at all.

It is a question about the trace - which is also the unit a message is. So that is what “one trace through every service” asks: that each rule is satisfied somewhere in this trace.

What it is for

Three rules, one per service, two of them narrowed by an attribute:

    (service = erp-adapter)
OR  (service = order-validator  AND http.url = ...)
OR  (service = b2b-gateway      AND edi.interchange_control_ref = ...)

Read as a union, a trace that satisfies the b2b-gateway rule is pulled into the integration whatever its order-validator span says, because that one rule has been satisfied on its own. The condition on the other rule never gets to exclude anything. Under one trace through every service it does, because the trace has to clear all three.

Including the first one: a rule with no attribute conditions still has to be satisfied. In that mode the example above also requires the trace to have an erp-adapter span, where before that rule merely offered another way in.

”Or” inside one service

In this mode, the conditions on one service can only be joined with and for now. The editor shows “or” as unavailable there and says why, and it will not switch an integration into this mode while one of its services joins its conditions with “or”.

The reason is how rules are stored. A service whose conditions are joined with “or” is saved as one group per alternative, and this mode requires every group - so “order-gateway where x = 1 or x = 2” would mean a span with x = 1 and another with x = 2, in the same trace. Until the stored groups record which service they came from (#39), the editor keeps “or” out of this mode rather than save a rule that means something else. An integration already saved that way keeps its current behaviour, and its Services tab flags the affected service.

What the mode reaches

The mode belongs to the integration rather than to a screen, and the integration’s slice of traffic is what everything about it is built from - the message list, trace and error counts, the flow graph, alerts, and the traffic a user granted that integration is allowed to see. They all narrow together.

Logs and metrics

Logs carry a trace id, so they follow the mode like everything else.

Metrics carry no trace, and “one trace through every service” is a claim about a trace. For metrics the conditions apply exactly as written, and the mode changes nothing.

Asking for every rule costs an extra pass over the selected window, since the traces have to be settled before the spans can be. Measured on 1.2M spans with both rules matching very nearly every trace - the worst case for this - a message search went from 0.64s to 1.73s. Rules that select more narrowly pay less than that.

The mode is part of the integration’s configuration, so it travels with it: a config export carries it, an import restores it, and cloning an integration clones it.

Over the API

Both halves of an attribute condition travel over the API: the operator, and whether the condition takes the subtree with it.

Integration matchers carry a boolean include_descendants, on POST /api/v1/integrations, POST /api/v1/integrations/{id}/matchers and PUT /api/v1/integrations/{id}/matchers. It is read per match_group: one matcher in a group setting it is enough for the whole group, so you do not have to repeat it across the matchers that make up a single rule.

Message filters carry a boolean includeDescendants per filter row, on POST /api/v1/messages/search and in the rows of a saved view.

Replacing the whole set. PUT /api/v1/integrations/{id}/matchers takes the complete matcher list, and optionally rule_match, and swaps them in one transaction: every row is validated before anything is written, and the set lands whole or not at all. It is what the Services tab saves with. Leave rule_match out to keep the mode as it is.

The rule mode is not part of a condition but of the integration itself: a rule_match field, either "any" or "all", accepted on POST /api/v1/integrations, on PUT /api/v1/integrations/{id}, and on PUT /api/v1/integrations/{id}/matchers.

The operator names on the wire are the same across the span, metric and log surfaces: eq, neq, contains, not_contains, starts_with, ends_with, matches, gt, gte, lt, lte, and now exists and not_exists. Integration matchers use the longer spellings - equals, not_equals, prefix, suffix, contains, regex, exists, not_exists.

exists and not_exists ask about the key rather than the value, so they carry no value - the UI hides the value input, and the API ignores anything sent in it.

Existing matchers, saved views and alert rules are untouched by all of this and mean exactly what they meant before. Both booleans default to false and rule_match defaults to "any", which is the behaviour everything had until now: a condition selects the spans that carry the attribute and nothing else, and one satisfied rule is enough to put traffic in an integration. There is nothing to migrate.