diff --git a/NOTICE b/NOTICE index 0fef61473bc2..0493ef0ee243 100644 --- a/NOTICE +++ b/NOTICE @@ -24,3 +24,10 @@ The Apache Software Foundation (http://www.apache.org/). The original idea and initial implementation of the maven-wrapper module is derived from the Gradle Wrapper which was written originally by Hans Dockter and Adam Murdoch. Copyright 2007 the original author or authors. +----------------------------------------------------------------------- +This product contains proto files from the Envoy Project: + +Envoy +Copyright The Envoy Project Authors + +Licensed under Apache License 2.0. See LICENSE for terms. diff --git a/dist-material/release-docs/NOTICE b/dist-material/release-docs/NOTICE index 90fdf3a8b552..0eb9145afe10 100755 --- a/dist-material/release-docs/NOTICE +++ b/dist-material/release-docs/NOTICE @@ -791,3 +791,16 @@ The above copyright notice and this permission notice shall be included in all c THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. =========================================================================== + +------ + +=========================================================================== +Envoy Notice +=========================================================================== + +Envoy +Copyright The Envoy Project Authors + +Licensed under Apache License 2.0. See LICENSE for terms. + +=========================================================================== diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index 4f6e81d7e186..9a32edf3ae3a 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -3,6 +3,7 @@ #### Project * Update the e2e mock sender's OTLP proto to v1.11.1. A current OpenTelemetry Collector adds fields the old copy lacked, such as a metric's `metadata`, and the sender's strict JSON parser refused a recording that carries them. The Windows e2e now also replays such a recording, of windows_exporter `0.31.8` through OpenTelemetry Collector `0.158.0` taken on a GitHub-hosted Windows runner, and asserts its memory metrics. * Replace the GenAI e2e cases' Spring AI application with an in-repo `e2e-spring-ai-service` module pinned to the released Spring AI 2.0.1, and move the mock LLM endpoints out of `e2e-service-provider` into a dedicated `e2e-mock-llm-server` module that all three GenAI cases share. The Spring AI application was previously built at test time by cloning `spring-projects/spring-ai-examples` and running Maven inside the image build, which resolved `spring-ai:2.0.0-SNAPSHOT`: a line that has since been abandoned, whose surviving builds all predate the 2.0.0 GA by a day and whose older builds have been pruned, so the fixture was pinned to nothing released and could no longer be rebuilt from the same bytes. Both modules are built by the existing e2e reactor and published to `ghcr.io/apache/skywalking` alongside the other e2e service images; `e2e-spring-ai-service` needs Java 17 and Spring Boot 4, so it is gated behind a new `jdk-17` profile in the e2e reactor and the jobs that build it now pin JDK 17 explicitly; `make -C test docker` refuses up front on an older JDK rather than packaging a missing or stale jar, since `clean` never reaches a module the profile excluded and compose would turn the absent jar into a directory. +* Add the Envoy project's NOTICE to `NOTICE` and to the binary distribution's NOTICE, for the Envoy proto files SkyWalking bundles in `receiver-proto`. #### OAP Server * Add customizable LLM-as-judge support for AI evaluation, with OpenAI-compatible endpoint / model / API key configuration, and persist the evaluation result as queryable `GenAIEvaluationRecord` rows for later inspection. @@ -50,7 +51,9 @@ * Build the dated Elasticsearch index names in one place, `IndexNames`, with the day step decided once per physical table from its models, replacing `TimeSeriesUtils` and `TimeRangeIndexNameGenerator`. Apart from the fixes above, index names are unchanged. * Fix a service instance dropping out of the instance list queried with a time range when a record carrying an older minute updates it, such as the segment of a request that started minutes earlier arriving after the instance's heartbeat. `InstanceTraffic` took the `last_ping` of the last record written instead of keeping the latest one, as `EndpointTraffic` now does. * Fix eBPF access-log timestamps losing all sub-second precision. `NodeInfo.parseTimestamp` truncated the nanosecond-since-boot offset to whole seconds and also dropped `bootTime.nanos` before converting to millis, so short mesh HTTP latency (`end - start`) was quantized to 0 or 1000ms depending on whether the request crossed a whole-second boundary, and minute buckets could land in the wrong minute near minute boundaries. The conversion is now `Instant.ofEpochSecond(bootSeconds, bootNanos).plusNanos(offset).toEpochMilli()`, the same Instant-plus-nanos style Envoy ALS uses for wall time. -* Fix the Envoy ALS sidecar internal request and response latencies dropping whole seconds: only the `nanos` part of each protobuf `Duration` was read. The internal response latency is now `0` when the access log lacks the upstream or downstream timing it is computed from, such as when Envoy answers an upstream timeout with its own 504, instead of counting the whole wait or going negative. +* Fix the Envoy ALS sidecar internal request and response latencies dropping whole seconds: only the `nanos` part of each protobuf `Duration` was read. The internal response latency is now left out of the internal latency metrics when the access log lacks the upstream or downstream timing it is computed from, such as when Envoy answers an upstream timeout with its own 504, instead of counting the whole wait or going negative. +* Sync the Envoy access log proto, `envoy/data/accesslog/v3/accesslog.proto`, with Envoy v1.40.0, adding the `envoy/annotations/deprecation.proto` it imports. With it, the Envoy ALS sidecar internal latencies leave out what the access log cannot measure instead of counting it: the internal request latency of an HTTP request Envoy retried or hedged, or of a TCP connection that took more than one connect attempt (`upstream_request_attempt_count` above 1), as Envoy measures the upstream timings from the start of the downstream request or connection, so the time to the first upstream byte can include the earlier attempts and the delays between them; and the internal response latency when the response details are set to anything but `via_upstream`, as for Envoy's own 504 of a timeout during a retry backoff, which can carry an earlier attempt's upstream timing, and, conservatively, for an upstream response that a later reset or timeout ended. The adapters report such a latency as `-1`, and the `*_sidecar_internal_req_latency_nanos` and `*_sidecar_internal_resp_latency_nanos` metrics in `mesh.oal` filter it out with `filter(... >= 0)`. A customized `mesh.oal` needs the same filters, or the `-1` values are averaged in. +* Fix OAL `>`, `>=`, `<` and `<=` filters on a `long` or `double` source field, and `==` on a `long` one, failing to compile when they compare it with a number, such as `filter(sideCar.internalRequestLatencyNanos >= 0)`. The generated code passed the number as an `int`, and the comparison matchers have no `match(long, int)` or `match(double, int)` overload, as Javassist does not widen an argument. The number is now written in the field's type, `0L` or `0.0`; filters on `int` fields, which already worked, generate the same code. #### UI * Add a Virtual GenAI evaluation-record page and evaluation-score chart in Horizon UI, so operators can inspect evaluation result, level, reason, judge model, timestamp, trace linkage, and the `gen_ai_model_evaluation_score_ppm` trend for evaluated records. diff --git a/docs/en/concepts-and-designs/oal.md b/docs/en/concepts-and-designs/oal.md index e0becc4dbfe9..887da30d5f9b 100644 --- a/docs/en/concepts-and-designs/oal.md +++ b/docs/en/concepts-and-designs/oal.md @@ -59,6 +59,8 @@ Supported operators: - String matching: `like %...`, `like ...%`, `like %...%` (e.g., `name like "serv%"`) - Tag matching: `contain`, `not contain` (e.g., `tags contain "http.method:GET"`) +With `>`, `>=`, `<` and `<=`, a number is compared in the type of a `long` or `double` field, and with `==` in the type of a `long` field, e.g. `filter(sideCar.internalRequestLatencyNanos >= 0)`. + ## Aggregation Function The default functions are provided by the SkyWalking OAP core, and it is possible to implement additional functions. diff --git a/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricher.java b/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricher.java index d427f662f6a1..ef387f49c12a 100644 --- a/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricher.java +++ b/oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricher.java @@ -116,7 +116,8 @@ public CodeGenModel enrich(MetricDefinition metric) { CodeGenModel.SerializeFieldsV2 serializeFields = generateSerializeFields(fieldsFromSource, persistentFields); // 7. Convert filter expressions to template format - List filterExpressions = convertFilters(metric.getFilters()); + List filterExpressions = + convertFilters(metric.getFilters(), metric.getSource().getName()); // 8. Build CodeGenModel return CodeGenModel.builder() @@ -267,7 +268,7 @@ private CodeGenModel.EntranceMethodV2 buildEntranceMethod(Method entranceMethod, FilterExpression filterExpr = funcArg.asExpression(); String matcherClass = getMatcherClassName(filterExpr); String left = buildFilterLeft(filterExpr); - String right = buildFilterRight(filterExpr); + String right = buildFilterRight(filterExpr, metric.getSource().getName()); // Add as FilterExpressionV2 object for template access argsExpressions.add(CodeGenModel.FilterExpressionV2.builder() .expressionObject(matcherClass) @@ -397,10 +398,10 @@ private String mapExpression(String attribute) { /** * Build right side of filter expression. */ - private String buildFilterRight(FilterExpression filterExpr) { + private String buildFilterRight(FilterExpression filterExpr, String sourceName) { FilterValue value = filterExpr.getValue(); if (value.isNumber()) { - return String.valueOf(value.asNumber()); + return buildNumberLiteral(value.asNumber(), resolveAttributeType(sourceName, filterExpr.getFieldName())); } else if (value.isString()) { return "\"" + value.asString() + "\""; } else if (value.isBoolean()) { @@ -447,6 +448,53 @@ private String buildFilterRight(FilterExpression filterExpr) { throw new IllegalArgumentException("Unknown filter value type: " + value); } + /** + * Render a whole number as a literal of the long or double attribute it is compared with. Each comparison matcher + * has one overload per type, such as {@code match(long, long)}, and Javassist does not widen an argument, so a bare + * {@code 0} against a long attribute would look for {@code match(long, int)}. The literal itself carries the type, + * {@code 0L} or {@code 0.0}: Javassist reads the number inside a cast such as {@code (long)(3000000000)} as an int + * and overflows it. Any other attribute or number keeps the literal as written. + */ + static String buildNumberLiteral(Number number, Class attributeType) { + boolean wholeNumber = number instanceof Long || number instanceof Integer; + if (wholeNumber && attributeType == long.class) { + return number + "L"; + } + if (wholeNumber && attributeType == double.class) { + return number + ".0"; + } + return String.valueOf(number); + } + + /** + * Resolve the type of the source attribute a filter compares, following nested getters such as + * {@code sideCar.internalRequestLatencyNanos}. + * + * @return the attribute type, or null when it cannot be resolved, as for a map expression. + */ + private Class resolveAttributeType(String sourceName, String fieldName) { + if (isMapExpression(fieldName)) { + return null; + } + try { + Class type = Class.forName(sourcePackage + sourceName); + for (String attribute : fieldName.split("\\.")) { + type = findAccessor(type, attribute).getReturnType(); + } + return type; + } catch (ClassNotFoundException | NoSuchMethodException e) { + return null; + } + } + + private static Method findAccessor(Class type, String attribute) throws NoSuchMethodException { + try { + return type.getMethod(ClassMethodUtil.toGetMethod(attribute)); + } catch (NoSuchMethodException e) { + return type.getMethod(ClassMethodUtil.toIsMethod(attribute)); + } + } + /** * Map filter operator and value type to matcher expression type. * @@ -575,12 +623,12 @@ private CodeGenModel.SerializeFieldsV2 generateSerializeFields( /** * Convert filter expressions to template-ready format. */ - private List convertFilters(List filters) { + private List convertFilters(List filters, String sourceName) { List result = new ArrayList<>(); for (FilterExpression filter : filters) { String matcherClass = getMatcherClassName(filter); String left = buildFilterLeft(filter); - String right = buildFilterRight(filter); + String right = buildFilterRight(filter, sourceName); // Verbatim ANTLR slice from the parser — e.g. // "filter(detectPoint == DetectPoint.SERVER)". Used as the // captured filter record's sourceText so the UI matches against diff --git a/oap-server/oal-rt/src/test/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricherTest.java b/oap-server/oal-rt/src/test/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricherTest.java index b79a6ecc00fe..4e24836964a9 100644 --- a/oap-server/oal-rt/src/test/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricherTest.java +++ b/oap-server/oal-rt/src/test/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricherTest.java @@ -18,9 +18,14 @@ package org.apache.skywalking.oal.v2.generator; +import java.lang.reflect.Method; import java.util.List; +import javassist.ClassPool; +import javassist.CtClass; +import javassist.CtNewMethod; import org.apache.skywalking.oal.v2.model.MetricDefinition; import org.apache.skywalking.oal.v2.parser.OALScriptParserV2; +import org.apache.skywalking.oap.server.core.analysis.metrics.expression.GreaterEqualMatch; import org.apache.skywalking.oap.server.core.source.DefaultScopeDefine; import org.apache.skywalking.oap.server.core.source.Endpoint; import org.apache.skywalking.oap.server.core.source.K8SService; @@ -387,6 +392,55 @@ public void testNumericComparisonFilter() throws Exception { assertEquals("1000", filterExpr.getRight()); } + /** + * Test a number compared with a long attribute: filter(sideCar.internalRequestLatencyNanos >= 0) + * The literal must be a long, or the generated code looks for GreaterEqualMatch#match(long, int). + */ + @Test + public void testNumericComparisonFilterOnLongAttribute() throws Exception { + String oal = "relation_internal_req_latency = from(ServiceRelation.sideCar.internalRequestLatencyNanos)" + + ".filter(sideCar.internalRequestLatencyNanos >= 0).longAvg();"; + + MetricDefinitionEnricher enricher = new MetricDefinitionEnricher(SOURCE_PACKAGE, METRICS_PACKAGE); + OALScriptParserV2 parser = OALScriptParserV2.parse(oal); + MetricDefinition metric = parser.getMetrics().get(0); + CodeGenModel model = enricher.enrich(metric); + + CodeGenModel.FilterExpressionV2 filterExpr = model.getFilterExpressions().get(0); + assertTrue(filterExpr.getExpressionObject().contains("GreaterEqualMatch"), + "Expected GreaterEqualMatch but got: " + filterExpr.getExpressionObject()); + assertEquals("source.getSideCar().getInternalRequestLatencyNanos()", filterExpr.getLeft()); + assertEquals("0L", filterExpr.getRight()); + } + + /** + * A number above Integer.MAX_VALUE keeps its value once Javassist compiles the literal. A cast such as + * (long)(3000000000) would overflow to -1294967296 and turn the filter around. + */ + @Test + public void testNumberLiteralAboveIntRangeKeepsItsValue() throws Exception { + String longLiteral = MetricDefinitionEnricher.buildNumberLiteral(3_000_000_000L, long.class); + String doubleLiteral = MetricDefinitionEnricher.buildNumberLiteral(3_000_000_000L, double.class); + assertEquals("3000000000L", longLiteral); + assertEquals("3000000000.0", doubleLiteral); + + String matcher = GreaterEqualMatch.class.getName(); + ClassPool pool = new ClassPool(true); + CtClass probe = pool.makeClass(MetricDefinitionEnricherTest.class.getPackage().getName() + ".NumberLiteralProbe"); + probe.addMethod(CtNewMethod.make("public static boolean longAtLeast(long value) { return new " + matcher + + "().match(value, " + longLiteral + "); }", probe)); + probe.addMethod(CtNewMethod.make("public static boolean doubleAtLeast(double value) { return new " + matcher + + "().match(value, " + doubleLiteral + "); }", probe)); + Class probeClass = probe.toClass(MetricDefinitionEnricherTest.class); + + Method longAtLeast = probeClass.getMethod("longAtLeast", long.class); + assertFalse((Boolean) longAtLeast.invoke(null, 1L)); + assertTrue((Boolean) longAtLeast.invoke(null, 3_000_000_000L)); + Method doubleAtLeast = probeClass.getMethod("doubleAtLeast", double.class); + assertFalse((Boolean) doubleAtLeast.invoke(null, 1.0)); + assertTrue((Boolean) doubleAtLeast.invoke(null, 3.0E9)); + } + /** * Test enum comparison: filter(detectPoint == DetectPoint.CLIENT) */ diff --git a/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapter.java b/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapter.java index 0644b3cf32dd..1faf4d018eff 100644 --- a/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapter.java +++ b/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapter.java @@ -45,6 +45,14 @@ */ @RequiredArgsConstructor public class LogEntry2MetricsAdapter { + /** + * An internal latency the access log cannot measure. The internal latency metrics in {@code mesh.oal} leave it + * out with {@code filter(... >= 0)}. + */ + public static final long UNKNOWN_INTERNAL_LATENCY = -1; + + private static final String VIA_UPSTREAM = "via_upstream"; + /** * The access log entry that is to be adapted into metrics builders. */ @@ -104,14 +112,23 @@ public HTTPServiceMeshMetric.Builder adaptCommonPart() { final Protocol protocol = requestProtocol(entry.getRequest()); final String tlsMode = parseTLS(properties.getTlsProperties()); final String internalErrorCode = parseInternalErrorCode(properties.getResponseFlags()); - final long internalRequestLatencyNanos = toNanos(properties.getTimeToFirstUpstreamTxByte()); - // Envoy leaves out a timing that never happened, and an absent Duration reads as zero, so the - // difference would be the whole wait or negative. + // The upstream timings count from the start of the downstream request, so after a retry or a hedged + // attempt the first upstream byte can include the earlier attempts and the backoff between them. + final long internalRequestLatencyNanos = + properties.hasTimeToFirstUpstreamTxByte() && properties.getUpstreamRequestAttemptCount() <= 1 + ? toNanos(properties.getTimeToFirstUpstreamTxByte()) + : UNKNOWN_INTERNAL_LATENCY; + // Envoy leaves out a timing that never happened, and an absent Duration reads as zero. Its own replies, such + // as the 504 of a timeout during a retry backoff, can carry an earlier attempt's upstream timing, so only a + // response whose details say it came via the upstream is measured. A later reset or timeout replaces those + // details, so such a response is left out too. An Envoy that reports no details leaves them empty. + final String responseCodeDetails = entry.getResponse().getResponseCodeDetails(); final long internalResponseLatencyNanos = - properties.hasTimeToFirstUpstreamRxByte() && properties.hasTimeToFirstDownstreamTxByte() + (responseCodeDetails.isEmpty() || VIA_UPSTREAM.equals(responseCodeDetails)) + && properties.hasTimeToFirstUpstreamRxByte() && properties.hasTimeToFirstDownstreamTxByte() ? toNanos(properties.getTimeToFirstDownstreamTxByte()) - toNanos(properties.getTimeToFirstUpstreamRxByte()) - : 0; + : UNKNOWN_INTERNAL_LATENCY; final HTTPServiceMeshMetric.Builder builder = HTTPServiceMeshMetric diff --git a/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/tcp/TCPLogEntry2MetricsAdapter.java b/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/tcp/TCPLogEntry2MetricsAdapter.java index 0dd5465ebb2e..0588ffdb257e 100644 --- a/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/tcp/TCPLogEntry2MetricsAdapter.java +++ b/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/tcp/TCPLogEntry2MetricsAdapter.java @@ -29,6 +29,7 @@ import org.apache.skywalking.apm.network.servicemesh.v3.TCPServiceMeshMetric; import org.apache.skywalking.oap.server.receiver.envoy.als.ServiceMetaInfo; +import static org.apache.skywalking.oap.server.receiver.envoy.als.LogEntry2MetricsAdapter.UNKNOWN_INTERNAL_LATENCY; import static org.apache.skywalking.oap.server.receiver.envoy.als.LogEntry2MetricsAdapter.formatAsLong; import static org.apache.skywalking.oap.server.receiver.envoy.als.LogEntry2MetricsAdapter.parseInternalErrorCode; import static org.apache.skywalking.oap.server.receiver.envoy.als.LogEntry2MetricsAdapter.parseTLS; @@ -87,13 +88,17 @@ public TCPServiceMeshMetric.Builder adaptCommonPart() { final ConnectionProperties connectionProperties = entry.getConnectionProperties(); final String tlsMode = parseTLS(properties.getTlsProperties()); final String internalErrorCode = parseInternalErrorCode(properties.getResponseFlags()); - final long internalRequestLatencyNanos = toNanos(properties.getTimeToFirstUpstreamTxByte()); - // An absent timing reads as zero, see LogEntry2MetricsAdapter#adaptCommonPart. + // The attempt count here is the TCP proxy's connect attempts. See LogEntry2MetricsAdapter#adaptCommonPart + // for why a retried request's internal request latency is unknown, and why an absent timing reads as zero. + final long internalRequestLatencyNanos = + properties.hasTimeToFirstUpstreamTxByte() && properties.getUpstreamRequestAttemptCount() <= 1 + ? toNanos(properties.getTimeToFirstUpstreamTxByte()) + : UNKNOWN_INTERNAL_LATENCY; final long internalResponseLatencyNanos = properties.hasTimeToFirstUpstreamRxByte() && properties.hasTimeToLastDownstreamTxByte() ? toNanos(properties.getTimeToLastDownstreamTxByte()) - toNanos(properties.getTimeToFirstUpstreamRxByte()) - : 0; + : UNKNOWN_INTERNAL_LATENCY; final TCPServiceMeshMetric.Builder builder = TCPServiceMeshMetric diff --git a/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapterTest.java b/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapterTest.java index 30f6dd107b12..bfa6dbd33351 100644 --- a/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapterTest.java +++ b/oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapterTest.java @@ -22,12 +22,14 @@ import com.google.protobuf.Timestamp; import io.envoyproxy.envoy.data.accesslog.v3.AccessLogCommon; import io.envoyproxy.envoy.data.accesslog.v3.HTTPAccessLogEntry; +import io.envoyproxy.envoy.data.accesslog.v3.HTTPResponseProperties; import io.envoyproxy.envoy.data.accesslog.v3.TCPAccessLogEntry; import org.apache.skywalking.apm.network.servicemesh.v3.HTTPServiceMeshMetric; import org.apache.skywalking.apm.network.servicemesh.v3.TCPServiceMeshMetric; import org.apache.skywalking.oap.server.receiver.envoy.als.tcp.TCPLogEntry2MetricsAdapter; import org.junit.jupiter.api.Test; +import static org.apache.skywalking.oap.server.receiver.envoy.als.LogEntry2MetricsAdapter.UNKNOWN_INTERNAL_LATENCY; import static org.junit.jupiter.api.Assertions.assertEquals; public class LogEntry2MetricsAdapterTest { @@ -57,6 +59,44 @@ public class LogEntry2MetricsAdapterTest { .setTimeToFirstUpstreamRxByte(Duration.newBuilder().setSeconds(2).setNanos(900_000_000)) .build(); + // A retry: the first attempt got a 503 after 3s, and the second went upstream at 3.029s, got its first response + // byte at 3.5s, and the first byte was sent back at 3.6s. + private static final AccessLogCommon RETRIED = AccessLogCommon.newBuilder() + .setStartTime(Timestamp.newBuilder().setSeconds(1700000000)) + .setUpstreamRequestAttemptCount(2) + .setTimeToFirstUpstreamTxByte(Duration.newBuilder().setSeconds(3).setNanos(29_000_000)) + .setTimeToFirstUpstreamRxByte(Duration.newBuilder().setSeconds(3).setNanos(500_000_000)) + .setTimeToFirstDownstreamTxByte(Duration.newBuilder().setSeconds(3).setNanos(600_000_000)) + .setTimeToLastDownstreamTxByte(Duration.newBuilder().setSeconds(3).setNanos(600_000_000)) + .build(); + + // A TCP proxy tunneling over HTTP CONNECT that connected on its second attempt. The TCP proxy logs no downstream + // byte timings. + private static final AccessLogCommon RETRIED_CONNECT = AccessLogCommon.newBuilder() + .setStartTime(Timestamp.newBuilder().setSeconds(1700000000)) + .setUpstreamRequestAttemptCount(2) + .setTimeToFirstUpstreamTxByte(Duration.newBuilder().setSeconds(3).setNanos(29_000_000)) + .setTimeToFirstUpstreamRxByte(Duration.newBuilder().setSeconds(3).setNanos(500_000_000)) + .build(); + + // The global timeout fired during the backoff after the first attempt's 503 at 3s, before the retry started, so the + // entry keeps that attempt's timings and the count of 1, and Envoy sent its own 504 at 3.5s. + private static final AccessLogCommon TIMEOUT_IN_RETRY_BACKOFF = AccessLogCommon.newBuilder() + .setStartTime(Timestamp.newBuilder().setSeconds(1700000000)) + .setUpstreamRequestAttemptCount(1) + .setTimeToFirstUpstreamTxByte(Duration.newBuilder().setNanos(2_000_000)) + .setTimeToFirstUpstreamRxByte(Duration.newBuilder().setSeconds(3)) + .setTimeToFirstDownstreamTxByte(Duration.newBuilder().setSeconds(3).setNanos(500_000_000)) + .setTimeToLastDownstreamTxByte(Duration.newBuilder().setSeconds(3).setNanos(500_000_000)) + .build(); + + // A filter answered with its own reply at 1ms, so nothing went upstream. + private static final AccessLogCommon NO_UPSTREAM = AccessLogCommon.newBuilder() + .setStartTime(Timestamp.newBuilder().setSeconds(1700000000)) + .setTimeToFirstDownstreamTxByte(Duration.newBuilder().setNanos(1_000_000)) + .setTimeToLastDownstreamTxByte(Duration.newBuilder().setNanos(1_000_000)) + .build(); + @Test public void testHTTPInternalLatencyCountsWholeSeconds() { final HTTPAccessLogEntry entry = HTTPAccessLogEntry.newBuilder().setCommonProperties(COMMON).build(); @@ -74,7 +114,7 @@ public void testHTTPInternalResponseLatencyWithoutUpstreamResponse() { new LogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); assertEquals(2_000_000L, metrics.getInternalRequestLatencyNanos()); - assertEquals(0L, metrics.getInternalResponseLatencyNanos()); + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalResponseLatencyNanos()); } @Test @@ -84,7 +124,63 @@ public void testHTTPInternalResponseLatencyWithoutDownstreamResponse() { new LogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); assertEquals(1_500_000_000L, metrics.getInternalRequestLatencyNanos()); - assertEquals(0L, metrics.getInternalResponseLatencyNanos()); + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalResponseLatencyNanos()); + } + + @Test + public void testHTTPInternalLatencyWithOneAttempt() { + final AccessLogCommon oneAttempt = COMMON.toBuilder().setUpstreamRequestAttemptCount(1).build(); + final HTTPAccessLogEntry entry = HTTPAccessLogEntry.newBuilder().setCommonProperties(oneAttempt).build(); + final HTTPServiceMeshMetric.Builder metrics = + new LogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); + + assertEquals(1_500_000_000L, metrics.getInternalRequestLatencyNanos()); + assertEquals(200_000_000L, metrics.getInternalResponseLatencyNanos()); + } + + @Test + public void testHTTPInternalRequestLatencyAfterRetry() { + final HTTPAccessLogEntry entry = HTTPAccessLogEntry.newBuilder().setCommonProperties(RETRIED).build(); + final HTTPServiceMeshMetric.Builder metrics = + new LogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); + + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalRequestLatencyNanos()); + assertEquals(100_000_000L, metrics.getInternalResponseLatencyNanos()); + } + + @Test + public void testHTTPInternalResponseLatencyViaUpstream() { + final HTTPAccessLogEntry entry = HTTPAccessLogEntry.newBuilder() + .setCommonProperties(COMMON) + .setResponse(HTTPResponseProperties.newBuilder().setResponseCodeDetails("via_upstream")) + .build(); + final HTTPServiceMeshMetric.Builder metrics = + new LogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); + + assertEquals(200_000_000L, metrics.getInternalResponseLatencyNanos()); + } + + @Test + public void testHTTPInternalResponseLatencyOfEnvoyReply() { + final HTTPAccessLogEntry entry = HTTPAccessLogEntry.newBuilder() + .setCommonProperties(TIMEOUT_IN_RETRY_BACKOFF) + .setResponse(HTTPResponseProperties.newBuilder().setResponseCodeDetails("response_timeout")) + .build(); + final HTTPServiceMeshMetric.Builder metrics = + new LogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); + + assertEquals(2_000_000L, metrics.getInternalRequestLatencyNanos()); + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalResponseLatencyNanos()); + } + + @Test + public void testHTTPInternalLatencyWithoutUpstreamRequest() { + final HTTPAccessLogEntry entry = HTTPAccessLogEntry.newBuilder().setCommonProperties(NO_UPSTREAM).build(); + final HTTPServiceMeshMetric.Builder metrics = + new LogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); + + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalRequestLatencyNanos()); + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalResponseLatencyNanos()); } @Test @@ -104,6 +200,26 @@ public void testTCPInternalResponseLatencyWithoutDownstreamResponse() { new TCPLogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); assertEquals(1_500_000_000L, metrics.getInternalRequestLatencyNanos()); - assertEquals(0L, metrics.getInternalResponseLatencyNanos()); + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalResponseLatencyNanos()); + } + + @Test + public void testTCPInternalRequestLatencyAfterRetriedConnect() { + final TCPAccessLogEntry entry = TCPAccessLogEntry.newBuilder().setCommonProperties(RETRIED_CONNECT).build(); + final TCPServiceMeshMetric.Builder metrics = + new TCPLogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); + + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalRequestLatencyNanos()); + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalResponseLatencyNanos()); + } + + @Test + public void testTCPInternalLatencyWithoutUpstreamTimings() { + final TCPAccessLogEntry entry = TCPAccessLogEntry.newBuilder().setCommonProperties(NO_UPSTREAM).build(); + final TCPServiceMeshMetric.Builder metrics = + new TCPLogEntry2MetricsAdapter(entry, null, null).adaptToDownstreamMetrics(); + + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalRequestLatencyNanos()); + assertEquals(UNKNOWN_INTERNAL_LATENCY, metrics.getInternalResponseLatencyNanos()); } } diff --git a/oap-server/server-receiver-plugin/receiver-proto/src/main/proto/envoy/annotations/deprecation.proto b/oap-server/server-receiver-plugin/receiver-proto/src/main/proto/envoy/annotations/deprecation.proto new file mode 100644 index 000000000000..c9a96f1ae27d --- /dev/null +++ b/oap-server/server-receiver-plugin/receiver-proto/src/main/proto/envoy/annotations/deprecation.proto @@ -0,0 +1,34 @@ +syntax = "proto3"; + +package envoy.annotations; +option go_package = "github.com/envoyproxy/go-control-plane/envoy/annotations"; + +import "google/protobuf/descriptor.proto"; + +// [#protodoc-title: Deprecation] +// Adds annotations for deprecated fields and enums to allow tagging proto +// fields as fatal by default and the minor version on which the field was +// deprecated. One Envoy release after deprecation, deprecated fields will be +// disallowed by default, a state which is reversible with +// :ref:`runtime overrides `. + +// Magic number in this file derived from top 28bit of SHA256 digest of +// "envoy.annotation.disallowed_by_default" and "envoy.annotation.deprecated_at_minor_version" +extend google.protobuf.FieldOptions { + bool disallowed_by_default = 189503207; + + // The API major and minor version on which the field was deprecated + // (e.g., "3.5" for major version 3 and minor version 5). + string deprecated_at_minor_version = 157299826; +} + +// Magic number in this file derived from top 28bit of SHA256 digest of +// "envoy.annotation.disallowed_by_default_enum" and +// "envoy.annotation.deprecated_at_minor_version_eum" +extend google.protobuf.EnumValueOptions { + bool disallowed_by_default_enum = 70100853; + + // The API major and minor version on which the enum value was deprecated + // (e.g., "3.5" for major version 3 and minor version 5). + string deprecated_at_minor_version_enum = 181198657; +} diff --git a/oap-server/server-receiver-plugin/receiver-proto/src/main/proto/envoy/data/accesslog/v3/accesslog.proto b/oap-server/server-receiver-plugin/receiver-proto/src/main/proto/envoy/data/accesslog/v3/accesslog.proto index a10dfb607323..6f476c6ed643 100644 --- a/oap-server/server-receiver-plugin/receiver-proto/src/main/proto/envoy/data/accesslog/v3/accesslog.proto +++ b/oap-server/server-receiver-plugin/receiver-proto/src/main/proto/envoy/data/accesslog/v3/accesslog.proto @@ -10,6 +10,7 @@ import "google/protobuf/duration.proto"; import "google/protobuf/timestamp.proto"; import "google/protobuf/wrappers.proto"; +import "envoy/annotations/deprecation.proto"; import "udpa/annotations/status.proto"; import "udpa/annotations/versioning.proto"; import "validate/validate.proto"; @@ -31,6 +32,24 @@ option (udpa.annotations.file_status).package_version_status = ACTIVE; // Fields describing *upstream* interaction will explicitly include ``upstream`` // in their name. +enum AccessLogType { + NotSet = 0; + TcpUpstreamConnected = 1; + TcpPeriodic = 2; + TcpConnectionStart = 14; + TcpConnectionEnd = 3; + DownstreamStart = 4; + DownstreamPeriodic = 5; + DownstreamEnd = 6; + UpstreamPoolReady = 7; + UpstreamPeriodic = 8; + UpstreamEnd = 9; + DownstreamTunnelSuccessfullyEstablished = 10; + UdpTunnelUpstreamConnected = 11; + UdpPeriodic = 12; + UdpSessionEnd = 13; +} + message TCPAccessLogEntry { option (udpa.annotations.versioning).previous_message_type = "envoy.data.accesslog.v2.TCPAccessLogEntry"; @@ -80,7 +99,7 @@ message ConnectionProperties { } // Defines fields that are shared by all Envoy access logs. -// [#next-free-field: 23] +// [#next-free-field: 34] message AccessLogCommon { option (udpa.annotations.versioning).previous_message_type = "envoy.data.accesslog.v2.AccessLogCommon"; @@ -91,14 +110,16 @@ message AccessLogCommon { double sample_rate = 1 [(validate.rules).double = {lte: 1.0 gt: 0.0}]; // This field is the remote/origin address on which the request from the user was received. - // Note: This may not be the physical peer. E.g, if the remote address is inferred from for - // example the x-forwarder-for header, proxy protocol, etc. + // + // .. note:: + // This may not be the actual peer address. For example, it might be derived from headers like ``x-forwarded-for``, + // the proxy protocol, or similar sources. config.core.v3.Address downstream_remote_address = 2; // This field is the local/destination address on which the request from the user was received. config.core.v3.Address downstream_local_address = 3; - // If the connection is secure,S this field will contain TLS properties. + // If the connection is secure, this field will contain TLS properties. TLSProperties tls_properties = 4; // The time that Envoy started servicing this request. This is effectively the time that the first @@ -110,14 +131,14 @@ message AccessLogCommon { google.protobuf.Duration time_to_last_rx_byte = 6; // Interval between the first downstream byte received and the first upstream byte sent. There may - // by considerable delta between *time_to_last_rx_byte* and this value due to filters. - // Additionally, the same caveats apply as documented in *time_to_last_downstream_tx_byte* about + // be considerable delta between ``time_to_last_rx_byte`` and this value due to filters. + // Additionally, the same caveats apply as documented in ``time_to_last_downstream_tx_byte`` about // not accounting for kernel socket buffer time, etc. google.protobuf.Duration time_to_first_upstream_tx_byte = 7; // Interval between the first downstream byte received and the last upstream byte sent. There may - // by considerable delta between *time_to_last_rx_byte* and this value due to filters. - // Additionally, the same caveats apply as documented in *time_to_last_downstream_tx_byte* about + // by considerable delta between ``time_to_last_rx_byte`` and this value due to filters. + // Additionally, the same caveats apply as documented in ``time_to_last_downstream_tx_byte`` about // not accounting for kernel socket buffer time, etc. google.protobuf.Duration time_to_last_upstream_tx_byte = 8; @@ -130,14 +151,14 @@ message AccessLogCommon { google.protobuf.Duration time_to_last_upstream_rx_byte = 10; // Interval between the first downstream byte received and the first downstream byte sent. - // There may be a considerable delta between the *time_to_first_upstream_rx_byte* and this field + // There may be a considerable delta between the ``time_to_first_upstream_rx_byte`` and this field // due to filters. Additionally, the same caveats apply as documented in - // *time_to_last_downstream_tx_byte* about not accounting for kernel socket buffer time, etc. + // ``time_to_last_downstream_tx_byte`` about not accounting for kernel socket buffer time, etc. google.protobuf.Duration time_to_first_downstream_tx_byte = 11; // Interval between the first downstream byte received and the last downstream byte sent. // Depending on protocol, buffering, windowing, filters, etc. there may be a considerable delta - // between *time_to_last_upstream_rx_byte* and this field. Note also that this is an approximate + // between ``time_to_last_upstream_rx_byte`` and this field. Note also that this is an approximate // time. In the current implementation it does not include kernel socket buffer time. In the // current implementation it also does not include send window buffering inside the HTTP/2 codec. // In the future it is likely that work will be done to make this duration more accurate. @@ -150,7 +171,7 @@ message AccessLogCommon { // The upstream local/origin address that handles this exchange. This does not include retries. config.core.v3.Address upstream_local_address = 14; - // The upstream cluster that *upstream_remote_address* belongs to. + // The upstream cluster that ``upstream_remote_address`` belongs to. string upstream_cluster = 15; // Flags indicating occurrences during request/response processing. @@ -169,7 +190,7 @@ message AccessLogCommon { // If upstream connection failed due to transport socket (e.g. TLS handshake), provides the // failure reason from the transport socket. The format of this field depends on the configured // upstream transport socket. Common TLS failures are in - // :ref:`TLS trouble shooting `. + // :ref:`TLS troubleshooting `. string upstream_transport_failure_reason = 18; // The name of the route @@ -181,18 +202,80 @@ message AccessLogCommon { config.core.v3.Address downstream_direct_remote_address = 20; // Map of filter state in stream info that have been configured to be logged. If the filter - // state serialized to any message other than `google.protobuf.Any` it will be packed into - // `google.protobuf.Any`. + // state serialized to any message other than ``google.protobuf.Any`` it will be packed into + // ``google.protobuf.Any``. map filter_state_objects = 21; // A list of custom tags, which annotate logs with additional information. - // To configure this value, users should configure + // To configure this value, see the documentation for // :ref:`custom_tags `. map custom_tags = 22; + + // For HTTP: Total duration in milliseconds of the request from the start time to the last byte out. + // For TCP: Total duration in milliseconds of the downstream connection. + // This is the total duration of the request (i.e., when the request's ActiveStream is destroyed) + // and may be longer than ``time_to_last_downstream_tx_byte``. + google.protobuf.Duration duration = 23; + + // For HTTP: Number of times the request is attempted upstream. Note that the field is omitted when the request was never attempted upstream. + // For TCP: Number of times the connection request is attempted upstream. Note that the field is omitted when the connect request was never attempted upstream. + uint32 upstream_request_attempt_count = 24; + + // Connection termination details may provide additional information about why the connection was terminated by Envoy for L4 reasons. + string connection_termination_details = 25; + + // Optional unique id of stream (TCP connection, long-live HTTP2 stream, HTTP request) for logging and tracing. + // This could be any format string that could be used to identify one stream. + string stream_id = 26; + + // Indicates whether this log entry is the final entry (flushed after the stream completed) or an intermediate entry + // (flushed periodically during the stream). + // + // For long-lived streams (e.g., TCP connections or long-lived HTTP/2 streams), there may be multiple intermediate + // entries and only one final entry. + // + // If needed, a unique identifier (see :ref:`stream_id `) + // can be used to correlate all intermediate and final log entries for the same stream. + // + // .. attention:: + // + // This field is deprecated in favor of ``access_log_type``, which provides a clearer indication of the log entry + // type. + bool intermediate_log_entry = 27 + [deprecated = true, (envoy.annotations.deprecated_at_minor_version) = "3.0"]; + + // If downstream connection in listener failed due to transport socket (e.g. TLS handshake), provides the + // failure reason from the transport socket. The format of this field depends on the configured downstream + // transport socket. Common TLS failures are in :ref:`TLS troubleshooting `. + string downstream_transport_failure_reason = 28; + + // For HTTP: Total number of bytes sent to the downstream by the http stream. + // For TCP: Total number of bytes sent to the downstream by the :ref:`TCP Proxy `. + uint64 downstream_wire_bytes_sent = 29; + + // For HTTP: Total number of bytes received from the downstream by the http stream. Envoy over counts sizes of received HTTP/1.1 pipelined requests by adding up bytes of requests in the pipeline to the one currently being processed. + // For TCP: Total number of bytes received from the downstream by the :ref:`TCP Proxy `. + uint64 downstream_wire_bytes_received = 30; + + // For HTTP: Total number of bytes sent to the upstream by the http stream. This value accumulates during upstream retries. + // For TCP: Total number of bytes sent to the upstream by the :ref:`TCP Proxy `. + uint64 upstream_wire_bytes_sent = 31; + + // For HTTP: Total number of bytes received from the upstream by the http stream. + // For TCP: Total number of bytes sent to the upstream by the :ref:`TCP Proxy `. + uint64 upstream_wire_bytes_received = 32; + + // The type of the access log, which indicates when the log was recorded. + // See :ref:`ACCESS_LOG_TYPE ` for the available values. + // In case the access log was recorded by a flow which does not correspond to one of the supported + // values, then the default value will be ``NotSet``. + // For more information about how access log behaves and when it is being recorded, + // please refer to :ref:`access logging `. + AccessLogType access_log_type = 33; } // Flags indicating occurrences during request/response processing. -// [#next-free-field: 27] +// [#next-free-field: 29] message ResponseFlags { option (udpa.annotations.versioning).previous_message_type = "envoy.data.accesslog.v2.ResponseFlags"; @@ -218,7 +301,7 @@ message ResponseFlags { // Indicates there was no healthy upstream. bool no_healthy_upstream = 2; - // Indicates an there was an upstream request timeout. + // Indicates there was an upstream request timeout. bool upstream_request_timeout = 3; // Indicates local codec level reset was sent on the stream. @@ -279,7 +362,7 @@ message ResponseFlags { // Indicates that a filter configuration is not available. bool no_filter_config_found = 22; - // Indicates that request or connection exceeded the downstream connection duration. + // Indicates that the request or connection exceeded the downstream connection duration. bool duration_timeout = 23; // Indicates there was an HTTP protocol error in the upstream response. @@ -290,10 +373,16 @@ message ResponseFlags { // Indicates overload manager terminated the request. bool overload_manager = 26; + + // Indicates a DNS resolution failed. + bool dns_resolution_failure = 27; + + // Indicates a downstream remote codec level reset was received on the stream + bool downstream_remote_reset = 28; } // Properties of a negotiated TLS connection. -// [#next-free-field: 7] +// [#next-free-field: 8] message TLSProperties { option (udpa.annotations.versioning).previous_message_type = "envoy.data.accesslog.v2.TLSProperties"; @@ -327,6 +416,9 @@ message TLSProperties { // The subject field of the certificate. string subject = 2; + + // The issuer field of the certificate. + string issuer = 3; } // Version of TLS that was negotiated. @@ -350,9 +442,12 @@ message TLSProperties { // The TLS session ID. string tls_session_id = 6; + + // The ``JA3`` fingerprint when ``JA3`` fingerprinting is enabled. + string ja3_fingerprint = 7; } -// [#next-free-field: 14] +// [#next-free-field: 16] message HTTPRequestProperties { option (udpa.annotations.versioning).previous_message_type = "envoy.data.accesslog.v2.HTTPRequestProperties"; @@ -389,7 +484,7 @@ message HTTPRequestProperties { // do not already have a request ID. string request_id = 9; - // Value of the ``X-Envoy-Original-Path`` request header. + // Value of the ``x-envoy-original-path`` request header. string original_path = 10; // Size of the HTTP request headers in bytes. @@ -406,9 +501,17 @@ message HTTPRequestProperties { // Map of additional headers that have been configured to be logged. map request_headers = 13; + + // Number of header bytes sent to the upstream by the http stream, including protocol overhead. + // + // This value accumulates during upstream retries. + uint64 upstream_header_bytes_sent = 14; + + // Number of header bytes received from the downstream by the http stream, including protocol overhead. + uint64 downstream_header_bytes_received = 15; } -// [#next-free-field: 7] +// [#next-free-field: 9] message HTTPResponseProperties { option (udpa.annotations.versioning).previous_message_type = "envoy.data.accesslog.v2.HTTPResponseProperties"; @@ -419,7 +522,7 @@ message HTTPResponseProperties { // Size of the HTTP response headers in bytes. // // This value is captured from the OSI layer 7 perspective, i.e. it does not - // include overhead from framing or encoding at other networking layers. + // include protocol overhead or overhead from framing or encoding at other networking layers. uint64 response_headers_bytes = 2; // Size of the HTTP response body in bytes. @@ -436,4 +539,10 @@ message HTTPResponseProperties { // The HTTP response code details. string response_code_details = 6; + + // Number of header bytes received from the upstream by the http stream, including protocol overhead. + uint64 upstream_header_bytes_received = 7; + + // Number of header bytes sent to the downstream by the http stream, including protocol overhead. + uint64 downstream_header_bytes_sent = 8; } diff --git a/oap-server/server-starter/src/main/resources/oal/mesh.oal b/oap-server/server-starter/src/main/resources/oal/mesh.oal index d4bd9bac29e6..6dcfb58f130d 100755 --- a/oap-server/server-starter/src/main/resources/oal/mesh.oal +++ b/oap-server/server-starter/src/main/resources/oal/mesh.oal @@ -16,21 +16,22 @@ * */ -service_sidecar_internal_req_latency_nanos = from(Service.sideCar.internalRequestLatencyNanos).longAvg(); -service_sidecar_internal_resp_latency_nanos = from(Service.sideCar.internalResponseLatencyNanos).longAvg(); +// A negative internal latency is one the access log could not measure, see LogEntry2MetricsAdapter#UNKNOWN_INTERNAL_LATENCY. +service_sidecar_internal_req_latency_nanos = from(Service.sideCar.internalRequestLatencyNanos).filter(sideCar.internalRequestLatencyNanos >= 0).longAvg(); +service_sidecar_internal_resp_latency_nanos = from(Service.sideCar.internalResponseLatencyNanos).filter(sideCar.internalResponseLatencyNanos >= 0).longAvg(); -service_instance_sidecar_internal_req_latency_nanos = from(ServiceInstance.sideCar.internalRequestLatencyNanos).longAvg(); -service_instance_sidecar_internal_resp_latency_nanos = from(ServiceInstance.sideCar.internalResponseLatencyNanos).longAvg(); +service_instance_sidecar_internal_req_latency_nanos = from(ServiceInstance.sideCar.internalRequestLatencyNanos).filter(sideCar.internalRequestLatencyNanos >= 0).longAvg(); +service_instance_sidecar_internal_resp_latency_nanos = from(ServiceInstance.sideCar.internalResponseLatencyNanos).filter(sideCar.internalResponseLatencyNanos >= 0).longAvg(); -endpoint_sidecar_internal_req_latency_nanos = from(Endpoint.sideCar.internalRequestLatencyNanos).longAvg(); -endpoint_sidecar_internal_resp_latency_nanos = from(Endpoint.sideCar.internalResponseLatencyNanos).longAvg(); +endpoint_sidecar_internal_req_latency_nanos = from(Endpoint.sideCar.internalRequestLatencyNanos).filter(sideCar.internalRequestLatencyNanos >= 0).longAvg(); +endpoint_sidecar_internal_resp_latency_nanos = from(Endpoint.sideCar.internalResponseLatencyNanos).filter(sideCar.internalResponseLatencyNanos >= 0).longAvg(); -service_client_sidecar_internal_req_latency_nanos = from(ServiceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).longAvg(); -service_server_sidecar_internal_req_latency_nanos = from(ServiceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.SERVER).longAvg(); -service_client_sidecar_internal_resp_latency_nanos = from(ServiceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).longAvg(); -service_server_sidecar_internal_resp_latency_nanos = from(ServiceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.SERVER).longAvg(); +service_client_sidecar_internal_req_latency_nanos = from(ServiceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).filter(sideCar.internalRequestLatencyNanos >= 0).longAvg(); +service_server_sidecar_internal_req_latency_nanos = from(ServiceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.SERVER).filter(sideCar.internalRequestLatencyNanos >= 0).longAvg(); +service_client_sidecar_internal_resp_latency_nanos = from(ServiceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).filter(sideCar.internalResponseLatencyNanos >= 0).longAvg(); +service_server_sidecar_internal_resp_latency_nanos = from(ServiceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.SERVER).filter(sideCar.internalResponseLatencyNanos >= 0).longAvg(); -instance_client_sidecar_internal_req_latency_nanos = from(ServiceInstanceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).longAvg(); -instance_server_sidecar_internal_req_latency_nanos = from(ServiceInstanceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.SERVER).longAvg(); -instance_client_sidecar_internal_resp_latency_nanos = from(ServiceInstanceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).longAvg(); -instance_server_sidecar_internal_resp_latency_nanos = from(ServiceInstanceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.SERVER).longAvg(); +instance_client_sidecar_internal_req_latency_nanos = from(ServiceInstanceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).filter(sideCar.internalRequestLatencyNanos >= 0).longAvg(); +instance_server_sidecar_internal_req_latency_nanos = from(ServiceInstanceRelation.sideCar.internalRequestLatencyNanos).filter(detectPoint == DetectPoint.SERVER).filter(sideCar.internalRequestLatencyNanos >= 0).longAvg(); +instance_client_sidecar_internal_resp_latency_nanos = from(ServiceInstanceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.CLIENT).filter(sideCar.internalResponseLatencyNanos >= 0).longAvg(); +instance_server_sidecar_internal_resp_latency_nanos = from(ServiceInstanceRelation.sideCar.internalResponseLatencyNanos).filter(detectPoint == DetectPoint.SERVER).filter(sideCar.internalResponseLatencyNanos >= 0).longAvg();