Sync the Envoy access log proto and leave unmeasurable sidecar internal latencies out

NOTICE

@@ -24,3 +24,10 @@ The Apache Software Foundation (http://www.apache.org/).
 The original idea and initial implementation of the maven-wrapper module is derived 
 from the Gradle Wrapper which was written originally by Hans Dockter and Adam Murdoch.
 Copyright 2007 the original author or authors.
+-----------------------------------------------------------------------
+This product contains proto files from the Envoy Project:
+
+Envoy
+Copyright The Envoy Project Authors
+
+Licensed under Apache License 2.0.  See LICENSE for terms.

dist-material/release-docs/NOTICE

@@ -791,3 +791,16 @@ The above copyright notice and this permission notice shall be included in all c
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
 ===========================================================================
+
+------
+
+===========================================================================
+Envoy Notice
+===========================================================================
+
+Envoy
+Copyright The Envoy Project Authors
+
+Licensed under Apache License 2.0.  See LICENSE for terms.
+
+===========================================================================

docs/en/changes/changes.md

@@ -3,6 +3,7 @@
 #### Project
 * Update the e2e mock sender's OTLP proto to v1.11.1. A current OpenTelemetry Collector adds fields the old copy lacked, such as a metric's `metadata`, and the sender's strict JSON parser refused a recording that carries them. The Windows e2e now also replays such a recording, of windows_exporter `0.31.8` through OpenTelemetry Collector `0.158.0` taken on a GitHub-hosted Windows runner, and asserts its memory metrics.
 * Replace the GenAI e2e cases' Spring AI application with an in-repo `e2e-spring-ai-service` module pinned to the released Spring AI 2.0.1, and move the mock LLM endpoints out of `e2e-service-provider` into a dedicated `e2e-mock-llm-server` module that all three GenAI cases share. The Spring AI application was previously built at test time by cloning `spring-projects/spring-ai-examples` and running Maven inside the image build, which resolved `spring-ai:2.0.0-SNAPSHOT`: a line that has since been abandoned, whose surviving builds all predate the 2.0.0 GA by a day and whose older builds have been pruned, so the fixture was pinned to nothing released and could no longer be rebuilt from the same bytes. Both modules are built by the existing e2e reactor and published to `ghcr.io/apache/skywalking` alongside the other e2e service images; `e2e-spring-ai-service` needs Java 17 and Spring Boot 4, so it is gated behind a new `jdk-17` profile in the e2e reactor and the jobs that build it now pin JDK 17 explicitly; `make -C test docker` refuses up front on an older JDK rather than packaging a missing or stale jar, since `clean` never reaches a module the profile excluded and compose would turn the absent jar into a directory.
+* Add the Envoy project's NOTICE to `NOTICE` and to the binary distribution's NOTICE, for the Envoy proto files SkyWalking bundles in `receiver-proto`.
 
 #### OAP Server
 * Add customizable LLM-as-judge support for AI evaluation, with OpenAI-compatible endpoint / model / API key configuration, and persist the evaluation result as queryable `GenAIEvaluationRecord` rows for later inspection.
@@ -50,7 +51,9 @@
 * Build the dated Elasticsearch index names in one place, `IndexNames`, with the day step decided once per physical table from its models, replacing `TimeSeriesUtils` and `TimeRangeIndexNameGenerator`. Apart from the fixes above, index names are unchanged.
 * Fix a service instance dropping out of the instance list queried with a time range when a record carrying an older minute updates it, such as the segment of a request that started minutes earlier arriving after the instance's heartbeat. `InstanceTraffic` took the `last_ping` of the last record written instead of keeping the latest one, as `EndpointTraffic` now does.
 * Fix eBPF access-log timestamps losing all sub-second precision. `NodeInfo.parseTimestamp` truncated the nanosecond-since-boot offset to whole seconds and also dropped `bootTime.nanos` before converting to millis, so short mesh HTTP latency (`end - start`) was quantized to 0 or 1000ms depending on whether the request crossed a whole-second boundary, and minute buckets could land in the wrong minute near minute boundaries. The conversion is now `Instant.ofEpochSecond(bootSeconds, bootNanos).plusNanos(offset).toEpochMilli()`, the same Instant-plus-nanos style Envoy ALS uses for wall time.
-* Fix the Envoy ALS sidecar internal request and response latencies dropping whole seconds: only the `nanos` part of each protobuf `Duration` was read. The internal response latency is now `0` when the access log lacks the upstream or downstream timing it is computed from, such as when Envoy answers an upstream timeout with its own 504, instead of counting the whole wait or going negative.
+* Fix the Envoy ALS sidecar internal request and response latencies dropping whole seconds: only the `nanos` part of each protobuf `Duration` was read. The internal response latency is now left out of the internal latency metrics when the access log lacks the upstream or downstream timing it is computed from, such as when Envoy answers an upstream timeout with its own 504, instead of counting the whole wait or going negative.
+* Sync the Envoy access log proto, `envoy/data/accesslog/v3/accesslog.proto`, with Envoy v1.40.0, adding the `envoy/annotations/deprecation.proto` it imports. With it, the Envoy ALS sidecar internal latencies leave out what the access log cannot measure instead of counting it: the internal request latency of an HTTP request Envoy retried or hedged, or of a TCP connection that took more than one connect attempt (`upstream_request_attempt_count` above 1), as Envoy measures the upstream timings from the start of the downstream request or connection, so the time to the first upstream byte can include the earlier attempts and the delays between them; and the internal response latency when the response details are set to anything but `via_upstream`, as for Envoy's own 504 of a timeout during a retry backoff, which can carry an earlier attempt's upstream timing, and, conservatively, for an upstream response that a later reset or timeout ended. The adapters report such a latency as `-1`, and the `*_sidecar_internal_req_latency_nanos` and `*_sidecar_internal_resp_latency_nanos` metrics in `mesh.oal` filter it out with `filter(... >= 0)`. A customized `mesh.oal` needs the same filters, or the `-1` values are averaged in.
+* Fix OAL `>`, `>=`, `<` and `<=` filters on a `long` or `double` source field, and `==` on a `long` one, failing to compile when they compare it with a number, such as `filter(sideCar.internalRequestLatencyNanos >= 0)`. The generated code passed the number as an `int`, and the comparison matchers have no `match(long, int)` or `match(double, int)` overload, as Javassist does not widen an argument. The number is now written in the field's type, `0L` or `0.0`; filters on `int` fields, which already worked, generate the same code.
 
 #### UI
 * Add a Virtual GenAI evaluation-record page and evaluation-score chart in Horizon UI, so operators can inspect evaluation result, level, reason, judge model, timestamp, trace linkage, and the `gen_ai_model_evaluation_score_ppm` trend for evaluated records.

docs/en/concepts-and-designs/oal.md

@@ -59,6 +59,8 @@ Supported operators:
 - String matching: `like %...`, `like ...%`, `like %...%` (e.g., `name like "serv%"`)
 - Tag matching: `contain`, `not contain` (e.g., `tags contain "http.method:GET"`) 
 
+With `>`, `>=`, `<` and `<=`, a number is compared in the type of a `long` or `double` field, and with `==` in the type of a `long` field, e.g. `filter(sideCar.internalRequestLatencyNanos >= 0)`.
+
 ## Aggregation Function
 The default functions are provided by the SkyWalking OAP core, and it is possible to implement additional functions. 
 

oap-server/oal-rt/src/main/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricher.java

@@ -116,7 +116,8 @@ public CodeGenModel enrich(MetricDefinition metric) {
         CodeGenModel.SerializeFieldsV2 serializeFields = generateSerializeFields(fieldsFromSource, persistentFields);
 
         // 7. Convert filter expressions to template format
-        List<CodeGenModel.FilterExpressionV2> filterExpressions = convertFilters(metric.getFilters());
+        List<CodeGenModel.FilterExpressionV2> filterExpressions =
+            convertFilters(metric.getFilters(), metric.getSource().getName());
 
         // 8. Build CodeGenModel
         return CodeGenModel.builder()
@@ -267,7 +268,7 @@ private CodeGenModel.EntranceMethodV2 buildEntranceMethod(Method entranceMethod,
                         FilterExpression filterExpr = funcArg.asExpression();
                         String matcherClass = getMatcherClassName(filterExpr);
                         String left = buildFilterLeft(filterExpr);
-                        String right = buildFilterRight(filterExpr);
+                        String right = buildFilterRight(filterExpr, metric.getSource().getName());
                         // Add as FilterExpressionV2 object for template access
                         argsExpressions.add(CodeGenModel.FilterExpressionV2.builder()
                             .expressionObject(matcherClass)
@@ -397,10 +398,10 @@ private String mapExpression(String attribute) {
     /**
      * Build right side of filter expression.
      */
-    private String buildFilterRight(FilterExpression filterExpr) {
+    private String buildFilterRight(FilterExpression filterExpr, String sourceName) {
         FilterValue value = filterExpr.getValue();
         if (value.isNumber()) {
-            return String.valueOf(value.asNumber());
+            return buildNumberLiteral(value.asNumber(), resolveAttributeType(sourceName, filterExpr.getFieldName()));
         } else if (value.isString()) {
             return "\"" + value.asString() + "\"";
         } else if (value.isBoolean()) {
@@ -447,6 +448,53 @@ private String buildFilterRight(FilterExpression filterExpr) {
         throw new IllegalArgumentException("Unknown filter value type: " + value);
     }
 
+    /**
+     * Render a whole number as a literal of the long or double attribute it is compared with. Each comparison matcher
+     * has one overload per type, such as {@code match(long, long)}, and Javassist does not widen an argument, so a bare
+     * {@code 0} against a long attribute would look for {@code match(long, int)}. The literal itself carries the type,
+     * {@code 0L} or {@code 0.0}: Javassist reads the number inside a cast such as {@code (long)(3000000000)} as an int
+     * and overflows it. Any other attribute or number keeps the literal as written.
+     */
+    static String buildNumberLiteral(Number number, Class<?> attributeType) {
+        boolean wholeNumber = number instanceof Long || number instanceof Integer;
+        if (wholeNumber && attributeType == long.class) {
+            return number + "L";
+        }
+        if (wholeNumber && attributeType == double.class) {
+            return number + ".0";
+        }
+        return String.valueOf(number);
+    }
+
+    /**
+     * Resolve the type of the source attribute a filter compares, following nested getters such as
+     * {@code sideCar.internalRequestLatencyNanos}.
+     *
+     * @return the attribute type, or null when it cannot be resolved, as for a map expression.
+     */
+    private Class<?> resolveAttributeType(String sourceName, String fieldName) {
+        if (isMapExpression(fieldName)) {
+            return null;
+        }
+        try {
+            Class<?> type = Class.forName(sourcePackage + sourceName);
+            for (String attribute : fieldName.split("\\.")) {
+                type = findAccessor(type, attribute).getReturnType();
+            }
+            return type;
+        } catch (ClassNotFoundException | NoSuchMethodException e) {
+            return null;
+        }
+    }
+
+    private static Method findAccessor(Class<?> type, String attribute) throws NoSuchMethodException {
+        try {
+            return type.getMethod(ClassMethodUtil.toGetMethod(attribute));
+        } catch (NoSuchMethodException e) {
+            return type.getMethod(ClassMethodUtil.toIsMethod(attribute));
+        }
+    }
+
     /**
      * Map filter operator and value type to matcher expression type.
      *
@@ -575,12 +623,12 @@ private CodeGenModel.SerializeFieldsV2 generateSerializeFields(
     /**
      * Convert filter expressions to template-ready format.
      */
-    private List<CodeGenModel.FilterExpressionV2> convertFilters(List<FilterExpression> filters) {
+    private List<CodeGenModel.FilterExpressionV2> convertFilters(List<FilterExpression> filters, String sourceName) {
         List<CodeGenModel.FilterExpressionV2> result = new ArrayList<>();
         for (FilterExpression filter : filters) {
             String matcherClass = getMatcherClassName(filter);
             String left = buildFilterLeft(filter);
-            String right = buildFilterRight(filter);
+            String right = buildFilterRight(filter, sourceName);
             // Verbatim ANTLR slice from the parser — e.g.
             // "filter(detectPoint == DetectPoint.SERVER)". Used as the
             // captured filter record's sourceText so the UI matches against

oap-server/oal-rt/src/test/java/org/apache/skywalking/oal/v2/generator/MetricDefinitionEnricherTest.java

@@ -18,9 +18,14 @@
 
 package org.apache.skywalking.oal.v2.generator;
 
+import java.lang.reflect.Method;
 import java.util.List;
+import javassist.ClassPool;
+import javassist.CtClass;
+import javassist.CtNewMethod;
 import org.apache.skywalking.oal.v2.model.MetricDefinition;
 import org.apache.skywalking.oal.v2.parser.OALScriptParserV2;
+import org.apache.skywalking.oap.server.core.analysis.metrics.expression.GreaterEqualMatch;
 import org.apache.skywalking.oap.server.core.source.DefaultScopeDefine;
 import org.apache.skywalking.oap.server.core.source.Endpoint;
 import org.apache.skywalking.oap.server.core.source.K8SService;
@@ -387,6 +392,55 @@ public void testNumericComparisonFilter() throws Exception {
         assertEquals("1000", filterExpr.getRight());
     }
 
+    /**
+     * Test a number compared with a long attribute: filter(sideCar.internalRequestLatencyNanos >= 0)
+     * The literal must be a long, or the generated code looks for GreaterEqualMatch#match(long, int).
+     */
+    @Test
+    public void testNumericComparisonFilterOnLongAttribute() throws Exception {
+        String oal = "relation_internal_req_latency = from(ServiceRelation.sideCar.internalRequestLatencyNanos)"
+            + ".filter(sideCar.internalRequestLatencyNanos >= 0).longAvg();";
+
+        MetricDefinitionEnricher enricher = new MetricDefinitionEnricher(SOURCE_PACKAGE, METRICS_PACKAGE);
+        OALScriptParserV2 parser = OALScriptParserV2.parse(oal);
+        MetricDefinition metric = parser.getMetrics().get(0);
+        CodeGenModel model = enricher.enrich(metric);
+
+        CodeGenModel.FilterExpressionV2 filterExpr = model.getFilterExpressions().get(0);
+        assertTrue(filterExpr.getExpressionObject().contains("GreaterEqualMatch"),
+            "Expected GreaterEqualMatch but got: " + filterExpr.getExpressionObject());
+        assertEquals("source.getSideCar().getInternalRequestLatencyNanos()", filterExpr.getLeft());
+        assertEquals("0L", filterExpr.getRight());
+    }
+
+    /**
+     * A number above Integer.MAX_VALUE keeps its value once Javassist compiles the literal. A cast such as
+     * (long)(3000000000) would overflow to -1294967296 and turn the filter around.
+     */
+    @Test
+    public void testNumberLiteralAboveIntRangeKeepsItsValue() throws Exception {
+        String longLiteral = MetricDefinitionEnricher.buildNumberLiteral(3_000_000_000L, long.class);
+        String doubleLiteral = MetricDefinitionEnricher.buildNumberLiteral(3_000_000_000L, double.class);
+        assertEquals("3000000000L", longLiteral);
+        assertEquals("3000000000.0", doubleLiteral);
+
+        String matcher = GreaterEqualMatch.class.getName();
+        ClassPool pool = new ClassPool(true);
+        CtClass probe = pool.makeClass(MetricDefinitionEnricherTest.class.getPackage().getName() + ".NumberLiteralProbe");
+        probe.addMethod(CtNewMethod.make("public static boolean longAtLeast(long value) { return new " + matcher
+            + "().match(value, " + longLiteral + "); }", probe));
+        probe.addMethod(CtNewMethod.make("public static boolean doubleAtLeast(double value) { return new " + matcher
+            + "().match(value, " + doubleLiteral + "); }", probe));
+        Class<?> probeClass = probe.toClass(MetricDefinitionEnricherTest.class);
+
+        Method longAtLeast = probeClass.getMethod("longAtLeast", long.class);
+        assertFalse((Boolean) longAtLeast.invoke(null, 1L));
+        assertTrue((Boolean) longAtLeast.invoke(null, 3_000_000_000L));
+        Method doubleAtLeast = probeClass.getMethod("doubleAtLeast", double.class);
+        assertFalse((Boolean) doubleAtLeast.invoke(null, 1.0));
+        assertTrue((Boolean) doubleAtLeast.invoke(null, 3.0E9));
+    }
+
     /**
      * Test enum comparison: filter(detectPoint == DetectPoint.CLIENT)
      */

oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/als/LogEntry2MetricsAdapter.java

@@ -45,6 +45,14 @@
  */
 @RequiredArgsConstructor
 public class LogEntry2MetricsAdapter {
+    /**
+     * An internal latency the access log cannot measure. The internal latency metrics in {@code mesh.oal} leave it
+     * out with {@code filter(... >= 0)}.
+     */
+    public static final long UNKNOWN_INTERNAL_LATENCY = -1;
+
+    private static final String VIA_UPSTREAM = "via_upstream";
+
     /**
      * The access log entry that is to be adapted into metrics builders.
      */
@@ -104,14 +112,23 @@ public HTTPServiceMeshMetric.Builder adaptCommonPart() {
         final Protocol protocol = requestProtocol(entry.getRequest());
         final String tlsMode = parseTLS(properties.getTlsProperties());
         final String internalErrorCode = parseInternalErrorCode(properties.getResponseFlags());
-        final long internalRequestLatencyNanos = toNanos(properties.getTimeToFirstUpstreamTxByte());
-        // Envoy leaves out a timing that never happened, and an absent Duration reads as zero, so the
-        // difference would be the whole wait or negative.
+        // The upstream timings count from the start of the downstream request, so after a retry or a hedged
+        // attempt the first upstream byte can include the earlier attempts and the backoff between them.
+        final long internalRequestLatencyNanos =
+            properties.hasTimeToFirstUpstreamTxByte() && properties.getUpstreamRequestAttemptCount() <= 1
+                ? toNanos(properties.getTimeToFirstUpstreamTxByte())
+                : UNKNOWN_INTERNAL_LATENCY;
+        // Envoy leaves out a timing that never happened, and an absent Duration reads as zero. Its own replies, such
+        // as the 504 of a timeout during a retry backoff, can carry an earlier attempt's upstream timing, so only a
+        // response whose details say it came via the upstream is measured. A later reset or timeout replaces those
+        // details, so such a response is left out too. An Envoy that reports no details leaves them empty.
+        final String responseCodeDetails = entry.getResponse().getResponseCodeDetails();
         final long internalResponseLatencyNanos =
-            properties.hasTimeToFirstUpstreamRxByte() && properties.hasTimeToFirstDownstreamTxByte()
+            (responseCodeDetails.isEmpty() || VIA_UPSTREAM.equals(responseCodeDetails))
+                && properties.hasTimeToFirstUpstreamRxByte() && properties.hasTimeToFirstDownstreamTxByte()
                 ? toNanos(properties.getTimeToFirstDownstreamTxByte())
                     - toNanos(properties.getTimeToFirstUpstreamRxByte())
-                : 0;
+                : UNKNOWN_INTERNAL_LATENCY;
 
         final HTTPServiceMeshMetric.Builder builder =
             HTTPServiceMeshMetric