Skip to content

run-make-fulldeps/pgo-branch-weights fails on AArch64 Linux #78226

@pietroalbini

Description

@pietroalbini
Member

The most recent builds on AArch64 Linux are failing on this test:

2020-10-22T08:53:26.0713848Z ---- [run-make] run-make-fulldeps/pgo-branch-weights stdout ----
2020-10-22T08:53:26.0723060Z 
2020-10-22T08:53:26.0727694Z error: make failed
2020-10-22T08:53:26.0732974Z status: exit code: 2
2020-10-22T08:53:26.0738071Z command: "make"
2020-10-22T08:53:26.0742831Z stdout:
2020-10-22T08:53:26.0750881Z ------------------------------------------
2020-10-22T08:53:26.0763884Z # We don't compile `opaque` with either optimizations or instrumentation.
2020-10-22T08:53:26.0783737Z LD_LIBRARY_PATH="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights:/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0-bootstrap-tools/aarch64-unknown-linux-gnu/release/deps:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights -L /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights   opaque.rs || exit 1
2020-10-22T08:53:26.0791260Z # Compile the test program with instrumentation
2020-10-22T08:53:26.0794012Z mkdir -p "/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/prof_data_dir" || exit 1
2020-10-22T08:53:26.0803441Z LD_LIBRARY_PATH="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights:/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0-bootstrap-tools/aarch64-unknown-linux-gnu/release/deps:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights -L /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights   interesting.rs \
2020-10-22T08:53:26.0813242Z 	-Cprofile-generate="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/prof_data_dir" -O -Ccodegen-units=1 || exit 1
2020-10-22T08:53:26.0824507Z LD_LIBRARY_PATH="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights:/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0-bootstrap-tools/aarch64-unknown-linux-gnu/release/deps:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights -L /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights   main.rs -Cprofile-generate="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/prof_data_dir" -O || exit 1
2020-10-22T08:53:26.0833472Z # The argument below generates to the expected branch weights
2020-10-22T08:53:26.0841671Z LD_LIBRARY_PATH="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights:/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0-bootstrap-tools/aarch64-unknown-linux-gnu/release/deps:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/lib" /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/main aaaaaaaaaaaa2bbbbbbbbbbbb2bbbbbbbbbbbbbbbbcc || exit 1
2020-10-22T08:53:26.0849409Z "/checkout/obj/build/aarch64-unknown-linux-gnu/llvm/build/bin/llvm-profdata" merge \
2020-10-22T08:53:26.0852410Z 	-o "/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/prof_data_dir/merged.profdata" \
2020-10-22T08:53:26.0855865Z 	"/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/prof_data_dir" || exit 1
2020-10-22T08:53:26.0864853Z LD_LIBRARY_PATH="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights:/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0-bootstrap-tools/aarch64-unknown-linux-gnu/release/deps:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights -L /checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights   interesting.rs \
2020-10-22T08:53:26.0874373Z 	-Cprofile-use="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/prof_data_dir/merged.profdata" -O \
2020-10-22T08:53:26.0876942Z 	-Ccodegen-units=1 --emit=llvm-ir || exit 1
2020-10-22T08:53:26.0880673Z cat "/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make-fulldeps/pgo-branch-weights/pgo-branch-weights/interesting.ll" | "/checkout/obj/build/aarch64-unknown-linux-gnu/llvm/build/bin/FileCheck" filecheck-patterns.txt
2020-10-22T08:53:26.0883188Z 
2020-10-22T08:53:26.0884121Z ------------------------------------------
2020-10-22T08:53:26.0884722Z stderr:
2020-10-22T08:53:26.0885663Z ------------------------------------------
2020-10-22T08:53:26.0887152Z filecheck-patterns.txt:5:8: error: CHECK: expected string not found in input
2020-10-22T08:53:26.0889855Z CHECK: define void @function_called_twice(i32 %c) {{.*}} !prof [[function_called_twice_id:![0-9]+]] {
2020-10-22T08:53:26.0890870Z        ^
2020-10-22T08:53:26.0891548Z <stdin>:1:1: note: scanning from here
2020-10-22T08:53:26.0893037Z ; ModuleID = 'interesting.3a1fbbbh-cgu.0'
2020-10-22T08:53:26.0893984Z ^
2020-10-22T08:53:26.0894732Z <stdin>:7:1: note: possible intended match here
2020-10-22T08:53:26.0895788Z define void @function_called_twice(i32 %c) unnamed_addr #0 {
2020-10-22T08:53:26.0896578Z ^
2020-10-22T08:53:26.0896922Z 
2020-10-22T08:53:26.0897496Z Input file: <stdin>
2020-10-22T08:53:26.0898859Z Check file: filecheck-patterns.txt
2020-10-22T08:53:26.0899576Z 
2020-10-22T08:53:26.0900782Z -dump-input=help explains the following input dump.
2020-10-22T08:53:26.0901455Z 
2020-10-22T08:53:26.0902005Z Input was:
2020-10-22T08:53:26.0902571Z <<<<<<
2020-10-22T08:53:26.0903979Z            1: ; ModuleID = 'interesting.3a1fbbbh-cgu.0'
2020-10-22T08:53:26.0905561Z check:5'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
2020-10-22T08:53:26.0908184Z            2: source_filename = "interesting.3a1fbbbh-cgu.0"
2020-10-22T08:53:26.0909684Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0911280Z            3: target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
2020-10-22T08:53:26.0912831Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0914249Z            4: target triple = "aarch64-unknown-linux-gnu"
2020-10-22T08:53:26.0915583Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0916181Z            5: 
2020-10-22T08:53:26.0917010Z check:5'0     ~
2020-10-22T08:53:26.0918142Z            6: ; Function Attrs: noinline nonlazybind uwtable
2020-10-22T08:53:26.0919605Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0920542Z            7: define void @function_called_twice(i32 %c) unnamed_addr #0 {
2020-10-22T08:53:26.0921934Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0923158Z check:5'1     ?                                                            possible intended match
2020-10-22T08:53:26.0923948Z            8: start:
2020-10-22T08:53:26.0924831Z check:5'0     ~~~~~~
2020-10-22T08:53:26.0925481Z            9:  %0 = icmp eq i32 %c, 50
2020-10-22T08:53:26.0926480Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0927169Z           10:  br i1 %0, label %bb2, label %bb1
2020-10-22T08:53:26.0928608Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0929312Z           11: 
2020-10-22T08:53:26.0930503Z check:5'0     ~
2020-10-22T08:53:26.0931285Z           12: bb1: ; preds = %start
2020-10-22T08:53:26.0932694Z check:5'0     ~~~~~~~~~~~~~~~~~~~~~
2020-10-22T08:53:26.0933375Z            .
2020-10-22T08:53:26.0934086Z            .
2020-10-22T08:53:26.0934834Z            .
2020-10-22T08:53:26.0935442Z >>>>>>
2020-10-22T08:53:26.0936135Z make: *** [Makefile:42: all] Error 1
2020-10-22T08:53:26.0936642Z 
2020-10-22T08:53:26.0937694Z ------------------------------------------
2020-10-22T08:53:26.0938123Z 
2020-10-22T08:53:26.0938461Z 
2020-10-22T08:53:26.0938808Z 
2020-10-22T08:53:26.0939349Z failures:
2020-10-22T08:53:26.0940684Z     [run-make] run-make-fulldeps/pgo-branch-weights
2020-10-22T08:53:26.0941493Z 
2020-10-22T08:53:26.0942824Z test result: FAILED. 211 passed; 1 failed; 7 ignored; 0 measured; 0 filtered out

Activity

added
O-ArmTarget: 32-bit Arm processors (armv6, armv7, thumb...), including 64-bit Arm in AArch32 state
C-bugCategory: This is a bug.
on Oct 22, 2020
pietroalbini

pietroalbini commented on Oct 22, 2020

@pietroalbini
MemberAuthor

@rustbot ping arm

rustbot

rustbot commented on Oct 22, 2020

@rustbot
Collaborator

Hey ARM Group! This bug has been identified as a good "ARM candidate".
In case it's useful, here are some instructions for tackling these sorts of
bugs. Maybe take a look?
Thanks! <3

cc @JamieCunliffe @joaopaulocarreiro @raw-bin @Stammark @vigoux

JohnTitor

JohnTitor commented on Oct 22, 2020

@JohnTitor
Member

Looks like my rollup (#78212) has the cause.

raw-bin

raw-bin commented on Oct 22, 2020

@raw-bin

@JohnTitor: Thanks for the tip! We're looking into this.

JamieCunliffe

JamieCunliffe commented on Oct 22, 2020

@JamieCunliffe
Contributor

Based on that I reverted #77554 (git revert 813066c4429e7582b08dbf3af2c12a5f2e1b2a16 -m 1) and the test then passed.

pietroalbini

pietroalbini commented on Oct 28, 2020

@pietroalbini
MemberAuthor

This seems to be fixed now, CI is passing again.

Mark-Simulacrum

Mark-Simulacrum commented on Jun 20, 2021

@Mark-Simulacrum
Member

Reopening this issue as it's hitting us (again) on the beta promotion PR, #86413.

I've tried various things to debug but so far haven't really arrived at any conclusions. It doesn't seem immediately related to anything in the PR itself, which should in theory have minimal effect on runtime behavior here.

@rustbot ping arm -- since this is only failing on the aarch64 builder, I am currently presuming some relationship, but it's not clear. The test does not fail on the only aarch64 machine I currently readily have access to when I tried to run it indirectly (i.e., copying the input files and simulating the make file via some local manual invocations)>

rustbot

rustbot commented on Jun 20, 2021

@rustbot
Collaborator

Error: Parsing ping command in comment failed: ...'t ping arm' | error: expected end of command at >| ' -- since '...

Please let @rust-lang/release know if you're having trouble with this bot.

Mark-Simulacrum

Mark-Simulacrum commented on Jun 20, 2021

@Mark-Simulacrum
Member

@rustbot ping arm

25 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCategory: This is a bug.O-ArmTarget: 32-bit Arm processors (armv6, armv7, thumb...), including 64-bit Arm in AArch32 stateP-criticalCritical priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-betaPerformance or correctness regression from stable to beta.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @pnkfelix@raw-bin@michaelwoerister@pietroalbini@Mark-Simulacrum

        Issue actions

          run-make-fulldeps/pgo-branch-weights fails on AArch64 Linux · Issue #78226 · rust-lang/rust