Cadvisor for ARM #1236

mboussaa · 2016-04-21T10:18:45Z

I guess the current version of Cadvisor doest not support RPI docker images. I am not sure about that since I did not test it yet.
Is it possible to provide support for ARM images?

vishh · 2016-04-21T17:15:22Z

I wonder if cross compilation for arm just works.. cc @luxas

luxas · 2016-04-21T17:48:18Z

I would just add GOARCH=arm to go build command in build/build.sh.
It should just work.

However, if we want to cross-build a docker image, we probably have to base it on debian for other arches than amd64. See the hyperkube image how to do this: https://github.com/kubernetes/kubernetes/blob/master/cluster/images/hyperkube/Makefile

If we decide on a naming convention for non-amd64 cAdvisor and think we should do it, I may send a PR that builds cAdvisor images for arm, arm64, ppc64le.

vishh · 2016-04-21T17:54:36Z

It might help to automate the docker image release process in cadvisor to
ensure that arm images are built for every release.

On Thu, Apr 21, 2016 at 10:48 AM, Lucas Käldström [email protected]
wrote:

I would just add GOARCH=arm to go build command in build/build.sh.
It should just work.

However, if we want to cross-build a docker image, we probably have to
base it on debian for other arches than amd64. See the hyperkube image
how to do this:
https://github.com/kubernetes/kubernetes/blob/master/cluster/images/hyperkube/Makefile

If we decide on a naming convention for non-amd64 cAdvisor and think we
should do it, I may send a PR that builds cAdvisor images for arm, arm64,
ppc64le.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1236 (comment)

luxas · 2016-04-21T17:55:23Z

I may do it if you want it

vishh · 2016-04-21T18:37:05Z

@luxas: That will be awesome!!

On Thu, Apr 21, 2016 at 10:55 AM, Lucas Käldström [email protected]
wrote:

I may do it if you want it

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1236 (comment)

mboussaa · 2016-04-22T08:04:49Z

So for instance cAdvisor image does not handle ARM image?

luxas · 2016-04-22T19:56:39Z

@mboussaa I'm not really sure what you mean.
On an ARM host, cAdvisor built for ARM may monitor ARM containers without problems.

The current cadvisor image on docker hub is built for linux/amd64 (like the most other things), so it won't run on an ARM host at all

solidnerd · 2016-09-25T13:15:42Z

@luxas
I tried to cross compile with Docker 4 Mac and a golang:1.6.3 container.

docker run -it --rm golang:1.6.3
apt-get update && apt-get install vim
go get -d github.com/google/cadvisor
cd $GOPATH/src/github.com/google/cadvisor
vim build/build.sh and add env GOARCH=arm GOBIN=$PWD go "$GO_CMD" ${GO_FLAGS} -ldflags "${ldflags}" "${repo_path}"
make build

I get the following error:

vendor/github.com/opencontainers/runc/libcontainer/cgroups/systemd/apply_systemd.go:17:2: no buildable Go source files in /go/src/github.com/google/cadvisor/vendor/github.com/coreos/go-systemd/util
Makefile:35: recipe for target 'build' failed

Do you know what I'm doing wrong ?

fgardt · 2017-01-01T17:28:37Z

I compiled cadvisor and managed to edit the dockerfile to work on my raspberry pi.
I uploaded the result here.
I decided to compile it my self because any other versions weren't up to date :D

rjlee · 2017-01-01T23:51:53Z

@Brain-Gamer could you provide the dockerfile/instructions so others can do the same ?

fgardt · 2017-01-02T01:00:28Z

@rjlee sure! It takes maybe a few days but I will do it ;)

fgardt · 2017-01-02T13:24:33Z

@rjlee I forked this repo and added the information you asked for in the readme. If you need anything else just ask ;)

rjlee · 2017-01-02T13:29:50Z

@Brain-Gamer many thanks for sharing.

vaibhavsood · 2017-10-04T10:58:58Z

Hi, want to check if the cadvisor docker image can be considered for being extended to be multi-arch on docker hub? (as per above comments right now its for intel/amd64 arch)

The process for doing so would be along the lines as per here: https://github.com/docker-library/official-images#multiple-architectures

My interest here is to ultimately see a cadvisor docker image under https://hub.docker.com/u/ppc64le/

However since cadvisor is not a official Docker image, the first step i guess would be to make it official https://docs.docker.com/docker-hub/official_repos/#how-do-i-create-a-new-official-repository and then move to making the dockerfiles multi-arch (this process is as per Docker)

Want to check if this makes sense/any comments?

whatever4711 · 2018-03-21T10:25:37Z

Hi,

currently we tried to port cAdvisor to a multiarchitecture Docker Image. The source repository is unibaktr/docker-cadvisor.

Therefore, we restructured the Dockerfile to use a slim Debian based image, since all binaries, cAdvisor uses, are rarely available for Alpine based multiarchitecture images.

Feel free to try it out
docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --volume=/dev/disk/:/dev/disk:ro --publish=8080:8080 --detach=true --name=cadvisor unibaktr/cadvisor

Perhaps we can find a way to integrate our work it into the official cAdvisor Repo.

Best regards

ZCube · 2019-10-27T13:57:45Z

The images released above are outdated. I created a cadvisor v0.34.0 docker images for my Raspberry Pi. I have used docker buildx which is still experimental. It supports multi architecture(kubernetes/kubernetes#73365) (arm, arm64, 386, x86_64). I would like to have a look if someone is interested.

https://github.com/ZCube/cadvisor-docker
https://hub.docker.com/r/zcube/cadvisor

rhuss · 2020-01-24T11:27:35Z

FYI, I managed to build and push an ARM image with this multistage Docker build & buildkit:

FROM golang:1.13-buster AS builder
ARG VERSION

RUN apt-get update \
 && apt-get install make git bash gcc \
 && mkdir -p $GOPATH/src/github.com/google \
 && git clone https://github.com/google/cadvisor.git $GOPATH/src/github.com/google/cadvisor

WORKDIR $GOPATH/src/github.com/google/cadvisor
RUN git fetch --tags \
 && git checkout $VERSION \
 && make build \
 && cp ./cadvisor /

# ------------------------------------------
# Copied over from deploy/Dockerfile except that the "zfs" dependency has been removed 
# a its not available fro Alpine on ARM
FROM alpine:3.10
MAINTAINER [email protected] [email protected] [email protected] [email protected] [email protected]

RUN apk --no-cache add libc6-compat device-mapper findutils && \
    apk --no-cache add thin-provisioning-tools --repository http://dl-3.alpinelinux.org/alpine/edge/main/ && \
    echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf && \
    rm -rf /var/cache/apk/*

# Grab cadvisor from the staging directory.
COPY --from=builder /cadvisor /usr/bin/cadvisor

EXPOSE 8080

HEALTHCHECK --interval=30s --timeout=3s \
  CMD wget --quiet --tries=1 --spider http://localhost:8080/healthz || exit 1

ENTRYPOINT ["/usr/bin/cadvisor", "-logtostderr"]

and

# Prepare
docker buildx create --name xbuilder
docker buildx use xbuilder

# Cross build
VERSION=v0.35.0
docker buildx build --build-arg VERSION=${VERSION} --platform linux/arm/v7,linux/amd64 --push -t rhuss/cadvisor:${VERSION} .

rhuss · 2020-01-24T11:30:22Z

Ah, just saw its more or less the same as in https://github.com/ZCube/cadvisor-docker

(not sure why apk add zfs didn't work for me, though)

rhuss · 2020-01-24T11:35:32Z

(not sure why apk add zfs didn't work for me, though)

well, it doesn't work over there for arm, too :)

whatever4711 · 2020-04-03T11:38:56Z

The images released above are outdated. I created a cadvisor v0.34.0 docker images for my Raspberry Pi. I have used docker buildx which is still experimental. It supports multi architecture(kubernetes/kubernetes#73365) (arm, arm64, 386, x86_64). I would like to have a look if someone is interested.

https://github.com/ZCube/cadvisor-docker
https://hub.docker.com/r/zcube/cadvisor

I updated our images to use buildx with CircleCI, so feel free to use unibaktr/cadvisor again ;)

muzammil360 · 2020-04-09T12:20:19Z

I think it would be great if we can have (non-docker) binaries for arm64 in addition to amd in assets. Right now, I am cloning the repo and building it using make build. But it doesn't seem like a clean approach. Definitely would prefer something that I can just download using wget and unzip. Prometheus and all Prometheus related monitoring applications do that.

alex-voigt · 2020-06-01T09:45:12Z

still no ARM support? :-/

sabretus · 2020-06-30T14:29:16Z

ARM is not only RPI but also for example m6g on Amazon. Why it's so hard to build for ARM too?

thaJeztah · 2021-01-04T11:53:07Z

Perhaps the release script could be updated to build a multi-arch image using buildx;

cadvisor/build/release.sh

Lines 43 to 46 in 6fd4cd9

    
           # Build the docker image 
        
           echo ">> building cadvisor docker image" 
        
           gcr_tag="gcr.io/cadvisor/cadvisor:$VERSION" 
        
           docker build -t $gcr_tag -f deploy/Dockerfile .

iwankgb · 2021-01-04T18:18:53Z

We could probably use goreleaser to build downloadable binaries and container images for any platform that we want. I don't think that any official ARM images make sense before #2744 is merged: it seems that majority of reported issues is related to RPi.
@sabretus I will make sure to test these changes against Graviton 2 and any other server ARM platform I will be able to get hold of :)

muzammil360 · 2021-01-07T16:26:49Z

As a workaround i have been building cadvisor from src for arm64. But of course this is problematic since it needs to be run again every time cicd job runs on gitlab.

…

On Mon, Jan 4, 2021, 11:19 PM iwankgb ***@***.***> wrote: We could probably use goreleaser <https://github.com/goreleaser/goreleaser> to build downloadable binaries and container images for any platform that we want. I don't think that any official ARM images make sense before #2744 <#2744> is merged: it seems that majority of reported issues is related to RPi. @sabretus <https://github.com/sabretus> I will make sure to test these changes against Graviton 2 and any other server ARM platform I will be able to get hold of :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1236 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE3S5IJHVBP2YG6S3EGMLS3SYIBB5ANCNFSM4CBSE6VA> .

bahamut657 · 2021-05-01T23:22:41Z

@muzammil360 can you share the Dockerfile for arm64? How much time and resources are needed to build?

muzammil360 · 2021-05-01T23:48:56Z

@bahamut657 Building arm64 is trivial. I can share the basic code

git clone https://github.com/google/cadvisor.git 
cd cadvisor
bash ./build/assets.sh 
bash ./build/build.sh

Regarding the dockerfile, I don't have it right now. But putting above code and making one should be again trivial. Just use some golang image as base.

Shoot me for more questions.

muzammil360 · 2021-05-01T23:50:31Z

@bahamut657 just as I was looking, I found a dockerfile online. May be this will work faster for you.

https://github.com/Budry/cadvisor-arm

muzammil360 · 2021-05-01T23:55:15Z

@bobbypage @haircommander I am open to discussing adding arm64 release for cadvisor. But would need some guidance for that.

iwankgb · 2021-05-07T16:47:46Z

It's something that I want to address in #2835.

logopk · 2021-07-12T09:21:03Z

I used the buildx approach of @rhuss.

That works fine until v0.39.0. Starting from there I get exec errors. Like this for v0.40.0

cadvisor        | panic: unaligned 64-bit atomic operation
cadvisor        |
cadvisor        | goroutine 72 [running]:
cadvisor        | runtime/internal/atomic.panicUnaligned()
cadvisor        |       /usr/local/go/src/runtime/internal/atomic/unaligned.go:8 +0x24
cadvisor        | runtime/internal/atomic.Load64(0x31f336c, 0x72355c80, 0xc0331fbf)
cadvisor        |       /usr/local/go/src/runtime/internal/atomic/asm_arm.s:263 +0x14
cadvisor        | github.com/google/cadvisor/manager.(*containerData).updateStats(0x31f3200, 0x71d9d10f, 0xc0331fbf)
cadvisor        |       /src/cadvisor/manager/container.go:675 +0x84
cadvisor        | github.com/google/cadvisor/manager.(*containerData).housekeepingTick(0x31f3200, 0x332a040, 0x5f5e100, 0x0, 0x34fd800)
cadvisor        |       /src/cadvisor/manager/container.go:586 +0xe0
cadvisor        | github.com/google/cadvisor/manager.(*containerData).housekeeping(0x31f3200)
cadvisor        |       /src/cadvisor/manager/container.go:534 +0x200
cadvisor        | created by github.com/google/cadvisor/manager.(*containerData).Start
cadvisor        |       /src/cadvisor/manager/container.go:122 +0x2c

logopk · 2021-08-31T13:05:45Z

cadvisor | panic: unaligned 64-bit atomic operation

https://twitter.com/fatih/status/1371391996003295234?lang=de

golang/go#36606

introduced by #1837 Expose OOM event count to prometheus
fa07332

stats.OOMEvents = atomic.LoadUint64(&cd.oomEvents)

deathdefyer · 2022-01-07T22:19:31Z

+1 looking to build this on apple silicon

logopk · 2022-01-08T07:56:03Z

I created my own Dockerfile to build.
after the clone of the repo I remove the failing stats.OOMEvents-Statement and the import like this:

RUN sed -i 's#"sync/atomic"##g'  /src/cadvisor/manager/container.go
RUN sed -i "s/ atomic.LoadUint64(&cd.oomEvents)/0/g"  /src/cadvisor/manager/container.go

After this it's running fine in v0.40 - v0.43

martadinata666 · 2022-01-08T08:00:55Z

it already fixed in git master, just no release version yet. #3029 , me too still waiting release v0.4x with this fix

logopk · 2022-01-08T08:50:59Z

I don't understand the fix 🤯. could anyone explain why it matters that the uint64 comes first in the struct? or am I missing sth. obvious???

daveshow · 2022-04-16T19:58:06Z

Maybe an old issue but this might help if you want to build the image yourself. Thanks, @rhuss for the initial dockerfile. Made some modifications to the build process but if you have your own registry this should work fine.

https://gist.github.com/daveshow/e990656bba8451d85673640cac605e2c

thaJeztah · 2022-04-19T08:40:09Z

I don't understand the fix 🤯. could anyone explain why it matters that the uint64 comes first in the struct? or am I missing sth. obvious???

Yes, these can be confusing; search for "type alignment"; e.g., see https://go.dev/ref/spec#Size_and_alignment_guarantees and https://go101.org/article/memory-layout.html#alignment-guarantee

Logicer16 · 2022-10-20T12:16:55Z

Pretty sure this was fixed in #3141. Is there a reason this is still open?

bobbypage · 2022-10-20T21:34:15Z

Yes, it is done, I think we can close this.

luxas mentioned this issue Apr 22, 2016

Compile cAdvisor statically #1237

Closed

dashpole added help wanted kind/enhancement labels Mar 16, 2017

mkumatag mentioned this issue Feb 28, 2019

Build google/cadvisor:latest for multi architecture kubernetes/kubernetes#73365

Closed

nightah mentioned this issue Jun 19, 2020

Run on RPI? stefanprodan/dockprom#185

Closed

fuomag9 mentioned this issue Jan 3, 2021

cAdvisor dies with exec format error #2564

Open

bobbypage closed this as completed Oct 20, 2022

Cadvisor for ARM #1236

Cadvisor for ARM #1236

Comments

mboussaa commented Apr 21, 2016

vishh commented Apr 21, 2016

luxas commented Apr 21, 2016

vishh commented Apr 21, 2016

luxas commented Apr 21, 2016

vishh commented Apr 21, 2016

mboussaa commented Apr 22, 2016

luxas commented Apr 22, 2016

solidnerd commented Sep 25, 2016

fgardt commented Jan 1, 2017

rjlee commented Jan 1, 2017

fgardt commented Jan 2, 2017

fgardt commented Jan 2, 2017

rjlee commented Jan 2, 2017

vaibhavsood commented Oct 4, 2017

whatever4711 commented Mar 21, 2018

ZCube commented Oct 27, 2019

rhuss commented Jan 24, 2020

rhuss commented Jan 24, 2020

rhuss commented Jan 24, 2020

whatever4711 commented Apr 3, 2020

muzammil360 commented Apr 9, 2020

alex-voigt commented Jun 1, 2020

sabretus commented Jun 30, 2020

thaJeztah commented Jan 4, 2021

iwankgb commented Jan 4, 2021

muzammil360 commented Jan 7, 2021 via email

bahamut657 commented May 1, 2021 • edited Loading

muzammil360 commented May 1, 2021

muzammil360 commented May 1, 2021

muzammil360 commented May 1, 2021

iwankgb commented May 7, 2021

logopk commented Jul 12, 2021

logopk commented Aug 31, 2021 • edited Loading

deathdefyer commented Jan 7, 2022

logopk commented Jan 8, 2022 • edited Loading

martadinata666 commented Jan 8, 2022

logopk commented Jan 8, 2022

daveshow commented Apr 16, 2022

thaJeztah commented Apr 19, 2022

Logicer16 commented Oct 20, 2022

bobbypage commented Oct 20, 2022

bahamut657 commented May 1, 2021 •

edited

Loading

logopk commented Aug 31, 2021 •

edited

Loading

logopk commented Jan 8, 2022 •

edited

Loading