Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 3b74831

Browse files
authoredJan 8, 2025··
Merge pull request #15 from Inkedstinct/7_doc/nld_proofread
7_doc/nld proofread
2 parents ee25e5d + e7dffa0 commit 3b74831

25 files changed

+3549
-464
lines changed
 

‎README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,23 @@
22
PowerAPI website use [MKDocs](https://www.mkdocs.org/).
33

44
You can push your changes on `master` branch to deploy them on the website.
5+
6+
## Local development
7+
8+
When redacting changes for this documentation, you may want to visualize the
9+
output the way it will be presented in the final website once compiled.
10+
11+
One way to do so is to:
12+
13+
1. Install the necessary packages : mkdocs, mkdocs-material, mkdocs-material-extensions
14+
This can be done with: `pip install mkdocs mkdocs-material mkdocs-material-extensions`
15+
16+
2. Have a local copy of this repository (taking care of *checkout-ing* the right ref)
17+
18+
3. From the CLI, have this particular repository as PWD
19+
20+
4. Use mkdocs in order to serve locally (default on http://localhost:8000)
21+
This can be done with: `mkdocs serve -o`
22+
23+
5. Check the URL it serves to, targets will be rebuild on modifications in the
24+
current directory or in any nested elements (use -W to add paths to be considered for hot reloading)

‎docs/assets/images/reference/sensors/PowerAPI_HWPCSensorOverview.drawio

Lines changed: 99 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/assets/images/reference/sensors/PowerAPI_HWPCSensorOverview.drawio.svg

Lines changed: 4 additions & 0 deletions
Loading

‎docs/getting_started.md

Lines changed: 72 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,57 +1,92 @@
11
# Getting started
22

3-
If you want to monitor the energy consumption of your process we have some
4-
ready-to-use tools
3+
In this tutorial, we will guide you through the first steps to get started with PowerAPI.
4+
The objective is to get a quick view of the capabilities of PowerAPI, by monitoring a process and getting a quick glimpse at the energy consumption.
5+
A few things are required before we start:
56

6-
???+ info "Source and Destination"
7-
In order to use any Formula, you need to run a Source and a Destination. The former is used by a Sensor to store metrics. The later allows the Formula to make available the estimations. For starting, you can use [MongoDB](https://hub.docker.com/_/mongo) as Source and [InfluxDB:2.X](https://hub.docker.com/_/influxdb) as Destination by installing them as Docker containers.
8-
For more details about Sources and Destinations please check this [section](reference/database/sources_destinations.md).
7+
- A compatible processor (you can see the compatible CPU architecture [here](./reference/sensors/hwpc-sensor.md#)), and you can look on the following pages to find your CPU architecture:
8+
* For [Intel Processor](https://en.wikipedia.org/wiki/List_of_Intel_processors)
9+
* For [Intel Xeon Processor](https://en.wikipedia.org/wiki/List_of_Intel_Xeon_processors)
10+
* For [AMD Processor](https://en.wikipedia.org/wiki/Table_of_AMD_processors)
11+
- A python installation ready
12+
- Docker & Docker-Compose ready (refer to [this official documentation](https://docs.docker.com/engine/install/) and the [post-install steps](https://docs.docker.com/engine/install/linux-postinstall/) if needed !)
13+
- Root access
14+
- Optional : [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) to proceed by cloning the repository
915

16+
## Which components to get a complete stack
1017

11-
<!---
12-
## **RAPL Formula**
18+
If you wish to get started as soon as possible, the archive below will allow you to deploy the following elements:
1319

14-
!!! note ""
15-
for monitoring the energy consumption of your device
20+
1. A MongoDB instance to store the [Sensor](./reference/sensors/hwpc-sensor.md)
21+
Reports
1622

17-
RAPL Formula is made for tracking the energy consumption of your machine.
18-
To install RAPL Formula on a baremetal server or a PC run [the following
19-
script](script/rapl_install.sh) in a Terminal.
23+
3. An [HWPC-Sensor](./reference/sensors/hwpc-sensor.md) that outputs its
24+
[HWPC Reports](./reference/reports/reports.md#hwpc-reports) in a MongoDB Database,
25+
within the HWPC Report Collection
2026

21-
The script explains what it will do and then pauses before it does it.
27+
4. A [SmartWatts](./reference/formulas/smartwatts.md) that streams the
28+
[HWPC Reports](./reference/reports/reports.md#hwpc-reports) from the MongoDB
29+
Database Collection, processes it and outputs its
30+
[Power Reports](./reference/reports/reports.md#power-reports) as CSV files for a
31+
quick glimpse
2232

23-
Please notice that you need a **Linux distribution** in order to use the HWPC Sensor installed by the script as
24-
well as a **comptible Intel** (Sandy Bridge and newer) or **AMD Processor** (Zen). **Power/ARM/RISCV are not supported** architectures. HWPC Sensor will **not work on a Virtual Machine**. However, you can install the Formula by hand in a Virtual Machine if need it.
25-
-->
33+
## Preparation
2634

27-
## **SmartWatts Formula**
35+
You can either download the archive by cloning the repository or using wget.
2836

29-
!!! note ""
30-
for monitoring the power consumption of your process
37+
=== "git"
38+
```sh
39+
git clone https://github.com/powerapi-ng/powerapi-ng.github.io.git
40+
cd powerapi-ng.github.io/docs/script/getting-started
41+
```
3142

32-
Smartwatts is made for tracking the power consumption of processes on a
33-
machine.
34-
To install Smartwatts on a baremetal server or a PC run [the following
35-
script](script/smartwatts_install.sh) in a Terminal. Please notice that you will need [pip](https://pip.pypa.io/en/stable/installation/) or [docker](https://docs.docker.com/engine/install/) in order to use the Formula.
43+
=== "wget"
44+
```
45+
wget -c https://github.com/raw/powerapi-ng/powerapi-ng.github.io/refs/heads/master/docs/script/getting_started.tar.gz -O - | tar -xz
46+
cd getting_started
47+
```
3648

37-
The script explains what it will do and then pauses before it does it.
49+
At this stage, you will have all the essential files to begin. Let's go through each element in detail.
50+
### Archive content
3851

39-
Please notice that you need a **Linux distribution** in order to use the HWPC Sensor installed by the script as
40-
well as a **comptible Intel** (Sandy Bridge and newer) or **AMD Processor** (Zen). You also need [docker](https://docs.docker.com/engine/install/). **Power/ARM/RISCV are not supported** architectures. HWPC Sensor will **not work on a Virtual Machine**. However, you can install the Formula by hand in a Virtual Machine if need it.
52+
```sh
53+
getting_started/
54+
|--csv/
55+
|--formula/
56+
|----smartwatts-mongodb-csv.json
57+
|--sensor/
58+
|----hwpc-mongodb.json
59+
|--start.sh
60+
|--start.py
61+
|--stop.sh
62+
|--pretty_print.py
63+
|--docker-compose.yaml
64+
|--.env
65+
```
4166

67+
#### HWPC-Sensor and SmartWatts Configuration
4268

69+
As described in the [HWPC-Sensor Documentation](./reference/sensors/hwpc-sensor.md#global-parameters) and in the [SmartWatts Documentation](./reference/formulas/smartwatts.md#global-parameters),
70+
several parameters can be set, both globally and for specific Groups monitored for the sensor or the formula.
4371

44-
#### CGroups
45-
If you need to monitor a process or a group of process via SmartWatts by using HWPC Sensor **version 1.2 or older**, you can follow this [tutorial](reference/cgroup/cgroup.md). Please notice that **cgroup V1** is required **only** for HWPC Sensor **version 1.2 or older**. If you need to enable this `cgroup` version please follow this [tutorial](reference/cgroup/cgroup_v1_activation.md).
72+
The provided docker-compose.yaml file uses configuration files and the **.env** to set those parameters.
73+
You can find examples of both those configuration files in the archive under the **formula** and **sensor** directories.
4674

47-
<!---
48-
## **Jouleit**
4975

50-
!!! note ""
51-
for mesuring the energy consumption of a program
76+
## Turn the key
5277

53-
Jouleit is made for tracking the energy consumption of a program.
54-
Jouleit need `gawk` to run.
55-
You can get the script from the [github repository](https://github.com/powerapi-ng/jouleit)
56-
Start jouleit by using `./jouleit.sh cmd`.
57-
-->
78+
Once all set, you shall be able to initiate the stack with:
79+
80+
```sh
81+
python3 start.py
82+
```
83+
84+
After the 2 minutes of monitoring, you will be able to see the result inside the **csv** directory.
85+
If you have trouble understanding the output, you can read the [Power Report documentation](./reference/reports/reports.md#power-reports).
86+
87+
!!! info "Quick results overview"
88+
Only in the context of this testing archive, after the monitoring, you can use the following command to get a pretty print of the result directly inside the terminal.
89+
90+
```sh
91+
python3 pretty_print.py
92+
```

‎docs/reference/database/sources_destinations.md

Lines changed: 23 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,23 @@
1-
# Sources and Destinations
1+
# Storage Options
22

3-
A PowerAPI Formula uses Sources and Destinations in order to retrieve metrics and store estimations.
3+
Different storage options are available to serve different purpose both for [Sensors](../overview.md#Sensor) and [Formulas](../overview.md#Formula).
44

5-
For each Source/Destination the parameters to specify are different. For each one of them,
6-
its parameters are specified in following sections.
5+
Storage is needed to save reports produced by each components.
6+
- Sensors store their usage reports
7+
- Formulas retrieve usage reports and store energy consumption reports
8+
- Visualization tools or individuals need to access reports for analysis
79

810
## Summary
9-
| Name | Source | Destination | CLI `input`/`ouput` parameter value | JSON `type` tag parameter value |
10-
| ------------- | ----- | ------------- | ------------- | ------------------------------------ |
11-
| MongoDB | Yes | Yes | mongodb | mongodb |
12-
| InfluxDB2 | No | Yes | influxdb2 | influxdb2 |
13-
| CSV | Yes | Yes | csv | csv |
14-
| Socket | Yes | No | socket | socket |
15-
| File Database | Yes | Yes | filedb | filedb |
16-
| Prometheus | No | Yes | prometheus | prometheus |
11+
12+
The following table defines the existing storage options for Sensors usage reports :
13+
14+
| Name | CLI `ouput` parameter value | JSON `type` tag parameter value|
15+
| ------------ | --------------------------------------| -------------------------------------------|
16+
| MongoDB | mongodb | mongodb |
17+
| CSV | csv | csv |
18+
| Socket | socket | socket |
19+
| File Database | filedb | filedb |
20+
1721

1822
## MongoDB
1923

@@ -30,7 +34,7 @@ The list of accepted parameters are:
3034
|`db` (`database` for `HWPCSensor`) | string | `d` (`D` for `HWPCSensor`) | N/A | Yes | The name of your database |
3135
|`collection` | string | `c` (`C` for `HWPCSensor`) | N/A | Yes | The name of the collection inside `db` |
3236
|`name` | string | `n` | `"puller_mongodb"` (Source), `pusher_mongodb` (Destination)| No | The related puller/pusher name. This parameter is not used by `HWPCSensor` |
33-
|`model` | string | `m` | `"HWPCReport"` (Source), `PowerReport` (Destination) | No | The Report type stored by the database |
37+
|`model` | string | `m` | `"HWPC Report"` (Source), `Power Report` (Destination) | No | The Report type stored by the database |
3438

3539
### JSON File Excerpt
3640

@@ -64,7 +68,7 @@ The list of accepted parameters are:
6468
|`org` | string | `g` | N/A | Yes | The name of the organization associated to the bucket |
6569
|`tags` | string | `t` | N/A | No | List of metadata keys of the report separated by `,` that will be kept. `sensor` and `target` are always kept as report metadata |
6670
|`name` | string | `n` | `"pusher_influxdb2"` | No | The related pusher name |
67-
|`model` | string | `m` | `"PowerReport"` | No | The Report type stored by the database |
71+
|`model` | string | `m` | `"Power Report"` | No | The Report type stored by the database |
6872

6973

7074
InfluxDB2 can only be used as a Destination.
@@ -75,7 +79,7 @@ Below you find an example of configuration excerpt for this kind of Destination.
7579

7680
```json
7781
{
78-
"model": "PowerReport",
82+
"model": "Power Report",
7983
"type": "influxdb2",
8084
"uri": "http://127.0.0.1",
8185
"port": 8086,
@@ -100,7 +104,7 @@ The list of accepted parameters are:
100104
|`files`(Source)| string | `f` | Empty list | No | The list of input CSV files with the format file1,file2,file3... |
101105
|`directory` (Destination and `HWPCSensor`)| string |`d` (`U` for `HWPCSensor`) | Current directory | No |The directory where output CSV files will be written |
102106
|`name` | string | `n` | `"puller_csv"` (Source), `"pusher_csv"` (Destination)| No | The related puller/pusher name. This parameter is not used by `HWPCSensor` |
103-
|`model` | string | `m` | `"HWPCReport"` (Source), `"PowerReport"` (Destination) | No | The Report type stored in CSV files. This parameter is not used by `HWPCSensor` |
107+
|`model` | string | `m` | `"HWPC Report"` (Source), `"Power Report"` (Destination) | No | The Report type stored in CSV files. This parameter is not used by `HWPCSensor` |
104108

105109
### JSON File Excerpt
106110

@@ -128,7 +132,7 @@ The list of accepted parameters are:
128132
|`port` | int | `P` | N/A | Yes | The port of communication |
129133
|`uri`/ `host` | int | `U` | N/A | Yes | The IP address of the machine running the socket |
130134
|`name` | string | `n` | `"puller_socket"`| No | The related puller name |
131-
|`model` | string | `m` | `"HWPCReport"` | No | The Report type managed by the socket |
135+
|`model` | string | `m` | `"HWPC Report"` | No | The Report type managed by the socket |
132136

133137

134138
### JSON File Excerpt
@@ -158,7 +162,7 @@ The list of accepted parameters are:
158162
| ------------- | ----- | ------------- | -------------| ---------- | ------------------------------------ |
159163
|`filename` | int | `f` | N/A | Yes | The name of the file |
160164
|`name` | string | `n` | `"pusher_filedb"` | No | The related pusher name |
161-
|`model` | string | `m` | `"HWPCReport"` (Source) `"PowerReport"` (Destination)| No | The Report type stored in the file |
165+
|`model` | string | `m` | `"HWPC Report"` (Source) `"Power Report"` (Destination)| No | The Report type stored in the file |
162166

163167
### JSON File Excerpt
164168

@@ -188,7 +192,7 @@ The list of accepted parameters are:
188192
|`metric-name` | string | `M` | N/A | Yes | The exposed metric name |
189193
|`metric-description` | string | `d` | `"energy consumption"` | No | The exposed metric description |
190194
|`name` | string | `n` | `"pusher_prom"` | No | The related pusher name |
191-
|`model` | string | `m` | `"PowerReport"` | No | The Report type exposed by Prometheus |
195+
|`model` | string | `m` | `"Power Report"` | No | The Report type exposed by Prometheus |
192196

193197

194198
Prometheus can only be used as a Destination that monitors reports but they will be not stored by this service.

‎docs/reference/formulas/configuration_files.md

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,14 +11,11 @@ The table below shows basic parameters.
1111
| `verbose` | `bool` (flag) | `v` |`NOTSET` | Verbose or quiet mode |
1212
| `stream` | `bool` (flag) | `s` |`False` | Real time or post-mortem mode |
1313
| `sensor-report-sampling-interval` | `int` | N/A | `1000` | The time in milliseconds between two reports (`stream` = `True`) |
14-
| `input` | `string` (Source) | N/A | N/A | Source used as input. More information about Sources and their related parameters can be found [here](../database/sources_destinations.md)|
15-
| `output` | `string` (Destination)| N/A | N/A | Destination used as output. More information about Destinations and their related parameters can be found [here](../database/sources_destinations.md)|
14+
| `input` | `string` | N/A | N/A | SmartWatts input, shall match an existing Sensor output and contain HPWCReports. See [here](./smartwatts.md#smartwatts-inputs) |
15+
| `output` | `string` | N/A | N/A | SmartWatts output to store Power Report. See [here](./smartwatts.md#smartwatts-outputs) |
1616
| `pre-processor` | `string` | N/A | N/A | Pre-Processor to modify reports generated by a sensor. More information about Processors and their related parameters can be found [here](../processors/processors.md) |
1717
| `post-processor` | `string` | N/A | N/A | Post-Processor to modify reports generated by a formula. More information about Processors and their related parameters can be found [here](../processors/processors.md) |
1818

19-
???+ tip "Sources and Destinations' values"
20-
- Sources: `mongodb`, `csv`, `socket`, `filedb`
21-
- Destinations: `mongodb`, `influxedb`, `influxedb2`, `csv`, `socket`, `filedb`, `prom`
2219

2320
???+ tip "Processors' values"
2421
- Pre-Processors: `k8s`, `libvirt`
@@ -52,5 +49,5 @@ PowerAPI Formulas use `json` files. These files follow the next template:
5249
$formula_parameters
5350
}
5451
```
55-
???+ info "Sources and Destinations' `json` tags"
52+
???+ info "input and output' `json` tags"
5653
More information related to `json` tags for each Source/Destination can be found [here](../database/sources_destinations.md)

‎docs/reference/formulas/rapl.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ The RAPL Formula is designed to mesure power consumption of domains (CPU or RAM)
77
in real time.
88

99
The RAPL Formula takes HWPC Report with RAPL event for each domains. It then
10-
returns the mesured power in a PowerReport for each domain.
10+
returns the mesured power in a Power Report for each domain.
1111
This Formula does not perform any other computation as its goal is only to track
1212
global power consumption in a more readable way than raw RAPL.
1313

@@ -29,7 +29,7 @@ You can use [the following script](../script/rapl_install.sh) to install RAPL Fo
2929

3030
## Usage
3131

32-
For running the RAPL Formula you need: a Source and a Destination, a Sensor that provides `HWPCReports` and a configuration.
32+
For running the RAPL Formula you need: a Source and a Destination, a Sensor that provides `HWPC Reports` and a configuration.
3333

3434
### Source and Destination
3535

@@ -42,7 +42,7 @@ docker run -d --name mongo_source_destination -p 27017:27017 mongo
4242
```
4343

4444
### Sensor
45-
[HWPC Sensor](../sensors/hwpc-sensor.md) is used in order to get `HWPCReports`. Start by installing the HWPC Sensor (see
45+
[HWPC Sensor](../sensors/hwpc-sensor.md) is used in order to get `HWPC Reports`. Start by installing the HWPC Sensor (see
4646
[here](../sensors/hwpc-sensor.md#installation)) and start it (see
4747
[here](../sensors/hwpc-sensor.md#usage)).
4848

@@ -69,7 +69,7 @@ Below an example is provided by using MongoDB as Source and Destination.
6969
"stream": true,
7070
"input": {
7171
"puller": {
72-
"model": "HWPCReport",
72+
"model": "HWPC Report",
7373
"type": "mongodb",
7474
"uri": "mongodb://127.0.0.1",
7575
"db": "test",
@@ -79,7 +79,7 @@ Below an example is provided by using MongoDB as Source and Destination.
7979
"output": {
8080
"pusher_power": {
8181
"type": "mongodb",
82-
"model": "PowerReport",
82+
"model": "Power Report",
8383
"type": "mongodb",
8484
"uri": "mongodb://127.0.0.1",
8585
"db": "test",
@@ -124,8 +124,8 @@ the installation you use:
124124
docker run -t \
125125
--net=host \
126126
powerapi/powerapi --verbose \
127-
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
128-
--output mongodb --model PowerReport --uri mongodb://127.0.0.1 --db test --collection results \
127+
--input mongodb --model HWPC Report --uri mongodb://127.0.0.1 --db test --collection prep \
128+
--output mongodb --model Power Report --uri mongodb://127.0.0.1 --db test --collection results \
129129
--disable-dram-formula \
130130
--sensor-report-sampling-interval 500
131131
```
@@ -134,14 +134,14 @@ the installation you use:
134134

135135
```sh
136136
python -m powerapi --verbose \
137-
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
138-
--output mongodb --model PowerReport --uri mongodb://127.0.0.1 --db test --collection results \
137+
--input mongodb --model HWPC Report --uri mongodb://127.0.0.1 --db test --collection prep \
138+
--output mongodb --model Power Report --uri mongodb://127.0.0.1 --db test --collection results \
139139
--disable-dram-formula \
140140
--sensor-report-sampling-interval 500
141141
```
142142

143143
???+ info "Estimations' Storage"
144-
Your `PowerReports` will be stored on MongoDB.
144+
Your `Power Reports` will be stored on MongoDB.
145145

146146
???+ tip "Using shortcuts for parameters' names"
147147
You use `-` instead of `--`.

‎docs/reference/formulas/smartwatts.md

Lines changed: 559 additions & 166 deletions
Large diffs are not rendered by default.

‎docs/reference/formulas/smartwatts.md.save

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ You can use [the following script](../script/smartwatts_install.sh) to install S
3939

4040
## Usage
4141

42-
For running the SmartWatts Formula you need: a Source and a Destination, a Sensor that provides `HWPCReports` and a configuration.
42+
For running the SmartWatts Formula you need: a Source and a Destination, a Sensor that provides `HWPC Reports` and a configuration.
4343

4444
### Source and Destination
4545
For running SmartWatts we are using MongoDB as Source and InfluxDB as Destination as dockers containers.
@@ -57,7 +57,7 @@ docker run -d --name influx_dest -p 8086:8086 influxdb:1.8
5757

5858

5959
### Sensor
60-
[HWPC Sensor](../sensors/hwpc-sensor.md) is used in order to get `HWPCReports`. Start by installing the HWPC Sensor (see
60+
[HWPC Sensor](../sensors/hwpc-sensor.md) is used in order to get `HWPC Reports`. Start by installing the HWPC Sensor (see
6161
[here](../sensors/hwpc-sensor.md#installation)) and start it (see
6262
[here](../sensors/hwpc-sensor.md#usage)).
6363

@@ -91,7 +91,7 @@ Below an example is provided by using MongoDB as Source and InfluxDB as Destinat
9191
"stream": true,
9292
"input": {
9393
"puller": {
94-
"model": "HWPCReport",
94+
"model": "HWPC Report",
9595
"type": "mongodb",
9696
"uri": "mongodb://127.0.0.1",
9797
"db": "test",
@@ -148,8 +148,8 @@ the installation you used:
148148
docker run -t \
149149
--net=host \
150150
powerapi/smartwatts-formula --verbose \
151-
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
152-
--output influxdb --model PowerReport --uri 127.0.0.1 --port 8086 --db test_result \
151+
--input mongodb --model HWPC Report --uri mongodb://127.0.0.1 --db test --collection prep \
152+
--output influxdb --model Power Report --uri 127.0.0.1 --port 8086 --db test_result \
153153
--cpu-base-freq 1900 \
154154
--cpu-error-threshold 2.0 \
155155
--disable-dram-formula \
@@ -161,8 +161,8 @@ the installation you used:
161161
```sh
162162
python -m smartwatts \
163163
--verbose \
164-
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
165-
--output influxdb --model PowerReport --uri 127.0.0.1 --port 8086 --db test_result \
164+
--input mongodb --model HWPC Report --uri mongodb://127.0.0.1 --db test --collection prep \
165+
--output influxdb --model Power Report --uri 127.0.0.1 --port 8086 --db test_result \
166166
--cpu-base-freq 1900 \
167167
--cpu-error-threshold 2.0 \
168168
--disable-dram-formula \
@@ -172,7 +172,7 @@ the installation you used:
172172

173173

174174
???+ info "Estimations' Storage"
175-
Your `PowerReports` will be stored on InfluxDB. You can watch them in a grafana by using the [following tutorial](../grafana/grafana.md).
175+
Your `Power Reports` will be stored on InfluxDB. You can watch them in a grafana by using the [following tutorial](../grafana/grafana.md).
176176

177177
???+ tip "Using shortcuts for parameters' names"
178178
You use `-` instead of `--`.

‎docs/reference/grafana/grafana.md

Lines changed: 32 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,27 @@
22

33
Here it is explained how to visualize the power estimation computed by a Formula on a Grafana dashboard to obtain this kind of visualisation:
44

5-
<img
5+
![Visualization by process](../../assets/images/reference/grafana/viz_by_process.png){ width="1000px"}
6+
7+
<!--img
68
src="https://powerapi.org/assets/images/reference/grafana/viz_by_process.png"
7-
alt="viz_by_process" width="1000px">
9+
alt="viz_by_process" width="1000px"-->
10+
11+
This screenshot shows the visualisation of power consumption of a `Firefox` web browser and tools used for monitoring (MongoDB, InfluxDB, Sensor, Formula)
812

9-
This screenshot shows the visualisation of power consumption of a Web browser and tools used for monitoring (Source, Destination, Sensor, Formula)
13+
In this tutorial, we describe how to connect a Formula to a Grafana instance by using InfluxDB 2.X as output for Power Report.
14+
Then, we will see how to configure Grafana to visualize the power estimation computed by the Formula.
1015

11-
In this tutorial, we describe how to connect a Formula to a Grafana instance by using InfluxDB 2.X as Destination.
12-
Then, we will see how to configure Grafana to visualize the power estimation computed by the Formula.
16+
This tutorial assumes that you know how launch a Formula and a Sensor to compute power estimation and that you have an InfluxDB 2.X running on your local machine.
1317

14-
This tutorial assumes that you know how launch a Formula and a Sensor to compute power estimation and that you have an InfluxDB 2.X and a Grafana instances running on your local machine.
15-
The InfluxDB 2.X instance listen on port `8086`.
18+
The InfluxDB 2.X instances listen on port `8086` by default.
1619

1720
## Setup Grafana
1821

22+
The default installation is done through Docker container.
23+
The different images can be found on the [Docker Hub](https://hub.docker.com/r/grafana/grafana)
24+
25+
Here is an example to deploy the latest image version available.
1926
```sh
2027
docker run -d -p 3000:3000 grafana/grafana
2128
```
@@ -24,49 +31,40 @@ After the launch, Grafana will be available at http://localhost:3000. On the sig
2431

2532
## Connect Grafana to the InfluxDB 2.X instance
2633

27-
Connect to your Grafana instance and go to the `"Data sources"`` section (in the configuration part of the side bar).
28-
29-
<img
30-
src="https://powerapi.org/assets/images/reference/grafana/grafana_home.png"
31-
alt="grafana_home" width="300px">
32-
33-
Click on the `"Add new data source"` button and select `"InfluxDB"`. Enter:
34-
35-
1. A data source *Name* (here we choose "InfluxDB-2"),
36-
2. A *Query Language*, i.e., `InfluxQL`
37-
3. An instance *URL* (`http://localhost:8086`)
38-
4. A *Custom HTTP Header* called `Authorization` with Value `Token <mytoken>`, where `<mytoken>` is the token provided by InfluxDB 2.X for your organization.
39-
5. A *Database* name, (here we choose `power_consumption`) that is the `db` value of your destination defined in your formula configuration.
34+
Follow [this documentation](https://grafana.com/docs/grafana/latest/datasources/influxdb/#configure-the-data-source) in order to add InfluxDB as a DataSource.
35+
Be carefull to use the documentation matching your Grafana version.
4036

41-
Then click on the `"Save & test"` button. *User* and *Password* are not required as we use a token for authentification.
42-
43-
44-
<img
45-
src="https://powerapi.org/assets/images/reference/grafana/add_db.png"
46-
alt="add_db" width="400px">
4737

4838
## Visualize the power consumption on a dashboard in real-time
4939

50-
Go to the `"Dashboard"` section on the side bar and select on `New > New dashboard`. Then click on `+ Add visualisation` Then select `influxdb-2` as data source.
40+
1. Go to the `"Dashboard"` section on the side bar and select on `New > New dashboard`
41+
2. Then click on `+ Add visualisation`
42+
3. Then select the previously added datasource as data source
43+
44+
![Add Dashboard](../../assets/images/reference/grafana/add_dashboard.png){ width="1000px"}
5145

52-
<img
46+
<!--img
5347
src="https://powerapi.org/assets/images/reference/grafana/add_dashboard.png"
54-
alt="add_dashboard" width="1000px">
48+
alt="add_dashboard" width="1000px"-->
5549

56-
Click on the query edition button (the one with a pencil on it) and write the following query on the `Query` field to request the power estimations from the InfluxDB 2.X measurement:
50+
4. Click on the query edition button (the one with a pencil on it) and write the following query on the `Query` field to request the power estimations from the InfluxDB 2.X measurement:
5751

5852
```sql
5953
SELECT power FROM "power_consumption" GROUP BY target
6054
```
6155

6256
Then write `$tag_target` on the `ALIAS BY` field to label each graph with the target name
6357

64-
<img
58+
![Add query by process](../../assets/images/reference/grafana/add_query_by_process.png){ width="1000px"}
59+
60+
<!--img
6561
src="https://powerapi.org/assets/images/reference/grafana/add_query_by_process.png"
66-
alt="add_query_by_process" width="600px">
62+
alt="add_query_by_process" width="600px"-->
6763

6864
To display the power consumption in real time, you can update the range of the visualisation to `last 5 minutes` and the `refresh dashboard` parameter to `5s`. This parameter are on the top-right corner of the UI.
6965

70-
<img
66+
![Change the refresh interval](../../assets/images/reference/grafana/refresh.png){ width="1000px"}
67+
68+
<!--img
7169
src="https://powerapi.org/assets/images/reference/grafana/refresh.png"
72-
alt="refresh" width="600px">
70+
alt="refresh" width="600px"-->

‎docs/reference/overview.md

Lines changed: 45 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,63 +1,71 @@
11
# PowerAPI Overview
22

3-
The goal of this project is to provide a set of tools to go forward a greener
4-
computing.
5-
The idea is to provide software-defined PowerMeters to mesure the power
6-
consumption of programs.
7-
The core of this project is the [PowerAPI](https://github.com/powerapi-ng/powerapi) toolkit for building
8-
such PowerMeters.
9-
10-
3+
This project aims to offer a range of tools to promote greener computing. The focus is on creating software-defined PowerMeters to measure the power consumption of various programs.
4+
At the heart of this initiative is the [PowerAPI](https://github.com/powerapi-ng/powerapi) toolkit, which facilitates the development of these PowerMeters.
115

126
## Software PowerMeters
137

14-
A software PowerMeter is an application built with the PowerAPI components that can
15-
measure the power consumption of software running on a single machine or on a
16-
cluster of machine.
8+
A software PowerMeter is an application built using PowerAPI components, designed to measure the power consumption of software running on a single machine or across a cluster of machines.
179

18-
The Figure below depicts the global architecture of a software PowerMeter in PowerAPI.
10+
The diagram below illustrates the overall architecture of a PowerMeter within the PowerAPI framework.
1911

2012
![PowerAPI Architecture Overview](../assets/images/reference/overview/global-architecture.jpg){ width="1000px"}
2113

14+
A PowerMeter consists of two essential components:
2215

23-
<!--img
24-
src="assets/images/intro/global-architecture.png"
25-
alt="PowerMeter Architecture" width="1000px"-->
16+
- a [Sensor](#Sensor), which collects system usage metrics and generates usage reports.
17+
- a [Formula](#Formula), which applies a computational model to the usage reports, producing power consumption data (in watts).
2618

27-
A PowerMeter has two components, a Sensor and a Formula, used to
28-
produce an estimation of the power consumption of a monitored software.
19+
Additionally, [Preprocessors](./overview.md#Preprocessors) can be utilized to modify usage reports before they are processed by the Formula.
2920

3021
## Sensor
3122

32-
The Sensor is an independent software that collects raw data (metrics) correlated with the power consumption of the
33-
monitored software.
23+
The Sensor is a standalone software tool that gathers basic data related to the power usage of the software it monitors.
24+
25+
It collects this data by querying the hardware on the same machine where the monitored software runs. The Sensor needs to operate in parallel to the monitored software for the entire duration of its execution.
3426

35-
Data are collected by querying the hardware’s machine that hosts the monitored
36-
software. The sensor must be executed on the same machine as the monitored
37-
software. The data are collected throughout the duration of the software. For
38-
this reason, the sensor must operate in parallel.
27+
The raw data collected is then stored as usage Reports in various formats, either locally or remotely. These formats include CSV, MongoDB, FileDB, and UNIX sockets, among others.
3928

40-
Collected raw data are stored in an external Source to make the data available to
41-
the Formula. This Source may be hosted on an other machine.
29+
### Existing Sensors
4230

43-
### Usage
31+
| Sensor | Documentation | Repository | Description | Supported |
32+
| :---------------: |----------------------|--------------------------|----------------|---------------|
33+
| HWPC | [HWPC Documentation](./sensors/hwpc-sensor.md) | [Github](https://github.com/powerapi-ng/hwpc-sensor) | Hardware Performance Counters monitoring agent for containers | :material-check: Supported |
4434

45-
Currently, PowerAPI proposes one Sensor: [HWPC](sensors/hwpc-sensor.md).
46-
Refer to the Sensor documentation to know how to use it.
4735

4836
## Formula
4937

50-
A Formula is a computational module that computes an estimation of the power
51-
consumption of monitored software from the data collected by the sensor.
38+
A Formula is a computational module that estimates the power consumption of the monitored software using the data collected by the Sensor.
39+
40+
There are two modes in which a Formula can operate:
41+
42+
- **Stream mode**: In this mode, the Formula processes data from the Sensor as it is generated, providing real-time insights
43+
44+
- **Post-mortem mode**: In this mode, the Formula analyzes data that has already been collected by the Sensor during a previous monitoring phase
45+
46+
### Existing Formulas
47+
48+
| Formula | Documentation | Repository | Description | Supported |
49+
| :---------------: |----------------------|--------------------------|----------------|---------------|
50+
| SmartWatts | [SmartWatts Documentation](./formulas/smatwatts.md) | [Github](https://github.com/powerapi-ng/smartwatts-formula) | HSmartWatts is a formula for a self-adaptive software-defined power meter based on the PowerAPI framework. | :material-check: Supported |
51+
52+
## Processors
53+
54+
Processors allow for the customized filtering and modification of Reports. While optional, they offer additional flexibility, but Software PowerMeters can still function without them.
55+
56+
The diagram below shows where Processors are integrated into the architecture of a Software PowerMeter.
57+
58+
![Processor integration in the processing pipeline](../assets/images/reference/processors/processors.jpg)
5259

53-
A Formula has two working modes:
60+
### Preprocessors
5461

55-
- `stream` mode where the Formula read the data from the Sensor as they are
56-
produced (in realtime).
62+
Preprocessors are positioned between the [Sensor](./overview.md#Sensor) and the [Formula](./overview.md#Formula).
63+
Their role is to pre-process usage reports before the power consumption estimations are computed.
5764

58-
- `post-mortem` mode where the Formula analyses the data already collected by the Sensor in a past monitoring phase.
65+
#### Existing Preprocessors
5966

60-
### Usage
67+
| Preprocessor | Documentation | Repository | Description | Supported |
68+
| :---------------: |----------------------|--------------------------|----------------|---------------|
69+
| k8sPreprocessor | [k8sPreprocessor Documentation](./processors/processors.md#k8spreprocessor) | [Github](https://github.com/powerapi-ng/powerapi/tree/master/src/powerapi/processor/pre/k8s) | Add K8S Specific metadata to corresponding Reports | :material-check: Supported |
70+
| libvirt | [libvirtPreprocessor Documentation](.processors/processors.md#libvirt) | [Github](https://github.com/powerapi-ng/powerapi/tree/master/src/powerapi/processor/pre/libvirt) | Replace `libvirt ID` in Reports with `Open Stack UUID` | :material-check: Supported |
6171

62-
Currently, there is one Formula: [SmartWatts](formulas/smartwatts.md).
63-
Refer to the Formula documentation to know how to use it.

‎docs/reference/processors/processors.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
Processors enable customized filtering and/or modifications of `Reports`.
44
There are two kinds of processors:
55

6-
- `PreProcessors`: They are located between the `Puller` and the `Dispatcher`. They are supposed to pre-process the `HWPCReports` before computing estimations.
7-
- `PostProcessors`: They are located between, the `Formula` and the `Pusher`. They process `PowerReports` before storing them on the `Destination`.
6+
- `PreProcessors`: They are located between the `Puller` and the `Dispatcher`. They are supposed to pre-process the `HWPC Reports` before computing estimations.
7+
- `PostProcessors`: They are located between, the `Formula` and the `Pusher`. They process `Power Reports` before storing them on the output storage option.
88

99
Figure below depicts where are they introduced in the architecture of a Software `PowerMeters`.
1010

@@ -60,8 +60,8 @@ As notice, a `PreProcessor` is defined inside the `pre-processor` group. In this
6060
docker run -t \
6161
--net=host \
6262
powerapi/smartwatts-formula --verbose \
63-
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
64-
--output influxdb --model PowerReport --uri 127.0.0.1 --port 8086 --db test_result \
63+
--input mongodb --model HWPC Report --uri mongodb://127.0.0.1 --db test --collection prep \
64+
--output influxdb --model Power Report --uri 127.0.0.1 --port 8086 --db test_result \
6565
{==--pre-processor k8s --name p1 --api-mode local --puller puller==} \
6666
--cpu-base-freq 1900 \
6767
--cpu-error-threshold 2.0 \
@@ -74,8 +74,8 @@ As notice, a `PreProcessor` is defined inside the `pre-processor` group. In this
7474
```sh
7575
python -m smartwatts \
7676
--verbose \
77-
--input mongodb --model HWPCReport --uri mongodb://127.0.0.1 --db test --collection prep \
78-
--output influxdb --model PowerReport --uri 127.0.0.1 --port 8086 --db test_result \
77+
--input mongodb --model HWPC Report --uri mongodb://127.0.0.1 --db test --collection prep \
78+
--output influxdb --model Power Report --uri 127.0.0.1 --port 8086 --db test_result \
7979
{==--pre-processor k8s --name p1 --api-mode local --puller puller==} \
8080
--cpu-base-freq 1900 \
8181
--cpu-error-threshold 2.0 \
@@ -96,12 +96,12 @@ As notice, a `PreProcessor` is defined inside the `pre-processor` group. In this
9696
-e POWERAPI_CPU_ERROR_THRESHOLD=2.0 \
9797
-e POWERAPI_DISABLE_DRAM_FORMULA=true \
9898
-e POWERAPI_SENSOR_REPORTS_FREQUENCY=1000 \
99-
-e POWERAPI_INPUT_PULLER_MODEL=HWPCReport \
99+
-e POWERAPI_INPUT_PULLER_MODEL=HWPC Report \
100100
-e POWERAPI_INPUT_PULLER_TYPE=mongodb \
101101
-e POWERAPI_INPUT_PULLER_URI=mongodb://127.0.0.1 \
102102
-e POWERAPI_INPUT_PULLER_DB=test \
103103
-e POWERAPI_INPUT_PULLER_COLLECTION=prep \
104-
-e POWERAPI_OUTPUT_PUSHER_POWER_MODEL=PowerReport \
104+
-e POWERAPI_OUTPUT_PUSHER_POWER_MODEL=Power Report \
105105
-e POWERAPI_OUTPUT_PUSHER_POWER_TYPE=influxdb \
106106
-e POWERAPI_OUTPUT_PUSHER_POWER_URI=127.0.0.1 \
107107
-e POWERAPI_OUTPUT_PUSHER_POWER_PORT=8086 \
@@ -121,12 +121,12 @@ As notice, a `PreProcessor` is defined inside the `pre-processor` group. In this
121121
export POWERAPI_CPU_ERROR_THRESHOLD=2.0
122122
export POWERAPI_DISABLE_DRAM_FORMULA=true
123123
export POWERAPI_SENSOR_REPORTS_FREQUENCY=1000
124-
export POWERAPI_INPUT_PULLER_MODEL=HWPCReport
124+
export POWERAPI_INPUT_PULLER_MODEL=HWPC Report
125125
export POWERAPI_INPUT_PULLER_TYPE=mongodb
126126
export POWERAPI_INPUT_PULLER_URI=mongodb://127.0.0.1
127127
export POWERAPI_INPUT_PULLER_DB=test
128128
export POWERAPI_INPUT_PULLER_COLLECTION=prep
129-
export POWERAPI_OUTPUT_PUSHER_POWER_MODEL=PowerReport
129+
export POWERAPI_OUTPUT_PUSHER_POWER_MODEL=Power Report
130130
export POWERAPI_OUTPUT_PUSHER_POWER_TYPE=influxdb
131131
export POWERAPI_OUTPUT_PUSHER_POWER_URI=127.0.0.1
132132
export POWERAPI_OUTPUT_PUSHER_POWER_PORT=8086
@@ -140,15 +140,15 @@ As notice, a `PreProcessor` is defined inside the `pre-processor` group. In this
140140

141141
### Example of Usage with SmartWatts Formula via a Configuration File
142142

143-
Below an example is provided by using MongoDB as Source and InfluxDB as Destination.
143+
Below an example is provided by using MongoDB as input and InfluxDB as output.
144144

145145
```json
146146
{
147147
"verbose": true,
148148
"stream": true,
149149
"input": {
150150
"puller": {
151-
"model": "HWPCReport",
151+
"model": "HWPC Report",
152152
"type": "mongodb",
153153
"uri": "mongodb://127.0.0.1",
154154
"db": "test",
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"$schema": "https://json-schema.org/draft/2020-12/schema",
3+
"$id": "https://raw.githubusercontent.com/powerapi-ng/powerapi-ng.github.io/refs/heads/master/docs/reference/reports/basis-reports.schema.json",
4+
"title": "Report Basis",
5+
"description": "A common basis for PowerAPI Reports",
6+
"type": "object",
7+
"properties": {
8+
"timestamp": {
9+
"type": "string",
10+
"minLength": 1
11+
},
12+
"sensor": {
13+
"type": "string",
14+
"minLength": 1
15+
},
16+
"target": {
17+
"type": "string",
18+
"minLength": 1
19+
},
20+
"report_specific_fields": {
21+
"type": "object",
22+
"properties": {},
23+
"required": []
24+
}
25+
},
26+
"required": [
27+
"timestamp",
28+
"sensor",
29+
"target",
30+
"report_specific_fields"
31+
]
32+
}

‎docs/reference/reports/reports.md

Lines changed: 17 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -2,59 +2,56 @@
22

33
PowerAPI toolkit is modular, which means any sensor can be plugged to any monitoring
44
tool as long as the needed information is provided.
5-
We fixed the way of encoding the information. Those encoding are called reports.
5+
We fixed the way of encoding the information. Those encoding are called `Reports`.
66

77
A report type specify the `json` fields that has to be provided to pass information of
88
a certain kind. All reports types have a common basis:
99

1010

11-
- `timestamp`: at the format "year-month-dayThour:minutes:seconds". The
12-
timestamp reflects the time at which the information correspond, not the
13-
time the information was computed.
14-
For example if a power consumption of a CPU is measured at time `t` and used to
15-
determine the power consumption of a `cgroup` in a `PowerReport`, this report
16-
has timestamp `t`.
11+
- `timestamp`: at the [format](https://en.wikipedia.org/wiki/ISO_8601) "YYYY-MM-DDThh\:mm\:ss\.sss". The timestamp indicates when the data was collected, not when it was processed.
1712

18-
- `target`: the target is the subject of the measure. For example if
19-
you produce a report that contain information relative to a program, domain,
20-
etc., the target refers to it. It corresponds to the `cgroup` name.
13+
- `target`: The target refers to the entity being measured. For example, if a report contains data related to a specific program, domain, or other entity, the target identifies that subject. In this context, it corresponds to the cgroup name.
2114

2215
- `sensor`: it's a name field that is used to identify the reports produced by or computed thanks to a sensor.
2316

24-
Therefore, a report have the following format:
17+
Therefore, a report shall match the following description:
2518

26-
```json
19+
```json
20+
{
2721
"timestamp":$timestamp,
2822
"target":$target,
2923
"sensor":$sensor,
3024
$report_specific_fields
25+
}
26+
```
27+
28+
A valid JSON-Schema to can be found [here](https://raw.githubusercontent.com/powerapi-ng/powerapi-ng.github.io/refs/heads/master/docs/reference/reports/basis-reports.schema.json).
3129

32-
```
3330
In the following sections we specify the `$report_specific_fields` for each type of report.
3431

3532
## HWPC Reports
3633

37-
A `HWPCReport` is used to report performance counters and RAPL.
34+
A `HWPC Report` is used to report performance counters and RAPL.
3835
Its specific fields are the following:
3936

4037
- `groups`: a list of subreport that can be of three kind, `rapl`, `core` and
4138
`msr`.
4239

4340
Each group is represented in the same way:
4441

45-
```json
42+
```json
4643
{
4744
$type: {
4845
$core_number : {
4946
$socket_number : {
50-
List of counter and their value
47+
Each counter name and their measured value for the type/core_number/socket_number considered triplet
5148
}
5249
}
5350
}
5451
}
55-
```
52+
```
5653

57-
Below you can find an example of `HWPCReport`:
54+
Below you can find an example of `HWPC Report`:
5855

5956
```json
6057
{
@@ -123,12 +120,12 @@ Below you can find an example of `HWPCReport`:
123120

124121
## Power Reports
125122

126-
A `PowerReport` is used to transfer information about power consumption estimations.
123+
A `Power Report` is used to transfer information about power consumption estimations.
127124
Its specific fields are the following:
128125

129126
- `power`: a power value in Watts.
130127

131-
Below you find an exemple of `PowerReport`:
128+
Below you find an exemple of `Power Report`:
132129

133130
```json
134131
{

‎docs/reference/sensors/hwpc-sensor.md

Lines changed: 246 additions & 111 deletions
Large diffs are not rendered by default.

‎docs/script/getting_started.tar.gz

17.7 KB
Binary file not shown.

‎docs/script/getting_started/.env

Whitespace-only changes.

‎docs/script/getting_started/cpu.csv

Lines changed: 1857 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
services:
2+
############################################
3+
# SOURCES & DESTINATIONS #
4+
############################################
5+
# MongoDB
6+
mongodb:
7+
container_name: mongodb
8+
image: ${MONGO_IMAGE}
9+
networks:
10+
- powerapi-network
11+
profiles:
12+
- mongodb
13+
14+
15+
############################################
16+
# POWERAPI #
17+
############################################
18+
# PowerAPI Sensor
19+
sensor:
20+
container_name: sensor
21+
image: ${SENSOR_IMAGE}
22+
privileged: true
23+
command:
24+
- "--config-file"
25+
- "/etc/sensor.json"
26+
volumes:
27+
- ${PWD}/sensor/hwpc-${POWERAPI_SOURCE}.json:/etc/sensor.json
28+
- type: bind
29+
source: /proc
30+
target: /proc
31+
- type: bind
32+
source: /sys
33+
target: /sys
34+
- type: bind
35+
source: /var/lib/docker/containers
36+
target: /var/lib/docker/containers
37+
depends_on:
38+
- ${POWERAPI_SOURCE}
39+
networks:
40+
- powerapi-network
41+
restart: unless-stopped
42+
43+
# PowerAPI Formula
44+
formula:
45+
user: ${UID}:${GUID}
46+
container_name: formula
47+
image: ${FORMULA_IMAGE}
48+
command:
49+
- "--config-file"
50+
- "/etc/formula.json"
51+
volumes:
52+
- ${PWD}/formula/smartwatts-${POWERAPI_SOURCE}-${POWERAPI_DESTINATION}.json:/etc/formula.json
53+
- ${PWD}/csv:/tmp/csv
54+
networks:
55+
- powerapi-network
56+
restart: unless-stopped
57+
58+
59+
60+
############################################
61+
# TOOLS #
62+
############################################
63+
64+
# Mongo Express
65+
mongo-express:
66+
container_name: mongo-express
67+
image: ${MONGOEXPRESS_IMAGE}
68+
environment:
69+
ME_CONFIG_MONGODB_SERVER: mongodb
70+
ME_CONFIG_MONGODB_URL: mongodb://mongodb:27017
71+
depends_on:
72+
- mongodb
73+
ports:
74+
- "8081:8081"
75+
networks:
76+
- powerapi-network
77+
profiles:
78+
- mongodb
79+
80+
############################################
81+
# DOCKER #
82+
############################################
83+
84+
networks:
85+
powerapi-network:
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Sensor image
2+
SENSOR_IMAGE=powerapi/hwpc-sensor:${HWPC_SENSOR_VERSION:-latest}
3+
4+
# Formula image
5+
FORMULA_IMAGE=powerapi/smartwatts-formula:${SMARTWATTS_VERSION:-latest}
6+
7+
# Source selection
8+
# Available options: mongodb, socket
9+
POWERAPI_SOURCE=mongodb
10+
11+
# Destination selection
12+
# Available options: influxdb2, prometheus, mongodb, csv
13+
POWERAPI_DESTINATION=csv
14+
15+
# Third party images
16+
MONGO_IMAGE=mongo:latest
17+
MONGOEXPRESS_IMAGE=mongo-express:latest
18+
INFLUXDB_IMAGE=influxdb:latest
19+
CSV_IMAGE=busybox:stable-glibc
20+
21+
# Docker compose profiles
22+
COMPOSE_PROFILES=${POWERAPI_SOURCE},${POWERAPI_DESTINATION}
23+
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"verbose": false,
3+
"stream": true,
4+
"input": {
5+
"puller_mongodb": {
6+
"model": "HWPCReport",
7+
"type": "mongodb",
8+
"name": "puller_mongodb",
9+
"uri": "mongodb://mongodb:27017",
10+
"db": "db_sensor",
11+
"collection": "prep"
12+
}
13+
},
14+
"output": {
15+
"pusher_csv": {
16+
"model": "PowerReport",
17+
"type": "csv",
18+
"name": "pusher_csv",
19+
"directory": "/tmp/csv"
20+
}
21+
},
22+
"cpu-base-freq": 1900,
23+
"cpu-error-threshold": 2.0,
24+
"disable-dram-formula": true,
25+
"sensor-reports-frequency": 1000,
26+
"cpu-tdp": 15
27+
}
Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"name": "sensor",
3+
"verbose": true,
4+
"frequency": 1000,
5+
"cgroup_basepath": "/sys/fs/cgroup/",
6+
"output": {
7+
"type": "mongodb",
8+
"uri": "mongodb://mongodb:27017",
9+
"database": "db_sensor",
10+
"collection": "prep"
11+
},
12+
"system": {
13+
"rapl": {
14+
"events": [
15+
"RAPL_ENERGY_PKG"
16+
],
17+
"monitoring_type": "MONITOR_ONE_CPU_PER_SOCKET"
18+
},
19+
"msr": {
20+
"events": [
21+
"TSC",
22+
"APERF",
23+
"MPERF"
24+
]
25+
}
26+
},
27+
"container": {
28+
"core": {
29+
"events": [
30+
"CPU_CLK_THREAD_UNHALTED:REF_P",
31+
"CPU_CLK_THREAD_UNHALTED:THREAD_P",
32+
"LLC_MISSES",
33+
"INSTRUCTIONS_RETIRED"
34+
]
35+
}
36+
}
37+
}

‎docs/script/getting_started/start.py

Lines changed: 334 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,334 @@
1+
# Copyright (c) 2024, INRIA
2+
# Copyright (c) 2024, University of Lille
3+
# All rights reserved.
4+
#
5+
# Redistribution and use in source and binary forms, with or without
6+
# modification, are permitted provided that the following conditions are met:
7+
#
8+
# * Redistributions of source code must retain the above copyright notice, this
9+
# list of conditions and the following disclaimer.
10+
#
11+
# * Redistributions in binary form must reproduce the above copyright notice,
12+
# this list of conditions and the following disclaimer in the documentation
13+
# and/or other materials provided with the distribution.
14+
#
15+
# * Neither the name of the copyright holder nor the names of its
16+
# contributors may be used to endorse or promote products derived from
17+
# this software without specific prior written permission.
18+
#
19+
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20+
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21+
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22+
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23+
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24+
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25+
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26+
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27+
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
29+
30+
import re
31+
import csv
32+
import os
33+
import sys
34+
import subprocess
35+
import json
36+
37+
38+
# List of available processor architectures Template: "n - Arch name"
39+
# If an arch is added, the case statement in the
40+
# start_demo function should be updated accordingly with the proper core events
41+
# https://powerapi.org/reference/sensors/hwpc-sensor/
42+
arch_tab = [["Sandy bridge", "Ivy bridge", "Haswell", "Broadwell", "Comet lake"],
43+
["Skylake", "Cascade lake", "Kaby Lake R", "Kaby Lake", "Coffee Lake", "Amber Lake", "Rocket lake", "Whiskey lake"],
44+
["Zen", "Zen+", "Zen 2"],
45+
["Zen 3", "Zen 4"]]
46+
47+
48+
def docker_start(time):
49+
"""
50+
Start the docker compose stack and the logs
51+
:param time: The duration of the demo
52+
"""
53+
id1 = os.popen("id -u").read()
54+
id2 = os.popen("id -g").read()
55+
with open('env_template', 'r', encoding='UTF-8') as firstfile, open('.env', 'a', encoding='UTF-8') as secondfile:
56+
for line in firstfile:
57+
secondfile.write(line)
58+
secondfile.write("UID=" + id1)
59+
secondfile.write("GUID=" + id2)
60+
os.system("docker compose up -d")
61+
os.system("docker compose logs sensor -f &")
62+
os.system("docker compose logs formula -f &")
63+
os.system("sleep " + str(time))
64+
docker_stop()
65+
66+
67+
def docker_stop():
68+
"""
69+
Stop the docker compose stack and clean the environment
70+
"""
71+
os.system("set -ueo pipefail")
72+
os.system("set +x")
73+
os.system("docker compose down")
74+
open('.env', 'w', encoding='UTF-8').close()
75+
76+
77+
def load_data():
78+
"""
79+
Load CSV files from the specified directory and return the data as a list of dictionaries.
80+
"""
81+
data = []
82+
with open("./cpu.csv", mode='r', newline='', encoding='UTF-8') as f:
83+
data.extend(csv.DictReader(f))
84+
return data
85+
86+
87+
def load_result(directory='./csv'):
88+
"""
89+
Load CSV files from the specified directory and return the data as a list of dictionaries.
90+
"""
91+
data = []
92+
for root, _, files in os.walk(directory):
93+
for filename in files:
94+
if filename.endswith('.csv'):
95+
file_path = os.path.join(root, filename)
96+
with open(file_path, mode='r', newline='', encoding='UTF-8') as f:
97+
data.extend(csv.DictReader(f))
98+
return data
99+
100+
101+
def calculate_statistics(data, scope):
102+
"""
103+
Calculate average, maximum, and minimum consumption for the given scope (cpu or dram).
104+
"""
105+
stats = {}
106+
for row in data:
107+
if row['scope'] == scope and row['target'] != 'rapl':
108+
target = row['target']
109+
consumption = float(row['power'])
110+
stats.setdefault(target, []).append(consumption)
111+
112+
return {
113+
target: {
114+
'avg': sum(consumptions) / len(consumptions),
115+
'max': max(consumptions),
116+
'min': min(consumptions)
117+
} for target, consumptions in stats.items()
118+
}
119+
120+
121+
def print_statistics(stats, title):
122+
"""
123+
Print statistics (average, max, min) for the given data in a formatted table.
124+
"""
125+
if not stats:
126+
return
127+
128+
print(f"\n{title}\n")
129+
print(f"{'Target':<20} {'Average consumption':<20} {'Maximum consumption':<20} {'Minimum consumption':<20}")
130+
print("=" * 80)
131+
132+
total = {'avg': 0, 'max': 0, 'min': 0}
133+
for target, values in stats.items():
134+
if target != 'global':
135+
print(f"{target:<20} {values['avg']:<20.2f} {values['max']:<20.2f} {values['min']:<20.2f}")
136+
else:
137+
total = values
138+
139+
print("-" * 80)
140+
print(f"{'Global':<20} {total['avg']:<20.2f} {total['max']:<20.2f} {total['min']:<20.2f}")
141+
142+
143+
def start_pretty_print():
144+
"""
145+
Pretty print the CPU and DRAM power consumption statistics from CSV files.
146+
"""
147+
print("The consumptions are given in Watt, note that the precision depends on the configuration file\n")
148+
149+
data = load_result()
150+
151+
# Calculate and print CPU statistics
152+
cpu_stats = calculate_statistics(data, 'cpu')
153+
print_statistics(cpu_stats, "CPU Consumption Statistics :")
154+
155+
# Calculate and print DRAM statistics
156+
dram_stats = calculate_statistics(data, 'dram')
157+
print_statistics(dram_stats, "DRAM Consumption Statistics :")
158+
159+
# Could add the GPU statistics here
160+
161+
print("\nFor more precise evaluation, consult the PowerAPI documentation to adjust configurations.\n")
162+
163+
164+
def find_cpu(data):
165+
"""
166+
Find the cpu in the list of compatible cpu
167+
"""
168+
option = []
169+
line = "cat /proc/cpuinfo | grep 'model name'"
170+
result = subprocess.check_output(line, shell=True, text=True).split("\n")
171+
print("The CPU found is" + result[0].split(":")[1])
172+
parse = parse_processor_name(result[0])
173+
for row in data:
174+
if parse[0] in row["Name"] and row["Manufacturer"] == parse[1]:
175+
option.append(row)
176+
177+
if len(option) == 0:
178+
print("It looks like you cpu is not supported by PowerAPI")
179+
sys.exit()
180+
elif len(option) == 1:
181+
print("Your CPU should be supported by PowerAPI")
182+
cpu = option[0]
183+
else:
184+
print("Please select your CPU from this list")
185+
for i in range(len(option)):
186+
print(str(i) + " - " + option[i]["Name"])
187+
choice = int(input())
188+
print("You have selected : " + option[choice]["Name"])
189+
cpu = option[choice]
190+
return cpu
191+
192+
193+
def parse_processor_name(name):
194+
"""
195+
Parse the processor name to extract the id and the brand
196+
:param name: Name of the processor, extracted from /proc/cpuinfo
197+
:return: id adn brand of the processor
198+
"""
199+
if "Intel" in name:
200+
brand = "Intel"
201+
elif "AMD" in name:
202+
brand = "AMD"
203+
else:
204+
brand = "Unknown"
205+
id_pattern = r"\b\d{3,4}[A-Z0-9]*\b"
206+
207+
id_res = re.search(id_pattern, name)
208+
209+
return id_res.group(), brand
210+
211+
212+
def start_demo():
213+
"""
214+
Start the demo by selecting the processor architecture
215+
this will update the sensor configuration file
216+
"""
217+
print("PowerAPI demo")
218+
print("=" * 80)
219+
cpu = find_cpu(load_data())
220+
221+
print("\nSetting up configuration files...")
222+
print("-" * 80)
223+
# Update core events in the sensor configuration
224+
# file based on the selected processor architecture
225+
with open('sensor/hwpc-mongodb.json', encoding='UTF-8') as f:
226+
data = json.load(f)
227+
228+
if cpu["Family"] in arch_tab[0]:
229+
data['container']['core']['events'] = [
230+
"CPU_CLK_UNHALTED:REF_P",
231+
"CPU_CLK_UNHALTED:THREAD_P",
232+
"LLC_MISSES",
233+
"INSTRUCTIONS_RETIRED"
234+
]
235+
elif cpu["Family"] in arch_tab[1]:
236+
data['container']['core']['events'] = [
237+
"CPU_CLK_THREAD_UNHALTED:REF_P",
238+
"CPU_CLK_THREAD_UNHALTED:THREAD_P",
239+
"LLC_MISSES",
240+
"INSTRUCTIONS_RETIRED"
241+
]
242+
elif cpu["Family"] in arch_tab[2]:
243+
data['container']['core']['events'] = [
244+
"CYCLES_NOT_IN_HALT",
245+
"RETIRED_INSTRUCTIONS",
246+
"RETIRED_UOPS"
247+
]
248+
elif cpu["Family"] in arch_tab[3]:
249+
data['container']['core']['events'] = [
250+
"CYCLES_NOT_IN_HALT",
251+
"RETIRED_INSTRUCTIONS",
252+
"RETIRED_OPS"
253+
]
254+
255+
print("Core events updated")
256+
257+
# Check for the cgroup version and update
258+
# the sensor configuration file accordingly
259+
cgroup = subprocess.run(["stat", "-fc", "%T", "/sys/fs/cgroup/"],
260+
text=True, capture_output=True, check=True)
261+
if cgroup.stdout == "cgroup2fs\n":
262+
data["cgroup_basepath"] = "/sys/fs/cgroup/"
263+
else:
264+
data["cgroup_basepath"] = "/sys/fs/cgroup/perf_event"
265+
266+
print("Cgroup version updated")
267+
268+
with open('sensor/hwpc-mongodb.json', 'w', encoding='UTF-8') as f:
269+
json.dump(data, f, indent=4)
270+
271+
# Update parameters in the formula configuration
272+
with open('formula/smartwatts-mongodb-csv.json', encoding='UTF-8') as f:
273+
formula_config = json.load(f)
274+
275+
if cpu["Base frequency"] != '':
276+
formula_config["cpu-base-freq"] = int(float(cpu["Base frequency"])*1000)
277+
print("Base frequency updated")
278+
279+
if cpu["TDP"] != '':
280+
formula_config["cpu-tdp"] = int(cpu["TDP"][:-1])
281+
print("TDP updated\n")
282+
283+
with open('formula/smartwatts-mongodb-csv.json', 'w', encoding='UTF-8') as f:
284+
json.dump(formula_config, f, indent=4)
285+
286+
print("Please enter the number of second you want the demo to run for (minimum 30) or exit to quit:")
287+
waiting = True
288+
while waiting:
289+
try:
290+
val = input()
291+
val = int(val)
292+
if val < 30:
293+
print("Invalid input, please enter a valid number or exit to quit")
294+
else:
295+
waiting = False
296+
except ValueError:
297+
if val == "exit":
298+
print("Exiting...")
299+
sys.exit()
300+
else:
301+
print("Invalid input, please enter a valid number or exit to quit")
302+
303+
print("\nStarting the demo...")
304+
print("-" * 80)
305+
print("The demo will run for " + str(val) + " seconds\n")
306+
print("If you wish to stop it, Ctrl-C will do so and stop the docker compose stack\n")
307+
308+
docker_start(val)
309+
310+
verification = 0
311+
312+
# Get all the csv power report in the csv directory
313+
for root, _, files in os.walk('./csv'):
314+
for filename in files:
315+
if filename.endswith('.csv'):
316+
verification += 1
317+
file_path = os.path.join(root, filename)
318+
print("The power report is available at: " + file_path)
319+
320+
if verification == 0:
321+
print("\nNo power report available, "
322+
"please check the configuration file "
323+
"and the sensor availability for your "
324+
"processor architecture\n")
325+
else:
326+
print("\nThe demo has ended, "
327+
"you can see the result under the /csv directory, but"
328+
" here is a quick summary\n")
329+
330+
start_pretty_print()
331+
332+
333+
if __name__ == '__main__':
334+
start_demo()

‎docs/script/smartwatts_auto_config.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ echo "
1717
"stream": true,
1818
"input": {
1919
"puller": {
20-
"model": "HWPCReport",
20+
"model": "HWPC Report",
2121
"type": "socket",
2222
"uri": "127.0.0.1",
2323
"port": 8080,

‎mkdocs.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -84,9 +84,9 @@ nav:
8484
- Formulas:
8585
#- RAPL: reference/formulas/rapl.md
8686
- SmartWatts: reference/formulas/smartwatts.md
87-
- Configuration Files: reference/formulas/configuration_files.md
88-
- Sources/Destinations:
89-
- Description: reference/database/sources_destinations.md
87+
#- Configuration Files: reference/formulas/configuration_files.md
88+
# - Sources/Destinations:
89+
# - Description: reference/database/sources_destinations.md
9090
- Processors:
9191
- Description: reference/processors/processors.md
9292
- Reports:

0 commit comments

Comments
 (0)
Please sign in to comment.