Skip to content

benchmark: remove cycle counter #88

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 7, 2017
+59 −104
Merged

benchmark: remove cycle counter #88

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5708f0a
benchmark: remove cycle counter
gilles-peskine-arm Mar 29, 2017
486f2c6
Remove cycle counter regexp from benchmark CI file
Jun 6, 2017
9aca063
Update benchmark docs due to cycle counter removal
Jun 7, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions benchmark/README.md
View file
Original file line number Diff line number Diff line change
@@ -13,21 +13,21 @@ You can also compile this example with the [mbed Online Compiler](https://develo
The output in the terminal window should be similar to this:

```
SHA-256 : 1673 KB/s, 70 cycles/byte
SHA-512 : 546 KB/s, 215 cycles/byte
AES-CBC-128 : 1428 KB/s, 82 cycles/byte
AES-CBC-192 : 1260 KB/s, 93 cycles/byte
AES-CBC-256 : 1127 KB/s, 104 cycles/byte
AES-GCM-128 : 486 KB/s, 242 cycles/byte
AES-GCM-192 : 464 KB/s, 253 cycles/byte
AES-GCM-256 : 445 KB/s, 264 cycles/byte
AES-CCM-128 : 610 KB/s, 192 cycles/byte
AES-CCM-192 : 547 KB/s, 214 cycles/byte
AES-CCM-256 : 496 KB/s, 237 cycles/byte
CTR_DRBG (NOPR) : 1139 KB/s, 102 cycles/byte
CTR_DRBG (PR) : 826 KB/s, 142 cycles/byte
HMAC_DRBG SHA-256 (NOPR) : 193 KB/s, 611 cycles/byte
HMAC_DRBG SHA-256 (PR) : 170 KB/s, 695 cycles/byte
SHA-256 : 1673 KB/s
SHA-512 : 546 KB/s
AES-CBC-128 : 1428 KB/s
AES-CBC-192 : 1260 KB/s
AES-CBC-256 : 1127 KB/s
AES-GCM-128 : 486 KB/s
AES-GCM-192 : 464 KB/s
AES-GCM-256 : 445 KB/s
AES-CCM-128 : 610 KB/s
AES-CCM-192 : 547 KB/s
AES-CCM-256 : 496 KB/s
CTR_DRBG (NOPR) : 1139 KB/s
CTR_DRBG (PR) : 826 KB/s
HMAC_DRBG SHA-256 (NOPR) : 193 KB/s
HMAC_DRBG SHA-256 (PR) : 170 KB/s
RSA-2048 : 28 ms/ public
RSA-2048 : 953 ms/private
RSA-4096 : 93 ms/ public
26 changes: 3 additions & 23 deletions benchmark/main.cpp
View file
Original file line number Diff line number Diff line change
@@ -180,25 +180,12 @@
mbedtls_printf( "FAILED: -0x%04x\r\n", -ret );
#endif

static unsigned long mbedtls_timing_hardclock( void )
{
static int dwt_started = 0;

if( dwt_started == 0 )
{
CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk;
DWT->CTRL |= DWT_CTRL_CYCCNTENA_Msk;
}

return( DWT->CYCCNT );
}

static volatile int alarmed;
static void alarm() { alarmed = 1; }

#define TIME_AND_TSC( TITLE, CODE ) \
do { \
unsigned long i, j, tsc; \
unsigned long i; \
Timeout t; \
\
mbedtls_printf( HEADER_FORMAT, TITLE ); \
@@ -208,15 +195,8 @@ do { \
CODE; \
} \
\
tsc = mbedtls_timing_hardclock(); \
for( j = 0; j < 1024; j++ ) \
{ \
CODE; \
} \
\
mbedtls_printf( "%9lu KB/s, %9lu cycles/byte\r\n", \
i * BUFSIZE / 1024, \
( mbedtls_timing_hardclock() - tsc ) / ( j * BUFSIZE ) ); \
mbedtls_printf( "%9lu KB/s\r\n", \
i * BUFSIZE / 1024 ); \
} while( 0 )

#if defined(MBEDTLS_MEMORY_BUFFER_ALLOC_C) && defined(MBEDTLS_MEMORY_DEBUG)
77 changes: 26 additions & 51 deletions tests/README.md
View file
Original file line number Diff line number Diff line change
@@ -99,57 +99,32 @@ Please observe above that all the lines that have data that changes from executi
Another example with regular examples is shown below:

```

SHA-256 :\s*\d+ Kb/s,\s*\d+ cycles/byte | SHA-256 : 1922 Kb/s, 61 cycl

SHA-512 :\s*\d+ Kb/s,\s*\d+ cycles/byte | SHA-512 : 614 Kb/s, 191 cycl

AES-CBC-128 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-CBC-128 : 1401 Kb/s, 83 cycl

AES-CBC-192 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-CBC-192 : 1231 Kb/s, 95 cycl

AES-CBC-256 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-CBC-256 : 1097 Kb/s, 106 cycl

AES-GCM-128 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-GCM-128 : 429 Kb/s, 273 cycl

AES-GCM-192 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-GCM-192 : 412 Kb/s, 285 cycl

AES-GCM-256 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-GCM-256 : 395 Kb/s, 297 cycl

AES-CCM-128 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-CCM-128 : 604 Kb/s, 194 cycl

AES-CCM-192 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-CCM-192 : 539 Kb/s, 217 cycl

AES-CCM-256 :\s*\d+ Kb/s,\s*\d+ cycles/byte | AES-CCM-256 : 487 Kb/s, 241 cycl

CTR_DRBG \(NOPR\) :\s*\d+ Kb/s,\s*\d+ cycles/byte | CTR_DRBG (NOPR) : 1145 Kb/s, 102 cycl

CTR_DRBG \(PR\) :\s*\d+ Kb/s,\s*\d+ cycles/byte | CTR_DRBG (PR) : 821 Kb/s, 142 cycl

HMAC_DRBG SHA-256 \(NOPR\) :\s*\d+ Kb/s,\s*\d+ cycles/byte | HMAC_DRBG SHA-256 (NOPR) : 219 Kb/s, 537 cycl

HMAC_DRBG SHA-256 \(PR\) :\s*\d+ Kb/s,\s*\d+ cycles/byte | HMAC_DRBG SHA-256 (PR) : 193 Kb/s, 612 cycl

RSA-2048 :\s*\d+ ms/ public | RSA-2048 : 30 ms/ public

RSA-2048 :\s*\d+ ms/private | RSA-2048 : 1054 ms/private

RSA-4096 :\s*\d+ ms/ public | RSA-4096 : 101 ms/ public

RSA-4096 :\s*\d+ ms/private | RSA-4096 : 5790 ms/private

ECDHE-secp384r1 :\s*\d+ ms/handshake | ECDHE-secp384r1 : 1023 ms/handshake

ECDHE-secp256r1 :\s*\d+ ms/handshake | ECDHE-secp256r1 : 678 ms/handshake

ECDHE-Curve25519 :\s*\d+ ms/handshake | ECDHE-Curve25519 : 580 ms/handshake

ECDH-secp384r1 :\s*\d+ ms/handshake | ECDH-secp384r1 : 503 ms/handshake

ECDH-secp256r1 :\s*\d+ ms/handshake | ECDH-secp256r1 : 336 ms/handshake

ECDH-Curve25519 :\s*\d+ ms/handshake | ECDH-Curve25519 : 300 ms/handshake

SHA-256 :\s*\d+ KB/s | SHA-256 : 1922 KB/s
SHA-512 :\s*\d+ KB/s | SHA-512 : 614 KB/s
AES-CBC-128 :\s*\d+ KB/s | AES-CBC-128 : 1401 KB/s
AES-CBC-192 :\s*\d+ KB/s | AES-CBC-192 : 1231 KB/s
AES-CBC-256 :\s*\d+ KB/s | AES-CBC-256 : 1097 KB/s
AES-GCM-128 :\s*\d+ KB/s | AES-GCM-128 : 429 KB/s
AES-GCM-192 :\s*\d+ KB/s | AES-GCM-192 : 412 KB/s
AES-GCM-256 :\s*\d+ KB/s | AES-GCM-256 : 395 KB/s
AES-CCM-128 :\s*\d+ KB/s | AES-CCM-128 : 604 KB/s
AES-CCM-192 :\s*\d+ KB/s | AES-CCM-192 : 539 KB/s
AES-CCM-256 :\s*\d+ KB/s | AES-CCM-256 : 487 KB/s
CTR_DRBG \(NOPR\) :\s*\d+ KB/s | CTR_DRBG (NOPR) : 1145 KB/s
CTR_DRBG \(PR\) :\s*\d+ KB/s | CTR_DRBG (PR) : 821 KB/s
HMAC_DRBG SHA-256 \(NOPR\) :\s*\d+ KB/s | HMAC_DRBG SHA-256 (NOPR) : 219 KB/s
HMAC_DRBG SHA-256 \(PR\) :\s*\d+ KB/s | HMAC_DRBG SHA-256 (PR) : 193 KB/s
RSA-2048 :\s*\d+ ms/ public | RSA-2048 : 30 ms/ public
RSA-2048 :\s*\d+ ms/private | RSA-2048 : 1054 ms/private
RSA-4096 :\s*\d+ ms/ public | RSA-4096 : 101 ms/ public
RSA-4096 :\s*\d+ ms/private | RSA-4096 : 5790 ms/private
ECDHE-secp384r1 :\s*\d+ ms/handshake | ECDHE-secp384r1 : 1023 ms/handshake
ECDHE-secp256r1 :\s*\d+ ms/handshake | ECDHE-secp256r1 : 678 ms/handshake
ECDHE-Curve25519 :\s*\d+ ms/handshake | ECDHE-Curve25519 : 580 ms/handshake
ECDH-secp384r1 :\s*\d+ ms/handshake | ECDH-secp384r1 : 503 ms/handshake
ECDH-secp256r1 :\s*\d+ ms/handshake | ECDH-secp256r1 : 336 ms/handshake
ECDH-Curve25519 :\s*\d+ ms/handshake | ECDH-Curve25519 : 300 ms/handshake
DONE | DONE
```

More details about ```htrun``` are [here](https://github.com/ARMmbed/htrun#testing-mbed-os-examples).
30 changes: 15 additions & 15 deletions tests/benchmark.log
View file
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
SHA-256 :\s*\d+ KB/s,\s*\d+ cycles/byte
SHA-512 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-CBC-128 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-CBC-192 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-CBC-256 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-GCM-128 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-GCM-192 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-GCM-256 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-CCM-128 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-CCM-192 :\s*\d+ KB/s,\s*\d+ cycles/byte
AES-CCM-256 :\s*\d+ KB/s,\s*\d+ cycles/byte
CTR_DRBG \(NOPR\) :\s*\d+ KB/s,\s*\d+ cycles/byte
CTR_DRBG \(PR\) :\s*\d+ KB/s,\s*\d+ cycles/byte
HMAC_DRBG SHA-256 \(NOPR\) :\s*\d+ KB/s,\s*\d+ cycles/byte
HMAC_DRBG SHA-256 \(PR\) :\s*\d+ KB/s,\s*\d+ cycles/byte
SHA-256 :\s*\d+ KB/s
SHA-512 :\s*\d+ KB/s
AES-CBC-128 :\s*\d+ KB/s
AES-CBC-192 :\s*\d+ KB/s
AES-CBC-256 :\s*\d+ KB/s
AES-GCM-128 :\s*\d+ KB/s
AES-GCM-192 :\s*\d+ KB/s
AES-GCM-256 :\s*\d+ KB/s
AES-CCM-128 :\s*\d+ KB/s
AES-CCM-192 :\s*\d+ KB/s
AES-CCM-256 :\s*\d+ KB/s
CTR_DRBG \(NOPR\) :\s*\d+ KB/s
CTR_DRBG \(PR\) :\s*\d+ KB/s
HMAC_DRBG SHA-256 \(NOPR\) :\s*\d+ KB/s
HMAC_DRBG SHA-256 \(PR\) :\s*\d+ KB/s
RSA-2048 :\s*\d+ ms/ public
RSA-2048 :\s*\d+ ms/private
RSA-4096 :\s*\d+ ms/ public