Syx snapshot fixes and tuning #112

enisrat · 2025-04-08T06:44:08Z

Fix: CONFIG_SOFTMMU not available in blk
HMP commands for SYX testing & debugging
SYX in TLB streamlining
- We should make proper use of QEMU`s dirty RAM functions
- Integrate with notdirty_write, this is essentially the same as what we want
Add libafl_blk_write to write to blkdev with SYX enabled

The reason is that CONFIG_SOFTMMU is not available for the "block" source set. And it is not required, because "block" will only be built for system mode and tools, which we can deactivate using configure.

…me DEBUG print statements

…pshot restore Also flush TLB and TB more specifically

rmalmain · 2025-05-16T04:17:15Z

accel/tcg/cputlb.c

    return flags;
 }

+//// --- Begin LibAFL code ---
+// Use this snippet multiple times below
+#define SYX_SNAPSHOT_DIRTY_LIST_ADD_HOSTADDR_PROBE(dbg, access_type, addr, entry_full, phost) { \


this should be moved to top of this file

rmalmain · 2025-05-16T04:23:52Z

block/block-backend.c

@@ -29,9 +29,8 @@
 #include "migration/misc.h"

 //// --- Begin LibAFL code ---
-#ifdef CONFIG_SOFTMMU
+//#define CONFIG_DEBUG_SYX


rmalmain · 2025-05-16T04:33:59Z

include/libafl/syx-snapshot/syx-snapshot.h

@@ -16,6 +16,7 @@
 #include "device-save.h"
 #include "syx-cow-cache.h"

+//#define CONFIG_DEBUG_SYX


clean up this

rmalmain · 2025-05-16T04:34:45Z

libafl/blkdev.c

+    *(int *)opaque = ret;
+}
+
+int libafl_blk_write(BlockBackend *blk, void *buf, int64_t offset, int64_t sz)


why do we need this? i don't see it used anywhere else in your pr.

Ah yes you are right, it is not related directly to SYX.

I placed this in system.h.
I use it a lot in my branches for writing the fuzzing input to blockdev. The write is catched by SYX.
Often in embedded firmwares, fuzzing blocks of Flash or EMMC (firmware headers, partition tables, structures, etc.)
Think of it as cpu_physical_memory_rw for blockdev.

This is only low-level api yet, I can remove it of course.

rmalmain · 2025-05-16T04:35:10Z

libafl/syx-snapshot/syx-cow-cache.c

@@ -227,9 +227,12 @@ void syx_cow_cache_read_entry(SyxCowCache* scc, BlockBackend* blk,
    if (!QTAILQ_EMPTY(&scc->layers)) {
        for (; qiov_offset < qiov->size;
             blk_offset += chunk_size, qiov_offset += chunk_size) {
+            int i = 0;


clean up this

rmalmain · 2025-05-16T04:35:16Z

libafl/syx-snapshot/syx-cow-cache.c

            QTAILQ_FOREACH(layer, &scc->layers, next)
            {
                chunk_size = layer->chunk_size;
+                //printf("[SYX] check cache layer %d\n", i);


clean up this

rmalmain · 2025-05-16T04:35:23Z

libafl/syx-snapshot/syx-cow-cache.c

@@ -217,8 +218,7 @@ void syx_cow_cache_read_entry(SyxCowCache* scc, BlockBackend* blk,
    size_t qiov_offset = 0;
    uint64_t chunk_size = 0;

-    // printf("[%s] Read 0x%zx bytes @addr %lx\n", blk_name(blk), qiov->size,
-    // offset);
+    //printf("[SYX] [%s] Read 0x%zx bytes @addr %lx\n", blk_name(blk), qiov->size, offset);


clean up this

rmalmain · 2025-05-16T04:36:41Z

libafl/syx-snapshot/syx-hmp.c

@@ -0,0 +1,34 @@
+#include "libafl/syx-snapshot/syx-snapshot.h"


it this useful? for full snapshots i'd use qemu snapshots instead, which are already available with hmp.
may be useful for debugging maybe?

It is for debugging only. I used it to resolve the issues fixed in this PR. I have a personal branch with more features and also many more HMP commands.
As SYX (still don't know what that stands for ;-) ) is quite complicated I cannot debug it via the fuzzer lib. I usually compile the brigde stand-alone and do manual tests.

rmalmain · 2025-05-16T04:44:28Z

accel/tcg/cputlb.c

@@ -1406,16 +1412,19 @@ static int probe_access_internal(CPUState *cpu, vaddr addr,

    /* Everything else is RAM. */
    *phost = (void *)((uintptr_t)addr + entry->addend);
-//// --- Begin LibAFL code ---


why move this to higher-level functions?

i start to think we should not keep this call in probe, there are many places where memory gets probed for write but it in fact never written to.

it's only a matter of optimization though, nothing too problematic.

The probes are used by guest helpers. Even though not every probe may lead to an actual write, we have no way to get this feedback as of now. This is how the QEMU TCG is set up.

A rule of thumb for me was the following:
Whenever QEMU uses notdirty_write, we want to make a backup of the page (syx_snapshot_dirty_list_add_hostaddr). Because these are mostly the same features.

But we cannot use notdirty_write, it misses hostaddr, so I came up with this.

enisrat · 2025-05-17T16:15:14Z

Hi, thanks for the review!

I did some cleanup.

Also, please note that I removed CONFIG_SOFTMMU from block-backend.c because CONFIG_SOFTMMU is not available in block_ss.
This means that before this SYX features was not even compiled.
Now the QEMU block tools and tests will not compile anymore, so we have to use --disable-tools --disable-tests. If that is a problem we could address it in a future PR.

enisrat added 11 commits April 8, 2025 08:11

[add] HMP command for Syx snapshot

7462199

[add] debug messages to syx

12ddfc3

[fix+add] debug syx and TLB

fdb2844

[add] flush TLB on new snapshot tracking

5ea6158

[fix] rename syx-snapshot-hmp.c -> syx-hmp.c

ee8ec01

[add] libafl_blk_write. Also remove CONFIG_SOFTMMU from block-backend.c

5b0a3cc

The reason is that CONFIG_SOFTMMU is not available for the "block" source set. And it is not required, because "block" will only be built for system mode and tools, which we can deactivate using configure.

[fix+debug] Fix syx_snapshot_state.before_fuzz_cache handling. Add so…

596b0a1

…me DEBUG print statements

[fix] Mark RAM as notdirty before tracking new snapshot, or after sna…

80791c9

…pshot restore Also flush TLB and TB more specifically

[fix] SYX: keep in line with notdirty_write

8d8f145

[add] SYX: DMA flatview_write_continue_step

6c4e904

[fix] Cleanup and remove debug printfs

d2936b1

enisrat mentioned this pull request Apr 8, 2025

Use of syx_snapshot_dirty_list_add_hostaddr in cputlb.c #105

Open

rmalmain reviewed May 16, 2025

View reviewed changes

[fix] cleanup

99f3ff6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Syx snapshot fixes and tuning #112

Syx snapshot fixes and tuning #112

Uh oh!

enisrat commented Apr 8, 2025 •

edited

Loading

Uh oh!

rmalmain May 16, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

enisrat May 17, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

enisrat May 17, 2025

Uh oh!

rmalmain May 16, 2025

Uh oh!

enisrat May 17, 2025

Uh oh!

enisrat commented May 17, 2025

Uh oh!

Uh oh!

		@@ -0,0 +1,34 @@
		#include "libafl/syx-snapshot/syx-snapshot.h"

Syx snapshot fixes and tuning #112

Are you sure you want to change the base?

Syx snapshot fixes and tuning #112

Uh oh!

Conversation

enisrat commented Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

enisrat commented May 17, 2025

Uh oh!

Uh oh!

enisrat commented Apr 8, 2025 •

edited

Loading