Running Qwen3 in Android LlamaDemo App shows an error while loading tokenizer

[Arpit2601]

🐛 Describe the bug

I'm trying to run Qwen3 0.6B model in Android LlamaDemo app by following the steps from docs and getting the following error in logcat.

E0000 00:00:1748953425.022463    9321 re2.cc:237] Error parsing '((?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s...': invalid perl operator: (?!

Steps to convert model:

python3 -m examples.models.llama.export_llama \
  --model qwen3-0_6b \
  --params examples/models/qwen3/0_6b_config.json \
  -kv \
  --use_sdpa_with_kv_cache \
  -d fp32 \
  -X \
  --xnnpack-extended-ops \
  -qmode 8da4w \
  --metadata '{"get_bos_id": 151644, "get_eos_ids":[151645]}' \
  --output_name="qwen3-0_6b.pte" \
  --verbose \
  --max_context_length 2048 \
  --max_seq_length 2048

Steps to build android library from main branch (d4906e2):

1. ./install_executorch.sh --clean
2. sh examples/demo-apps/android/LlamaDemo/setup.sh

Android NDK version: 28.0.12916984

There is a similar issue which mentions to add -DSUPPORT_REGEX_LOOKAHEAD=ON cmake argument. But I can see that this is already present in build_android_library.sh.

Am I missing something, do I have to add this cmake argument somewhere else.

Versions

Output of collect_env.py.

Collecting environment information...
PyTorch version: 2.8.0.dev20250422
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 14.6.1 (arm64)
GCC version: Could not collect
Clang version: 16.0.0 (clang-1600.0.26.6)
CMake version: version 3.31.6
Libc version: N/A

Python version: 3.10.11 (v3.10.11:7d4cc5aa85, Apr  4 2023, 19:05:19) [Clang 13.0.0 (clang-1300.0.29.30)] (64-bit runtime)
Python platform: macOS-14.6.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1

Versions of relevant libraries:
[pip3] executorch==0.7.0a0+d4906e2
[pip3] numpy==2.2.1
[pip3] onnx==1.15.0
[pip3] onnxconverter-common==1.14.0
[pip3] onnxruntime==1.13.1
[pip3] pytorch-lightning==2.4.0
[pip3] pytorch-tokenizers==0.1.0
[pip3] torch==2.8.0.dev20250422
[pip3] torchao==0.12.0+git214e7047
[pip3] torchaudio==2.6.0.dev20250422
[pip3] torchdata==0.11.0
[pip3] torchmetrics==1.6.0
[pip3] torchsr==1.0.4
[pip3] torchtune==0.6.1
[pip3] torchvision==0.22.0.dev20250422
[conda] Could not collect

cc @mergennachin @iseeyuan @lucylq @helunwencser @tarun292 @kimishpatel @jackzhxng @kirklandsign @cbilgin @larryliu0820 @cccclai

[jackzhxng]

@kirklandsign

[kirklandsign]

Curious where did you get the tokenizer file?

[kirklandsign]

It should be a json file like ~/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/a9c98e602b9d36d2a2f7ba1eb0f5f31e4e8e5143/tokenizer.json

[Arpit2601]

I took the tokenizer.json file from https://huggingface.co/Qwen/Qwen3-0.6B/blob/main/tokenizer.json

[Arpit2601]

@kirklandsign as suggested tried out json file present at ~/.cache/huggingface/hub/models--Qwen--Qwen3-0.6B/snapshots/a9c98e602b9d36d2a2f7ba1eb0f5f31e4e8e5143/tokenizer.json, but still getting the same error. Is there anything else I should try out.