feat(vm): [Phase 1 - 4] Dalvik VM interpreter — Interpreter Basic, Core Instruction Set, Method Dispatch & Android API Stubs

[haeter525]

Key Changes

Phase 1 - Interpreter Basic

Execution Engine

• engine.py: DalvikVM main loop with call-frame stack and invoke handling
• errors.py: DalvikVM runtime error types
• cmd_run.py: new dextrace run subcommand to run the Dalvik VM interpreter
• _io.py: shared DEX/APK loader; handles APK with missing classes.dex and malformed DEX input

Register File

• register_file.py: Dalvik register file (v0–v15) with bounds checking
• int_ops.py: integer arithmetic with overflow semantics

Call Stack & Frame

• call_frame.py: per-call register snapshot/restore for callee isolation
• state.py: VMState dataclass with pending_result lifecycle for invoke/move-result

Phase 2 - Core Instruction Set Support

• handlers/: arithmetic, array, branch, compare, field, move, type_conv opcode handlers

Phase 3 - Method Dispatch

Vtable Dispatch

• class_hierarchy.py: ClassHierarchy — builds vtable per class via DFS topological sort; provides O(1) resolve_virtual(class_desc, name, proto) → code_off
• engine.py: invoke-virtual handler reads runtime class from heap, dispatches through vtable; new-instance handler allocates object on heap
• dex_class_iter.py: iter_class_defs / iter_class_data_methods — low-level class_def and encoded_method iteration used by vtable builder

Object Heap

• heap.py: ObjectHeap — integer handle → class descriptor; handle 0 reserved as null; reset between run() calls for isolation

Verbose Trace

• cmd_run.py: --verbose flag wires trace_sink into DalvikVM; emits [INFO] messages for new-instance, invoke-virtual dispatch, and entry execution
• cmd_run.py: --dump-regs flag prints non-zero register values after execution

Phase 4 - Android API Stubs & IoC Capture

Stub Registry

• vm/android_stubs/__init__.py: REGISTRY keyed by static callee signature; return-type sentinels (VOID, Value, Wide, ObjectRef); StubCallable protocol so each stub gets (args, heap, api_calls) and records its own trace entry
• vm/android_stubs/sms.py: Landroid/telephony/SmsManager;->getDefault() and sendTextMessage(...) — first concrete IoC-capturing stubs

Engine Wiring

• engine.py: stub dispatch sits in front of vtable resolution and the external-miss path; Policy C external-miss (void → silent no-op, non-void → DexTraceNotImplementedError with full sig + pc); --strict-stubs escalates void misses to errors; vm.run() accepts str args and materializes them as Ljava/lang/String; heap handles
• heap.py: per-handle value slot so stubs can read back analyst-supplied strings via heap.get_value(handle)
• class_hierarchy.py: has_class() probe lets invoke-virtual distinguish in-DEX classes from external ones (external falls through to stub/external-miss instead of vtable miss)

CLI

• cmd_run.py: --arg auto-parses ints (decimal or 0x-hex) and falls back to string (+-prefixed values stay as strings so phone-number IoCs aren't silently downgraded); --args '[...]' for explicit JSON typing of mixed lists; --trace prints captured api_calls as JSON; --strict-stubs flips the engine's external-miss policy

Tests + Sample Fetcher

• tests/test_vm_run_p4.py: runs Ahmyth's SMSManager.sendSMS(String,String)Z end-to-end and asserts the captured trace contains the analyst-supplied phone + message text
• tests/fixtures/p4_ahmyth_fetcher.py: SHA256-locked lazy fetcher ($DEXTRACE_AHMYTH_APK → ~/.cache/dextrace/p4_ahmyth.apk → MalwareBazaar); pytest.skip on FetcherError so offline runs stay green

Docs

• DESIGN.md: adds the P4 [ERROR] unknown Android API: <sig> (pc=...) format and a decision-log entry explaining why analysts need full sig + pc
• TODOS.md: tracks scope intentionally cut from P4/P5/P6 MVPs (StringBuilder/Uri/static fields, IOSocket URL demo, try/catch dispatch, const-string heap migration, invoke-interface, taint tracking, --explain)

How to use

Installation

git clone -b android_emulator_enhance https://github.com/haeter525/DexTrace.git
cd DexTrace
pip install -e .

Phase 1 — execute const/16 v0, #42 → return-void

$ dextrace run tests/fixtures/samples/p1_const_return.dex --entry 'Lp1;->main()V' --dump-regs
return: void
registers: v0=42

• the register value v0 should be 42
• the return value should be void

Phase 2 — execute recursive Fibonacci

$ dextrace run tests/fixtures/samples/p2_fib_recursive.dex --entry 'Lp2/Fib;->fib(I)I' --arg 10
return: 55

• the return value should be 55

Phase 3 — vtable dispatch through inheritance

$ dextrace run tests/fixtures/samples/p3_inheritance.dex --entry 'Lp3/Main;->entry()I'
return: 2

• Lp3/Main;->entry() allocates a Lp3/Mid; object and calls foo() via invoke-virtual
• vtable dispatch resolves Lp3/Base;->foo()I → Lp3/Mid;->foo()I (override), returning 2

$ dextrace run tests/fixtures/samples/p3_inheritance.dex --entry 'Lp3/Main;->entry()I' --verbose
[INFO]  executing Lp3/Main;->entry()I with args=[]
[INFO]  new-instance: Lp3/Mid; → handle #1
[INFO]  invoke-virtual: Lp3/Base;->foo()I → Lp3/Mid;->foo()I
return: 2

Phase 4 — capture an Android RAT's SMS exfil call

$ dextrace run /path/to/Ahmyth.apk \
    --entry 'Lahmyth/mine/king/ahmyth/SMSManager;->sendSMS(Ljava/lang/String;Ljava/lang/String;)Z' \
    --arg 'hello from dextrace p4'
return: 1

Tests

• All tests pass: pytest tests
• Phase 1, Phase 2, Phase 3, and Phase 4 commands above produce the expected output (Phase 4 requires an Ahmyth sample via $DEXTRACE_AHMYTH_APK; otherwise the P4 integration test skips)