English | 中文
Before: AI writes the code, then you still have to run the app, tap through flows, copy logs, inspect network traffic, and decide whether it worked.
Now: AI can build and install the app, operate real flows, read real UI state, inspect network and logs, and verify the result itself.
AI App Bridge gives autonomous AI agents a runtime interface to running Android, iOS, Flutter, WebView/WKWebView, and desktop Web targets. Agents can inspect the current screen, operate native UI and Web content, read View tree / Widget tree / DOM data, collect network requests and logs, verify outcomes, and keep iterating from real evidence.
Its goal is to help AI agents move through an observe -> act -> read results -> verify -> iterate loop, instead of guessing without runtime evidence.
Supported targets:
- Android native apps through the debug runtime plus ADB/UIAutomator for device-level evidence and actions
- Android WebView/H5 through runtime DOM/eval/click/input/wait/scroll and optional DevTools/CDP network/console capture
- Flutter apps on Android and iOS through widget snapshots, operable nodes, runtime actions, text input, scroll, and H5 adapters
- iOS native apps through
AiAppBridgeIOSfor UIKit/WKWebView/logs/network/state/events plus WebDriverAgent/XCUITest for screenshots, UI tree, tap, input, swipe, and system UI - Desktop Web Bridge sessions through the browser SDK for DOM, logs, network, state, events, whitelisted commands, click/input/wait, and scroll
MCP command domains: core (status, tree, uia-tree, screenshot, logs, network, state, events), app (install, clear data, launch, freeze/thaw, permissions, appops), action (tap, input, swipe, keyevent, wait, keyboard), flutter, webview, ios, web, diagnostics, and advanced (batch, port forwarding).
The default MCP surface is compact: call capabilities to discover domains, commands, and options, then call run with the chosen command.
Screenshot-only automation is fragile. For autonomous iteration, an AI agent needs both runtime evidence and a way to act on the running app.
- What screen is currently visible?
- What native View, WebView DOM, and Flutter Widget structure exists?
- Which elements can be tapped, typed into, or scrolled? What scripts can run inside a WebView?
- How can the agent precisely operate UI and enter text?
- What network requests, logs, state changes, and events happened after an action?
- Did the app actually move into the expected state after a code change or runtime action?
android/ai-app-bridge-android Android runtime SDK
android/ai-app-bridge-gradle-plugin Debug build instrumentation plugin
ios/ai-app-bridge-ios iOS Swift runtime SDK
flutter/ai_app_bridge_flutter Flutter plugin
web/ai-app-bridge-web Browser SDK for desktop Web Bridge sessions
desktop/ai-app-bridge-cli Node CLI and MCP stdio server
examples/android-native-sample Clean Android sample app
examples/ios-native-sample Clean iOS sample app for runtime install validation
docs Design, integration, and test notes
- Local bridge status on the first available port starting at
127.0.0.1:18080 - Android View tree, window tree, and screenshots
- Native UI operations, with desktop-side ADB / UIAutomator fallback operations
- iOS UIKit tree, WKWebView DOM/eval, screenshots, and XCUITest/WebDriverAgent actions
- Native Android WebView DOM snapshots, JavaScript evaluation, and debug DevTools/CDP network/console capture
- Flutter Widget snapshots, semantic action metadata, and runtime action handling
- Flutter H5 operations and DOM snapshots through a Dart-side H5 adapter registry
- Desktop Web Bridge sessions with browser SDK DOM/log/network/state/event evidence and whitelisted page commands
- Logs, network requests, state records, and event buffers with incremental
sinceId/sinceMsreads - Debug Gradle plugin support for OkHttp auto capture
- Node CLI / MCP stdio server for connecting AI tools to runtime capabilities
Add the Android runtime SDK to debug builds:
settings.gradle.kts:
dependencyResolutionManagement {
repositories {
google()
mavenCentral()
maven("https://jitpack.io")
}
}app/build.gradle.kts:
dependencies {
debugImplementation("com.github.mobileAiDev.ai-app-bridge:ai-app-bridge-android:0.2.8")
}The runtime SDK starts automatically in debuggable Android apps through its init provider.
The Android runtime supports minSdk 19+.
Optional OkHttp auto capture is provided by the debug Gradle plugin:
settings.gradle.kts:
pluginManagement {
repositories {
google()
mavenCentral()
gradlePluginPortal()
maven("https://jitpack.io")
}
resolutionStrategy {
eachPlugin {
if (requested.id.id == "io.github.mobileaidev.aiappbridge.android") {
useModule("com.github.mobileAiDev.ai-app-bridge:ai-app-bridge-gradle-plugin:${requested.version}")
}
}
}
}app/build.gradle.kts:
plugins {
id("io.github.mobileaidev.aiappbridge.android") version "0.2.8"
}
aiAppBridge {
setOkHttpCaptureEnabled(true)
}The same plugin id selects the AGP backend automatically: AGP 7+ uses Android Components instrumentation, and AGP 4.x uses the legacy Transform API.
Add the Swift runtime to debug builds through Swift Package Manager:
.package(url: "https://github.com/mobileAiDev/ai-app-bridge.git", from: "0.2.11")Start the runtime once in the debug app process:
#if DEBUG
import AiAppBridgeIOS
AiAppBridge.shared.start(appName: "your_ios_app")
#endifInstall the desktop CLI and verify the full-control stack:
npm install -g @mobileaidev/ai-app-bridge
ai-app-bridge ios-doctor --device-id <device-or-udid> --bundle-id <ios.bundle.id>
ai-app-bridge ios-setup --device-id <device-or-udid> --bundle-id <ios.bundle.id> --team-id <APPLE_TEAM_ID> --start-wdaFull iOS control requires Xcode, a trusted/unlocked device with Developer Mode enabled, the app debug runtime, and WebDriverAgent/XCUITest signed and reachable. The CLI vendors appium-webdriveragent and can start it with ios-setup --start-wda --team-id <APPLE_TEAM_ID>, using a unique default WDA bundle id unless --wda-bundle-id is supplied. On physical devices, reuse the WDA URL returned by setup; it may be a CoreDevice tunnel such as http://[fdxx::1]:8100 rather than 127.0.0.1.
Flutter projects only need the pub package. The plugin's Android debug variant automatically includes the ai-app-bridge-android runtime that starts the in-app bridge server; the iOS plugin starts the Swift runtime in the debug app process. Release builds should not expose the debug runtime automatically.
Add the Flutter plugin:
dependencies:
ai_app_bridge_flutter: ^0.2.4Initialize once:
import 'package:ai_app_bridge_flutter/ai_app_bridge_flutter.dart';
import 'package:flutter/widgets.dart';
void main() {
WidgetsFlutterBinding.ensureInitialized();
AiAppBridge.instance.initialize(appName: 'your_app_name');
runApp(const MyApp());
}For Flutter WebView DOM support, register an H5 adapter because the WebView controller lives in Dart:
AiAppBridge.instance.registerH5Adapter(
AiAppBridgeH5Adapter(
id: 'main-webview',
source: 'webview_flutter',
evaluateJavascript: (script) {
return controller.runJavaScriptReturningResult(script);
},
),
);If you are an AI agent, model client, automation framework, or MCP caller reading this README: before using AI App Bridge to operate an app, load the in-repo ai-app-bridge-use usage guide.
- If your client supports skills, project instructions, or custom instructions, load
skills/ai-app-bridge-use/SKILL.mdas the dedicated usage guide. - If your client does not support skills, at minimum include the contents of
skills/ai-app-bridge-use/SKILL.mdin the system prompt, developer prompt, or project instructions for the task. - If you use Codex or another client compatible with Codex skill folders, copy
skills/ai-app-bridge-usefrom this repo into that client's skills directory, then restart or refresh the session.
Install the in-repo skill / usage guide into a Codex-compatible skills directory.
macOS / Linux:
CODEX_HOME="${CODEX_HOME:-$HOME/.codex}"
mkdir -p "$CODEX_HOME/skills"
cp -R skills/ai-app-bridge-use "$CODEX_HOME/skills/ai-app-bridge-use"Windows PowerShell:
New-Item -ItemType Directory -Force "$env:USERPROFILE\.codex\skills" | Out-Null
Copy-Item -LiteralPath "skills\ai-app-bridge-use" -Destination "$env:USERPROFILE\.codex\skills\ai-app-bridge-use" -Recurse -Forcenpm install -g @mobileaidev/ai-app-bridgeAdd this MCP server to your AI agent, model client, or IDE MCP config.
macOS / Linux:
{
"mcpServers": {
"ai-app-bridge": {
"command": "ai-app-bridge-mcp"
}
}
}Windows:
{
"mcpServers": {
"ai-app-bridge": {
"command": "cmd",
"args": ["/c", "ai-app-bridge-mcp"]
}
}
}AI App Bridge exposes runtime inspection and operation surfaces. Wire it into debug builds only. Do not ship it in production / release builds unless you have completed a deliberate security review for your own environment.
AI App Bridge is licensed under the Apache License 2.0.
If you distribute modified versions, keep the license and copyright notices and clearly state that your version is based on or modified from AI App Bridge. See NOTICE.