Android Integration Guide

This guide explains how to integrate the Ink Engine into your Android application using InkView.

Prerequisites

• Android Studio (Ladybug or later)
• Android SDK (API Level 24+)
• NDK (r25c or later)
• Rust Toolchain (1.70+)

1. Add Dependencies

First, you need to obtain the Ink Android library (AAR).

1. Download the latest AAR artifact from the Android Build Workflow.
2. Place the downloaded AAR file (e.g., ink-debug.aar) into your app's libs directory.

Gradle Setup

In your app's build.gradle.kts:

dependencies {
    // The core Ink Android library
    implementation(files("libs/ink-debug.aar"))
    
    // Used for efficient CBOR serialization for IPC between Java and Native
    implementation("com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.14.2")
    
    // Provides Kotlin support for Jackson serialization
    implementation("com.fasterxml.jackson.module:jackson-module-kotlin:2.14.2")
}

2. Add InkView to Layout

You can add InkView directly to your XML layout file or create it programmatically.

Custom Attributes (Optional)

To enable configuring directories via XML, you need to define the attributes in your project's res/values/attrs.xml:

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <declare-styleable name="InkView">
        <attr name="runtimeDir" format="string" />
        <attr name="dataDir" format="string" />
    </declare-styleable>
</resources>

XML Layout

You can configure runtimeDir (for extraction) and dataDir (for persistent data) directly in XML.

<com.rokid.jsui.ink.InkView
    android:id="@+id/inkView"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:runtimeDir="/sdcard/ink/runtime"
    app:dataDir="/sdcard/ink/data" />

Note: If not provided, InkView will use context.cacheDir + "/ink_runtime" and context.filesDir + "/ink_data" by default.

Programmatic Creation

When creating InkView programmatically, you can either use the default directories or provide custom ones via the constructor. nativeEngineInit is called automatically in the constructor.

Using Default Directories:

val inkView = InkView(context)
layout.addView(inkView)

Using Custom Directories:

val inkView = InkView(
    context = context,
    runtimeDir = "/sdcard/ink/runtime",
    dataDir = "/sdcard/ink/data"
)
layout.addView(inkView)

3. Initialize and Open Mini Program

In your Activity or Fragment, initialize InkView and open a mini program.

class MainActivity : AppCompatActivity() {
    private lateinit var inkView: InkView

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        inkView = findViewById(R.id.inkView)
        
        // Open a mini program from a local directory
        val appPath = filesDir.absolutePath + "/mini-app"
        inkView.open(appPath)
    }
}

4. Implement System Capabilities

Ink relies on the host application to provide system capabilities like Media, Network, and Speech. You need to implement the corresponding interfaces.

4.1 Implement Capabilities

Here is an example of implementing the MediaCapability interface to handle media requests. Other capabilities (Networking, Speech, Audio) follow a similar pattern.

inkView.addCapability(object : InkView.MediaCapability {
    override fun startAudioRecording(
        reqId: String, 
        targetId: String, 
        options: AudioRecordingOptions, 
        callback: InkView.VoidCallback
    ) {
        try {
            // Initialize and start audio recording
            myAudioRecorder.start(options.sampleRate)
            callback.success()
        } catch (e: Exception) {
            callback.error(-1, e.message ?: "Unknown error")
        }
    }

    override fun stopAudioRecording(reqId: String, callback: InkView.VoidCallback) {
        myAudioRecorder.stop()
        callback.success()
    }

    // Implement other methods (pause, resume, takePhoto) similarly...
})

4.2 Dispatch Events

Ink supports two types of event dispatching: Target-Specific Events and Global Events.

Target-Specific Events

These events are dispatched to a specific target (e.g., an audio recording session, a network socket, or an audio player). You need to provide the targetId (or handle), which is typically passed to you via the Capability Implementation methods.

dispatchEvent(targetId: String, eventData: MediaEventData)

Dispatches a media-related event to a specific media session.

• Parameters:
  • targetId: The UUID of the target component/session (received in startAudioRecording).
  • eventData: The specific media event payload (e.g., MediaEventData.AudioRecordingStarted, MediaEventData.AudioFrameRecorded).

dispatchEvent(targetId: String, eventData: NetworkingEventData)

Dispatches a networking-related event to a specific socket connection.

• Parameters:
  • targetId: The UUID of the socket handle (received in openSocket).
  • eventData: The networking event payload (e.g., NetworkingEventData.Data).

dispatchEvent(targetId: String, eventData: AudioEventData)

Dispatches an audio-related event to a specific audio player.

• Parameters:
  • targetId: The UUID of the audio player (received in create).
  • eventData: The audio event payload (e.g., AudioEventData.OnPlay, AudioEventData.OnEnded).

Example:

// Dispatch a media event (e.g., recording started)
inkView.dispatchEvent(recordingSessionId, MediaEventData.AudioRecordingStarted)

// Dispatch a networking event (e.g., received data)
val data = "Hello".toByteArray()
inkView.dispatchEvent(socketHandle, NetworkingEventData.Data(data))

// Dispatch an audio event (e.g., playback started)
inkView.dispatchEvent(audioPlayerId, AudioEventData.OnPlay)

Global Events

These events are not bound to a specific target and affect the global state or are general input events.

dispatchEvent(type: InputEventType, code: String, timestamp: Long): Boolean

Dispatches a global input event (e.g., keyboard key press).

• Parameters:
  • type: The type of input event (InputEventType.KeyUp or InputEventType.KeyDown).
  • code: The key code string (e.g., "KeyA", "Enter").
  • timestamp: The event timestamp in milliseconds.
• Returns:
  • true if the event's default action was prevented by the JS side (i.e., event.preventDefault() was called).
  • false otherwise.

dispatchVoiceWakeup(keyword: String, timestamp: Long)

Dispatches a voice wakeup event (e.g., when a wake-word like "若琪" is detected).

• Parameters:
  • keyword: The wakeup keyword detected.
  • timestamp: The event timestamp in milliseconds.

dispatchConnectivityEvent(status: ConnectivityStatus)

Dispatches a connectivity event (e.g., when the device goes online or offline).

• Parameters:
  • status: The connectivity status, either ConnectivityStatus.ONLINE or ConnectivityStatus.OFFLINE.

Example:

// Dispatch a global input event
inkView.dispatchEvent(InkView.InputEventType.KeyDown, "KeyA", System.currentTimeMillis())

// Dispatch a voice wakeup event
inkView.dispatchVoiceWakeup("乐奇", System.currentTimeMillis())

// Dispatch a connectivity event
inkView.dispatchConnectivityEvent(InkView.ConnectivityStatus.ONLINE)

4.3 System Capabilities Reference

This section details the available system capabilities, their methods, and the events they should dispatch.

Media (InkView.MediaCapability)

Handles audio recording and camera operations.

inkView.addCapability(object : InkView.MediaCapability {
    override fun startAudioRecording(
        reqId: String, 
        targetId: String, 
        options: InkView.AudioRecordingOptions, 
        callback: InkView.VoidCallback
    ) {
        // Start recording...
        callback.success()
    }
    
    // Implement other methods...
})

Methods:

startAudioRecording(reqId: String, targetId: String, options: AudioRecordingOptions, callback: VoidCallback)

Request to start audio recording.

• Parameters:
  • reqId: Unique request ID.
  • targetId: UUID of the recording session.
  • options: Configuration (sample rate, channels, etc.).
  • callback: Callback to report success or error.

stopAudioRecording(reqId: String, callback: VoidCallback)

Request to stop audio recording.

pauseAudioRecording(reqId: String, callback: VoidCallback)

Request to pause audio recording.

resumeAudioRecording(reqId: String, callback: VoidCallback)

Request to resume audio recording.

takePhoto(reqId: String, options: TakePhotoOptions, callback: PhotoCallback)

Request to capture a photo.

• Callback: success(data: ByteArray, mimeType: String)

Events (InkView.MediaEventData):

Event Name	Data Type	Description
AudioRecordingStarted	Unit	Recording has started.
AudioRecordingStopped	String	Recording stopped (with optional reason/ID).
AudioRecordingPaused	Unit	Recording paused.
AudioRecordingResumed	Unit	Recording resumed.
AudioFrameRecorded	ByteArray	New audio frame data.
AudioRecordingError	String	Error occurred.
AudioRecordingInterruptionBegin	Unit	Recording interrupted (e.g. call).
AudioRecordingInterruptionEnd	Unit	Interruption ended.

Networking (InkView.NetworkingCapability)

Handles socket operations.

inkView.addCapability(object : InkView.NetworkingCapability {
    override fun openSocket(
        reqId: String, 
        handle: String, 
        name: String, 
        port: Int, 
        isSsl: Boolean, 
        callback: InkView.VoidCallback
    ) {
        // Open socket...
        callback.success()
    }

    // Implement other methods...
})

Methods:

openSocket(reqId: String, handle: String, name: String, port: Int, isSsl: Boolean, callback: VoidCallback)

Request to open a socket connection.

• Parameters:
  • reqId: Unique request ID.
  • handle: Unique UUID for this socket.
  • name: Hostname or IP address.
  • port: Port number.
  • isSsl: Whether to use SSL/TLS.
  • callback: Callback to report success or error.

closeSocket(reqId: String, handle: String, callback: VoidCallback)

Request to close a socket.

send(reqId: String, handle: String, data: ByteArray, callback: VoidCallback)

Request to send data over the socket.

Events (InkView.NetworkingEventData):

Event Name	Data Type	Description
Data	ByteArray	Received data from the socket.

Speech (InkView.SpeechCapability)

Handles Text-To-Speech (TTS) and Automatic Speech Recognition (ASR).

inkView.addCapability(object : InkView.SpeechCapability {
    override fun playTts(
        reqId: String, 
        text: String, 
        callback: InkView.VoidCallback
    ) {
        // Play TTS...
        callback.success()
    }

    override fun startAsr(
        reqId: String, 
        callback: InkView.StringCallback
    ) {
        // Start ASR...
        callback.success("Recognized Text")
    }
})

Methods:

playTts(reqId: String, text: String, callback: VoidCallback)

Request to play text-to-speech.

• Parameters:
  • reqId: Unique request ID.
  • text: Text to speak.
  • callback: Callback to report completion.

startAsr(reqId: String, callback: StringCallback)

Request to start speech recognition.

• Callback: success(text: String) with recognized text.

Events:

Event Name	Data Type	Description
VoiceWakeup	VoiceWakeupEventData	Triggered when a voice wakeup (wake-word) is detected.

Audio (InkView.AudioCapability)

Handles audio playback and streaming.

inkView.addCapability(object : InkView.AudioCapability {
    override fun create(reqId: String, id: String, options: AudioStreamOptions?, callback: InkView.VoidCallback) {
        // Create audio player...
        callback.success()
    }
    
    // Implement other methods...
})

Methods:

create(reqId: String, id: String, options: AudioStreamOptions?, callback: VoidCallback)

destroy(reqId: String, id: String, callback: VoidCallback)

play(reqId: String, id: String, callback: VoidCallback)

pause(reqId: String, id: String, callback: VoidCallback)

stop(reqId: String, id: String, callback: VoidCallback)

seek(reqId: String, id: String, position: Double, callback: VoidCallback)

setBuffer(reqId: String, id: String, data: ByteArray, hint: String, callback: VoidCallback)

setLoop(reqId: String, id: String, loop: Boolean, callback: VoidCallback)

append(reqId: String, id: String, data: ByteArray, callback: VoidCallback)

finish(reqId: String, id: String, callback: VoidCallback)

Events (InkView.AudioEventData):

Event Name	Data Type	Description
OnCanPlay	Unit	Audio is ready to play.
OnPlay	Unit	Playback started.
OnPause	Unit	Playback paused.
OnStop	Unit	Playback stopped.
OnEnded	Unit	Playback ended.
OnTimeUpdate	Double	Playback position updated.
OnError	String	Error occurred.
OnWaiting	Unit	Waiting for data.
OnSeeking	Unit	Seeking to a position.
OnSeeked	Unit	Seek operation completed.

6. Lifecycle Management

Ensure you handle the lifecycle of InkView correctly. InkView is designed to preserve the native engine instance even when the view is detached from the window (e.g., in a scrolling list).

override fun onDestroy() {
    super.onDestroy()
    // While InkView manages its internal surface lifecycle, you should 
    // ensure any host-side resources are cleaned up when the activity is destroyed.
}

When an InkView is detached from the window, it stops its rendering thread but keeps the native instance and JavaScript application state alive. When it's re-attached, it automatically resumes rendering with the same state.