Blog

At Google I/O 2024, Google showed off a completely reworked Assistant running on its Gemini LLM. Gone is the rigid "Hey Google" command pattern. The new Assistant holds continuous conversations, remembers context, and chains tasks across apps. Sensitive data stays on-device for privacy. For developers, that means rethinking how we build voice apps — and especially how we secure them.

What Changed Under the Hood

Under the hood, Google split intent recognition from action execution. The Gemini model handles natural language understanding and dialogue management, while a new lightweight runtime — the Assistant Extension Framework (AEF) — executes tasks via app-specific plugins. This decoupling lets developers write actions using standard Android APIs without giving the Assistant full access to their app’s data. Instead, the AEF requests only the minimum permissions needed for each turn, and the user can revoke those permissions mid-conversation.

Security and Privacy Implications for Developers

Because the new Assistant can chain actions — for example, finding a recipe, adding ingredients to a shopping list, and sending the list to a messaging app — the attack surface broadens. A malicious action plugin could, in theory, exfiltrate data if permissions are too broad. Google mitigates this by requiring all plugins to declare a strict permission model using the new AssistantActionManifest schema. Developers must specify exactly which intents their plugin handles and what data it reads or writes. Plugins that request access to contacts, location, or biometric data undergo an additional manual review.

Voice recordings also stay on your device for most queries now. The on-device speech recognizer handles wake-word detection and simple commands locally. Only when the Assistant needs to invoke a cloud-based Gemini model does it stream anonymized context. As a developer, you should never log raw audio or user utterances in your plugin. Instead, use the provided UtteranceSummary object, which strips personally identifiable information before your code receives it.

Setting Up a Safe Development Environment

To experiment with the new Assistant APIs without risking real user data, create a dedicated Android Virtual Device (AVD) running Android 14 or later. Enable the "Assistant Developer Mode" under Settings > System > Developer Options. This mode logs all intents sent to your plugin and lets you simulate conversation turns from the command line using adb shell am broadcast. You can also test permission prompts by toggling runtime permissions in the emulator’s settings.

When building your first plugin, start with a simple action that does not require network access — for instance, a timer action that reads no external data. Gradually add features while monitoring the permission dialog that appears each time your plugin requests a new capability. If you notice the Assistant asking for permissions you did not declare, inspect your manifest for unintended intent filters.

Practical Security Checklist for Assistant Plugins

• Scoped permissions: Use <uses-permission-sdk-34> with maxSdkVersion to prevent your plugin from requesting permissions on older devices where the security model differs.
• Input validation: Even though the Assistant filters profanity and SQL injection patterns, always sanitize parameters your plugin receives — treat them as untrusted user input.
• Data minimisation: Do not store conversation history. If you need to persist state, use Android’s EncryptedSharedPreferences and tie the key to the current session ID.
• Logging hygiene: Never log Intent.getExtras() directly. Strip fields that could contain user names, addresses, or timestamps before writing to logcat.

What This Means for End-User Digital Hygiene

As a developer, you also help users protect themselves. The new Assistant includes a privacy dashboard that shows every plugin’s permission usage over the last 24 hours. Encourage users to review this dashboard weekly. Additionally, the Assistant now supports voice deletion of conversation snippets — users can say "Hey Google, delete what I just said" and the on-device model will purge that segment from local storage. Your plugin should respect the onDeleteUtterance callback to remove any cached data when the user invokes this command.

For those running home labs or penetration testing environments, the Assistant’s new sandbox mode (enabled via adb shell setprop persist.assistant.sandbox 1) isolates all plugin processes into separate Linux namespaces. This allows you to test privilege escalation scenarios without affecting system stability.

A Concrete Next Step

Download the latest Android Studio Hedgehog (2023.1.1) and create a new project using the "Assistant Plugin" template. Deploy it to an emulator with the Developer Mode enabled, then trigger your action by saying "Hey Google, ask my test plugin for the current time." Observe the permission dialog — it should request nothing. Then add a feature that reads the device’s time zone, redeploy, and watch the system prompt the user for ACCESS_BACKGROUND_LOCATION — even though you only wanted time zone. That mismatch highlights exactly why you must audit every permission your plugin declares. Fix the manifest to request TIME_ZONE only, and the dialog disappears. That one exercise teaches more about the new Assistant’s security model than any documentation page.