Single-Step Actions in AskUI

Core Interaction Commands

Tools

Assertion and Information Commands

Mouse Interactions

Keyboard Interactions

Scrolling and Touch Interactions

Welcome to the home of your new documentation

Single-Step Actions

Starter Kit

Welcome to the AskUI documentation. This guide will help you understand what Vision Agents are, their key features, benefits, and how they can empower you to build AI agents for various use cases. 

Overview

How AskUI Works

Quickstart Tutorial

Prerequisites

Installation Guide

Configuration & Setup

On this page you will learn the different recommended practices

Selecting Elements

Agentic Mode

AskUI Agent OS provides a comprehensive set of commands to manage your AskUI projects, controllers, and settings. This page provides an overview of the available commands and their parameters.

Commands Overview

This page explains how to set up **AskUI with GitHub Actions**, including dependency setup, environment variables, test execution, and optional report generation.

Supported Actions

Reteaching Models

Proxy & TLS Configuration

Learn how to use and configure different AI models

Model Usage

Github Actions

This page explains how to set up and run **AskUI workflows in a CI/CD pipeline**, including dependencies, environment variables, and optional report generation, with a full example on **GitLab**.

Gitlab CI/CD

This page explains how to set up and run **AskUI workflows in Azure DevOps**, including pipeline configuration, environment variables, Docker setup, and report generation, with a full example on **GitHub**.

Azure DevOps

SSO with AskUI

Changelog

This page provides a glossary of key AskUI terms and definitions to help you understand the core concepts and components of the platform.

Glossary

Workspace Management

Access Tokens

Change your subscription and manage invoices

Billing and Invoices

Common Issues

Reporting Bugs & Providing Feedback

TypeScript Docs

Status Page

Trust Center

Community

Blog

API Reference & Extensibility

Account & Settings

Login

Support

Sign up

Create Access Token

Delete Access Token

List Access Tokens

Converts a sensitive global-level access token into its stable identifier (access_token_id). The access token can be base64 encoded (similar to Authorization header) or passed raw. This enables clients to reference tokens without transmitting sensitive values, e.g., when trying to delete the token or retrieve usage for the token.

Lookup Global Access Token ID

Converts a sensitive workspace-level access token into its stable identifier (access_token_id) within the specified workspace. The access token can be base64 encoded (similar to Authorization header) or passed raw. This enables clients to reference tokens without transmitting sensitive values, e.g., when trying to delete the token or retrieve usage for the token.

Lookup Workspace Access Token ID

Create Agent Execution

List all agent executions matching the query parameters sorted anti-chronologically by when they were last updated (`updated_at`).

List Agent Executions

Update Agent Execution

Create Agent

List all agents matching the query parameters sorted anti-chronologically by when they were last updated (`updated_at`).

List Agents

Update Agent

Create Customer Portal Session

Get Workspace Subscription

Delete a file at the specified path.

- Deletes only one file with the given file path. Not bulk deletion.
- If there is no file at the specified path, the operation will be a no-op, i.e., it will still return a 204 status code.

Delete File

List files.

- To list files within a workspace, the `prefix` (query parameter) must start with `workspaces/{workspace_id}/`
- Cannot list files across multiple workspaces, i.e., if the `prefix` does not start with `workspaces/{workspace_id}/`, it will return an empty list.

List Files

Set http-only, secure (signed) cookies for accessing (only READ access) files via AWS CloudFront across all accessible workspaces. If no workspace id is provided, no cookies will be set.

Set Signed Cookies

Upload a file (max. 5 GB) to the specified path.

- Specify the `Content-Type` header for accurate file type handling.
- If `Content-Type` is omitted, the system will attempt to infer it.
- For workspace-specific uploads, use path: `workspaces/{workspace_id}/...`
- If a file with the same file path already exists, it will be overwritten.
- If there are unsupported characters in the file path, they will be removed or replaced.
- If the file path is longer than `1024` characters, the file path will be shortened starting from the end.
- Set `strict=true` to fail the request if the file path would be modified during sanitization (e.g., if the file path contains unsupported characters or is too long).

Upload File

Complete

Lease

Ping

List Runs

Create Schedule

Suspend Schedule

List usage events sorted chronologically by timestamp. Events can be filtered by workspace, user, or access token.

Filter combinations:
- Multiple filters (e.g. workspace_id & user_id) use AND logic
- Multiple values for same filter (e.g. workspace_id=123&workspace_id=456) use OR logic

Pagination:
- Use starting_after with event IDs for cursor-based navigation
- has_more indicates if more events exist beyond limit
- total_count (optional) shows total matching events up to 100,000

	Description	Models	Example (Python)
`click()`	Clicks on an element described by text	All	`agent.click('Login button')`
`expect()`	Asserts that an element exists	All	`agent.expect('Login successful')`
`type()`	Types text into a focused element	All	`agent.type('username@example.com')`

	Description	Example (Python)
`os`	Provides OS-level operations	`agent.tools.os.get_clipboard_text()`
`webbrowser`	Controls web browser operations	`agent.tools.webbrowser.open_new("https://askui.com")`
`clipboard`	Manages clipboard operations	`agent.tools.clipboard.copy("Text to copy")`

	Description	Models	Example (Python)
`mouseDoubleLeftClick()`	Performs a double left click	All	`agent.mouseDoubleLeftClick('icon')`
`mouseDoubleMiddleClick()`	Performs a double middle click	All	`agent.mouseDoubleMiddleClick('tab')`
`mouseDoubleRightClick()`	Performs a double right click	All	`agent.mouseDoubleRightClick('file')`
`mouseLeftClick()`	Performs a left click	All	`agent.mouseLeftClick('button')`
`mouseMiddleClick()`	Performs a middle click	All	`agent.mouseMiddleClick('link')`
`mouseRightClick()`	Performs a right click	All	`agent.mouseRightClick('context menu')`
`mouseToggleDown()`	Holds down a mouse button	All	`agent.mouseToggleDown('left')`
`mouseToggleUp()`	Releases a mouse button	All	`agent.mouseToggleUp('left')`
`moveMouse()`	Moves mouse to coordinates	All	`agent.moveMouse(100, 200)`
`moveMouseRelatively()`	Moves mouse by offset	All	`agent.moveMouseRelatively(10, 20)`
`moveMouseRelativelyTo()`	Moves mouse relative to element	All	`agent.moveMouseRelativelyTo('button', 5, 5)`
`moveMouseTo()`	Moves mouse to element	All	`agent.moveMouseTo('search field')`

	Description	Models	Example (Python)
`pressAndroidKey()`	Presses Android-specific key	All	`agent.pressAndroidKey('home')`
`pressAndroidTwoKey()`	Presses two Android keys	All	`agent.pressAndroidTwoKey('shift', 'home')`
`pressKey()`	Presses a keyboard key	All	`agent.pressKey('enter')`
`pressThreeKeys()`	Presses three keys simultaneously	All	`agent.pressThreeKeys('ctrl', 'shift', 'esc')`
`pressTwoKeys()`	Presses two keys simultaneously	All	`agent.pressTwoKeys('ctrl', 'c')`

	Description	Models	Example (Python)
`scroll()`	Scrolls the page	All	`agent.scroll('down', 500)`
`scrollInside()`	Scrolls inside an element	All	`agent.scrollInside('dropdown menu', 'down', 200)`
`swipe()`	Performs a swipe gesture	All	`agent.swipe('left', 300)`

Introduction

Getting Started

Core Concepts

Agent OS

Model Usage & Configuration

Integrations & Advanced Usage

Updates & Glossary

Single-Step Actions

Single-Step Actions in AskUI

Core Interaction Commands

Tools

Assertion and Information Commands

Mouse Interactions

Keyboard Interactions

Scrolling and Touch Interactions

Introduction

Getting Started

Core Concepts

Agent OS

Model Usage & Configuration

Integrations & Advanced Usage

Updates & Glossary

​Single-Step Actions in AskUI

​Core Interaction Commands

​Tools

​Assertion and Information Commands

​Mouse Interactions

​Keyboard Interactions

​Scrolling and Touch Interactions

Single-Step Actions in AskUI

Core Interaction Commands

Tools

Assertion and Information Commands

Mouse Interactions

Keyboard Interactions

Scrolling and Touch Interactions