API Reference

class agentdesk.ConnectConfig(*, agentd_url: str | None = None, vm: str | None = None, storage_uri: str = 'file://.media', type_min_interval: float = 0.05, type_max_interval: float = 0.25, move_mouse_duration: float = 1.0, mouse_tween: str = 'easeInOutQuad', store_img: bool = False, requires_proxy: bool = True, proxy_type: str = 'process', proxy_port: int = 8000, private_ssh_key: str | None = None, ssh_port: int = 22)

Bases: BaseModel

agentd_url: str | None

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'agentd_url': FieldInfo(annotation=Union[str, NoneType], required=False), 'mouse_tween': FieldInfo(annotation=str, required=False, default='easeInOutQuad'), 'move_mouse_duration': FieldInfo(annotation=float, required=False, default=1.0), 'private_ssh_key': FieldInfo(annotation=Union[str, NoneType], required=False), 'proxy_port': FieldInfo(annotation=int, required=False, default=8000), 'proxy_type': FieldInfo(annotation=str, required=False, default='process'), 'requires_proxy': FieldInfo(annotation=bool, required=False, default=True), 'ssh_port': FieldInfo(annotation=int, required=False, default=22), 'storage_uri': FieldInfo(annotation=str, required=False, default='file://.media'), 'store_img': FieldInfo(annotation=bool, required=False, default=False), 'type_max_interval': FieldInfo(annotation=float, required=False, default=0.25), 'type_min_interval': FieldInfo(annotation=float, required=False, default=0.05), 'vm': FieldInfo(annotation=Union[str, NoneType], required=False)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

mouse_tween: str

move_mouse_duration: float

private_ssh_key: str | None

proxy_port: int

proxy_type: str

requires_proxy: bool

ssh_port: int

storage_uri: str

store_img: bool

type_max_interval: float

type_min_interval: float

vm: str | None

class agentdesk.Desktop(agentd_url: str | None = None, vm: DesktopVM | None = None, storage_uri: str = 'file://.media', type_min_interval: float = 0.05, type_max_interval: float = 0.25, move_mouse_duration: float = 1.0, mouse_tween: str = 'easeInOutQuad', store_img: bool = False, requires_proxy: bool = True, proxy_type: str = 'process', proxy_port: int = 8000, private_ssh_key: str | None = None, ssh_port: int = 22, check_health: bool = True)

Bases: Device

Desktop OS as a device via agentd

click(button: str = 'left', x: int | None = None, y: int | None = None) → None

Click mouse button

Parameters:

• button (str, optional) – Button to click. Defaults to “left”.

• x (Optional[int], optional) – X coordinate to move to, if not provided it will click on current location. Defaults to None.

• y (Optional[int], optional) – Y coordinate to move to, if not provided it will click on current location. Defaults to None.

close()

A method that should be implemented by subclasses to handle the closing of the tool.

This method is intended to provide a way to release any resources or perform any cleanup necessary when the tool is no longer needed.

classmethod connect(config: ConnectConfig) → Desktop

Connect to a device from a configuration

Parameters:

config (C) – Config

Returns:

The device

Return type:

D

connect_config() → ConnectConfig

Connect configuration

Returns:

Connect configuration for this device

Return type:

C

classmethod connect_config_type() → Type[ConnectConfig]

Type of connect configuration

Returns:

Type of connect configuration

Return type:

Type[C]

classmethod create(name: str | None = None, config: ProvisionConfig = ProvisionConfig(provider=V1ProviderData(type='qemu', args=None), image=None, memory=4, cpus=2, disk='30gb', reserve_ip=False, ssh_key_pair=None, proxy_port=8000)) → Desktop

Create a desktop VM

disconnect() → None

Disconnect from the device

double_click() → None

Double click the mouse

drag_mouse(x: int, y: int) → None

Drag the mouse

Parameters:

• x (int) – x coordinate

• y (int) – y coordinate

classmethod ec2(name: str | None = None, region: str | None = None, image: str | None = None, memory: int = 4, cpus: int = 2, disk: str = '30gb', reserve_ip: bool = False, ssh_key_pair: str | None = None) → Desktop

Create a desktop VM on EC2

classmethod ensure(name: str, config: ProvisionConfig) → Desktop

Find or create a desktop

classmethod find(**kwargs: Any) → list[DesktopVM]

List all desktops

Returns:

A list of desktop vms

Return type:

list[DesktopVM]

classmethod from_vm(vm: DesktopVM, proxy_type: str = 'process', proxy_port: int = 8000, check_health: bool = True) → Desktop

Create a desktop from a VM

Parameters:

• vm (DesktopVM) – VM to use

• proxy_type (str, optional) – The type of proxy to use. Defaults to process.

• proxy_port (int, optional) – The port to use for the proxy. Defaults to 8000.

• check_health (bool, optional) – Check the health of the VM. Defaults to True.

Returns:

A desktop

Return type:

Desktop

classmethod gce(name: str | None = None, project: str | None = None, zone: str = 'us-central1-a', region: str = 'us-central1', image: str | None = None, memory: int = 4, cpus: int = 2, disk: str = '30gb', reserve_ip: bool = False, ssh_key_pair: str | None = None) → Desktop

Create a desktop VM on GCE

classmethod get(name: str) → DesktopVM | None

Get a desktop by name

Parameters:

name (str) – Name of the desktop

Returns:

A desktop

Return type:

Desktop

health() → dict

Health of agentd

Returns:

Agentd health

Return type:

dict

hot_key(keys: List[str]) → None

Press a hot key. For example ctrl+c

Parameters:

keys (List[str]) – Which keys to press. Options are: [ “t”, “n”, “r”, “ “, “!”, ‘"’, “#”, “$”, “%”, “&”, “'”, “(”, “)”, “*”, “+”, “,”, “-”, “.”, “/”, “0”, “1”, “2”, “3”, “4”, “5”, “6”, “7”, “8”, “9”, “:”, “;”, “<”, “=”, “>”, “?”, “@”, “[”, “\”, “]”, “^”, “_”, “`”, “a”, “b”, “c”, “d”, “e”, “f”, “g”, “h”, “i”, “j”, “k”, “l”, “m”, “n”, “o”, “p”, “q”, “r”, “s”, “t”, “u”, “v”, “w”, “x”, “y”, “z”, “{”, “|”, “}”, “~”, “accept”, “add”, “alt”, “altleft”, “altright”, “apps”, “backspace”, “browserback”, “browserfavorites”, “browserforward”, “browserhome”, “browserrefresh”, “browsersearch”, “browserstop”, “capslock”, “clear”, “convert”, “ctrl”, “ctrlleft”, “ctrlright”, “decimal”, “del”, “delete”, “divide”, “down”, “end”, “enter”, “esc”, “escape”, “execute”, “f1”, “f10”, “f11”, “f12”, “f13”, “f14”, “f15”, “f16”, “f17”, “f18”, “f19”, “f2”, “f20”, “f21”, “f22”, “f23”, “f24”, “f3”, “f4”, “f5”, “f6”, “f7”, “f8”, “f9”, “final”, “fn”, “help”, “home”, “insert”, “left”, “numlock”, “pagedown”, “pageup”, “pause”, “pgdn”, “pgup”, “playpause”, “prevtrack”, “print”, “printscreen”, “prntscrn”, “prtsc”, “prtscr”, “return”, “right”, “scrolllock”, “select”, “separator”, “shift”, “shiftleft”, “shiftright”, “sleep”, “space”, “stop”, “subtract”, “tab”, “up”, “volumedown”, “volumemute”, “volumeup”, “win”, “winleft”, “winright”, “yen”, “command”, “option”, “optionleft”, “optionright” ]

info() → dict

Get info on the desktop runtime

Returns:

A dictionary of info

Return type:

dict

classmethod local(name: str | None = None, memory: int = 4, cpus: int = 2) → Desktop

Create a local VM

Parameters:

• name (str, optional) – Name of the vm. Defaults to None.

• memory (int, optional) – Memory the VM has. Defaults to 4.

• cpus (int, optional) – CPUs the VM has. Defaults to 2.

Returns:

A desktop

Return type:

Desktop

mouse_coordinates() → Tuple[int, int]

Get the current mouse coordinates

Returns:

x, y coordinates

Return type:

Tuple[int, int]

move_mouse(x: int, y: int) → None

Move mouse to a position

Parameters:

• x (int) – x coordinate

• y (int) – y coordinate

open_url(url: str) → None

Open a URL in chromium

Parameters:

url (str) – URL to open

press_key(key: str) → None

Press a key

Parameters:

key (str) – Which key to press. Options are: [ “t”, “n”, “r”, “ “, “!”, ‘"’, “#”, “$”, “%”, “&”, “'”, “(”, “)”, “*”, “+”, “,”, “-”, “.”, “/”, “0”, “1”, “2”, “3”, “4”, “5”, “6”, “7”, “8”, “9”, “:”, “;”, “<”, “=”, “>”, “?”, “@”, “[”, “\”, “]”, “^”, “_”, “`”, “a”, “b”, “c”, “d”, “e”, “f”, “g”, “h”, “i”, “j”, “k”, “l”, “m”, “n”, “o”, “p”, “q”, “r”, “s”, “t”, “u”, “v”, “w”, “x”, “y”, “z”, “{”, “|”, “}”, “~”, “accept”, “add”, “alt”, “altleft”, “altright”, “apps”, “backspace”, “browserback”, “browserfavorites”, “browserforward”, “browserhome”, “browserrefresh”, “browsersearch”, “browserstop”, “capslock”, “clear”, “convert”, “ctrl”, “ctrlleft”, “ctrlright”, “decimal”, “del”, “delete”, “divide”, “down”, “end”, “enter”, “esc”, “escape”, “execute”, “f1”, “f10”, “f11”, “f12”, “f13”, “f14”, “f15”, “f16”, “f17”, “f18”, “f19”, “f2”, “f20”, “f21”, “f22”, “f23”, “f24”, “f3”, “f4”, “f5”, “f6”, “f7”, “f8”, “f9”, “final”, “fn”, “help”, “home”, “insert”, “left”, “numlock”, “pagedown”, “pageup”, “pause”, “pgdn”, “pgup”, “playpause”, “prevtrack”, “print”, “printscreen”, “prntscrn”, “prtsc”, “prtscr”, “return”, “right”, “scrolllock”, “select”, “separator”, “shift”, “shiftleft”, “shiftright”, “sleep”, “space”, “stop”, “subtract”, “tab”, “up”, “volumedown”, “volumemute”, “volumeup”, “win”, “winleft”, “winright”, “yen”, “command”, “option”, “optionleft”, “optionright” ]

classmethod provision_config_type() → Type[ProvisionConfig]

Type of provision configuration

Returns:

Type of provisioner configuration

Return type:

Type[P]

classmethod react_component() → ReactComponent

React component for the device

Returns:

React component

Return type:

Optional[ReactComponent]

scroll(clicks: int = -3) → None

Scroll the screen

Parameters:

clicks (int, optional) – Number of clicks, negative scrolls down, positive scrolls up. Defaults to -3.

take_screenshot() → str

Take screenshot

Returns:

b64 encoded image or URI of the image depending on instance settings

Return type:

str

type_text(text: str) → None

Type text

Parameters:

text (str) – Text to type

view(background: bool = False) → None

View the desktop

Parameters:

background (bool, optional) – Whether to run in the background and not block. Defaults to False.

class agentdesk.ProvisionConfig(*, provider: V1ProviderData = V1ProviderData(type='qemu', args=None), image: str | None = None, memory: int = 4, cpus: int = 2, disk: str = '30gb', reserve_ip: bool = False, ssh_key_pair: str | None = None, proxy_port: int = 8000)

Bases: BaseModel

cpus: int

disk: str

image: str | None

memory: int

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'cpus': FieldInfo(annotation=int, required=False, default=2), 'disk': FieldInfo(annotation=str, required=False, default='30gb'), 'image': FieldInfo(annotation=Union[str, NoneType], required=False), 'memory': FieldInfo(annotation=int, required=False, default=4), 'provider': FieldInfo(annotation=V1ProviderData, required=False, default=V1ProviderData(type='qemu', args=None)), 'proxy_port': FieldInfo(annotation=int, required=False, default=8000), 'reserve_ip': FieldInfo(annotation=bool, required=False, default=False), 'ssh_key_pair': FieldInfo(annotation=Union[str, NoneType], required=False)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

provider: V1ProviderData

proxy_port: int

reserve_ip: bool

ssh_key_pair: str | None

class agentdesk.SimpleDesktop(agentd_url: str | None = None, vm: DesktopVM | None = None, storage_uri: str = 'file://.media', type_min_interval: float = 0.05, type_max_interval: float = 0.25, move_mouse_duration: float = 1.0, mouse_tween: str = 'easeInOutQuad', store_img: bool = False, requires_proxy: bool = True, proxy_type: str = 'process', proxy_port: int = 8000, private_ssh_key: str | None = None, ssh_port: int = 22, check_health: bool = True)

Bases: Desktop

A more simple desktop

actions() → List[Action]

Actions the agent can take

Returns:

List of actions

Return type:

List[Action]

class agentdesk.V1Desktop(*, id: str | None = None, name: str | None = None, addr: str | None = None, status: str | None = None, created: float | None = None, image: str | None = None, memory: int | None = None, cpu: int | None = None, disk: str | None = None, memory_usage: float | None = None, cpu_usage: float | None = None, disk_usage: float | None = None, reserved_ip: bool | None = None, provider: V1ProviderData | None = None, meta: dict | None = None, owner_id: str | None = None, key_pair_name: str | None = None)

Bases: BaseModel

addr: str | None

cpu: int | None

cpu_usage: float | None

created: float | None

disk: str | None

disk_usage: float | None

id: str | None

image: str | None

key_pair_name: str | None

memory: int | None

memory_usage: float | None

meta: dict | None

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'addr': FieldInfo(annotation=Union[str, NoneType], required=False), 'cpu': FieldInfo(annotation=Union[int, NoneType], required=False), 'cpu_usage': FieldInfo(annotation=Union[float, NoneType], required=False), 'created': FieldInfo(annotation=Union[float, NoneType], required=False), 'disk': FieldInfo(annotation=Union[str, NoneType], required=False), 'disk_usage': FieldInfo(annotation=Union[float, NoneType], required=False), 'id': FieldInfo(annotation=Union[str, NoneType], required=False), 'image': FieldInfo(annotation=Union[str, NoneType], required=False), 'key_pair_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'memory': FieldInfo(annotation=Union[int, NoneType], required=False), 'memory_usage': FieldInfo(annotation=Union[float, NoneType], required=False), 'meta': FieldInfo(annotation=Union[dict, NoneType], required=False), 'name': FieldInfo(annotation=Union[str, NoneType], required=False), 'owner_id': FieldInfo(annotation=Union[str, NoneType], required=False), 'provider': FieldInfo(annotation=Union[V1ProviderData, NoneType], required=False), 'reserved_ip': FieldInfo(annotation=Union[bool, NoneType], required=False), 'status': FieldInfo(annotation=Union[str, NoneType], required=False)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str | None

owner_id: str | None

provider: V1ProviderData | None

reserved_ip: bool | None

status: str | None

class agentdesk.V1ProviderData(*, type: str | None = None, args: dict | None = None)

Bases: BaseModel

args: dict | None

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'args': FieldInfo(annotation=Union[dict, NoneType], required=False), 'type': FieldInfo(annotation=Union[str, NoneType], required=False)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

type: str | None