Since: API Level v2
public final class

SikuliPlugin

com.webmbt.plugin.SikuliPlugin

Class Overview

Sikuli (beta) plugin provides a set of mscript functions to perform image based automation using SikuliX (Sikuli).

The first time that you use this pllugin you may need to restart TestOptimal server and logoff and re-login your computer for SikuliX to complete the installation.

This plugin is in BETA state. Please contact Support@testoptimal.com for additional information.

Locator: control is identified by the image file stored in uimap folder within the model folder. Only png files are supported.

Copyright © 2008 - 2017 TestOptimal LLC. All Rights Reserved.

Summary

Public Constructors
SikuliPlugin()
Public Methods
boolean click()
perform mouse click on the current UI element.
boolean click(String uiImgFile_p)
performs mouse click on the UI element identified by the image passed in.
boolean clickOffSet(String deltaX_p, String deltaY_p)
perform click action with offset from the click position of last match.
boolean doubleClick(String uiImgFile_p)
performs mouse double click on the UI element identified by the image passed in.
boolean doubleClick()
perform double click on current UI element
boolean doubleClickOffSet(String deltaX_p, String deltaY_p)
perform double click action with offset from the click position of last match.
boolean dragDrop(String fromUIImg_p, String toUIImg_p)
perform drag and drop.
boolean find(String uiImgFile_p)
attempts to find the UI element identified by the image passed in.
int getCurMonitorHeight()
returns the current monitor's height in pixels
int getCurMonitorId()
return the current monitor id
int getCurMonitorWidth()
returns the current monitor's width in pixels
int getMonitorCount()
returns the number of monitors visible to SikuliX
int getPrimaryMonitorId()
return the monitor id of the primary monitor
String getText(String deltaX_p, String deltaY_p, String width_p, String height_p)
performs OCR on the region specified.
void highlightLastMatch(String millis_p)
highlights the last matched UI region for the specified duration.
boolean isPresent(String uiImgFile_p)
checks if the UI element identified by the image passed in is present /visible on the screen.
void mouseDown(String buttons_p)
perform mouse down with the specified mouse button
void mouseDown()
perform mouse down action on current UI element
void mouseDownOffSet(String deltaX_p, String deltaY_p)
perform mouseDown action with offset from the click position of last match.
boolean mouseOver()
move mouse over (hover) current UI element
boolean mouseOverOffSet(String deltaX_p, String deltaY_p)
perform mouseover action with offset from the click position of last match.
void mouseUp()
perform mouse up on current UI element
void mouseUp(String buttons_p)
perform mouse up with the specified mouse button
void offsetLastMatch(String deltaX_p, String deltaY_p)
adjust the target position of last match for action like click.
boolean rightClick()
perform right mouse click on current UI element
boolean rightClickOffSet(String deltaX_p, String deltaY_p)
perform right click action with offset from the click position of last match.
boolean selectMonitor(String monitorID_p)
set the specified monitor to be the current monitor
void setMinSimilarity(String minSimilarity_p)
sets the sensitivity for matching the image.
void setOCR(String onFlag_p)
turns on or off OCR feature.
void setSearchRegion(String x_p, String y_p, String w_p, String h_p)
sets the screen region on the current screen to limit Siklul's search spacce for better performance.
String snapScreen(String x_p, String y_p, String width_p, String height_p, String fileName_p)
takes a screen shot of the specified region on the current screen/monitor and saves it to the file specified.
String snapScreen(String fileName_p)
takes a screen shot and saves it to the file specified.
boolean waitFor(String uiImgFile_p, String timeoutMillis_p)
wait for the UI element up to the timeout in milliseconds.
boolean waitForVanish(String uiImgFile_p, String timeoutMillis_p)
wait for the UI element to vanish/disappear from the screen, up to the timeout in milliseconds specified.

Public Constructors

public SikuliPlugin ()

Since: API Level v2

Public Methods

public boolean click ()

Since: API Level v2

perform mouse click on the current UI element.

Returns
  • true if click is successful

public boolean click (String uiImgFile_p)

Since: API Level v2

performs mouse click on the UI element identified by the image passed in. If UI element found, it also sets it to be the current UI element.

Returns
  • true if UI element is found and click is performed

public boolean clickOffSet (String deltaX_p, String deltaY_p)

Since: API Level v2

perform click action with offset from the click position of last match.

public boolean doubleClick (String uiImgFile_p)

Since: API Level v2

performs mouse double click on the UI element identified by the image passed in. If UI element found, it also sets it to be the current UI element.

Returns
  • true if UI element is found and double click is performed

public boolean doubleClick ()

Since: API Level v2

perform double click on current UI element

Returns
  • true if successful

public boolean doubleClickOffSet (String deltaX_p, String deltaY_p)

Since: API Level v2

perform double click action with offset from the click position of last match.

public boolean dragDrop (String fromUIImg_p, String toUIImg_p)

Since: API Level v2

perform drag and drop.

Parameters
fromUIImg_p drag to start at this UI element
toUIImg_p drop over this UI element
Returns
  • true if successful

public boolean find (String uiImgFile_p)

Since: API Level v2

attempts to find the UI element identified by the image passed in. If UI element is found, it also sets the UI element found to be the current UI element on which future actions will be performed.

Parameters
uiImgFile_p name of the image file located in uimap folder in the model folder. You may also use absolute file path.
Returns
  • true if found

public int getCurMonitorHeight ()

Since: API Level v2

returns the current monitor's height in pixels

public int getCurMonitorId ()

Since: API Level v2

return the current monitor id

Returns
  • monitor id

public int getCurMonitorWidth ()

Since: API Level v2

returns the current monitor's width in pixels

public int getMonitorCount ()

Since: API Level v2

returns the number of monitors visible to SikuliX

Returns
  • number of monitors

public int getPrimaryMonitorId ()

Since: API Level v2

return the monitor id of the primary monitor

Returns
  • monitor id

public String getText (String deltaX_p, String deltaY_p, String width_p, String height_p)

Since: API Level v2

performs OCR on the region specified. The region specified is relative from the current UI element location and the specified width and height.

Experimental use only.

Parameters
deltaX_p relative x position from current UI element's x location.
deltaY_p relative y position from current UI element's y location.
width_p width of the target region
height_p height of the target region
Throws
MBTException

public void highlightLastMatch (String millis_p)

Since: API Level v2

highlights the last matched UI region for the specified duration.

public boolean isPresent (String uiImgFile_p)

Since: API Level v2

checks if the UI element identified by the image passed in is present /visible on the screen.

Returns
  • true if present/visible

public void mouseDown (String buttons_p)

Since: API Level v2

perform mouse down with the specified mouse button

Parameters
buttons_p L for left, M for middle, R for right. String the button codes to perform mouse down with multiple mouse buttons.

public void mouseDown ()

Since: API Level v2

perform mouse down action on current UI element

public void mouseDownOffSet (String deltaX_p, String deltaY_p)

Since: API Level v2

perform mouseDown action with offset from the click position of last match.

public boolean mouseOver ()

Since: API Level v2

move mouse over (hover) current UI element

Returns
  • true if successful

public boolean mouseOverOffSet (String deltaX_p, String deltaY_p)

Since: API Level v2

perform mouseover action with offset from the click position of last match.

public void mouseUp ()

Since: API Level v2

perform mouse up on current UI element

public void mouseUp (String buttons_p)

Since: API Level v2

perform mouse up with the specified mouse button

Parameters
buttons_p L for left, M for middle, R for right. String the button codes to perform mouse down with multiple mouse buttons. Pass in blank to release all mouse buttons

public void offsetLastMatch (String deltaX_p, String deltaY_p)

Since: API Level v2

adjust the target position of last match for action like click.

public boolean rightClick ()

Since: API Level v2

perform right mouse click on current UI element

Returns
  • true if successful

public boolean rightClickOffSet (String deltaX_p, String deltaY_p)

Since: API Level v2

perform right click action with offset from the click position of last match.

public boolean selectMonitor (String monitorID_p)

Since: API Level v2

set the specified monitor to be the current monitor

Returns
  • true if successful

public void setMinSimilarity (String minSimilarity_p)

Since: API Level v2

sets the sensitivity for matching the image.

Parameters
minSimilarity_p float value between 0 and 1 with 1 being identical. Default is 0.75.

public void setOCR (String onFlag_p)

Since: API Level v2

turns on or off OCR feature. OCR feature is currently in experimental state. Use with caution.

Parameters
onFlag_p Y to turn on OCR, N to turn off OCR.

public void setSearchRegion (String x_p, String y_p, String w_p, String h_p)

Since: API Level v2

sets the screen region on the current screen to limit Siklul's search spacce for better performance.

public String snapScreen (String x_p, String y_p, String width_p, String height_p, String fileName_p)

Since: API Level v2

takes a screen shot of the specified region on the current screen/monitor and saves it to the file specified.

To capture screenshot of a specific screen/monitor, call $selectMonitor('id') to switch to that screen/monitor before calling this function.

Throws
Exception

public String snapScreen (String fileName_p)

Since: API Level v2

takes a screen shot and saves it to the file specified.

Throws
Exception

public boolean waitFor (String uiImgFile_p, String timeoutMillis_p)

Since: API Level v2

wait for the UI element up to the timeout in milliseconds.

Returns
  • true if found

public boolean waitForVanish (String uiImgFile_p, String timeoutMillis_p)

Since: API Level v2

wait for the UI element to vanish/disappear from the screen, up to the timeout in milliseconds specified.

Returns
  • true if UI element vanishes before time out