Sikuli Script

Platform Independence

Screenshot driven it is a visual technology to search and automate GUI using images
Intuitive open-source visual scripting tool
Scripts are written in Python
integrates with jUnit and supports unit testing for GUI
ability to import entire libs or only a selected subset of methods (reuse code and images)

Test folder

Sikuli source/executable script directory (.sikuli)
Python source file (.py)
Images files (.png)
While saving a script using Sikuli IDE, an HTML file is created

Using Sikuli IDE

Predefined Sikuli Global functions
IDE Command List – exists,find, findAll, wait, click,doubleClick, dragDrop, type

• switchApp("App Name")
• keyDown(Key.SHIFT+ "a")
• keyUp(Key.SHIFT+ "a")
• popup("message")
• userText=input("msg", "default")

Regions a rectangular area: Region(x, y, w, h)

Finding Regions:
Acting with Regions
Extending Regions

Tuning the Vision Algorithm

Advantages

• Allow programmer to make visual references instead of using names or keywords
• More intuitive and natural approach
• Useful for specific applications that involve manipulation of GUI

Limitation

• Sikuli Script operates only in the visible screen space and thus is not applicable to invisible GUI elements ex. elements hidden underneath other windows, in another tab, or scrolled out of view
• Dealing with moving, animated objects was hard