Getting Started with
In this section we work through a number of examples of automating user
interfaces using the
dip also includes the
dip-automate tool which runs a PyQt application (it
does not need to be a dip application) under the control of an automation
script written using the
dip.automate module. The only requirement
imposed on the PyQt application being automated is that individual widgets that
are to be automated have their
objectName property set as this is how they
are refered to by the automation script.
Automation is typically used for the following reasons:
- during debugging to get the application into a particular state that would be time consuming to do manually
- to unit test user interfaces, usually by verifying that the effects on the model that the user interface is bound to is as expected
- to produce canned demonstrations of an applications functionality.
Of course in a Test Driven Development environment the first stage would be to create an automation script that demonstrates a bug. Once the bug is fixed then the script is added to the test suite so that any future regressions are quickly identified.
How Automation Works¶
When using the default PyQt5 toolkit, the
dip.automate module is
implemented using the
QtTest module. The
module simulates keyboard and mouse events and applies them to specific
widgets. This very low level of functionality has some problems:
- you need a reference to the widget
- you need to know the type of the widget so that you know which events to simulate to achieve the desired behaviour.
The second problem is particularly significant. It means that whenever you change the GUI’s implementation (i.e. the widgets it uses) then you have to change the automation. Also, in a dip application, the automation would be specific to a particular. What you really want is to only have to change the automation when you change what the GUI does (i.e. what changes it makes to the model it is bound to).
dip.automate module solves the first problem by requiring that
widgets to be automated have their
objectName property set. Means are
provided to easily find widgets with a particular name, and to distinguish
between multiple widgets with the same name. When widgets are created from
declarative definitions by the
dip.ui module they are automatically
given predictable names. They can also be given names explicitly by setting
The second problem is solved by the provision of the
IAutomated interface and its sub-classes. These define
the simulated operations that different sorts of widgets support. A toolkit
will provide adapters that implement those operations on the
particular widgets that the toolkit creates.
The most common simulated operation is
set which is supported by any widget
that can be adapted to the
For example, the default PyQt5 toolkit provides adapters for, amongst others,
Therefore if you change your GUI to use a spin box rather than a combo box then
you do not need to change any automation as dip will automatically pick the
correct adapter that will simulate the appropriate keyboard and mouse events.
A Simple Example¶
The following is an automated version of our very first example. It can be
from dip.automate import Robot from dip.ui import Application, Form # Every application needs an Application. app = Application() # Create the model. model = dict(name='') # Define the view. view = Form() # Create an instance of the view bound to the model. ui = view(model) # Make the instance of the view visible. ui.show() # Simulate the events to set the name. Robot.simulate('name', 'set', "Bill", delay=200) # Show the value of the model. print("Name:", model['name'])
There are only a couple of changes. The first one, below, imports from the
from dip.automate import Robot
The second change, below, replaces the call to the
execute() method that enters the event loop:
Robot.simulate('name', 'set', "Bill", delay=200)
The first argument to
simulate() is the
objectName of the widget to be automated. When the view is created the
objectName property of all editors are automatically set to
id of the corresponding editor factory. The
id is the name of the model attribute that the
editor is bound to. So, in this case,
'name' identifies the widget that is
bound to the
'name' attribute of the model.
The second argument,
'set' in this case, is the high-level command to be
simulated. There is no definitive list of these commands. Instead the adapter
that is created under the covers that implements the
IAutomated interface is introspected for methods whose
name is of the form
xyz can then be used as a
high-level command to
The remaining non-keyword arguments are passed as arguments to the high-level
command. In this case
'set' takes a single argument,
"Bill" that is
the value to set.
Any other arguments are keyword arguments and are optional. In our example we
delay of 200 milliseconds. This is the delay between simulated
events. If we didn’t specify a delay then, at least with an asynchronous
system such as X11, we wouldn’t actually see anything displayed even though
the automation is happening. For the purposes of this example the delay means
that you can see exactly what is happening.
More on Identifying Widgets¶
When dip searches a GUI for the widget to be automated it stops when it finds
a visible widget with the required value of the
objectName property and
that can be adapted to implement the
interface or one of its sub-classes. Most of the time this is sufficient, but
there may be cases where there is more than one widget that meets these
criteria. This is particularly true if your application may have more than
one top-level widget (including dialogs) being displayed at the same time. It
shouldn’t be necessary for the developer of an automation script for one part
of an application to be aware of how the GUI’s of other parts of the
application that might be displayed at the same time are implemented
The string passed as the first argument to
simulate(), as well as being a simple object
name, can also be a sequence of object names each separated by a colon
character. Each object name is searched for in turn starting at the widget
found by the previous search (or all top-level widgets if this is the first
Note that you don’t need to provide a complete widget path, just sufficient information to resolve any potential ambiguities. It is recommended that you adopt a naming convention for any temporary top-level widgets, i.e. dialogs and wizards, and use this name (separated by a colon character) with the name of the particular widget.
Building Up Automation Sequences¶
When testing a modal user interface such as a typical dialog then it is actually necessary to record the simulated commands in this way so that they can be played once the event loop has started.
In the following more complicated example, which can be downloaded from
here, we show how to do this. We
also show the use of scoped object names to identify widgets, as described in
the previous section.
from dip.automate import Robot from dip.ui import Application, Dialog, IDialog, SpinBox # Every application needs an Application. app = Application() # Create the model. model = dict(name='', age=0) # Define the view. view = Dialog('name', SpinBox('age', suffix=" years"), id='dialog.person') # Create an instance of the view for the model. ui = view(model) # Create a robot with a default delay of 200ms between events. robot = Robot(delay=200) # Enter data into the two editors. robot.record('dialog.person:name', 'set', "Bill") robot.record('dialog.person:age', 'set', 30) # Click the Ok button. robot.record('dialog.person', 'click', 'ok') # Play the commands as soon as the event loop starts. robot.play(after=0) # Enter the dialog's modal event loop. IDialog(ui).execute() # Show the value of the model. print("Name:", model['name']) print("Age:", model['age'])
The following line declaratively defines the view as a dialog. Note that it
explicitly sets an
id which will be used as the
value of the
objectName property of the dialog widget that will be created.
view = Dialog('name', SpinBox('age', suffix=" years"), id='dialog.person')
The following line creates the
Robot instance. By
default the robot will delay for 200ms between simulating events.
robot = Robot(delay=200)
The following lines record the
set high-level commands that will simulate
the events for the widgets that are bound to the
attributes of the model.
robot.record('dialog.person:name', 'set', "Bill") robot.record('dialog.person:age', 'set', 30)
The widgets are identified using the
id of the
Dialog as a scope so that they don’t get confused with any
other widget with the same
age object names that the
application may be displaying at the same time.
The following line records the
'click' high-level command that will
simulate a click on the
robot.record('dialog.person', 'click', 'ok')
The following line plays the sequence of commands recorded so far. By default
commands are played immediately. In our example we must delay this until the
event loop starts, so we use the
after keyword argument to specify that
play should start 0ms after the start of the event loop.
Note that the sequences of commands can be played repeatedly if so desired.
Creating Automation Scripts¶
So far we have automated our examples by adding the automation calls to the
example code itself. This is fine when using automation to produce unit tests
as it makes sense to keep the automation and the code together in one place.
However when automating an existing application (either to get the application
to a state that a bug is apparent, or to produce a canned demonstration) you
don’t want to have to change the application code itself. dip provides the
dip-automate tool to allow a PyQt application to be run under the control
of an automation script.
As an example we will use a pure PyQt equivalent of the dialog example above.
This is shown below and can be dowloaded from
import sys from PyQt5.QtWidgets import (QApplication, QDialog, QDialogButtonBox, QFormLayout, QLineEdit, QSpinBox, QVBoxLayout) class Dialog(QDialog): """ Create a dialog allowing a person's name and age to be entered. """ def __init__(self, model): """ Initialise the dialog. """ super(Dialog, self).__init__(objectName='dialog.person') self._model = model # Create the dialog contents. dialog_layout = QVBoxLayout() form_layout = QFormLayout() self._name_editor = QLineEdit(objectName='name') self._name_editor.setText(self._model['name']) form_layout.addRow("Name", self._name_editor) self._age_editor = QSpinBox(objectName='age', suffix=" years") self._age_editor.setValue(self._model['age']) form_layout.addRow("Age", self._age_editor) dialog_layout.addLayout(form_layout) button_box = QDialogButtonBox( QDialogButtonBox.Ok | QDialogButtonBox.Cancel, accepted=self._on_accept, rejected=self.reject) dialog_layout.addWidget(button_box) self.setLayout(dialog_layout) def _on_accept(self): """ Invoke when the dialog is accepted. """ self._model['name'] = self._name_editor.text() self._model['age'] = self._age_editor.value() self.accept() # Every PyQt GUI application needs a QApplication. app = QApplication(sys.argv) # Create the model. model = dict(name='', age=0) # Create the dialog. ui = Dialog(model) # Enter the dialog's modal event loop. ui.exec() # Show the value of the model. print("Name:", model['name']) print("Age:", model['age'])
We won’t go through this code as it should be self explanatory. However it is interesting to note how many more lines of code are needed than the dip version.
dip-automate is a command line tool that takes the name of the application
as its argument. Any preceeding command line flags are handled by
dip-automate itself. Any following command line flags and arguments are
passed to the application being run. By default
dip-automate will run the
application as normal, without any automation. To automate our example we have
to write an automation script which we pass to
dip-automate using the
--commands command line flag.
The automation script itself is shown below and can be dowloaded from
from dip.automate import AutomationCommands # We are automating a non-dip application so we need to explicitly reference # the toolkit so that the adapters are registered. from dip.toolkits import Toolkit Toolkit().instance class AutomateName(AutomationCommands): """ This automation command will set the 'name' widget in the dialog. """ def record(self, robot): robot.record('dialog.person:name', 'set', self.value) class AutomateAge(AutomationCommands): """ This automation command will set the 'age' widget in the dialog. """ def record(self, robot): robot.record('dialog.person:age', 'set', self.value) class AutomateOk(AutomationCommands): """ This automation command will click the Ok button of the dialog. """ def record(self, robot): robot.record('dialog.person', 'click', 'ok') # Create the command sequence. automation_commands = (AutomateName("Bill"), AutomateAge(30), AutomateOk())
An automation script is an ordinary Python script that is run by
dip-automate before it runs the application. After it is run,
dip-automate inspects the script’s module dictionary for an object called
automation_commands which should be a sequence of
AutomationCommands instances. After the application’s
event loop is entered these commands will be executed in the order in which
they appear in the sequence.
We will now go through the script looking at the important sections.
The following line causes the automation adapters for the default toolkit to be imported. We have to do this only because the application being automated is not a dip application. A dip application would implicitly or explicitly reference a toolkit.
The following lines define a sub-class of
AutomationCommands that records the individual
high-level commands (only the single
'set' in this case) that implement the
command to set the dialog’s
The other class definitions are very similar and define the commands to set the
'age' widget and to click the
Note that in a complex environment these classes would probably be defined in separate modules and imported by each automation script that uses them.
Finally, the following lines show the definition of the sequence of automation commands that will be executed.
automation_commands = (AutomateName("Bill"), AutomateAge(30), AutomateOk())
All that needs to be done now is to run
dip-automate as follows:
dip-automate --commands=automate_pyqt_dialog.py pyqt_dialog.py