Observe, decide, act, check
A workflow can look for the image of a button or read text using optical character recognition, usually called OCR. It then uses that evidence to choose the next step.
For example, a report tool displays “Ready” after loading. The workflow checks for that state, selects Export and verifies the result. Each observation has a purpose in the sequence.
When the screen is the available interface
This approach can help with desktop or legacy applications that lack a useful integration. It can also complement other automation methods when part of a task remains manual.
A supported API or accessible UI control can be a better option when available. Visual matching is useful because it operates on the visible interface, but that also makes it sensitive to how the interface is displayed.
Recording and state checks solve different parts
A recording captures a sequence of interactions. A state check asks whether the application is ready for the next one. Many useful workflows combine both.
Waiting for a recognisable state can reduce dependence on a fixed pause. It does not make the workflow immune to redesigns, unexpected dialogs or incorrect recognition.
Build around evidence and limits
Keep recognition regions focused and decide how the workflow should finish before you run it repeatedly.
- Define an observable start state and an observable result.
- Choose images for recognisable appearance and OCR when text content matters.
- Set timeouts, bounded retries and a deliberate failure outcome.
- Test changes in timing, scale, focus and expected error states.