Windows GUI自动化控制工具之python uiAutomation

对 Windows GUI进行自动化控制的工具有很多，比如pywinauto、pyautogui、pywin32、Autoit、airtest、UIAutomation等，UI Automation API是微软提供的自动化框架，可在支持 Windows Presentation Foundation (WPF) 的所有操作系统上使用，支持的应用类型更多。本文介绍封装了UI Automation API的Python uiautomation 模块的使用方法。

目录

• 环境准备
• • uiautomation安装
  • 进程查看器
  • • inspect.exe
    • Accessibility Insights
• 控件对象模型
• uiautomation库示例
• • 控制计算器
• 参考文档

Python uiautomation 模块由yinkaisheng 开发，封装了微软 UI Automation API，支持自动化Win32，MFC，WPF，Modern UI(Metro UI), Qt, IE, Firefox, Chrome和基于Electron开发的应用程序。

uiautomation安装

最新版uiautomation2.0只支持Python 3版本，但不要使用3.7.6和3.8.1这两个版本，因为comtypes包在这两个版本中不能正常工作。

pip安装uiautomation：

$ pip install uiautomation

检查是否安装成功：

$ pip list | findstr uiautomationuiautomation       2.0.18

安装完成后，在Python的Scripts(我的路径为C:\Program Files\Python37\Scripts)目录中会有一个文件automation.py，是用来枚举控件树结构的一个脚本。

可运行 automation.py -h查看命令帮助:

$ python automation.py -hUIAutomation 2.0.18 (Python 3.7.2, 64 bit)usage-h      show command help-t      delay time, default 3 seconds, begin to enumerate after Value seconds, this must be an integer        you can delay a few seconds and make a window active so automation can enumerate the active window-d      enumerate tree depth, this must be an integer, if it is null, enumerate the whole tree-r      enumerate from root:Desktop window, if it is null, enumerate from foreground window-f      enumerate from focused control, if it is null, enumerate from foreground window-c      enumerate the control under cursor, if depth is < 0, enumerate from its ancestor up to depth-a      show ancestors of the control under cursor-n      show control full name, if it is null, show first 30 characters of control's name in console,        always show full name in log file @AutomationLog.txt-p      show process id of controlsif UnicodeError or LookupError occurred when printing,try to change the active code page of console window by using chcp or see the log file @AutomationLog.txtchcp, get current active code pagechcp 936, set active code page to gbkchcp 65001, set active code page to utf-8examples:automation.py -t3automation.py -t3 -r -d1 -m -nautomation.py -c -t3

进程查看器

对 Windows GUI进行自动化控制需要使用进程查看器工具对GUI界面元素进行定位，定位工具有很多，这里推荐使用微软提供的inspect.exe 或者 Accessibility Insights 这两款工具。

inspect.exe

inspect.exe 是 Windows SDK 自带的一个进程查看器，可以用来查看系统正在运行的进程信息、模块、线程、堆栈跟踪等详细数据。

Windows SDK下载地址为：https://developer.microsoft.com/en-us/windows/downloads/sdk-archive/

建议直接到这里下载inspect.exe：https://github.com/yinkaisheng/Python-UIAutomation-for-Windows/tree/master/inspect

64位系统版本的inspect.exe也可以点击这里下载。

Accessibility Insights

Accessibility Insights 是微软开发的一款辅助功能测试工具。它可以帮助开发者测试 web 应用、Windows 桌面应用和 Android 应用的可访问性，确保这些应用程序符合无障碍标准。

Accessibility Insights获取的控件属性信息没有inspect.exe全面，使用起来更加流畅。下载为：https://accessibilityinsights.io/downloads/

微软 UIAutomation API定义了支持的控件类型和对应的模型（Pattern），所有支持的控件类型可参考：https://learn.microsoft.com/en-us/windows/win32/winauto/uiauto-controlpatternmapping

python uiautomation库对UIAutomation API定义的各个Control和Pattern进行了封装。

下面来看使用python uiautomation操作Windows自带计算器的例子。

控制计算器

可以使用inspect.exe来定位计算器元素：

示例脚本如下：

import osimport uiautomation as autoimport subprocessclass uiautoCalc(Loggers):    """uiautomation控制计算器    """    def __init__(self):        super().__init__()        self.logger = Loggers().myLogger()        auto.uiautomation.DEBUG_SEARCH_TIME =True         auto.uiautomation.SetGlobalSearchTimeout(2) # 设置全局搜索超时时间        self.calcWindow = auto.WindowControl(searchDepth=1, Name='计算器', desc='计算器窗口') # 计算器窗口        if not self.calcWindow.Exists(0,0):            subprocess.Popen('calc')# 设置窗口前置            self.calcWindow = auto.WindowControl(            searchDepth=1, Name='计算器', desc='计算器窗口')        self.calcWindow.SetActive() # 激活窗口        self.calcWindow.SetTopmost(True) # 设置为顶层    def gotoScientific(self):        self.calcWindow.ButtonControl(AutomationId='TogglePaneButton', desc='打开导航').Click(waitTime=0.01)                self.calcWindow.ListItemControl(AutomationId='Scientific', desc='选择科学计算器').Click(waitTime=0.01)        clearButton = self.calcWindow.ButtonControl(AutomationId='clearEntryButton', desc='点击CE清空输入')        if clearButton.Exists(0,0):            clearButton.Click(waitTime=0)        else:            self.calcWindow.ButtonControl(AutomationId='clearButton', desc='点击C清空输入').Click(waitTime=0.01)    def getKeyControl(self):        automationId2key ={'num0Button':'0','num1Button':'1','num2Button':'2','num3Button':'3','num4Button':'4','num5Button':'5','num6Button':'6','num7Button':'7','num8Button':'8','num9Button':'9','decimalSeparatorButton':'.','plusButton':'+','minusButton':'-','multiplyButton':'*','divideButton':'/','equalButton':'=','openParenthesisButton':'(','closeParenthesisButton':')'}                calckeys = self.calcWindow.GroupControl(ClassName='LandmarkTarget')        keyControl ={}        for control, depth in auto.WalkControl(calckeys, maxDepth=3):            if control.AutomationId in automationId2key:                self.logger.info(control.AutomationId)                keyControl[automationId2key[control.AutomationId]]= control        return keyControl    def calculate(self, expression, keyControl):        expression =''.join(expression.split())        if not expression.endswith('='):            expression +='='            for char in expression:                keyControl[char].Click(waitTime=0)        self.calcWindow.SendKeys('{Ctrl}c', waitTime=0.1)        return auto.GetClipboardText()    def calc_demo(self):        """计算器示例        :return :         """                self.gotoScientific() # 选择科学计算器                keyControl = self.getKeyControl() # 获取按键控件        result     = self.calculate('(1 + 2 - 3) * 4 / 5.6 - 7', keyControl)        print('(1 + 2 - 3) * 4 / 5.6 - 7 =', result)        self.calcWindow.CaptureToImage('calc.png', x=7, y=0, width=-14, height=-7) # 截图        self.calcWindow.GetWindowPattern().Close() # 关闭计算机if __name__ == "__main__":    ui = uiautoCalc()    ui.calc_demo()

脚本执行动图：

1. https://github.com/pywinauto/pywinauto

2. https://cloud.tencent.com/developer/article/2213048

3. https://github.com/yinkaisheng/Python-UIAutomation-for-Windows

4. Python UIAutomation文档：https://github.com/yinkaisheng/Python-UIAutomation-for-Windows/blob/master/readme_cn.md

5. https://www.cnblogs.com/Yinkaisheng/p/3444132.html

6. GitHub - jacexh/pyautoit: Python binding for AutoItX3.dll

7. GitHub - mhammond/pywin32: Python for Windows (pywin32) Extensions

8. Accessibility tools - Inspect - Win32 apps | Microsoft Learn

9. Accessibility Insights

--THE END--

来源地址：https://blog.csdn.net/u010698107/article/details/130786941