I'm trying to conduct UI automation using AutoIt through use of the UI Automation library provided by @junkew. The reason why I'm using the UI Automation library and not just AutoIt's out of box controls is because the application I'm trying to automate was created using both standard and non-standard windows controls. Some were built in-house which is likely the reason for the difficulty I'm experiencing, but I believe there should be a way to do it because both the spy tool provided by @junkew and the Windows inspect.exe tool can both provide information about individual elements of the Grid, but I'm failing to find a coded solution. I would appreciate help in the right direction.
NOTE: I changed the name of the element of interest as well as the Window since the application is not mine to dispense.
At this moment I'm trying to solve two problems:
- Perform some allowed UI Automation action on the element of a Grid with the specified text.
- Extract all the elements within the Grid for analysis. Similar to the idea of a dump I guess.
The simplespy.exe application produces this code, which works to some degree. It puts the Grid in focus which brings the previously clicked row into focus, but it does not select a desired element or row with the text specified.
;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=WindowTitle;controltype:=UIA_WindowControlTypeId;class:=ATL:4476F950", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WTL_SplitterWindow", $treescope_children) _UIA_Action($oP3,"setfocus") Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WTL_PaneContainer", $treescope_children) _UIA_Action($oP2,"setfocus") Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770", $treescope_children) _UIA_Action($oP1,"setfocus") Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770", $treescope_children) _UIA_Action($oP0,"setfocus") _UIA_setVar("SalesorderEntry.mainwindow","title:=RowOfInterestText;classname:=GRID") _UIA_action("SalesorderEntry.mainwindow","setfocus")
simplespy.exe also produced this information about the focused element:
Mouse position is retrieved 2177-480 At least we have an element [RowOfInterestText][GRID] Having the following values for all properties: Title is: <RowOfInterestText> Class := <GRID> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 2037;318;588;466 *** Parent Information top down *** 5: Title is: <Desktop> Class := <#32769> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 0;0;1920;1200 "Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769" 4: Title is: <WindowTitle> Class := <ATL:4476F950> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) 2020;250;1256;583 "Title:=WindowTitle;controltype:=UIA_WindowControlTypeId;class:=ATL:4476F950" 3: Title is: <> Class := <WTL_SplitterWindow> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 2028;300;1240;502 "Title:=;controltype:=UIA_PaneControlTypeId;class:=WTL_SplitterWindow" 2: Title is: <> Class := <WTL_PaneContainer> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 2028;300;1240;502 "Title:=;controltype:=UIA_PaneControlTypeId;class:=WTL_PaneContainer" 1: Title is: <> Class := <#32770> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 2028;300;1240;502 "Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770" 0: Title is: <> Class := <#32770> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 2037;305;1224;479 "Title:=;controltype:=UIA_PaneControlTypeId;class:=#32770"
AutoIt
*** Detailed properties of the highlighted element *** UIA_title:= <RowOfInterestText> UIA_text:= <RowOfInterestText> UIA_regexptitle:= <RowOfInterestText> UIA_class:= <GRID> UIA_regexpclass:= <GRID> UIA_iaccessiblechildId:= <0> UIA_id:= <249> UIA_handle:= <4526846> UIA_RuntimeId:= <42;4526846> UIA_BoundingRectangle:= <2037;318;588;466> UIA_ProcessId:= <6892> UIA_ControlType:= <50033> UIA_LocalizedControlType:= <pane> UIA_Name:= <RowOfInterestText> UIA_HasKeyboardFocus:= <True> UIA_IsKeyboardFocusable:= <True> UIA_IsEnabled:= <True> UIA_AutomationId:= <249> UIA_ClassName:= <GRID> UIA_Culture:= <0> UIA_IsControlElement:= <True> UIA_IsContentElement:= <True> UIA_IsPassword:= <False> UIA_NativeWindowHandle:= <4526846> UIA_IsOffscreen:= <False> UIA_Orientation:= <0> UIA_FrameworkId:= <Win32> UIA_IsRequiredForForm:= <False> UIA_IsDockPatternAvailable:= <False> UIA_IsExpandCollapsePatternAvailable:= <False> UIA_IsGridItemPatternAvailable:= <False> UIA_IsGridPatternAvailable:= <False> UIA_IsInvokePatternAvailable:= <False> UIA_IsMultipleViewPatternAvailable:= <False> UIA_IsRangeValuePatternAvailable:= <False> UIA_IsScrollPatternAvailable:= <False> UIA_IsScrollItemPatternAvailable:= <False> UIA_IsSelectionItemPatternAvailable:= <False> UIA_IsSelectionPatternAvailable:= <False> UIA_IsTablePatternAvailable:= <False> UIA_IsTableItemPatternAvailable:= <False> UIA_IsTextPatternAvailable:= <False> UIA_IsTogglePatternAvailable:= <False> UIA_IsTransformPatternAvailable:= <False> UIA_IsValuePatternAvailable:= <False> UIA_IsWindowPatternAvailable:= <False> UIA_ValueIsReadOnly:= <True> UIA_RangeValueValue:= <0> UIA_RangeValueIsReadOnly:= <True> UIA_RangeValueMinimum:= <0> UIA_RangeValueMaximum:= <0> UIA_RangeValueLargeChange:= <0> UIA_RangeValueSmallChange:= <0> UIA_ScrollHorizontalScrollPercent:= <0> UIA_ScrollHorizontalViewSize:= <100> UIA_ScrollVerticalScrollPercent:= <0> UIA_ScrollVerticalViewSize:= <100> UIA_ScrollHorizontallyScrollable:= <False> UIA_ScrollVerticallyScrollable:= <False> UIA_SelectionCanSelectMultiple:= <False> UIA_SelectionIsSelectionRequired:= <False> UIA_GridRowCount:= <0> UIA_GridColumnCount:= <0> UIA_GridItemRow:= <0> UIA_GridItemColumn:= <0> UIA_GridItemRowSpan:= <1> UIA_GridItemColumnSpan:= <1> UIA_DockDockPosition:= <5> UIA_ExpandCollapseExpandCollapseState:= <3> UIA_MultipleViewCurrentView:= <0> UIA_WindowCanMaximize:= <False> UIA_WindowCanMinimize:= <False> UIA_WindowWindowVisualState:= <0> UIA_WindowWindowInteractionState:= <0> UIA_WindowIsModal:= <False> UIA_WindowIsTopmost:= <False> UIA_SelectionItemIsSelected:= <False> UIA_TableRowOrColumnMajor:= <2> UIA_ToggleToggleState:= <2> UIA_TransformCanMove:= <False> UIA_TransformCanResize:= <False> UIA_TransformCanRotate:= <False> UIA_IsLegacyIAccessiblePatternAvailable:= <True> UIA_LegacyIAccessibleChildId:= <0> UIA_LegacyIAccessibleName:= <RowOfInterestText> UIA_LegacyIAccessibleRole:= <10> UIA_LegacyIAccessibleState:= <1048580> UIA_IsDataValidForForm:= <False> UIA_ProviderDescription:= <[pid:9208,hwnd:0x4512FE Main:Nested [pid:6892,hwnd:0x4512FE Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]> UIA_IsItemContainerPatternAvailable:= <False> UIA_IsVirtualizedItemPatternAvailable:= <False> UIA_IsSynchronizedInputPatternAvailable:= <False>
I experienced success as well with the Windows inspect.exe in identifying the text of the element of interest when I put focus on it:
AutoIt
This is what inspect.exe shows: How found: Mouse move (2161,483) hwnd=0x004512FE 32bit class="GRID" style=0x52810371 ex=0x4 RuntimeId: "[42.4526846]" BoundingRectangle: {l:2037 t:318 r:2625 b:784} ProcessId: 6892 ControlType: UIA_PaneControlTypeId (0xC371) LocalizedControlType: "pane" Name: "RowOfInterestText" AccessKey: "" HasKeyboardFocus: true IsKeyboardFocusable: true IsEnabled: true AutomationId: "249" ClassName: "GRID" HelpText: "" IsPassword: false NativeWindowHandle: 0x4512FE IsOffscreen: false FrameworkId: "Win32" ProviderDescription: "[pid:6872,hwnd:0x4512FE Main:Nested [pid:6892,hwnd:0x4512FE Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]" LegacyIAccessible.ChildId: 0 LegacyIAccessible.DefaultAction: "" LegacyIAccessible.Description: "" LegacyIAccessible.Help: "" LegacyIAccessible.KeyboardShortcut: "" LegacyIAccessible.Name: "RowOfInterestText" LegacyIAccessible.Role: client (0xA) LegacyIAccessible.State: focused,focusable (0x100004) LegacyIAccessible.Value: "" IsDockPatternAvailable: false IsExpandCollapsePatternAvailable: false IsGridItemPatternAvailable: false IsGridPatternAvailable: false IsInvokePatternAvailable: false IsLegacyIAccessiblePatternAvailable: true IsMultipleViewPatternAvailable: false IsRangeValuePatternAvailable: false IsScrollPatternAvailable: false IsScrollItemPatternAvailable: false IsSelectionItemPatternAvailable: false IsSelectionPatternAvailable: false IsTablePatternAvailable: false IsTableItemPatternAvailable: false IsTextPatternAvailable: false IsTogglePatternAvailable: false IsTransformPatternAvailable: false IsValuePatternAvailable: false IsWindowPatternAvailable: false IsItemContainerPatternAvailable: false IsVirtualizedItemPatternAvailable: false IsSynchronizedInputPatternAvailable: false FirstChild: [null] LastChild: [null] Next: "Menu" text Previous: [null] Other Props: Object has no additional properties Children: Container has no children Ancestors: "" dialog "" dialog "" pane "" pane "WindowTitle" window "Desktop" pane [ No Parent ]