The Automation item is analagous to the Test item in 8.2, and they've made one huge change that I consider unnecessary and bothersome. They've moved Test Settings under the File menu. That's probably a more logical place, but here's the problem: in the past, when you couldn't remember where to go to change the Row Iterations, or what URL to use to open the browser, or Function Library references, chances were that if you chose Record and Run Settings and didn't find it, you could be assured that you could just choose "the other one in that menu" and it would be there. Now there are not one, not two, but three different places you have to go, none of them close together, in order to set up Test Run Settings. FileSettings, ToolsOptions, AutomationRecord and Run Settings - each of these menu chunks has the word "Run" in it, and who besides me can't remember which one has what?
One of my fondest hopes for QTP 9 was the ability to map many more functions to hotkeys. Alas, I'll have to wait a little longer. With the new clicks further obfuscating their most basic functionality, I'm find a lot of the seconds I'm saving on an improved feature taken away by another. I want to be able to go to Action Properties with a simple hotkey, but I can't map my own. This makes me sad.
I also can't change the annoying behavior that causes ALT+Backspace to erase an entire line of code instead of just the previous word. I don't know what interface precedent told them that was a good idea, but I'd advise them either to follow the Visual Studio interface (let's face it, the V in VBScript stands for Visual, which means Microsoft), or to give me mappings for every blessed function they provide. It's driving me nuts!
Ok, so I've covered the new menu bar and various niggling annoyances. Next I'll talk a little about the new window pane behavior, including the new right-click menu options and toolbars.
|W|P|114348288925867514|W|P|New Interface Features, Quirks, and What-have-yous in QuickTest 9.0 - Part I|W|P|mmerrell@gmail.comHow do you measure the overall quality of a product? One way is to take the spec, take the final product, and compare the two feature by feature. Well, QTP 9 went GA today, and we've pored over it so thoroughly you'd think we've had it for months. First I'll present the summary, then over the next few days we will dissect the minutae of what they did, how well they did it, and whether or not it's going to make your lives easier. Here's a bullet-by-bullet summary of how well they did against their own press release:
Whenever you call a checkpoint, there's a lot of stuff going on, very little of which you have control over. You can put in some things that make it more flexible, like regular expressions and comparisons to other output values, but in the end you have this big black box of stuff that you can't really get at. Plus, in the end, you have a Reporter.ReportEvent that goes to the test results as a Pass or Fail, and you have no way to alter the strings. The problem with that is, as I've said in previous posts, I want different tests to behave differently in different circumstances, but I don't want to rewrite any of the code. When I'm not running deep or strict tests, I want the test results to be sparse, just letting me know when the broad strokes of the tests have been completed. The Test Results Viewer that comes with QTP is difficult enough to use (e.g. it doesn't remember filter settings, etc.), so when I filter the test results based on pass/fail, I want to zero in quickly on the tests that were important to me when I ran the tests. Sometimes we use test actions as a means to an end, not an end in themselves, and in those cases I don't want to see every detail. I have a lot of other problems with the checkpoints as they are, but some of my problems may be a lack of education. My team abandoned checkpoints too early to really explore the ins and outs and what-have-yous (mainly because of the reporting problem), so I'd like to hear from anyone who is still using them and gets what they need out of them. I'll state some of the problems below. I'm not 100% sure that I'm right on these, so pretend that with all of them I've added "This is the experience I've had, and I've seen no evidence to contradict it. If you have some knowledge to impart, by all means let me know." Some of these problems are:
Like I said, there may be other ways to do this, but I haven't found them. When I have coded around things to shoehorn my checkpoints into working, I've often later found them to be very fragile. I had an occurrence recently of a long, drawn-out test script, that probably contained over a thousand lines of code. The one line of code that broke consistently and with no good error message was the one leftover checkpoint from my earlier days. I finally removed it and replaced it with our new, preferred method.
What is that new, preferred method? I'll talk about that soon. To give you a hint, think about JUnit.
In the meantime I want to talk a little more about Test Level, and the different levels we've defined. There are 6 levels, and we're retaining the option to have more. Each level gives an indication of how strictly it behaves, and how its behavior is recorded into the test results.
The nice thing is that you can ratchet up or down the Test Level as the test goes on. You can test shallow in order to achieve some sort of setup, like object or scenario creation, then knock it up a level or two when you start the really important part of the test.
Our company has tasked us with asking a bit more out of our automated tests than is normally recommended. We understand that you're not usually successful when you try to do outrageously large regression tests, or to simulate manual scripts. But they want something in between "normal" automation and 100% regression coverage, and techniques like this allow us to move much closer to that goal than ever before.
The ability to take the same scripts that perform the low-level field validation, then employ them as "just a step along the way", has been paramount. It takes a while to develop the initial tests, but the development gets faster and faster as we build our library. Then, with very little effort, we can build a smart suite that can be reused against new builds, against customer-deployed sites, and against new versions of the product.
And we don't use Mercury's checkpoints anywhere.
Next, I'll post an example of another Smart method, the one we use for WebLists. It employs heavy use of Test Level to determine its behavior and logging.
|W|P|114193312526571647|W|P|Test Level and Checkpoints in QuickTest Pro (9 and earlier)|W|P|mmerrell@gmail.com
Public Function WebRadioGroupSelectByText ( ByRef oRadGroup, sSelectText, sWhereOpt, sIndexOpt )
If isNull(sWhereOpt) Then
sWhereOpt = "afterEnd"
End If
If isNull(sIndexOpt) Then
sIndexOpt = 0
End If
sMatches = Array()
Set oParent = oRadGroup.GetTOProperty("parent")
radName = oRadGroup.GetROProperty("name")
Set oRadioButtons = oParent.Object.GetElementsByName( radName )
For each oElement in oRadioButtons
sName = oElement.getAdjacentText("afterEnd")
If sSelectText = sName Then
ArrayPush sMatches, oElement.Value
End If
Next
If Ubound(sMatches) < 0 Then
' TODO: report error...item doesn't exist in list.
Exit Function
End If
If Ubound(sMatches) < sIndexOpt Then
sIndexOpt = Ubound(sMatches)
' TODO: report warning...index doesn't exist. selecting last match
End If
sSelectValue = sMatches(sIndexOpt)
oRadGroup.Select sSelectValue
End Function
RegisterUserFunc "WebRadioGroup", "SelectByText", "WebRadioGroupSelectByText"
Now, to select the radio button I want, I just to this:
Browser("B").Page("P").WebRadioGroup("All Options").SelectByText "Option B"
Note that this performs an exact string match. The next step is to create our WebRadioGroup.SmartSelect function, which would be much more flexible, and would allow for partial matches of either adjascent text or the traditional button values. It would also allow for pattern matching via regular expressions. We'll post that one later.|W|P|114295813788482253|W|P|QuickTest Pro (9.0 or 8.2) method: WebRadioGroup.SelectByText|W|P|mmerrell@gmail.comYou know what? Those are all slightly different wordings of the same thing. I only want to write the code once. Ever.
How do we do this in QuickTest Pro? Well, that's the thing. Checkpoints and Outputs are not the most flexible things in the world. Neither are External Actions. Neither, really, is the Object Repository. We've come up with a way to accomplish the above goals without having to rely on the dodgy parts of QTP. It means we're leaving huge swaths of Mercury's code untouched, but sometimes that's the only way to get things done, isn't it?
More soon.
|W|P|114282502530542944|W|P|Strictness (or: The Problem of Inflexibility in QuickTest Pro)|W|P|mmerrell@gmail.comprintsentence(
'Subject' => 'The quick brown fox',
'Predicate' => 'jumps over the lazy dog.'
);
sub printsentence {
my %args = @_;
print "$args{'Subject'} $args{'Predicate'}";
}
I found an article on 4guysfromrolla.com about using optional arguments in VBScript. It suggests passing an array of arguments to the function. This is a good first step, but I don't like having to remember which value is which in the array. I still wanted my named arguments. Obviously I couldn't use a hash for this, so I used VBScript's nearest data type: a dictionary object. Once I have my parameters in a dictionary object, I can call them by name using syntax like oOptions("optionname"). The question was how to easily get all my options in a dictionary object.
I came up with a function I call DictBuild (insert platypus-related humor here). It takes an array and turns it into a dictionary object. The first value in the array is a dictionary element, the second value is that element's value, third - element, fourth - value...and so on. Basically, all even numbered values are dictionary elements and the odd value after each element name is its value.
This allows me to easily build a dictionary object with all my parameters.
printsentence ARRAY( _
"Subject" , "The quick brown fox" _
,"Predicate" , "jumps over the lazy dog." _
)
Public Function printsentence(aOptions)
Set oOptions = BuildDict(aOptions)
MsgBox oOptions("Subject") & " " & oOptions("Predicate")
End Function
Now this is finally becoming something I can work with, but what about defaults for these parameters? This is where my next function comes in: GetOpts. GetOpts takes two arrays as arguments and returns a dictionary object. The first array is the default values for the function. The second array is the custom options array that was passed to the function. GetOpts starts by building a dictionary object from the defaults array. Then it overwrites the default values with the custom options array if they exist. I also wrote some discipline for myself into GetOpts. If a option isn't listed in the defaults, it won't make it into the final dictionary.
So, now my function looks like this:
printsentence ARRAY( _
"Subject" , "The quick brown fox" _
,"Predicate" , "jumps over the lazy dog" _
)
Public Function printsentence(aOptions)
Set oOptions = GetOpts( ARRAY( _
"Subject" , "Coke" _
,"Predicate" , "is it" _
,"Punctuation" , "!" _
), aOptions)
MsgBox oOptions("Subject") & " " & oOptions("Predicate") & _
oOptions("Punctuation")
End Function
I think that about does it for my method of using named optional arguments in VBScript functions. I should note that I only use this for optional arguments, I put the variable names directly in the function signature for required values. So one of my real function delcarations looks like
Public Function WebCheckBoxSelectByText ( ByRef oCheckBox, sSelectText, aOptions )The code for BuildDict and GetOpts is below.
Public Function GetOpts(ByRef aDefaults, ByRef aCustom) Set oDefaults = DictBuild(aDefaults) Set oCustom = DictBuild(aCustom) For Each vKey in oDefaults If oCustom.Exists(vKey) Then oDefaults.Item(vKey) = oCustom.Item(vKey) End If Next Set GetOpts = oDefaults End Function
Public Function DictBuild(ByRef aArray)
bWord = True
Dim oDict
Set oDict = CreateObject("Scripting.Dictionary")
Dim sWord, sDef
If isArray(aArray) Then
For Each sString in aArray
If bWord Then
sWord = sString
bWord = False
Else
sDef = sString
oDict.Add sWord, sDef
bWord = True
sWord = NULL
sDef = NULL
End If
Next
End If
set DictBuild = oDict
End Function
Technorati Tags: VBScript, Quicktest Pro, Software Inquisition, Software TestingWhile we're talking about QTP 9, let's put together a list of nice-to-haves that we currently lack. I've found that most of my time is spent fighting with QuickTest Pro, not composing automated tests. I have to solve the same problems multiple times, and I get stuck doing things that should be extremely easy. In their attempt to make an automation tool that is easy for both basic and advanced users, Mercury has given the advanced users the short shrift.
Here's a list of things that would make my life easier. I want this list to be a lot longer, so please, comment away. We've found the MIC is pretty receptive to suggestions like this (all of these have been submitted as SRs, by the way), so keep them coming!
I'll probably have a 2.0 version of this, as well as an update whenever QTP version 9 comes out. Until then, let's make this list very very long.
|W|P|114193324034857291|W|P|Wish List for QuickTest Pro 9.0|W|P|mmerrell@gmail.comPublic Function WebCheckBoxSmartSet (ByRef oCheckBox, Setting)Start with the function declaration. Note that the first parameter is the CheckBox in question. Whenever you register a user-defined function to one of Mercury's Test Objects, the object itself is the first parameter. The second is a Variant, meaning it will accept any primitive (i.e. non-object) data type.
'The checkbox will be turned OFF unless it is explicitly set ON
sFinalSetting = "OFF"
The first thing to do is create a variable called "sFinalSetting", which is set to "OFF" by default. At the end of the method, we will call the native WebCheckBox.Set() method using this variable, and the bulk of the code in the method is bent on scrubbing the Setting parameter, and guaranteeing that the sFinalSetting will either be "ON" or "OFF".
Select Case TypeName(Setting)
Case "Integer"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
Case "Byte"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
.
.
.
End Select
The only thing that can happen in the Select statement coming up is that sFinalSetting can be changed from "OFF" to "ON". If none of the Case statements are TRUE, it drops off the end and remains set to "OFF".
oCheckBox.Set sFinalSetting
End Function
Finally, we call the native Set method, and end the function. The only thing left to do is register the method to the WebCheckBox object.
RegisterUserFunc "WebCheckBox", "SmartSet", "WebCheckBoxSmartSet", TrueNow you can send it just about anything and it'll be able to make a good guess as to how the checkbox will be set. Flexible = good. Stiff as a 2x4 = bad. Here's the full code, showing all data types, which ones we've left out, and why:
Public Function WebCheckBoxSmartSet (ByRef oCheckBox, Setting)
'The checkbox will be turned OFF unless it is explicitly set ON
sFinalSetting = "OFF"
Select Case TypeName(Setting)
Case "Integer"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
Case "Byte"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
Case "Long"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
Case "Single"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
Case "Double"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
Case "Decimal"
If Setting <> 0 Then
sFinalSetting = "ON"
End If
Case "String"
If Setting = "ON" Then
sFinalSetting = "ON"
End If
Case "Boolean"
If Setting Then
sFinalSetting = "ON"
End If
'If it is Empty, Null, Nothing, or Unknown, we'll let it remain "OFF"
'We're not handling the following types: Date, Object, Unknown,
'Currency, or Error, they'll remain "OFF"
'If it's anything else, it'll remain "OFF"
'It should be sufficiently scrubbed at this point to be valid.
End Select
oCheckBox.Set sFinalSetting
End Function
RegisterUserFunc "WebCheckBox", "SmartSet", "WebCheckBoxSmartSet", True
|W|P|114226594199110540|W|P|QuickTest Pro method: WebCheckBox.SmartSet|W|P|mmerrell@gmail.comregsvr32 RepositoryUtil.dll
In the same directory with the DLL is a short rtf document detailing some things about the API, including sample code. It is by no means comprehensive, but the examples are good enough to get you going.
There is an AddObject method, though it's best used when you're opening one OR, reading an object from it, and putting this object into another OR. You can create objects from scratch, but there is a lot of initialization involved, and I've never had a compelling enough case to go through all that. I'm sure you could do it with some perserverance. If you do, let us know and post some code.
Here's a snippet of VB code that, once the dll is registered, you should be able to use to get an OR object, and to start manipulating and changing it:
Dim ImageObj, PageObj, RepositoryFrom, RepositoryTo
Set RepositoryFrom = CreateObject ("Mercury.ObjectRepositoryUtil")
Set RepositoryTo = CreateObject ("Mercury.ObjectRepositoryUtil")
RepositoryFrom.LoadActionRepository "E:\Temp\Tests\Test1", "Action1"
RepositoryTo.Load "E:\Temp\Tests\Default.tsr"
That gets you access to the object so you can start making method calls.
I haven't written much code in VBS for this stuff, so it's time to move to Perl. The syntax is almost the same in this kind of script, so you shouldn't have any trouble following along. This code doesn't do much, but it should give you a basic idea. In the next few days I'll be able to produce my XML-generation code. I posted it to the Mercury forum once, but I think it got buried (as usual) beneath all the crap.
use Win32::OLE;
my ($ImageObj, $PageObj);
my $Rep = Win32::OLE->new("Mercury.ObjectRepositoryUtil");
$Repository->Load('HC.tsr');
#########################################################
# The function enumerates all the Test Objects under a
# given root, printing out the keys and values.
# This could be used to generate a hash, which can
# then be used to generate test script code or an
# XML dump.
#########################################################
sub enumProps
{
my $root = shift;
my ($TOCollection, $TestObject, $PropertiesCollection, $Property, $Msg);
$TOCollection = $Repository->GetAllObjects();
for my $i (0..$TOCollection->Count - 1)
{
$TestObject = $TOCollection->Item($i);
$name = $Repository->GetLogicalName($TestObject) . "\n";
$PropertiesCollection = $TestObject->GetTOProperties;
for (0..$PropertiesCollection->Count - 1)
{
print $PropertiesCollection->Item(n);
print $Property->Name . "-" . $Property->Value . "\n";
}
}
return $or;
}
my $or = enumProps('Browser');
1;
|W|P|114201090356556968|W|P|Using the Object Repository API in QuickTest Pro|W|P|mmerrell@gmail.comPublic Function WebObjectGetBrowser (oWebObject)
Set oWebObject2 = oWebObject
While Not oWebObject2.GetROProperty("micclass") = "Browser"
Set oWebObject2 = oWebObject2.GetTOProperty("parent")
Wend
Set WebObjectGetBrowser = oWebObject2
End Function
Now, with the WebObjectGetBrowser function if I am operating on a table and need to do a sync before the next operation to keep an exist(0) from failing, I can use code like this.
'...operating on the object oTable... Set oBrowser = WebObjectGetBrowser(oTable) oBrowser.Sync '...continue working with oTable...This works, but it is still one line too long and creates the oBrowser object that isn't otherwise necessary. I do a lot of syncing, so I wrote another function and registered it to WebTable, Frame and several other classes.
Public Function WebObjectSync(oWebObject) Set oBrowser = WebObjectGetBrowser(oWebObject) oBrowser.Sync End Function RegisterUserFunc "Frame", "Sync", "WebObjectSync" RegisterUserFunc "Image", "Sync", "WebObjectSync" '...more register entriesNow my previous two lines can be reduced to one.
oTable.Sync