Sitefinity + Test Studio = Better Websites; Faster Launches (Webinar) Video Lecture | Study Telerik TestStudio:Automated Testing made Easy - Software Testing | Best Video for Software Testing

all right we're going to go ahead and
get things kicked off today once again
thank you for attending today's webinar
entitled Sitefinity plus tests to do
better websites faster launches a couple
of housekeeping items to go through real
quick all callers are muted throughout
the call but please feel free to submit
questions or comments in the questions
panel or the chat panel and we'll have a
follow-up blog post with all the popular
questions and answers as well as a link
to the recording so we will be recording
today's session as well first by way of
introduction my name is Andy Weiland I'm
a senior sales engineer for test studio
and certified scrum master and we're
working with telluric and progress for
about four and a half years and first of
all I want to start with the end
basically what most people look to get
out of test automation so a lot of our
customers come to us initially trying to
get away from either a manual process or
what we refer to as a bloated coated
process for test automation and what
they hope to achieve is this sort of
post automation day-in-the-life where
you essentially can start your day with
email results or results of your test
list via email that have been
automatically sent to you possibly from
tesla's that have been running overnight
or over the weekends or something like
that and this actually allows you to
focus on what's failing instead of
what's what's working and that's the
catch 22 of the QA world where if you
spend 30 minutes trying to prove
something works and it does it's as if
you just waste at that time because you
didn't find any failure so test to do
really turns that around it lets you
review that failure information and go
to what's not working in the application
so you don't have to waste your time
you'll see in the foreground here the
failure details we'll be taking a look
at that along the way as we create some
tests also we have a great integration
for bug tracking tools available to
submit bugs directly into the
development process and this really
empowers the QA to be more agile and to
be more of a participant in the
application lifecycle as well this also
creates a feedback loop not just the
buzz word that everybody likes to throw
around but an actual feedback loop
and that powers again the QA team they
can simply report bugs review things
review test failure details see if it's
a test issue see if it's a application
issue and this allows them to then
communicate it back into the team with
integrations to different bug tracking
tools here as well as custom integration
if you D if you do need to create an
above tracking integration and beyond
what's in the list there we also have
Tesla scheduling we've got a lot of data
that's captured at the point of failure
that you're going to see - and this can
all be also integrated into continuous
integration processes and build servers
like Jenkins the other big benefit here
on failure details is we we love to get
failure details from our customers
because we do have a great support team
it's engineer level support it's 24 hour
guaranteed response time support so when
you have a a test that's failing you
have the option to get help from the
development staff it's if it's an
application issue or zip it and attach
it to a support ticket and get help from
our support if it's a test issue or if
it's something you need help with on
test studio now test to do takes a
different approach to test creation and
test maintenance that you're gonna see
today first of all we have four
solutions and every problem this means
as you're creating a test if it doesn't
work one way you've got three other
options to make it work and three out of
these four are coatless options so
ultimately we try to reduce code as much
as possible so that our developers can
spend time on creating features rather
than test maintenance and our QAS can
have more capability and be part of the
process and be more involved and also be
able to maintain the tests rather than
relying on our development resources so
the first of these solutions is point
click record will hit record we'll do it
a user does will capture all the actions
and elements along the way if that
doesn't work we can invoke our build a
step in our toolbar as you can see
here's an example of creating an image
verification we'll also play with today
- still a codeless process if that
doesn't work we can use location-based
interactions or
XY coordinates which is where a lot of
other tools live any any tool that's
using XPath for example is simply
finding something by a location really
guessing that something is in the right
location whereas testico approaches this
very differently but that is an option
and then if that doesn't work you can
code it in c-sharp for vb.net you have
the ability to really work with it looks
like we have maybe a little bit of an
audio hitch but I think we're coming in
clear now so ultimately four ways in
which you can create up do we or we
still lose an audio we may may be having
a little bit of audio trouble here only
for some okay all right so four
different solutions to basically solve
every problem in creating that test this
also gives us the ability to reuse a
theme throughout tests to do is reuse so
when you capture an element which may be
a button an input a checkbox a grid we
basically will recognize that store it
globally across the project and reuse it
create new references to it along the
way if you need to use it another test
this allows you to update a broken or
changed element and apply it across all
of the tests our tests can be evolved
very very easily rather than being
thrown away and starting over all the
time we also can of course record a test
in one browser and run it across four
other browsers so five browsers
total that we can execute against and
that comes by way of our sophisticated
fine logic which you'll see next we also
have the ability to modify a base URL
setting and then hit different
deployments of the application so when
you finish getting everything ready on
your site Finity sites in staging and
you promote it to production you can
actually rerun the same list of tests
against production to make sure that
everything did translate over correctly
in that final final change they're also
tests a step you'll see this in action
to test within other tests why you know
reinvent the wheel you can just reuse
what what works for you so if you've got
a common function like logging into an
application you can simply just reuse
that rather than
creating it every time and also we'll
take a quick look at load testing today
but performance and load testing are
included in test studio --all ultimate
and you can certainly just reuse your
functional test your functional
regression test that you create as
performance tests to see where the
bottlenecks are in your application and
inside of our load test to see what
capacities your test or your servers can
handle also just ease of use copy and
paste capability throughout this makes
it easy to work with under the hood this
is what we call the Corvette engine of
test studio the ability to locate
elements in a way that's responsive in a
way that's sophisticated we locate
elements as you could see in the example
the background here locating a login
button by the fact that it says the word
login on the button that's how users
work so we can do that but we can also
locate things by ID or maybe an H ref or
all sorts of other attributes of the
elements that don't tie it to a location
if you also do have the ability to use
location-based identifiers but they are
at the bottom of the list for reason
they're weaker identifiers they're
subject to breaking easily when things
change and things can change from
browser to browser this is why our tests
work across browser because this logic
doesn't change from browser to browser
if it says login and I each should say
login and Chrome right you'll also see
an example later where I can date a
drive text content when I'm using it as
an attribute and actually run the same
test in multiple languages now the
project level you can see here where we
can prioritize the attributes to create
a backup search as well so tests to do
and this is completely customizable for
what type of application you're working
with SOPA test REO can look for
automation IDs unique IDs labels and
buttons or text within rows for dynamic
grits and things like that and move on
down the list to the weak or weaker
attributes and tags so this is something
that can come you can custom tailor for
your sites to make sure that test studio
knows best how to work with your
application and provides a backup method
when the best method doesn't work so a
lot of times you'll
if you do have a failing test it'll be a
failure with suggestions the ability to
create chain find expressions to see an
example here in a couple slides where we
talk about that and also of course the
ability to fix these elements code
lessly and apply it across the entire
project a couple other items on codelist
tests maintenance you get a tool bar
that gives you a lot of power to add
verifications in there's actually some
specific verifications to our tool Erik
controls which siphon today is
chock-full of so you can get some added
benefits just by using this with insight
Finity you can see an example of kind of
specific kind of grid specific ones but
we also work with third-party custom
controls really any web application we
can work with so you can easily add
verifications on-the-fly with the
element highlighter you can edit
elements against the live UI to make
sure it's working correctly working
right you can create very advanced
things like elements element drag and
drops were my favorite things because
when elements change because you've got
a responsive design application when it
renders down to a different size or
form-factor those elements move and if
you're trying to rely on XY coordinates
and XPath that's not going to work with
with the way we approach it with our
smart sophisticated file logic we can go
find element a go find element B and
always make the right connection and a
drag and drop scenario so that's a
really powerful capability that you get
again from our codeless capabilities
here you can see an example of an
element that's been changed and applying
that change across all of the tests that
use that elements that reuse in action
there's a lot of things here to help you
keep the tests codeless right minimizing
the need for code wouldn't wouldn't you
now here's more about the dynamic grid
situation because a lot of people will
say well we don't believe that this is
true right we've been burned by test
automation in the past that made
promises like this it's true you can see
for yourself and we've got some great
videos as well if you'd like to check
those out that go into tutorial and how
to do this but basically we can
consistently locate specific records
within dynamic grid situations so you
can handle grids that are sorted
filtered
aged and basically rather than using an
XPath we can use a chain finding
expression as we call it that can locate
the grid itself the row within the grid
with maybe a partial contains operator
on an inner text and then also leave the
edit button in this case that belongs to
that particular row so this is very
challenging to do it's almost really
impossible to do this with location best
based identifier so this will really
give you a lot more precision and power
in your testing we can use a combination
of location and content if you want to
as well so we can make sure something's
in the right cell of a grid for example
or we can go find it no matter which
cell it's in or we can even page through
that grid and find it on different pages
as well we can even day to drive this
mic and data drive is one component of
this fine logic and actually created a
loop out of that as well that would say
okay now go to another row and click
it's edit button then another row and
click it's edit button so a lot of very
strong capabilities that we've built in
for them now behind all this we have the
test studio licensing where you might
have ultimate licenses to create and
maintain tests that's where your QA czar
living all day and then you've got your
runtime licenses that give you a Swiss
Army knife to the test environment you
can create test scheduling servers you
can create execution agents you can have
performance monitoring and profiling
with the runtime so the runtimes bring
together a lot of components they're
very inexpensive and they're a great way
to build out your test lab so that you
can make sure you can cover all the
versions of all the browsers and things
that you need to test and you can have
that all running while you're asleep
this is brought to you of course my
progress and of course it comes along
with the support that we're known for
and as you can see is an example here
just on the help tab within tests to do
you have access to forums and videos and
white papers and blogs and training and
roadmaps and all sorts of stuff to get
you what you need we've got a great
community behind all our products and of
course test studio is included in that
so you can certainly go out into the
forums and
and share and find solutions with with
coworkers and co users I should say but
also you've got the ability to to ask
for help from support so again having
that support is critical to making sure
you get success and of course if you
need additional help we've got training
options and professional services along
the way too so we're gonna jump over now
to test studio now I'll give you a
little bit of a tour around here real
quick what we're looking at is a test
open and the test tab in the middle here
on the Left we have our project
organized and this project is actually
synchronized with TFS you see here's
source control options we do integrate
our source control options with git as
well as TFS and this again promotes
collaboration we're storing the tests
where the code is stored we're basically
living in the same world as the
development team and providing a lot of
rich data into that process we can check
in we can check out
we have chain of custody we have
versioning we have a conflict resolution
that comes with that as well in addition
you can see some integration points for
Team polls project management or quality
Center if you're still using that as
well as bug tracking options to team
Team Foundation server again JIRA as
well this allows us to create that
feedback loop and as I mentioned there's
great documentation you can get to just
by clicking in the link at the bottom
here if you need to create a custom bug
tracking plugin I would also encourage
you to check out the forums as somebody
may have already done the hard work for
you tester has got lots of settings to
customize and fit with each environment
we'd be always happy to help you get the
most out of yours if you need help when
configuring your fine logic strategy and
things like that you absolutely have us
help along the way with that as well so
lots of things to make it work best for
the application under test also
throughout the the environment here I've
got an element repository I've got an
output you can see here I've just
committed something to source control it
gave me some feedback on that a compiler
some syntax errors down here I'm
actually gonna pin this one out of the
way and here we have the
repository with the page-level and the
element level also whatever I'm clicking
on and focusing on will see the
properties for that in the properties
panel and at the top right we've got the
step builder and of course you can
remove these panels around pin them back
pop them out we've got some custom or
some presets up here too if you'd like
to move your panels to sort of preset
area as well so next up we're gonna go
ahead and start with creating our first
test and this is going to be just a
registration test so we'll go ahead and
say create on that clips I missed one
thing there have my filter turned on and
I forgot to give it the right name so do
that again because we want to call this
the quantum reservation or registration
test now for those of you that are
familiar you've probably seen quantum
before it is a great example site of
Sitefinity and from here what I'm gonna
do is just right click on this and
choose record so this is a great way to
just jump in and get started we're gonna
hit record this is gonna give me three
different options I can create my test
in one of these three browsers or all
three of these browsers you can mix and
match you're not stuck with anything
this allows you to go get what you need
to put a test together that ultimately
will be able to execute across all three
of these as well as edge and Safari
those two we don't record in but you'll
see those in the execution list in a
moment so I'm gonna start out with I
here my apologies I'm actually gonna
start with Chrome just to switch things
up so I'll do that again
yeah so here's my recording browser this
is my starting screen this is the same
starting screen you'll see on I Chrome
or Firefox and essentially now I can
just provide the URL to wherever is I
need to go so we'll drop that in we'll
hit start recording and now we're in
that solution one the hit record do what
user does and test studio will capture
for you you can see in the background
here it's already captured my navigate
step for me and as my start wakes my
site starts to wake up here you can also
see it's attached the recorder at the
bottom the recording toolbar it shows us
we're in the live record mode while I'm
here I'm just gonna hit the highlighter
real quick and add a verification up it
went away a little bit too quickly but
we can actually add verifications and
timing adjustments so if I have that
start up screen there I could actually
have it wait for that to go away
centrally so really at this point
whatever I type in will be captured so
I'll turn off my element highlighter
we're just gonna start by registering so
we'll click register now you will see
that it's capturing keyboard presses too
so if I do jump from field to field so
for instance if I put in tester here and
I tap to the last name field it will
capture that now a lot of tools will use
tab keys and keyboard actions like this
to cheat test studio doesn't actually
need that everything is built into the
test step itself so when I have enter
guy and in the last name field we have
everything we need to go find that field
and populate the data within it so the
tab key actually becomes an extra step
and any extra step you can eliminate you
can either toggle it off if you don't
know whether you need it or not and run
the test without it or if you know you
don't need it you can just delete it and
you can edit this test while you're
recording you're encouraged to stay in a
live record mode and edit things and
modify things and even run things back
you'll see from my right-click menu I
can run to a certain spot of the test
from a certain spot or
even selected steps on the fly as well
oops again I hit my tab key but we'll
continue on here add in the email the
username and of course the password
perfect
so as I'm capturing all these test steps
I'm also capturing the corresponding
elements down here at the bottom and
just by clicking on a particular test
step you'll see a highlight the
corresponding element for you so you get
a really nice connection between element
and step we can also right click on this
go see where it's being used this is
currently just being used by this one
particular test we could even
right-click this and edit the element
against the live UI itself so this pulls
up the ver the values that are currently
being used to identify the element and
shows me other options that are here as
well so when we have something like a
dynamic ID for instance we might trim
this and basically just leave behind the
static portion that makes the most sense
in this case user name for instance now
this happens to be a static ID
it just looks dynamic but if it was a
dynamic ID this is one way to work with
it to continue to allow tests to do to
find the appropriate element in the
appropriate field but ignoring the
dynamic portion of the ID and focusing
on that static portion that's there so
we're looking for an input that contains
this particular user name ID the other
thing that you're getting is when you
click validate you'll see on the actual
live UI itself that it's highlighting
the field so you have that feedback from
the application itself in addition to
this message here that says yes we're
able to find that so it's very easy to
get in an update and troubleshoot and
fix an element and even dive deep into
an elements logic like that while we're
in the live record mode and even be able
to right click and say let's go ahead
and run this select step right back on
the application while
we're here so we don't have to start the
test all over again to see if we just
fix the thing that's broken and you can
see that it executed that and passed so
it's very easy to create these tests now
what I'm going to do is actually hop out
of Chrome
and we're going to just run this test
now to the last step I'm going to use
now run to here and the way to use this
is just to jump into the middle of the
test if you need to edit something or
troubleshoot something or update
something or add on to your test you can
use run to here to get to any point you
need to it's going to execute the test
up to that point including that point
and then attach the recorder so we
started recording this in Chrome now I'm
gonna go finish recording it in IE so as
you can see it goes pretty quickly
through the process fills out all those
fields for me attaches the recorder
we're now back in a live record mode and
we can continue building this test they
go ahead and push this around of it so
you can see what's happening in the
background here still and I'm gonna just
click register now of course every test
needs proof right especially if you want
these tests to be able to run while
you're asleep so we need something here
because right now I could actually put
any combination of data together and
click Submit it's what happens after I
do that that's what tells me if I've
succeeded or not here's a perfect
example where it says success we've got
that figured out right well this is
where I can turn on the element
highlighter and I can cruise around and
see all of the different elements and
layers that make up the page you can see
when our custom translators kick in and
recognize a specific kendo tree node for
example or kendo menu item right with
great additional bonus features here as
well as the basics right so we we will
do the same for all elements whether
their customer third-party or
what-have-you but when it comes to our
own you'll get some added advantages and
we could see how things are set without
actually having to exercise them in many
cases right so we can add verifications
in like that but for proof I want more
of a functional verification which is
perfect for this success message I'll
just float over that under quick steps
we have verified text contains success
we could go ahead and use that as the
proof if we've succeeded we should get
that message right now this orientation
one is important but it's not
necessarily mission-critical
right so I don't want it to ruin my
functional test case I do want to know
if it's broken though so what I'm going
to do is set this to a continue on
failure option what you can see just
gives me that icon there so this way
this can fail if the orientations
vertical instead of horizontal but I can
still see if my registration process
succeeds it will still go on to this
final verification for me right so
go ahead and close our browser that ends
the recording process for us whoops not
that that's the wrong X button there we
go and we'll just run this back and when
we go to execute now I've got the five
options so let's go ahead and execute
this one in Safari for example so
safaris going to come up and now I have
my full registration test you'll see it
fly through that and at the end of
everything looks like we did have a
failure at the end now I'm gonna run
that back and slow things down but
before I do let's take advantage of the
fact that we've got two failure details
here
now every time you execute a test you do
get a log file that's produced and
currently we're using we call quick
execution this is a temporary set of
data it's still valuable and I can still
act upon it but it's temporary if I
rerun the test quick execution mode
we'll get new data but this tells me
every what I need to know what browser
and version we're running against all
the steps that pass the step that failed
a little bit about the failure detail
here as well and the total time it took
to run that which was for under four and
a half seconds to execute that entire
test now let me go ahead and run that
again but this time with our annotations
turned on so you can really see what's
happening on a step by step basis
annotations are gonna highlight for us
each of the elements along the way it's
gonna basically show us sort of a split
play-by-play if you will also at the
bottom right you'll be able to see
what's happening on a step level as well
and you can pause execution if you need
to inspect something or grab screenshots
and things like that
so this is
great way to inspect in what was
happening now one thing I didn't show
you was the failure details that come
out of this as well but we'll probably
have the same failure this time I
suspect so jumping into this red X
double-clicking on this will give us the
full failure details now even though
this is a temporary result again I could
still act on it so if we find bugs
during crash test creation we could
still submit them which is a common need
of course here you can see that we're
trying to match the the texts contain
success says you can find that didn't
match we also have images that show us
the expected state on the right and the
failed state on the left so we can
capture and compare what we expected to
happen and what actually happened we
have the page Dom at the point of
failure too so we can inspect what was
going on behind the scenes when that
failure occurred and the ability to fix
the failure if it's a test issue in this
case it's going to give us the ability
to maybe switch out which type of
attribute we're looking at maybe
different operators if we need to or
even customize or update the text that
it's looking for depending on the type
of failure the resolve failure tab will
look much different if it's a failure to
find an element for example we'll be
able to fix that problem back on the
failure tab here we also have the link
to the complete test log again and that
time it took us 28 and a half seconds
with the annotations turned on well
we've got the complete test log as well
as an exception log so all this data is
captured are magically for you at the
point of failure and allows you as a QA
or as the developer to review it
determine if it's a application issue or
a test issue determine if it's something
you could fix or something you need help
with and then from here we can easily
copy paste it to clipboard or export a
to zip file or we can just hit the
submit bugs and pump this directly into
our bug tracking system it populates the
name and description for us and attaches
the failure details so I hit submit it
goes to my developers and they get all
of that particular data all of the
attachments for the images the expected
state the failed state
the steps to reproduce the issue within
the complete test log and all that and I
get a little bit of feedback here from
TFS letting me know that I just created
bug 22 right
similarly if it's a test issue if you
need help from our own staff from
support it export zip it up attach it to
a support ticket our guys love to get
failure details because they know
they're gonna get a lot of that data
that's gonna help them help you so
what's going on here and it's a little
tough to see possibly on the screenshot
there but I have already used up the
uniqueness within this test so I need to
provide something unique again my
username for example an email address
need to change for this test to operate
correctly now we certainly can data
drive things and I have some great ways
to go get random data and populate them
into these tests as well if you just
didn't those but what I'm going to do
real quick is just update these two
let's see if we can't get a pass out of
this
so I execute that back one more time
we'll stick with Safari for consistency
and hopefully we get a pass now when it
comes to verifications there are both
functional that tell us whether we're
succeeding or not like is two plus two
equal to four or you know is it does it
say success at the end when we get to
the end of this process or does it say
log out after we log in for example so
there's lots of different types of
functional verifications that provide
proof and then there's more of the
superficial verifications that provide
us the stylistics the things that tell
us whether or not it looks the way it
should the human eye kind of variable so
it looks like we got a pass on that
everything passed that time around so
let's jump into another test I'm going
to show you a quick login test now that
we have got a couple of logins have been
created here this is a basic test
navigates to site clicks the login link
provides the credentials which by the
way if you need to you can certainly
encrypt we do have the ability here to
take a password for example like my
super secret password here and actually
either encrypt which automatically
checks is password or you can separately
check is password which can be undone as
soon as I use encrypted though it's a
one-way operation you can now not
encrypt that unless you're pretty good
hacker I guess better but basically
that's all now been protected data so we
don't have to share secret information
through through our tests essentially so
basic login test we have a few different
things along the way we'll go ahead and
run this real quick just to see that
it's working correctly and I'll just use
this as a chance to try out another
browser so let's go hit edge here with
this simple login test the reason I want
to run this one real quick is so that we
can build on top of it once you create a
building block like a simple login test
like this you can certainly achieve a
lot you know you can basically start all
of your other tests with a log in B go
do something else you can just use this
login test as that a portion of it also
my day to drive this login test to show
different crit sets of credentials and
what happens when they log in as well
and I see I have the inappropriate
username there that's an easy fix here
you can see the feedback as well so
great to be able to see that what their
annotations turned on but if I didn't if
I wasn't present for this I could review
that failure details again and see what
what it was that went on this time you
can see it's a failure to locate an
element so it's looking for a div that
contains the text content of hello
tester 13 it couldn't find that keep in
mind if I hadn't had this verification
at the end that I would have gotten a
false positive so it's important to have
verifications throughout your test in
fact you may even want to have even more
verifications than this one at the very
end this time the resolve failure option
gives me the ability to fix a broken
element and guess what it's come with
suggestions this is when an application
or a sorry an element has changed and
tests studio will kick in the back
methods so it was looking for the
element based off its text content tag
name combined couldn't find that right
what it brought back in this case was
login button login text you know all
these different kind of things that
correspond to too long so we didn't
exactly get there we could see from the
images then we didn't get to that
particular spot this is where it was
looking for log out I believe instead of
log in so nice try and this wasn't the
solution but gives me lots of data that
I can share with the team and report
back if need be now I'm gonna try just
taking the tester guy or changing this
to test your guy for here and we'll
rerun that and we'll see if we've got to
pass once you've got your test working
the way you want we'll drop it in the
test list we'll set it up on a schedule
or call it from a continuous integration
process and basically have it just run
as often as you need to to make sure
everything's working for you now I'll
show you an example once we get a clean
run of this how we can data drive this
as well oops oh I got a trial message
there and I didn't
but we can data drive the text entry of
the of the test itself so the username
the password we can also data drive the
find logic or the even the verifications
that are involved so there's an easy way
to do that we can essentially take our
test so we're out here jumping back to
the project menu you can see that you
can bring in different data sources
Excel CSV XML or even a live database
and it's ask that to your project level
as you can see I've got quite a few
already in here a couple sequel
databases as well as some different
spreadsheets and you can edit those as
well as needed and of course update
those into source control too and check
those in and check them out along the
way too so now I've got a couple of
those already and I'm just going to
choose my test click bind test choose my
data source and choose my table and
we'll go ahead and take a look at what's
what's in this one here it looks like
there's a number of different options
ten different records I can filter these
down maybe I want to get this test
working for just two records to begin
with so I'll go ahead and filter these
down and see if it's working for these
two I will give it three I suppose you
also have further column level filters
as well so this is really useful
especially when you're doing a database
datasource you'll pick your table or
you'll provide a query but then even
after you do that you still have some
additional column level filters to
fine-tune your dataset once I'm ready
I'll click bind this will then change
the icon indicating that there's data on
this test and now at the test step level
I can go to the bindings properties and
in the drop down I have all of the
columns of data so for the user name and
see this is the user name one so we'll
just click user name column we'll hit
that to set and I now have a data driven
test that will iterate three times since
if I've told it to use three records so
we've got a user name password is
actually the same in this case so I'll
have to worry too much about that we're
gonna click login and then we're going
to verify that it says
hello tester right but I need to update
this as well so I'll go ahead day to
drive this verification to include the
user name and we'll just modify this to
say contains and pull off the hello part
of that it'll actually insert it doesn't
really matter what's in there because it
will actually insert the value from the
data driven portion of it so now we can
execute this test I'm going to turn off
annotation for this so it goes a little
bit quicker and I don't think we've hit
Firefox yet so we'll go execute against
that one now here in a moment we're
gonna go take a look at load testing as
well after we get if
a bit further here
now test studio uses timeouts you could
see at the very very bottom it's
counting down right now as it's waiting
for the verification to complete
timeouts are great because we can go as
quickly as the application allows but
allow for an amount of time before we
give up so you can modify this at a step
level at a test level at a test list
level as well so a lot of timing
adjustments really allow you to make
sure that the test doesn't outpace the
application now I suspect I might have
three failures after this because it
looks like they've all had the incorrect
combination being used there but that's
okay with the iterative tests you'll see
the iteration results up here and we can
drill down into each of those and see
the failure details of those this one
seems to become across all of the
different iterations which is a result
of me providing some bad data
essentially but once we have our login
tests the way we'd like once we go to
create another test for instance if I
start up another one here we'll say
product search for example sorry I have
my quantum filter on quantum processor
we'll just go ahead and jump in instead
of hitting record this time I'm gonna
hit open and I'm gonna start using my
step builder step builder allows you to
provide a lot of steps a lot of common
things that you might need things like
clearing a browser cache or maximizing
the browser or adding delays even
jumping into code things you'd like to
and by the way any of our test steps you
can simply right click on an edit in
code but the thing I'm going to use this
for first is test a step which is one of
our most popular features the ability
for me to go
grab a test that I've been working with
and and just pull it into this
particular test now before I do that I
want to just double check that I have
the right one here that's your word so
test a step and you can see there's
quite a few of my tests you also of
course have the same filtering
capabilities that you might want long
away will grab this quantum log in test
to select that and now we have a dynamic
call to go execute the login process and
then whatever else it is I want to build
on top of that so a great way to
continue from here is to simply
right-click on the tests of step use run
to here and this will execute the test
through the login process and attach the
recorder leaving me right at a spot
where I can continue building on this
and reusing that one common login
function this also achieves that cut
down of maintenance or the reduction of
maintenance where if I have a problem
with my login test I can go to one place
and fix it for everything that uses it
and please
if you do have questions that are
popping up in your mind or comments
please feel free to add those into the
questions panel along the way be happy
to to get answers for you if not during
session at least and follow up on our
blog but as you can see I am in a live
record mode now the recorder is attached
whatever I might do is going to be
simply added on to this test so let's
say we want to first of all take care of
some cosmetic verifications let's make
sure that those human eye types of
things are working correctly so to do
that again I'm going to toggle on my
element highlighter we can go up and see
that things are showing up correctly
that things are enabled if we need
timing adjustments we can add a weight
which is a 30-second timeout to say does
it you know make sure something goes
away before I try the next step or make
sure something's visible or exists also
we can use extraction which allows us to
actually grab data from the live UI
creating a runtime variable that we
might day to drive something else with
which is great for unique variables that
pop up along the way here in the quick
steps we'll just do a verify text
contains on that I'll make sure that
this pre-order button is enabled and
let's see we'll jump up to this quantum
image here and let's do an image
verification now you'll notice in the
quick steps there's lots of options
there's JavaScript events that we can
fire against it there's mouse actions
some of the more difficult ones that you
can easily set up like right click or
hover over double click scroll actions
and then you've got a couple other
options here which invoke the toolbar so
if I do the build step option it invokes
this toolbar which I'm now going to
undock and basically this is another
level this is a little bit more advanced
capability where I don't have to code
still but as you can see when I click
around through the dom here it will
highlight the corresponding element for
me and over here on the right I have
different verification options and
actions as well so if we can't simply
hit record and do what a user does
here's a nice more advanced way of
creating an action or interaction
against a specific element and making
sure that we're working on the
appropriate layer there may be sometimes
buttons on top of buttons and things so
this really helps you see those and work
with the appropriate one also this is
where you're going to find element to
element drag and drop I'm going to jump
over the verifications here you can have
some of your basic verifications but
then also some of those more cosmetic
things and let's make sure it's got the
appropriate href on the stylistic side
we can look at either computed or inline
and we can drill down and look at font
color background display options text
options you know for a responsive design
application for example we might check
to see if it's how the alignment is and
to make sure it says auto right or to
check the width or the height of things
now as I make selections here you'll see
that it automatically populates the
value for that so if I go back to width
for example this is the current width of
that particular element so you've got
that live feedback and the ability to
just say at that step without having to
know to plug in those values so lots of
different verification options that you
can create but also image verification
this is really nice this is pixel level
image verification very precise we do of
course provide a threshold because there
is some sort of a variability especially
when you go from one browser to the next
this is also case where I might make a
browser specific step like an IE image
verification versus a firefox one so I'm
basically saying I want to go find this
image and make sure that it has all of
its pixels in the right place and
they're the right color within the ten
percent threshold right so that 90
percent of these pixels have to match
what I recorded against versus what I
execute against now all right now I have
verify entire image but we can also just
go in and draw an area you'll see the XY
coordinates are being used and this is
that third level that I mentioned before
coding things XY coordinates are used
very effectively with tests to do
because this still goes to the idea of
finding the quantum image wherever it
exists and the zero zero point of it is
the top left corner not the entire page
so when you are using XY coordinates
you're typically doing that base off of
finding an element where it exists at
execution and then offsetting from you
know the top-left corner of that element
or the center of that element to
specific spot and that gives you a lot
more accuracy throughout so we'll add
this one in too so there's lots of tools
here we'll go ahead and dock the toolbar
back and we can continue recording or we
can even just go back and try running
from here for example and just see how
these test steps execute on the fly now
we also support the execution or the
automation of dialog handlers for logon
like windows authentication for example
or al the download upload even generic
ones these are automatically detected
during the recording phase but you can
always add them and drop them in if need
be as well also conditional logic
if-else while loop loop options there's
all sorts of great things that we've got
just built in out-of-the-box for you I
want to say that I've got an admin test
here that might have and it felts one of
these tests I believe has it felts in it
but really easy to drop those in and use
those as well I'm gonna pull back my
filter here real quick nope I did you
see it okay I want to show you another
test here which is the language test
because this one actually takes it a
step further it has a data-driven URL so
I can run the same test against
different sites
it has data-driven verifications and
under the hood it has data-driven find
logic so let me take you through a real
quick glance at what this shop link
element looks like from the element file
logic because what I've done is I've
taken the initial structure of how I'm
locating it which is by text content and
I've given it
value I've got a drop down here to
choose different columns to actually
feed into the element fine logic so this
is a very very dynamic thing to do very
complex thing to do but again a way in
which we're doing that code lessly to my
data in this case is attached from an
Excel spreadsheet but you also could
build a local data set within the test
itself I should also show too along the
way the storyboard is capturing images
during your recording process and that's
actually where the expected state comes
from I'm sorry I clicked on that it went
back real quick so this is all the
expected states that that were captured
during the recording originally but my
data source itself is coming from an
Excel spreadsheet and in this excel
spreadsheet I have the English and the
Spanish version of these words so this
one particular test we can execute
across different languages and in fact
really it's different site because
they're two different URLs so let's go
ahead and run this real quick and then
I'm going to take you over to load
testing to show you that as well but
while I'm working on that I want to talk
about the conditional logic a little bit
the if-else capabilities give you a lot
of ability to take your tests maybe
modularize them you might have sub tests
within your else if statements as well
but here you can also see the
verifications working out through the
layers by the way with the element
annotations turned on and this test has
a lot of verifications built-in but
again with a conditional file logic you
could really start to branch things out
you know if I log in as an admin can I
do what an admin should be able to do
maybe go down that road else go down the
road of the end user experience you can
put together the different roles and
data drive those this also has a mouse
hover over on this so it hovers over
that now switched into Spanish a couple
of these buttons like logging in search
are still in English but it's going to
go through the same test now the same
verifications now on the Spanish version
of this site and of course if I ran this
without annotation
it would have been done probably 10
seconds ago so whenever your
box change whenever you're updating new
content on your application it's easy to
get in and evolve your test so that you
don't have to start from scratch and so
that you can build reliability on that
test and get a lot of mileage out of
that test over time you'll see here both
results were 100% everything passed and
actually in my log file here we can see
the overall result here's iteration one
the steps there iteration two and the
values that were used and all the
passing steps there and of course the
total time as well as each individual
execution or iterations time - so if you
think about that I can take that test up
a lot by continuing through the
application and the Spanish versus the
English or maybe expanding across even
more languages all with this one test
just by either changing up the data
source and adding maybe another row of
language or of course extending and
adding on to this test as well alright
so we're going to jump really quickly
over to load testing I went ahead Oh
before I do that I do want to show you
the ability here to right-click a test
step if need be an editing code again
it's the last thing that we do in tests
you know but we have a great built in
IDE for you by clicking that it will
produce the code-behind file and we'll
push some panels out of the way here
real quick because you can also see just
how easy this is to work with we do
provide both the standalone environment
that you've seen today as well as a
plugin for visual studio so here you can
see on the left side my virtually
codelist tests except for there's one
step and the coded step here as well
okay so you can get in here and extend
and customize this you can type in
there's an intellisense capability
that's here so you can easily create
what it is that you need to do for your
test steps so lots of ease-of-use IDE
built in you don't even have to have a
visual studio installed to use test
Studios IDE it's already here for you
but if you do have the visual studio
you'll get the plug-in that comes with
test studio as well okay so great if you
use minimal coding but when you need it
it's easily - easy to access as well
alright so as I mentioned I'm going to
jump into a load test now load testing
is great to see how your site if your
site's ready once it's once it's going
live or on staging ultimately it's
probably best to do production load test
when possible but test geo allows you to
leverage your existing tests so I'm
going to run through real quick here and
find some tests work with just do this
simple registration test and we'll pick
a browser this will then execute the
test just as we've seen before and
return back any data that found whoops I
grabbed the wrong test
but basically captures the traffic and
it comes back with all of the first of
all dynamic targets that are detected so
any of these session IDs we need to work
with any of those that things like
authentication tokens that we need to
interact with tests to do will detect
and you can disable these and enable
these basically when you enable these
you're telling tests to do to make them
to treat them as dynamic if I uncheck
these this will actually be treated as
static which means each execution each
virtual user will reuse the same ID
that's here but by turning it on I'm
telling tests to do that each virtual
user needs to get its own here we have
all of the posts yet and we can jump
into the post operations and see that we
can des to drive these as well we could
bring in the data source if we'd want if
we'd like to and maybe provide the
username and password so we could data
drive our load scenarios once we've got
our scenarios we're going to add our
workload so we may be adding certain
amount of workload to one versus another
varying those up we'll put the majority
here on this guy and you've got
performance monitoring performance
monitoring is going to allow you to
inspect the applications server the app
server that's hosting it maybe the
database server for the application to
make sure that it's keeping up to
check on memory or physical disk
processor so we can add in some
performance counters there and maybe
even monitor multiple machines as well
on the right we've got our strategy so
I'm going to ramp this up to 50
concurrent users with a ramp at the
beginning and run for three minutes
first now we can add more profiles here
you don't just have to use our existing
web tests you can also capture from the
live browser or from an API or from
mobile devices as well don't forget test
studio mobile as well as test to do API
testing also is part of our ultimate
package - when you're ready to run your
load test you can set goals and this
allows you to basically sit back or
sleep while it's executing and not have
to worry about it and you're basically
spelling out how what what's required of
the test for it to be considered a pass
versus a fail so we'll go ahead and run
this test and we're gonna run out of
time as well
but hopefully we'll get some result data
back here in the next minute so that we
could take a look at that the maximum
users of load is it's up to you we've
got customers that are tested or load
testing at hundreds of thousands of
users and then we've got some of their
testing in the in the simple hundreds
sometimes it depends on if it's a
public-facing application or if it's a
local line of business application as
this is turning up here we'll be able to
start seeing average response time
currently use errors per second
responses received per second machine
level of variables as well you can
filter it down by machines that you're
monitoring too and we can watch as it
progresses I have a sample rate right
now of every five seconds so it's going
to return data for me every five seconds
and I could even go run or my functional
tests or performance tests while the
load test is running tubes you can
combine these together
on the overview side we're getting goals
reporting on here to get some data back
as well and it tells us that we're
currently not yet at our virtual user
goal we haven't had any errors yet our
average response time is low and our
responses received per second isn't
quite where we want it yet either we
also have page specific metrics which if
there's any errors would give us the
appropriate code and count and response
time too and if you'd like you can get
to all of your raw data at the bottom
when you're done capturing it with that
export option but it's better to just
set the goals so that you don't have to
worry about that
that's where we're gonna end up leaving
it off for today again I want to thank
every one of those a lot of content that
we cover in this session thank you for
your time and attention feel free to
re-watch to see if this see this again
in the video that we'll be providing in
the follow-up email and any other
questions that you might have feel free
to contribute it on the blogs as well
and become a part of a community if you
don't already have a trial of tests to
do you can get one do everything I've
shown you today we've also got some
great examples out there and certainly
can set up any private discussions that
you might need and get you the
appropriate pricing as well but thanks
again everybody for joining and have a
great day