Educational Institutions Rely on Test Studio (Webinar) Video Lecture | Telerik TestStudio:Automated Testing made Easy - Software Testing

again thanks for joining today's
telluric test studio webinar we're gonna
go ahead and get things kicked off
today's webinar is entitled educational
institutions rely on tests to do first I
want to thank everybody for joining in
today we're still getting a few more
folks trickling in here but we're gonna
go ahead and kick things off because
there's lots to show you in a short
amount of time
first of all just by way of introduction
my name is Andy Weiland I'm a senior
sales engineer for teller test studio
and also a certified scrum master I've
been working with this product on the
company for about four and a half years
at this point and what we're going to
start out with is just a few slides to
go through then we're gonna jump into
the application itself because I really
want to show you where test studio is
going to help you but before I get going
here I want to also of course let you
know couple housekeeping items real
quick all the colors are muted
throughout the call but definitely feel
free to use the questions or comments
panel if you're having any trouble
hearing or seeing things you should be
able to see my screen at this point and
hear my voice
we'll also be trying to provide some
follow-up or providing follow-up to any
questions that that we can't answer
along the way but definitely feel free
to jump in with any questions that you
might have and we'll try to get those
answers also if there's time we'll try
to have a few answers at the end of the
session if not follow up via email with
you to make sure everything gets
answered correctly so where I want to
start out with is where a lot of our
customers are trying to get to and being
an education it's institution you have a
lot of tools that you have to rely on
either ERP line of business type of
solutions our CRM or student enrollment
student registration content rich
websites e-learning there's just a
multitude of technologies of course
mobile apps as well so students have
every access point they they could get
to the omni-channel kind of
methodologies and they and the internet
of things that all the universities
really have to be involved in and so
what I wanted to start with was just
what we do at the end result what a lot
of people are looking to get out of test
automation and that is sort of the day
in the life of what it looks like after
you've got your test item
in place and it's generating results for
you because you can really begin by
looking at result data you can start
your day as you can see in the
background here an email notification
that's been automatically sent out
following a test list that has been run
this could this could be running over
the weekend or it could be running on a
nightly basis or daily basis and
essentially it spits out this email
notification to anybody that needs to be
notified with attachments of a result
data so this allows you to start by
looking at what's failing instead of
wasting time on trying to figure out
what's working and also provide some
very rich failure details data that
we're gonna be looking at to help the
the Q A's and the BAS and the folks that
are in charge of making sure
everything's working correctly to allow
them to evaluate if the problem is
application related if it's test problem
what it might be but then also be able
to communicate that to the appropriate
people so this really empowers the QA
process even a manual non-technical user
can really get a lot of advantages out
of test to do but then we also make it
approachable for a developer tester as
well we really want to encourage the
collaboration between the development
and QA to make sure that there's a great
feedback loop and this is where the the
failure details is is very valuable for
a tester to be able to send back to the
development team if it's an application
issue we'll look through the failure
details here more more detail but
basically a lot of rich data that's
provided automatically at the point that
the test fails that can be easily
communicated back into the team with our
bug tracking integration you can easily
submit the bug directly into the bug
tracking system and with all the
attachments and so forth that go with
that we also of course provide
integration for continuous integration
options or build server automation so
there's lots of ways in which you can
get the data in the right hands of those
folks that need it also of course if
there's a test issue involved this is a
well supported and known for its support
product it's engineer level support with
a 24-hour guaranteed response time
you can always take a failure details
zip it attach it to your support ticket
and it helps our support guys help you
get the answers you need as well so
either way you go you've got some great
data that's gonna allow you to to get
the assistance you need to make sure
your applications are all working
correctly and your websites as well so
test video has a really different
approach than a lot of tools out there
for the creation and the maintenance of
tests and we call this our four
solutions to every problem so during a
test creation process which we'll go
through here in a minute you can first
hit record do what a user does and
essentially capture all of those actions
and elements by point and click record
and you can play that back across
multiple browsers and get a lot of data
back about how well your application is
working now if that simple
point-and-click doesn't do the job we
have the ability to build a step with a
toolbar that has a very powerful set of
tools in fact here's an example up on
top we're setting up an image
verification that's looking at pixel
level verification make sure an image is
correct among other types of very
advanced functionality that's built in
that build a step 3rd if we can't do it
through those first two methods we can
utilize X Y coordinates or location
based interactions and that's where a
lot of tools live today if you're
looking at testing tools out there they
rely on X paths or X Y coordinates or
images or something like that to to
locate things whereas test EO takes a
much more sophisticated approach you're
gonna see more on that in a few moments
but basically what this boils down to is
ease of use and of low learning curve
you can basically step in and you have
three different solutions here that are
codeless before you have to utilize code
that means that we can typically
automate between 80 90 percent of web
based applications without having to
rely on on development team that allows
the developers to really focus on
creating new functionality so you can
use your best assets to make the the
best out of that application and of
course empower the QA x' to be able to
be more agile in the process as well
there's a theme of reusability
throughout test studio we'll go through
this real quickly first of all when we
capture
and we're talking about inputs buttons
anything that you come across as you're
creating a test will store that globally
across the project will reuse the
elements whenever we come across the
same one again this way when when things
change when things break we can fix it
across the entire project instead of
having to go from script to script to do
that we've got cross browser
compatibility this is due to our
sophisticated biologic which you'll see
next we've got deployment environments
so you can reuse these tests across
different deployment environments as it
moves from staging to production and so
forth we have the ability to use tests
as steps a very popular feature that
we've got where any repetitive process
or common function can be called from
another test so you can actually take
you can call tests from other tests and
reuse those tests make modular kind of
pieces that you can put together in a
lot of different combinations then of
course you could take your functional
regression test that you're creating and
run them as performance tests to see
where the bottlenecks are in your
application and run them as a load test
to see what capacities you can handle of
course when you're coming around to
registration time you want to make sure
that all of the students who want to get
on and get the classes they need can get
on and get access and and that the the
system can handle that type of look
of course copy-paste ease of use
throughout tests to do just another way
in which we help reuse things and here's
more on that sophisticated fine logic so
under the hood of tests to do this is
what we call the Corvette engine
basically test to do is looking for the
elements by the attributes of the
element by the properties of the element
and here in the background is an example
of how we might locate a login button
for example in this example it's using
the text content which is the label on
the button and that's how users work so
we can actually do that or there may be
even stronger methods like using IDs or
an href or other things and as we see
this prioritized list the weaker and
weaker identifiers are closer to the
bottom we'll see this in more details a
bit of a smaller version but you'll see
this from real-life in a few moments on
the foreground here is how we can tailor
the project settings to fit the
application under test
today you're gonna see an app an angular
application brought to us by Michigan
colleges online they've allowed us to
use their site as an example for you
guys today and an angular application
may have a very different tuning of the
find logic strategy then a typical html5
application so this can be customized
and tailored to fit the application
under test to make sure that we find the
strongest method of locating elements
and and of course create a backup
prioritization as well so when there is
a failure it can fail with suggestions
as you can see in the background where
there's suggestions listed here of how
you might repair or modify this element
we can create a very powerful change
find expressions and of course we can
fix and maintain these elements without
code and here's a bit more on codeless
test maintenance on the fly as you're
creating your test you can add
verifications from our element
highlighter lots of great powerful quick
steps that we've populated in there and
of course the ability to jump in and
build a step if you need to through our
toolbar we've got of course that editing
the elements against the live UI so you
get feedback from the application to
tell you yes this is working no it's not
of course
creating element net to element drag and
drops we're responsive design
applications these days are going to be
moving our buttons around and we want
them to we want them to be basically fit
the form factor that the user is is
bringing to the table so that means we
need to make sure we have a testability
of those of those buttons of those
elements when they move and test studio
can do that we can go find element a and
drag it to element B no matter where
they move around and of course when you
have a change that applies to many
different tests you can apply it across
the whole project instantly with a
couple clicks now on a test tool testing
tools out there struggle with more of
the complex widgets we've recently put
out a tutorial we've got a lot of great
videos that have recently been added as
well by our product management team and
and really when it comes to dynamic
grids and dynamic situations test studio
excels we'll take a look at some of this
today and
we can work with a grid can be sorted
filtered page all those different things
and how we can go find something
accurately using location or content or
a combination of both of those and then
even data drive that to make a loop out
of it this is all of course accomplished
by using test Studios ultimate license
to create and maintain the tests and
then also having our tests of your
runtime licenses to be able to execute
tests across different maybe browser
versions maybe operating systems that
you need to test against so you can
create a lab environment of virtual
machines often that you can execute
multiple tests and parallel against and
get a multitude of result data as well
so lots of information that you can get
and lots of mileage that you can get out
of every test that we create the runtime
also gives you the performance
monitoring capability that's used for
performance testing as well as load
testing and of course we can work in a
continuous and Gration environment
integrate to source control I get TFS
integrate to the bug tracking system so
we can be a part of the process the
development process that you already
have underway and of course would
provide this with our legendary support
24 hour engineer level guaranteed
response time we got webinars that we
host weekly like this one tons of
resources here's a look at the help tab
if you haven't already looked at the
help tab in test studio it's not just
what version am I using but you have
direct access to access our feedback
portal for instance you can tell us how
we should shape the product you can vote
up features to make sure we prioritize
them on the roadmap and comment and be a
part of that community or jump into the
forums and provide solutions or if you
need help from support click the submit
ticket button so this gives you a world
of access to lots of resources that are
always being updated so next we're gonna
jump over to test studio itself and here
let's see that's the other one that I
got a couple projects up and the first
thing we're going to take a look guys
just give you some familiarity with what
we're seeing here there's a lot on the
screen right
and this is something that you can
configure to liking there are lots of
tabs that we have for accessing whether
it's properties or our step builder that
lets you build steps very quickly and
easily or the project tab and the
integration point to source control I
have this project integrated to get
source control so I could commit changes
I kind of revert to server and that way
all of my team members can access the
most up-to-date versions of these tests
we can keep track of the versioning and
the chain of custody and conflict
resolution as well so lots of things to
see I'm going to pin back for now my
project tab because you can move around
and modify these tabs to wherever you'd
like and we'll just give ourselves a
little bit more room on the test tab
here to see what we've got
so the first thing we're going to take a
look at is how we can create a test so
we're gonna be utilizing again the
michigan colleges online org site and
the first thing i'm going to do is just
hit record and do what a user does
basically so we'll start recording here
in firefox to begin with you can record
from IE firefox or chrome and now my
recorder is attached so we'll just
provide that URL here hit start
recording and i think now we're where we
should be so we have the recorder
attached and now we can basically go
through the site as a typical user we're
going to click on this little burger
button you can see it captures that
click for me and we're going to ask the
question are we ready for online courses
so back here you can see as it's
capturing my actions we're just going to
go through a quick self-assessment this
opens up a pop-up window you'll see
tests to do will connect and continue
recording in the pop-up window as well
and we have a lot of radial buttons so
i've read through these questions
previously but don't take my answers as
literal or just going to go through real
quick and just sort of make up a little
bit of time by clicking through some of
these questions for this particular
self-assessment here
there we go and at the end of that whole
process I could see there's a total up
here lets me know distance earning is a
good possibility for me and of course
all those questions along the way were
pertinent to can you work on your own
and things like that but the point of
this is that we've captured all those
steps we'll go take a look at that in a
minute but before we go on to the test I
want to show you that test studio gives
us the ability here to also interpret
layering on the page with the element
highlighter I can toggle this on and
quickly add in verifications and modify
or sorry a JavaScript events as well
mouse actions scroll actions there's
lots of things just packed into the
quick steps here also just within the
quick steps we can verify something
shows up or is enabled we can wait for
something to show up if we need the
appropriate timing adjustment we can
also extract values from the actual live
UI itself and then of course if we need
additional more advanced kind of
capabilities we can jump down to build
step which will take us into our tool
bar so we can do a little bit more
advanced stuff as well so I'm going to
just utilize this as a verification at
the end of my tests to make sure that
that actually shows up and let's see if
we can actually verify the content
itself real
there's my score
so we'll just do a quick step
verification to verify that that says 45
and that should be all we need
so we'll go ahead and close the browser
this ends the recording back here on my
test you can see all of the steps that
we've captured and of course all of the
elements that are captured along the way
too we can filter this down to just this
test a lot of radio buttons that we
captured there for us but we're going to
just quickly run this back and I
recorded that I think I ended up
recording that on ie so we'll go ahead
and execute on Firefox as you can see
here now
we have the five options for execution
and we'll hit run and we'll watch that
run right back for us so this is that
first solution hit record do what a user
does as you can see it's going to jump
right back in to that self-assessment
toggle all those checkboxes and then
return our result for us and we have 34
out of 34 very quickly we'll look at the
view log here this shows us that it was
run against Firefox 40 802
all the test steps that passed here of
course that took about ten and a half
seconds to complete and we can move on
to the next test but before we do that I
must slow this down using annotations
I'm going to rerun this test for you
we'll also hit another browser we'll run
this one in Safari for a good example
and the annotations are going to slow
down the test execution adding 400
milliseconds between each test step and
also highlighting all the steps along
the way so we'll get more of a
play-by-play here
so now you
as we go now it's connecting to the pop
up and it should complete the actual
assessment and highlight each of those
elements as well so that it can be that
easy it's very quick to get in and
create something that's repetitive that
can be used across multiple browsers
it's consistent and reliable like this
and ultimately at that at the end of
this we should have the verification
verify that it's the correct amount that
we expect it to be so as this wraps up
we're going to jump into another test
where we've got a failure to
troubleshoot so I'll show you what that
process looks like and what that failure
details looks like as well believe we've
got one more page of this assessment
this is a great way to
debug and to really just take a closer
look at what the test is doing when
you've got problems or when you're
trying to fix something that's broken I
always love to turn on annotations and
run through the test to see what it can
show me so here's our final verification
and it's closing out the window closing
out the browser returning our results
this by the way is what we call quick
execution so again we had a 34 out of 34
past but this quick execution just
provides a temporary result it is
actionable data so if we have a failure
we can act on it but if we clear the
results or re-execute this result is
gone this is done on purpose
so that you can separate the tests that
are under construction for the test from
the tests that are ready for primetime
so this way I can run this it can fail
multiple times but it's not going to
impact my reports or my results because
it's considered under construction once
I put it in a test list we'll run it and
we'll get real results real report data
from that point
all right so jumping over to my next one
here
we're going to go through the MCO
registration test and there's a couple
things I'm going to point out in this
test but for now what I'll do is just
run it without annotations so we can get
it click a quick result that we can view
and start to work with so we'll hit edge
this time for an execution
so this test is going through some
dropdowns and doing a course search and
should be selecting a course for us for
registration purposes as well I believe
it's gonna be an accounting course now
tester view it utilizes time outs and
this is a test that we had set up to
fail
it passed 14 out of 15 here but I want
to talk more about the time outs because
it took a little bit longer to run that
test since at the end it was getting to
a point where it actually failed that's
because it utilized more time I was able
to use up to 15 seconds to make sure
confine the appropriate element or up to
in my case 70 seconds to make sure the
webpage is ready at the beginning of
that test so we've built in these
timeouts throughout at a step level at a
test level and a test list level so that
you can get your tests running as
quickly as possible but allow for ample
amount of time when needed when things
are loading more slowly or grid has to
populate we can make sure that the
timing is adjusted accordingly so we had
a failure here I want to double click on
this red X this will give me the full
failure details which includes the
complete test log just like what we saw
before from the log link this has the
complete test log contained within the
failure details as well and it shows me
what's passed and actually ultimately
what failed down here it shows me a
little bit about that failure as well
you can see there was a ng-app
identifier that was being used we have
an angular application under test and of
course it took 21 seconds that time to
run this particular test we've also
captured the exception log and of course
images and the images show us a very
different state between the expected
state and the failed state so it looks
like this has the expectation that it's
expanded it's open during the failed
state looks like it was not at that
particular point so we have a little bit
of a mismatch but we've got great way a
great way to see what's expected what
happened at the point of failure also
we've captured the page Dom at the point
of failure too so you can see what was
going on behind the scenes when that
failure occurred as well and on the
resolve failure tab we'll see that there
was some suggestions provided so
back to the failure type really quickly
I just want to show you in the in the
center section here it was basically
saying it couldn't find the element it
was looking for this expander on the
grid
it was utilizing this logic this
combination of things to find that
particular element and it failed but a
backup search method kicked in and was
able to locate something in this case it
was XPath now I'm not gonna use XPath as
the solution but often times XPath is a
great way to find additional suggestions
so we'll look at what XPath brought us
back and that is where you'll find on
the resolve failure tab the list of
suggestions so this is what XPath found
at the point of failure this is the
logic that was trying to be used to
locate the expander in the first place
so looking at an ID to a row this is a
sort of a body level or grid level to a
row level within the grid and then
ultimately a class which in this case
was looking at a down arrow and at the
point of failure it captured the class
which was actually an angle up arrow now
I'm recalling that during the creation
of this test I actually copied and
pasted a test step I get to the right
screen there I copy and pasted a test
step for that particular step and
instead of it being up and down or
instead of it being down at this point
is actually up so I should be able to
just modify my class switching out the
correct one for the wrong one here and
hit validate and I'm getting a great
we're able to find the element using
those settings and I essentially can fix
that for the entire project also I can
connect up to the live environment to
make sure that I've got the latest data
and information from the suggestions
here and we'll hit OK you can see this
is being used by more than one step in
this test and we're just gonna modify it
for step 15 but again if it's in more
than one test multiple steps you'll be
able to update that element if it's used
in other places all across the project
in one file suit
so now we'll re execute this test in
fact this time I'm just going to use an
option we call run two here this is a
great way to jump back into troubleshoot
a test if I need to rerecord something
and of course when you're no
recorder you can also run from here a
run selected steps as well so that way
you don't have to jump in and out of the
recording process you can stay there and
continue to work on it so this time
we'll do run to here with Internet
Explorer as the recording tool browser
and we'll see if we solved our problem
now it looks like I may not have had the
right timing in there because it jumped
right back out and said that it failed
to find that view link so I'm actually
just gonna try to rerun from here now
that I'm in a live recorder the guy had
just a bit of a too much of a timing
delay that can also be adjusted by
adding more time at that test level I'm
sorry at the test step level
great and now you can see back here on
the test itself it went through and
verified first of all that the view link
was there clicked on the view link it
did a search for classes it verified
that it could find the Astronomy 104
class that it contains the this text
within that that is three hundred and
eighty dollars and of course if we slow
that down with the annotations you see a
little bit more of that detail there but
basically we can see also that it's
selected it and that time it actually
closed it so this is where it did all of
the verifications but $380 the credit
hours even the course equivalency search
as well and toggled it to be selected so
this leaves us now at the end of that
particular test step still in a live
record mode and that time it did pass
with the correct expander that was
clicked there so I want to show you a
few more things before we leave this
testament to turn on my end of my
element highlighter again you can see
that it really helps us find all the
elements that make up the page this time
I'm going to use this image and we'll
just do a quick build step so sometimes
if it's not a basic is it visible or not
visible type of verification if you want
a little bit more cosmetic types of
verifications we'll use the toolbar and
that basically brings me to this point
it's highlighting for me the element on
the screen that's been selected from me
just hover over that and go to build a
step so it takes you to the right spot
in the document object model here on the
toolbar and over here on the right we
have the ability to add some actions or
verifications against that particular
element as you can see a lot of these
mouse actions here we naturally record
but there's some more advanced ones in
here too like right-click or double
click or hover over or drag and drop
lots of advanced capabilities that you
can build directly from here also on the
verification side we have the basics
again just like we saw in the quick
quick tasks but you also have more of
the cosmetic ones things like looking at
style either computed or inline we could
check the font color and background text
display we can make sure that things are
showing up in the right spot on the
screen they're the right color the right
font all of those things that really the
human
verifications that you think of and also
we can make sure as you're creating
these it will automatically populate the
current value that's present on the page
it makes it very easy for you to just
select what you want click Add step it
adds that step to the test on the stone
the attribute side here we can also see
there may be a class there's an ng hide
there's a source as well so we can even
verify that the right image file is
being used or for an image we can also
jump down to image verification which is
the pixel level comparison now this is
very specific it is looking to make sure
that all the pixels of this image match
from execution from recording to
execution so it's going to take what
we've captured here during the recording
process and during execution make sure
that it's within the same threshold so
at this point I have to have 99% of
these pixels match for this test step to
pass now we want to maybe make that a
little bit more generous there so we'll
make it a 10% threshold that way it'll
allow that image to be slightly
different
but the nice thing about this is it will
go find this image wherever it exists we
still have that sophisticated by logic
to go locate the image and then just
focus on this instead of the entire
screen even when we want to verify just
a part of the image let's say maybe this
logo portion of it on the right you'll
see it start to use XY coordinates but
these are not based off of the entire
page they're based off first finding
that image on the page and then taking a
portion of that into consideration for
this test so even when we use XY
coordinates it's very sophisticated
we'll add that step as well and we'll go
ahead and close our browser down take a
look back at the test sorry there's a
mobile project we're going to be looking
at in a few moments we'll take a look
back at the test and you can see where
our verifications can verify the color
we've got the source and also our
verification for the percent tolerance
here now since these are more of our
cosmetic types of verifications I like
to utilize our continued on failure
option to make sure that our tests can
continue it will capture our data it may
not be a functional verification these
appear maybe
more important to making sure things are
functioning correctly so I do want it to
fail if this fail to test out if this
particular step fails these maybe a
little bit more cosmetic I still want to
capture information if there's a failure
but I don't necessarily want my test to
end so that way you can really make sure
to get all of the data that you need
whether it's a cosmetic issue or whether
it's a true functional issue within the
application also on our step builder we
can easily jump in and create test two
steps or incorporate tested steps
oftentimes we might build a test that
has a login process and then a couple
other processes that we combine together
so you can just grab an existing test
drop it in here it looks like I just
dropped that at the bottom but you can
always edit your test very easily -
maybe I need to log in before I go to go
through that process so I can put that
up at the top of the test super easy to
work with lots of things that are packed
in here and of course at any point you
can always right click and edit
something in code this will open up the
IDE within test to do and generate the
code behind file and in fact we can make
it a little bit more roomy for us to be
able to work and we can drop this down
side by side to see art mostly codeless
tests on the left and then that coded
step converted the code here on the
right that we can continue to grow and
expand if we need to you can see there's
a great intellisense built in - there's
a lot of great tooling built into the
actual IDE here and of course we also
provide this through Visual Studio
directly so this so far has been in our
standalone environment but we also
provide a visual studio plug-in when
necessary but keep in mind this
conversion to code is typically not
necessary for the general test and the
general application under test so we can
easily convert that back if we'd like to
and get back to sort of the codeless
test now we're going to transition very
quickly back to the project Lowell so I
want to show you a few more things
before we jump over to some mobile
testing as well
studio provides a lot of different
project types and just to show you what
you see at the start you'll have a
choice between our web and desktop
project type that has our coverage for
WPF silverlight and web applications our
mobile type of project that covers your
iOS and Android mobile web applications
and also our new API this is actually
our beta version of API projects that
currently cover rests and Web API
testing as well so today you're seeing
so far web-based testing with the web
and desktop inside of web and desktop
you also get the ability to utilize
performance and load testing a
performance is basically just a matter
of selecting a functional test and then
clicking on the performance tab load
testing is a different type of test but
still utilizes the same tests that
you've created for your functional tests
so I'm going to just pin some tabs back
here to give us a little bit more room
to see because when we design our load
tests we can capture our traffic from
our existing web tests so all of those
tests that we just made related to MCO
for instance a lot of great search and
filter capabilities as you can see here
- now I can go grab one of those capture
the traffic from that test and that
results in us getting the actual traffic
from that that looks something like this
it captures all the test steps are all
the get some posts also some think time
this is time spent on the page and this
can be customized in duration and
deviation so you can add more
randomization to your load test so once
you've gone through and created your
functional tests that prove whether or
not your application is functioning
correctly and that students can register
for courses and so forth then you can
take that capture the traffic from that
functional test customize it even day to
drive it as you can see on the post
operation this has been linked up to a
database that has a number of different
user names and passwords so that each
virtual user that the load test
generates will be logging in with
different credentials so we've got a
test capture
will oftentimes capture from multiple
scenarios because we're usually going to
see a combination of things happening on
our application at the same time people
registering some people logging in some
people browsing so there's a combination
of things there and you can modify that
workload as well to fit the according
according to what the real world is as
well we also support the use of
identities for windows authentication
applications also of course you can
capture not just from your existing web
tests but from a live session from any
of the browsers here also from API is
traffic and also from mobile device
traffic so that way you can bring
together all of the endpoints into that
test and be able to accommodate a
combination of not just scenarios but
simulating what users accessing but how
the users are accessing it as well so
you can capture that also one other
capture process you can utilize is
importing your your user scenarios from
fiddler files those sa Z files so if
you're familiar with fiddler another
product within the progress stack you
can use fiddler it's a free product go
capture traffic just like we do capture
traffic here it's a little bit more
sophisticated it's a very popular
developer tool and you can import those
as another way to bring your scenarios
into your load test in addition to the
scenarios and the workload we have our
performance monitoring this taps into
test Studios profiler that allows us
access into the windows performance
counters so that we can see how well
things are working not just if it's
working or not but how close to the edge
of the cliff are we when something's
beginning to fail and what leads up to
that so we can really inspect all the
available performance counters that are
available on that machine these are
going to be different depending on what
type of machine you're looking at mine's
got hyper-v sequel Express and all sorts
of things in here but generally speaking
we might be looking at processor
physical disk things like that memory
consumption and basically you're
building out a template of things that
can be analyzed
during that load test this
by the way is the same thing that you do
during a performance test and it may be
coming from multiple machines because
you want to see what the student or user
perspective looks like and then also
what's happening on the actual
application server when this is
happening or on the the database server
so you may be monitoring multiple
machines you can adjust your strategy in
your scenario here to basically sit fit
whatever your scenario might be you can
also of course ramp or or not ramp if
you're doing a multiple hour soak test
to see how performance may degrade over
time you can accomplish that or if you
wanted to do the typical stress test and
just go from the 1 to 10,000 you can do
that as well and then of course when you
go to run one of the most compelling
things about test Studios load testing
is it's again very easy to use you
capture traffic from many different
methods one of those being your existing
functional test that you've already got
and then you can set goals and the goals
let you sit back and relax and have test
studio do the hard work for you there's
a lot of load testing tools there load
tests produce a ton of data and a ton of
results and typically you'll have to do
a lot of number crunching to see whether
you know whether or not you've succeeded
well test studio does that for you you
can use our built-in goals here to see
if your average response time across all
machines involved is staying sub 3
seconds if your errors are within a
threshold or if you're getting up to the
appropriate responses received per
second of course if you're getting to
your concurrent usage that you want to
this is actually how many people can we
handle simultaneously and then you can
also customize these goals and choose
any of the performance counters from
those machines that you've got turned on
so if I wanted to look at pooled
connections in sequel or transactions
per second or something like that I can
actually look at very very detailed data
from any of the machines and build my
goals around that to make sure that
we're getting what we need so in order
to pass a load test and test studio I
have to pass all my goals jumping over
the analyze screen here we'll show you
some result data here's
four minute test with a hundred users
and it looks like it only passed two out
of six goals if we look at the goals
here we had a non-response actually that
looks like a bad go let me get back to
one that's actually got some data in it
so here's one with average response time
we've got created virtual users or
current virtual users errors per second
we can also scroll down and see all of
those performance counters are here as
well so we can look at processor
interrupt time or privilege time or
something like that and overlay those on
the graphing here you can even combine
result data from multiple tests to
compare those to each other see if you
get big spikes occurring here you can
see the current virtual user level got
up to a hundred at the max the median
was a hundred as well average was 89
minimum was eight and that was at a one
scale so it'll automatically scale this
for you as well so lots of great data
that you can get out of your failed test
of course you also get page specific
metrics if there are errors that are
occurring you'll get the error codes and
the count and the timing related to that
as well so lots of great data that you
can gather from your existing functional
tests now running as a load test and
then last but not least we're going to
jump over to my my other project here
I'll leave this one up but we'll get out
of the way because here we have a test
to do mobile project now you have the
ability to create your iOS and your
Android tests your mobile web test
through record and playback and still
utilizing much of the same approach
where we identify we automatically
capture elements during recording you
hit record and do what a user does and
we'll capture those elements as you can
see here and if we jump in and edit one
of these you can see that we're using
the same type of process to locate
elements we don't go to where they are
on the screen we go to where the element
is it's not a location-based logic like
a lot of tools used it's more
sophisticated than that so I can go find
that by ID or even customize the
properties for it to look for that
element and of course use some of the
other properties like index or class if
need be too so very much the same
approach to element identification this
is very very powerful because you can
record a test in portrait run it in
landscape the same test can work even
across multiple devices so if I record
it on iPod I can run it on iPad it
doesn't matter because we're looking for
the element where the element exists the
form factor can be different now when we
need to we can utilize gestures and
gestures are even with with test vo you
don't have to you could do a lot with
gestures it's our second choice you know
it's not our first choice to how we
would automate because we want to go
find the element where the element
exists but there are times where
gestures are the only method and we have
very sophisticated ones at that let me
go ahead and bring up for you a actual
device itself and we'll run a couple
tests so test video mobile is a little
bit of a younger part of our application
but it's got a great future in front of
it there's a giant roadmap to really
provide the same ease of use that you've
already come to know from our web and
desktop solution so follows a lot of the
same principles now as I'm bringing up
my device here should be seeing that on
screen here in a moment we'll also jump
over and see there's also of course lots
of tabs properties step builder things
to work with you can always push these
out of the way and dock them and undock
them and also turn turn off ones that
you don't necessarily want or need
I should also show you the help tab here
gives you access directly to submit
tickets or forms and videos lots of
things at your fingertips and as we're
jumping into this I also want to point
out something that's new and and really
cool the ability to actually go get an
application from your device itself
instrument it so it becomes testable and
then deploy it to either the same device
or even another device so test to do is
created now a wizard to where you don't
have to go to the development team at
all you can just
caPSURE really you set your application
up for test with a couple button clicks
and our screen sharing app is having a
rough time here so let me see if I can
bring that over this is going to be a
mirror of a real device once it comes
awake we go now while we're waiting for
that to connect I'm going to jump back
over here because we do see that it's a
connected device it's listed as a
connected device here so we could take
our tests and go execute it across that
particular device and it will run
actually directly on the device itself
now we use emulators or simulators as
well so you're not just stuck with real
devices and of course we we have a
partnership with a device cloud host
called tests droid that you can utilize
your tests to do tests across hundreds
of different devices simultaneously as
well and now I apologize looks like my
screen mirroring app is have a little
bit of a tough time being shared out I
try to connect to that one more time
because it's always a lot better when
you can actually see the device itself
instead of a black screen now this
sharing app it is mirroring app I'm
using has nothing to do with test to do
it's just for my ability to be able to
share it with you through the power of
the webinar so wait to see if it's going
to cooperate this time
okay kick the tires a little bit now
while that's still getting tuned up
there you'll see that it executed all of
my actions here there's some taps
there's some verifications as well we
can verify property values along the way
so it's very easy you still have the
element highlighter too that happens for
you and can you can basically highlight
while you're looking at the real device
which I'm hoping to get a chance to show
you this particular test actually failed
there's a hardware key press down here
it was the back button it looks like and
we can look at the error details here
too it's more than likely I had a
actually a couple duplicate test steps
there so that's probably what caused
that okay we're gonna give our straight
screen stream one more chance here to
mirror for us and I think it might have
just gotten itself together here so
there we go perfect awesome so this is a
nexus 9 and it's a little tablet
basically I'm just gonna hit execute
here flip back over so you can see it
will launch the actual application on
the device itself fill out some data for
us go through a sign-in process looks
like it's making a shopping list here
and then ultimately return the results
for us by the way that sample app I'm
working with is also a native script app
native script is a framework that we
provide to allow you to build native
applications for both iOS and Android
and that's available through our teller
platform but again we have the same
method to the madness here the ability
to click record have an instrumented
application and a device connected to
test studio and basically from there be
able to create a test so here's our
actual mobile testing agent that's got a
dart our current connection if we want
to we can jump in tell it what
application we want to test against
we'll hit
start that will kick off the recorder on
the actual device its launched the
application for us see I've got the
wrong one up here let me go ahead and
pull up the right test for that
application and we can either run this
test now that we're in a live record
mode we also have run to here run
selected steps run from here so those
have carried over from what we already
do well in tests to do web and I want to
show you two we're gonna do just a quick
run from here at the end of this test to
show you what it looks like when you do
have gestures as you can see the blue
dots show the actual gestures whether
it's five fingers zooming in and out or
whether it's a swipe or a pinch or
what-have-you
so when we do use gestures it's still
very sophisticated type of gesture
versus when we can go find a particular
element locate that specific element
like you can see here and utilizing the
define logic that's that's strongest
essentially so you've got a lot of
options at your fingertips
no pun intended but again I'll show you
how we can just run those gestures out
and of course take screenshots along the
way oops there it goes and of course the
record
capability if I wanted to continue to
add on to this test I'm still in a live
record mode so I can just go back to my
device here and just start clicking away
and even jump into the Dom Explorer top
so you can see it added some new steps
down here and I can even jump into the
Dom Explorer tab and we'll try to share
the screen here for you and on the Dom
Explorer tab I can go drill down into
the layer of the application that I
might want to add a verification on
you'll see the red highlighter as it
closes in for us on the area that I'm
looking at so here's for example our
button here's our checkbox and if we
combine this with our properties tab we
can see the actual live properties of
this checkbox and even add verifications
against any of these properties as well
so here's our M transform that shows
checkbox our positioning our true or
false whether it's actually checked or
not so we can actually do verifications
on all of those things just by looking
at what it is that we might want to
verify finding the appropriate property
here's mtech for example I'll flip over
to the step builder from here I can add
common things like a screenshot so let's
make sure that from a screenshot
standpoint it is actually showing is
checked and from a verification
standpoint we'll verify this property
value we'll drill down into the M
checked
option it shows true click OK and that's
how we can easily quickly add a
verification against a mobile
application so that's going to be about
it for today I know we've covered a lot
of ground here sorry for a little bit of
the process of trying to get devices
showing but there's lots of to talk
about I know we've got a couple
questions in the chat log and the
questions panel already so we'll take a
quick look at those
here for another moment but I want to
thank everybody for attending and have a
great day if we don't get to your
questions we'll definitely follow up via
email and feel free if there's any
additional questions to pop those in at
this point if you've seen enough thanks
a lot for your time and attention and
have a great day