okay a viral i'm gonna talk about some work often recently so for testing web
services or all the testing the client codes which we use and i still to
web services
so what is web service because of examples to go contacts people you to i
guess a web
open street maps a few also downloading map files facebook scruffy ple basically any hasty
gpus H T S service which
can be and is used by the desktop
i don't specifically limiting my work eight H T P and he cts because it
makes everything a whole is
and doesn't really
restrictive anyway because
pretty much every web service uses one of the same because you don't get problems
of a levels
in that sense
so
when you think about testing web services you go to the big question you can
ones there is what you want to test
you want to test your point code you wanna make sure isn't crash but you
want tested the point conforms to the current behaviour of the server or some cost
behaviour which is what you actually programmed against
the user would say you want current behaviour because that's what i actually want to
use but that's quite tricky to do because that means you gonna test online which
gives you of those problems with connectivity and race conditions and slowness and all sorts
so we wanna catch regressions in point carried but we also wanna catch that isn't
it changes in the save as behaviour which require changes in your point codes the
new features or
whatever the changed online
so
you can do it two ways you can do offline testing or online testing and
one talking about here is offline testing developing working on
the bait basic idea of offline testing is that you somehow record how the server
hey and then test against are offline not using that work not touching the curve
javier the server so it's reliable because younger always network problem you've eliminated the web
service and all the network infrastructure in between from your system on the test
so massively cut down a good place
but it's not what users care about because the web service changes your point code
isn't getting tested against like situation
you can also
test against the carburetor the server
which is as i said what we use just care about but it requires a
network connections the instantly eliminates any kind of continuous integration or testing on a seven
provided by know because it's just so unreliable it's hassles that so and network admins
don't like it
it's unreliable
and that's a problem that's it's and me a lot in the G data which
is more based this work on where basically every time a run make check it
failed and that would be D to
transient network problems or subtle changes in the so that it behaviour that S how
to compensated for but the point was actually point
another thing about online testing is that web services change so if you're gonna do
offline testing you need to
be able to change your tests and update them so they represent the lexus takes
of a server and that was easy because you don't wanna spend time doing that
you got much better things today
finally what services are distributed systems which is a problem it's an lots
which means that roger circuit a good wasn't example later just have one server which
has a leo contacts on they have hundreds and as it turns out if you
say for example instead to contact in your address book and it says another one
and then query your address book one you contacts
often they don't here for ten twenty seconds
which means that your test code has to get to sleep the channel twenty seconds
and that's it means your tests take ages to run which is horrible because you
can get most of the way three make check and then it'll five and you
gotta start again
so that's one of the major reasons why online testing bad
there are also other problems with
online testing do you to things like resource limits and
the fact that you have to use not a natural online account so testing against
cable services they limit you to say a thousand press today so if you wanna
run make checking continuous integration you gonna hit against that limits and also new tests
are gonna fail for us today and there's nothing you can do about that
similarly if you wanna run test from multiple locations so one developer in europe and
one america and they both whatever make czech good people is gonna say well that
test account is be involved in two places at once
i'm gonna look at out and then again make czech stops working
so it would be working on is
maybe oscillation for offline testing
where you record H T P messages between client and server you save into a
file you commit that get you then run only a test again start but periodically
you can manually check and compare the trace files against the carburetor was of the
and check other the same or if the difference then you know what's changed so
you cannot take your code in your tests to match but it means that's for
most the time this of one testing can be done in continuous integration it can
be a lot faster sleep and you don't have always problems with synchronization or resource
limits
so there are various other existing solutions i mean it's a pretty simple idea but
most of the large over only so the first three they basically exactly what describes
why more is a separate process which makes in the paying to set up
rest drive a is rest only which eliminates a huge number one services but
also means you can test if you don't know number one services and betamax children
as well
question they will smoke an e-mail of all the open source projects which is similar
sort of things with the cost and hardware but obviously not exceeding gonna translate and
fractions
so the approach of taking would look at it is to a broad enough is
a revelation
we directly apply requests to a local in precisely but other and then from that
replay the trace and compare every incoming H T you request with the trains and
then reply with respondent race well
if they don't if they you can make the press that match what's in that
race while then you get never your tests fail and everyone stopping one happy
alternatively you can not a variety S and the test will go straight to the
online server and then you can compare the responses not with the choice while and
see if the servers changed compared to last time you save the choices
so you this operates at H T P level using them so you which is
simple things a lot this transpose can be human readable and
does debug able
it means you can do
simulation of H T P errors like timeouts and for a forcible server iras every
five hundreds things like that with relative ease you not filling around of the lower
levels of T C P I P which would just be too much for fire
it gives you extra power but for the most part it's not needed for more
web services "'cause" there will be H T to you know like
unlike a one or more
the server that's your testing against the offline server is kept in process so you
don't have process synchronisation signal tests and this mock server so some for things a
lot
and because it's overriding the N S it requires minimal code changes to your point
so of the solutions that people
suggest would be to take your clients and split it out into interface isn't and
reimplement interfaces with an online petition offline version and that becomes fourty complicated cost many
changes and is
a real fast
so by writing the N S you basically have to change a client to
specimen H T people to use because it'll be a wide range one so you'd
have to run three but you can easily switch between online and offline just by
turning off the gas reservation
so diagrams everyone much diagrams
so this is have things currently all you've got your system on the test which
is your point code running on the desktop and you've got on one server which
are testing against you got always this page T V transactions going between them
so
now introduce a lots of the choice while and you locally say C T V
transactions to that race while so we still doing online stuff but once when you
first set of tests you save more to try as well
and commitment to get
you can and the your testing case that race while eliminate on one server everything
goes remote server by redirecting guess the choice well get right into that and of
course happy
you can also compare your choice well against the online so the point again switching
to doing the transactions online but also reading interest well i'm comparing now same time
so you can see what's changed you do not manually once a month or whatever
one of if you like it or something develop to take trough and you continuous
integration so
so he's a example
of how we change your unit test so the test it is here
whatever it is something sensation he request gets response pulses it doesn't ounce bottle of
and what you have is
beginning dedicated and dedicated around each unit test
which was a trace the stock price
it's cool since your code somewhat susceptible to use because you gonna use this high
range pull so you don't wanna use rate don't just use port eighty
and then you also tell the D N S resolver which your is to have
a right so if you testing able services you know growing people dot com and
some supplements that when you set up to run results you'll be back seven
this also means that any requests that you don't expect to other domains however unlikely
that they would gets a D N S translation arrow and so you can catch
buttons that way
probably not gonna happen
and you can wrap as a lot than us and macros or something so you
basically just a one line before and after each unit test
there is some code i'm not showing here which you and you log handler which
will be htk mess in haiti messages like tricks while but that's just a few
lines didn't feel
so i've implemented this village a data which is the project a hard on about
most
so make czech was over ten minutes because of all these sleeps to wait for
the synchronisation in the street a system on the server side
you can emanate willows and make czech negatives down two hundred and thirty seven seconds
done with a quick run of point make check
you can cover interesting about the same after implementing this it's likely increased because i
did a few tests for every conditions which i can nowadays so i can now
simulates and this of the giving you an access denied error or saying your accounts
been compromised
so you can add test cases for those which you wouldn't be able to do
for
because you can't manipulate so state
but generally it's been useful
i haven't
applied the store the libraries yes but there's a whole later than can be modified
this that and help also there's the facebook graphic the only as a go a
do you able a no one counts
do the network as it could possibly use it as well this
you know as a lot of stuff that could have it and it was is
become a bit more stable
but there are some problems so
in your traditional unit tests that's a you have a set of function the unit
test function attend on function and you can reuse the set of content and functions
for multiple unit tests because they do common things like you have won the ads
the contact and that at a functional remove contacts and then you have there is
unit tests do different things with a new contract
unfortunately this means that's
if you have a choice while
per unit test
and they have point these in them
not identify your contact for example
the only these are generated by the server will differ from indication of set up
to different invocations that so
and there's a bit of written in the trace files which means that your unit
tests will need
well that response for each unit test believe modifying afterwards to make a list id
is the same
which is something i haven't worked out how to fix yet
probably isn't too hard
similarly you go from the time stamps if you have H T P response which
includes time stamps well in the body or in that is a nice it up
to choice while and then you run test against it next month it's on some
so gonna get acetate and i could cause problems in some cases
again i think is work around
and then also if you may detest against cancellation the messages the current way of
will be noticed that race problems knots
particularly great
and needs a bit of work
so currently this work is being done in the G data and i haven't yet
split up into separate project i'm in the process of doing that and it'll be
committed to get doris shortly
sorry please on other people can start using it
there's a lot of potential running standard error responses way like return an error five
hundred or narrow for a for or whatever and it be good to implements
some kind of easy support for that in they
the testing code so that every point code uses it doesn't have to read that
and goals need to what of the libraries
so that was
representation of it and also construct a bit of a discussion about this because it's
an area when i'm can improve a lot basically most of a networking code websites
is good is
effectively untested that lines so
this is a like a brief implementation of my ideas and i'm sure there's a
lot of stuff can be added an improved
and i'd welcome any suggestions people have
well questions
i do you deal with the be catching a bit encrypted connections
as in H T us encryption
catching
well if i understood well the moat server cash the request
yes that catches the H T P requests so actually yes
packaging of that this is not don't within the tries postal remote server has its
own dummy certificates and K which you have to commit to get and then you
also have to model for your point code to say no don't districts of course
in checks we so there are being work that for that kind of like so
you've done some work in yes i mean would you know to use it H
T yes and also working point obviously you modifications district see a checks would be
the okay like and should never be used in production but
this is a few lines
so i wanted to ask you to will who are offering some kind of staging
services like to just it on production servers or
is there something else
so i don't know about anything else but giggle don't have testing service they have
an online api playground which is web only when you can talk in requests on
a web page and test them out but that's mostly documentation thing so what the
test cases that i have run against productions others and that's what you run into
these problems with and request limits and
accounting not so
and it would be nice if they provided
playground sins a staging service
like that
no questions okay so because will be upon victorious hopefully in the next day at
the moment it's in the G data in the tests directory and wants to take
a look and or controlled we have to is about me
thoughts you have so thank you much
thank you