one as they've it's adam again it's mine's and might all not report your you expert and there are a couple of things about the title of the stock and the first one is that it based on my master's thesis so everything i'm gonna talk about here on is based on the research i do it it'd during my master's thesis and the actual title of my thesis this a bit longer than you know report your you expired it's something like support supporting and user reporting of human computer interaction issues than open sort supper and then read that this is not a what if i told you need to do better than this then and we ended up with how to not report on your you expired even though the i'm gonna talk and not only about how not to do it but also about how to do it based on what i've been looking into and what i've been talking with people in the community so let me introduce me so i'm gonna small i got involved you know community last year this is my second product i was and now for each in turn working the island and big you works project a so i did a redesign for to use application right now i'm contributing to the web seem and also to the marketing team and i do a lot of work and that's and as i said i did my master's thesis on how to support on users in reporting on you what's buds on user interface bugs user experience bugs that are actually helpful for designers actually visible for designers to work so what i did something up and not gonna go into a lot of the details of my research because they think that's gonna be boring it's a lot of mythological stuff so what i did they did for interviews with us designers working in and three an open source projects and found that the door indignant project and i also then an analysis of i hundred and forty seven bucks in big numbers a lot i will only but i don't are on usability and you why we the so basically what we could roughly filter as our usability and you what buttons based on that i came up with a couple findings about how are designers work with on reports and how they tackle feedback how they used it's true to work and an iterative designs and everything and by analysing a couple but they could understand a bit how we doing you experts apparently and and by comparing this to what designers told me i can understand better on what is the gaps that we're dealing with between what we were doing in what we should do so i'm gonna start with telling you guys why should you care why should you care about reporting your you what's bugs in well i mean why should you actually more about doing this and there are two reasons that i would say it's very important for you to report that's in a way that people can actually act upon this i'm right i'm i'd like to remind that i'm talking about you what's bug so i'm certainly arguments that are specific to this kind about and also i'm calling this how to not reporting you want but i think this is a top motes what's more not only able bugs and but still itself about feedback on your legs and then how you feel about the interfaces and the experience we have with interfaces so the first reason is that a lot of what we report "'em" but still and the feedback we both and google pause or two meter or you what those we use this for design evaluation for it are aiding and about winning hard designs are doing and how they are attempting the users needs and how exactly how our computer are we doing well with this actually attending the users needs is it not so we can iterate in the design and fix it so we can actually better serve our users the next you should be comic that i think it's a good example one why you should evaluate and be attentive and concentrate about our users needs and people about lighting in being cautious about your design pages things on average think everybody that has been university can and relate to this that you university website they're pretty much useless you can never find what you need or if you can find you have to make it small quest to find it so i guess that's what happens and that happens because that there is a considerable gap between what the person that is actually conceiving a design and the person that is actually using the design this order person can see there is a gap year and even not worth designs they study a lot and they work i want to make this got them in as minimal as possible this it there is there is going to be this gap and that's why it's really important first to evaluate so we can understand the ways and people they are using i mean you never know when you do some usability testing or when you know whatever kind of evaluation you never know you you're always surprised about how people appropriate themselves of technology and the way is the you soft are always like i'd never talked about using this a preferred to this thing and it's always amazing how unexpected things from to come up come to surface when you when you without which are designs with users so about evaluation method itself i think that most people want to talk about waiting on what they think about usability testing straightaway it's the first thing that pops into amount i think people are very passionate about usability testing they really enjoyed and the first thing that comes to my and it turns out that use with lasting is not the only way to really and speak about free an open source operand how we work and how with about some great how we created and how we how hard really cycle is actually is ability this is not quite suitable for how we work it takes a lot of time onto plan to execute to analyse the data collected and it turns out that see as you wanna are really cycle is very short so by the time we plan the usability testing in the previous versions they have an owl triplicate we already released people and perhaps even people twelve and so it's actually not very suitable for our names it's all those about a lot about how the problems people face when you get to observe people and everything five i think in terms of three n-tuples of our software on usability testing is much more useful when you together a sense of how people wired to better know your users because that's a problem we will you have an open source to know who we're dealing it with who are reproducing so for two another one that's quite popular is with the voice and i think some people might have heard about it but it's a an evaluation technique that do not and will user participation on it's only the designer that on and what or lessons back the whole interface having in mind some guy right lines and then try to identify possible meatballs for users based on this and this and they're popular one but there is much more to evaluation methods on our user interface design they're dreaming categories of evolution methods the first one it's investigation methods which are and i was that do involve or dispassionate users but they're more okay and finding out to the user is what do they do one a mindset wanted to ask they needed to accomplish and this kind of thing so it's more like a preliminary thing that usually do with both like thirties questionnaires some kind of you work so you go to where people work where they do their task that you just observe how they used up there's also inspection methods that they said they were think evolution is an inspection method they usually do not in both or the speech from the user they are basically or one but by the designer himself and he just put himself in the place of the user and trying to go to think about what are the flaws in the design and what are the problems and all kind of stuff is the thing we do a lot in our community in them and i guess innovative other projects with it's especially because we have a really difficult time to find users it talk to them and figure out what what's happening and there's also observation methods which are methods that are also as a investigation methods they also involve participation from users but if you those people observing as the name says help be more useful very healthy are interacting of it how a they use it how the appropriate themselves of the technology we got hurt and what i want to propose here is that we think about but reports as a do for observation still one when i would work a problem that a experience that i had this is actually a way to design a can observe is a large expand the problems that i'm facing and i think that's one of the main uses of but reports from a design perspective i don't think you think about bug reports of the contribution and would you would you can to be told that for example doesn't work at all would you can recall that does not solve the problem that this does not fix an issue or even more is would you can to be go that only works for you but i mean only you can see that problem was fixed or something like that would anyone do this or at your expense a honorary the for product with you right implementation that only you can understand or i don't know would you design the software that only you can use just think the contribution would you would work about that no one can understand what you're talking about would you report about that only you know what you're talking about so it's really important that we are able to communicate well that we are able to establish use but reports as a as a breakfast at between are committed in different parts of maybe as we do with any other kind of contributions we got in our community so even having is that why you guys should care about reporting your you expect well i'm gonna start try telling you guys for to not do this and i think to begin with the first one is that a guy that a lot of you guys know and that is a quote that of all the designers are interview have mentioned in the exact same way do i think that should you want and they all raised exactly like this there's a very different a very it's very different when you say i think something should be like something else and when you say well i try to do one thing and then i was not able to do this thing and this make me frustrated or a lot a lot of time and i expected to do this there's other way and perhaps you can do this so it's a it's really a matter of difference rating problem and solution but report sort of new use if you only offer a solution so i think about when you go to adapt or do you go to the doctor and you say i have the flu that's it i need this and this medication know you go to the doctor and you say well you know maybe not really very well i have fever i have this in this pain and i've been not feeling very well you know what's happening to me and what kind of medication should you so that use you go to the doctor and you say this is the prescription just try to tear right so this is the same thing with what it's i mean you it's much helpful when you state the problem that you're actually facing and not just observe the solution and there's a lot of things that are also related it is i mean there is a not a minimum a number of reasons why it's really hard for designers work on but reports and feedback in general that only state the solution because i mean when you're bored i mean actually really nice that everyone wants to be a the kinder and everybody but sometimes people just not understand this the underlying technology what the problem we deal with and there's a lot of restrictions sometimes in terms of P how the cold is itself or actually about their design vision that's it's being taken by D by in general i mean overall for the project so there's a lot of information that sometimes people were laughing and that it actually makes a solution so it's okay to propose solution please don't think that i'm not saying you never you never services that i mean this is how we work we work together and by an extent i want to by giving the guidelines on exclude anyone from the discussions that take place i think this is just sell way so we can get a quick stored on the report and like this a couple of course because if you do not observer problem what will happen is that the design it will come to you and say okay what is exactly the problem that you're having because i mean i can just add this button perhaps it's not the best solution you know i mean perhaps something else able soldier problem better and you really don't want to solve the problem that is not a problem but you have so it's a proposal isn't as it's okay to submit a patch but i mean you can you one here just submit a bug with just about your slides apply it i mean that's not something we do and it's not something we should do with design bugs it as well so it's a possible solution but it should not be your main point it should not only be able to solution it you think it's gonna be appropriate for your for so i think this is the most important part of it let others brainstorm with you letter hours thing together with you use data probably and you that everyone brainstorm with you designers developers and everybody out that that's all still experiencing this problem they can see about reporting can leave a note of they i arrived in also having this problem with that in this order a quality or characteristic or anything and this is the solution and everybody can work in this together you know much more clear way because you you're not actually solving up you're not giving users what they ask for but actually what we need no one is it sucks which comes are several different forms it's comes it's broken it's an hour problem i don't like it for whatever reason i just don't like it it doesn't work L it's just doesn't i don't i don't really know what's broken but it's difficult and then it then the next you want they are kind of like this they are likely that they pretend to be nice feedback for the and actually not help what L it's not intuitive what is it which i mean it's just not helpful and it's not user friendly i think that one i mean want that to be evenly maybe it is just this doesn't say anything this kind of feedback it is not going to the pretend to be helpful but there because they don't answer as i said before the problem they did not say why in the why is it that why is it broken why in the way it doesn't work for you why what exactly about this whole interface you don't like so i mean it's very it's very easy for us when we were like we really upset with operates really easy first you just like the anger go with socks you're such an incompetent are stored but really it's not helpful and i mean we once problem our problems should be solved after all i mean we don't want to stick with terrible wouldn't experiences and terrible winter pieces rubber so explain why you have a certain problem why didn't exactly the problem that you have at least that you don't want to solve a problem that is not the problem that you have so if you just say it sucks then i can start i prototyping in which ways it sucks and i don't know me it sucks for me i think all it's probably how it sucks for him so i'm just gonna i'm just gonna focus problem and in a month or something that's gonna couples again it's not i'm using at or something like that i'm not using you know many more the aim of about report is to enable the programmer you see the program flailing in front of them this is true for a lot of callbacks so you just described much and then you try to and use the programmer to reproduce that like that you had so that you can see the problem failing are crashing in front of them but it's kind of tricky for you what's but because a lot of times that you whisper the experience problem already usability problem the really personal matter it it's really about how you felt about the interface always took a lot of time it was a to struggle like i could not use it i don't really competent about it so it's much harder to just give a designer about and an expectation to reproduce it to make it the to make the problem fail because the designer have a different experience they have a different mindset you have the different personality and that he therefore you definitely there and we have different to stop or so this prime the context i've been i've been helping with the i have fights working with you know my also which may which were dressed and i've been helping them in the forums and a lot of that they work that it sometimes that in different they just want to change customise the website somehow and it's a lot of C S hope and then you just don't have a again find some success that works for them and so their problem and it's really help for me when people come and see how to build the first thing about that i really don't know this thing because it really gives me a start point a starting point and how to talk to this user how to solve the problem for down and or also is different things to me like i don't know i guess that's what i like reading on it i've been trying to solve it but i think my favourite i can so this so you know this person doesn't know C S but he wants to know see as that so i can help him in a different way i usually try to explain to be there how to use fire about how to do certain things so it's really make when you provide complex when you provide information on line to do certain things what you how you we got to start and stop it really helps people to impress your problems to help you and to choose so whereabouts and the third one is most users most users would find this an intuitive it combines to present or most users i dunno wouldn't want to button to be somewhere else and i mean did you actually talk to most users did you actually like it up fast in your neighbourhood and you set up a most users definitely don't like this in my neighbourhood and then it would be able to the neighbourhood but not most users that all so expenses is that is really personal so you should really talk about yourself about how you feel how this because they're first learning convex because of all you you've been dealing with it why is it is it a problem for you not for most users because you're just assuming it's a problem for most users and it turns out that several it's actually a very specific use case that only you have still it's not most users perhaps the most users had it's very specific use case then you it's another thing i don't know but in general you lex is a fairly personal matter so you should always say i want i still i wish i struggle i think it's about you are experiencing that about that is that the same thing with code you cannot take it most it precious for most users you know i mean it was just for you you're using this machine you're having this kind of application running and whatever you have a certain encompass after nine think that only it's only britain and for your experience number of words to really minor and this is the thing and something you think we should review what the box watching of all right and think about this right now are gonna minus you i or usability and usability bugs not you wax as a very minor and as we want to talk more about our experiences i mean usability problems usually a remove something somewhere or like this is not consistent with something L's or we should rename this button and it's usually like the lack of absence of the presence or absence of a certain design characteristic which i understand that something minor but i mean alright are you what's but also ready minor issues it's one cannot figure out how to do something and they just are not able to do something is there's really it's a very have a really minor issue it's the not understand what system is the only know if they cannot and they get this message this dialogue and they have no idea what to do and how to recover officer dinner it is really a minor issue it's does not know whole to continue what that's great and that's it is again assume a minor issue it's one even more seats one something and they thought it was very successful and only later do you realise it was not and they were like but actually did it is it really a minor issue i mean you what the user to think he was able to accomplish something when he actually it like seems fine to me but it actually it user actually failed to accomplish what he was trying to our goals issues and i'm asking this all guys minor issues or is it like a block or issue a critical matter for that experience i mean if we are compromising the ability of things which are probably can do what he it is really a minor issue the user is not able to use the system to use needs and then well as a side i think this is should we should this is something we should be who was a community together and perhaps we should think about better i got mine sorry you want like to specifically for experience but because they can really compromise the whole experience and the whole use of soccer there is this is a very ratings and they have a different purpose they therefore so that a designer can right but on box problems feasibly problem that he found after doing and usability testing or something or heuristic evaluation and then write them and then decide what should be big first and this kind of thing so it's not really if something that the user or the report it uses but it's a start i guess so the scale is more like they're all going to reduce the problem too with the cosmetic horrible treats a minor issue for the major issue and five it's a usability catastrophe like it really compromises the whole thing so you know i'm gonna talk about how to do it and after my researcher or talking with what the designers are interview and i came up with a list of six information types that it would be really helpful in reports can include most of them so they are there and freezing them as questions and they are what are you trying to do why did you want to do it what did you do what exactly do you do doing but tracks and what happened what were your expectations and what are you running and again i interviewed designed it's working with open sores free open source projects so this is really pertinent for did there is the main on offer in general reporting usability guidelines or whatever so i'm gonna talk a little bit about each of this questions and everything is what we're trying to do which is also associated a bit with the problem so state what is it that you're actually trying to do what was your main goal when you started this interaction i wanted to print a file it's much more helpful to say i wanted to print a file then just say like insert this checkbox somewhere that you can be an option to do something related sprinting or something like that so this those are some quotes that i got from the designers that in their interview we try to get more information about what is it that they were trying to do because that has to be the drive the solution this has to be what guides the solution you don't you don't want to so that the problem that is not the problem that you have as i said many times why did you want to do it and this is a tricky question and it's not something that will be always pregnant or or it will not always of white your but sometimes it doesn't really make sense but it's a question related to the complex why want what motivates you to want to do certain axe and then that's got that comes of information is really valuable to me it helps me understand the motivation behind the request or sort of problem so there was an example that was someone was using a startup location and they want their something smaller and why would you want your compact smaller why is the reason for that and then the user would explain that i want my complex smaller because i have a lot of contacts and no not all of them have out there are so it's not really helpful for me to have a big i got and all that so it gets a lot of it get similar background that will change the way the designer will tackle the solution and it's not related to exactly what you're trying to do because what he was trying to do what you make a picture smaller but two related to why you want to walk money made it seem to want to do a certain action and of course if you're a i had a problem to try to connect to the internet why do you want to connect to the internet i mean it doesn't make sense to me that i wanted to get to interact so i could like would work picture for cats or something like that i mean it's not relevant to design it all and it's really hard to make this judgement on when there's this person and when it's not but it's something to keep in the back of your head when we are reported bugs what did you and this is about describing the whole thing this one happened during the whole interaction it's great one reports our narrative you know every interaction problem is very much like the story so i wanted to do this and then i tried this thing and then that thing happened and i didn't expect is because expected that sort of thing to happen and then to work around this i try to do that sort of thing but that didn't work too silly tells the designer everything that's you try to do and it might seem like a bit lost embedding when we when i ran some experience some studies trying to ask people to report things answering this question most of the time they were able to report things using like three or cricket lines describing what is it that they were to they did how the expects the whole interactive works and it's really helpful especially when you mentioned the things that you try to do to solve a problem this else to do this tells a lot of the designer about how you think and how you actually how what is your experience what's operate how you used to do things and brand that we do not know exactly how what is our user base this is really helpful is a real helpful to try to understand for a certain perspective on how the user is and because it really doesn't make sense told this is the first speaker building your report tell us we or what your experience in really it's not helpful still there is we have to find another way to understand so the user is and what are see experience and how we can better address the problems that he's facing so you know the very helpful this is what i wanted to do this is where i started this is what i did this is where a one problem and this is how i try to work around it and if you will write this and that you should never be afraid of are offering too much information because if you want for too much information the design there will just ignore that part of the information that he doesn't need so it's much better if you provide a lot of information then if you lack information because then the whole the whole processing of about becomes really time consuming you just have to engage with the reporter and go back and forth and try to make all those questions that i'm stating and and this takes time and then your bags gonna take longer to be solved and all this kind of things what happened out then you may have fairly lengthy interactions with people just trying to understand what is it that your experience so when you face a problem tell exactly describe exactly what is it that happens you that you found it was a problem that you think this is the problem i have to report this to the committee i have to tell them and we have to fix it make supper better and it's really hard to understand i i've read five hundred forty seven bytes so it's really hard to understand before talking about it to be a lot of time and i was like i cannot understand this person at all i mean and it was all english and of like that's an english so it's really hard to innocent looking for talking about the mission not a lot or and you don't know where this top or is or at certain button that is this there has of the label when you don't know when you wonder if you in the right place and you cannot find what this person is talking about so it's really important to be really descriptive about what is it that is going wrong and how you perceive is how do you see this is what we're expectations this is another really important bit of information because it tells it as what you did what where your steps true to work on the problem and also tells you a lot of our user a interacts with software how this user fangs how what is the experience this user has so a lot of the feedback i get is always think that should be why and we don't know if that's because the reporter just think that's a better idea or if it's something related to your actual expectations their users and experience so it's really it's really helpful to make this distinguish this distinction because it it's very different if someone thinks this should be done like this because he had this problem is like how serious and you things a bit hard and i guess that would work that would solve it and it's different when the user expected some it's the meat it solution it's something that it's natural or we can say intuitive for forty users so it's really important to understand what is it that you expected to happen that didn't happen so expect to needs usually very revealing this is where i think the best information comes from the expectations and the difference between i think actually you why and i try to do wax and i expected why but then Z happened and i wasn't able to accomplish X that definitely much more helpful so i guess this except this is a good example of the difference on how on a more complete report enables it wanted to do much more for you than if you might to waste i think that should be white which has i said was a very public and what are you running in this is especially more than in our compass of free and open source operations we are always like really things operate quite a quite stored in sort much or at intervals and always and iterating and always on the mideast also it's really important you know what is the margin that's been running on so you can actually understand what is it that people were talking about so a lot of time it's because of the version number that i know what the user is talking about because some things sometimes you report a bug because you're using is our version and then another version this thing doesn't exist anymore anything i have this problem with this thing and that place and then the very using another version is i don't see that thing and sometimes you just like and they are that's result and tell the users it's all but sometimes it's not that easy to understand so it's really important she to say what is the version of the for are talk about your experience others that a lot of the bugs that we report nowadays are minor issues so i'll change the way about this button and on we just spacing here and there are make that's consistent with of the up and those but please don't get me wrong there really important they really make experiences ten dollars on usability votes there definitely crucial for the experience but we should also talk about how we feel when we use offer because we use it everyday if it doesn't make us happy what is the point i mean especially at me we deal with some for every day it should be something pleasant it should be something that it's a joyful experience sell that user experience is definitely about if something takes a lot of time to do it is about if you're if it makes you like it's a lot of work you don't you don't feel sick say if you don't feel like confident that is going somewhere this is about it's not supposed to be like that you should be content that when you do something it will create this on a result and it or that if you're gonna you have like you have to send email you want to take twenty minutes adding an e-mail you want to be a straight for express because it's a really simple things of all the little bits of express their really important for software so easily about thirty like usable software so cool being concerned about usability matters is really important it's great but and other said it's also condition for a good you wax but it's not all so we should definitely be talking more about our experiences on possible i i'm sure that's kind of feedback it's in most part of the experience feedback is like the so called it sucks but reports it's like people don't talk they just at you can say it's a and experience problem if you can see that there really and happy using that feasible for body you don't know why it just say it sucks and we got a lot of this feedback so i guess people want to talk about this but they don't talk about this probably so i think this is important to keep in mind answer yes is that right as much as needed be specific describe describe what you're seeing what you're doing what interaction it something else three or more race to be done it's some specify why didn't exactly way you try to do it be verbose don't have this don't be afraid to write a lot if something is not pretend to design it will just ignore it it's better if we provide more information that we've are we letter lacking information to work on describe that you see describing to did and most of the most important describe how you felt it made me frustrated and don't be and like don't be an awful able this just a like i felt frustrated i didn't do it is it was hard for me you don't have to append anyone because to i understand that when you're talking about your experience it can be come quite personal and you get a lot of anger in your heart and you have to let go but we're trying to working together to make things better so describe i didn't get prints ring it's really helpful if people provide screenshot of the problem expense and they can point this is what i'm talking about this is the bottom and talking about this is the barn talking about this is the thing the it a it there is a lot of difficult as and verbal communication and sometimes we don't really get to understand it shorter but if we provide visual aids to reports this makes it much helpful much easier for the device work and if you guys once you walk the extra miles and i guess this is not very so related to but report itself but more to just be back to a larger extent please i raise it's really good if you guys you raise to design to this offer easy if it makes you happy if it helps you get your things don if it if it makes your like that or you raise so people that's like this that this new feature is awesome it enables you to do so much more it's really hard to know when we're doing things right because we only know we only hear about when we're doing things wrong and it's almost like know use it's gonna news if you don't hear anything it's probably working and it's actually not true because sometimes people just do not care enough they are i can work around as i can live with that in you want people to just they have with what you i mean it's are like that just operate it something they're getting together we want people to should be proud of it and to be happy with it i want them to level you know a great help people money right tell people wanting it and there's also help us know we're doing we are going in the right that thank you that won't we assume so because the only use money designer z-norm projects most of the from line in when it comes to boss it's on the web developers and the thing is if i have given a solution like that immediately say this is are you out of this is a code if he knows and this inbox close not of are because it is the design of so okay this thing is that the record is working it what it was supposed to should alleviate opened should i do well i guess that something really need to i mean i guess you know they have been doing a very good job and getting developers and decided to work together and i think as we get as a get better wax reports on the just have to work harder and on well switcher because i mean you're definitely we need a i don't know i'm tell so identified as i mean i i'm pretty sure alan already with all the reports regardless so and the ability to work together with the design and that has we have a spring them and we met leave them excellent i think this you not be problem i guess it's something we just it's something we need to think about as a committee a and i definitely don't think something's are eight i think there's a lot of like that i i'm pretty sure somebody years that you got your not so it's physical problem are a so you know i problem and sometimes on will get some usability bikes that you're like i don't know perhaps it is difficult and a lot of times it is so i don't think it's a problem that doesn't exist not is it it's just the problem that we already have this one so i guess like a lot of like but we tools to do are very right you do you do the subject line than that you get a box where you can write a story if you want to write or you could write poems and letters to of about short something which basically there are very few so we knew we know you some very small this to use your masters to you know you are or you who do or something about a well i think we are trying to see how people would report correctly they were giving those exact questions that i presented to you guys and i guess people provided a lot of and experts the only certain what do why do you want to do something question that people to confuse with it and that is that it's not always burden and it doesn't really make sense us that so in general people when they were guided like this is what i need to know i feel that people are provided much better reports that were much more helpful and but i said it's very much tax it's very eccentric and i guess there is a problem that is a financing this kind of communication that it's not very useful for a big report so i get there is a lot of research and that actually observed that on it's not a very useful very helpful a very easy model to just write text like crazy on sale need something they have to think about we know but what i found like one of the nicer things about your for because i filled it out as well and my friend and neighbour fill it out that and my friend and neighbour be he was saying like so i followed several bugs and i was i was like it was kind of hyper right because he didn't have anywhere to be angry because none of the questions were like what were you doing like how are you doing it but is but he couldn't find anywhere to round in that for at all so you didn't of course right which i thought it is my opinion there is no i think when you like what do you mean only about you don't them gives you a lot of space to be angry at all as you're talking about yourself we are talking about how i mean that every night i like that you did something wrong it is not and a lot and the outer yourself and that you like very personally so you gonna be rewriting or who proposed guidelines these i can i can migrate i have there was it's really hard to make this request in public but what so forget about you probably recommend to have more structured it's just feels and reporting box but so we just to use but new houses all cool by the this book report form which is probably not very configurable and will look very are we wanna comes to the H T M L code which doesn't it's and C S the last time we look the that so it's probably from ten years ago and can know had accustomed patch what is one big text field but you normally have time to description from your bug reports and we had something in there like steps to reproduce want to three expect that's results actual results ripper disability dispatched wasn't reapplied after the last operates looks a lot from version two to three and i guess these are all areas that we should consider at some point so i'm saying this also is a run reminder to myself