Value of unit test
Mustafina, Katya
Little over 10 years ago, we have learned and adapted test-driven development (TDD). TDD is a huge step forward to software quality, as well as to the value of your software as a company asset. Functionalities and architectures wrapped with tests can be changed under control and evolve — one of the biggest achievements in the industry during the last decade. When codebase grows in size and complexity and dependencies start degrading team productivity, the problem of how to maintain automated test base comes to stage. Production code changes over time due to new functionality, bug fixes, etc. – and it affects test’s code and lifetime of the tests. If a test fails due to changes in code, it means an error has been detected early and the test has done its job. This is point where a developer should fix a problem and a key to fix is understanding of the test, why it fails. This leads to good test design as key component of continuous integration. We cannot submit on red build and then cannot change and develop our application. We generally want to run the whole the test suite before every code change submission and how long does it take to run - plays significant role for big projects. if whole test suite takes more than 15 minutes, it becomes an obstacle for doing frequent changes. Teams start running not full suite or try to detect relevant tests for this particular change, etc. Problem of redundancy in unit testing due to the large number of people is working on a project become important once test code grown in size. Developers write tests for their components, and some tests overlay each other—often covering the same code over and over again. This is not a problem from quality point of view, but maintenance can be an issue. Here we come to distinction of unit test and integration test. The important quality metric is code coverage. There are aspects that this metric does not reflect: sensitivity to input data. Many workflows are vitally dependent from input data: small difference in data would only add couple of lines of code to coverage, but makes huge difference in calculations and final results. However, we often do not take these effects in account when measuring quality metrics. Our most valuable tests are those that fail more often than other. The right test is the test consciously addresses right scope, have expected time to run and easy to understand when fails. I my talk I will be speaking about how good test design affects productivity of agile project based on large and legacy codebase. I will be focusing on lessons learned from existing project, I have been involved with, coming up with examples of good and bad test design and its consequences in project delivery. I will be giving live code examples on the go.
 Hi. Before we start, I wanted to ask a question. Who is writing unit tests? Super. Many people write unit tests. We will use C-sharp. Many people use C-sharp. Java. C++. Because two people, three people. Right. So, did you ever have a problem maintaining your tests and trying to read it and not understand it? Have you ever spent hours in understanding the test? Did anyone have this problem? Yeah, a few people. So, we'll talk about that. About how we can write the test that is easy to maintain. That it's easy to work with later on. My name is Kaisa Mostafina. I'm a developer. I developed software for about 17 years in different companies in countries. I worked for six years in the Norway Technology Center. Which is part of the biggest company that produces oil services. We produce software solutions for the oil industry. The flagship product we produce here in Norway is called Petrel. Petrel is a software that helps people find oil. That is a tool for surface modeling. If you're an oil industry, you can do with Petrel basically whatever you want. The coolest feature of Petrel is 3D visualization. That's why I have this cool picture on that slide. That is considered to be one of the largest of the packages in the world. It has hundreds, developed by hundreds of developers and millions of lines of code. This is a successful product that is being sold in all the continents. The key component of that success is automated testing. We have starting developing Petrel in 1995 here in Norway when TDD wasn't really around. We had to adapt to it. We had to start using it at some point. We have gone all the long way from total hate to total love. I will try to connect the good practices of writing a unit test to our experiences and to what worked and what didn't work for us. TDD, unit testing, it became a religious issue. Everybody is talking about it. Some people believe it. Some people are deep fanatics. Some people follow it to some degree. There are some people who don't think we don't need it. There are still some people like that. But generally, as a programming community, we have adapted to unit testing. We are all committed to spend some time to write tests for our software. That's generally happened because people tried it and saw the value of it. They realized there is a benefit for us. We started spending less time on bug fixing, less time on testing. Bug fixing has now been our favorite activity. So we recognize the value of it. We really like that. That is a value that we all share these days. Like 10 years ago, it would be normal to talk to someone and realize, okay, they don't have an automated test at all. They don't have automated tests. Today, it's nearly impossible to have that kind of conversation. I think everybody raised their hands when I asked people to write the test. I had this kind of talk recently with some peers who don't have unit tests. But that's actually immediately smells. They spend a lot of time bug fixing. They spend a lot of time testing. It's probably not that exciting on what they are doing on an everyday basis. So, and we have done a great job in using tests as our helpers. And this talk will be very much about there is more than we can do about that. Better quality with the less effort. And let's outline our problem statement here. So we are using automated tests and our safety net. Well spotted by Martin Fowler some years back who invented that acronym. So it's so easy to refactor. It's so easy to change the code that we have a test for. So we can easily detect the problem. And that is all very good. But then we write more code and more tests. And more code and more tests. And there are several releases we have gone through with this test. And sometimes they start failing. We have to go back to them. We have to understand what is happening. And that brings us to maintenance problems. If you cannot easily detect what is happening with the test, if readability is bad, then we spend a lot of time on that. And I spoke to some colleagues some years back and they just started scramming their team. And that iterative development actually made it very visible of what they are doing immediately. And they were complaining like we spend an iteration just fixing the unit test. And next iteration we spend just fixing the unit test. We are not doing anything good. We are just fixing the unit test. And that is the situation that we want to avoid. We don't want our safety net to bring us to this. We don't want that. We want to spend less time on maintaining our test base. We want to make it easy. And at the end we are not paid to do tests. If we could ship the software without the testing machine, that's fine. And no one would ask us where do you have automated test suite. That would be fine. But this is our own internal tool to do that. So let's talk about what makes this hard to maintain. What we can do to make things easy to maintain. And here brings us to the question what is a good test. And looking at the good test, what we can think about. Can we think about how it looks? Can we think about what a test which is coverage? Can we think about coverage? And we talk about coverage all the time. It's our main metric of the quality. But generally should we think about coverage or should we think about other things? Should we think about when we run our tests? Do we run them all in the same time? Or should we probably run some integration tests afterwards? Or where we test? Which part of code we choose to test? Or we should blindly follow the coverage and gain for the coverage. We'll talk about in a minute. And the first thing I wanted to talk about is readability. We'll talk about how it tests. Anyone had that situation? Some people had. Right. And why we care about readability at all? That is not a production code. And the thing is that we want our test to fail at some time in the future. Anyone had a situation when someone told, okay, I had written that test and that was so good because it saved me so much time afterwards. Unfortunately, we cannot predict it. But we actually want them to fail sometime in the future. And when it fails, someone needs to go and understand why it fails. What happened, actually? And we want to minimize that effort because at this point we are doing something else. We are developing software that is not related to maybe what this test is doing and that part of software that we have written. So when we do that, it's ‑‑ if you have to spend a lot of time actually understanding and maintaining the test, that is a pain for us. We don't want that context switch. We want to continue to do it. We want this to be a light activity to go and understand the test. That is because of that and then fix it and then go on. And I really like the Roy O'Sheriff phrase that he wrote in his book who said that readability is the connecting thread between the one who wrote the test and the poor soul who will read it a few months later. Tests are stories that we tell the next generation of programmers in the project. And that is so true. We want to tell that story and we will tell a good story. So let's look at the part that would actually help us to reach ‑‑ to achieve readability. How can we get there to write a good readable test? And let's look at the part that is called a range actor. Anyone is using that part when writing their tests? Yeah, a few people. And that has been blocked I think in 2003. So I have a link below that. And it's very simple. It's just ‑‑ it couldn't be simpler. But it actually explains what data being set up, what data being executed, what concerns we are interested in this test, and what kind of verification we are. So when we actually organize our test with a range actor third, that is becoming so easy to achieve readability and to help people, those poor souls who will read it a few months later. So before we go and look at each step of range actor third, let's look at the example. That is an example. That is trying to test something. And I called it messy test example. Is it easy to understand what it is doing? Can we easily identify what is actually happening in this test? It's probably not that easy. So let's look where a range an actor served in this test. Let's look at that. So if you try to understand and read it carefully, and it took me some time to actually color it and understand it, so you are setting up something and then you are doing some mix of a solution and execution, and then you are setting up something else, then you are doing something more, and you are doing all of that stuff. And I think if we look at that test, we see a several arrange, and sometimes a range actor third is even mixed up all three together. So someone who will read that test will be actually confused. What is this setting up? What is happening? What is going to be? And he is going to spend a lot of time to understand it, and he doesn't want that. So maybe one day he will say, okay, I'll look at that later on. I'll put this test to hold. I'll mute it until sometime. And that actually never works because there is a tendency this test to stay there forever. So that is a violation example. So what we can do to make it better. And let's look at the arrange part. Arrange part is responsible for data set up. And the important thing here is that it is all consolidated at the beginning of the test. There is nothing that you can set up something, then execute something, then set up something more, execute something. We are setting this up at the beginning of the test. And to do that, there are several actually partners that I used to clarify of what you are setting up. Two of them, for example, are object, mother, and fluent interface. Those are partners that we can use, and I'll go into example in a minute just to clarify what is being set up. There are some links on Martin folder, what you can do about them more. Let's look at the example again. So this is another example. And that's quite simple. So let's look at the test on the left. What it is actually doing? That is trying to set up some points. What is this for? That is trying to create some plane. So probably point three is a class that is a point in a three-dimensional coordinate system. And then plane three, some plane in a space. But what is this? What is this? 0, 0, 0, 1, 0, 0, 1, 1, 0. We have to kind of think about it. Why is those digits are here and stuff like that? And that is an arranged part. That is consolidated at the beginning, but we can't actually read what is happening here very well and just at a glance. So let's look at what we can do. What are we actually doing? I will call it we are creating XY plane. We are creating XY plane here by those numbers 0, 0, 0, 1, 0, 0, 1, 1, 1. That is just XY plane, the plane perpendicular or token out of the Z axis. This plane is well-known and people understand what it is. So we can just create a method, a helper method that would just create XY plane and just name it that we are creating XY plane. And if you go to right-hand side, the one who is reading, he is actually understanding that this is XY plane created. Okay, I understand that. And what is the point? Where is this point? We create a point at the axis. So we understand there is a point exactly on the axis that we are creating. And those are matters, those are data that we are going to test. So here is the helper matters and we could call them object mothers. So those are methods that we consolidated to actually make a better readability for this test, with simple example. And we emphasize what data are being set up. And we could actually abstract from the constructor details as well. And we could reuse it in several other classes. And we could create objects that we are familiar with. For example, we can create diagonal plane. And we could create horizontal plane. And we could work with those objects in our test framework, in our communication framework. So we understand what kind of dummy objects we are creating to test. And those helper methods actually serve us as a mother for those objects. There are many methods to create XY plane, but we only use those points. But we don't want to know what kind of points we use just to, we want to abstract on those details. That makes our test readable and our setup readable. And fluent interface, I don't have example, but it's very similar if you have a long constructor, you can actually set up special methods like we's, I have a girl with a balloon or go without a balloon. And then you can build that particular object and return it. So you create sort of a specific number of dummy objects that you are working with in your test framework. And that is very handy. And this is very good for readability. So that was arranged. And there are more buttons for arrange. There is this book, X-Unit Test Partons that Geraldine Marsalis has written. This explains all of the things that we do in detail. It's 900 pages if you want to read it. Let's look at the act. What is actually act? It is code execution. So what we can say about it, we just execute the code. But there are some things that we can think about as a back practices. We could make it visible. Yeah, some people insert asserts in the act part. That shouldn't be there. We should assume everything works here. We are only executing what we are trying to execute. We don't know anything else. And if you want to actually isolate ourselves from the environment, we can use stubbing and mocking. And use the different dummy fake objects for those that our object depends on to make it isolated. Let's look at this example again. What is this act part? It's not quite clear. It's actually here. And it's mixed up with assert. So the one who is reading that, he has to understand, he has to read it like a just wide line, which is not very handy. I think it's just a mixed up. But on the right-hand side, we can see a call. And we can actually understand what is happening here. We actually make a plane and calculate a distance to a point. And we get this number. That's what the one who is reading the task would be interested in. And this is much more readable than we have it here on the left-hand side. So the act is separated here. And it is clearly visible. And those are some other examples of act that are what we do here. We just execute some code, calculate some context, do the work that we need to do. And now we come to assert. And the assert is a verification stage. It could be state verification and it could be behavior verification. And state verification is where we verify the state of our objects, what happened to our objects, what are stated. And behavior verification is a little bit more tricky where we verify actually how our object influenced the other object. Have it been called? That is also a button and we should do one or another or not mix of it. And the best practices here is to have one assert. So, and why we should have one assert? Mostly it is because you will never execute multiple asserts in the test. The test framework wouldn't let you to do that whatever the test framework is. And Microsoft framework or G-unit, they will just stop at the first assertion. That's how they design. And you will never be able to execute all of them. But you want to focus on one thing, what you are doing. And you want to write a clear message for that assertion, what actually happened. This is only achievable with one assert. And if, for example, some people just try to write an assertion or let the asserts that assert in different properties of one object, that is easier to do just comparing objects. And just one assert should be a rule, I think, for everybody. Let's look at the example here. So here we have two asserts. So that is testing completely different things. So we actually can divide this test in two different tests. But on the right-hand side, this is clearly visible what is being tested on the assertion. So that was arranged after assert. A pattern that if you use it, you actually almost achieve readability for free. Because I used to write the messy test myself. And when I was trying to understand what this test is doing and someone else, it's just what are concerns being tested? What are data being set up? And what is being verified? And once you reply to yourself on those questions, that is easy to make a decision what actually happened in a code. But what else we can think about when we think about how the test should look and what it should do? Let's look, let's talk about the test scope. The scope of the test. It should think, it should test one thing. It should focus on one thing to test, not on multiple things, on multiple aspects, maybe the problem with multiple properties of an object. An indication that we are testing several things would be that we are having some logic in the test. If you're having if you have a statement, if you have switch statement, if you have for loop, we are, it's very high probability that we are testing different things in one test. That we are trying to test multiple aspects of that. And of course, if you have multiple asserts, we are almost certainly testing different things. If you are wondering about that, you can try to name your test just by looking at all the asserts and just write the sentence that this test is doing, this distance, you get a long sentence if you have multiple asserts. It will be several things. So that's a good practice to have one thing and it will help people to understand it later how actually test is, how test is working, what test is testing, the size of it. What is a good test size? That is a religious issue again. So I believe in five lines. Some people believe in maybe another amount. But generally, I think the perfect test is a five line, maybe six lines, I don't know the exact number, but again, it's a religious issue. I think six, I can imagine the test that would have 16 lines and would still test in one thing. I have never seen test 25 lines testing one thing ever. Anyone seen that? Okay. And that was testing different things. Yes. So it should be small and I believe this function should be small, but the test should be small to test one thing, to focus on one concern. And let's look at the good test naming. That is very interesting issue. So we have that concept, don't repeat yourself in coding. That means we are, we want to remove duplication. Duplication is a maintenance for us. We don't want that in a code. We don't want to repeat ourselves. But from the other side, we have the idea of writing everything in descriptive and meaningful phrases which is coming to us from the main specific languages. And there is kind of a balance between it because the more we write in descriptive and meaningful phrases, we could actually violate that don't repeat yourself principle. And I think test lives in its own little world. It's isolated. It shouldn't have any knowledge from the outside world. And because of that, sometimes trying to remove duplication, we are actually sacrificing the business logic and the meaning of the test. So in testing, being descriptive and meaningful is more important than removing duplication. Sometimes that's fine to have some sort of duplication in a test than actually trying to force people to read some other code. I was actually struggling to get an example for that. I actually pulled out some code I written a while back and that was a good explanation. But it was so difficult to understand in such a short time. So I just removed the example. But think about that. And being descriptive is much more important in tests because they live in their own little world. And that's fine if you repeat sometimes. It's a test code at the end. It's never executed in production. Good test coverage. That is my favorite slide. How much we should test? We talk about coverage all the time. Management uses it as a metric to detect quality. Is it really true? So I will tell you a story. Three programmers asked the gray master of what coverage should I aim for. The first one asked and the great master said, don't worry about that. Just go and try some good tests. And the second one asked about that. What coverage should I aim? The great master said, how many grain rice should I put in that pot? That depends on the second programmer. It depends because how many people you have to feed. What other food you're serving. What actually you are, what is this, how hungry they are. And those things are how I can tell you. That's what he said. And the master said, yeah, that's right. And that programmer left. And the third one came. What coverage should I aim for? 80% at no less. Said the great master. And then the followers came to the great master and asked him, we heard that you did the three different answers to the same questions to different people why it is. And he said, the first one was new to unit testing. So he had to just focus on writing some good tests. And the second one was experienced. So he understood and realized with my answer that there is no simple answer to that. The third one just wanted the simple answer to the question that doesn't have any simple answers. So he got it. 80% at no less. And in coverage, it is very important that we should balance. Does anyone has 100% coverage? We are saying that we just have to do it. Do we have to blindly follow the coverage? Do we have to blindly aim 100% or 80% or whatever number, simple number? I can't believe that it could be some tests, some projects with 100% coverage. They still have bugs in that. We have to focus on something else and just test coverage. We have to focus what we are testing and where we are testing and what we are testing. And we can talk about also coverage duplication here. And this more brings us to the questions when we are testing. So we have isolated tests that test local things. And we have tests that are integration tests, results, stubborn and morking. The isolated tests are unit tests referred to. So we are focusing on one class, one object. It's very local. And then we have end-to-end tests. And we have to have end-to-end tests. Because there is no way that we can test the system. And we don't have end-to-end tests. However, anyone had a situation on one change in the code cost hundreds of tests? Yeah, a few people. That happens because we have some coverage duplication in our tests. And the good practice that we found here is just to run the isolated tests, local tests before and make sure that our local systems work fine. And then run our integration tests, end-to-end tests afterwards. And there should be the way less integration tests than the local tests. It's a normal situation. And then when we run the integration test, there's just a few. They test the actually components communication. They don't test the local things. And we call that smoke test that we run. And we run them first. And we make sure everything is fine. And then we can allow execution of the integration test. So that is a very good practice that we found for ourselves. Good tests fail. And they fail. And they want them to fail. But they want them to fail just for one reason. One good reason is that the functionality code change. We don't want tests to fail randomly due to some isolation problems. We don't want them to fail because there are bugs in them. We want them to fail just because we change the code and it fails. And there is a dependency on that. But we don't want random failures. But they appear that random failures and isolation problems and test bugs, they're quite easy to fix. But to find the tests that fail where we want it is quite difficult. We want to write a test that will save us some time in future but we cannot predict the future very well. How we can predict the failure information. Where should I test? Each test is associated with a cost. I have to spend time writing it. People have to spend time executing that. And we have to think carefully how we are going to write the test. Where? And here is Ken back. And he said, I should test where I'm likely to make a mistake. Not the things that always work. That's fine. But how do we know where we'll make a mistake? That is probabilistic investment. We don't know. That's a bet. We just... But we can think about it maybe. And he's saying, for example, I tend to make mistake around conditionals so I am trying to write a test around conditionals because I know I'm making mistakes there. But what else we can think about when we talk about the good test value? Can we predict the failure information? We normally don't think about it very much. But if you have a test and you have a code and everything works, can we think about when it will fail? We can try to comment out production fault and watch it fails. We can try to change production code in a way that we think it will evolve in the future and watch it fails. And that is a very good exercise in determining when the test will fail. And we normally never do that exercise when we write the test. We can look at the test as an API, as a documentation to our code. And we... This is a very good practice as well because we have... We are seeing what we are trying to achieve. And here we can predict maybe how it will evolve, how it will change. So we want the test when we feel like they are valuable, but not before. And that is a thing that what we can think about when we write the test. Which test to write and which test not to write. So what is a good test? So the question that we ask ourselves when we write a good test is how easy I can understand it. And more important is not like I can understand it and how easy someone else would be able to understand it. It is very, very likely that you will not be reading that test. You are doing it for your peers. You are doing it for the one... The poorest soul who will come to the project a few months later and say, ah. And then when it will fail, will it fail? What kind of mistake I should make so it fails? And where I can predict the bug? And we can think about coverage, of course. And we should think about coverage. But we should understand that this is not a simple answer to the coverage question. So we figure that the good test when we look from the maintainability point of view, it is more important to think about how it looks and where it tests. And then thinking about when it will be run and what it is actually testing and focusing on coverage. That was it. Thank you very much.
 Hi. Before we start, I wanted to ask a question. Who is writing unit tests? Super. Many people  write unit tests. We will use C-sharp. Many people use C-sharp. Java. C++. Because two  people, three people. Right. So, did you ever have a problem maintaining your tests and  trying to read it and not understand it? Have you ever spent hours in understanding the test?  Did anyone have this problem? Yeah, a few people. So, we'll talk about that. About how we can  write the test that is easy to maintain. That it's easy to work with later on. My name is  Kaisa Mostafina. I'm a developer. I developed software for about 17 years in different  companies in countries. I worked for six years in the Norway Technology Center. Which is part  of the biggest company that produces oil services. We produce software solutions for the oil  industry. The flagship product we produce here in Norway is called Petrel. Petrel is a software  that helps people find oil. That is a tool for surface modeling. If you're an oil industry,  you can do with Petrel basically whatever you want. The coolest feature of Petrel is  3D visualization. That's why I have this cool picture on that slide. That is considered  to be one of the largest of the packages in the world. It has hundreds, developed by hundreds  of developers and millions of lines of code. This is a successful product that is being  sold in all the continents. The key component of that success is automated testing. We have  starting developing Petrel in 1995 here in Norway when TDD wasn't really around. We had  to adapt to it. We had to start using it at some point. We have gone all the long way  from total hate to total love. I will try to connect the good practices of writing a  unit test to our experiences and to what worked and what didn't work for us. TDD, unit testing,  it became a religious issue. Everybody is talking about it. Some people believe it.  Some people are deep fanatics. Some people follow it to some degree. There are some people  who don't think we don't need it. There are still some people like that. But generally,  as a programming community, we have adapted to unit testing. We are all committed to spend  some time to write tests for our software. That's generally happened because people tried  it and saw the value of it. They realized there is a benefit for us. We started spending  less time on bug fixing, less time on testing. Bug fixing has now been our favorite activity.  So we recognize the value of it. We really like that. That is a value that we all share  these days. Like 10 years ago, it would be normal to talk to someone and realize, okay,  they don't have an automated test at all. They don't have automated tests. Today, it's  nearly impossible to have that kind of conversation. I think everybody raised their hands when  I asked people to write the test. I had this kind of talk recently with some peers who  don't have unit tests. But that's actually immediately smells. They spend a lot of time  bug fixing. They spend a lot of time testing. It's probably not that exciting on what they  are doing on an everyday basis. So, and we have done a great job in using tests as our  helpers. And this talk will be very much about there is more than we can do about that. Better  quality with the less effort. And let's outline our problem statement here. So we are using  automated tests and our safety net. Well spotted by Martin Fowler some years back who invented  that acronym. So it's so easy to refactor. It's so easy to change the code that we have  a test for. So we can easily detect the problem. And that is all very good. But then we write  more code and more tests. And more code and more tests. And there are several releases  we have gone through with this test. And sometimes they start failing. We have to go back to  them. We have to understand what is happening. And that brings us to maintenance problems.  If you cannot easily detect what is happening with the test, if readability is bad, then  we spend a lot of time on that. And I spoke to some colleagues some years back and they  just started scramming their team. And that iterative development actually made it very  visible of what they are doing immediately. And they were complaining like we spend an  iteration just fixing the unit test. And next iteration we spend just fixing the unit test.  We are not doing anything good. We are just fixing the unit test. And that is the situation  that we want to avoid. We don't want our safety net to bring us to this. We don't want that.  We want to spend less time on maintaining our test base. We want to make it easy. And  at the end we are not paid to do tests. If we could ship the software without the testing  machine, that's fine. And no one would ask us where do you have automated test suite.  That would be fine. But this is our own internal tool to do that. So let's talk about what  makes this hard to maintain. What we can do to make things easy to maintain. And here  brings us to the question what is a good test. And looking at the good test, what we can  think about. Can we think about how it looks? Can we think about what a test which is coverage?  Can we think about coverage? And we talk about coverage all the time. It's our main metric  of the quality. But generally should we think about coverage or should we think about other  things? Should we think about when we run our tests? Do we run them all in the same  time? Or should we probably run some integration tests afterwards? Or where we test? Which  part of code we choose to test? Or we should blindly follow the coverage and gain for the  coverage. We'll talk about in a minute. And the first thing I wanted to talk about is  readability. We'll talk about how it tests. Anyone had that situation? Some people had.  Right. And why we care about readability at all? That is not a production code. And the  thing is that we want our test to fail at some time in the future. Anyone had a situation  when someone told, okay, I had written that test and that was so good because it saved  me so much time afterwards. Unfortunately, we cannot predict it. But we actually want  them to fail sometime in the future. And when it fails, someone needs to go and understand  why it fails. What happened, actually? And we want to minimize that effort because at  this point we are doing something else. We are developing software that is not related  to maybe what this test is doing and that part of software that we have written. So  when we do that, it's ‑‑ if you have to spend a lot of time actually understanding  and maintaining the test, that is a pain for us. We don't want that context switch. We  want to continue to do it. We want this to be a light activity to go and understand the  test. That is because of that and then fix it and then go on. And I really like the Roy  O'Sheriff phrase that he wrote in his book who said that readability is the connecting  thread between the one who wrote the test and the poor soul who will read it a few months  later. Tests are stories that we tell the next generation of programmers in the project.  And that is so true. We want to tell that story and we will tell a good story. So let's  look at the part that would actually help us to reach ‑‑ to achieve readability.  How can we get there to write a good readable test? And let's look at the part that is called  a range actor. Anyone is using that part when writing their tests? Yeah, a few people. And  that has been blocked I think in 2003. So I have a link below that. And it's very simple.  It's just ‑‑ it couldn't be simpler. But it actually explains what data being set up,  what data being executed, what concerns we are interested in this test, and what kind  of verification we are. So when we actually organize our test with a range actor third,  that is becoming so easy to achieve readability and to help people, those poor souls who will  read it a few months later. So before we go and look at each step of range  actor third, let's look at the example. That is an example. That is trying to test something.  And I called it messy test example. Is it easy to understand what it is doing? Can we  easily identify what is actually happening in this test? It's probably not that easy.  So let's look where a range an actor served in this test. Let's look at that. So if you  try to understand and read it carefully, and it took me some time to actually color it and  understand it, so you are setting up something and then you are doing some mix of a solution  and execution, and then you are setting up something else, then you are doing something  more, and you are doing all of that stuff. And I think if we look at that test, we see  a several arrange, and sometimes a range actor third is even mixed up all three together.  So someone who will read that test will be actually confused. What is this setting up?  What is happening? What is going to be? And he is going to spend a lot of time to understand  it, and he doesn't want that. So maybe one day he will say, okay, I'll look at that later  on. I'll put this test to hold. I'll mute it until sometime. And that actually never  works because there is a tendency this test to stay there forever. So that is a violation  example. So what we can do to make it better. And let's look at the arrange part. Arrange  part is responsible for data set up. And the important thing here is that it is all consolidated  at the beginning of the test. There is nothing that you can set up something, then execute  something, then set up something more, execute something. We are setting this up at the beginning  of the test. And to do that, there are several actually partners that I used to clarify of  what you are setting up. Two of them, for example, are object, mother, and fluent interface.  Those are partners that we can use, and I'll go into example in a minute just to clarify  what is being set up. There are some links on Martin folder, what you can do about them  more. Let's look at the example again. So this is another example. And that's quite  simple. So let's look at the test on the left. What it is actually doing? That is trying  to set up some points. What is this for? That is trying to create some plane. So probably  point three is a class that is a point in a three-dimensional coordinate system. And  then plane three, some plane in a space. But what is this? What is this? 0, 0, 0, 1, 0,  0, 1, 1, 0. We have to kind of think about it. Why is those digits are here and stuff  like that? And that is an arranged part. That is consolidated at the beginning, but we can't  actually read what is happening here very well and just at a glance. So let's look at  what we can do. What are we actually doing? I will call it we are creating XY plane. We  are creating XY plane here by those numbers 0, 0, 0, 1, 0, 0, 1, 1, 1. That is just XY  plane, the plane perpendicular or token out of the Z axis. This plane is well-known and  people understand what it is. So we can just create a method, a helper method that would  just create XY plane and just name it that we are creating XY plane. And if you go to  right-hand side, the one who is reading, he is actually understanding that this is XY  plane created. Okay, I understand that. And what is the point? Where is this point? We  create a point at the axis. So we understand there is a point exactly on the axis that  we are creating. And those are matters, those are data that we are going to test. So here  is the helper matters and we could call them object mothers. So those are methods that  we consolidated to actually make a better readability for this test, with simple example. And we  emphasize what data are being set up. And we could actually abstract from the constructor  details as well. And we could reuse it in several other classes. And we could create  objects that we are familiar with. For example, we can create diagonal plane. And we could  create horizontal plane. And we could work with those objects in our test framework,  in our communication framework. So we understand what kind of dummy objects we are creating  to test. And those helper methods actually serve us as a mother for those objects. There  are many methods to create XY plane, but we only use those points. But we don't want to  know what kind of points we use just to, we want to abstract on those details. That makes  our test readable and our setup readable. And fluent interface, I don't have example,  but it's very similar if you have a long constructor, you can actually set up special methods like  we's, I have a girl with a balloon or go without a balloon. And then you can build that particular  object and return it. So you create sort of a specific number of dummy objects that you  are working with in your test framework. And that is very handy. And this is very good  for readability. So that was arranged. And there are more buttons for arrange. There  is this book, X-Unit Test Partons that Geraldine Marsalis has written. This explains all of  the things that we do in detail. It's 900 pages if you want to read it. Let's look at  the act. What is actually act? It is code execution. So what we can say about it, we  just execute the code. But there are some things that we can think about as a back  practices. We could make it visible. Yeah, some people insert asserts in the act part.  That shouldn't be there. We should assume everything works here. We are only executing  what we are trying to execute. We don't know anything else. And if you want to actually  isolate ourselves from the environment, we can use stubbing and mocking. And use the  different dummy fake objects for those that our object depends on to make it isolated.  Let's look at this example again. What is this act part? It's not quite clear. It's  actually here. And it's mixed up with assert. So the one who is reading that, he has to  understand, he has to read it like a just wide line, which is not very handy. I think  it's just a mixed up. But on the right-hand side, we can see a call. And we can actually  understand what is happening here. We actually make a plane and calculate a distance to a  point. And we get this number. That's what the one who is reading the task would be interested  in. And this is much more readable than we have it here on the left-hand side. So the  act is separated here. And it is clearly visible. And those are some other examples of act that  are what we do here. We just execute some code, calculate some context, do the work  that we need to do. And now we come to assert. And the assert is a verification stage. It  could be state verification and it could be behavior verification. And state verification  is where we verify the state of our objects, what happened to our objects, what are stated.  And behavior verification is a little bit more tricky where we verify actually how our  object influenced the other object. Have it been called? That is also a button and we  should do one or another or not mix of it. And the best practices here is to have one  assert. So, and why we should have one assert? Mostly it is because you will never execute  multiple asserts in the test. The test framework wouldn't let you to do that whatever the test  framework is. And Microsoft framework or G-unit, they will just stop at the first assertion.  That's how they design. And you will never be able to execute all of them. But you want  to focus on one thing, what you are doing. And you want to write a clear message for  that assertion, what actually happened. This is only achievable with one assert. And if,  for example, some people just try to write an assertion or let the asserts that assert  in different properties of one object, that is easier to do just comparing objects. And  just one assert should be a rule, I think, for everybody. Let's look at the example here.  So here we have two asserts. So that is testing completely different things. So we actually  can divide this test in two different tests. But on the right-hand side, this is clearly  visible what is being tested on the assertion. So that was arranged after assert. A pattern  that if you use it, you actually almost achieve readability for free. Because I used to write  the messy test myself. And when I was trying to understand what this test is doing and  someone else, it's just what are concerns being tested? What are data being set up?  And what is being verified? And once you reply to yourself on those questions, that is easy  to make a decision what actually happened in a code. But what else we can think about  when we think about how the test should look and what it should do? Let's look, let's talk  about the test scope. The scope of the test. It should think, it should test one thing.  It should focus on one thing to test, not on multiple things, on multiple aspects, maybe  the problem with multiple properties of an object. An indication that we are testing  several things would be that we are having some logic in the test. If you're having if  you have a statement, if you have switch statement, if you have for loop, we are, it's very high  probability that we are testing different things in one test. That we are trying to test  multiple aspects of that. And of course, if you have multiple asserts, we are almost certainly  testing different things. If you are wondering about that, you can try to name your test just  by looking at all the asserts and just write the sentence that this test is doing, this  distance, you get a long sentence if you have multiple asserts. It will be several things.  So that's a good practice to have one thing and it will help people to understand it later  how actually test is, how test is working, what test is testing, the size of it. What  is a good test size? That is a religious issue again. So I believe in five lines. Some people  believe in maybe another amount. But generally, I think the perfect test is a five line, maybe  six lines, I don't know the exact number, but again, it's a religious issue. I think  six, I can imagine the test that would have 16 lines and would still test in one thing.  I have never seen test 25 lines testing one thing ever. Anyone seen that? Okay. And that  was testing different things. Yes. So it should be small and I believe this function should  be small, but the test should be small to test one thing, to focus on one concern. And  let's look at the good test naming. That is very interesting issue. So we have that concept,  don't repeat yourself in coding. That means we are, we want to remove duplication. Duplication  is a maintenance for us. We don't want that in a code. We don't want to repeat ourselves.  But from the other side, we have the idea of writing everything in descriptive and meaningful  phrases which is coming to us from the main specific languages. And there is kind of a  balance between it because the more we write in descriptive and meaningful phrases, we  could actually violate that don't repeat yourself principle. And I think test lives in its own  little world. It's isolated. It shouldn't have any knowledge from the outside world. And  because of that, sometimes trying to remove duplication, we are actually sacrificing the  business logic and the meaning of the test. So in testing, being descriptive and meaningful  is more important than removing duplication. Sometimes that's fine to have some sort of  duplication in a test than actually trying to force people to read some other code. I  was actually struggling to get an example for that. I actually pulled out some code I  written a while back and that was a good explanation. But it was so difficult to understand in  such a short time. So I just removed the example. But think about that. And being descriptive  is much more important in tests because they live in their own little world. And that's  fine if you repeat sometimes. It's a test code at the end. It's never executed in production.  Good test coverage. That is my favorite slide. How much we should test? We talk about coverage  all the time. Management uses it as a metric to detect quality. Is it really true? So I  will tell you a story. Three programmers asked the gray master of what coverage should I  aim for. The first one asked and the great master said, don't worry about that. Just  go and try some good tests. And the second one asked about that. What coverage should  I aim? The great master said, how many grain rice should I put in that pot? That depends  on the second programmer. It depends because how many people you have to feed. What other  food you're serving. What actually you are, what is this, how hungry they are. And those  things are how I can tell you. That's what he said. And the master said, yeah, that's  right. And that programmer left. And the third one came. What coverage should I aim for?  80% at no less. Said the great master. And then the followers came to the great master  and asked him, we heard that you did the three different answers to the same questions to  different people why it is. And he said, the first one was new to unit testing. So he had  to just focus on writing some good tests. And the second one was experienced. So he understood  and realized with my answer that there is no simple answer to that. The third one just  wanted the simple answer to the question that doesn't have any simple answers. So he got  it. 80% at no less. And in coverage, it is very important that we should balance. Does  anyone has 100% coverage? We are saying that we just have to do it. Do we have to blindly  follow the coverage? Do we have to blindly aim 100% or 80% or whatever number, simple  number? I can't believe that it could be some tests, some projects with 100% coverage. They  still have bugs in that. We have to focus on something else and just test coverage. We  have to focus what we are testing and where we are testing and what we are testing. And  we can talk about also coverage duplication here. And this more brings us to the questions  when we are testing. So we have isolated tests that test local things. And we have tests  that are integration tests, results, stubborn and morking. The isolated tests are unit tests  referred to. So we are focusing on one class, one object. It's very local. And then we have  end-to-end tests. And we have to have end-to-end tests. Because there is no way that we can  test the system. And we don't have end-to-end tests. However, anyone had a situation on  one change in the code cost hundreds of tests? Yeah, a few people. That happens because we  have some coverage duplication in our tests. And the good practice that we found here is  just to run the isolated tests, local tests before and make sure that our local systems  work fine. And then run our integration tests, end-to-end tests afterwards. And there should  be the way less integration tests than the local tests. It's a normal situation. And  then when we run the integration test, there's just a few. They test the actually components  communication. They don't test the local things. And we call that smoke test that we run. And  we run them first. And we make sure everything is fine. And then we can allow execution of  the integration test. So that is a very good practice that we found for ourselves. Good  tests fail. And they fail. And they want them to fail. But they want them to fail just for  one reason. One good reason is that the functionality code change. We don't want tests to fail randomly  due to some isolation problems. We don't want them to fail because there are bugs in them.  We want them to fail just because we change the code and it fails. And there is a dependency  on that. But we don't want random failures. But they appear that random failures and isolation  problems and test bugs, they're quite easy to fix. But to find the tests that fail where  we want it is quite difficult. We want to write a test that will save us some time in  future but we cannot predict the future very well. How we can predict the failure information.  Where should I test? Each test is associated with a cost. I have to spend time writing  it. People have to spend time executing that. And we have to think carefully how we are  going to write the test. Where? And here is Ken back. And he said, I should test where  I'm likely to make a mistake. Not the things that always work. That's fine. But how do  we know where we'll make a mistake? That is probabilistic investment. We don't know.  That's a bet. We just... But we can think about it maybe. And he's saying, for example,  I tend to make mistake around conditionals so I am trying to write a test around conditionals  because I know I'm making mistakes there. But what else we can think about when we talk  about the good test value? Can we predict the failure information? We normally don't  think about it very much. But if you have a test and you have a code and everything  works, can we think about when it will fail? We can try to comment out production fault  and watch it fails. We can try to change production code in a way that we think it will evolve  in the future and watch it fails. And that is a very good exercise in determining when  the test will fail. And we normally never do that exercise when we write the test. We  can look at the test as an API, as a documentation to our code. And we... This is a very good  practice as well because we have... We are seeing what we are trying to achieve. And  here we can predict maybe how it will evolve, how it will change. So we want the test when  we feel like they are valuable, but not before. And that is a thing that what we can think  about when we write the test. Which test to write and which test not to write. So what  is a good test? So the question that we ask ourselves when we write a good test is how  easy I can understand it. And more important is not like I can understand it and how easy  someone else would be able to understand it. It is very, very likely that you will not  be reading that test. You are doing it for your peers. You are doing it for the one...  The poorest soul who will come to the project a few months later and say, ah. And then when  it will fail, will it fail? What kind of mistake I should make so it fails? And where I can  predict the bug? And we can think about coverage, of course. And we should think about coverage.  But we should understand that this is not a simple answer to the coverage question. So  we figure that the good test when we look from the maintainability point of view, it  is more important to think about how it looks and where it tests. And then thinking about  when it will be run and what it is actually testing and focusing on coverage. That was  it. Thank you very much.
Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_0_1295.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_1296_2039.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_2040_3911.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_3912_6659.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_6660_7091.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_7092_8435.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_8436_8483.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_8484_9503.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_9504_10739.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_10740_12851.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_12852_13763.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_13764_15263.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_15264_15647.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_15648_15911.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_15912_17447.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_17448_18239.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_18240_18335.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_18336_18383.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_18384_18467.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_18468_18599.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_18600_18755.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_18756_19787.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_19788_20123.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_20124_24287.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_24288_25367.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_25368_25499.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_25500_25847.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_25848_26579.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_26580_26783.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_26784_26927.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_26928_27767.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_27768_28451.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_28452_29231.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_29232_29723.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_29724_30623.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_30624_32627.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_32628_33935.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_33936_35711.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_35712_36839.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_36840_40667.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_40668_42851.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_42852_44063.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_44064_45287.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_45288_47015.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_47016_48167.png Katya_Mustafina_-_Value_of_unit_test_-_way_from_good_design_to_easy_maintenance-68378252_48168_48904.png
Hi. Before we start, I wanted to ask a question. Who is writing unit tests? Super. Many people write unit tests. We will use C-sharp. Many people use C-sharp. Java. C++. Because two people, three people. Right. So, did you ever have a problem maintaining your tests and trying to read it and not understand it? Have you ever spent hours in understanding the test? Did anyone have this problem? Yeah, a few people. So, we'll talk about that. About how we can write the test that is easy to maintain. That it's easy to work with later on. My name is Kaisa Mostafina. I'm a developer. I developed software for about 17 years in different companies in countries. I worked for six years in the Norway Technology Center. Which is part of the biggest company that produces oil services. We produce software solutions for the oil industry. The flagship product we produce here in Norway is called Petrel. Petrel is a software that helps people find oil. That is a tool for surface modeling. If you're an oil industry, you can do with Petrel basically whatever you want. The coolest feature of Petrel is 3D visualization. That's why I have this cool picture on that slide. That is considered to be one of the largest of the packages in the world. It has hundreds, developed by hundreds of developers and millions of lines of code. This is a successful product that is being sold in all the continents. The key component of that success is automated testing. We have starting developing Petrel in 1995 here in Norway when TDD wasn't really around. We had to adapt to it. We had to start using it at some point. We have gone all the long way from total hate to total love. I will try to connect the good practices of writing a unit test to our experiences and to what worked and what didn't work for us. TDD, unit testing, it became a religious issue. Everybody is talking about it. Some people believe it. Some people are deep fanatics. Some people follow it to some degree. There are some people who don't think we don't need it. There are still some people like that. But generally, as a programming community, we have adapted to unit testing. We are all committed to spend some time to write tests for our software. That's generally happened because people tried it and saw the value of it. They realized there is a benefit for us. We started spending less time on bug fixing, less time on testing. Bug fixing has now been our favorite activity. So we recognize the value of it. We really like that. That is a value that we all share these days. Like 10 years ago, it would be normal to talk to someone and realize, okay, they don't have an automated test at all. They don't have automated tests. Today, it's nearly impossible to have that kind of conversation. I think everybody raised their hands when I asked people to write the test. I had this kind of talk recently with some peers who don't have unit tests. But that's actually immediately smells. They spend a lot of time bug fixing. They spend a lot of time testing. It's probably not that exciting on what they are doing on an everyday basis. So, and we have done a great job in using tests as our helpers. And this talk will be very much about there is more than we can do about that. Better quality with the less effort. And let's outline our problem statement here. So we are using automated tests and our safety net. Well spotted by Martin Fowler some years back who invented that acronym. So it's so easy to refactor. It's so easy to change the code that we have a test for. So we can easily detect the problem. And that is all very good. But then we write more code and more tests. And more code and more tests. And there are several releases we have gone through with this test. And sometimes they start failing. We have to go back to them. We have to understand what is happening. And that brings us to maintenance problems. If you cannot easily detect what is happening with the test, if readability is bad, then we spend a lot of time on that. And I spoke to some colleagues some years back and they just started scramming their team. And that iterative development actually made it very visible of what they are doing immediately. And they were complaining like we spend an iteration just fixing the unit test. And next iteration we spend just fixing the unit test. We are not doing anything good. We are just fixing the unit test. And that is the situation that we want to avoid. We don't want our safety net to bring us to this. We don't want that. We want to spend less time on maintaining our test base. We want to make it easy. And at the end we are not paid to do tests. If we could ship the software without the testing machine, that's fine. And no one would ask us where do you have automated test suite. That would be fine. But this is our own internal tool to do that. So let's talk about what makes this hard to maintain. What we can do to make things easy to maintain. And here brings us to the question what is a good test. And looking at the good test, what we can think about. Can we think about how it looks? Can we think about what a test which is coverage? Can we think about coverage? And we talk about coverage all the time. It's our main metric of the quality. But generally should we think about coverage or should we think about other things? Should we think about when we run our tests? Do we run them all in the same time? Or should we probably run some integration tests afterwards? Or where we test? Which part of code we choose to test? Or we should blindly follow the coverage and gain for the coverage. We'll talk about in a minute. And the first thing I wanted to talk about is readability. We'll talk about how it tests. Anyone had that situation? Some people had. Right. And why we care about readability at all? That is not a production code. And the thing is that we want our test to fail at some time in the future. Anyone had a situation when someone told, okay, I had written that test and that was so good because it saved me so much time afterwards. Unfortunately, we cannot predict it. But we actually want them to fail sometime in the future. And when it fails, someone needs to go and understand why it fails. What happened, actually? And we want to minimize that effort because at this point we are doing something else. We are developing software that is not related to maybe what this test is doing and that part of software that we have written. So when we do that, it's ‑‑ if you have to spend a lot of time actually understanding and maintaining the test, that is a pain for us. We don't want that context switch. We want to continue to do it. We want this to be a light activity to go and understand the test. That is because of that and then fix it and then go on. And I really like the Roy O'Sheriff phrase that he wrote in his book who said that readability is the connecting thread between the one who wrote the test and the poor soul who will read it a few months later. Tests are stories that we tell the next generation of programmers in the project. And that is so true. We want to tell that story and we will tell a good story. So let's look at the part that would actually help us to reach ‑‑ to achieve readability. How can we get there to write a good readable test? And let's look at the part that is called a range actor. Anyone is using that part when writing their tests? Yeah, a few people. And that has been blocked I think in 2003. So I have a link below that. And it's very simple. It's just ‑‑ it couldn't be simpler. But it actually explains what data being set up, what data being executed, what concerns we are interested in this test, and what kind of verification we are. So when we actually organize our test with a range actor third, that is becoming so easy to achieve readability and to help people, those poor souls who will read it a few months later. So before we go and look at each step of range actor third, let's look at the example. That is an example. That is trying to test something. And I called it messy test example. Is it easy to understand what it is doing? Can we easily identify what is actually happening in this test? It's probably not that easy. So let's look where a range an actor served in this test. Let's look at that. So if you try to understand and read it carefully, and it took me some time to actually color it and understand it, so you are setting up something and then you are doing some mix of a solution and execution, and then you are setting up something else, then you are doing something more, and you are doing all of that stuff. And I think if we look at that test, we see a several arrange, and sometimes a range actor third is even mixed up all three together. So someone who will read that test will be actually confused. What is this setting up? What is happening? What is going to be? And he is going to spend a lot of time to understand it, and he doesn't want that. So maybe one day he will say, okay, I'll look at that later on. I'll put this test to hold. I'll mute it until sometime. And that actually never works because there is a tendency this test to stay there forever. So that is a violation example. So what we can do to make it better. And let's look at the arrange part. Arrange part is responsible for data set up. And the important thing here is that it is all consolidated at the beginning of the test. There is nothing that you can set up something, then execute something, then set up something more, execute something. We are setting this up at the beginning of the test. And to do that, there are several actually partners that I used to clarify of what you are setting up. Two of them, for example, are object, mother, and fluent interface. Those are partners that we can use, and I'll go into example in a minute just to clarify what is being set up. There are some links on Martin folder, what you can do about them more. Let's look at the example again. So this is another example. And that's quite simple. So let's look at the test on the left. What it is actually doing? That is trying to set up some points. What is this for? That is trying to create some plane. So probably point three is a class that is a point in a three-dimensional coordinate system. And then plane three, some plane in a space. But what is this? What is this? 0, 0, 0, 1, 0, 0, 1, 1, 0. We have to kind of think about it. Why is those digits are here and stuff like that? And that is an arranged part. That is consolidated at the beginning, but we can't actually read what is happening here very well and just at a glance. So let's look at what we can do. What are we actually doing? I will call it we are creating XY plane. We are creating XY plane here by those numbers 0, 0, 0, 1, 0, 0, 1, 1, 1. That is just XY plane, the plane perpendicular or token out of the Z axis. This plane is well-known and people understand what it is. So we can just create a method, a helper method that would just create XY plane and just name it that we are creating XY plane. And if you go to right-hand side, the one who is reading, he is actually understanding that this is XY plane created. Okay, I understand that. And what is the point? Where is this point? We create a point at the axis. So we understand there is a point exactly on the axis that we are creating. And those are matters, those are data that we are going to test. So here is the helper matters and we could call them object mothers. So those are methods that we consolidated to actually make a better readability for this test, with simple example. And we emphasize what data are being set up. And we could actually abstract from the constructor details as well. And we could reuse it in several other classes. And we could create objects that we are familiar with. For example, we can create diagonal plane. And we could create horizontal plane. And we could work with those objects in our test framework, in our communication framework. So we understand what kind of dummy objects we are creating to test. And those helper methods actually serve us as a mother for those objects. There are many methods to create XY plane, but we only use those points. But we don't want to know what kind of points we use just to, we want to abstract on those details. That makes our test readable and our setup readable. And fluent interface, I don't have example, but it's very similar if you have a long constructor, you can actually set up special methods like we's, I have a girl with a balloon or go without a balloon. And then you can build that particular object and return it. So you create sort of a specific number of dummy objects that you are working with in your test framework. And that is very handy. And this is very good for readability. So that was arranged. And there are more buttons for arrange. There is this book, X-Unit Test Partons that Geraldine Marsalis has written. This explains all of the things that we do in detail. It's 900 pages if you want to read it. Let's look at the act. What is actually act? It is code execution. So what we can say about it, we just execute the code. But there are some things that we can think about as a back practices. We could make it visible. Yeah, some people insert asserts in the act part. That shouldn't be there. We should assume everything works here. We are only executing what we are trying to execute. We don't know anything else. And if you want to actually isolate ourselves from the environment, we can use stubbing and mocking. And use the different dummy fake objects for those that our object depends on to make it isolated. Let's look at this example again. What is this act part? It's not quite clear. It's actually here. And it's mixed up with assert. So the one who is reading that, he has to understand, he has to read it like a just wide line, which is not very handy. I think it's just a mixed up. But on the right-hand side, we can see a call. And we can actually understand what is happening here. We actually make a plane and calculate a distance to a point. And we get this number. That's what the one who is reading the task would be interested in. And this is much more readable than we have it here on the left-hand side. So the act is separated here. And it is clearly visible. And those are some other examples of act that are what we do here. We just execute some code, calculate some context, do the work that we need to do. And now we come to assert. And the assert is a verification stage. It could be state verification and it could be behavior verification. And state verification is where we verify the state of our objects, what happened to our objects, what are stated. And behavior verification is a little bit more tricky where we verify actually how our object influenced the other object. Have it been called? That is also a button and we should do one or another or not mix of it. And the best practices here is to have one assert. So, and why we should have one assert? Mostly it is because you will never execute multiple asserts in the test. The test framework wouldn't let you to do that whatever the test framework is. And Microsoft framework or G-unit, they will just stop at the first assertion. That's how they design. And you will never be able to execute all of them. But you want to focus on one thing, what you are doing. And you want to write a clear message for that assertion, what actually happened. This is only achievable with one assert. And if, for example, some people just try to write an assertion or let the asserts that assert in different properties of one object, that is easier to do just comparing objects. And just one assert should be a rule, I think, for everybody. Let's look at the example here. So here we have two asserts. So that is testing completely different things. So we actually can divide this test in two different tests. But on the right-hand side, this is clearly visible what is being tested on the assertion. So that was arranged after assert. A pattern that if you use it, you actually almost achieve readability for free. Because I used to write the messy test myself. And when I was trying to understand what this test is doing and someone else, it's just what are concerns being tested? What are data being set up? And what is being verified? And once you reply to yourself on those questions, that is easy to make a decision what actually happened in a code. But what else we can think about when we think about how the test should look and what it should do? Let's look, let's talk about the test scope. The scope of the test. It should think, it should test one thing. It should focus on one thing to test, not on multiple things, on multiple aspects, maybe the problem with multiple properties of an object. An indication that we are testing several things would be that we are having some logic in the test. If you're having if you have a statement, if you have switch statement, if you have for loop, we are, it's very high probability that we are testing different things in one test. That we are trying to test multiple aspects of that. And of course, if you have multiple asserts, we are almost certainly testing different things. If you are wondering about that, you can try to name your test just by looking at all the asserts and just write the sentence that this test is doing, this distance, you get a long sentence if you have multiple asserts. It will be several things. So that's a good practice to have one thing and it will help people to understand it later how actually test is, how test is working, what test is testing, the size of it. What is a good test size? That is a religious issue again. So I believe in five lines. Some people believe in maybe another amount. But generally, I think the perfect test is a five line, maybe six lines, I don't know the exact number, but again, it's a religious issue. I think six, I can imagine the test that would have 16 lines and would still test in one thing. I have never seen test 25 lines testing one thing ever. Anyone seen that? Okay. And that was testing different things. Yes. So it should be small and I believe this function should be small, but the test should be small to test one thing, to focus on one concern. And let's look at the good test naming. That is very interesting issue. So we have that concept, don't repeat yourself in coding. That means we are, we want to remove duplication. Duplication is a maintenance for us. We don't want that in a code. We don't want to repeat ourselves. But from the other side, we have the idea of writing everything in descriptive and meaningful phrases which is coming to us from the main specific languages. And there is kind of a balance between it because the more we write in descriptive and meaningful phrases, we could actually violate that don't repeat yourself principle. And I think test lives in its own little world. It's isolated. It shouldn't have any knowledge from the outside world. And because of that, sometimes trying to remove duplication, we are actually sacrificing the business logic and the meaning of the test. So in testing, being descriptive and meaningful is more important than removing duplication. Sometimes that's fine to have some sort of duplication in a test than actually trying to force people to read some other code. I was actually struggling to get an example for that. I actually pulled out some code I written a while back and that was a good explanation. But it was so difficult to understand in such a short time. So I just removed the example. But think about that. And being descriptive is much more important in tests because they live in their own little world. And that's fine if you repeat sometimes. It's a test code at the end. It's never executed in production. Good test coverage. That is my favorite slide. How much we should test? We talk about coverage all the time. Management uses it as a metric to detect quality. Is it really true? So I will tell you a story. Three programmers asked the gray master of what coverage should I aim for. The first one asked and the great master said, don't worry about that. Just go and try some good tests. And the second one asked about that. What coverage should I aim? The great master said, how many grain rice should I put in that pot? That depends on the second programmer. It depends because how many people you have to feed. What other food you're serving. What actually you are, what is this, how hungry they are. And those things are how I can tell you. That's what he said. And the master said, yeah, that's right. And that programmer left. And the third one came. What coverage should I aim for? 80% at no less. Said the great master. And then the followers came to the great master and asked him, we heard that you did the three different answers to the same questions to different people why it is. And he said, the first one was new to unit testing. So he had to just focus on writing some good tests. And the second one was experienced. So he understood and realized with my answer that there is no simple answer to that. The third one just wanted the simple answer to the question that doesn't have any simple answers. So he got it. 80% at no less. And in coverage, it is very important that we should balance. Does anyone has 100% coverage? We are saying that we just have to do it. Do we have to blindly follow the coverage? Do we have to blindly aim 100% or 80% or whatever number, simple number? I can't believe that it could be some tests, some projects with 100% coverage. They still have bugs in that. We have to focus on something else and just test coverage. We have to focus what we are testing and where we are testing and what we are testing. And we can talk about also coverage duplication here. And this more brings us to the questions when we are testing. So we have isolated tests that test local things. And we have tests that are integration tests, results, stubborn and morking. The isolated tests are unit tests referred to. So we are focusing on one class, one object. It's very local. And then we have end-to-end tests. And we have to have end-to-end tests. Because there is no way that we can test the system. And we don't have end-to-end tests. However, anyone had a situation on one change in the code cost hundreds of tests? Yeah, a few people. That happens because we have some coverage duplication in our tests. And the good practice that we found here is just to run the isolated tests, local tests before and make sure that our local systems work fine. And then run our integration tests, end-to-end tests afterwards. And there should be the way less integration tests than the local tests. It's a normal situation. And then when we run the integration test, there's just a few. They test the actually components communication. They don't test the local things. And we call that smoke test that we run. And we run them first. And we make sure everything is fine. And then we can allow execution of the integration test. So that is a very good practice that we found for ourselves. Good tests fail. And they fail. And they want them to fail. But they want them to fail just for one reason. One good reason is that the functionality code change. We don't want tests to fail randomly due to some isolation problems. We don't want them to fail because there are bugs in them. We want them to fail just because we change the code and it fails. And there is a dependency on that. But we don't want random failures. But they appear that random failures and isolation problems and test bugs, they're quite easy to fix. But to find the tests that fail where we want it is quite difficult. We want to write a test that will save us some time in future but we cannot predict the future very well. How we can predict the failure information. Where should I test? Each test is associated with a cost. I have to spend time writing it. People have to spend time executing that. And we have to think carefully how we are going to write the test. Where? And here is Ken back. And he said, I should test where I'm likely to make a mistake. Not the things that always work. That's fine. But how do we know where we'll make a mistake? That is probabilistic investment. We don't know. That's a bet. We just... But we can think about it maybe. And he's saying, for example, I tend to make mistake around conditionals so I am trying to write a test around conditionals because I know I'm making mistakes there. But what else we can think about when we talk about the good test value? Can we predict the failure information? We normally don't think about it very much. But if you have a test and you have a code and everything works, can we think about when it will fail? We can try to comment out production fault and watch it fails. We can try to change production code in a way that we think it will evolve in the future and watch it fails. And that is a very good exercise in determining when the test will fail. And we normally never do that exercise when we write the test. We can look at the test as an API, as a documentation to our code. And we... This is a very good practice as well because we have... We are seeing what we are trying to achieve. And here we can predict maybe how it will evolve, how it will change. So we want the test when we feel like they are valuable, but not before. And that is a thing that what we can think about when we write the test. Which test to write and which test not to write. So what is a good test? So the question that we ask ourselves when we write a good test is how easy I can understand it. And more important is not like I can understand it and how easy someone else would be able to understand it. It is very, very likely that you will not be reading that test. You are doing it for your peers. You are doing it for the one... The poorest soul who will come to the project a few months later and say, ah. And then when it will fail, will it fail? What kind of mistake I should make so it fails? And where I can predict the bug? And we can think about coverage, of course. And we should think about coverage. But we should understand that this is not a simple answer to the coverage question. So we figure that the good test when we look from the maintainability point of view, it is more important to think about how it looks and where it tests. And then thinking about when it will be run and what it is actually testing and focusing on coverage. That was it. Thank you very much.
have a problem maintaining your tests and trying that is easy to maintain. That it's easy to work developer. I developed software for about 17 Center. Which is part of the biggest company that of code. This is a successful product that is component of that success is automated testing. point. We have gone all the long way from total work for us. TDD, unit testing, it became a have adapted to unit testing. We are all committed to spend some time to write tests for activity. So we recognize the value of it. We share these days. Like 10 years ago, it would be They spend a lot of time bug fixing. They spend a basis. So, and we have done a great job in using brings us to maintenance problems. If you cannot readability is bad, then we spend a lot of time team. And that iterative development actually fixing the unit test. And that is the situation automated test suite. That would be fine. But metric of the quality. But generally should we think about coverage or should we think about our tests? Do we run them all in the same time? someone needs to go and understand why it fails. a lot of time actually understanding and because of that and then fix it and then go on. few people. And that has been blocked I think in going to spend a lot of time to understand it, clarify what is being set up. There are some those are matters, those are data that we are more readable than we have it here on the examples of act that are what we do here. We just assert. And the assert is a verification stage. only achievable with one assert. And if, for example, some people just try to write an talk about the test scope. The scope of the test. just write the sentence that this test is doing, actually test is, how test is working, what test size? That is a religious issue again. So I should be small, but the test should be small to important in tests because they live in their own Management uses it as a metric to detect quality. some tests, some projects with 100% coverage. run the integration test, there's just a few. They test the actually components communication. is that the functionality code change. We don't want tests to fail randomly due to some isolation because we change the code and it fails. And they're quite easy to fix. But to find the tests that fail where we want it is quite difficult. We can think about when we talk about the good test you have a test and you have a code and well because we have... We are seeing what we are how it will evolve, how it will change. So we valuable, but not before. And that is a thing important is not like I can understand it and how poorest soul who will come to the project a few of view, it is more important to think about how about when it will be run and what it is actually testing and focusing on coverage. That was it.