However, when I looked at actual examples, they looked sophisticated and obscure, the opposite of what I feel unit tests should be: obvious and simple, simplistic to the point of stupidity. I couldn't figure out at a glance what the expected behavior was, what was being tested and what was environment.
So I never used mocks in practice, meaning my opinions could not go beyond being superficial. Fortunately, I was given the task of porting a fairly large Objective-C project to OS X (yes, you read that right: "to OS X" ), and it was heavily mock-tested.
As far as I could tell, most of the vague premonitions I had about mock testing were borne out in that project: obscure mock tests, mock tests that didn't actually test anything except their own expectations and mock tests that were deeply coupled to implementation details.
Again, though, that could just be my misunderstandings, certainly people for whom I have a great deal of respect advocate for mock tests, but I was heartened when I heard in the recent DHH/Fowler/Beck TDD death-matches friendly conversations that neither Kent nor Martin are great fans of mocking, and certainly not of deeply nested mocks.
However, it was DHH's comments that finally made me realize that what really bothered was something more subtle, and much more pervasive. The talk is about "mocking the database", or mocking some other component. While not proof positive, this kind of mocking seems indicative of not letting the tests drive the design towards simplicity, because the design is already set in stone.
As a result, you're going to have constant pain, because the tests will continuously try to drive you towards simplifying your design, which you resist by putting in mocks.
Instead of putting in mocks of presumed components, let the tests tell you what counterparts they want. Then build those counterparts, again in simplest way possible. You will likely discover that a lot of your assumptions about the required environment for your application turn out not to be true.
For example, when building SportStats v2 at the BBC we thought we needed a database for persistence. But we didn't build it in until we needed it, and we didn't mock it out either. We waited until the code told us that we now needed a database.
It never did.
So we discovered that our problem was simpler than we had originally thought, and therefore our architecture could be as well. Mocking eliminates that feedback.
So don't mock. Because it's impolite to not listen to what your code is trying to tell you.
18 comments:
> We waited until the code told us that we now needed a database. It never did.
This sounds really interesting. Could you elaborate a bit on this?
I don't understand the database example. If I design outside-in with mocks, then I will either reach the point where I'll need some collaborator that will provide me some information or not. In the former case, I mock that role without thinking about the persistence mechanism and other implementation details. At this point, I'm still not commited to use a database.
Besides the benefits of speed when running tests, can anyone think of any benefits to using mocks over regular objects?
> Besides the benefits of speed when running tests, can anyone think of any benefits to using mocks over regular objects?
It is a design tool. There is a book about this, called GOOS.
> Besides the benefits of speed when running tests, can anyone think of any benefits to using mocks over regular objects?
For example you can mock a REST api, to avoid doing real calls, either for speed or to avoid doing unwanted actions.
> We waited until the code told us that we now needed a database. It never did.
Sounds very similar to Uncle Bob's story when they were creating FitNesse.
Mocking is useful in cases such as testing a module that depends on a third party API. You mock the API so that you can test only the module itself, and not the external API it depends on. You can test the external API itself separately. This way if your module tests fail, you know it is a problem with the module itself and not with the external API dependency.
> Sounds very similar to Uncle Bob's story when they were creating FitNesse.
They end up using a file system store instead of a RDBS, if I recall correctly. At first they created a stub, later some in-memory storage.
This seems to be entirely orthogonal to mocking to me. In fact they used stubs too.
You don't mock at all and still getting a decoupled design. Its very interesting! How can you get a decoupled design without mock? Do you let your class under test "touch" another domain classes?
About avoid dependencies until your code require them: when you must add them, how your tests keeps decoupled from it ?
this is something I realized as well. but it is very hard to explain, and I actually never managed to convey what this means completely. in this article I can see the same hardship in exposing this concept.
I have this mental model of the code: there is an evident input and output (parameter and return values) then there is an external state (database data, third party api call) and an internal state (local variables and members)
input and output are dead easy to test if there are no operation done on external and internal state.
operation on internal state are as quell easy to prepare and verify exposing the right method on the object to be tested.
that leaves us with the external state. this is where mocking actually helps but: I prefer decoupling instead of mocking. I don't query the database, I create a dao that is passed as local state - then the dao can be made to return fixed data on testing. easy to see, easy to follow, easy to debug.
now, you have to test the dao, and here you are stuck with moking or integration test, however, this pushing to delegation of external state make the logic code dead easy to test.
as external state routines take exclusively the form of load and store, even if they are painful to test, they are easy as well.
In large projects with many layers mocks allow you to not have to construct complicated data scenarios.
@Cory Gross
If the third party lib fails testing then deal with that first.
If the lib didn't fail testing but would have caused some other component to fail testing surely you would want to know about that even if it is harder to track down the bug than if it had been found in the tests for the library.
> operation on internal state are as quell easy to prepare and verify exposing the right method on the object to be tested.
Please don't. This is almost always wrong when unit testing in an OO language. You should not expose internal state in order to test it. This would degrade your object's public interface and would make your design worse. This is actually one case where test doubles work well.
Instead of putting in mocks of presumed components, let the tests tell you what counterparts they want. Then build those counterparts, again in simplest way possible.
Consider that you need to integrate with third party systems. In fact, the name of your application is about integration with a proprietary service, software or platform... Sometimes not even proprietary, but the one that cannot be run on your machine.
Do you build Google AdWords, because you need to integrate with it? Or do you mock it out? Do you try to reverse engineer SAS? Rewrite Hadoop? Rewrite SAP? and the list goes on... There are thousands of systems that you can't just go and use the actual object/API/whatever.
I don't like mocking as well. I don't advocate or do it for anything other than external(uncontrollable) component. But then again... I don't do excessive unit testing, preferring integration testing.
It is also a bad habit of writing unit tests before you start coding. As those will either make you adhere to a certain design/architecture(that is fixed beforehand) or be a nightmare to maintain. Mocking components is just the extension of that issue.
@Anoymous: Yes, I was quite amused when I saw Uncle Bob's Fitnesse story when I saw his Ruby Midwest 2011 keynote: Architecture, the Lost Years. While I don't agree with everything he says, there is a lot of good stuff there.
And no, they did not stub out (which is different from mocking) the storage. They just didn't need storage at all for al long time. When they started needing storage, they also didn't stub it out, they just implemented the simplest thing that could possibly work, which happened to be a file store.
So they discovered a lot about their problem space, particularly that it was simpler than they had suspected, just as we did. Simplicity is good, and very often the drivers of unnecessary complexity are we the programmers, mostly due to faulty assumptions.
I think more people would make this discovery if they let their tests guide them in their architectural discovery. Mind you, the tests do not induce an architecture, and certainly not a good one. You do need to design it yourself, and it is almost always helpful to think about that architecture beforehand. You just want to adapt that design to the discoveries that you make along the way, and tests are a good mechanism for enhancing the feedback you get.
Mocks in many cases let you shut out that feedback making it easier to not discover anything interesting about your architecture.
One reason to mock objects is so you don't have to setup supporting data needed for a test case, you just embed the data in the mock objects. Many call those stubs instead of mocks, which I find make for more understandable tests. For example, testing a credit card processing back charge would first require you to run something to create the charge in the first place.
Mocking is pretty much a must if you do unit testing. You can only have coarse grained tests without it (assembling most / all of your app for your tests).
Eg. all your tests may blow up because of a small issue in a commonly used dependency somewhere down the chain. You may get away with that on a small/simple project, but can be very time consuming on a more complex one.
When it comes to integrations / infrastructure you need to do some stubbing in any case - unless you go even further and rely on integration tests only, which further limits the set of projects that can allow to do this.
Hi Marcel. I am wondering what you think of the following paper where Steve Freeman, Nat Pryce, et al. see mocking as a tool for interface discovery.
http://www.jmock.org/oopsla2004.pdf
Perhaps you can shed some insight on their mocked presumptions in the context of roles and expected interactions.
Another thing I'm curious about is DHH's comment on "mocking the database". DHH assumes that he needs to mock a database, but maybe what he really needed to mock was a component playing the role of a data store, which could be a file store, data base, web service, etc. After mocking the role of a data store, we could implement it using a file store to start.
As someone trying to grok both schools of TDD, I'm very curious to hear your thoughts!
Post a Comment