Mocks considered harmful?

You said:

in the original paper on mock objects, they complained a lot about external systems / libs out of our control. is there any other use they suggest for mocks besides this?

ChatGPT said:

In the original paper on mock objects, “Endo-Testing: Unit Testing with Mock Objects” by Tim Mackinnon, Steve Freeman, and Philip Craig, the authors focus primarily on using mock objects to deal with external systems or libraries that are outside of the developer’s control. This was one of the major motivations behind their approach: external systems are often difficult to interact with in unit tests because they may be slow, non-deterministic, unavailable, or have side effects.

However, they also suggest other uses for mock objects beyond just isolating code from external dependencies:

1. Design Feedback: Mocks are used to drive the design of software by enforcing communication between objects. When you create mock objects, you specify the interactions (i.e., method calls) that are expected. This can help you design cleaner, more decoupled interfaces since you are forced to think carefully about how objects communicate. This process, sometimes referred to as Test-Driven Development (TDD) with Mocks, helps shape the API of a system by focusing on the roles and responsibilities of objects.
2. Unit Testing in Isolation: Beyond external systems, mock objects are used to isolate the unit under test from its collaborators. The authors emphasize the importance of testing a class in isolation to ensure that tests focus only on the behavior of the unit itself, rather than on the behavior of its dependencies. This makes tests more reliable and faster, since they don’t depend on other parts of the system.
3. Interaction-Based Testing: Mock objects allow for interaction-based testing, where the test focuses on the interactions between objects (e.g., method calls, arguments passed). This is different from state-based testing, which checks the final state of the system. With mocks, you can verify that certain methods were called with the expected parameters, ensuring that the tested object communicates with its dependencies correctly. This is particularly useful when the outcome of a test depends on complex collaborations between objects.

So, while the primary motivation was dealing with external systems, the paper also discusses the value of mocks in shaping object-oriented design, enabling isolated unit tests, and verifying object interactions.

You said:

these secondary topics sound dangerous, method calls and arguments between closely related collaborating objects is an implementation detail and should not leak into tests (even unit). some of the inter-class refactorings (move method, extract class) will be blocked too.

ChatGPT said:

You’re absolutely right to raise concerns about the potential dangers of relying too heavily on testing method calls and arguments between closely related objects. This can indeed lead to brittle tests that break during refactoring, even when the behavior of the system remains unchanged. Let’s break down the key issues:

1. Testing Implementation Details (Leaky Abstractions)

• Problem: When tests focus on verifying specific method calls and arguments, they can become tightly coupled to the internal implementation of the code. This turns the tests into a reflection of how something is done rather than what the code should accomplish.
• Risk: If the internal implementation changes (e.g., refactoring a method to be called in a different way or changing the flow between collaborators), the tests might fail even though the overall behavior of the system remains correct.
• Example: Imagine two classes, Order and PaymentProcessor, where the test verifies that Order calls PaymentProcessor.process(payment). If the logic changes and Order delegates to another object to handle the payment processing (e.g., introducing a PaymentHandler), the test would fail even if the overall functionality of processing a payment still works fine.

2. Hindering Refactorings

• Problem: Refactorings like Move Method or Extract Class aim to improve the internal design of the system without changing its observable behavior. However, if tests are based on the interactions between objects (rather than their externally observable behavior), refactoring the code will break the tests.
• Risk: Tests that rely on mocks for internal interactions make it harder to refactor the code, as the refactoring might require changing not only the code but also the tests, even though the behavior hasn’t changed. This undermines the agility of the development process.
• Example: If you want to move a method from one class to another, but the test asserts that a particular object receives a call to that method, you’ll need to update the tests along with the code, making the refactoring more tedious.

3. Over-reliance on Interaction-Based Testing

• Problem: The paper’s approach of interaction-based testing emphasizes verifying how objects collaborate, i.e., checking that methods are called with the right arguments. However, focusing on interactions rather than on the final state or behavior can make tests brittle and harder to maintain.
• Risk: Instead of writing tests that focus on what the system should do (its observable behavior), you’re testing how it accomplishes it. The implementation details become part of the test, which may block future changes and lead to tests that are more likely to break without real bugs being introduced.
• Better Approach: State-based testing (or behavior verification) is usually preferred in unit testing because it verifies that the system’s behavior meets its specification without dictating how that behavior should be achieved.

4. Collaborator Tests vs. Contract Tests

• Collaborator Tests (Interaction-based): These tests verify that specific collaborators are called in specific ways. This can easily lead to brittleness.
• Contract Tests (State-based): Instead of focusing on how the object interacts with its dependencies, the focus is on the contract or the outcomes of the interactions. This is more resilient to internal changes as long as the contract remains valid.

Alternative Recommendations

1. Favor State-based Testing:

• Focus on the outcome rather than the process. Instead of checking that a specific method was called, check that the object under test behaves correctly (i.e., returns the correct values or modifies state as expected).

1. Use Mocks Sparingly:

• Only mock external systems or boundaries where it’s hard or impractical to test the real implementation (e.g., databases, third-party APIs).
• For internal collaborators, prefer using real objects when possible, or rely on stubs rather than mocks to provide predictable behavior without over-specifying interactions.

1. Mock Only Stable Collaborations:

• If you do need to mock interactions, mock interfaces or stable APIs that represent well-defined contracts rather than fragile internal method calls. This ensures that the mocked behavior is less likely to change during refactoring.

1. Refactor Tests with Code:

• If your code evolves to improve its internal structure, consider evolving your tests in a way that maintains flexibility. Avoid locking tests to specific method calls or sequences unless they are core to the business logic.

In summary, while the mock objects approach can be useful, the pitfalls you’re describing are real, particularly when tests focus too much on method calls and arguments. By emphasizing behavior and state over interaction details, you can keep tests more robust and flexible, allowing for refactorings without unnecessary test breakage.

You said:

there is this GOOS book, where the unit tests were too mock heavy. does each class map exactly to one layer of the system (mocking between layers would be fine by me)? or there are classes belonging to the same layer, thus they also mocked closely related collaborating objects too?

ChatGPT said:

You’re referring to “Growing Object-Oriented Software, Guided by Tests” (GOOS) by Steve Freeman and Nat Pryce. This book is influential in the Test-Driven Development (TDD) community, but it is also known for its fairly heavy use of mocks, especially in unit tests.

GOOS Approach to Layering and Mocks

In GOOS, the approach is based on interaction-based testing, where classes are often tested in isolation by mocking their collaborators. The book emphasizes that TDD is not just about verifying behavior but also about driving good design, which is why mock-heavy tests are sometimes used to shape the interactions between objects. However, this does lead to questions about layering and whether the use of mocks crosses boundaries that should ideally be kept stable.

Do Classes Map One-to-One with Layers?

The answer is: not always. In GOOS, the classes do not strictly map to one specific layer of the system. The focus is more on collaborations between objects and how objects interact, regardless of their layer. As a result, it is common for the tests to mock objects that belong to the same layer, which can lead to some of the concerns you’re mentioning, where closely related objects are mocked out even though they are part of the same logical layer of the system.

Layers in GOOS:

• Layers like Domain, Application, Infrastructure:
• GOOS does acknowledge common architectural layers, and mocking would be more acceptable across layers (e.g., mocking the persistence layer while testing domain logic).
• If a class from the domain layer interacts with a class in the infrastructure layer (such as a database or messaging system), then mocking would generally be fine here since it isolates the domain logic from external systems.
• Mocking within the Same Layer:
• However, GOOS often mocks objects within the same layer, particularly within the domain layer. This could involve mocking objects that represent domain logic collaborators. For example, if two domain objects are supposed to work together to achieve a certain behavior, one might be mocked during the unit test of the other.

Mocking Closely Related Collaborators in the Same Layer

GOOS’s reliance on mocking can lead to issues, especially when classes within the same layer are mocked:

1. Mocking closely related domain objects: If two domain objects work together and one is mocked during testing, this leads to tight coupling between the tests and the interactions (i.e., how objects communicate). This can introduce fragility because internal refactorings within the same layer (such as splitting a class or changing communication patterns) can break the tests, even though the overall functionality remains the same.
2. Risk of Brittle Tests: As you pointed out, mocking within the same layer might block certain refactorings. For example, if the design evolves to distribute responsibilities differently (e.g., using Move Method or Extract Class refactorings), tests might break even though the behavior remains correct.
3. Layering Over-Emphasized at Object Level: While the system as a whole might be layered (e.g., UI, application, domain, infrastructure), in GOOS, the unit tests often focus on small, collaborating objects within the same layer. These interactions are then mocked, leading to tests that verify how objects collaborate rather than their broader behavior across layers.

When is Mocking Appropriate Between Layers?

Mocking between layers would generally be more acceptable, as this allows you to test the higher-level behavior without involving lower-level concerns (such as mocking out the infrastructure or services that deal with databases, messaging systems, or external APIs). This way, the layers remain decoupled, and the tests focus on the functionality of each layer in isolation.

For example:

• Mocking the database layer while testing domain logic is often a good practice because it allows you to test the domain logic in isolation from the persistence concerns.
• Mocking the UI layer while testing application logic makes sense because you don’t want your tests to depend on UI interactions.

Issues with Mocking Collaborators Within the Same Layer

However, mocking objects within the same layer (particularly in the domain or application layer) can lead to:

• Fragile tests: Tests become tied to implementation details like method calls between domain objects, leading to failures when those interactions change.
• Blocked refactorings: Refactoring becomes harder because changes in how objects interact within a layer might break tests unnecessarily.
• Focus on interactions, not behavior: Tests verify whether one object calls another in a specific way rather than ensuring that the system behaves correctly, leading to over-specification.

Alternatives to Mock-Heavy Testing

1. Use Real Collaborators for Same-Layer Tests: Instead of mocking objects in the same layer, use real instances of the objects and focus on testing the outcome (state-based testing) rather than interactions. This leads to more robust tests that are less sensitive to internal changes.
2. Test Larger Units of Behavior: Instead of focusing too narrowly on individual classes, consider testing larger units of behavior that span multiple objects. For example, you can test a service or aggregate object that encapsulates the behavior of multiple domain objects without mocking them out individually.
3. Mock Only Cross-Layer Interactions: Save mocks for interactions between different layers (e.g., domain to infrastructure, or UI to application logic). This way, the system’s internal design can evolve freely without breaking the tests.

Conclusion

In GOOS, classes often don’t map strictly to layers, and mocks are sometimes used even between closely related objects within the same layer. This can lead to brittle tests that break during refactorings and focus too much on interactions instead of behavior. While the GOOS approach emphasizes the use of mocks to drive design, it’s important to be cautious about overusing them, especially within the same layer, and to consider more behavior-focused testing strategies to maintain test flexibility and support refactoring.