pleroma/docs/development/mox_testing.md

Using Mox for Testing in Pleroma

Introduction

This guide explains how to use Mox for testing in Pleroma and how to migrate existing tests from Mock/meck to Mox. Mox is a library for defining concurrent mocks in Elixir that offers several key advantages:

• Async-safe testing: Mox supports concurrent testing with async: true
• Explicit contract through behaviors: Enforces implementation of behavior callbacks
• No module redefinition: Avoids runtime issues caused by redefining modules
• Expectations scoped to the current process: Prevents test state from leaking between tests

Why Migrate from Mock/meck to Mox?

Problems with Mock/meck

1. Not async-safe: Tests using Mock/meck cannot safely run with async: true, which slows down the test suite
2. Global state: Mocked functions are global, leading to potential cross-test contamination
3. No explicit contract: No guarantee that mocked functions match the actual implementation
4. Module redefinition: Can lead to hard-to-debug runtime issues

Benefits of Mox

1. Async-safe testing: Tests can run concurrently with async: true, significantly speeding up the test suite
2. Process isolation: Expectations are set per process, preventing leakage between tests
3. Explicit contracts via behaviors: Ensures mocks implement all required functions
4. Compile-time checks: Prevents mocking non-existent functions
5. No module redefinition: Mocks are defined at compile time, not runtime

Existing Mox Setup in Pleroma

Pleroma already has a basic Mox setup in the Pleroma.DataCase module, which handles some common mocking scenarios automatically. Here's what's included:

Default Mox Configuration

The setup function in DataCase does the following:

1. Sets up Mox for either async or non-async tests
2. Verifies all mock expectations on test exit
3. Stubs common dependencies with their real implementations

# From test/support/data_case.ex
setup tags do
  setup_multi_process_mode(tags)
  setup_streamer(tags)
  stub_pipeline()

  Mox.verify_on_exit!()

  :ok
end

Async vs. Non-Async Test Setup

Pleroma configures Mox differently depending on whether your test is async or not:

def setup_multi_process_mode(tags) do
  :ok = Ecto.Adapters.SQL.Sandbox.checkout(Pleroma.Repo)

  if tags[:async] do
    # For async tests, use process-specific mocks and stub CachexMock with NullCache
    Mox.stub_with(Pleroma.CachexMock, Pleroma.NullCache)
    Mox.set_mox_private()
  else
    # For non-async tests, use global mocks and stub CachexMock with CachexProxy
    Ecto.Adapters.SQL.Sandbox.mode(Pleroma.Repo, {:shared, self()})

    Mox.set_mox_global()
    Mox.stub_with(Pleroma.CachexMock, Pleroma.CachexProxy)
    clear_cachex()
  end

  :ok
end

Default Pipeline Stubs

Pleroma automatically stubs several core components with their real implementations:

def stub_pipeline do
  Mox.stub_with(Pleroma.Web.ActivityPub.SideEffectsMock, Pleroma.Web.ActivityPub.SideEffects)
  Mox.stub_with(Pleroma.Web.ActivityPub.ObjectValidatorMock, Pleroma.Web.ActivityPub.ObjectValidator)
  Mox.stub_with(Pleroma.Web.ActivityPub.MRFMock, Pleroma.Web.ActivityPub.MRF)
  Mox.stub_with(Pleroma.Web.ActivityPub.ActivityPubMock, Pleroma.Web.ActivityPub.ActivityPub)
  Mox.stub_with(Pleroma.Web.FederatorMock, Pleroma.Web.Federator)
  Mox.stub_with(Pleroma.ConfigMock, Pleroma.Config)
  Mox.stub_with(Pleroma.StaticStubbedConfigMock, Pleroma.Test.StaticConfig)
  Mox.stub_with(Pleroma.StubbedHTTPSignaturesMock, Pleroma.Test.HTTPSignaturesProxy)
end

This means that by default, these mocks will behave like their real implementations unless you explicitly override them with expectations in your tests.

Configuration in Async Tests

Understanding clear_config Limitations

The clear_config helper is commonly used in Pleroma tests to modify configuration for specific tests. However, it's important to understand that clear_config is not async-safe and should not be used in tests with async: true.

Here's why:

# Implementation of clear_config in test/support/helpers.ex
defmacro clear_config(config_path, temp_setting) do
  quote do
    clear_config(unquote(config_path)) do
      Config.put(unquote(config_path), unquote(temp_setting))
    end
  end
end

defmacro clear_config(config_path, do: yield) do
  quote do
    initial_setting = Config.fetch(unquote(config_path))

    unquote(yield)

    on_exit(fn ->
      case initial_setting do
        :error ->
          Config.delete(unquote(config_path))

        {:ok, value} ->
          Config.put(unquote(config_path), value)
      end
    end)

    :ok
  end
end

The issue is that clear_config:

1. Modifies the global application environment
2. Uses on_exit to restore the original value after the test
3. Can lead to race conditions when multiple async tests modify the same configuration

Async-Safe Configuration Approaches

When writing async tests with Mox, use these approaches instead of clear_config:

1. Dependency Injection with Module Attributes:

   # In your module
   @config_impl Application.compile_env(:pleroma, [__MODULE__, :config_impl], Pleroma.Config)
   
   def some_function do
     value = @config_impl.get([:some, :config])
     # ...
   end
   

2. Mock the Config Module:

   # In your test
   Pleroma.ConfigMock
   |> expect(:get, fn [:some, :config] -> "test_value" end)
   

3. Use Test-Specific Implementations:

   # Define a test-specific implementation
   defmodule TestConfig do
     def get([:some, :config]), do: "test_value"
     def get(_), do: nil
   end
   
   # In your test
   Mox.stub_with(Pleroma.ConfigMock, TestConfig)
   

4. Pass Configuration as Arguments:

   # Refactor functions to accept configuration as arguments
   def some_function(config \\ nil) do
     config = config || Pleroma.Config.get([:some, :config])
     # ...
   end
   
   # In your test
   some_function("test_value")
   

By using these approaches, you can safely run tests with async: true without worrying about configuration conflicts.

Setting Up Mox in Pleroma

Step 1: Define a Behavior

Start by defining a behavior for the module you want to mock. This specifies the contract that both the real implementation and mocks must follow.

# In your implementation module (e.g., lib/pleroma/uploaders/s3.ex)
defmodule Pleroma.Uploaders.S3.ExAwsAPI do
  @callback request(op :: ExAws.Operation.t()) :: {:ok, ExAws.Operation.t()} | {:error, term()}
end

Step 2: Make Your Implementation Configurable

Modify your module to use a configurable implementation. This allows for dependency injection and easier testing.

# In your implementation module
@ex_aws_impl Application.compile_env(:pleroma, [__MODULE__, :ex_aws_impl], ExAws)
@config_impl Application.compile_env(:pleroma, [__MODULE__, :config_impl], Pleroma.Config)

def put_file(%Pleroma.Upload{} = upload) do
  # Use @ex_aws_impl instead of ExAws directly
  case @ex_aws_impl.request(op) do
    {:ok, _} ->
      {:ok, {:file, s3_name}}

    error ->
      Logger.error("#{__MODULE__}: #{inspect(error)}")
      error
  end
end

Step 3: Define the Mock in test/support/mocks.ex

Add your mock definition in the central mocks file:

# In test/support/mocks.ex
Mox.defmock(Pleroma.Uploaders.S3.ExAwsMock, for: Pleroma.Uploaders.S3.ExAwsAPI)

Step 4: Configure the Mock in Test Environment

In your test configuration (e.g., config/test.exs), specify which mock implementation to use:

config :pleroma, Pleroma.Uploaders.S3, ex_aws_impl: Pleroma.Uploaders.S3.ExAwsMock
config :pleroma, Pleroma.Uploaders.S3, config_impl: Pleroma.UnstubbedConfigMock

Writing Tests with Mox

Setting Up Your Test

defmodule Pleroma.Uploaders.S3Test do
  use Pleroma.DataCase, async: true  # Note: async: true is now possible!

  alias Pleroma.Uploaders.S3
  alias Pleroma.Uploaders.S3.ExAwsMock
  alias Pleroma.UnstubbedConfigMock, as: ConfigMock

  import Mox  # Import Mox functions
  
  # Note: verify_on_exit! is already called in DataCase setup
  # so you don't need to add it explicitly in your test module
end

Setting Expectations with Mox

Mox uses an explicit expectation system. Here's how to use it:

# Basic expectation for a function call
ExAwsMock
|> expect(:request, fn _req -> {:ok, %{status_code: 200}} end)

# Expectation for multiple calls with same response
ExAwsMock
|> expect(:request, 3, fn _req -> {:ok, %{status_code: 200}} end)

# Expectation with specific arguments
ExAwsMock
|> expect(:request, fn %{bucket: "test_bucket"} -> {:ok, %{status_code: 200}} end)

# Complex configuration mocking
ConfigMock
|> expect(:get, fn key ->
  [
    {Pleroma.Upload, [uploader: Pleroma.Uploaders.S3, base_url: "https://s3.amazonaws.com"]},
    {Pleroma.Uploaders.S3, [bucket: "test_bucket"]}
  ]
  |> get_in(key)
end)

Understanding Mox Modes in Pleroma

Pleroma's DataCase automatically configures Mox differently based on whether your test is async or not:

1. Async tests (async: true):

   • Uses Mox.set_mox_private() - expectations are scoped to the current process
   • Stubs Pleroma.CachexMock with Pleroma.NullCache
   • Each test process has its own isolated mock expectations

2. Non-async tests (async: false):

   • Uses Mox.set_mox_global() - expectations are shared across processes
   • Stubs Pleroma.CachexMock with Pleroma.CachexProxy
   • Mock expectations can be set in one process and called from another

Choose the appropriate mode based on your test requirements. For most tests, async mode is preferred for better performance.

Migrating from Mock/meck to Mox

Here's a step-by-step guide for migrating existing tests from Mock/meck to Mox:

1. Identify the Module to Mock

Look for with_mock or test_with_mock calls in your tests:

# Old approach with Mock
with_mock ExAws, request: fn _ -> {:ok, :ok} end do
  assert S3.put_file(file_upload) == {:ok, {:file, "test_folder/image-tet.jpg"}}
end

2. Define a Behavior for the Module

Create a behavior that defines the functions you want to mock:

defmodule Pleroma.Uploaders.S3.ExAwsAPI do
  @callback request(op :: ExAws.Operation.t()) :: {:ok, ExAws.Operation.t()} | {:error, term()}
end

3. Update Your Implementation to Use a Configurable Dependency

# Old
def put_file(%Pleroma.Upload{} = upload) do
  case ExAws.request(op) do
    # ...
  end
end

# New
@ex_aws_impl Application.compile_env(:pleroma, [__MODULE__, :ex_aws_impl], ExAws)

def put_file(%Pleroma.Upload{} = upload) do
  case @ex_aws_impl.request(op) do
    # ...
  end
end

4. Define the Mock in mocks.ex

Mox.defmock(Pleroma.Uploaders.S3.ExAwsMock, for: Pleroma.Uploaders.S3.ExAwsAPI)

5. Configure the Test Environment

config :pleroma, Pleroma.Uploaders.S3, ex_aws_impl: Pleroma.Uploaders.S3.ExAwsMock

6. Update Your Tests to Use Mox

# Old (with Mock)
test_with_mock "save file", ExAws, request: fn _ -> {:ok, :ok} end do
  assert S3.put_file(file_upload) == {:ok, {:file, "test_folder/image-tet.jpg"}}
  assert_called(ExAws.request(:_))
end

# New (with Mox)
test "save file" do
  ExAwsMock
  |> expect(:request, fn _req -> {:ok, %{status_code: 200}} end)

  assert S3.put_file(file_upload) == {:ok, {:file, "test_folder/image-tet.jpg"}}
end

7. Enable Async Testing

Now you can safely enable async: true in your test module:

use Pleroma.DataCase, async: true

Best Practices

1. Always define behaviors: They serve as contracts and documentation
2. Keep mocks in a central location: Use test/support/mocks.ex for all mock definitions
3. Use verify_on_exit!: This is already set up in DataCase, ensuring all expected calls were made
4. Use specific expectations: Be as specific as possible with your expectations
5. Enable async: true: Take advantage of Mox's concurrent testing capability
6. Don't over-mock: Only mock external dependencies that are difficult to test directly
7. Leverage existing stubs: Use the default stubs provided by DataCase when possible
8. Avoid clear_config in async tests: Use dependency injection and mocking instead

Example: Complete Migration

For a complete example of migrating a test from Mock/meck to Mox, you can refer to commit 90a47ca050c5839e8b4dc3bac315dc436d49152d in the Pleroma repository, which shows how the S3 uploader tests were migrated.

Conclusion

Migrating tests from Mock/meck to Mox provides significant benefits for the Pleroma test suite, including faster test execution through async testing, better isolation between tests, and more robust mocking through explicit contracts. By following this guide, you can successfully migrate existing tests and write new tests using Mox.