# Test a module

`smf_pipeline` has a library for the tests of modules, `package:smf_pipeline/testing.dart`, with three tools. The contract harness generates every app that matters for a module in memory and checks it against the rules of the module model. `ModulePackage` checks what the package of a module imports and depends on, and `roleClassNameProblems` checks that its code takes the names of the classes of roles from the roles. The built-in modules test themselves with the harness and `ModulePackage`, and the tests of the CLI run `roleClassNameProblems` over all of them. Use all three for yours. A package that defines a role has a fourth, `roleNoteProblems`, for the note of the role in the guide for coding agents.

## The contract harness

```dart title="test/counter_module_test.dart"
import 'package:smf_bloc/smf_bloc.dart';
import 'package:smf_counter/smf_counter.dart';
import 'package:smf_flutter_core/smf_flutter_core.dart';
import 'package:smf_gen_l10n/smf_gen_l10n.dart';
import 'package:smf_get_it/smf_get_it.dart';
import 'package:smf_go_router/smf_go_router.dart';
import 'package:smf_pipeline/smf_pipeline.dart';
import 'package:smf_pipeline/testing.dart';
import 'package:smf_riverpod/smf_riverpod.dart';
import 'package:smf_shared_preferences/smf_shared_preferences.dart';
import 'package:test/test.dart';

void main() {
  test('every app of the module follows the rules of its roles', () async {
    final harness = ContractHarness(
      ModuleRegistry(const [
        FlutterCoreModule(),
        GoRouterModule(),
        GetItModule(),
        BlocModule(),
        RiverpodModule(),
        // The localization, which the counter uses for the label of its
        // tab, and the preferences that the localization requires.
        GenL10nModule(),
        SharedPreferencesModule(),
        CounterModule(),
      ]),
    );
    final results = await harness.checkAll();
    for (final result in results) {
      expect(result.errors, isEmpty, reason: '${result.contractCase}');
    }
  });
}
```

The registry has a module for the localization, which the counter uses for the label of its tab, and one for the preferences, which the localization requires. Without them the harness builds no app of the counter with that role, and a counter that lists the role and forgets to give it the label passes. With them the test reports the mistake: "The module gives the router role the text "label" in its data, but not the localization role, so the app would show it in English in every language."

`checkAll()` builds a case for every module and every role of the registry. For a module, it builds an app for every combination of a provider of the role of its variants and a provider of each role it requires. It repeats this for each subset of the roles that the module only uses, the largest first, with a provider of each role of the subset in the combination too. A combination that would give the app two providers of a role that takes one is left out, such as one with another router for a module that depends on `go_router`. When no combination for a subset can be in an app with the module, the harness keeps them all, and their cases report why.

A case is named after its module, the provider of each role that has several in the registry, and the roles of its subset. For the counter above, the cases are `counter (bloc)` and `counter (riverpod)`, without the localization that it only uses, and `counter (bloc) with localization` and `counter (riverpod) with localization`. If the counter also used the analytics role, and the registry had two providers of it, one of its cases would be `counter (bloc, firebase_analytics) with analytics`.

It also builds a case of the module with each provider in the registry that brings a package which the module adds, itself or in a variant, when the two can be in one app (see [the packages of a provider](https://doc.saymyframe.com/extending/role-provider.md#the-packages-of-a-provider)). The case is named `<module> with <provider>`, such as `banner with bloc` for a module without variants that adds `flutter_bloc`. When the provider provides another role than that of the module's variants, the case is `<module> (<provider of the variant>) with <provider>`, one for each variant that adds the package, or for the first variant when only the module itself adds it. A case whose app another case builds already is left out.

For each case, the harness runs the stages of the pipeline up to rendering, the way a run without a terminal that skips external setup would. It answers the questions of the roles like a user who presses Enter, and renders the app in memory. Then it checks, among other things:

- the rules of the module's kind, the access rules of sockets and roles, and the roles in `when`;
- the module rules of every role, and its structural rules over the parsed Dart files of the app;
- that every provider generates the symbols its role requires, and no file at the path of a symbol whose condition on a choice does not hold in the app, such as the fallback start screen of the app entry in an app that a route starts;
- the tags of the sockets in the templates, and templates that mustache would copy with their braces;
- that every import finds its file and every imported package is a dependency of the app;
- that a module imports only the files it may use: its own, those of the modules it depends on directly and those of its roles. The imports of a template count for the module of the template, and an import that the pipeline adds for a fragment counts for the module of the fragment, as the errors say with "in the template of …" and "for a fragment of …";
- that only what a render hook of a role gives imports the files of the modules that give the role data, such as the screens that a router routes: a fragment of the hook, or a file that the hook generates. That holds for the template of a role and for its providers. The template of a brick gets no such allowance, and neither does a contribution for a fragment of its own, since both are the same whatever the modules give the role;
- that a module which imports or exports the package of a provider of a role in the app, in a template or for a fragment, adds that package itself, even when it depends on the provider. A provider of a role may import, for a fragment or in a file that its render hook generates, the package that a module which gives the role data adds, as the data may need it;
- that no file of the app, of a brick or of a render hook, is at a path where code generation or the localizations of Flutter write their files once the app is rendered;
- that each path that the guide for coding agents of the app names, below a top-level directory of a Flutter project or to a Dart file, is a file or a directory of the app.

The files that a render hook generates are files of the app like those of the bricks, and the harness checks them the same way.

The result of a case also has the warnings about what its app is made of, in `ContractResult.warnings`, each with its code. The harness makes the choices of every app once more without a terminal, and reports a role whose warnings differ there. It also reports a `choose` hook that warns through the logger, whose warnings have no code, and a structural rule that warns. So a test of a module can expect the warnings of an app by their codes, such as `router.no_start` for an app of a feature that marks no route as a start screen:

```dart
expect(
  result.warnings.map((warning) => warning.code),
  [RouterRole.noStartWarning],
);
```

What the registry holds decides what the harness can check, so put these modules into it:

- `FlutterCoreModule`, the app entry that every app needs;
- every module that your module depends on, which the registry requires anyway;
- every provider of a role your module requires or uses that you want it checked with, since the harness builds an app with each, such as `SharedPreferencesModule` for a module that keeps a setting, `SettingsModule` for one with a row on the settings screen and `GenL10nModule` for one with texts. Such a provider needs what it requires in the registry too: `SettingsModule` a router, such as `GoRouterModule`, and `GenL10nModule` the preferences. [Add a setting](https://doc.saymyframe.com/extending/settings.md#test-the-module) has such a registry;
- every provider that your module has a variant for, which the registry requires as well;
- the provider of every package that your module adds and that a provider of a role brings, such as `RiverpodModule` for a module that adds `flutter_riverpod`.

SMF allows the package of a provider only in a variant for that provider or with a dependency on it, and `smf create` checks this only in apps that have the provider. With the provider in the registry, `checkAll()` builds such an app of your module, so your tests find the mistake rather than a user of your module.

`harness.uncheckedProviders()` lists, one line each, the providers that no case checks a module with. Each is a provider of a role that a module of the registry requires or uses, and it can be in an app with the module, but no app of the module's cases has it as the provider of that role. That happens when the apps with the provider do not resolve, such as when no module of the registry provides a role that the provider requires. Expect the list to be empty, as the tests of the CLI do. Each module then stays checked with every provider of its roles, also when a role gets another provider:

```dart
expect(await harness.uncheckedProviders(), isEmpty);
```

## The modes of a role

A role can have a [mode option](https://doc.saymyframe.com/extending/new-role.md#a-mode-option), each value of which makes another app, as `--auth-mode` of the auth role does. For such a role, `checkAll()` builds one more case of each provider of the role for each value of the option after the first. It is the first case of the provider, which has every role that the role uses, with the value among the options of the case, and its name ends with `--<option>=<value>`. With `firebase_auth` and a router in the registry, these are `auth by firebase_auth with router --auth-mode=guest` and `auth by firebase_auth with router --auth-mode=anonymous`. The other cases leave the option out, so their apps have its first value.

So the tests of a provider of such a role render its hook for every value, and a second provider gets these cases without a change of the tests. The harness reads the options from the roles of its registry and names no role itself.

In every app with such a role that it renders, the harness makes the choices twice: in a terminal, where it answers as a user who presses Enter, and without one, as `smf create --no-input` makes them. It reports a role:

- whose choice with a value of the option is not that value, or whose choice without the option is not the first value;
- whose two choices differ;
- that fails or asks in the run without a terminal.

It also reports a value that a case gives an option which does not take it, since the command line would refuse it.

`harness.casesOfModes()` has the cases of the apps with every module, those of `casesOfAll()`, once more for each other combination of the values of the mode options of the roles of each app. SMF's CI generates apps from them; see [Contributing](https://doc.saymyframe.com/contributing.md#generated-apps). An option that you give every case, with `ContractHarness(registry, roleOptions: {...})`, has that value in every app and no cases of its own.

## Checking one app

`harness.check(case)` builds one app and returns its result, whose `app` holds the rendered files:

```dart
final result = await ContractHarness(registry).check(
  const ContractCase('counter', requested: [CounterModule.id, BlocModule.id]),
);
final screen = result.app!.files['lib/features/counter/counter_screen.dart']!;
```

`ContractCase` takes the modules to request, the provider to pick for a role with several, and the values of role options in `roleOptions`, such as `{'start': '/counter'}` or `{'auth-mode': 'guest'}`. Check the rendered code as code rather than as text, by parsing it with the analyzer and looking for the declarations and calls you expect.

Your module does not know which module provides a role, and its tests should not either. So they check what the module gives a role through the role, and not in the files that the provider of the role renders: `lib/core/router/app_router_factory.dart` of the router, `lib/core/di/dependencies.dart` of the DI container, `lib/core/layout/app_shell.dart` of the layout, `lib/core/l10n/l10n.dart` and the files of the texts, such as the ARB files of `gen_l10n`, the screen of the module that provides the settings screen, and `lib/bootstrap.dart`, `lib/main.dart` and `lib/app.dart`, into which the provider of the app entry renders the start-up and the root of the app. Another provider renders them differently. The result gives two views of the app that do not depend on the provider:

- `result.hook` is the `RoleHookRequest` of the app: the data of the roles, the roles that are present, the context of the harness and the choices of the roles, as the hooks of the roles got them when the harness rendered the app. A role builds the input of its hooks from it with `hookInput`, so a test reads the data as every provider of the role gets it. `diRole.graphOf(diRole.hookInput(result.hook!))` has the registrations of the services, `routerRole.startIn(routerRole.hookInput(result.hook!))` the route that the router role chose to start the app on, and `layoutRole.destinationsIn(layoutRole.hookInput(result.hook!))` the destinations of the main navigation. In the same way, `settingsScreenRole.entriesIn` has the entries of the settings screen, in the order the screen shows them, `localizationRole.textsIn` the texts of the app with their getters, `localizationRole.localesIn` its languages, and `authRole.modeIn` the sign-in mode that the auth role chose.
- `result.app!.socketOrders` has the contributions of every socket that got any, in the order that the pipeline rendered them into the tags of the socket, including the fragments of the render hooks. What the modules and the templates of the roles put into the platform phase of the start-up is in `socketOrders[AppEntryRole.bootstrapPlatform]`, whichever module renders `bootstrap()`.

```dart
final registrations =
    diRole.graphOf(diRole.hookInput(result.hook!)).registrations;
final platform = result.app!.socketOrders[AppEntryRole.bootstrapPlatform]!;
final startUp = [
  for (final collected in platform.contributions)
    (collected.contribution as SocketContribution).fragment!.code,
];
```

`result.app!.entriesOf(socket)` gives what a keyed socket got, in the order the socket got it: the contributor, the key and the value of each entry. With it a test reads the notes of the guide for coding agents, each with the heading of its section, without reading `AGENTS.md`. This finds the note of the preferences role in the app of a result:

```dart
final ofRole = [
  for (final (origin, _, note)
      in result.app!.entriesOf(AppEntryRole.agentSections))
    if (origin == const RoleTemplateOrigin(preferencesRole)) note.text,
].single;
```

The tests of the built-in modules check where the note of their module is: in the section of its role, after the note of the role, or in a section of its own. For the names of code that a note tells agents to use, they check that the note gives each name in inline code and that a file of the app declares or uses it, with `DartFileIndexer` of `package:smf_pipeline/testing.dart`. What a sentence of a note says about the code, only a test of that code can pin.

`result.manualSteps` has the steps by hand of the app, each once with its contributors, in the order that its README lists them; see [Steps by hand](https://doc.saymyframe.com/extending/infrastructure-module.md#in-tests).

When a test compares the app of your module with the app without it, the files of the providers of its roles differ too, since they render the data of your module. Tell those files apart by their owner rather than by their paths: a rendered file whose `owner` is a module among `result.resolution!.providersOf(role)` belongs to a provider of that role.

A module whose tests need a main navigation, but which is not a feature itself, can declare small features and a layout of its own in `test/`, as `smf_firebase_analytics` does in `test/support/navigation.dart`.

## The rules of a module package

```dart title="test/architecture_test.dart"
import 'package:smf_pipeline/testing.dart';
import 'package:test/test.dart';

void main() {
  test('the package follows the rules of a module package', () {
    expect(
      const ModulePackage(
        'smf_counter',
        dependencies: {'mason'},
        testModules: {
          'smf_bloc',
          'smf_flutter_core',
          'smf_gen_l10n',
          'smf_get_it',
          'smf_go_router',
          'smf_riverpod',
          'smf_shared_preferences',
        },
      ).problems(),
      isEmpty,
    );
  });
}
```

`problems()` reads the package in the current directory, which is the package's own when `dart test` runs, and reports every package and import that breaks a rule:

- The code in `lib/` imports only the module model of `smf_contracts`, its own files, the public libraries of its `dependencies`, and libraries of Dart that do not reach the machine, which rules out `dart:io`.
- Of the module packages, the tests use only the module model, the package itself, its dependencies, `smf_pipeline` and the `testModules`.
- The package depends on `smf_contracts` and its `dependencies` and on nothing else. Its dev dependencies among the module packages are `smf_pipeline` and the `testModules`.

The module packages are `smf_contracts`, `smf_pipeline` and every package that depends on `smf_contracts`, whatever its name. `problems()` recognizes them by their pubspecs, which it finds through the package config that `dart pub get` writes, in the directory of the package or in the root of its workspace. When the package config is missing, or the pubspec of a package that it has to check, the problems ask you to run `dart pub get`.

pub.dev resolves the dev dependencies of a package when it analyzes it, so the dev dependencies between module packages must not form a cycle. When two modules would need each other in their tests, one of them tests with a small provider of its own in `test/`.

## The names of the classes of roles

A role can require a class, such as the `AppShell` that every layout generates, or the `Destination` that the template of the layout role generates. A module that generates code which creates such a class takes the name from the role, as in `'${LayoutRole.appShell.name}('`, so that the generated code follows the role. `roleClassNameProblems(roles)` reads the package in the current directory and the module packages it depends on, and reports each string in their `lib/` that spells out the name of a class of those roles followed by `(`, with its path and line:

```dart
test('the code takes the names of the classes of roles from the roles', () {
  expect(roleClassNameProblems(ModuleRegistry(modules).roles), isEmpty);
});
```

Here `modules` are the modules of the registry of your tests. Comments, code outside strings, such as the `Destination(...)` of a route, and the bundles in `lib/bundles/`, which the bricks generate, do not count. `roleClassNameProblems` finds the module packages as `ModulePackage` does, and asks you to run `dart pub get` when the package config is missing.

## The notes of roles

The template of a role adds the note of the role to the [guide for coding agents](https://doc.saymyframe.com/extending/infrastructure-module.md#notes-for-coding-agents), and the note has to hold whichever modules provide the roles. A package that defines such a role checks it with `roleNoteProblems`, in the apps of the results of the harness:

```dart
test('the notes of the roles name nothing of their providers', () async {
  final results = await ContractHarness(ModuleRegistry(modules)).checkAll();
  expect(
    roleNoteProblems(
      results,
      AppEntryRole.agentSections,
      textOf: (note) => note.text,
    ),
    isEmpty,
  );
});
```

The check holds each note of a role against the providers of every role in those apps, and against the modules that they depend on. The note may have none of their ids or packages. In its code, it may name nothing that only their files declare, and no file that only they generate. What the roles guarantee is free, such as the symbols of their interfaces.

The check knows a module only from the generated files, so it takes a name of Flutter that only files with one package have for a name of that package. Pass such a name in `allowed` with the reason, as in `allowed: {'NavigatorObserver': 'A class of Flutter.'}`. Do the same when a line is about a note of `smf_contracts`, which you cannot change. The check also reports an entry of `allowed` that excuses nothing.

## Beyond the harness

The harness renders apps but does not build them. To see your module in a real app, generate apps with your command, as [Extending SMF](https://doc.saymyframe.com/extending.md#how-a-module-reaches-an-app) shows, with each state manager and each provider your module works with, and run `flutter analyze` on them. SMF's CI does this for the built-in modules.

A module with a [guard](https://doc.saymyframe.com/extending/feature-module.md#a-guard) keeps the user from every other screen until its guard allows. So a test that starts an app with the module sees only the flow of the guard, unless it first puts what the guard reads into the state that allows, as a test in an app with [`onboarding`](https://doc.saymyframe.com/modules/onboarding.md#in-tests) does.

A module whose guard asks for an account opens it for such a test by signing an account up before the app starts, as the tests of the apps with [`sign_in`](https://doc.saymyframe.com/modules/sign-in.md) do. A call of `appSession` made before the start-up waits for it, and the start-up completes after such calls. So the test calls `appSession.signUp()` before `bootstrap()`, without awaiting it, and the app starts with that account. It makes the call outside the body of a test, as in a `setUpAll`: made in the body of a widget test, the call waits in the fake time of that body, and a start-up that runs in real time never completes. [Sign-in](https://doc.saymyframe.com/guides/sign-in.md#in-tests) has more on tests of the session.

A test that starts an app runs as its default flavor, `dev`, since `flutter test` takes no `--flavor`. A test of a module with a Firebase service needs a mock of the platform side of that service, and the options of `dev` stay a placeholder otherwise.

A module that makes requests goes through `appRest`, and a test of it gets a local server from `RestTestServer` of `test/core/rest/rest_test_server.dart`, or scripts the answers with `appRest.use(transport: …)`. [REST API](https://doc.saymyframe.com/guides/rest.md#in-tests) tells what `flutter test` does to a request.

A module with a `PostGenStep` that writes into the native projects, as a provider of the app icon does, shows the result only once the step has run. Neither the rules nor a snapshot see it, so its tests read the files of an app that `smf create` generated, as an app test of its module.

A module that remembers a setting has more to show in a running app: that a choice is saved, and that the next launch reads it. [Preferences and settings](https://doc.saymyframe.com/guides/preferences.md#in-tests) describes both checks, which hold whichever module provides the preferences.

A module whose generated code is plain Dart can go further in its tests. It can write the rendered files to a temporary directory with the packages of the tests, analyze them and run them in the Dart VM. The tests of `smf_get_it` and `smf_event_bus` run the generated services with the real get_it and event_bus this way.

### The state of a variant

The state of a screen is close to plain Dart even when the screen is not: a cubit or a notifier uses little of Flutter. So the tests of a feature with [variants](https://doc.saymyframe.com/extending/feature-module.md#a-look-that-the-variants-share) can run the state of each variant with the Dart SDK alone, as the tests of `smf_sign_in` do:

- They write the Dart files of a rendered app to a temporary directory, with stand-ins for what the state uses of Flutter, such as `ChangeNotifier` and `ValueListenable` of the foundation library.
- The package of each state manager is a stand-in too, which exports the package it builds on: `flutter_bloc` exports `package:bloc/bloc.dart`, and `flutter_riverpod` exports `package:riverpod/riverpod.dart`. Both are plain Dart, and the package of the module has them as dev dependencies.
- A script of the test imports the state of a screen and no widget, gives the session a service that it scripts, and runs in the Dart VM. The same scenarios run with each variant: a call that is on its way, a failure, a second call, a user who leaves the screen.

So the tests of the package need no Flutter SDK. The widgets need more of Flutter than a stand-in gives, so the analyzer checks them, and they run in the tests of generated apps.