Contributing

SMF is developed in the open at saymyframe/smf_flutter_cli. Bug reports and ideas go to GitHub Issues, and questions are welcome on Discord. This page is an overview, and the repository has the details. CONTRIBUTING.md describes the process, and AGENTS.md describes the layout, the rules and the conventions. AGENTS.md is written for coding agents, but it serves people just as well.

The repository

The repository is a Dart pub workspace managed with Melos. Every package is versioned and published on its own.

packages/
  smf_contracts/            the module model: modules, roles, sockets, contributions
  smf_pipeline/             the stages of smf create and the contract harness
    fixture_registry/       apps from the fixtures: their snapshots, their matrix
                            and app tests, the app of several providers, and the
                            apps of the providers with a known bug
    test/fixtures/          fake modules for features that no real module uses yet,
                            and providers of roles with a known bug
  smf_modules/
    smf_<module>/           the modules: flutter_core, go_router, home, ...
  smf_flutter_cli/          the smf command: its modules and the machine
tools/                      bundling of bricks, the banlist, the version of the CLI,
                            the coverage check, and tests of the package graph
                            and of CI

Dependencies point one way: smf_contracts ← smf_pipeline and the modules ← smf_flutter_cli. The pipeline knows no module and no concrete role, and modules use smf_pipeline only in their tests.

Set up

CI pins Dart 3.12.2, and the output of the formatter depends on the version, so use that Dart, with Flutter 3.44.2 for the apps. Then run:

dart pub global activate melos
dart pub global activate mason_cli
melos bootstrap

melos bootstrap resolves the workspace and bundles every brick into lib/bundles/ with the Mason CLI. The bundles are generated, so change the brick, bundle it again and commit both. CI fails when a committed bundle differs from its brick.

Checks

melos run check

runs, in this order:

Script	Checks
melos run format:check	The formatting of the Dart code of every package, its app_tests/ and example/ included, and of tools/. melos run format formats it.
melos run analyze	dart analyze --fatal-infos --fatal-warnings in every package and in tools/.
melos run banlist	That no file uses what the repository bans; see tools/banlist.dart. It bans, among other things, the names that the module model replaced, reading the workspace: key of the root pubspec.yaml by hand anywhere but in tools/workspace_members.dart, and, in the tests of a module, the paths and constants of the files that the provider of another role renders, such as lib/core/di/dependencies.dart or lib/bootstrap.dart.
melos run test	The tests of every package and of tools/. The tests in tools/ also check that the published packages, dev dependencies included, do not depend on each other in a cycle. pub.dev resolves the dev dependencies of a package when it analyzes it.

CI also measures coverage. The tests must run every line and take every branch of lib/ in every package. To check it locally, install the coverage tool once with dart pub global activate coverage, then run:

melos run test:coverage
melos run coverage:check

coverage:check names each line and branch that no test reaches, as path:line. Code that no test can run, such as the terminal of a user, goes between // coverage:ignore-start and // coverage:ignore-end, with the reason in a comment.

When a test of a package or of tools/ fails in CI, the page of the run lists it at the top as an annotation with the name of the test, its message, and its file and line. Every run of tests in the workflows writes a JSON report to build/test-results/ of the directory it runs in. A later step of each job with tests, which runs even when a step failed or the job was cancelled, reads those reports with tools/test_annotations.dart and prints an annotation for each test that failed, or that started and never finished. A test in tools/ fails when a run of tests in a workflow writes no report, when two runs of a job write the same report, and when no later step of the job runs the tool with if: always().

Tests are hermetic. They use no network and no Flutter SDK, and they remove their temporary directories. A test of a tool that the app gets, such as the script of firebase_auth that enables the sign-in methods, runs it with a stand-in for the command that it calls, here the Firebase CLI. A few tests need more and skip themselves without it. The test of smf_flutter_core that compares its brick with an app that flutter create wrote needs that app, which the CI job Brick of flutter_core creates with the pinned Flutter. Two others run only in the jobs on macOS and Windows, described below. Two more run only when CI gives them an app that it configures with Firebase, described in Starting the apps: one of smf_firebase_core, which configures the app, and one of smf_firebase_crashlytics, which then fixes its build phase for Crashlytics on iOS. A test of generated code parses or analyzes it rather than comparing strings. A bug that a change finds but does not fix gets a test of the correct behavior, marked skip: 'Bug: <description>'.

Snapshots

The CLI and the fixture registry keep snapshots of the apps they generate, in packages/smf_flutter_cli/test/snapshots/ and packages/smf_pipeline/fixture_registry/test/snapshots/. When a change alters what an app renders, update the snapshots and review their diff:

cd packages/smf_flutter_cli
SMF_UPDATE_SNAPSHOTS=1 dart test test/snapshot_test.dart

The CLI keeps no snapshot of an app with every module for another value of a mode option, such as another sign-in mode. The snapshot of the app of the provider of the role for that value shows what the value changes.

Notes for coding agents

Every module that the CLI offers has a note in the guide for coding agents of the apps with it, and so does every role with a template. test/agent_notes_test.dart of smf_flutter_cli fails on one without a note. A module or a role with nothing to tell a coding agent is listed in that test with the reason, and so is a module whose only note is for the apps with another role.

The same test runs roleNoteProblems over the modules of the CLI, and the test of that name in the fixture registry runs it over the fixtures, so that the note of a role names nothing of a module that provides a role. Notes for coding agents has the rules of a note, and Test a module describes the check.

Generated apps

Besides the job of the checks above, CI has jobs with Flutter, which run with the pinned Flutter 3.44.2 on pushes to main and on pull requests into it. They generate apps with the matrix tool of each registry: packages/smf_flutter_cli/tool/matrix.dart for the modules of the CLI, and packages/smf_pipeline/fixture_registry/tool/matrix.dart for the fixture modules. The matrix of a registry has the apps of the contract harness for its modules, and the apps with every module, one for each combination of the providers of the roles that take one, such as one for each state manager.

The tool generates each app with smf create --no-input --skip-external-setup --no-dart-fix --strict and runs flutter analyze on it. --no-dart-fix, a hidden option, skips the full dart fix of the app. Then it copies into the app the tests that apply to it, from the app_tests/ directories of the packages, adds the packages these tests use with flutter pub add dev:…, runs flutter analyze again and then flutter test. These tests check what only a running app shows, such as the start-up of Firebase with the platform side of its plugins mocked, and they are not part of the apps that smf create generates. flutter test runs them next to the one test that every generated app has, the widget test of the fallback start screen of flutter_core. See App tests.

The number of apps with every module is the product of the numbers of providers of those roles, so it grows fast. The matrix generates a pairwise covering of them: every pair of providers of two different roles that fit in one app is in one of its apps, and so is every module. The tool still renders every combination in memory with the contract harness, and a combination with errors fails the run.

A role with a mode option makes another app for each of its values, as the auth role does with --auth-mode. So a matrix has each app with every module once more for every combination of the values of the mode options of its roles, except the combination of the first values, which is that app itself. Such an app has the modules of the app with the first values, and its name ends with its values, as in every module (bloc) --auth-mode=guest. For a covering, the value of a mode option is one more thing that an app has, as the provider of a role is: a pairwise covering takes the apps that have, between them, each other value with each provider of a role that has several, and with each value of another mode option.

An app with the first value of a mode option gets no value of it on the command line. The role chooses the first value without the option, so CI generates the app as a user without the option does.

The plan of CI

CI spreads the apps over many jobs by a plan. The job Plan runs the matrix tools with --plan, which print the plan as JSON. It has the covering of the apps with every module that the matrices check, pairwise, as combinations, and these lists:

• shards: the shards of each matrix, one for about each 24 apps that its jobs check, about 20 minutes each;
• apps: a pairwise covering of the apps with every module of the CLI;
• start: a pairwise covering of the apps with every module of the CLI but the modules whose steps need an external service, such as Firebase, and the modules that need them;
• entries: the app with every module of the CLI for each provider of the app entry, the role whose provider owns the native projects and their configuration.

The apps of the other values of a mode option count for shards, and they are never in apps, start or entries. So the apps that CI builds for a platform or starts on a device have the first value of every mode option. The values differ in Dart code only, which the jobs of the matrices check.

Each job of this table takes one shard or one app of the plan from its matrix:

Job	Takes	Does
Generated apps (<registry>, <shard>)	a shard of shards	Checks its share of the matrix of its registry, real for the modules of the CLI or fixtures, in a directory with a space in its name. Then it checks its share of the apps with every module again, in a directory whose name has letters beyond ASCII too.
Android (<app>)	an app of apps	Builds the app with flutter build apk --debug.
iOS (<app>)	an app of apps	Builds the app for the iOS simulator.
Windows (<app>)	an app of apps	Checks the app as the matrix does, and generates it once more as a user does.
Android start (<app>), iOS start (<app>)	an app of start	Starts the app on an Android emulator or on an iOS simulator.
iOS archive (<app>)	an app of entries	Archives the app with the build phase for Crashlytics.
Android with Firebase (<app>), iOS with Firebase (<app>)	an app of entries	Configures the app with Firebase and starts it.

The other jobs have no matrix: Generated apps (several providers) and Generated apps (broken providers), described in App tests, macOS and Windows, which run the tests of the packages, Brick of flutter_core, which compares the brick of flutter_core with the app of flutter create, and Firebase key, which tells the jobs with Firebase whether the key of the Firebase project of CI is there.

A job shows its app in its name, such as Android (every module (riverpod)), and the matrix tool generates the app with --create --app, named after the providers other than the first of their roles, such as android_app_riverpod. So a new provider or module changes the plan, not the workflows, and a job that takes its app or shard from the plan does not take longer as the apps get more: only the number of jobs grows.

What Gradle and Swift Package Manager download for the native builds goes into caches of the repository. Each cache is keyed by what the app of a job builds with and by the week, and serves every version of Flutter. A job restores the cache of its app, or else the newest one of an earlier week or of another app, and the runs on main save the cache of an app when its build added more than 100 MB to what the job restored. Maven Central answers 403 to a runner that downloads too much, so the jobs that build for Android also take what Gradle would download from Maven Central from the mirror that Google keeps, through the init script .github/gradle/maven_central_mirror.gradle.

Running the matrix locally

To run the matrices locally, with flutter on the PATH:

dart run packages/smf_flutter_cli/tool/matrix.dart /tmp/smf_apps
dart run packages/smf_pipeline/fixture_registry/tool/matrix.dart /tmp/smf_fixture_apps

To check only some apps of the matrix, name them after the directory:

dart run packages/smf_flutter_cli/tool/matrix.dart /tmp/smf_apps 'go_router with layout'

These options go before the directory:

Option	Checks
--every-module	Only the apps with every module, one for each combination of providers. It leaves out the apps of the other values of a mode option.
--combinations 3-wise, --combinations all	A covering of every triple of providers of three roles, or every combination, of the apps with every module, rather than a pairwise covering. The values of the mode options count as the providers do.
--app '<name>'	With --every-module, only the app with every module of that name, such as 'every module (riverpod)', as a job of CI does.
--shard <i>/<n>	Only the share i of n of the apps, as a job of CI does.

--plan alone prints the plan, and --plan --every-combination the plan of the nightly run with the latest Flutter.

--create generates apps with every module without checking them, for the native builds and the starts. It takes a directory and a name, names each app after the providers other than the first of their roles, such as my_app and my_app_riverpod, and passes the arguments after the name to smf create. It generates a pairwise covering of the apps with every module, or the apps of --combinations or --app, and none of the apps of the other values of a mode option. With --without-external-steps after --create, it leaves out the modules whose steps need an external service, and the modules that need them, so the apps start as they are generated.

dart run packages/smf_flutter_cli/tool/matrix.dart --create --without-external-steps /tmp/smf_start start_app

In a directory whose path has letters beyond ASCII, the matrix analyzes the apps with dart analyze --fatal-infos, because flutter analyze of Flutter 3.44 and 3.47 fails in such a directory on every system.

To run the CLI from source:

cd packages/smf_flutter_cli
dart run bin/smf_flutter.dart create my_app -o /tmp/out --org com.example --no-input --on-conflict replace

App tests

Each directory right in the app_tests/ of a package holds the tests of one MatrixAppTest. The package of each matrix tool registers them in its lib/matrix_app_tests.dart, each with the apps it appliesTo: smfAppTests() of smf_flutter_cli for the CLI and its modules, and fixtureAppTests() of fixture_registry for the fixture modules. A test that applies to no app fails the run, and test/matrix_app_tests_test.dart of the package fails on it without Flutter. A test in tools/ fails when no matrix tool lists a directory of app tests, because its tests would never run. app_tests/ stays out of the analysis of its package, of its published archive and of SonarCloud, but melos run format formats it. Give each testWidgets there an explicit timeout.

In the files of a test, {{app_name}} becomes the name of the package of the app, and {{<key>}} a value of MatrixAppTest.values. A test that needs to know what the modules of an app give a role, such as the services they register, gets files that the matrix writes next to it from the data of the roles of the app, with MatrixAppTest.generatedFiles. The start check of the CLI, described in Starting the apps, applies to every app, so the matrix analyzes it in each of them, and flutter test runs in each of them.

The tests check roles, not the modules that happen to provide them today:

• A test that a package of modules keeps applies only to the apps with its modules. What it needs of the other modules of an app comes from the roles of the app through MatrixApp.hook, such as whether the app has a router and the route it starts on.
• A test of the contract of a role, such as the listeners of the screen of the router role, names the role in MatrixAppTest.roles and selects its apps by the roles of the app, never by the ids of the modules that provide it. The run fails when such a test applies to no app of a provider of its role, or would select other apps, take other values or generate other files with another id in place of that of a provider.
• The run also fails when a role that the tests of a registry must check, in MatrixAppTests.testedRoles, has no test. So a new router gets the tests of the router role without a change of the tests.
• A test of what only one provider does, such as a refresh of go_router, names no role and selects the apps of that provider by its id, as an exception that tools/app_tests_test.dart lists.
• A test takes the mode of its app from the choice of the role, with a function of the role over MatrixApp.hook, such as authRole.modeIn. It reads neither the name of the app nor the options that the matrix generates it with. The run fails on a test that does something else once an app has another name or no options, and says what the test read.

Matrix	Roles that its tests must check
The CLI	Router, DI, events, preferences, localization, theme, authentication, app entry, settings screen
The fixtures	Router, DI, events, preferences, localization, theme, authentication, app entry, layout
The app of several providers	Analytics, crash reporting

The fixture registry has every module of the CLI that provides the router or the layout role, so its tests of those roles run with each of them.

Tests in tools/ also fail when:

• an app test that a package of modules keeps applies to an app without those modules;
• a matrix tool selects the apps of a test, fills the values of its files or generates files for it by the id of a module that the test does not know;
• a matrix tool imports a package of modules whose app tests it does not register;
• a file of app_tests/ imports a file of the app that neither a role nor a brick of its own modules generates;
• an app test calls a function of a role that an app can have several providers of, such as createCrashReporter(), unless it tests the contract of that role. The call reaches every provider, and only the tests of each provider know what it does there, so the test of Crashlytics calls createCrashlyticsCrashReporter() instead. A test of the role names the role in MatrixAppTest.roles, lives in a package of no module, such as the fixture registry, and looks only at what reaches the providers that its package knows.

A module whose start-up or services call a platform channel keeps the mocks of its platform side with its app tests, as a function in a file in test/, and declares the function in MatrixAppTest.mocks, as the Firebase modules and shared_preferences do. The matrix writes test/flutter_test_config.dart into each app with such tests. Before the tests of each test file, it sets up the mocks of every test that applies to the app, so the tests of any module can run the start-up of the whole app. A module with a guard of the routes opens the guard in those mocks, since the tests of the other modules expect the screens that the guard would keep them from, and the walk of the routes fails on a guard that does not allow, by its name. The mocks of onboarding finish the onboarding before the app starts, with onboardingStatus.complete(), unless a test file of the onboarding itself tells them to leave it alone. The mocks of sign_in, whose guards ask for an account, sign one up: a call of appSession.signUp() made before the app starts, which nothing awaits, gives the app that account before its first frame. So every test file of every module in an app with sign_in starts with an account signed in, and both guards allow. The two test files of the module that go through a first launch tell the mocks to sign nobody up.

The mocks of firebase_auth are a backend in memory that starts with no account and nobody signed in, so every test of the app sees a first launch. The firebase_auth plugin brings no mocks for tests, so they answer the messages of the plugin themselves.

A widget test that starts the app runs main() in tester.runAsync, so that a start-up that waits for a timer or for input and output does not keep the fake time of the test waiting forever. The test fails on an error of main(), which runAsync would only report to FlutterError.onError. Once main() returns, it puts back FlutterError.onError, PlatformDispatcher.instance.onError and ErrorWidget.builder, which a start-up may replace, as crash reporting replaces the handlers of errors. startApp in fixture_registry/app_tests/router_screens/test/screens.dart does all of this.

The app tests of the modules of the CLI must pass whatever else an app has. The job Generated apps (several providers) runs them in the app with every module of the registry of several providers, severalProvidersModules() of the fixture registry. The app has:

• each module of the CLI that provides a role an app can have several providers of, such as crash reporting, next to fixture providers of the same roles, whose start-up and services call platform channels of their own;
• a fixture provider of both roles that notes each call it gets, and a fixture whose start-up waits for a timer;
• get_it, the fixture events and shared_preferences, for the tests of the DI, events and preferences roles;
• settings with two fixtures that have a setting, for the tests of the settings screen role. The widgets of the two settings have one class name, so the app analyzes only if the screen imports the file of each with a prefix of its own;
• material_theme, for the tests of the theme role, and the fixture texts, a provider of the localization role;
• onboarding and home, for the app tests that the two modules keep. The mocks of the first finish the onboarding for the other tests of the app, and the test of the second runs in an app that starts on the screen of another feature;
• firebase_auth, for the app test that the module keeps and for the test of the auth role, in the default mode of the role;
• sign_in, for the app test that the module keeps, and with it bloc and riverpod, since a registry has every provider that one of its modules has a variant for. The registry has such an app for each of the two state managers, and the tool checks the first.

The app is also where the tests of the analytics and crash reporting roles run; see The roles at run time. A test of the fixture registry fails when such a module of the CLI is missing from the registry. The tool checks its one app and takes no names of apps:

dart run packages/smf_pipeline/fixture_registry/tool/several_providers_matrix.dart /tmp/smf_several_providers

The roles at run time

The contract harness checks the files of an app. The matrices check what a provider of a role does in a running app, with tests of the role in the apps of every provider of the role that their registry has:

Role	Tests	What they check
Router	router_walk of the CLI	The test starts the app with main() and goes with go() to each location that needs no values, up to the first 20. Each location shows the page of its route on top of the innermost navigator, named after the route, and the screen of the route, with no ErrorWidget and no error from Flutter. Every guard of the routes has to allow, so that the walk reaches every route outside the flows, and the test fails on one that does not, by its name. The flows are over then, so for a location of a flow the walk expects the screen that the app starts on.
Router, layout	the tests of the fixture registry	The listeners of the screen, each called on its own, the observers of each navigator, the back button of the system, the fallback screen, the StateError of push() and replace() with a location of the main navigation, the config created once, and the switches between the destinations of the layout. On the screen that the router shows for a location that it cannot show, the back button throws nothing and closes a dialog over it, and push() and replace() from that screen show their location, of which the listeners hear once. go() out of the main navigation and back to a destination throws nothing, in one turn too. In each language of the app, label(context) of every destination returns the text of its module in that language, and the layout shows no label in a language that the app is no longer in: as a text, in a tooltip or to the semantics.
Router: guards	router_guards, router_guards_fallback, layout_guards and router_walk_guards of the fixture registry	While a gate does not allow, the router shows its target in place of the location that the app starts on, and of each location outside the flow of the guard that go(), push() or replace() asks for. The target is then the only page, and such a push() completes with null. When a guard starts or stops allowing, the router shows what the role answers: the target, in place of the pages that the guard keeps the user from, or, once the guards allow, the latest location that was asked for, the location below the pushed pages, or the screen that the app starts on. The routes of a flow show only while a guard with that flow does not allow: once it is over, a location of the flow shows the screen that the app starts on. The router asks the guards by stage, whatever the order of the modules. Once a guard that does not bring the user back stopped allowing, the user comes to the screen that the app starts on. A notification of a guard that changes nothing leaves the stack as it is. The guards hold over the fallback screen of the app entry too. With a layout the user comes back to the destination that a pushed page was opened from, and each branch is back on its destination, also when a gate closes and opens in one turn. The walk holds with a guard closed and then open.
Router: conditions	router_conditions, router_flow_opens and layout_guards of the fixture registry	While a condition does not hold, go(), push() and replace() of a route that asks for it open the target of its guard over the page that the user is on, and the router builds no screen of that route. Back returns to that page, and such a push() completes with null. Once the condition holds, the router closes the flow and does what was asked. A further request shows nothing while the flow is open, and the first one waits. A route that asks for two conditions opens the flow of each guard in turn, and a location from the platform opens the flow over the screen that the app starts on. A link to a page of the flow itself opens that page over the screen that the app starts on, and back leads there. One request opens the target once, and the router throws nothing later, as from a microtask. When the condition stops holding, the router closes the pages that ask for it. A gate decides for every route, and the tests pin the known limit: a location that asks for the condition and is asked for behind a gate is lost when the gate allows first. With a layout, the flow opens over the main navigation, which stays below it. The listeners of the screen hear only of the pages that the user saw.
DI	di_role of the CLI, di_disposal of the fixture registry	After the start-up, every registered service resolves, a singleton or lazy singleton to the same instance each time. None resolves after resetDependencies(), and all resolve again after registerDependencies(). resetDependencies() disposes of the services of the fixtures in the reverse order of their registration and creates no lazy singleton just to dispose of it.
Events	events_role of the CLI	Each listener of a type gets each event of that type once, in the order the events were fired, and a listener of an unrelated type gets neither the event nor an error. The stream of on<T>() has only the events fired after it is listened to, and a cancelled subscription gets no more events.
Preferences	preferences_role of the CLI, preferences_restorers of the fixture registry	A value of each type is read back as it was saved, and as null by the reads of the other types, which do not throw. A removed key has no value, and a write replaces what its key had, a value of another type too. The preferences keep a copy of a list they are given, and a read returns a copy of it. The next start, initPreferences() again, reads what was saved and nothing that was removed. The start-up gives each restorer the preferences once, a setting that nothing saved keeps the value it had, and a restorer that throws keeps no other from restoring and does not stop the start.
Settings screen	settings_screen_role of the CLI	The route that the provider names shows the screen. The screen shows every entry once, one below the other in the order of the role, each on a Material, in a list that scrolls. It sets the width of each entry, the same for every entry, and puts no limit on its height.
Localization	localization_role and language_setting of the CLI	The root of the app supports the languages of the app and no other, with a delegate of each kind of localizations for each. Once the user chose a language with appLocale.choose(), the app is in it, and each text reads in it: its translation, or its English text. While the user chose none, the app is in the language that the device prefers among its own, and in its first one when the device asks for none of them. A choice is saved under the key of the role and removed when the app follows the device again, and the next start restores the language that was saved. In an app with a settings screen, the entry of the language shows the choice as the value of its row. A tap opens a sheet over the main navigation, with each language of the app and the languages of the device, and the one check mark on the chosen option. A tap on an option closes the sheet and chooses it. On a small phone with a large text size, the choice is below the title of the entry, and the sheet scrolls to its last option.
Theme, app entry	theme_role and theme_setting of the CLI	The root MaterialApp takes each mode that appThemeMode.choose() chooses. The screen below it gets a light theme in the light mode and a dark one in the dark mode, with the color schemes that the two functions of the provider return, and in the mode of the device the theme that the device asks for. Around the navigator of the app, the root asks the system for dark icons of the status bar on a light theme and for light ones on a dark theme. A choice is saved under the key of the role, and the next start has the mode that was saved. The root rebuilds when the mode changes, whichever module provides the app entry. In an app with a settings screen, the entry of the theme mode shows once and offers each mode, with its icon above its name and the mode of the app selected, and a tap on a mode chooses it. On a small phone with a large text size, the names of the modes grow only by half and stay in the control of the entry.
Authentication	auth_role of the CLI	The constant authMode of the app is the mode that the role chose, and the test expects what that mode says, so the same files hold in an app of each mode. On the service of the provider, when a call completes, currentUser is its result. A wrong password and an address without an account both fail with invalidCredentials, and a sign-up with an address that has an account fails with emailInUse. A call that fails leaves whoever is signed in. Each anonymous user has an id of its own and keeps it with an account, a deletion removes the account, and userChanges tells each listener of every change. appSession goes through a sign-up, a sign-out, a sign-in and the deletion of the account as its mode says, and only an app in the mode anonymous signs a user in when the user comes back to it. The next start, initAuth() again, has the user of the device as soon as it is over. An account that is signed up before the app starts is signed in when bootstrap() is over.
Analytics, crash reporting	analytics_role and crash_reporting_role of the fixture registry	In the app of several providers, every provider gets each call of the service of the role once, and every crash reporter gets each error that nothing catches once. A provider that throws or fails keeps no other from the call, and its failure reaches neither the code that called nor the handlers of errors. Each analytics provider gets its own copy of the parameters, and a provider whose factory or start fails is left out.

A test of a role knows only the role. The matrix writes what the test needs to know of the modules of an app next to it, from the data of the roles of the app, such as the services that the modules register, the locations of their routes or their texts in each language.

• The matrix of the CLI runs its tests in every app with the role, and the test of the DI role in those whose modules register services.
• The matrix of the fixtures runs them only in some of its apps: the tests of the events role, of the preferences role and of the localization role and the walk in its apps with every module, which run flutter test for other tests anyway, and the test of the DI role in its apps whose services have every lifetime. It runs the test of the auth role in every app with the role, which a fixture sign-in with accounts in memory provides there.
• The walk, the check of the services of the DI role, the checks of the preferences and the check of the languages use no test framework, so the start check also runs them on devices; see Starting the apps. The tests of the settings screen have no such probe, since the walk goes to the screen on a device. The test of the auth role has none either, since nobody can sign in on a device.
• Under flutter test, the flows of the guards are over, so the walk shows none of their screens. Those are for the tests of the module of each guard.
• The test of the auth role calls the service of the provider itself, through createAuthService(), since the contract of the service is what it checks. The tests of a module go through appSession, as the code of the module does.

The tests of the guards run in the apps of the fixtures that have the two fixture features with guards. One has two gates that a test closes and opens, and that are open otherwise, and two guards that stand for the conditions of a fixture role, which routes of another fixture feature ask for. The other has a gate of the later stage that does not bring the user back. Both depend on one of the two fixture state managers, so the apps with every fixture come with guards and without them, and the other tests of the router and of the layout run in both. In the apps without, the routes that ask for a condition have no guard that stands for it, and show like any other route.

For the walk, the matrix writes more next to the locations of an app: shownFor(), which gives the page and the screen that the role says the router shows for a location, startOfApp, the screen that the app starts on, and closedGuards(), the names of the guards that do not allow. No test can open a guard on a device, such as one that asks for a signed-in user. So the probe of the walk holds there whichever guards allow. For a location that a guard keeps the user from, it expects the page and the screen of the target of that guard, and for any other location in a flow that is over, the screen that the app starts on. In an app with sign_in in the mode required, the gate does not allow on a device. The walk then shows the three screens of the sign-in and expects the sign-in in place of every other location.

A module keeps the tests of what only it does. The test of shared_preferences checks that a value that the preferences of the module save reaches the package under the same key and with the same type, that a number is read only as the type that it was saved as, and that a list is read in the form that each platform returns it in. The test of settings puts the app into each language that the module has its title in: the screen shows the title in that language, as its header for a screen reader. Below the title, the screen shows one group, a card with a line between each two entries, or, in an app whose modules have no setting, the note for the developer of the app. The note names the file of the screen, a tap on the path copies it, and on a small phone with a large text size the screen scrolls and shows the path in full. The screen has a back button, in an app bar, only on top of another screen, where the button leads back. The test of onboarding starts the app as its first launch finds it, with nothing saved: the app shows the onboarding and does not build the screen that it starts on, and Skip and Get started, the button on the last page, save onboarding.completed and show that screen. The marks of the pages and the buttons follow the page, and in an app that asks for less motion, Next shows the last page at once. Both pages show their texts in each language of the app, with their titles as headers for a screen reader, on a small phone with a large text size too, where the letters and the number in the cell of a page keep their size. Started with the onboarding finished, the app never builds the onboarding, and go(), push() and replace() of its route show the screen that the app starts on: only onboardingStatus.restart() shows it again. Its probe goes through the onboarding on a device. The test of firebase_auth uses the service of the module on the real Firebase packages, over mocks that answer the messages of the plugin. It checks what each call sends to Firebase, the reason of the role for each code of Firebase in the form of Android and of iOS, with the command of the script that enables a way to sign in in the hint of notConfigured, the language of the message of a password reset, and the user whom a start finds on the device. Its probe checks on a device that the session of the app has the user that the plugin has. The test of home goes to the start screen in each language of the app: the screen names the app, shows the mark from the image that the app bundles, and has the card and the three steps, each one button for a screen reader, and a tap on a step copies its path. It shows each greeting at its first and its last hour, the card in a light and in a dark theme, and the parts of the screen as they come in, at once in an app that asks for less motion. On a small phone with a large text size, nothing overflows and the list scrolls to each step. The test of sign_in runs in every app with the module, in each mode of the auth role: its files read authMode of the app and expect what that mode says. Two of its five files go on with the provider of the auth role of the app, from a first launch and from a link to the sign-in. In the mode required the app shows the sign-in in place of the screen that it starts on, a sign-up shows that screen, and a sign-out shows the sign-in again. In the other modes the sign-in opens over a screen, and after a sign-in the user is on that same page. The other three give the session a service of the test, which fails a call with a reason or holds it back. They check the entry of the settings screen and the screen of the account with its sheet, what a form says and does while its call is on its way, the text of each reason of a failure in each language of the app, and the screens on a small phone with a large text size and with a keyboard. The test has no probe: nobody can sign in on a device.

The fixture registry keeps the tests of what only go_router does: the back button of the system on its error screen, push() and replace() while go_router has no page of a route, a main navigation that comes back as a new one after a go() out of it, a refresh of its routes while the flow of a condition is open, a second link to the page on top, and the calls of go_router that the router makes itself. It also keeps the test of the bar that bottom_tabs draws: what each tab says to a screen reader, the tick of a tap, the color and the size of the selected tab, the screen of a destination that fades in, no motion for a user who asks for less, and large text.

A test shows that a setting is remembered by writing its key through AppPreferences, running initPreferences() again and reading the state, and that it is saved by reading the key after a choice. Running initPreferences() again right after a choice proves nothing, since memory has the value already.

Broken providers

A test of a role that passes whatever the provider does checks nothing. So the fixture package fake_broken has providers of roles with one known bug each, which brokenProviders() in fixture_registry/lib/broken_providers.dart lists with their bugs:

Role	The bug of each provider
Router, the screens	Six routers: they tell the listeners of the same screen again, stop at a listener that throws, push a location of the main navigation over a page over it, create their config again, show another screen in the page of a route, or throw when the back button of the system has no route to close.
Router, the guards	Three routers: they ask the guards only as the app starts and when one of them changes, do not show what the guards answer when one of them changes, or leave the stack as it is when a guard keeps the user from the location of a replace().
Router, the conditions	Five routers: they tell the guards of a pushed page under the route of the location below it, ask about a location by the name of its top-level route, show the target of a guard that stands for a condition in place of the stack, or drop the request that waits when the pages of its flow close. The fifth is a go_router whose redirect asks the guards within the calls of the router.
Router, the main navigation and the guards	Three routers: they leave the selected branch its pages when they show what the guards answer, leave the pages of every branch where they are, or keep the navigators of the branches when the main navigation leaves the stack.
Layout	Two layouts: an AppShell that gives only its first destination to the code that reads them, and one that keeps the labels of its tabs from its first build, in the language of before.
Events	A service whose on<T>() gives a listener the events of every type.
Preferences	Three providers: writes that never reach the disk, lists that are never copied, and reads that cast the value to the type they ask for.
Localization	Two providers: a delegate that supports English only, and texts that are always in the first language of the app.
Theme	A createDarkTheme() that returns the light theme.
App entry	A root that keeps the MaterialApp that it built first, so it does not follow the theme mode.
Authentication	Two providers: one returns its service before it has read who is signed in on the device, and one gives an anonymous user a new id when the user gets an account.
DI	A container that registers one service only in a condition that is false when the app runs.
Settings screen	A screen that creates the widget of every entry and shows all of them but the last.
Analytics	A service that notes each call twice.
Crash reporting	A reporter whose start takes the handler of the errors of Flutter for a report of its own.

Each keeps the rules that the contract harness checks, so only a running app shows its bug, and none is in a registry of apps that must work.

The app of a broken provider gets no role option, so that each role chooses as it does for an app that got none. A provider whose bug the tests show in full only in another mode of a role names the option, as the fixture sign-in that gives an anonymous user a new id does with --auth-mode=anonymous.

The job Generated apps (broken providers) checks the tests with them:

1. It generates for each provider a small app with only the modules that its tests need.
2. It adds the app tests of the fixtures and of the app of several providers that apply to the app, and runs flutter analyze and flutter test.
3. Each test of the role that the registry names for the bug must fail, first on the expectation with the reason that the registry gives, and every other test of the app must pass.

So the job fails when a test of a role passes whatever the provider does, or fails for another reason. The job does the same for the apps of brokenModuleApps(), whose tests must fail for what a module does and not for the bug of a provider: in the app of the fixture gates that start closed, which nothing opens for the tests, the test of the walk must fail on the guards, by their names.

A test of the fixture registry checks the registry without Flutter. Among other things, each expected failure names a test that the app has, and a reason that the test file writes, or a file that it imports by a relative path. A reason is a real piece of that text: it has to match at least twelve characters that the file itself writes. A test of the fixture registry fails when a role whose contract the app tests check has no broken provider, unless brokenProviderExemptions exempts it with a reason. The Build workflow runs the job, and the nightly run only with the latest Flutter. To run it locally:

dart run packages/smf_pipeline/fixture_registry/tool/broken_providers_matrix.dart /tmp/smf_broken_providers

macOS and Windows

The jobs macOS and Windows run the tests of every package and of tools/ on the runners of GitHub, with Flutter 3.44.2. A test of something that only one system does runs only there, marked @TestOn('mac-os') or testOn: 'windows': on macOS, the Ruby of the machine and its gem xcodeproj; on Windows, the batch files that the CLI runs without a shell.

The jobs on macOS generate their apps in a directory whose name has a space and letters beyond ASCII. iOS archive (<app>) archives its app with flutter build ipa --no-codesign, in test/crashlytics_archive_test.dart of smf_firebase_crashlytics, which takes the app from SMF_CRASHLYTICS_APP and skips an app without Crashlytics. Before the archive, the test adds the build phase for Crashlytics with the gem xcodeproj, as flutterfire configure does, and fixes the phase with the command from the Crashlytics section of the README of the app. GitHub runs only a few macOS jobs of a repository at a time, so the jobs iOS (<app>) and iOS start (<app>) run two at a time.

Windows (<app>) checks its app as the matrix does, in a directory with a space in its name and then in one whose name has letters beyond ASCII too. Then it generates the app once more as a user does, with the full dart fix, in a directory whose name has letters beyond ASCII, and analyzes and tests it. SMF generates the apps in %TEMP% and moves them to another drive. The job Windows also runs the FlutterFire CLI through dart.bat, as SMF does, and checks that smf create --explain finds it for the app of each provider of the app entry whose modules need it.

The job Firebase CLI install (Windows) of the Build workflow runs the PowerShell script that installs the Firebase CLI, for real. The script changes the machine, so its test runs only when SMF_INSTALL_FIREBASE_CLI is 1.

Starting the apps

CI also starts apps, on an Android emulator and on an iOS simulator. A check that the CLI keeps in app_tests/start/integration_test/start_check.dart becomes the entry of the app. It runs the main() of the app, takes back the error handlers that crash reporting installs, waits for the first screen to settle and checks that the root widget of the app shows it without an error. It writes passed, or failed: and the problems, to the file smf_start_check in the temporary directory of the app, and prints SMF_START_CHECK: with the same line. It knows no module, only main() in lib/main.dart, which the app entry role puts into every app whichever module provides it, and a WidgetsApp at the root, which MaterialApp and CupertinoApp build. So an app with another provider of the app entry role gets the same check.

Once the first screen settled, the check also runs the probes of the tests of the app, those of its roles and of its modules, one after another and for at most a minute in all, and adds their problems to its line. A probe depends on no other: it holds whichever probes ran before it, and whichever guards of the routes allow.

Probe of	What it does on the device
router_walk	Goes to each location of the app that needs no values. It expects the target of a guard for each location that the guard keeps the user from, and the screen that the app starts on for any other location in a flow that is over.
di_role	Resolves each service of the app, a singleton or lazy singleton twice, without calling it, since on a device the apps with Firebase use a real project.
preferences_role	Saves values under keys of its own, reads them back and removes them.
shared_preferences	Opens the preferences again and reads back what it saved, as the platform returns it.
localization_role	Checks the texts in the language that it finds, goes through the first languages of the app, and puts back the choice that it found.
firebase_auth	Reads the user that the session of the app has and the user that the plugin has, which must agree: nobody, an anonymous user or the user of an account, with the same id. It signs nobody in and makes no request, so it holds in every mode and without a network.
onboarding	Reads whether the onboarding is finished, in the preferences and in the app, which must agree. If it is not finished, it goes to the route of the onboarding, which must show its screen, and finishes it as the button of its last page does: the preferences must have that saved, and the router must leave the screen.

A test declares its probe in MatrixAppTest.startProbe: a function in its integration_test/ that goes through the running app without a test framework and returns what is wrong. The matrix tool lists the probes of the tests that it adds to an app in integration_test/start_probes.dart, so the check knows none of them itself. The check prints the probes that it ran to the log of the device, with the time each took, so the log shows them even when the check passes.

.github/scripts/start_app.sh builds a debug app with the check as its entry, installs and launches it with adb or xcrun simctl, and reads that file. Nothing connects to the app, unlike flutter test, which sometimes never reaches the app on the emulators and simulators of CI. The script fails when the check fails, when the app stops before the check wrote its result, as when it crashes at start-up, and when the result does not come in time. Then it prints the lines of Flutter, the end of the log of the device and, on iOS, the crash report.

Two groups of jobs start apps:

• Android start (<app>) and iOS start (<app>) start the apps of start, which need nothing outside the app.
• Android with Firebase (<app>) and iOS with Firebase (<app>) start the app of each provider of the app entry after flutterfire configure configured it in the Firebase project of CI.

Each job adds the check to its app with --add-app-tests and --app, the app of the plan that it generated, with --without-external-steps for the apps of start, so the check runs the probes of the tests of that app. The tool adds the dev dependencies of those tests with flutter pub add.

The jobs with Firebase do more before they start the app:

• A test of smf_firebase_core runs the command of the README of the app for one platform, with the key of a service account from the secret FIREBASE_SERVICE_ACCOUNT. It runs only when CI gives it the app, the project and the key.
• Once the app is configured, test/enable_sign_in_test.dart of smf_firebase_auth asks the project for its sign-in methods as a client does, with the API key of the app: it signs in with a password for an address that has no account, which creates nothing. A project with Email/Password answers that the credentials are wrong. The test fails on a project where the method is off or where Authentication was never set up, and says who enables it. It skips an app without firebase_auth. The apps of CI are in the mode required, so it asks for Email/Password alone, and no job creates an anonymous user.
• In a run of the Build workflow, one job enables the methods first. Android with Firebase of the first app of the plan runs the script of the app with the command of its README and the key of the service account, and expects the line with which the Firebase CLI names the methods that it enabled. It is one job, so that two never write to the project at once. The Firebase CLI sets up a web app of the project for this, so the service account needs serviceusage.services.enable, which the role Service Usage Admin has and Firebase Admin lacks. When the script fails for a permission, the test names the permission that the Firebase CLI printed as denied, and a role that has it.
• The Firebase CLI of these jobs is the latest one of npm. firebase_auth needs 15.6.0 or later, and its check of the machine and the script of the app refuse an older one themselves. tools/firebase_cli_version_test.dart fails when a workflow or a script of CI installs or runs firebase-tools in a version that it writes.
• On iOS, test/flutterfire_phase_test.dart of smf_firebase_crashlytics then fixes the build phase for Crashlytics that flutterfire added, with the command from the Crashlytics section of the README, as SMF does right after it configures an app. It takes the app from SMF_CONFIGURED_APP and skips an app without Crashlytics.
• These jobs, like the job Windows, activate the FlutterFire CLI in the version that tool/flutterfire_version.dart of smf_firebase_core prints, the one that the module activates for its users. tools/flutterfire_version_test.dart fails when a workflow or a script of CI activates the FlutterFire CLI in a version that it writes itself, or in none.

These apps are named firebase_start_app for the first provider of the app entry, and firebase_start_app_<providers> for another, after the providers other than the first of their roles. The names stay when another role gets a second provider, so flutterfire finds the Firebase apps that an earlier run registered.

flutterfire registers a Firebase app when the project has none with the id of the app. Runs at the same time, such as those of the nightly run, could register one each, and the test fails on a project with more than one app with the id. So only the runs of the Build workflow let flutterfire register an app: the run of the pull request into main that brings a new provider of the app entry registers its apps, one job for each platform. In any other run, the test fails before flutterfire when the project has no app with the id, and asks to run Build first, which you can start by hand. tools/firebase_registration_test.dart fails when another workflow lets the test register apps. Without the secret, as in a pull request from a fork, the jobs with Firebase do not run, and the job Firebase key says so in a notice.

To start an app that CI starts, generate it as CI does and add the check for the same app, then start it on a running Android emulator or device, or on a booted iOS simulator:

dart run packages/smf_flutter_cli/tool/matrix.dart --create --without-external-steps --app 'every module (bloc)' /tmp/smf_start start_app
dart run packages/smf_flutter_cli/tool/matrix.dart --add-app-tests --without-external-steps --app 'every module (bloc)' /tmp/smf_start/start_app packages/smf_flutter_cli/app_tests/start
cd /tmp/smf_start/start_app
<repository>/.github/scripts/start_app.sh android <device> 240   # such as emulator-5554
<repository>/.github/scripts/start_app.sh ios <UDID of the simulator> 240

--app names the app with every module that the app was generated as, and --without-external-steps goes with it when --create had it. The tool then adds the tests of that app that have a probe, with the files that they read, such as the locations of its routes, and their dev dependencies. To start an app of yours, leave both out: --add-app-tests <app> packages/smf_flutter_cli/app_tests/start adds the check, which then runs no probe. The last argument of the script is how many seconds the app has to start once it is built. The build itself has no limit, as it can take minutes longer on a busy machine. flutter run -d <device> -t integration_test/start_check.dart runs the check too and prints its result. An app with Firebase starts only once flutterfire configure configured it, as its README says.

Nightly run

Every night, CI runs the jobs with Flutter again, on every stable release of Flutter that the generated apps allow. tool/flutter_versions.dart of the CLI finds them: it takes the Flutter and Dart constraints that the modules put into the pubspec.yaml of the apps, and the releases from the lists that Flutter publishes. The jobs on Linux run on all of them, and the jobs on macOS and Windows on the oldest and on the latest patch of each minor version. The jobs that the nightly run leaves out for a release show up as skipped, such as those on macOS and Windows of the other releases. A skipped job does not fail the run, so a red nightly run always has a job that really failed. To start the run by hand, use Run workflow on the Nightly workflow in the Actions tab.

With the latest release, the jobs of the matrices check every combination of the apps with every module, the apps of the other values of the mode options among them, or a 3-wise covering of them when there are more than 100, in more shards. The apps that CI builds and starts stay the pairwise coverings on every release. The job Generated apps (broken providers) runs only with the latest release: whether the tests of the roles fail on the bugs hardly depends on the version of Flutter, apart from how flutter_test reports a failure.

The nightly run skips the job of the checks, which needs the pinned Dart, and the job that installs the Firebase CLI. It also skips the comparison of the brick of flutter_core with flutter create, because the brick follows the template of the pinned Flutter. Only its jobs with the latest Flutter save the caches of Gradle and Swift Package Manager; the others restore them. So the caches get new versions of plugins within a week even when nothing is pushed. If the nightly run fails while the run with the pinned Flutter passes, a release of Flutter broke something.

It also checks what users install. On each release of Flutter, the job CLI from pub.dev activates smf_flutter_cli from pub.dev, generates with it the apps with every module of that release, with the full dart fix, and runs flutter analyze, flutter test and flutter build apk --debug in each. The published CLI has the modules of its release, which may differ from those in the repository, so tool/every_module_apps.dart of the CLI lists the apps from the matrix of that release, one for each combination of its providers, and the workflow names no module. A release from before the plan has no covering, so this job alone checks every combination, one app after another. A failure there affects the users of the published version, and its fix needs a release.

Changing the workflows

The workflows take their apps from the plan, and they name neither modules nor apps. tools/workflow_apps_test.dart checks the workflows and the scripts in .github/scripts, and fails when a step:

• selects an app of a matrix tool by its name, or refers to an app of --create by its name, as in "$apps/android_app". .github/scripts/each_app.sh runs a command in each app of a directory instead, and fails at the end when it failed in any of them;
• runs smf create itself, or names the modules of an app with -m;
• refers to the package of a module, unless it runs the tests of the module, or runs with dart run a tool of the module that prints what the step needs to know of it, such as the version of the FlutterFire CLI, and the test lists the step as an exception, with its reason;
• adds the tests of a matrix tool to an app with --add-app-tests but without --app, which names the app of the plan that the job generated;
• generates or checks a whole selection of apps in one job, as --create without --app, --every-module without --app or --shard, or the matrix without --shard do, or takes an --app or a --shard that does not come from the matrix of its job.

A job takes one app or one shard, so it gets a fixed number of minutes, and the test fails when a job or a step computes its timeout-minutes. It fails when a job whose outputs feed a matrix refers to a secret, since the runner leaves out an output with a line of a secret, such as the { of a key in JSON. So the job Plan refers to no secret, and Firebase key tells the other jobs whether the key is there. And it fails when a job on Linux or macOS generates an app to build it but restores no cache of Gradle or of Swift Package Manager.

A new job in build.yml goes into the needs of the job CI, and a test in tools/ fails when it does not.

Changes

• Keep modules independent. A change never teaches one module about another to fix a problem quickly. See Module independence.
• A module of the CLI with variants has one for every provider of their role that the CLI offers, as sign_in has for bloc and riverpod. A test of the CLI fails otherwise, since an app with the module and the other provider could not be generated.
• Use Conventional Commits with the short name of the package as the scope, such as fix(go_router): … or feat(contracts): …. melos version derives the versions and changelogs from them, so don't bump versions or edit changelogs by hand.
• Name branches fix/<topic>, feat/<topic>, docs/<topic>, test/<topic> or chore/<topic>. Keep a pull request to one concern and a commit to one logical change.
• A pull request is squash-merged, and the commit takes the title of the pull request as its only line, so the title must be a Conventional Commit too, and CI checks it. Mark a breaking change with ! in the title, such as feat(contracts)!: …, since a BREAKING CHANGE: footer in a commit of the branch does not reach main.
• A larger change may collect its pull requests in a branch of its own. The pull request of that branch into main is merged with a merge commit rather than squashed, so that the squash commit of each of its pull requests reaches main as it is and gets its own entry in the changelogs. The ! of a breaking change then goes into the title of the pull request into that branch that makes the change.
• A pull request needs two checks to pass: PR title, and CI, which fails when another job of the Build workflow failed or was cancelled, and, in a run that runs every job, when one was skipped.
• Pull requests into main and pushes to it run every job of the Build workflow. The jobs with Flutter are many and take about half an hour, so a pull request into another branch, such as one that collects the pull requests of a larger change, runs only PR title and the job of the checks. The jobs with Flutter run once that branch goes into main through a pull request of its own. To run them on a branch before that, use Run workflow on the Build workflow in the Actions tab.
• A new run of the Build workflow cancels the run of the same pull request or branch that is still in progress, and the run of a pull request stops once the pull request is merged or closed.
• Once a branch is pushed, add new commits on top instead of rewriting it.