Connect a database
The operations a connect flow drives to add a database the project does not own: list and check connections live, read the project's database state, write development values, repair, edit and remove a connection, and sample its rows.
Overview
A connect flow, such as Start from an existing database in @antelopejs/dms-assistant, drives these operations. They are served wherever the builder is, in development or under builder.enabled, and CheckDatabaseConnection answers in development only.
| Operation | Answers |
|---|---|
ListDatabaseConnections() | Each catalog connection as { id, engine?, version?, database?, role?, error? }: what its provider's Info() answers, or why it did not. |
CheckDatabaseConnection({ engine, url }) | { reachable, latencyMs?, user?, canRead, canWrite, reason? }. See Check a connection before writing it. |
GetDatabaseProject() | { modules, self, ownDatabase?, databasePins, dmsDatabase?, interfaceDatabase?, drivers }. See What connecting a database writes. |
ConfigureProject(steps, { variables }) | As ConfigureProject, with the variables' development values written to .env first. See Development values. |
RepairDatabaseConnections({ dryRun? }) | The configuration change, or a success with no file when nothing is missing. See Repair a connection made before. |
GetDatabaseConnection(connection) | { wiring, details? }: the driver, the tables modules and the variables wired for the connection, and its URL's parts without the password. not_found when no module declares it. |
SetDatabaseConnectionUrl(connection, url) | { variable }, the variable the URL was written to. A URL with no password keeps the current one. |
RemoveDatabaseConnection(connection, { dryRun? }) | The configuration change. See Remove a connection. |
ListEntryConnections(module) | The interface-database and interface-database-catalog bindings of one module entry as antelope.config.ts writes them, { interface, source, id?, access? } each, whether core loads that module or not, a disabled: true entry included. access is read-write only when the dms-database entry's config.externalConnections.<id>.access says so. not_found when no entry has that id, and invalid_config for an entry, importOverrides or binding written as a variable or a spread. It writes nothing. |
GetLatestAnalysis(connection) | The connection's running analysis, or else the last to finish. not_found when none ran since the start. |
SampleDatabaseRows(connection, { relation, columns?, limit }) | The rows the provider's SampleRows answers. not_found for an unknown connection, and invalid_config with the provider's message when it refuses. |
Check a connection before writing it
CheckDatabaseConnection tests a database before anything is written to antelope.config.ts or .env. The driver module is not configured at that point, so the builder calls the probe its package exports from dist/probe.js: @antelopejs/postgres, @antelopejs/mysql, @antelopejs/mongodb or @antelopejs/rethinkdb. The probe signs in and decides what the account may read and write with the driver's own rule, the one its catalog applies, and its types are Probe, ProbeConfig and ProbeAnswer of @antelopejs/interface-database-catalog. The builder requires the driver only when its engine is checked, first from the DMS and then from the project. When neither installs a release of the driver that exports probe, the answer is reachable: false with a reason such as Install @antelopejs/postgres to connect a PostgreSQL database.
DATABASE_ENGINE_SCHEMES lists the engines the check takes, each with the URL schemes it accepts, such as ["postgres:", "postgresql:"] for postgres. A form that offers engines can compare its own list with it.
Outside development the call fails with unsupported and opens nothing, since it connects to a host the caller names. An engine other than postgres, mysql, mongodb or rethinkdb, or a url whose scheme is not that engine's, fails with invalid_config, naming the fields and never their values.
The answer reads as follows:
reachableistruewhen a server answered, even to refuse the sign-in.latencyMsis the round trip of the first query of the signed-in account, anduseris the account the server reports.canWriteandcanReadare the driver's answer, as it is.canWriteistruewhen the account may insert, update or delete in any relation it can see, andcanReadwhen it may read any of them, or when it sees none. A MongoDB server without access control gives full privileges, whatever the database holds.reasonis a short sentence for the person who connects, present when the database is unreachable, refuses the sign-in or cannot be read. It never holds the URL, the password or a query a driver echoes in its message.
mysql checks MySQL and MariaDB alike, with mysql:// URLs, and a grant on some columns of a relation only does not make it readable. A MySQL, MariaDB or RethinkDB URL with no database after the port is answered once the account has signed in, with reachable: true, user, no access and a reason saying the URL names no database.
The check ends within three times the connect timeout of five seconds. When the deadline passes first, it answers that the database did not answer in time and aborts the probe's signal, on which the probe closes every connection it opened. A probe never waits for a server to close its end before it answers, and changes no row: for RethinkDB it deletes a key no row has to learn whether the account may write. An answer that is not a ProbeAnswer is reachable: false with a reason saying the driver answered in a way the check does not understand.
What connecting a database writes
A connect flow plans one ConfigureProject batch from GetDatabaseProject(). Its answer names self, the DMS's id in antelope.config.ts, ownDatabase, the project's default interface-database provider, databasePins, the modules to pin to it, and dmsDatabase, the id of the dms-database entry. ownDatabase and dmsDatabase are absent when the project has no such database or module. interfaceDatabase is the version of @antelopejs/interface-database core resolves for the project, and drivers gives each engine's driver package with a range of only its releases built for that version, such as >=0.4.1 <0.5.0 for @antelopejs/postgres on 0.4.2. A range starts at the driver's first release that exports probe and never reaches a release built for another line of the interface, which the project could not start with. An engine with no driver built for that version is absent from drivers.
GetDatabaseProject fails with invalid_config when the configuration does not say plainly which database is the default, or when a module that imports interface-database without providing it, or dms-database itself, is written as a version string or with a spread or a member under a computed name, so the builder cannot write its pin. An entry with a spread that pins interface-database in an importOverrides written after its last spread counts as pinned, since a later key wins.
The batch writes these steps in order:
AddModulefor the engine's driver, at the package and rangedriversgives, with its URL read from a variable.CreateLocalModulefor the module that holds the tables, pinned to the driver.ConfigureModuleonself, declaring the catalog connection under the connection's id.ConfigureModuleondms-database, when the project has it and an own database. For a connectionerpserved by the driverbuilder-db-erpover a project whose own database ismongodb, the entry then reads:const modules = { "dms-database": { importOverrides: [ { interface: "@antelopejs/interface-database", source: "mongodb" }, { interface: "@antelopejs/interface-database", source: "builder-db-erp", id: "erp" }, { interface: "@antelopejs/interface-database-catalog", source: "builder-db-erp", id: "erp" }, ], config: { externalConnections: { erp: { access: "read" } } }, }, };
The firstinterface-databaseentry without an id is the binding a plain import reaches, so the own database comes first. The connection keeps one id on both interfaces and inexternalConnections.- One
ConfigureModulefor each module indatabasePins. Once a secondinterface-databaseprovider exists, core refuses to start a module that importsinterface-databasewithout a pin, so each one that does not provide it gets a pin to the own database.
ConfigureModule writes an override without an id ahead of the first named connection of its interface, and merges a nested config object key by key, so the connections already registered with dms-database stay. It replaces a scalar or an array held where it writes an object. A shorthand member of the key it writes, such as url in config: { url }, is replaced by an assignment when it writes a scalar or an array there, and the other members stay. It refuses the step with invalid_config where what it would replace is held in a way it cannot read, and writes nothing. That covers an object written into a shorthand member, whose value is a variable; a config written as a variable, such as config: SHARED_CONFIG, or as a shorthand config; an object with a spread, a variable or a call where it writes an object; and an object that holds a member under a computed name, a method or an accessor, since that member may be the key it writes. The refusal names the module and the key path, such as config of module "dms-database", says it is held in a way the builder cannot read, and asks for a plain object literal with plain names there.
Development values
ConfigureProject(steps, { variables: [{ name, value }] }) writes each variable's development value into the project's .env, once a dry run has accepted the steps, then writes the configuration. Each variable must be one the steps map in their env. When .env refuses one, nothing is written, and when the configuration then fails, .env goes back to what it held. .env must be ignored by git. With dryRun, the variables are not written.
.env also refuses a variable that the environment which started the project sets to another value, such as one exported in its shell, because the project keeps that value over .env. The refusal names the variable, so the connect flow can say what to unset instead of waiting for a value the project never loads. SetDatabaseConnectionUrl refuses the same way.
Repair a connection made before
A project connected before the steps above has the catalog connection and nothing dms-database reads. RepairDatabaseConnections() adds what is missing for every catalog connection declared on the DMS's entry. It writes the dms-database overrides and the pins of step 5, and never access, so a value set in the dms-database console is kept. With { dryRun: true } it answers the changes without writing them. It refuses as GetDatabaseProject does.
A configuration that already holds everything answers { ok: true, data: { files: [] }, changes: [] } and is not written, so a second call changes nothing.
Next steps
- External databases shows the
ConfigureProjectsteps that wire a database, then its analysis and mapping. - Results and errors shows how to branch on what every operation answers.