Docs › Administer

Upgrading and licensing

Install a new release safely, and what the licence checks.

This page shows how to install a new KRISID release on a repository people are already using, without losing anything they have done. It also explains what the licence checks and what to do when the screen reports a problem with it.

Before you start

• You need a terminal on the server KRISID runs on, in the folder that holds the application.
• The upgrade takes a database backup with pg_dump, so pg_dump must be installed and able to reach the database. If the backup fails, the upgrade stops before changing anything.
• Pick a quiet time. The application is stopped while you upgrade.

Important: On a repository somebody is using, never run bash run.sh setup. It rebuilds the database from nothing, and every hand edit, saved view, connection and workflow is lost. Use bash run.sh upgrade, described below.

Upgrade to a new release

1. Stop the application. If you started it with bash run.sh app, press Ctrl+C in that terminal. If it runs as a service, stop the service.
2. Copy the new release's files over the old ones: the program files, the static, help and migrations folders, schema.sql, requirements.txt and run.sh. Don't delete the backups folder, your licence file, or any key file beside the application.
3. Install any new libraries the release needs with .venv/bin/pip install -r requirements.txt.
4. Run bash run.sh upgrade --dry-run to see what the upgrade would do. It changes nothing.
5. Run bash run.sh upgrade.
6. Start the application again with bash run.sh app, then check it with bash run.sh doctor from a second terminal.

Steps 3 to 6, as they are typed:

.venv/bin/pip install -r requirements.txt
bash run.sh upgrade --dry-run
bash run.sh upgrade
bash run.sh app
bash run.sh doctor

bash run.sh upgrade runs .venv/bin/python upgrade.py for you. You can run that directly with the same options if you prefer.

What the upgrade does, and what it prints

In order, the upgrade:

1. counts the rows in every table;
2. prints the database's version and the version this release needs, for example database at v38, this build wants v40, and lists the changes it will apply;
3. backs up the whole database and prints backed up to with the file name;
4. applies each change in turn, printing applied and its name. If one fails, nothing from that change is written, and the upgrade stops and prints the command to go back;
5. adds anything new the product supplies, such as new permissions or settings. It prints new settings, at their defaults and names them. It doesn't overwrite roles, permissions, names or descriptions you have changed, and it says what it left alone;
6. counts again and prints what changed:;
7. takes a second, clean backup and prints backed up again, clean, to with the file name.

It never changes users, passwords, the mail settings, saved views, workflows, or any asset, whether harvested or typed in.

If any of your data has fewer rows afterwards, the upgrade prints SOMETHING IS MISSING:, names each table with its counts before and after, and prints the command to restore the backup. The upgrade is also recorded on the System tab of the activity log.

When it has finished, press the corner at the top right of any screen — it names who the copy is licensed to — and the panel shows the new schema version, for example Schema v83. If it still shows the old number, the application you are looking at is not the one you upgraded.

Before a long upgrade, you can close the repository: People and Persona → Maintenance, tick Show the maintenance page. Everybody else is signed out and sees one page until you switch it off. See People, roles and access.

Backups, and going back

Backups go in the backups folder beside the application, unless your installation sets a different folder with KRISID_BACKUP_DIR. Each upgrade leaves two files, named for the release and the time, one taken before and one after:

krisid-before-v40-20260921-091502.dump
krisid-after-v40-20260921-091544.dump

To put the newest backup back:

bash run.sh upgrade --rollback

To restore a particular one, put its path after --rollback. You must type restore to confirm. Everything done since that backup was taken is lost, including anything typed today. Afterwards, start the application and run bash run.sh doctor.

To check at any time that nothing has gone missing since the last upgrade, run:

bash run.sh upgrade --compare

These backups only cover upgrades. Nightly backups of the database are separate, and your server team should schedule them.

Put back which source catalogued each asset

One-off, after upgrading to v57. Until then, asset.source_system recorded whichever source touched an asset last rather than the one that catalogued it, so a BI tool that merely reported your warehouse tables took them over and "how many assets catalogued from Snowflake" answered with a moving number. New harvests are recorded correctly from v57 onwards; this puts back the rows already affected.

bash run.sh repair-sources --dry-run
bash run.sh repair-sources

It reads each harvest run's record to find out which source added what, newest first, and commits as it goes in batches. It says where it has got to; you can stop it with Ctrl-C and run it again later, and it carries on rather than starting over. The application can stay up. On a repository of 280,000 assets with a hundred runs of history it takes about fifteen seconds.

Nothing waits on it. An asset it has not reached yet simply has the old wrong answer recorded against it; nothing is broken, and no other screen depends on it.

If a harvest or a long import is running while you do this, you may see it say that the asset table is busy and that it is carrying on more slowly. That is deliberate: it gives way rather than making everybody else wait behind it. Stop it and run it again when the repository is quieter if you would rather have it quick.

After a large harvest: build the context layer

Search does not read your assets directly. It reads a context layer KRISID builds for each one — its own words plus the words of what surrounds it — and an asset whose context has not been built yet cannot be found, however plainly it is named. That layer is kept current by itself. It builds fifty assets every five seconds when there is little to do, and five hundred when it is behind — because it can get behind on one keystroke: a description typed on a schema changes the searchable text of every table and column under it, which on a large warehouse is ten thousand assets. At the slower rate that was twenty-five minutes of search giving the old words; it is now about two.

A harvest through KRISID does get picked up by itself, and measured: a backlog drains at a hundred assets a second, so six hundred thousand new assets is under two hours. But there is one case where waiting does not work at all, and it is worth knowing which. Assets that arrived without KRISID writing them — a load straight into the database, a restore that left the context layer out — were never queued, so they are found instead by a background walk looking for assets with no context at all. That walk is a safety net rather than a catch-up: it builds about twenty-five of them each time it laps the whole repository, which on a large one means it will not get through a big block of them in any useful time. rebuild-context is the tool for that, and for any case where you would rather not wait.

The symptom is unmistakable: search goes quiet or slow, and counts come back far too small. This tells you where you stand:

psql krisid -c "SELECT (SELECT count(*) FROM asset) assets, (SELECT count(*) FROM asset_context) searchable;"

If searchable is well below assets:

bash run.sh rebuild-context --dry-run
bash run.sh rebuild-context

It builds about 2,500 assets a second — a quarter of a million in a couple of minutes — commits as it goes, says where it has got to, and can be stopped with Ctrl-C and run again later. The application can stay up.

After upgrading to v64: let search order from its index

One-off, and search is correct without it — only slower on one kind of question.

From v64 the context layer keeps its own copy of the three things a result list is ordered by, so that a question which narrows without naming any words — how many assets catalogued from Snowflake — can take its order from an index instead of sorting everything it matched. On a repository of ten million assets that step goes from about fifteen seconds to under one.

The copies start empty, because filling them is a rewrite of every row of the layer and an upgrade should never do work without a bound. Until they are filled, search works out the order the way it always has — the same order, the same answers, a slower route. To fill them:

bash run.sh rebuild-context --everything

It says when it is done, and turns the index order on by itself once every asset has been built. It is batched, it can be stopped with Ctrl-C and run again, and the application can stay up. On a repository of a quarter of a million assets it takes a couple of minutes.

To see where you stand:

psql krisid -c "SELECT value FROM repository_setting WHERE key = 'search_order_index';"

off means the order is being worked out the old way; on means it is coming from the index.

After upgrading to v65: a run can be stopped

Nothing to run. The upgrade adds two columns to the run record and one word to what a run's status may be, and a Stop this run button appears on any run in flight. It asks the run to stop rather than killing it — whatever a phase had committed stays committed — and records who asked. See Connections and harvesting.

After upgrading to v68 and v69: search and filters stop reading the whole estate

Nothing to run. These two add a function to the database and change what four screens ask it.

A Data Domain filter, a saved view narrowed to a domain, and the domain chip that appears beside every search result all used to work the answer out by walking membership across the whole repository and then keeping the part that was asked for. On a repository holding a few thousand assets you would not notice. On a large one you would, and the slowest case was the one that comes up most: a filter naming a domain that nothing is assigned to yet, where there is no earlier row to stop at.

Measured on a repository of ten million assets: a search took 7.7 seconds before and 0.8 after; asking a question, 7.4 and 0.7; a filter on an empty domain, 5.6 and 0.02. The answers are unchanged, and verify proves that over every asset — the numbers on a roll-up card and the rows the card opens still cannot disagree.

If you have saved views filtered on a domain, they need nothing done to them.

After upgrading to v70: the asset list stops reading the whole repository

Nothing to run. The upgrade adds one index and the list uses it.

Lists of assets — the plain list behind every type, and the fallback when a screen has no view of its own — were ordered by the type's position in the sidebar and then by name. That is two tables' worth of sort key, which no index can serve, so the list read every asset in the repository and sorted it to show a page of fifty. On a repository of ten million assets that took 6.3 seconds; it now takes 0.04, and a page of two thousand takes 0.08. The rows are the same, and verify compares them row for row.

One visible change, and it is a fix: where two assets of the same type share a name — two columns both called CHANNEL — they used to swap places between identical requests. They now come back in the same order every time.

The roll-up screen also stopped sending every member of every domain back and forth to count them; it counts in the database. On a large repository that screen is still slow, and the reason is recorded in the design notes as NEXT 220: it needs the same kind of maintained layer that search has, which is a piece of work in its own right and not part of this release.

After upgrading to v71: the screens stop reading the whole repository

Nothing to run. This release is almost entirely about how long screens take on a large repository, and the answers they give are unchanged — verify compares them against the old ones row for row.

Measured on a repository of ten million assets:

People and Persona is the one to know about. Every tab of that screen — even Teams, even Roles — used to fetch the Inheritance list along with everything else, and on a harvested estate that list is every table and every schema you have. So Teams was slow because of a list Teams does not show. The Inheritance tab now loads a page at a time, with a box to narrow it to one kind of asset or by name, and a Show the next 200 button. The number on the tab stops being exact past 500 and says 500+.

Some screens are now cached for a few minutes — the dashboard's counts, the metamodel, the semantic model and its YAML. They are summaries of the whole repository, and they are rebuilt as soon as anything in the repository changes, including a setting: you will not see a stale number after an edit. The first person to open one after a change waits for it; everybody after that does not. How much of the search layer is built is deliberately not cached that way, so the progress figure still moves while a rebuild is running.

After upgrading to v74: lineage and the clear-up screen

Nothing to run. Three migrations, all of them about how long a screen takes; the answers are unchanged and verify compares them against the old ones.

The two lineage views that did not load at all on a large repository are the ones to know about: the grain toggle at the top of a lineage diagram now works on an estate of any size. If you had concluded that the business view was broken, it was — it is not now.

On a very large repository the technical view still took a couple of seconds the first time, and on a very large one it was far worse than that. v82 fixes it — see below.

After upgrading to v80 and v81: the metamodel stops disturbing the estate

Nothing to run. Two migrations, both about what happens when somebody saves while somebody else is saving.

Adding or moving an asset type no longer rebuilds every asset of that type. Changing a type's name does, and should — the name is part of the words an asset is found by. Changing where the type sits in the list is one number, and it used to queue the whole estate anyway: on a 275,139-asset repository simply creating a type put 274,872 assets into the rebuild queue and the builder spent five minutes writing text that was already identical. On a repository of 600,000 that is twelve minutes. Worse, while it caught up the ordering copy was behind the metamodel, so the same question could come back in two different orders depending on which reading was in use. From v80 the new number is carried to the context layer by the background builder, in batches, and the ordering is consistent at every moment in between. You will see it finish within seconds of the save; the Rebuilding figure on the dashboard no longer jumps to the size of your repository when all you did was reorder a list.

Two people editing at the same time no longer collide. Everything that changes anything tells the search layer which assets to build again, and that note was being written in whatever order the database happened to produce — so two large edits landing together could each be waiting for the other, and Postgres would end one of them. On a 613,461-asset repository that showed up as a bulk link of ninety tables failing outright with nothing saved. From v81 the note is always written in the same order, and the background builder takes only rows nobody else is holding, so it never makes a request wait. If two edits really do touch the same things in the same instant, one of them now comes back with "Somebody else was changing the same things at the same moment, so nothing was written. Try that again" — a sentence and an intact repository, rather than a red box.

If an upgrade is interrupted

Each migration runs inside a single transaction and is recorded only when it finishes. Pressing Ctrl-C part way through rolls that migration back completely — the database stays on the version it was, nothing is half applied, and the backup taken beforehand is untouched.

Afterwards the application will refuse to serve, because it expects a newer schema than the database has. It says which version each is on, and what to do: run bash run.sh upgrade again. It does not tell you to run setup, and you should not — that rebuilds the database from nothing.

Stop the application before upgrading if you can. Some migrations need a table to themselves for an instant, and they would rather refuse after ten seconds with a message than queue behind a running query.

The licence

If a licence is wrong, write to contact@krisid.com with what the corner panel shows.

Each copy of KRISID comes with a licence file that names your organisation and the web addresses (hostnames) it may be used on. A name can be exact, such as metadata.example.com, or cover a whole domain, such as *.uat.example.com. The corner of each screen shows licensed to and your organisation's name.

To see the licence from the server:

bash run.sh licence

It prints who the licence is for, which hosts it covers, when it expires, and whether any shipped file has been changed.

What happens when something is wrong:

• Wrong address: if KRISID is opened at a hostname the licence doesn't list, it normally refuses, and the message names the licensed hosts. If the installation has moved, ask KRISID for a licence that names the new address. Opening it on the server itself, as localhost, always works, so you can't be locked out of your own machine.
• Expired: a banner says so, and everything keeps working.
• Edited: the licence is signed. If anyone changes it, even one hostname, the banner says the licence has been edited since it was issued, or it was not issued by KRISID. Don't edit the file. Ask for a new one.
• Missing: a banner says There is no licence file.

KRISID reads the licence when it starts. After you replace the licence file, restart the application.

Changed files

Each release also comes with a signed list of its files. bash run.sh licence compares the files on disk against it and names any that have been changed or are missing. This is reported, not blocked: a changed file is sometimes a support fix applied on purpose. It means nobody can change the product quietly. After an upgrade, no files should be listed unless you have been told to expect them.

Good to know

• --dry-run never writes anything. Run it first every time.
• If the upgrade stops and says a change that was already applied has since been edited, don't work around it. Contact whoever supplied the release.
• The mail password and every source's credentials are encrypted with a key kept outside the database: the KRISID_SECRET_KEY environment variable, or the .krisid_key file beside the application. Back that key up separately. Without it, a restored database cannot open them, and you will type the mail password and each source's credentials in again. Credentials saved by an earlier release are encrypted the first time this release starts.

Related

• The activity log
• The metamodel
• Troubleshooting and FAQ

After upgrading to v82: a diagram costs what it draws

Nothing to run. One migration, about how long a lineage diagram takes, and the answers are unchanged — verify compares every edge and every node against the old ones, row for row.

Measured on a repository of 613,461 assets, drawing a picture of 232 assets and 303 arrows:

The cause is worth knowing if you run a large repository, because it explains why the diagram used to get slower over time without anybody changing it. The walk held every lineage link in the whole repository in memory and then filtered that list on each step outward. So the cost of drawing one picture was set by the size of your repository rather than by the size of the picture, and every harvest that added links made every diagram slower. It now looks up only the links on the assets it is actually drawing.

The object and business views are unchanged, and were already fixed in v74.

If you had learned to avoid the lineage diagram on a big estate, stop avoiding it.

After upgrading to v83: who can see usernames and email addresses

Nothing to run, and nobody who was administering people loses anything — but one thing changes on a screen, so read this before somebody asks.

People and Persona used to show every person's Username and Email to anybody who could sign in. From v83 those two columns, and the Download the current access model button, need a new permission: See usernames and email addresses (PERSON_IDENTIFY). The rest of the screen is unchanged and still open to everybody — names, teams, roles, who is accountable for what — because that is how people find out who to ask.

Who has it after the upgrade. Every role that already held Manage users and roles (ACCESS_MANAGE) was given it, whatever that role is called in your repository. So whoever administers people carries on exactly as before. It is one tick on the Permissions tab if you want somebody else to have it, and one tick to take it away again.

What somebody without it now sees. The two columns are not there, and a line under the tabs says which permission shows them. A person's page says hidden where their address would be. The workbook download is replaced by the reason — the workbook is keyed on usernames, so there is no sensible copy of it with them removed. Their own username and address are still on their own profile, which is where they need them.

What has not changed. Usernames still appear in the person picker, in Viewing as, beside an approval and in the activity log. Email addresses still appear in the mail log on the Logs screen, which needs Read the logs (LOG_VIEW). If a username is itself sensitive where you work, say so and we will tell you what closing those would cost.

One more small thing: a role carrying the new permission can no longer be the role sign-up hands out. If that was your sign-up role — it almost certainly was not, since only administrators had the permission before now — the sign-up tab will say so and you choose another.