Heroku Postgres Maintenance
Last updated April 15, 2024
Table of Contents
From time to time, Heroku performs maintenance tasks on a Heroku Postgres database. Typical tasks include updating the underlying infrastructure of the database. For example, patching the operating system or required libraries, or upgrading Postgres itself. Heroku handles these maintenance tasks automatically.
You can also specify your maintenance windows or perform maintenances manually with the Data Maintenance CLI Plugin commands. Install the plugin before running the data:maintenances:*
commands in this article.
There are three ways to perform maintenance:
- Heroku automatically performs maintenance for you.
- You perform maintenance manually with maintenance mode.
- You perform maintenance manually without maintenance mode.
Maintenance windows are available for all Heroku Postgres database plans except for Essential-tier plans.
The Data Maintenance CLI Plugin commands improve and extend the functionality of the pg:maintenance
commands. The pg:maintenance
commands are still available, but Heroku plans to deprecate them for the commands in the Data Maintenance CLI Plugin.
Checking for Required Maintenance
You can check if maintenance is required on a database by using pg:info
:
$ heroku pg:info -a example-app
=== DATABASE_URL
Plan: Standard 4
Status: Available
Data Size: 26.1 MB
...
Maintenance: required by 2018-05-04 21:00:00 +0000
Maintenance: scheduled for 2018-05-01 14:30:00 +0000, required by 2018-05-04 21:00:00 +0000, replacement currently being prepared
Maintenance window: Tuesdays 14:30 to 18:30 UTC
You can also check with the data:maintenances:info
command:
$ heroku data:maintenances:info DATABASE -a example-app
Fetching maintenance... done
addon_attachments: DATABASE_URL
addon_description: Standard Non-HA
addon_kind: heroku-postgresql
addon_name: postgresql-octagonal-1234
addon_plan: standard-0
addon_window: Mondays 14:30 to 18:30 UTC
app_name: example-app
method: changeover
reason: routine_maintenance
required_by: 2023-06-30T10:10:13.831+00:00
scheduled_for: 2023-06-12T14:30:00.000+00:00
server_created_at: 2023-05-10T14:52:33.871+00:00
status: preparing
window: Mondays 14:30 to 18:30 UTC
Heroku Postgres databases are updated regularly to maintain important security patches in Postgres and the underlying software and hardware. Maintenance tasks occur at least once every 90 days.
Setting a Maintenance Window
When you provision a database, the default maintenance window is set to a time between Monday to Friday and 18:00 to 23:00 UTC (10:00 to 15:00 PT). You can specify a maintenance window for Standard, Premium, Private, and Shield Heroku Postgres plans. You can specify the day of the week and time in UTC when the window begins:
$ heroku data:maintenances:window:update DATABASE Sunday 14:30 -a example-app
Setting a maintenance window minimizes the impact of the maintenance on your application and users. Heroku recommends selecting a time when a maintenance event has the smallest impact on your business.
You receive an email letting you know the time of your upcoming maintenance window. You can verify when the time with the heroku data:maintenances:info
command. This command indicates when your maintenance event is scheduled and if it’s ready to run.
Maintenance windows are 4 hours long. Heroku attempts to begin the maintenance as close to the beginning of the window as possible. The duration of the maintenance event varies, but usually, your database is only offline for 10–60 seconds.
The window you specify must end before the required by
time indicated by the heroku data:maintenances:info
command.
Rescheduling the Maintenance Window
You can change your database’s maintenance window by running the data:maintenances:window:update
command again and providing a different start time for the window.
Performing Maintenance Manually
Before you perform maintenance manually, it’s recommended that you first put your database’s associated app in maintenance mode. Install the plugin before running the data:maintenances:*
commands in this article.
Automatically
The most common way to perform maintenance is to let Heroku do it for you. We use your database’s maintenance window to determine the best time to perform maintenance automatically. Your app restarts as part of the maintenance event, so some users can see errors or delays for a few minutes.
You can see errors, such as connection error
, pg is read only
, and so on in your logs. These errors are artifacts of the changeover process and are safe to ignore as long as everything comes back online.
With Maintenance Mode Enabled
The following commands demonstrate putting an app in maintenance mode and then manually performing maintenance on its associated database:
$ heroku maintenance:on -a example-app
Enabling maintenance mode for ⬢ example-app... done
$ heroku data:maintenances:run DATABASE -a example-app
Starting maintenance for postgresql-clean-29349... done
$ heroku maintenance:off -a example-app
Disabling maintenance mode for ⬢ example-app... done
This method involves putting your application into maintenance mode before performing the PostgreSQL failover. This method can have benefits such as providing a clean landing page for your users, but total downtime can be longer as it’s a manual process to enable and disable maintenance mode.
Without Maintenance Mode Enabled
Performing maintenance manually without first enabling maintenance mode requires the --confirm
flag with the app name:
$ heroku data:maintenances:run DATABASE --confirm example-app
This command starts the maintenance event immediately.
If you’ve built a read-only
or database-independent
mode into your app, you can use that as well:
# for example
heroku config:set READ_ONLY=true
heroku data:maintenances:run --confirm example-app
heroku pg:wait
heroku config:set READ_ONLY=false
Maintenance Types
When performing database maintenance tasks, Heroku uses one of several strategies, such as a restart, a changeover, or a failover. The strategy Heroku uses depends on the type of maintenance task and the database plan.
For example, for high-availability (HA) plans (Premium, Private, and Shield), Heroku uses a failover strategy whenever possible, unless the database is the follower of another database. For non-HA plans (Standard) and all follower databases regardless of plan, Heroku usually uses a changeover strategy.
If your database has one or more followers and a failover or changeover maintenance event occurs, Heroku attempts to point the followers to the replacement database. Pointing the followers to the replacement requires restarting the follower database.
In some cases, if we’re unable to point the follower at the replacement, we automatically prepare and promote a replacement database.
Monitoring and Initiating Changeover Maintenance Events
When Heroku schedules a changeover maintenance event for your database, it begins creating a replacement database. Creating a replacement takes time depending on the size of your database.
While the replacement is still being created, the data:maintenances:info
command displays the following:
$ heroku data:maintenances:info DATABASE --app example-app
Fetching maintenance... done
addon_attachments: DATABASE_URL
addon_description: Standard Non-HA
addon_kind: heroku-postgresql
addon_name: postgresql-octagonal-41284
addon_plan: standard-0
addon_window: Mondays 14:30 to 18:30 UTC
app_name: example-app
method: changeover
reason: customer_request
required_by: 2023-06-30T10:10:13.831+00:00
scheduled_for: 2023-06-12T14:30:00.000+00:00
server_created_at: 2023-05-10T14:52:33.871+00:00
status: preparing
window: Mondays 14:30 to 18:30 UTC
This info isn’t shown when the maintenance uses a failover or restart strategy.
When the replacement is ready and the maintenance can proceed, the data:maintenances:info
command shows the following:
$ heroku data:maintenances:info DATABASE --app example-app
Fetching maintenance... done
addon_attachments: DATABASE_URL
addon_description: Standard Non-HA
addon_kind: heroku-postgresql
addon_name: postgresql-octagonal-41284
addon_plan: standard-0
addon_window: Mondays 14:30 to 18:30 UTC
app_name: example-app
method: changeover
reason: customer_request
required_by: 2023-06-30T10:10:13.831+00:00
scheduled_for: 2023-06-12T14:30:00.000+00:00
server_created_at: 2023-05-10T14:52:33.871+00:00
status: ready
window: Mondays 14:30 to 18:30 UTC
Limitations
You can only set maintenance windows and manually perform maintenance on Standard, Premium, Private, and Shield plans.
Heroku makes the best effort to honor your maintenance window request but it isn’t guaranteed. If there’s an emergency where the security or integrity of your data is threatened, we can perform maintenance outside of your regular window at our discretion.