The following code goes in the theme_name.theme file:
```php
/**
* Implements hook_theme_suggestions_page_alter().
*/
function rootid_theme_theme_suggestions_page_alter(array &$suggestions, array $variables) {
$path_args = explode('/', trim(\Drupal::service('path.current')->getPath(), '/'));
$suggestions = theme_get_suggestions($path_args, 'page');
$http_error_suggestions = [
'system.401' => 'page__error_page',
'system.403' => 'page__error_page',
'system.404' => 'page__error_page',
];
$route_name = \Drupal::routeMatch()->getRouteName();
if (isset($http_error_suggestions[$route_name])) {
$suggestions[] = $http_error_suggestions[$route_name];
}
}
```
Then set up a `page--error-page.html.twig` template :) (Note the underscores in the code are transformed into hyphens in the filename)