__ __ / \ / \ ______ _ _ __ __ __ / /\ / /\ \ / __ \ | | | | / \ \ \ / / / / / / \ \ | | | | | | | | / /\ \ \ / \ \ \ \ / / | |__| | | |__| | / ____ \ | | \ \/ \ \/ / \_ ___/ \____/ /_/ \_\ |_| \__/ \__/ \ \__ \___\ by Red Hat Build, Store, and Distribute your Containers Startup timestamp: Mon Jul 13 05:53:32 PM UTC 2026 Running all default registry services Running init script '/quay-registry/conf/init/certs_install.sh' Running init script '/quay-registry/conf/init/client_certs.sh' Running init script '/quay-registry/conf/init/copy_config_files.sh' Running init script '/quay-registry/conf/init/d_validate_config_bundle.sh' Validating Configuration plpgsql pg_trgm +------------------------+-------+--------+ | Field Group | Error | Status | +------------------------+-------+--------+ | AccessSettings | - | 🟒 | +------------------------+-------+--------+ | ActionLogArchiving | - | 🟒 | +------------------------+-------+--------+ | AppTokenAuthentication | - | 🟒 | +------------------------+-------+--------+ | AutoPrune | - | 🟒 | +------------------------+-------+--------+ | BitbucketBuildTrigger | - | 🟒 | +------------------------+-------+--------+ | BuildManager | - | 🟒 | +------------------------+-------+--------+ | Database | - | 🟒 | +------------------------+-------+--------+ | DistributedStorage | - | 🟒 | +------------------------+-------+--------+ | ElasticSearch | - | 🟒 | +------------------------+-------+--------+ | Email | - | 🟒 | +------------------------+-------+--------+ | GitHubBuildTrigger | - | 🟒 | +------------------------+-------+--------+ | GitHubLogin | - | 🟒 | +------------------------+-------+--------+ | GitLabBuildTrigger | - | 🟒 | +------------------------+-------+--------+ | GoogleLogin | - | 🟒 | +------------------------+-------+--------+ | HostSettings | - | 🟒 | +------------------------+-------+--------+ | JWTAuthentication | - | 🟒 | +------------------------+-------+--------+ | LDAP | - | 🟒 | +------------------------+-------+--------+ | OIDC | - | 🟒 | +------------------------+-------+--------+ | QuayDocumentation | - | 🟒 | +------------------------+-------+--------+ | Redis | - | 🟒 | +------------------------+-------+--------+ | RepoMirror | - | 🟒 | +------------------------+-------+--------+ | SecurityScanner | - | 🟒 | +------------------------+-------+--------+ | TeamSyncing | - | 🟒 | +------------------------+-------+--------+ | TimeMachine | - | 🟒 | +------------------------+-------+--------+ | UserVisibleSettings | - | 🟒 | +------------------------+-------+--------+ Running init script '/quay-registry/conf/init/nginx_conf_create.sh' Running init script '/quay-registry/conf/init/runmigration.sh' fatal: not a git repository (or any of the parent directories): .git 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. fatal: not a git repository (or any of the parent directories): .git 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. 17:53:37 INFO [alembic.runtime.migration] Context impl PostgresqlImpl. 17:53:37 INFO [alembic.runtime.migration] Will assume non-transactional DDL. 17:53:37 INFO [alembic.runtime.migration] Running upgrade -> c156deb8845d, Reset our migrations with a required update. 17:53:38 INFO [alembic.runtime.migration] Running upgrade c156deb8845d -> 6c7014e84a5e, Add user prompt support. 17:53:38 INFO [alembic.runtime.migration] Running upgrade 6c7014e84a5e -> faf752bd2e0a, Add user metadata fields. 17:53:38 INFO [alembic.runtime.migration] Running upgrade faf752bd2e0a -> 94836b099894, Create new notification type. 17:53:38 INFO [alembic.runtime.migration] Running upgrade 94836b099894 -> 45fd8b9869d4, add_notification_type. 17:53:38 INFO [alembic.runtime.migration] Running upgrade 45fd8b9869d4 -> f5167870dd66, update queue item table indices. 17:53:39 INFO [alembic.runtime.migration] Running upgrade f5167870dd66 -> fc47c1ec019f, Add state_id field to QueueItem. 17:53:39 INFO [alembic.runtime.migration] Running upgrade fc47c1ec019f -> 3e8cc74a1e7b, Add severity and media_type to global messages. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 3e8cc74a1e7b -> d42c175b439a, Backfill state_id and make it unique. 17:53:39 INFO [alembic.runtime.migration] Running upgrade d42c175b439a -> e2894a3a3c19, Add full text search indexing for repo name and description. 17:53:39 INFO [alembic.runtime.migration] Running upgrade e2894a3a3c19 -> 7a525c68eb13, Add OCI/App models. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 7a525c68eb13 -> b8ae68ad3e52, Change BlobUpload fields to BigIntegers to allow layers > 8GB. 17:53:39 INFO [alembic.runtime.migration] Running upgrade b8ae68ad3e52 -> b4df55dea4b3, add repository kind. 17:53:39 INFO [alembic.runtime.migration] Running upgrade b4df55dea4b3 -> a6c463dfb9fe, back fill build expand_config. 17:53:39 INFO [alembic.runtime.migration] Running upgrade a6c463dfb9fe -> be8d1c402ce0, Add TeamSync table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade be8d1c402ce0 -> f30984525c86, Add RepositorySearchScore table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade f30984525c86 -> c3d4b7ebcdf7, Backfill RepositorySearchScore table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade c3d4b7ebcdf7 -> ed01e313d3cb, Add trust_enabled to repository. 17:53:39 INFO [alembic.runtime.migration] Running upgrade ed01e313d3cb -> 53e2ac668296, Remove reference to subdir. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 53e2ac668296 -> dc4af11a5f90, add notification number of failures column. 17:53:39 INFO [alembic.runtime.migration] Running upgrade dc4af11a5f90 -> d8989249f8f6, Add change_tag_expiration log type. 17:53:39 INFO [alembic.runtime.migration] Running upgrade d8989249f8f6 -> 7367229b38d9, Add support for app specific tokens. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 7367229b38d9 -> cbc8177760d9, Add user location field. 17:53:39 INFO [alembic.runtime.migration] Running upgrade cbc8177760d9 -> 152bb29a1bb3, Add maximum build queue count setting to user table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 152bb29a1bb3 -> c91c564aad34, Drop checksum on ImageStorage. 17:53:39 INFO [alembic.runtime.migration] Running upgrade c91c564aad34 -> 152edccba18c, Make BlodUpload byte_count not nullable. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 152edccba18c -> b4c2d45bc132, Add deleted namespace table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade b4c2d45bc132 -> 61cadbacb9fc, Add ability for build triggers to be disabled. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 61cadbacb9fc -> 17aff2e1354e, Add automatic disable of build triggers. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 17aff2e1354e -> 87fbbc224f10, Add disabled datetime to trigger. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 87fbbc224f10 -> 0cf50323c78b, Add creation date to User table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 0cf50323c78b -> b547bc139ad8, Add RobotAccountMetadata table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade b547bc139ad8 -> 224ce4c72c2f, Add last_accessed field to User table. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 224ce4c72c2f -> 5b7503aada1b, Cleanup old robots. 17:53:39 INFO [util.migrate.cleanup_old_robots] Found 0 robots to delete 17:53:39 INFO [alembic.runtime.migration] Running upgrade 5b7503aada1b -> 1783530bee68, Add LogEntry2 table - QUAY.IO ONLY 17:53:39 INFO [alembic.runtime.migration] Running upgrade 1783530bee68 -> 5cbbfc95bac7, Remove 'oci' tables not used by CNR. The rest will be migrated and renamed. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 5cbbfc95bac7 -> 610320e9dacf, Add new Appr-specific tables. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 610320e9dacf -> 5d463ea1e8a8, Backfill new appr tables. 17:53:39 INFO [alembic.runtime.migration] Running upgrade 5d463ea1e8a8 -> d17c695859ea, Delete old Appr tables. 17:53:39 INFO [alembic.runtime.migration] Running upgrade d17c695859ea -> 6c21e2cfb8b6, Change LogEntry to use a BigInteger as its primary key. /opt/app-root/lib64/python3.12/site-packages/alembic/ddl/postgresql.py:187: UserWarning: autoincrement and existing_autoincrement only make sense for MySQL super().alter_column( 17:53:40 INFO [alembic.runtime.migration] Running upgrade 6c21e2cfb8b6 -> 9093adccc784, Add V2_2 data models for Manifest, ManifestBlob and ManifestLegacyImage. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 9093adccc784 -> eafdeadcebc7, Remove blob_index from ManifestBlob table. 17:53:40 INFO [alembic.runtime.migration] Running upgrade eafdeadcebc7 -> 654e6df88b71, Change manifest_bytes to a UTF8 text field. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 654e6df88b71 -> 13411de1c0ff, Remove unique from TagManifestToManifest. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 13411de1c0ff -> 10f45ee2310b, Add Tag, TagKind and ManifestChild tables. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 10f45ee2310b -> 67f0abd172ae, Add TagToRepositoryTag table. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 67f0abd172ae -> c00a1f15968b 17:53:40 INFO [alembic.runtime.migration] Running upgrade c00a1f15968b -> 54492a68a3cf, Add NamespaceGeoRestriction table. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 54492a68a3cf -> 6ec8726c0ace, Add LogEntry3 table. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 6ec8726c0ace -> e184af42242d, Add missing index on UUID fields. 17:53:40 INFO [alembic.runtime.migration] Running upgrade e184af42242d -> b9045731c4de, Add lifetime end indexes to tag tables. 17:53:40 INFO [alembic.runtime.migration] Running upgrade b9045731c4de -> 481623ba00ba, Add index on logs_archived on repositorybuild. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 481623ba00ba -> b918abdbee43, Run full tag backfill. 17:53:40 INFO [alembic.runtime.migration] Running upgrade b918abdbee43 -> 5248ddf35167, Repository Mirror. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 5248ddf35167 -> c13c8052f7a6, Add new fields and tables for encrypted tokens. 17:53:40 INFO [alembic.runtime.migration] Running upgrade c13c8052f7a6 -> 703298a825c2, Backfill new encrypted fields. 17:53:40 INFO [703298a825c2_backfill_new_encrypted_fields_py] Backfilling encrypted credentials for access tokens 17:53:40 INFO [703298a825c2_backfill_new_encrypted_fields_py] Backfilling encrypted credentials for robots 17:53:40 INFO [703298a825c2_backfill_new_encrypted_fields_py] Backfilling encrypted credentials for repo build triggers 17:53:40 INFO [703298a825c2_backfill_new_encrypted_fields_py] Backfilling encrypted credentials for app specific auth tokens 17:53:40 INFO [703298a825c2_backfill_new_encrypted_fields_py] Backfilling credentials for OAuth access tokens 17:53:40 INFO [703298a825c2_backfill_new_encrypted_fields_py] Backfilling credentials for OAuth auth code 17:53:40 INFO [703298a825c2_backfill_new_encrypted_fields_py] Backfilling secret for OAuth applications 17:53:40 INFO [alembic.runtime.migration] Running upgrade 703298a825c2 -> 49e1138ed12d, Change token column types for encrypted columns. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 49e1138ed12d -> c059b952ed76, Remove unencrypted fields and data. 17:53:40 INFO [alembic.runtime.migration] Running upgrade c059b952ed76 -> cc6778199cdb, repository mirror notification. 17:53:40 INFO [alembic.runtime.migration] Running upgrade cc6778199cdb -> 34c8ef052ec9, repo mirror columns. 17:53:40 INFO [34c8ef052ec9_repo_mirror_columns_py] Migrating to external_reference from existing columns 17:53:40 INFO [34c8ef052ec9_repo_mirror_columns_py] Reencrypting existing columns 17:53:40 INFO [alembic.runtime.migration] Running upgrade 34c8ef052ec9 -> 4fd6b8463eb2, Add new DeletedRepository tracking table. 17:53:40 INFO [alembic.runtime.migration] Running upgrade 4fd6b8463eb2 -> 8e6a363784bb, add manifestsecuritystatus table 17:53:40 INFO [alembic.runtime.migration] Running upgrade 8e6a363784bb -> 04b9d2191450 17:53:40 INFO [alembic.runtime.migration] Running upgrade 04b9d2191450 -> 3383aad1e992, Add UploadedBlob table 17:53:40 INFO [alembic.runtime.migration] Running upgrade 3383aad1e992 -> 88e64904d000, Add new metadata columns to Manifest table 17:53:40 INFO [alembic.runtime.migration] Running upgrade 88e64904d000 -> 909d725887d3, Add composite index on manifestblob 17:53:40 INFO [alembic.runtime.migration] Running upgrade 909d725887d3 -> e9f3e4dbb979, quota_management_reporting 17:53:40 INFO [alembic.runtime.migration] Running upgrade e9f3e4dbb979 -> 10d34e6536f8, create proxy_cache_config table 17:53:40 INFO [alembic.runtime.migration] Running upgrade 10d34e6536f8 -> b43b44996c26, increase proxy cache config creds max len 17:53:40 INFO [alembic.runtime.migration] Running upgrade b43b44996c26 -> d94d328733e4, Increase repomirrorconfig char size 17:53:40 INFO [alembic.runtime.migration] Running upgrade d94d328733e4 -> 2d66ad598b56, Change metadata_json size in logentry3 table for MySQL 17:53:40 INFO [alembic.runtime.migration] Running upgrade 2d66ad598b56 -> 7bca88923f2c, add proxy cache logentrykind 17:53:40 INFO [alembic.runtime.migration] Running upgrade 7bca88923f2c -> b2d1e4b95fc2, add build triggers' logentrykind 17:53:40 INFO [alembic.runtime.migration] Running upgrade b2d1e4b95fc2 -> f0875fbc7c7a, β€œquota-v2” 17:53:40 INFO [alembic.runtime.migration] Running upgrade f0875fbc7c7a -> a648ab200ab7, add org auditing 17:53:40 INFO [alembic.runtime.migration] Running upgrade a648ab200ab7 -> 43fea47d641f, create RH subscriptions table 17:53:40 INFO [alembic.runtime.migration] Running upgrade 43fea47d641f -> 0246c2d0e750, add login logout logentrykind 17:53:40 INFO [alembic.runtime.migration] Running upgrade 0246c2d0e750 -> ab5a24478052, β€œadd_registry_size_table” 17:53:40 INFO [alembic.runtime.migration] Running upgrade ab5a24478052 -> 8cf90670bf38, add permanently delete tag log entry kind 17:53:40 INFO [alembic.runtime.migration] Running upgrade 8cf90670bf38 -> 46980ea2dde5, change removed_tag_expiration_s to BIGINT 17:53:40 INFO [alembic.runtime.migration] Running upgrade 46980ea2dde5 -> a0fc06d604a4, Unlink deprecated tables 17:53:40 INFO [alembic.runtime.migration] Running upgrade a0fc06d604a4 -> 8a70b8777089, drop deprecated tables 17:53:40 INFO [alembic.runtime.migration] Running upgrade 8a70b8777089 -> b82361fba1cd, Create table for org to sku relations 17:53:40 INFO [alembic.runtime.migration] Running upgrade b82361fba1cd -> 4366f54be43c, add indexes to quotanamespacesize and quotarepositorysize tables 17:53:41 INFO [alembic.runtime.migration] Running upgrade 4366f54be43c -> 222debc16e18, β€œautopruning” 17:53:41 INFO [alembic.runtime.migration] Running upgrade 222debc16e18 -> 2062bbd5ef0e, insert logentrykind 17:53:41 INFO [alembic.runtime.migration] Running upgrade 2062bbd5ef0e -> 8d47693829a0, auto prune policy audit logs 17:53:41 INFO [alembic.runtime.migration] Running upgrade 8d47693829a0 -> 3f8e3657bb67, add login and pull failure logentrykind 17:53:41 INFO [alembic.runtime.migration] Running upgrade 3f8e3657bb67 -> 41d15c93c299, add quantity field to orgRhSkus 17:53:41 INFO [alembic.runtime.migration] Running upgrade 41d15c93c299 -> b4da5b09c8df, repository autoprune policy 17:53:41 INFO [alembic.runtime.migration] Running upgrade b4da5b09c8df -> 135fd3e94615, insert oidc in loginservice 17:53:41 INFO [alembic.runtime.migration] Running upgrade 135fd3e94615 -> cead4183265c, recreate index for repository autoprune policy 17:53:41 INFO [alembic.runtime.migration] Running upgrade cead4183265c -> 36cd2b747a08, team sync logs 17:53:41 INFO [alembic.runtime.migration] Running upgrade 36cd2b747a08 -> 946f0e90f9c9, Create manifestsubject table 17:53:41 INFO [alembic.runtime.migration] Running upgrade 946f0e90f9c9 -> 0cdd1f27a450, add subject backfilled index 17:53:41 INFO [alembic.runtime.migration] Running upgrade 0cdd1f27a450 -> 0988213e0885, add oauth assigned token table 17:53:41 INFO [alembic.runtime.migration] Running upgrade 0988213e0885 -> 66147b81aad2, create repo_image_expiry notification type 17:53:41 INFO [alembic.runtime.migration] Running upgrade 66147b81aad2 -> f67fe4871771, create tag_notification_sent table 17:53:41 INFO [alembic.runtime.migration] Running upgrade f67fe4871771 -> 2664723e1b4b, add last_ran_ms to repository_notification_table 17:53:41 INFO [alembic.runtime.migration] Running upgrade 2664723e1b4b -> 8a7ba94c2e84, add assign token logentry 17:53:41 INFO [alembic.runtime.migration] Running upgrade 8a7ba94c2e84 -> 3f8d7acdf7f9, Add artifact_type column 17:53:41 INFO [alembic.runtime.migration] Running upgrade 3f8d7acdf7f9 -> a32e17bfad20, increase upstreamregistry's password length (PROJQUAY-7430) 17:53:41 INFO [alembic.runtime.migration] Running upgrade a32e17bfad20 -> 5b8dc452f5c3, add tag immutability column 17:53:41 INFO [alembic.runtime.migration] Running upgrade 5b8dc452f5c3 -> ba263f9be4a6, remove unique constraint for autoprunepolicy 17:53:41 INFO [alembic.runtime.migration] Running upgrade ba263f9be4a6 -> 9085e82074f2, Add log event kind for federated robot 17:53:41 INFO [alembic.runtime.migration] Running upgrade 9085e82074f2 -> 8e97c2cfee57, Add export logs auditing 17:53:41 INFO [alembic.runtime.migration] Running upgrade 8e97c2cfee57 -> 3634f2df3c5b, drop unique constraint on org sku table 17:53:41 INFO [alembic.runtime.migration] Running upgrade 3634f2df3c5b -> e8ed3fb547da, alter manifestblob id 17:53:41 INFO [alembic.runtime.migration] Running upgrade e8ed3fb547da -> 1623f40582ed, Add skopeo timeout to mirroring config 17:53:41 INFO [alembic.runtime.migration] Running upgrade 1623f40582ed -> 7078c84d14e8, add_pull_statistics_tables Running init script '/quay-registry/conf/init/supervisord_conf_create.sh' Running init script '/quay-registry/conf/init/zz_boot.sh' fatal: not a git repository (or any of the parent directories): .git 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. /quay-registry/boot.py:76: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). expiration = datetime.utcnow() + timedelta(minutes=minutes_until_expiration) 2026-07-13 17:53:44,067 INFO RPC interface 'supervisor' initialized 2026-07-13 17:53:44,067 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2026-07-13 17:53:44,067 INFO supervisord started with pid 14 2026-07-13 17:53:45,070 INFO spawned: 'stdout' with pid 77 2026-07-13 17:53:45,071 INFO spawned: 'autopruneworker' with pid 78 2026-07-13 17:53:45,073 INFO spawned: 'blobuploadcleanupworker' with pid 79 2026-07-13 17:53:45,075 INFO spawned: 'builder' with pid 80 2026-07-13 17:53:45,077 INFO spawned: 'buildlogsarchiver' with pid 81 2026-07-13 17:53:45,079 INFO spawned: 'chunkcleanupworker' with pid 82 2026-07-13 17:53:45,081 INFO spawned: 'dnsmasq' with pid 83 2026-07-13 17:53:45,083 INFO spawned: 'expiredappspecifictokenworker' with pid 84 2026-07-13 17:53:45,085 INFO spawned: 'exportactionlogsworker' with pid 85 2026-07-13 17:53:45,087 INFO spawned: 'gcworker' with pid 86 2026-07-13 17:53:45,090 INFO spawned: 'globalpromstats' with pid 87 2026-07-13 17:53:45,093 INFO spawned: 'gunicorn-registry' with pid 88 2026-07-13 17:53:45,095 INFO spawned: 'gunicorn-secscan' with pid 89 2026-07-13 17:53:45,097 INFO spawned: 'gunicorn-web' with pid 90 2026-07-13 17:53:45,099 INFO spawned: 'logrotateworker' with pid 91 2026-07-13 17:53:45,102 INFO spawned: 'manifestbackfillworker' with pid 92 2026-07-13 17:53:45,105 INFO spawned: 'manifestsubjectbackfillworker' with pid 93 2026-07-13 17:53:45,107 INFO spawned: 'memcache' with pid 94 2026-07-13 17:53:45,110 INFO spawned: 'namespacegcworker' with pid 95 2026-07-13 17:53:45,168 INFO spawned: 'nginx' with pid 96 2026-07-13 17:53:45,170 INFO spawned: 'notificationworker' with pid 97 2026-07-13 17:53:45,173 INFO spawned: 'proxycacheblobworker' with pid 98 2026-07-13 17:53:45,175 INFO spawned: 'pullstatsredisflushworker' with pid 106 2026-07-13 17:53:45,177 INFO spawned: 'pushgateway' with pid 109 2026-07-13 17:53:45,180 INFO spawned: 'queuecleanupworker' with pid 110 2026-07-13 17:53:45,182 INFO spawned: 'quotaregistrysizeworker' with pid 111 2026-07-13 17:53:45,185 INFO spawned: 'quotatotalworker' with pid 116 2026-07-13 17:53:45,188 INFO spawned: 'reconciliationworker' with pid 117 2026-07-13 17:53:45,191 INFO spawned: 'repositoryactioncounter' with pid 118 2026-07-13 17:53:45,269 INFO spawned: 'repositorygcworker' with pid 119 2026-07-13 17:53:45,272 INFO spawned: 'securityscanningnotificationworker' with pid 120 2026-07-13 17:53:45,274 INFO spawned: 'securityworker' with pid 121 2026-07-13 17:53:45,276 INFO spawned: 'servicekey' with pid 122 2026-07-13 17:53:45,279 INFO spawned: 'storagereplication' with pid 126 2026-07-13 17:53:45,282 INFO spawned: 'teamsyncworker' with pid 128 2026-07-13 17:53:46,372 INFO success: stdout entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: autopruneworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: blobuploadcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: builder entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: buildlogsarchiver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: chunkcleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: dnsmasq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: expiredappspecifictokenworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,372 INFO success: exportactionlogsworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: gcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: globalpromstats entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: gunicorn-registry entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: gunicorn-secscan entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: gunicorn-web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: logrotateworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: manifestbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: manifestsubjectbackfillworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: memcache entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,373 INFO success: namespacegcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,376 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,376 INFO success: notificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,376 INFO success: proxycacheblobworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,376 INFO success: pullstatsredisflushworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: pushgateway entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: queuecleanupworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: quotaregistrysizeworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: quotatotalworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: reconciliationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: repositoryactioncounter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: repositorygcworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: securityscanningnotificationworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: securityworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,377 INFO success: servicekey entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,378 INFO success: storagereplication entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2026-07-13 17:53:46,378 INFO success: teamsyncworker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) dnsmasq stderr | dnsmasq: started, version 2.85 cachesize 150 dnsmasq stderr | dnsmasq: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile dnsmasq stderr | dnsmasq: reading /etc/resolv.conf dnsmasq stderr | dnsmasq: using nameserver 10.96.0.10#53 dnsmasq stderr | dnsmasq: read /etc/hosts - 7 addresses nginx stdout | 2026/07/13 17:53:45 [warn] 96#96: the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:40 nginx stdout | 2026/07/13 17:53:45 [warn] 96#96: the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:67 nginx stdout | 2026/07/13 17:53:45 [warn] 96#96: the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:91 nginx stderr | nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:40 nginx stderr | nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:67 nginx stderr | nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /quay-registry/conf/nginx/nginx.conf:91 pushgateway stderr | ts=2026-07-13T17:53:45.277Z level=info caller=main.go:82 msg="starting pushgateway" version="(version=1.11.1, branch=HEAD, revision=aff0d834b730375fc965135e3e2d17d15f82a76f)" pushgateway stderr | ts=2026-07-13T17:53:45.277Z level=info caller=main.go:83 msg="Build context" build_context="(go=go1.24.1, platform=linux/amd64, user=root@672a818fb093, date=20250409-13:24:09, tags=unknown)" pushgateway stderr | ts=2026-07-13T17:53:45.279Z level=info caller=tls_config.go:347 msg="Listening on" address=[::]:9091 pushgateway stderr | ts=2026-07-13T17:53:45.279Z level=info caller=tls_config.go:350 msg="TLS is disabled." http2=false address=[::]:9091 nginx stdout | 2026/07/13 17:53:45 [alert] 129#129: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 132#132: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 133#133: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 130#130: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 138#138: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 139#139: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 131#131: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 134#134: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 144#144: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 141#141: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 140#140: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 135#135: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 136#136: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 137#137: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 143#143: setpriority(-10) failed (13: Permission denied) nginx stdout | 2026/07/13 17:53:45 [alert] 142#142: setpriority(-10) failed (13: Permission denied) buildlogsarchiver stderr | fatal: not a git repository (or any of the parent directories): .git chunkcleanupworker stderr | fatal: not a git repository (or any of the parent directories): .git logrotateworker stderr | fatal: not a git repository (or any of the parent directories): .git manifestsubjectbackfillworker stderr | fatal: not a git repository (or any of the parent directories): .git builder stderr | fatal: not a git repository (or any of the parent directories): .git repositorygcworker stderr | fatal: not a git repository (or any of the parent directories): .git reconciliationworker stderr | fatal: not a git repository (or any of the parent directories): .git gcworker stderr | fatal: not a git repository (or any of the parent directories): .git autopruneworker stderr | fatal: not a git repository (or any of the parent directories): .git gunicorn-secscan stderr | fatal: not a git repository (or any of the parent directories): .git exportactionlogsworker stderr | fatal: not a git repository (or any of the parent directories): .git namespacegcworker stderr | fatal: not a git repository (or any of the parent directories): .git expiredappspecifictokenworker stderr | fatal: not a git repository (or any of the parent directories): .git notificationworker stderr | fatal: not a git repository (or any of the parent directories): .git nginx stdout | 2026/07/13 17:53:49 [crit] 138#138: *1 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.244.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.244.0.43:8443" nginx stdout | 2026/07/13 17:53:49 [error] 138#138: *1 open() "/quay-registry/static/patternfly/quay-registry/static/502.html" failed (2: No such file or directory), client: 10.244.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock/health/instance", host: "10.244.0.43:8443" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:53:49 +0000] "GET /health/instance HTTP/2.0" 404 146 "-" "kube-probe/1.32" (0.000 47 0.000) quotaregistrysizeworker stderr | fatal: not a git repository (or any of the parent directories): .git blobuploadcleanupworker stderr | fatal: not a git repository (or any of the parent directories): .git pullstatsredisflushworker stderr | fatal: not a git repository (or any of the parent directories): .git repositoryactioncounter stderr | fatal: not a git repository (or any of the parent directories): .git gunicorn-web stderr | fatal: not a git repository (or any of the parent directories): .git queuecleanupworker stderr | fatal: not a git repository (or any of the parent directories): .git proxycacheblobworker stderr | fatal: not a git repository (or any of the parent directories): .git securityscanningnotificationworker stderr | fatal: not a git repository (or any of the parent directories): .git globalpromstats stderr | fatal: not a git repository (or any of the parent directories): .git securityworker stderr | fatal: not a git repository (or any of the parent directories): .git teamsyncworker stderr | fatal: not a git repository (or any of the parent directories): .git manifestbackfillworker stderr | fatal: not a git repository (or any of the parent directories): .git servicekey stderr | fatal: not a git repository (or any of the parent directories): .git storagereplication stderr | fatal: not a git repository (or any of the parent directories): .git builder stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. builder stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. quotatotalworker stderr | fatal: not a git repository (or any of the parent directories): .git buildlogsarchiver stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. buildlogsarchiver stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. exportactionlogsworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. exportactionlogsworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. repositorygcworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. repositorygcworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. namespacegcworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. namespacegcworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. buildlogsarchiver stdout | 2026-07-13 17:53:57,689 [81] [INFO] [apscheduler.scheduler] Scheduler started gunicorn-registry stderr | fatal: not a git repository (or any of the parent directories): .git buildlogsarchiver stdout | 2026-07-13 17:53:57,770 [81] [INFO] [apscheduler.scheduler] Added job "ArchiveBuildLogsWorker._archive_redis_buildlogs" to job store "default" chunkcleanupworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. chunkcleanupworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. 2026-07-13 17:53:58,372 INFO exited: builder (exit status 0; expected) autopruneworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. autopruneworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. gunicorn-secscan stdout | 2026-07-13 17:53:58,980 [89] [WARNING] [app] 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. gunicorn-secscan stdout | 2026-07-13 17:53:58,981 [89] [WARNING] [app] 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. gunicorn-secscan stdout | 2026-07-13 17:53:58,984 [89] [INFO] [util.ipresolver] Loading AWS IP ranges from disk repositorygcworker stdout | 2026-07-13 17:53:59,182 [119] [INFO] [apscheduler.scheduler] Scheduler started repositorygcworker stdout | 2026-07-13 17:53:59,184 [119] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" repositorygcworker stdout | 2026-07-13 17:53:59,185 [119] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" repositorygcworker stdout | 2026-07-13 17:53:59,186 [119] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" logrotateworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. logrotateworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. manifestsubjectbackfillworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. manifestsubjectbackfillworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. nginx stdout | 2026/07/13 17:53:59 [crit] 132#132: *3 connect() to unix:/tmp/gunicorn_web.sock failed (2: No such file or directory) while connecting to upstream, client: 10.244.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock:/health/instance", host: "10.244.0.43:8443" nginx stdout | 2026/07/13 17:53:59 [error] 132#132: *3 open() "/quay-registry/static/patternfly/quay-registry/static/502.html" failed (2: No such file or directory), client: 10.244.0.1, server: _, request: "GET /health/instance HTTP/2.0", upstream: "http://unix:/tmp/gunicorn_web.sock/health/instance", host: "10.244.0.43:8443" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:53:59 +0000] "GET /health/instance HTTP/2.0" 404 146 "-" "kube-probe/1.32" (0.000 47 0.000) proxycacheblobworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. proxycacheblobworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. gunicorn-secscan stdout | 2026-07-13 17:54:00,587 [89] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-secscan gunicorn-secscan stdout | 2026-07-13 17:54:00,587 [89] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-secscan stdout | 2026-07-13 17:54:00,589 [89] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2026-07-13 17:54:00,589 [89] [INFO] [data.secscan_model] Using split secscan model: `[]` gunicorn-secscan stdout | 2026-07-13 17:54:00,589 [89] [INFO] [data.secscan_model] =============================== gunicorn-secscan stdout | 2026-07-13 17:54:00,589 [89] [INFO] [data.logs_model] =============================== gunicorn-secscan stdout | 2026-07-13 17:54:00,589 [89] [INFO] [data.logs_model] Using logs model `` gunicorn-secscan stdout | 2026-07-13 17:54:00,589 [89] [INFO] [data.logs_model] =============================== namespacegcworker stdout | 2026-07-13 17:54:00,976 [95] [INFO] [apscheduler.scheduler] Scheduler started namespacegcworker stdout | 2026-07-13 17:54:00,978 [95] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" namespacegcworker stdout | 2026-07-13 17:54:00,979 [95] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" namespacegcworker stdout | 2026-07-13 17:54:00,980 [95] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" teamsyncworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. teamsyncworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. pullstatsredisflushworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. pullstatsredisflushworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. manifestsubjectbackfillworker stdout | 2026-07-13 17:54:01,428 [93] [INFO] [apscheduler.scheduler] Scheduler started manifestsubjectbackfillworker stdout | 2026-07-13 17:54:01,471 [93] [INFO] [apscheduler.scheduler] Added job "ManifestSubjectBackfillWorker._backfill_manifest_subject" to job store "default" manifestsubjectbackfillworker stdout | 2026-07-13 17:54:01,471 [93] [INFO] [apscheduler.scheduler] Added job "ManifestSubjectBackfillWorker._backfill_manifest_artifact_type" to job store "default" gunicorn-secscan stderr | Traceback (most recent call last): gunicorn-secscan stderr | File "src/gevent/_abstract_linkable.py", line 287, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-secscan stderr | File "src/gevent/_abstract_linkable.py", line 333, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-secscan stderr | AssertionError: (None, ) gunicorn-secscan stderr | 2026-07-13T17:54:01Z failed with AssertionError gcworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. gcworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. expiredappspecifictokenworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. expiredappspecifictokenworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. notificationworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. notificationworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. blobuploadcleanupworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. blobuploadcleanupworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. reconciliationworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. reconciliationworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. repositoryactioncounter stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. repositoryactioncounter stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. queuecleanupworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. queuecleanupworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. exportactionlogsworker stdout | 2026-07-13 17:54:03,883 [85] [INFO] [apscheduler.scheduler] Scheduler started exportactionlogsworker stdout | 2026-07-13 17:54:03,886 [85] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" exportactionlogsworker stdout | 2026-07-13 17:54:03,890 [85] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" exportactionlogsworker stdout | 2026-07-13 17:54:03,890 [85] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" gcworker stdout | 2026-07-13 17:54:04,378 [86] [INFO] [apscheduler.scheduler] Scheduler started gcworker stdout | 2026-07-13 17:54:04,379 [86] [INFO] [apscheduler.scheduler] Added job "GarbageCollectionWorker._garbage_collection_repos" to job store "default" expiredappspecifictokenworker stdout | 2026-07-13 17:54:04,684 [84] [INFO] [apscheduler.scheduler] Scheduler started expiredappspecifictokenworker stdout | 2026-07-13 17:54:04,685 [84] [INFO] [apscheduler.scheduler] Added job "ExpiredAppSpecificTokenWorker._gc_expired_tokens" to job store "default" notificationworker stdout | 2026-07-13 17:54:04,782 [97] [INFO] [apscheduler.scheduler] Scheduler started notificationworker stdout | 2026-07-13 17:54:04,784 [97] [INFO] [apscheduler.scheduler] Added job "QueueWorker.poll_queue" to job store "default" notificationworker stdout | 2026-07-13 17:54:04,784 [97] [INFO] [apscheduler.scheduler] Added job "QueueWorker.update_queue_metrics" to job store "default" notificationworker stdout | 2026-07-13 17:54:04,784 [97] [INFO] [apscheduler.scheduler] Added job "QueueWorker.run_watchdog" to job store "default" quotaregistrysizeworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. quotaregistrysizeworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. queuecleanupworker stdout | 2026-07-13 17:54:05,188 [110] [INFO] [apscheduler.scheduler] Scheduler started queuecleanupworker stdout | 2026-07-13 17:54:05,191 [110] [INFO] [apscheduler.scheduler] Added job "QueueCleanupWorker._cleanup_queue" to job store "default" repositoryactioncounter stdout | 2026-07-13 17:54:05,303 [118] [INFO] [apscheduler.scheduler] Scheduler started repositoryactioncounter stdout | 2026-07-13 17:54:05,329 [118] [INFO] [apscheduler.scheduler] Added job "RepositoryActionCountWorker._run_counting" to job store "default" gunicorn-web stdout | 2026-07-13 17:54:05,497 [90] [WARNING] [app] 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. gunicorn-web stdout | 2026-07-13 17:54:05,497 [90] [WARNING] [app] 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. gunicorn-web stdout | 2026-07-13 17:54:05,499 [90] [INFO] [util.ipresolver] Loading AWS IP ranges from disk blobuploadcleanupworker stdout | 2026-07-13 17:54:05,596 [79] [INFO] [apscheduler.scheduler] Scheduler started blobuploadcleanupworker stdout | 2026-07-13 17:54:05,597 [79] [INFO] [apscheduler.scheduler] Added job "BlobUploadCleanupWorker._try_cleanup_uploads" to job store "default" securityworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. securityworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. quotatotalworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. quotatotalworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. securityscanningnotificationworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. securityscanningnotificationworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. globalpromstats stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. globalpromstats stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. storagereplication stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. storagereplication stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. servicekey stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. servicekey stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. manifestbackfillworker stderr | 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. manifestbackfillworker stderr | 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. gunicorn-registry stdout | 2026-07-13 17:54:06,175 [88] [WARNING] [app] 🟑🟑🟑 Detected TESTING: true on startup. TESTING property is either missing from config.yaml or set to 'true'. gunicorn-registry stdout | 2026-07-13 17:54:06,175 [88] [WARNING] [app] 🟑🟑🟑 Quay starting in TESTING mode, some functionality such as e-mail sending will not be available. gunicorn-registry stdout | 2026-07-13 17:54:06,176 [88] [INFO] [util.ipresolver] Loading AWS IP ranges from disk gunicorn-web stdout | 2026-07-13 17:54:06,529 [90] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-web gunicorn-web stdout | 2026-07-13 17:54:06,529 [90] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2026-07-13 17:54:06,568 [90] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2026-07-13 17:54:06,568 [90] [INFO] [data.secscan_model] Using split secscan model: `[]` gunicorn-web stdout | 2026-07-13 17:54:06,568 [90] [INFO] [data.secscan_model] =============================== gunicorn-web stdout | 2026-07-13 17:54:06,568 [90] [INFO] [data.logs_model] =============================== gunicorn-web stdout | 2026-07-13 17:54:06,568 [90] [INFO] [data.logs_model] Using logs model `` gunicorn-web stdout | 2026-07-13 17:54:06,568 [90] [INFO] [data.logs_model] =============================== servicekey stdout | 2026-07-13 17:54:06,613 [122] [INFO] [apscheduler.scheduler] Scheduler started servicekey stdout | 2026-07-13 17:54:06,614 [122] [INFO] [apscheduler.scheduler] Added job "ServiceKeyWorker._refresh_service_key" to job store "default" gunicorn-registry stdout | 2026-07-13 17:54:06,618 [88] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-registry gunicorn-registry stdout | 2026-07-13 17:54:06,618 [88] [INFO] [data.database] Connection pooling enabled for postgresql; stale timeout: None; max connection count: None gunicorn-registry stdout | 2026-07-13 17:54:06,619 [88] [INFO] [data.secscan_model] =============================== gunicorn-registry stdout | 2026-07-13 17:54:06,619 [88] [INFO] [data.secscan_model] Using split secscan model: `[]` gunicorn-registry stdout | 2026-07-13 17:54:06,619 [88] [INFO] [data.secscan_model] =============================== gunicorn-registry stdout | 2026-07-13 17:54:06,619 [88] [INFO] [data.logs_model] =============================== gunicorn-registry stdout | 2026-07-13 17:54:06,619 [88] [INFO] [data.logs_model] Using logs model `` gunicorn-registry stdout | 2026-07-13 17:54:06,628 [88] [INFO] [data.logs_model] =============================== globalpromstats stdout | 2026-07-13 17:54:06,705 [87] [INFO] [apscheduler.scheduler] Scheduler started globalpromstats stdout | 2026-07-13 17:54:06,706 [87] [INFO] [apscheduler.scheduler] Added job "GlobalPrometheusStatsWorker._try_report_stats" to job store "default" manifestbackfillworker stdout | 2026-07-13 17:54:06,715 [92] [INFO] [apscheduler.scheduler] Scheduler started manifestbackfillworker stdout | 2026-07-13 17:54:06,716 [92] [INFO] [apscheduler.scheduler] Added job "ManifestBackfillWorker._backfill_manifests" to job store "default" gunicorn-registry stderr | Traceback (most recent call last): gunicorn-registry stderr | File "src/gevent/_abstract_linkable.py", line 287, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-registry stderr | File "src/gevent/_abstract_linkable.py", line 333, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-registry stderr | AssertionError: (None, ) gunicorn-registry stderr | 2026-07-13T17:54:07Z failed with AssertionError gunicorn-web stderr | Traceback (most recent call last): gunicorn-web stderr | File "src/gevent/_abstract_linkable.py", line 287, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-web stderr | File "src/gevent/_abstract_linkable.py", line 333, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links gunicorn-web stderr | AssertionError: (None, ) gunicorn-web stderr | 2026-07-13T17:54:07Z gunicorn-web stderr | failed with AssertionError notificationworker stdout | 2026-07-13 17:54:07,784 [97] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:07 GMT)" (scheduled at 2026-07-13 17:54:07.784427+00:00) notificationworker stdout | 2026-07-13 17:54:07,795 [97] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:17 GMT)" executed successfully buildlogsarchiver stdout | 2026-07-13 17:54:08,769 [81] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2026-07-13 17:54:08 GMT)" (scheduled at 2026-07-13 17:54:08.769082+00:00) buildlogsarchiver stdout | 2026-07-13 17:54:08,778 [81] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2026-07-13 17:54:38 GMT)" executed successfully gunicorn-web stdout | 2026-07-13 17:54:09,974 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:09 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.005 162 0.005) gunicorn-registry stdout | 2026-07-13 17:54:09,980 [369] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:09 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:09,994 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:09 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.003 159 0.002) gunicorn-web stdout | 2026-07-13 17:54:09,997 [354] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:09 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:10,009 [354] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-web gunicorn-web stdout | 2026-07-13 17:54:10,009 [354] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2026-07-13 17:54:10,016 [354] [INFO] [gunicorn.access] 10.244.0.1 - - [13/Jul/2026:17:54:10 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.32" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:54:10 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.32" (0.073 47 0.072) gunicorn-web stdout | 2026-07-13 17:54:10,022 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:10 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.005 162 0.005) gunicorn-registry stdout | 2026-07-13 17:54:10,027 [367] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:10 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:10,041 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:10 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.001 159 0.001) gunicorn-web stdout | 2026-07-13 17:54:10,042 [354] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:10 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:10,043 [354] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-web gunicorn-web stdout | 2026-07-13 17:54:10,043 [354] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2026-07-13 17:54:10,056 [354] [INFO] [gunicorn.access] 10.244.0.1 - - [13/Jul/2026:17:54:10 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.32" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:54:10 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.32" (0.036 47 0.036) gcworker stdout | 2026-07-13 17:54:15,380 [86] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2026-07-13 17:54:15 GMT)" (scheduled at 2026-07-13 17:54:15.379967+00:00) gcworker stdout | 2026-07-13 17:54:15,387 [86] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2026-07-13 17:54:45 GMT)" executed successfully notificationworker stdout | 2026-07-13 17:54:17,784 [97] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:27 GMT)" (scheduled at 2026-07-13 17:54:17.784427+00:00) notificationworker stdout | 2026-07-13 17:54:17,786 [97] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:27 GMT)" executed successfully gunicorn-web stdout | 2026-07-13 17:54:20,052 [353] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.002 162 0.002) gunicorn-registry stdout | 2026-07-13 17:54:20,054 [369] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:20 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:20,068 [353] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:20 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.002 159 0.002) gunicorn-web stdout | 2026-07-13 17:54:20,071 [353] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:20 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:20,083 [353] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-web gunicorn-web stdout | 2026-07-13 17:54:20,083 [353] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2026-07-13 17:54:20,091 [353] [INFO] [gunicorn.access] 10.244.0.1 - - [13/Jul/2026:17:54:20 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.32" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:54:20 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.32" (0.071 47 0.071) exportactionlogsworker stdout | 2026-07-13 17:54:22,887 [85] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2026-07-13 17:54:22 GMT)" (scheduled at 2026-07-13 17:54:22.887051+00:00) exportactionlogsworker stdout | 2026-07-13 17:54:22,895 [85] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:22 GMT)" executed successfully repositorygcworker stdout | 2026-07-13 17:54:24,185 [119] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2026-07-13 17:54:24 GMT)" (scheduled at 2026-07-13 17:54:24.185010+00:00) repositorygcworker stdout | 2026-07-13 17:54:24,194 [119] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:24 GMT)" executed successfully notificationworker stdout | 2026-07-13 17:54:26,785 [97] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:26 GMT)" (scheduled at 2026-07-13 17:54:26.785367+00:00) notificationworker stdout | 2026-07-13 17:54:26,785 [97] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:26 GMT)" executed successfully notificationworker stdout | 2026-07-13 17:54:27,784 [97] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:37 GMT)" (scheduled at 2026-07-13 17:54:27.784427+00:00) notificationworker stdout | 2026-07-13 17:54:27,786 [97] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:37 GMT)" executed successfully gunicorn-web stdout | 2026-07-13 17:54:30,034 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:30 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.005 162 0.005) gunicorn-registry stdout | 2026-07-13 17:54:30,040 [378] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:30 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:30,053 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:30 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.001 159 0.001) gunicorn-web stdout | 2026-07-13 17:54:30,054 [354] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:30 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:30,055 [354] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-web gunicorn-web stdout | 2026-07-13 17:54:30,055 [354] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2026-07-13 17:54:30,067 [354] [INFO] [gunicorn.access] 10.244.0.1 - - [13/Jul/2026:17:54:30 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.32" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:54:30 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.32" (0.047 47 0.047) exportactionlogsworker stdout | 2026-07-13 17:54:33,891 [85] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:33 GMT)" (scheduled at 2026-07-13 17:54:33.891205+00:00) exportactionlogsworker stdout | 2026-07-13 17:54:33,891 [85] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:33 GMT)" executed successfully notificationworker stdout | 2026-07-13 17:54:37,784 [97] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:47 GMT)" (scheduled at 2026-07-13 17:54:37.784427+00:00) notificationworker stdout | 2026-07-13 17:54:37,786 [97] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:47 GMT)" executed successfully buildlogsarchiver stdout | 2026-07-13 17:54:38,769 [81] [INFO] [apscheduler.executors.default] Running job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2026-07-13 17:55:08 GMT)" (scheduled at 2026-07-13 17:54:38.769082+00:00) buildlogsarchiver stdout | 2026-07-13 17:54:38,771 [81] [INFO] [apscheduler.executors.default] Job "ArchiveBuildLogsWorker._archive_redis_buildlogs (trigger: interval[0:00:30], next run at: 2026-07-13 17:55:08 GMT)" executed successfully manifestsubjectbackfillworker stdout | 2026-07-13 17:54:39,472 [93] [INFO] [apscheduler.executors.default] Running job "ManifestSubjectBackfillWorker._backfill_manifest_subject (trigger: interval[0:01:00], next run at: 2026-07-13 17:54:39 GMT)" (scheduled at 2026-07-13 17:54:39.471850+00:00) manifestsubjectbackfillworker stdout | 2026-07-13 17:54:39,481 [93] [INFO] [apscheduler.executors.default] Job "ManifestSubjectBackfillWorker._backfill_manifest_subject (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:39 GMT)" executed successfully gunicorn-web stdout | 2026-07-13 17:54:40,034 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:40 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.002 162 0.002) gunicorn-registry stdout | 2026-07-13 17:54:40,036 [378] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:40 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:40,049 [354] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:40 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.001 159 0.001) gunicorn-web stdout | 2026-07-13 17:54:40,050 [354] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:40 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:40,051 [354] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-web gunicorn-web stdout | 2026-07-13 17:54:40,051 [354] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2026-07-13 17:54:40,063 [354] [INFO] [gunicorn.access] 10.244.0.1 - - [13/Jul/2026:17:54:40 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.32" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:54:40 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.32" (0.043 47 0.043) namespacegcworker stdout | 2026-07-13 17:54:41,982 [95] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:54:41 GMT)" (scheduled at 2026-07-13 17:54:41.980682+00:00) namespacegcworker stdout | 2026-07-13 17:54:41,982 [95] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:54:41 GMT)" executed successfully manifestsubjectbackfillworker stdout | 2026-07-13 17:54:43,472 [93] [INFO] [apscheduler.executors.default] Running job "ManifestSubjectBackfillWorker._backfill_manifest_artifact_type (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:43 GMT)" (scheduled at 2026-07-13 17:54:43.472609+00:00) manifestsubjectbackfillworker stdout | 2026-07-13 17:54:43,474 [93] [INFO] [apscheduler.executors.default] Job "ManifestSubjectBackfillWorker._backfill_manifest_artifact_type (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:43 GMT)" executed successfully repositorygcworker stdout | 2026-07-13 17:54:44,187 [119] [INFO] [apscheduler.executors.default] Running job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:44 GMT)" (scheduled at 2026-07-13 17:54:44.186859+00:00) repositorygcworker stdout | 2026-07-13 17:54:44,187 [119] [INFO] [apscheduler.executors.default] Job "QueueWorker.run_watchdog (trigger: interval[0:01:00], next run at: 2026-07-13 17:55:44 GMT)" executed successfully gcworker stdout | 2026-07-13 17:54:45,380 [86] [INFO] [apscheduler.executors.default] Running job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2026-07-13 17:55:15 GMT)" (scheduled at 2026-07-13 17:54:45.379967+00:00) gcworker stdout | 2026-07-13 17:54:45,380 [86] [INFO] [apscheduler.executors.default] Job "GarbageCollectionWorker._garbage_collection_repos (trigger: interval[0:00:30], next run at: 2026-07-13 17:55:15 GMT)" executed successfully notificationworker stdout | 2026-07-13 17:54:45,785 [97] [INFO] [apscheduler.executors.default] Running job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2026-07-13 17:59:45 GMT)" (scheduled at 2026-07-13 17:54:45.785138+00:00) notificationworker stdout | 2026-07-13 17:54:45,789 [97] [INFO] [apscheduler.executors.default] Job "QueueWorker.update_queue_metrics (trigger: interval[0:05:00], next run at: 2026-07-13 17:59:45 GMT)" executed successfully notificationworker stdout | 2026-07-13 17:54:47,784 [97] [INFO] [apscheduler.executors.default] Running job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:57 GMT)" (scheduled at 2026-07-13 17:54:47.784427+00:00) notificationworker stdout | 2026-07-13 17:54:47,786 [97] [INFO] [apscheduler.executors.default] Job "QueueWorker.poll_queue (trigger: interval[0:00:10], next run at: 2026-07-13 17:54:57 GMT)" executed successfully gunicorn-web stdout | 2026-07-13 17:54:50,034 [353] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.001 162 0.001) gunicorn-registry stdout | 2026-07-13 17:54:50,035 [378] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:50 +0000] "GET /v1/_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:50,047 [353] [WARNING] [py.warnings] /opt/app-root/lib64/python3.12/site-packages/urllib3/connectionpool.py:1097: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings gunicorn-web stdout | warnings.warn( nginx stdout | 127.0.0.1 (-) - - [13/Jul/2026:17:54:50 +0000] "GET /_internal_ping HTTP/1.1" 200 4 "-" "python-requests/2.32.3" (0.001 159 0.001) gunicorn-web stdout | 2026-07-13 17:54:50,048 [354] [INFO] [gunicorn.access] 127.0.0.1 - - [13/Jul/2026:17:54:50 +0000] "GET /_internal_ping HTTP/1.0" 200 4 "-" "python-requests/2.32.3" gunicorn-web stdout | 2026-07-13 17:54:50,049 [353] [INFO] [data.database] Setting database host to quay-postgres.quay for worker gunicorn-web gunicorn-web stdout | 2026-07-13 17:54:50,049 [353] [INFO] [data.database] Connection pooling disabled for postgresql gunicorn-web stdout | 2026-07-13 17:54:50,062 [353] [INFO] [gunicorn.access] 10.244.0.1 - - [13/Jul/2026:17:54:50 +0000] "GET /health/instance HTTP/1.0" 200 152 "-" "kube-probe/1.32" nginx stdout | 10.244.0.1 (-) - - [13/Jul/2026:17:54:50 +0000] "GET /health/instance HTTP/2.0" 200 152 "-" "kube-probe/1.32" (0.042 47 0.042)