W0319 13:56:44.708985       1 cmd.go:257] Using insecure, self-signed certificates
I0319 13:56:44.958122       1 start.go:138] Unable to read service ca bundle: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory
I0319 13:56:44.958701       1 observer_polling.go:159] Starting file observer
I0319 13:56:46.166165       1 operator.go:76] Starting insights-operator v0.0.0-master+$Format:%H$
I0319 13:56:46.166486       1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"}
I0319 13:56:46.168742       1 secure_serving.go:57] Forcing use of http/1.1 only
W0319 13:56:46.168765       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.
W0319 13:56:46.168770       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.
W0319 13:56:46.168775       1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected.
W0319 13:56:46.168779       1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected.
W0319 13:56:46.168783       1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected.
W0319 13:56:46.168787       1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected.
I0319 13:56:46.168992       1 simple_featuregate_reader.go:171] Starting feature-gate-detector
I0319 13:56:46.170941       1 operator.go:141] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GCPClusterHostedDNSInstall GatewayAPI GatewayAPIController HighlyAvailableArbiter HyperShiftOnlyDynamicResourceAllocation ImageStreamImportMode ImageVolume KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS ManagedBootImagesAzure ManagedBootImagesvSphere MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages PreconfiguredUDNAddresses ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SigstoreImageVerification SigstoreImageVerificationPKI StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks VolumeAttributesClass AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSDualStackInstall AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureDualStackInstall AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement CBORServingAndStorage CRDCompatibilityRequirementOperator ClientsAllowCBOR ClientsPreferCBOR ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterAPIMachineManagement ClusterAPIMachineManagementVSphere ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud EtcdBackendQuota EventTTL EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall GCPDualStackInstall ImageModeStatusReporting IngressControllerDynamicConfigurationManager InsightsConfig InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesCPMS MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutableCSINodeAllocatableCount MutatingAdmissionPolicy NewOLM NewOLMBoxCutterRuntime NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterInstall NutanixMultiSubnets OSStreams OVNObservability OnPremDNSRecords OpenShiftPodSecurityAdmission ProvisioningRequestAvailable SELinuxMount ShortCertRotation SignatureStores TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeGroupSnapshot]
I0319 13:56:46.171409       1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"55403a75-de0c-4260-8bba-753e7f26f6aa", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GCPClusterHostedDNSInstall", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "HyperShiftOnlyDynamicResourceAllocation", "ImageStreamImportMode", "ImageVolume", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "PreconfiguredUDNAddresses", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SigstoreImageVerification", "SigstoreImageVerificationPKI", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks", "VolumeAttributesClass"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSDualStackInstall", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureDualStackInstall", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "CBORServingAndStorage", "CRDCompatibilityRequirementOperator", "ClientsAllowCBOR", "ClientsPreferCBOR", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterAPIMachineManagement", "ClusterAPIMachineManagementVSphere", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "EtcdBackendQuota", "EventTTL", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "GCPDualStackInstall", "ImageModeStatusReporting", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesCPMS", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutableCSINodeAllocatableCount", "MutatingAdmissionPolicy", "NewOLM", "NewOLMBoxCutterRuntime", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterInstall", "NutanixMultiSubnets", "OSStreams", "OVNObservability", "OnPremDNSRecords", "OpenShiftPodSecurityAdmission", "ProvisioningRequestAvailable", "SELinuxMount", "ShortCertRotation", "SignatureStores", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeGroupSnapshot"}}
I0319 13:56:46.173131       1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController
I0319 13:56:46.173154       1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController
I0319 13:56:46.173194       1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0319 13:56:46.173210       1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0319 13:56:46.173263       1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I0319 13:56:46.173328       1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0319 13:56:46.173645       1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/tmp/serving-cert-678815172/tls.crt::/tmp/serving-cert-678815172/tls.key"
I0319 13:56:46.173887       1 secure_serving.go:213] Serving securely on [::]:8443
I0319 13:56:46.173963       1 tlsconfig.go:243] "Starting DynamicServingCertificateController"
W0319 13:56:46.180813       1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used.
I0319 13:56:46.180883       1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status
I0319 13:56:46.180941       1 base_controller.go:76] Waiting for caches to sync for ConfigController
I0319 13:56:46.188867       1 secretconfigobserver.go:249] Found cloud.openshift.com token
I0319 13:56:46.188915       1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status
I0319 13:56:46.192996       1 secretconfigobserver.go:119] support secret does not exist
I0319 13:56:46.197437       1 secretconfigobserver.go:249] Found cloud.openshift.com token
I0319 13:56:46.201099       1 secretconfigobserver.go:119] support secret does not exist
I0319 13:56:46.203915       1 recorder.go:176] Pruning old reports every 6h7m52s, max age is 288h0m0s
I0319 13:56:46.209791       1 periodic.go:216] Running clusterconfig gatherer
I0319 13:56:46.209872       1 tasks_processing.go:45] number of workers: 32
I0319 13:56:46.209790       1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message=
I0319 13:56:46.209894       1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s
I0319 13:56:46.209812       1 controllerstatus.go:80] name=insightsreport healthy=true reason= message=
I0319 13:56:46.209935       1 insightsreport.go:296] Starting report retriever
I0319 13:56:46.209928       1 tasks_processing.go:69] worker 31 listening for tasks.
I0319 13:56:46.209942       1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s
I0319 13:56:46.209946       1 tasks_processing.go:71] worker 31 working on machine_configs task.
I0319 13:56:46.209951       1 tasks_processing.go:69] worker 0 listening for tasks.
I0319 13:56:46.209958       1 tasks_processing.go:71] worker 0 working on proxies task.
I0319 13:56:46.210156       1 tasks_processing.go:69] worker 1 listening for tasks.
I0319 13:56:46.210172       1 tasks_processing.go:69] worker 2 listening for tasks.
I0319 13:56:46.210180       1 tasks_processing.go:69] worker 3 listening for tasks.
I0319 13:56:46.210188       1 tasks_processing.go:69] worker 4 listening for tasks.
I0319 13:56:46.210197       1 tasks_processing.go:69] worker 5 listening for tasks.
I0319 13:56:46.210206       1 tasks_processing.go:69] worker 6 listening for tasks.
I0319 13:56:46.210215       1 tasks_processing.go:69] worker 7 listening for tasks.
I0319 13:56:46.210226       1 tasks_processing.go:69] worker 8 listening for tasks.
I0319 13:56:46.210234       1 tasks_processing.go:69] worker 9 listening for tasks.
I0319 13:56:46.210243       1 tasks_processing.go:69] worker 10 listening for tasks.
I0319 13:56:46.210252       1 tasks_processing.go:69] worker 11 listening for tasks.
I0319 13:56:46.210261       1 tasks_processing.go:69] worker 12 listening for tasks.
I0319 13:56:46.210269       1 tasks_processing.go:69] worker 13 listening for tasks.
I0319 13:56:46.210278       1 tasks_processing.go:69] worker 14 listening for tasks.
I0319 13:56:46.210285       1 tasks_processing.go:69] worker 15 listening for tasks.
I0319 13:56:46.210295       1 tasks_processing.go:69] worker 16 listening for tasks.
I0319 13:56:46.210336       1 tasks_processing.go:69] worker 17 listening for tasks.
I0319 13:56:46.210344       1 tasks_processing.go:69] worker 18 listening for tasks.
I0319 13:56:46.210353       1 tasks_processing.go:69] worker 19 listening for tasks.
I0319 13:56:46.210361       1 tasks_processing.go:69] worker 20 listening for tasks.
I0319 13:56:46.210369       1 tasks_processing.go:69] worker 21 listening for tasks.
I0319 13:56:46.210378       1 tasks_processing.go:69] worker 22 listening for tasks.
I0319 13:56:46.210386       1 tasks_processing.go:69] worker 23 listening for tasks.
I0319 13:56:46.210396       1 tasks_processing.go:69] worker 24 listening for tasks.
I0319 13:56:46.210422       1 tasks_processing.go:69] worker 29 listening for tasks.
I0319 13:56:46.210457       1 tasks_processing.go:69] worker 30 listening for tasks.
I0319 13:56:46.210462       1 tasks_processing.go:69] worker 28 listening for tasks.
I0319 13:56:46.210521       1 tasks_processing.go:71] worker 28 working on monitoring_persistent_volumes task.
I0319 13:56:46.210517       1 tasks_processing.go:71] worker 30 working on machine_sets task.
I0319 13:56:46.210807       1 tasks_processing.go:71] worker 1 working on crds task.
I0319 13:56:46.210473       1 tasks_processing.go:69] worker 25 listening for tasks.
I0319 13:56:46.210480       1 tasks_processing.go:69] worker 26 listening for tasks.
I0319 13:56:46.210487       1 tasks_processing.go:69] worker 27 listening for tasks.
I0319 13:56:46.210898       1 tasks_processing.go:71] worker 27 working on jaegers task.
I0319 13:56:46.211041       1 tasks_processing.go:71] worker 2 working on sap_datahubs task.
I0319 13:56:46.211123       1 tasks_processing.go:71] worker 25 working on certificate_signing_requests task.
I0319 13:56:46.211329       1 tasks_processing.go:71] worker 3 working on container_runtime_configs task.
I0319 13:56:46.211509       1 tasks_processing.go:71] worker 4 working on openshift_machine_api_events task.
I0319 13:56:46.211624       1 tasks_processing.go:71] worker 26 working on service_accounts task.
I0319 13:56:46.211948       1 tasks_processing.go:71] worker 5 working on silenced_alerts task.
W0319 13:56:46.212133       1 gather_silenced_alerts.go:38] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory
I0319 13:56:46.212148       1 tasks_processing.go:71] worker 5 working on networks task.
I0319 13:56:46.212009       1 tasks_processing.go:71] worker 15 working on machines task.
I0319 13:56:46.212015       1 tasks_processing.go:71] worker 6 working on openshift_logging task.
I0319 13:56:46.212460       1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 34.919µs to process 0 records
I0319 13:56:46.212019       1 tasks_processing.go:71] worker 7 working on pdbs task.
I0319 13:56:46.212022       1 tasks_processing.go:71] worker 8 working on schedulers task.
I0319 13:56:46.212028       1 tasks_processing.go:71] worker 9 working on nodes task.
I0319 13:56:46.212032       1 tasks_processing.go:71] worker 10 working on config_maps task.
I0319 13:56:46.212035       1 tasks_processing.go:71] worker 11 working on ceph_cluster task.
I0319 13:56:46.212039       1 tasks_processing.go:71] worker 12 working on storage_cluster task.
I0319 13:56:46.212037       1 tasks_processing.go:71] worker 20 working on nodenetworkstates task.
I0319 13:56:46.212043       1 tasks_processing.go:71] worker 13 working on machine_config_pools task.
I0319 13:56:46.212045       1 tasks_processing.go:71] worker 16 working on openstack_version task.
I0319 13:56:46.212051       1 tasks_processing.go:71] worker 17 working on cluster_apiserver task.
I0319 13:56:46.212046       1 tasks_processing.go:71] worker 14 working on storage_classes task.
I0319 13:56:46.212054       1 tasks_processing.go:71] worker 21 working on tsdb_status task.
W0319 13:56:46.214508       1 gather_prometheus_tsdb_status.go:38] Unable to load metrics client, tsdb status cannot be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory
I0319 13:56:46.214525       1 tasks_processing.go:71] worker 21 working on version task.
I0319 13:56:46.214579       1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 47.747µs to process 0 records
I0319 13:56:46.212055       1 tasks_processing.go:71] worker 18 working on authentication task.
I0319 13:56:46.212054       1 tasks_processing.go:71] worker 23 working on openstack_controlplanes task.
I0319 13:56:46.212057       1 tasks_processing.go:71] worker 19 working on sap_config task.
I0319 13:56:46.212065       1 tasks_processing.go:71] worker 24 working on pod_network_connectivity_checks task.
I0319 13:56:46.212067       1 tasks_processing.go:71] worker 22 working on image_pruners task.
I0319 13:56:46.212067       1 tasks_processing.go:71] worker 29 working on mutating_webhook_configurations task.
I0319 13:56:46.217047       1 tasks_processing.go:71] worker 30 working on validating_webhook_configurations task.
I0319 13:56:46.217084       1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 6.48475ms to process 0 records
I0319 13:56:46.217962       1 controller.go:129] Initializing last reported time to 0001-01-01T00:00:00Z
I0319 13:56:46.217996       1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready
I0319 13:56:46.218002       1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready
I0319 13:56:46.218006       1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready
I0319 13:56:46.218025       1 controller.go:531] The operator is still being initialized
I0319 13:56:46.218100       1 controller.go:554] The operator is healthy
I0319 13:56:46.218915       1 tasks_processing.go:71] worker 0 working on clusterroles task.
I0319 13:56:46.219113       1 recorder.go:75] Recording config/proxy with fingerprint=76d2510e326010fd9e59c2c5d219225789e01ebdd232dd821db9e4412e7f0385
I0319 13:56:46.219167       1 gather.go:177] gatherer "clusterconfig" function "proxies" took 8.943532ms to process 1 records
I0319 13:56:46.228562       1 tasks_processing.go:71] worker 6 working on active_alerts task.
W0319 13:56:46.228638       1 gather_active_alerts.go:54] Unable to load alerts client, no alerts will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory
I0319 13:56:46.228841       1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 16.175336ms to process 0 records
E0319 13:56:46.228883       1 gather.go:140] gatherer "clusterconfig" function "machine_configs" failed with the error: getting MachineConfigPools failed: the server could not find the requested resource (get machineconfigpools.machineconfiguration.openshift.io)
I0319 13:56:46.228930       1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499
I0319 13:56:46.228964       1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 18.607349ms to process 1 records
I0319 13:56:46.228994       1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 56.082µs to process 0 records
E0319 13:56:46.229022       1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope
I0319 13:56:46.229050       1 gather.go:177] gatherer "clusterconfig" function "machines" took 16.654244ms to process 0 records
I0319 13:56:46.229077       1 tasks_processing.go:71] worker 15 working on number_of_pods_and_netnamespaces_with_sdn_annotations task.
I0319 13:56:46.229863       1 tasks_processing.go:71] worker 6 working on infrastructures task.
I0319 13:56:46.230207       1 tasks_processing.go:71] worker 31 working on lokistack task.
I0319 13:56:46.230900       1 tasks_processing.go:71] worker 5 working on image_registries task.
I0319 13:56:46.231961       1 recorder.go:75] Recording config/network with fingerprint=107c7878a638d2a0ee209606bb7518de97e259543dedc5c3670ef7d839d6d926
I0319 13:56:46.232032       1 gather.go:177] gatherer "clusterconfig" function "networks" took 18.741541ms to process 1 records
I0319 13:56:46.233773       1 tasks_processing.go:71] worker 28 working on operators_pods_and_events task.
I0319 13:56:46.233994       1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 23.237119ms to process 0 records
I0319 13:56:46.236993       1 tasks_processing.go:71] worker 13 working on dvo_metrics task.
I0319 13:56:46.237157       1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 21.513691ms to process 0 records
I0319 13:56:46.239785       1 tasks_processing.go:71] worker 27 working on node_features task.
I0319 13:56:46.239923       1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 28.874616ms to process 0 records
I0319 13:56:46.239970       1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 28.465173ms to process 0 records
I0319 13:56:46.240001       1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 26.72698ms to process 0 records
I0319 13:56:46.240030       1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 26.617977ms to process 0 records
I0319 13:56:46.240057       1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 26.334807ms to process 0 records
I0319 13:56:46.240083       1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 26.060108ms to process 0 records
I0319 13:56:46.240109       1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 24.90594ms to process 0 records
I0319 13:56:46.240136       1 tasks_processing.go:71] worker 16 working on openstack_dataplanenodesets task.
I0319 13:56:46.240127       1 tasks_processing.go:71] worker 20 working on qemu_kubevirt_launcher_logs task.
E0319 13:56:46.240140       1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io)
I0319 13:56:46.240397       1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 24.774539ms to process 0 records
I0319 13:56:46.240412       1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 28.599372ms to process 0 records
I0319 13:56:46.240418       1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 25.353862ms to process 0 records
I0319 13:56:46.240621       1 tasks_processing.go:71] worker 4 working on machine_healthchecks task.
I0319 13:56:46.240681       1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=13e0b5259a99b65e307c6ecb765c359c3eef1025a81bdc6052444e89db64304e
I0319 13:56:46.240813       1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=d22c3b4cbb794ddbb1ec345d772dc98e55634710e501930adbb52f8e02eebe76
I0319 13:56:46.240841       1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=d956cab7d002d06e2e4b619e8196d5328a489e56348b650e5d1888c76ba2816d
I0319 13:56:46.240851       1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 23.091177ms to process 3 records
I0319 13:56:46.240132       1 tasks_processing.go:71] worker 12 working on feature_gates task.
I0319 13:56:46.240858       1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 29.109707ms to process 0 records
I0319 13:56:46.240869       1 tasks_processing.go:71] worker 30 working on operators task.
I0319 13:56:46.240918       1 tasks_processing.go:71] worker 2 working on support_secret task.
I0319 13:56:46.240931       1 tasks_processing.go:71] worker 23 working on node_logs task.
I0319 13:56:46.240175       1 tasks_processing.go:71] worker 24 working on oauths task.
I0319 13:56:46.240086       1 tasks_processing.go:71] worker 11 working on aggregated_monitoring_cr_names task.
I0319 13:56:46.240092       1 tasks_processing.go:71] worker 3 working on olm_operators task.
I0319 13:56:46.240168       1 tasks_processing.go:71] worker 19 working on ingress_certificates task.
I0319 13:56:46.251716       1 tasks_processing.go:71] worker 25 working on container_images task.
I0319 13:56:46.251753       1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 40.57062ms to process 0 records
I0319 13:56:46.256117       1 tasks_processing.go:71] worker 17 working on sap_pods task.
I0319 13:56:46.256882       1 recorder.go:75] Recording config/apiserver with fingerprint=7fc900bbbd7b9372bac1584d2c5b09b32959152bcab927b86da70bb51d4cbdd9
I0319 13:56:46.256898       1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 42.014476ms to process 1 records
I0319 13:56:46.257097       1 tasks_processing.go:71] worker 5 working on openstack_dataplanedeployments task.
I0319 13:56:46.257955       1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=6bfe14d0d2a6ba4b8c3647044b230b33e16835b93b3e04643f37975f2331fcc9
I0319 13:56:46.257974       1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 26.13318ms to process 1 records
I0319 13:56:46.264229       1 tasks_processing.go:71] worker 22 working on ingress task.
E0319 13:56:46.264470       1 gather_node_features.go:86] GatherNodeFeatures: NodeFeatures resource not found in openshift-nfd namespace (may not be installed)
I0319 13:56:46.264538       1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=d4cef5e6d389f497e05dac7cfc0cb0804c168c926d877f25d565a488aadfec3c
I0319 13:56:46.264599       1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 49.007258ms to process 1 records
I0319 13:56:46.264628       1 gather.go:177] gatherer "clusterconfig" function "node_features" took 24.731386ms to process 0 records
I0319 13:56:46.264656       1 tasks_processing.go:71] worker 27 working on nodenetworkconfigurationpolicies task.
I0319 13:56:46.264864       1 tasks_processing.go:71] worker 1 working on cost_management_metrics_configs task.
I0319 13:56:46.265484       1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=cce047a933a5058f26f40ff38396b87bce0a71a92d3afa8ca09d24b6b31ac6f1
I0319 13:56:46.265830       1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=d08b4f67ab355cbbe394ad36d69dc24566760aacbf77409427bc48ab3ffecb05
I0319 13:56:46.265845       1 gather.go:177] gatherer "clusterconfig" function "crds" took 53.842745ms to process 2 records
I0319 13:56:46.265940       1 tasks_processing.go:71] worker 18 working on machine_autoscalers task.
I0319 13:56:46.266116       1 recorder.go:75] Recording config/authentication with fingerprint=c2348af19bdb2787099e7af74420d0ee0def2db1628ffd2d8d2451013158ed7a
I0319 13:56:46.266169       1 gather.go:177] gatherer "clusterconfig" function "authentication" took 50.077479ms to process 1 records
I0319 13:56:46.266184       1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 8.606591ms to process 0 records
I0319 13:56:46.266191       1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 34.488674ms to process 0 records
I0319 13:56:46.266197       1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 7.644785ms to process 0 records
I0319 13:56:46.266204       1 tasks_processing.go:71] worker 17 working on metrics task.
E0319 13:56:46.266206       1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope
I0319 13:56:46.266215       1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 24.128122ms to process 0 records
W0319 13:56:46.266225       1 gather_most_recent_metrics.go:64] Unable to load metrics client, no metrics will be collected: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory
I0319 13:56:46.266241       1 tasks_processing.go:71] worker 5 working on install_plans task.
I0319 13:56:46.266367       1 recorder.go:75] Recording config/featuregate with fingerprint=9b430e21627ba20fa5be3997028feb415a59d57c580fa9d8864b0be458523faf
I0319 13:56:46.266382       1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 23.968499ms to process 1 records
I0319 13:56:46.266390       1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 25.223551ms to process 0 records
I0319 13:56:46.266396       1 gather.go:177] gatherer "clusterconfig" function "metrics" took 28.732µs to process 0 records
I0319 13:56:46.266481       1 tasks_processing.go:74] worker 12 stopped.
I0319 13:56:46.266530       1 tasks_processing.go:71] worker 17 working on overlapping_namespace_uids task.
I0319 13:56:46.266814       1 tasks_processing.go:74] worker 31 stopped.
I0319 13:56:46.266828       1 tasks_processing.go:71] worker 4 working on image task.
I0319 13:56:46.266968       1 tasks_processing.go:74] worker 16 stopped.
I0319 13:56:46.267598       1 gather_logs.go:145] no pods in  namespace were found
I0319 13:56:46.267611       1 tasks_processing.go:74] worker 20 stopped.
I0319 13:56:46.267620       1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 27.418791ms to process 0 records
I0319 13:56:46.268077       1 tasks_processing.go:74] worker 29 stopped.
I0319 13:56:46.268331       1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=13aa2d7ab1a9ecfa4351c32c8d07a8279bc676efd63ee6b503b0125699c381a1
I0319 13:56:46.268374       1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 52.788018ms to process 1 records
I0319 13:56:46.268757       1 tasks_processing.go:74] worker 7 stopped.
I0319 13:56:46.268856       1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=f177b8af603a6ec0663f2d4c057ec4255309f5f2ebd9e135bdcf170255a5bd68
I0319 13:56:46.268923       1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=200546d91b596323ab9c142edf28ac9c569261c8218e462eef26399f2d278c43
I0319 13:56:46.268951       1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=0782685707f10a026769dcdb400022139c13a1a8ec216906290d758e2fe451f4
I0319 13:56:46.268964       1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 56.273897ms to process 3 records
I0319 13:56:46.269431       1 tasks_processing.go:74] worker 2 stopped.
E0319 13:56:46.269450       1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found
I0319 13:56:46.269460       1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 28.462761ms to process 0 records
I0319 13:56:46.269784       1 tasks_processing.go:74] worker 24 stopped.
I0319 13:56:46.270056       1 recorder.go:75] Recording config/oauth with fingerprint=f704fd85d24aeb2604e8c242d401fea791e7d573fd8637f02fde59fafef83281
I0319 13:56:46.270133       1 gather.go:177] gatherer "clusterconfig" function "oauths" took 28.486732ms to process 1 records
I0319 13:56:46.270244       1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=2a9768e3c723d2eabb2f0bdaed86559213e99c21c1916871b0a8eab8cf9323ad
I0319 13:56:46.270897       1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 57.431417ms to process 1 records
W0319 13:56:46.270396       1 operator.go:328] started
I0319 13:56:46.270419       1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer
I0319 13:56:46.270476       1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s
I0319 13:56:46.270485       1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s
I0319 13:56:46.270645       1 tasks_processing.go:74] worker 8 stopped.
I0319 13:56:46.272911       1 tasks_processing.go:74] worker 6 stopped.
I0319 13:56:46.272992       1 recorder.go:75] Recording config/infrastructure with fingerprint=ec32afc2295d4415368a510796b7dab4ca42079023e7dcc1bf9c6a8c4a40214d
I0319 13:56:46.273034       1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 40.657198ms to process 1 records
I0319 13:56:46.273061       1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 29.441939ms to process 0 records
I0319 13:56:46.273283       1 tasks_processing.go:74] worker 23 stopped.
I0319 13:56:46.273764       1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=646cd31ee3a4bdb8f5acc24e4e7f460fbd7a96bb5d1a3fae4c1461cc0764f6d7
I0319 13:56:46.273866       1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=d552e7b40cd6cd3f19946f7fecc271b1ebf2aa34eec9e3f5b16a5084ddfcbe05
I0319 13:56:46.273894       1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 56.356356ms to process 2 records
I0319 13:56:46.273924       1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 8.858713ms to process 0 records
I0319 13:56:46.273949       1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 8.668252ms to process 0 records
I0319 13:56:46.273986       1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 7.666605ms to process 0 records
I0319 13:56:46.274016       1 tasks_processing.go:74] worker 18 stopped.
I0319 13:56:46.274043       1 tasks_processing.go:74] worker 14 stopped.
I0319 13:56:46.274067       1 tasks_processing.go:74] worker 27 stopped.
I0319 13:56:46.274089       1 tasks_processing.go:74] worker 1 stopped.
I0319 13:56:46.274941       1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0319 13:56:46.275044       1 tasks_processing.go:74] worker 9 stopped.
I0319 13:56:46.275844       1 recorder.go:75] Recording config/node/ip-10-0-0-219.ec2.internal with fingerprint=dd27b912bbb25b317eda4903f8e472cc1b511e39476c9d1e9c87d59b89e78106
I0319 13:56:46.276016       1 recorder.go:75] Recording config/node/ip-10-0-1-219.ec2.internal with fingerprint=99cd170271bc10baca5d32492d54706af1b33c187769043e07c84ecdb0757bd3
I0319 13:56:46.276098       1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController
I0319 13:56:46.276183       1 recorder.go:75] Recording config/node/ip-10-0-2-194.ec2.internal with fingerprint=09b123811e600955030f48c121665716a3c194de82671130aa63a732a119d8d3
I0319 13:56:46.276217       1 gather.go:177] gatherer "clusterconfig" function "nodes" took 62.289424ms to process 3 records
I0319 13:56:46.278628       1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0319 13:56:46.285266       1 tasks_processing.go:74] worker 4 stopped.
I0319 13:56:46.285476       1 recorder.go:75] Recording config/image with fingerprint=ae3985def4d9e866c39cbd43357e2e7b0ca0aab4e0f0065dd29a2f5ac41ff396
I0319 13:56:46.285537       1 gather.go:177] gatherer "clusterconfig" function "image" took 18.429827ms to process 1 records
I0319 13:56:46.287445       1 base_controller.go:82] Caches are synced for ConfigController 
I0319 13:56:46.287520       1 base_controller.go:119] Starting #1 worker of ConfigController controller ...
I0319 13:56:46.287941       1 controller.go:254] Source clusterTransferController *clustertransfer.Controller is not ready
I0319 13:56:46.287992       1 controller.go:254] Source periodic-clusterconfig *controllerstatus.Simple is not ready
I0319 13:56:46.288018       1 controller.go:254] Source periodic-conditional *controllerstatus.Simple is not ready
I0319 13:56:46.288041       1 controller.go:254] Source periodic-workloads *controllerstatus.Simple is not ready
I0319 13:56:46.288064       1 controller.go:254] Source scaController *sca.Controller is not ready
I0319 13:56:46.288124       1 controller.go:531] The operator is still being initialized
I0319 13:56:46.288203       1 controller.go:554] The operator is healthy
I0319 13:56:46.289906       1 tasks_processing.go:74] worker 25 stopped.
I0319 13:56:46.292529       1 prometheus_rules.go:88] Prometheus rules successfully created
I0319 13:56:46.293766       1 configmapobserver.go:84] configmaps "insights-config" not found
I0319 13:56:46.294903       1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-5b5d9b4544-z4d8m with fingerprint=958f4fc44bacc49b4c329c12005d0fec256e7bc0262358081f9e42ea5173b8fc
I0319 13:56:46.295475       1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-dbk75 with fingerprint=63c301578905178aecebe2fa92954ed843b020a5074e6960af544f268d344c47
I0319 13:56:46.295620       1 recorder.go:75] Recording config/running_containers with fingerprint=5c7dadf4f8cfbdd4f7b2c88b6eeb65efc1820e568c612b87613578edbe51e5e2
I0319 13:56:46.295666       1 gather.go:177] gatherer "clusterconfig" function "container_images" took 38.127627ms to process 3 records
I0319 13:56:46.295703       1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 48.901193ms to process 0 records
I0319 13:56:46.295858       1 tasks_processing.go:74] worker 3 stopped.
I0319 13:56:46.295923       1 tasks_processing.go:74] worker 0 stopped.
I0319 13:56:46.296008       1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=b6650e514ca872828b05645eaf37f30d19c7d50c75c804122f37664e6cf83db2
I0319 13:56:46.296190       1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=a7e995ed880a9282a49d10f6d60dd7f2e84a22d85751785ab8e0294f372a3724
I0319 13:56:46.296225       1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 75.055ms to process 2 records
I0319 13:56:46.296256       1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 52.62531ms to process 0 records
I0319 13:56:46.296442       1 tasks_processing.go:74] worker 11 stopped.
I0319 13:56:46.296759       1 tasks_processing.go:74] worker 22 stopped.
I0319 13:56:46.296851       1 recorder.go:75] Recording config/ingress with fingerprint=a22ded44afbc769bc03381435efb1b830bf249ba700177378e0e7de8f8a29be2
I0319 13:56:46.296906       1 gather.go:177] gatherer "clusterconfig" function "ingress" took 30.049899ms to process 1 records
W0319 13:56:46.298339       1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again.
I0319 13:56:46.302406       1 tasks_processing.go:74] worker 15 stopped.
I0319 13:56:46.302474       1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 73.300236ms to process 0 records
E0319 13:56:46.307585       1 cluster_transfer.go:95] failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27907ca170-f09b-47bb-9e7b-05cf08b8a4b0%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:35473->172.30.0.10:53: read: connection refused
I0319 13:56:46.307640       1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=Disconnected message=failed to pull cluster transfer: unable to retrieve cluster transfer data from https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/: Get "https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/?search=cluster_uuid+is+%27907ca170-f09b-47bb-9e7b-05cf08b8a4b0%27+and+status+is+%27accepted%27": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:35473->172.30.0.10:53: read: connection refused
I0319 13:56:46.309713       1 tasks_processing.go:74] worker 10 stopped.
E0319 13:56:46.309733       1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found
E0319 13:56:46.309801       1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found
E0319 13:56:46.309812       1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found
I0319 13:56:46.309825       1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=c8a9805405aae7d04b7f03c09c51097942070877009b6daebd5573ef8127446e
I0319 13:56:46.309869       1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=d476c7d3f5b104863f08f481b1264dcc68cc272ecefb0ecb709b18a6afab034d
I0319 13:56:46.309895       1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a
I0319 13:56:46.309905       1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=3b70b9787487b548204d806ebebf90b0c2d460145e34cb33c0d8d3a7a011d907
I0319 13:56:46.309911       1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
I0319 13:56:46.309964       1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0
I0319 13:56:46.309997       1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d
I0319 13:56:46.310023       1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 96.731468ms to process 7 records
I0319 13:56:46.332428       1 tasks_processing.go:74] worker 17 stopped.
I0319 13:56:46.332482       1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945
I0319 13:56:46.332500       1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 65.869455ms to process 1 records
I0319 13:56:46.360672       1 tasks_processing.go:74] worker 21 stopped.
I0319 13:56:46.360974       1 recorder.go:75] Recording config/version with fingerprint=6246ac70ff93097fa30be667b05916ca069a26cd3f51b688c0a356eae0ef7a86
I0319 13:56:46.360993       1 recorder.go:75] Recording config/id with fingerprint=afeb6c53d3e77f527a534453b2b4ad2357d0018a87c47955d0e4dd785a1c8b55
I0319 13:56:46.361003       1 gather.go:177] gatherer "clusterconfig" function "version" took 146.123909ms to process 2 records
I0319 13:56:46.377887       1 base_controller.go:82] Caches are synced for LoggingSyncer 
I0319 13:56:46.377905       1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ...
I0319 13:56:46.378197       1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload
W0319 13:56:46.387427       1 sca.go:161] Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:58146->172.30.0.10:53: read: connection refused
I0319 13:56:46.387446       1 controllerstatus.go:80] name=scaController healthy=true reason=NonHTTPError message=Failed to pull SCA certs from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: unable to retrieve SCA certs data from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates: Post "https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates": dial tcp: lookup api.openshift.com on 172.30.0.10:53: read udp 10.128.0.15:58146->172.30.0.10:53: read: connection refused
I0319 13:56:46.391134       1 tasks_processing.go:74] worker 19 stopped.
E0319 13:56:46.391153       1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found
E0319 13:56:46.391160       1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2p4qu53p30sfni25c42tk2t0l4poha1i-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2p4qu53p30sfni25c42tk2t0l4poha1i-primary-cert-bundle-secret" not found
I0319 13:56:46.391275       1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=fb79a967043945aa137579c4335e70e40b82b458e627736b4991ba14019ad695
I0319 13:56:46.391354       1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 149.415958ms to process 1 records
I0319 13:56:46.706105       1 gather_cluster_operator_pods_and_events.go:121] Found 16 pods with 18 containers
I0319 13:56:46.706120       1 gather_cluster_operator_pods_and_events.go:235] Maximum buffer size: 1398101 bytes
I0319 13:56:46.706392       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-frww6 pod in namespace openshift-dns (previous: false).
I0319 13:56:46.915865       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-frww6 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-frww6\" is waiting to start: ContainerCreating"
I0319 13:56:46.915886       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-frww6\" is waiting to start: ContainerCreating"
I0319 13:56:46.915895       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-frww6 pod in namespace openshift-dns (previous: false).
I0319 13:56:47.094901       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-frww6 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-frww6\" is waiting to start: ContainerCreating"
I0319 13:56:47.094921       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-frww6\" is waiting to start: ContainerCreating"
I0319 13:56:47.094931       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns container dns-default-l6fc7 pod in namespace openshift-dns (previous: false).
W0319 13:56:47.277571       1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again.
I0319 13:56:47.323500       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-l6fc7 pod in namespace openshift-dns for failing operator dns (previous: false): "container \"dns\" in pod \"dns-default-l6fc7\" is waiting to start: ContainerCreating"
I0319 13:56:47.323518       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"dns\" in pod \"dns-default-l6fc7\" is waiting to start: ContainerCreating"
I0319 13:56:47.323531       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for kube-rbac-proxy container dns-default-l6fc7 pod in namespace openshift-dns (previous: false).
I0319 13:56:47.497070       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for dns-default-l6fc7 pod in namespace openshift-dns for failing operator kube-rbac-proxy (previous: false): "container \"kube-rbac-proxy\" in pod \"dns-default-l6fc7\" is waiting to start: ContainerCreating"
I0319 13:56:47.497092       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"kube-rbac-proxy\" in pod \"dns-default-l6fc7\" is waiting to start: ContainerCreating"
I0319 13:56:47.497105       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-2k6v8 pod in namespace openshift-dns (previous: false).
I0319 13:56:47.716420       1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty"
I0319 13:56:47.716439       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-fxwgk pod in namespace openshift-dns (previous: false).
I0319 13:56:47.895233       1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty"
I0319 13:56:47.895252       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for dns-node-resolver container node-resolver-txmkt pod in namespace openshift-dns (previous: false).
I0319 13:56:47.906090       1 gather_cluster_operators.go:184] Unable to get configs.samples.operator.openshift.io resource due to: configs.samples.operator.openshift.io "cluster" not found
I0319 13:56:48.095454       1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty"
I0319 13:56:48.095485       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5877f87688-jdl9t pod in namespace openshift-image-registry (previous: false).
W0319 13:56:48.276320       1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again.
I0319 13:56:48.294749       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5877f87688-jdl9t pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5877f87688-jdl9t\" is waiting to start: ContainerCreating"
I0319 13:56:48.294765       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5877f87688-jdl9t\" is waiting to start: ContainerCreating"
I0319 13:56:48.294775       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-5877f87688-lk4tl pod in namespace openshift-image-registry (previous: false).
I0319 13:56:48.309300       1 tasks_processing.go:74] worker 30 stopped.
I0319 13:56:48.309359       1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=6a30b29cfe8aa6c793fc3be2db08a36fa764602c4ea8351e4ee61e5e46a8aaa0
I0319 13:56:48.309404       1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=c9eacc96c9efb6d7b1b4beac3a786e8e47f78fe6ab2f766986479f317989229d
I0319 13:56:48.309442       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e
I0319 13:56:48.309469       1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=7ba4a322aa3d86fda76e78005cc3145b2caea4894c4258536d9d2aeaf7a07a00
I0319 13:56:48.309484       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f
I0319 13:56:48.309510       1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=db44dd4a14eb69026024cfe358bc6d11a70a3eac729c4b80598d0bc7c0ebc415
I0319 13:56:48.309561       1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=7d2c460109adc5a56f2435bf8d19a3eeaf00a3b45b730eb7d9288de845562f1e
I0319 13:56:48.309601       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=b6a8ad443cb7342268d5f94c31e0fdd8a1f8431d91092149eb51863a937c429d
I0319 13:56:48.309648       1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=2d50b3cf9f9316dc1ac7799b4c89cf3ab4a8be32bc6c69c2160de810da91f7ad
I0319 13:56:48.309662       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=e5ff11d57817f84a678f6fa9565af55bd1120227c16a21933637ab62675a6d70
I0319 13:56:48.309689       1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=f7b4d6b4994597174c75ddfc01869c3bf11b3ec15b65429830b08657684e933c
I0319 13:56:48.309702       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762
I0319 13:56:48.309730       1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=fc6c7516044621968e958b7158d0cc5b4be59b1e646e8ae1fd9cd85cf02f73e4
I0319 13:56:48.309745       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11
I0319 13:56:48.309776       1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=ffac54b0bb9685df8ae99c7256d3cb153b139a758dbd3b2c7930ff58226d06ee
I0319 13:56:48.309786       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a
I0319 13:56:48.309800       1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=3827ab6999fd9e66eb19e452deaaaea4d5fba7aebfa21b469c6e8c591f062879
I0319 13:56:48.309813       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db
I0319 13:56:48.309838       1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=995cefc7b50bf8d32aaa99654cbe3d6b2e52960e6d8274231c696bab88c5313c
I0319 13:56:48.309987       1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=852ba0e1d7b08d45fd7eac73c9448815ba0f33d771e8107133e83b3daffeba2e
I0319 13:56:48.310001       1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab
I0319 13:56:48.310008       1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d
I0319 13:56:48.310036       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75
I0319 13:56:48.310071       1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=aa9d7aa10f66e2370ed246c0b2bc4477aea5f5a957627779e44779ac4d6b9907
I0319 13:56:48.310096       1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=ca2c86f7fc9d08eef2d6ae1bd4df94950d8130c7d75c2c6fff515d7a27e8ea28
I0319 13:56:48.310105       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76
I0319 13:56:48.310122       1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=df54da1ea3fb41f94366b5158c8027db4e895ada13e7ae3c87086438e8c4dd21
I0319 13:56:48.310138       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7
I0319 13:56:48.310156       1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=4e66233cd455f8f045b3a4e8d017d7c9ab3c45f0886cd1bbb3d583f70ee45166
I0319 13:56:48.310178       1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=2319e7c6967068329ccf173c5e0d2be45a144af9a5305560a5b05c3c6dee7246
I0319 13:56:48.310202       1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=add7b498ad05e13195a807a11f0d55f153912f33205956918aeea6bd3d7d1b77
I0319 13:56:48.310224       1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=4cab14ecf855ba7153ecc4188f5823f1927735e4f462368585b6572cf24e3478
I0319 13:56:48.310243       1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=bb95bf3a79cfb719cfef38244f7908de5d7c762cf75a94721732524f065d88e9
I0319 13:56:48.310282       1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=4ab2f55dce26ef6bbac0b756c55f2845b3e47bdbcbdaba293692a334e0ced09e
I0319 13:56:48.310299       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e
I0319 13:56:48.310319       1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad
I0319 13:56:48.310328       1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.068413252s to process 36 records
I0319 13:56:48.494745       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for image-registry-5877f87688-lk4tl pod in namespace openshift-image-registry for failing operator registry (previous: false): "container \"registry\" in pod \"image-registry-5877f87688-lk4tl\" is waiting to start: ContainerCreating"
I0319 13:56:48.494766       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"registry\" in pod \"image-registry-5877f87688-lk4tl\" is waiting to start: ContainerCreating"
I0319 13:56:48.494781       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for registry container image-registry-66697d9587-z6h7f pod in namespace openshift-image-registry (previous: false).
I0319 13:56:48.691226       1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty"
I0319 13:56:48.691243       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-9qjhb pod in namespace openshift-image-registry (previous: false).
I0319 13:56:48.895272       1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty"
I0319 13:56:48.895292       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-df7x2 pod in namespace openshift-image-registry (previous: false).
I0319 13:56:49.096608       1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty"
I0319 13:56:49.096625       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for node-ca container node-ca-p47jz pod in namespace openshift-image-registry (previous: false).
W0319 13:56:49.275150       1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again.
I0319 13:56:49.300542       1 gather_cluster_operator_pods_and_events.go:280] Error: "log buffer is empty"
I0319 13:56:49.300560       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-69cd7b44fc-x2s8l pod in namespace openshift-ingress (previous: false).
I0319 13:56:49.496878       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-69cd7b44fc-x2s8l pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-69cd7b44fc-x2s8l\" is waiting to start: ContainerCreating"
I0319 13:56:49.496964       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-69cd7b44fc-x2s8l\" is waiting to start: ContainerCreating"
I0319 13:56:49.497005       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-db6589979-5gkh7 pod in namespace openshift-ingress (previous: false).
I0319 13:56:49.701569       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-db6589979-5gkh7 pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-db6589979-5gkh7\" is waiting to start: ContainerCreating"
I0319 13:56:49.701589       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-db6589979-5gkh7\" is waiting to start: ContainerCreating"
I0319 13:56:49.701605       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for router container router-default-db6589979-pr7kx pod in namespace openshift-ingress (previous: false).
I0319 13:56:49.895898       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for router-default-db6589979-pr7kx pod in namespace openshift-ingress for failing operator router (previous: false): "container \"router\" in pod \"router-default-db6589979-pr7kx\" is waiting to start: ContainerCreating"
I0319 13:56:49.895919       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"router\" in pod \"router-default-db6589979-pr7kx\" is waiting to start: ContainerCreating"
I0319 13:56:49.895935       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-42t6p pod in namespace openshift-ingress-canary (previous: false).
I0319 13:56:50.095080       1 gather_cluster_operator_pods_and_events.go:408] Failed to fetch log for ingress-canary-42t6p pod in namespace openshift-ingress-canary for failing operator serve-healthcheck-canary (previous: false): "container \"serve-healthcheck-canary\" in pod \"ingress-canary-42t6p\" is waiting to start: ContainerCreating"
I0319 13:56:50.095099       1 gather_cluster_operator_pods_and_events.go:280] Error: "container \"serve-healthcheck-canary\" in pod \"ingress-canary-42t6p\" is waiting to start: ContainerCreating"
I0319 13:56:50.095109       1 gather_cluster_operator_pods_and_events.go:365] Fetching logs for serve-healthcheck-canary container ingress-canary-5fcr8 pod in namespace openshift-ingress-canary (previous: false).
W0319 13:56:50.274459       1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again.
I0319 13:56:50.295053       1 tasks_processing.go:74] worker 28 stopped.
I0319 13:56:50.295164       1 recorder.go:75] Recording events/openshift-dns with fingerprint=f91e22530e3feb932f639be683675c3bdc5d0b2a309ae18ad21b739e7a8a2b51
I0319 13:56:50.295266       1 recorder.go:75] Recording events/openshift-image-registry with fingerprint=de33f400b8e3462b8be0dbcac2038eeb5aa0064371035ec259289a164fc73cf1
I0319 13:56:50.295326       1 recorder.go:75] Recording events/openshift-ingress-operator with fingerprint=145c24a24b584b625d0a1255beb5ab21bddf15624766b5dc0bc2e9a33c25cdad
I0319 13:56:50.295388       1 recorder.go:75] Recording events/openshift-ingress with fingerprint=d28eba4651cf5bf063a3690ddaed6761d5a615f0473d0e3dcc7d7d3b462b96a0
I0319 13:56:50.295424       1 recorder.go:75] Recording events/openshift-ingress-canary with fingerprint=4ae4f6977bb790939d971233dc80e12d762c472f7232553416885f97aeea122c
I0319 13:56:50.295437       1 recorder.go:75] Recording config/pod/openshift-ingress-canary/logs/ingress-canary-5fcr8/serve-healthcheck-canary_current.log with fingerprint=e8f31fda67bfbd75a0b56048b9880ff73cc03a5043db10c63d30d713d0399a06
I0319 13:56:50.295449       1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 4.061210449s to process 6 records
W0319 13:56:51.274574       1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again.
W0319 13:56:51.274602       1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded
I0319 13:56:51.274621       1 tasks_processing.go:74] worker 13 stopped.
E0319 13:56:51.274631       1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded
I0319 13:56:51.274643       1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
W0319 13:56:51.274657       1 gather.go:155] issue recording gatherer "clusterconfig" function "dvo_metrics" result "config/dvo_metrics" because of the warning: warning: the record with the same fingerprint "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" was already recorded at path "config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt", recording another one with a different path "config/dvo_metrics"
I0319 13:56:51.274664       1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.037571585s to process 1 records
I0319 13:56:58.738488       1 tasks_processing.go:74] worker 5 stopped.
I0319 13:56:58.738555       1 recorder.go:75] Recording config/installplans with fingerprint=7b887df561a3a9e6ef0dc672845aa5d56e348505006b7496d3a2f83892b0c95b
I0319 13:56:58.738569       1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 12.472229934s to process 1 records
I0319 13:56:59.224410       1 configmapobserver.go:84] configmaps "insights-config" not found
I0319 13:56:59.417656       1 tasks_processing.go:74] worker 26 stopped.
I0319 13:56:59.418074       1 recorder.go:75] Recording config/serviceaccounts with fingerprint=7c1a3e3c431de69ce4ee6fb33aad2158a3d38446a91ec5899ca71ceaf725097e
I0319 13:56:59.418096       1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.205979956s to process 1 records
E0319 13:56:59.418203       1 periodic.go:254] "Unhandled Error" err="clusterconfig failed after 13.208s with: function \"machine_configs\" failed with an error, function \"machines\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error"
I0319 13:56:59.419341       1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "machine_configs" failed with an error, function "machines" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "machine_healthchecks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error
I0319 13:56:59.419363       1 periodic.go:216] Running workloads gatherer
I0319 13:56:59.419390       1 tasks_processing.go:45] number of workers: 2
I0319 13:56:59.419410       1 tasks_processing.go:69] worker 1 listening for tasks.
I0319 13:56:59.419417       1 tasks_processing.go:71] worker 1 working on workload_info task.
I0319 13:56:59.419426       1 tasks_processing.go:69] worker 0 listening for tasks.
I0319 13:56:59.419447       1 tasks_processing.go:71] worker 0 working on helmchart_info task.
I0319 13:56:59.444363       1 tasks_processing.go:74] worker 0 stopped.
I0319 13:56:59.444390       1 gather.go:177] gatherer "workloads" function "helmchart_info" took 24.90441ms to process 0 records
I0319 13:56:59.446631       1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 22s for image data
I0319 13:56:59.454255       1 gather_workloads_info.go:387] No image sha256:bbb5af28bbed69e8189825fdc6665f4e8dbcd0c0c83fa479943ea102fe9452d6 (8ms)
I0319 13:56:59.461832       1 gather_workloads_info.go:387] No image sha256:1aabd00f813e1afccdab16a845e6346288f5cf7e320fa5bb97c683702191c575 (8ms)
I0319 13:56:59.469291       1 gather_workloads_info.go:387] No image sha256:5efa227ddac82e56eddc03761e220af1f2d6a4b22ee00b2baf426523f26c0319 (7ms)
I0319 13:56:59.476480       1 gather_workloads_info.go:387] No image sha256:db40a3d4e50016c645fdb4a1f389dfcdbc9febc10c174816d681abe5c14153cd (7ms)
I0319 13:56:59.485999       1 gather_workloads_info.go:387] No image sha256:77d8ddf2957025a39616328b5cf2b012de6cedce63fb79eebc5270d7b3401574 (10ms)
I0319 13:56:59.493213       1 gather_workloads_info.go:387] No image sha256:ff8a8892f37210c582b3e526cbd2652b02ef9007d57be12b404434374dd0116a (7ms)
I0319 13:56:59.500255       1 gather_workloads_info.go:387] No image sha256:26e4f5b039a340bc1ae080fb9d92799700b57ae5df45120768f516682d6cc54b (7ms)
I0319 13:56:59.506931       1 gather_workloads_info.go:387] No image sha256:7ea1ff613255c1ee67b34ec82b68ec6f5a4a6d283620a8e0ad142d4a31eb9211 (7ms)
I0319 13:56:59.513728       1 gather_workloads_info.go:387] No image sha256:ab32e70fb149ae7470c941ca73bcfe9eba540bd5058a3d697a3ddf96597d0132 (7ms)
I0319 13:56:59.520800       1 gather_workloads_info.go:387] No image sha256:11a864539ae77a94afcf8d2e1abf9168081c5081a1bc06088f4be5206cdc02df (7ms)
I0319 13:56:59.554180       1 gather_workloads_info.go:387] No image sha256:862d5befeba504d5058264cf65f16fc307eea4eb3914fbbbf40fbe61714b0958 (33ms)
I0319 13:56:59.654146       1 gather_workloads_info.go:387] No image sha256:2a4f6a54cd5135c8121832236063419071fa5d00c10a051c197d3d01370e4d9c (100ms)
I0319 13:56:59.756585       1 gather_workloads_info.go:387] No image sha256:50b7c5185c4a6e63145753415c92723c59136d14b951ea0f4f06739b00893329 (102ms)
I0319 13:56:59.855517       1 gather_workloads_info.go:387] No image sha256:0b2cc6aadb1abc1d35a55da0c9884eb9426d7ff7a97174daeb09bb1e1a23cfae (99ms)
I0319 13:56:59.954950       1 gather_workloads_info.go:387] No image sha256:07f14370b57046dc2ac49be933bb2248aaf5943e8fb56ce9703eaa7c673b999f (99ms)
I0319 13:57:00.055026       1 gather_workloads_info.go:387] No image sha256:7dd9584d291967387f2a1ade991ad3bf8565bb59f49b013217951399e8273498 (100ms)
I0319 13:57:00.154943       1 gather_workloads_info.go:387] No image sha256:dd6a017092d57ab3a6663f3822a0fa0e3bce4365859d9522dd173405276cd49a (100ms)
I0319 13:57:00.254875       1 gather_workloads_info.go:387] No image sha256:3e889ea85c9c4748c548b84ae135a8adc88f42026ce8073538ca2fe7d89b8004 (100ms)
I0319 13:57:00.354194       1 gather_workloads_info.go:387] No image sha256:c85dd52f4140690eef0d762834ab8b9af2faedbbd3bd9dd5616d4cd0bebd4740 (99ms)
I0319 13:57:00.454921       1 gather_workloads_info.go:387] No image sha256:1c325fa866a794c56a7b30ca9d4fe225606566651bb12a2ecce8f60cbddb2809 (101ms)
I0319 13:57:00.554176       1 gather_workloads_info.go:387] No image sha256:1bf773bc4ec02bf6d68c82f32175366b01fb24f6da8c724b7621748692aec7dc (99ms)
I0319 13:57:00.654708       1 gather_workloads_info.go:387] No image sha256:04c7a2673571e9069c934d86b4308c0f57b2715b45db500837b7708ae1f20ae7 (101ms)
I0319 13:57:00.759420       1 gather_workloads_info.go:387] No image sha256:3e262d5207d5e2077e4a14f4fbddc70d4d2651dbaaf955147013feb58a18cddd (105ms)
I0319 13:57:00.855005       1 gather_workloads_info.go:387] No image sha256:188e4e2d697e2ca3c7c16772c0ab12253f00bd8433d8691ed7058fe7bafbbbf4 (96ms)
I0319 13:57:00.958388       1 gather_workloads_info.go:387] No image sha256:79fca57a4fd5ef00472a287218b84061b941905c61fcc55e89073dd93fa77648 (103ms)
I0319 13:57:01.054750       1 gather_workloads_info.go:387] No image sha256:04ee35aa6f87e954d386228808017cf2938c5a96a052059c6767078d5f3ad8d6 (96ms)
I0319 13:57:01.154050       1 gather_workloads_info.go:387] No image sha256:05376e7e2304d22dd866d8e25dd1e5ff8988b2232059fadfd12cb440af785635 (99ms)
I0319 13:57:01.257211       1 gather_workloads_info.go:387] No image sha256:0c90ca6acbe2f4fdbd253ae22f5b3286247c6940321f7d2b9801aea21ef5269f (103ms)
I0319 13:57:01.356809       1 gather_workloads_info.go:387] No image sha256:60648d27017709a420e43353f9a83d7c466017b3b771200672a4dbc3c3f7a3ae (100ms)
I0319 13:57:01.456054       1 gather_workloads_info.go:387] No image sha256:c15ca0c0ad60fe8757c2d5d1723fcdd7a1ed6c0251a90d22a7e6cae6811d01aa (99ms)
I0319 13:57:01.554774       1 gather_workloads_info.go:387] No image sha256:41e0dcd2f91df9234249f168b3fbe6d55cfdb3ebd2e598d5edfa47f905c046cb (99ms)
I0319 13:57:01.654127       1 gather_workloads_info.go:387] No image sha256:17e68da37c74bdc95a4eb43b51ce671027807ee610a4ee5237324a8c0dc9d044 (99ms)
I0319 13:57:01.754874       1 gather_workloads_info.go:387] No image sha256:25d9148ea34149935323231c64952679727b6f7f235febc3be1df63e0bf18c5e (101ms)
I0319 13:57:01.754906       1 tasks_processing.go:74] worker 1 stopped.
E0319 13:57:01.754918       1 gather.go:140] gatherer "workloads" function "workload_info" failed with the error: no running pods found for the insights-runtime-extractor statefulset
I0319 13:57:01.755151       1 recorder.go:75] Recording config/workload_info with fingerprint=d51d44913ed68c1df1b234e96902cc61b83583ab93c9e5ea03025b77e39878d6
I0319 13:57:01.755168       1 gather.go:177] gatherer "workloads" function "workload_info" took 2.335480808s to process 1 records
E0319 13:57:01.755191       1 periodic.go:254] "Unhandled Error" err="workloads failed after 2.335s with: function \"workload_info\" failed with an error"
I0319 13:57:01.756295       1 controllerstatus.go:89] name=periodic-workloads healthy=false reason=PeriodicGatherFailed message=Source workloads could not be retrieved: function "workload_info" failed with an error
I0319 13:57:01.756326       1 periodic.go:216] Running conditional gatherer
I0319 13:57:01.761193       1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.21.5/gathering_rules
I0319 13:57:01.768409       1 conditional_gatherer.go:107] Get "https://console.redhat.com/api/gathering/v2/4.21.5/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.15:45499->172.30.0.10:53: read: connection refused
E0319 13:57:01.768707       1 conditional_gatherer.go:322] unable to update alerts cache: open /var/run/configmaps/service-ca-bundle/service-ca.crt: no such file or directory
I0319 13:57:01.768766       1 conditional_gatherer.go:384] updating version cache for conditional gatherer
I0319 13:57:01.773611       1 conditional_gatherer.go:392] cluster version is '4.21.5'
E0319 13:57:01.773624       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773629       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773632       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773636       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773638       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773643       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773645       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773648       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
E0319 13:57:01.773651       1 conditional_gatherer.go:209] error checking conditions for a gathering rule: alerts cache is missing
I0319 13:57:01.773665       1 tasks_processing.go:45] number of workers: 3
I0319 13:57:01.773684       1 tasks_processing.go:69] worker 2 listening for tasks.
I0319 13:57:01.773690       1 tasks_processing.go:71] worker 2 working on remote_configuration task.
I0319 13:57:01.773689       1 tasks_processing.go:69] worker 0 listening for tasks.
I0319 13:57:01.773699       1 tasks_processing.go:69] worker 1 listening for tasks.
I0319 13:57:01.773703       1 tasks_processing.go:71] worker 0 working on rapid_container_logs task.
I0319 13:57:01.773707       1 tasks_processing.go:74] worker 1 stopped.
I0319 13:57:01.773714       1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task.
I0319 13:57:01.773742       1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=0394430c431eec4d48bb1811a90918e95161d2282c59af26f2473613cc0959db
I0319 13:57:01.773755       1 gather.go:177] gatherer "conditional" function "remote_configuration" took 983ns to process 1 records
I0319 13:57:01.773817       1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=7034af97e7e41c22e4b775abdd4b9066c8ebb19da33eb7f69f39bfd2eb5f6406
I0319 13:57:01.773826       1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.567µs to process 1 records
I0319 13:57:01.773831       1 tasks_processing.go:74] worker 2 stopped.
I0319 13:57:01.773953       1 tasks_processing.go:74] worker 0 stopped.
I0319 13:57:01.773964       1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 239.893µs to process 0 records
I0319 13:57:01.773981       1 controllerstatus.go:89] name=periodic-conditional healthy=false reason=NotAvailable message=Get "https://console.redhat.com/api/gathering/v2/4.21.5/gathering_rules": dial tcp: lookup console.redhat.com on 172.30.0.10:53: read udp 10.128.0.15:45499->172.30.0.10:53: read: connection refused
I0319 13:57:01.773997       1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=359de9c990c741675cec72fda96b5c3682221efdb4799f5eaa6e9805bcd3b5c1
I0319 13:57:01.794196       1 recorder.go:75] Recording insights-operator/gathers with fingerprint=ee3ebae0c64901790f621d53999c1d90ce75a1f6ffa0d273ee779a98762012eb
I0319 13:57:01.794335       1 diskrecorder.go:70] Writing 92 records to /var/lib/insights-operator/insights-2026-03-19-135701.tar.gz
I0319 13:57:01.800535       1 diskrecorder.go:51] Wrote 92 records to disk in 6ms
I0319 13:57:01.800563       1 periodic.go:285] Gathering cluster info every 2h0m0s
I0319 13:57:01.800578       1 periodic.go:286] Configuration is 
	dataReporting:
		interval: 2h0m0s,
		uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload,
		storagePath: /var/lib/insights-operator,
		downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, 
		conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules,
		obfuscation: []
	sca:
		disabled: false,
		endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates,
		interval: 8h0m0s
	alerting:
		disabled: false
	clusterTransfer:
		endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/,
		interval: 12h0m0s
	proxy:
		httpProxy: ,
		httpsProxy: ,
		noProxy: 
I0319 13:57:03.746850       1 configmapobserver.go:84] configmaps "insights-config" not found
I0319 13:57:03.947729       1 configmapobserver.go:84] configmaps "insights-config" not found
I0319 13:57:15.062273       1 configmapobserver.go:84] configmaps "insights-config" not found
I0319 13:58:09.959411       1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.crt" has been created (hash="f02c5849d7df19b19ed9f15ddfcf113e8531e396fbe497a290e0a64201fe6e44")
W0319 13:58:09.959445       1 builder.go:160] Restart triggered because of file /var/run/secrets/serving-cert/tls.crt was created
I0319 13:58:09.959505       1 simple_featuregate_reader.go:177] Shutting down feature-gate-detector
I0319 13:58:09.959520       1 observer_polling.go:111] Observed file "/var/run/secrets/serving-cert/tls.key" has been created (hash="8ce91617b172446e9915c956dde891ccf1c536ea76b24f0add45eaae9b8407e8")
I0319 13:58:09.959541       1 genericapiserver.go:693] "[graceful-termination] pre-shutdown hooks completed" name="PreShutdownHooksStopped"
I0319 13:58:09.959564       1 base_controller.go:181] Shutting down ConfigController ...
I0319 13:58:09.959576       1 genericapiserver.go:548] "[graceful-termination] shutdown event" name="ShutdownInitiated"
I0319 13:58:09.959587       1 base_controller.go:181] Shutting down LoggingSyncer ...
I0319 13:58:09.959606       1 base_controller.go:123] Shutting down worker of ConfigController controller ...
I0319 13:58:09.959614       1 base_controller.go:113] All ConfigController workers have been terminated
I0319 13:58:09.959621       1 genericapiserver.go:651] "[graceful-termination] not going to wait for active watch request(s) to drain"
I0319 13:58:09.959636       1 configmap_cafile_content.go:226] "Shutting down controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0319 13:58:09.959652       1 requestheader_controller.go:194] Shutting down RequestHeaderAuthRequestController
I0319 13:58:09.959687       1 observer_polling.go:111] Observed file "/var/run/configmaps/service-ca-bundle/service-ca.crt" has been created (hash="e1b7613c6606a8e8f3039bf18e3685a4a4d3ea58e4ace1b5c99b69251e8971c5")
I0319 13:58:09.959713       1 secure_serving.go:258] Stopped listening on [::]:8443
I0319 13:58:09.959654       1 object_count_tracker.go:151] "StorageObjectCountTracker pruner is exiting"
I0319 13:58:09.959662       1 base_controller.go:123] Shutting down worker of LoggingSyncer controller ...
I0319 13:58:09.959811       1 genericapiserver.go:713] [graceful-termination] apiserver is exiting
I0319 13:58:09.959817       1 base_controller.go:113] All LoggingSyncer workers have been terminated
I0319 13:58:09.959670       1 tlsconfig.go:258] "Shutting down DynamicServingCertificateController"