I0701 19:46:16.196742 1 cmd.go:253] Using service-serving-cert provided certificates I0701 19:46:16.197344 1 observer_polling.go:159] Starting file observer I0701 19:46:16.754629 1 operator.go:60] Starting insights-operator v0.0.0-master+$Format:%H$ I0701 19:46:16.754837 1 legacy_config.go:327] Current config: {"report":false,"storagePath":"/var/lib/insights-operator","interval":"2h","endpoint":"https://console.redhat.com/api/ingress/v1/upload","conditionalGathererEndpoint":"https://console.redhat.com/api/gathering/v2/%s/gathering_rules","pull_report":{"endpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports","delay":"60s","timeout":"3000s","min_retry":"30s"},"impersonate":"system:serviceaccount:openshift-insights:gather","enableGlobalObfuscation":false,"ocm":{"scaEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates","scaInterval":"8h","scaDisabled":false,"clusterTransferEndpoint":"https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/","clusterTransferInterval":"12h"},"disableInsightsAlerts":false,"processingStatusEndpoint":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status","reportEndpointTechPreview":"https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/report"} I0701 19:46:16.755369 1 secure_serving.go:57] Forcing use of http/1.1 only W0701 19:46:16.755393 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0701 19:46:16.755398 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. W0701 19:46:16.755402 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_GCM_SHA256' detected. W0701 19:46:16.755406 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_GCM_SHA384' detected. W0701 19:46:16.755409 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_128_CBC_SHA' detected. W0701 19:46:16.755413 1 secure_serving.go:69] Use of insecure cipher 'TLS_RSA_WITH_AES_256_CBC_SHA' detected. I0701 19:46:16.755455 1 simple_featuregate_reader.go:171] Starting feature-gate-detector I0701 19:46:16.758990 1 operator.go:125] FeatureGates initialized: knownFeatureGates=[AdditionalRoutingCapabilities AdminNetworkPolicy AlibabaPlatform AzureWorkloadIdentity BuildCSIVolumes CPMSMachineNamePrefix ConsolePluginContentSecurityPolicy ExternalOIDC ExternalOIDCWithUIDAndExtraClaimMappings GatewayAPI GatewayAPIController HighlyAvailableArbiter ImageVolume IngressControllerLBSubnetsAWS KMSv1 MachineConfigNodes ManagedBootImages ManagedBootImagesAWS MetricsCollectionProfiles NetworkDiagnosticsConfig NetworkLiveMigration NetworkSegmentation PinnedImages ProcMountType RouteAdvertisements RouteExternalCertificate ServiceAccountTokenNodeBinding SetEIPForNLBIngressController SigstoreImageVerification StoragePerformantSecurityPolicy UpgradeStatus UserNamespacesPodSecurityStandards UserNamespacesSupport VSphereMultiDisk VSphereMultiNetworks AWSClusterHostedDNS AWSClusterHostedDNSInstall AWSDedicatedHosts AWSServiceLBNetworkSecurityGroup AutomatedEtcdBackup AzureClusterHostedDNSInstall AzureDedicatedHosts AzureMultiDisk BootImageSkewEnforcement BootcNodeManagement ClusterAPIInstall ClusterAPIInstallIBMCloud ClusterMonitoringConfig ClusterVersionOperatorConfiguration DNSNameResolver DualReplica DyanmicServiceEndpointIBMCloud DynamicResourceAllocation EtcdBackendQuota EventedPLEG Example Example2 ExternalSnapshotMetadata GCPClusterHostedDNS GCPClusterHostedDNSInstall GCPCustomAPIEndpoints GCPCustomAPIEndpointsInstall ImageModeStatusReporting ImageStreamImportMode IngressControllerDynamicConfigurationManager InsightsConfig InsightsConfigAPI InsightsOnDemandDataGather IrreconcilableMachineConfig KMSEncryptionProvider MachineAPIMigration MachineAPIOperatorDisableMachineHealthCheckController ManagedBootImagesAzure ManagedBootImagesvSphere MaxUnavailableStatefulSet MinimumKubeletVersion MixedCPUsAllocation MultiArchInstallAzure MultiDiskSetup MutatingAdmissionPolicy NewOLM NewOLMCatalogdAPIV1Metas NewOLMOwnSingleNamespace NewOLMPreflightPermissionChecks NewOLMWebhookProviderOpenshiftServiceCA NoRegistryClusterOperations NodeSwap NutanixMultiSubnets OVNObservability OpenShiftPodSecurityAdmission PreconfiguredUDNAddresses SELinuxMount ShortCertRotation SignatureStores SigstoreImageVerificationPKI TranslateStreamCloseWebsocketRequests VSphereConfigurableMaxAllowedBlockVolumesPerNode VSphereHostVMGroupZonal VSphereMixedNodeEnv VolumeAttributesClass VolumeGroupSnapshot] I0701 19:46:16.758992 1 event.go:377] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-insights", Name:"insights-operator", UID:"3cfa0c51-13c2-4ffa-8086-a545175e7779", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FeatureGatesInitialized' FeatureGates updated to featuregates.Features{Enabled:[]v1.FeatureGateName{"AdditionalRoutingCapabilities", "AdminNetworkPolicy", "AlibabaPlatform", "AzureWorkloadIdentity", "BuildCSIVolumes", "CPMSMachineNamePrefix", "ConsolePluginContentSecurityPolicy", "ExternalOIDC", "ExternalOIDCWithUIDAndExtraClaimMappings", "GatewayAPI", "GatewayAPIController", "HighlyAvailableArbiter", "ImageVolume", "IngressControllerLBSubnetsAWS", "KMSv1", "MachineConfigNodes", "ManagedBootImages", "ManagedBootImagesAWS", "MetricsCollectionProfiles", "NetworkDiagnosticsConfig", "NetworkLiveMigration", "NetworkSegmentation", "PinnedImages", "ProcMountType", "RouteAdvertisements", "RouteExternalCertificate", "ServiceAccountTokenNodeBinding", "SetEIPForNLBIngressController", "SigstoreImageVerification", "StoragePerformantSecurityPolicy", "UpgradeStatus", "UserNamespacesPodSecurityStandards", "UserNamespacesSupport", "VSphereMultiDisk", "VSphereMultiNetworks"}, Disabled:[]v1.FeatureGateName{"AWSClusterHostedDNS", "AWSClusterHostedDNSInstall", "AWSDedicatedHosts", "AWSServiceLBNetworkSecurityGroup", "AutomatedEtcdBackup", "AzureClusterHostedDNSInstall", "AzureDedicatedHosts", "AzureMultiDisk", "BootImageSkewEnforcement", "BootcNodeManagement", "ClusterAPIInstall", "ClusterAPIInstallIBMCloud", "ClusterMonitoringConfig", "ClusterVersionOperatorConfiguration", "DNSNameResolver", "DualReplica", "DyanmicServiceEndpointIBMCloud", "DynamicResourceAllocation", "EtcdBackendQuota", "EventedPLEG", "Example", "Example2", "ExternalSnapshotMetadata", "GCPClusterHostedDNS", "GCPClusterHostedDNSInstall", "GCPCustomAPIEndpoints", "GCPCustomAPIEndpointsInstall", "ImageModeStatusReporting", "ImageStreamImportMode", "IngressControllerDynamicConfigurationManager", "InsightsConfig", "InsightsConfigAPI", "InsightsOnDemandDataGather", "IrreconcilableMachineConfig", "KMSEncryptionProvider", "MachineAPIMigration", "MachineAPIOperatorDisableMachineHealthCheckController", "ManagedBootImagesAzure", "ManagedBootImagesvSphere", "MaxUnavailableStatefulSet", "MinimumKubeletVersion", "MixedCPUsAllocation", "MultiArchInstallAzure", "MultiDiskSetup", "MutatingAdmissionPolicy", "NewOLM", "NewOLMCatalogdAPIV1Metas", "NewOLMOwnSingleNamespace", "NewOLMPreflightPermissionChecks", "NewOLMWebhookProviderOpenshiftServiceCA", "NoRegistryClusterOperations", "NodeSwap", "NutanixMultiSubnets", "OVNObservability", "OpenShiftPodSecurityAdmission", "PreconfiguredUDNAddresses", "SELinuxMount", "ShortCertRotation", "SignatureStores", "SigstoreImageVerificationPKI", "TranslateStreamCloseWebsocketRequests", "VSphereConfigurableMaxAllowedBlockVolumesPerNode", "VSphereHostVMGroupZonal", "VSphereMixedNodeEnv", "VolumeAttributesClass", "VolumeGroupSnapshot"}} I0701 19:46:16.759784 1 requestheader_controller.go:180] Starting RequestHeaderAuthRequestController I0701 19:46:16.759796 1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController I0701 19:46:16.759842 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0701 19:46:16.759863 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0701 19:46:16.759842 1 configmap_cafile_content.go:205] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0701 19:46:16.759878 1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0701 19:46:16.760082 1 dynamic_serving_content.go:135] "Starting controller" name="serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key" I0701 19:46:16.760326 1 secure_serving.go:213] Serving securely on [::]:8443 I0701 19:46:16.760363 1 tlsconfig.go:243] "Starting DynamicServingCertificateController" W0701 19:46:16.763976 1 configmapobserver.go:64] Cannot get the configuration config map: configmaps "insights-config" not found. Default configuration is used. I0701 19:46:16.763998 1 secretconfigobserver.go:216] Legacy configuration set: enabled=false endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=false reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0701 19:46:16.764074 1 base_controller.go:76] Waiting for caches to sync for ConfigController I0701 19:46:16.768969 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:46:16.769004 1 secretconfigobserver.go:204] Legacy configuration updated: enabled=true endpoint=https://console.redhat.com/api/ingress/v1/upload conditional_gatherer_endpoint=https://console.redhat.com/api/gathering/v2/%s/gathering_rules interval=2h0m0s token=true reportEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports initialPollingDelay=1m0s minRetryTime=30s pollingTimeout=50m0s processingStatusEndpoint=https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/request/%s/status I0701 19:46:16.774946 1 secretconfigobserver.go:119] support secret does not exist I0701 19:46:16.778554 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:46:16.781843 1 secretconfigobserver.go:119] support secret does not exist I0701 19:46:16.784997 1 recorder.go:161] Pruning old reports every 5h22m4s, max age is 288h0m0s I0701 19:46:16.789071 1 controllerstatus.go:80] name=insightsuploader healthy=true reason= message= I0701 19:46:16.789077 1 controllerstatus.go:80] name=insightsreport healthy=true reason= message= I0701 19:46:16.789087 1 insightsreport.go:296] Starting report retriever I0701 19:46:16.789091 1 insightsuploader.go:86] Reporting status periodically to https://console.redhat.com/api/ingress/v1/upload every 2h0m0s, starting in 1m30s I0701 19:46:16.789094 1 insightsreport.go:298] Insights analysis reports will be downloaded from the https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports endpoint with a delay of 1m0s I0701 19:46:16.789132 1 periodic.go:209] Running clusterconfig gatherer I0701 19:46:16.789192 1 tasks_processing.go:45] number of workers: 32 I0701 19:46:16.789217 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:46:16.789231 1 tasks_processing.go:69] worker 12 listening for tasks. I0701 19:46:16.789237 1 tasks_processing.go:71] worker 12 working on authentication task. I0701 19:46:16.789238 1 tasks_processing.go:69] worker 2 listening for tasks. I0701 19:46:16.789242 1 tasks_processing.go:69] worker 4 listening for tasks. I0701 19:46:16.789248 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:46:16.789247 1 tasks_processing.go:69] worker 13 listening for tasks. I0701 19:46:16.789256 1 tasks_processing.go:69] worker 14 listening for tasks. I0701 19:46:16.789258 1 tasks_processing.go:69] worker 23 listening for tasks. I0701 19:46:16.789263 1 tasks_processing.go:69] worker 27 listening for tasks. I0701 19:46:16.789264 1 tasks_processing.go:69] worker 15 listening for tasks. I0701 19:46:16.789269 1 tasks_processing.go:69] worker 8 listening for tasks. I0701 19:46:16.789271 1 tasks_processing.go:69] worker 16 listening for tasks. I0701 19:46:16.789276 1 tasks_processing.go:69] worker 24 listening for tasks. I0701 19:46:16.789282 1 tasks_processing.go:69] worker 5 listening for tasks. I0701 19:46:16.789282 1 tasks_processing.go:69] worker 17 listening for tasks. I0701 19:46:16.789287 1 tasks_processing.go:69] worker 25 listening for tasks. I0701 19:46:16.789289 1 tasks_processing.go:69] worker 18 listening for tasks. I0701 19:46:16.789295 1 tasks_processing.go:69] worker 26 listening for tasks. I0701 19:46:16.789297 1 tasks_processing.go:69] worker 6 listening for tasks. I0701 19:46:16.789303 1 tasks_processing.go:69] worker 19 listening for tasks. I0701 19:46:16.789303 1 tasks_processing.go:69] worker 28 listening for tasks. I0701 19:46:16.789308 1 tasks_processing.go:69] worker 7 listening for tasks. I0701 19:46:16.789307 1 tasks_processing.go:69] worker 30 listening for tasks. I0701 19:46:16.789315 1 tasks_processing.go:69] worker 31 listening for tasks. I0701 19:46:16.789316 1 tasks_processing.go:69] worker 29 listening for tasks. I0701 19:46:16.789321 1 tasks_processing.go:69] worker 20 listening for tasks. I0701 19:46:16.789324 1 tasks_processing.go:69] worker 21 listening for tasks. I0701 19:46:16.789328 1 tasks_processing.go:69] worker 22 listening for tasks. I0701 19:46:16.789332 1 tasks_processing.go:69] worker 11 listening for tasks. I0701 19:46:16.789332 1 tasks_processing.go:69] worker 10 listening for tasks. I0701 19:46:16.789334 1 tasks_processing.go:69] worker 9 listening for tasks. I0701 19:46:16.789338 1 tasks_processing.go:71] worker 14 working on active_alerts task. I0701 19:46:16.789341 1 tasks_processing.go:71] worker 4 working on node_logs task. I0701 19:46:16.789342 1 tasks_processing.go:71] worker 8 working on pdbs task. I0701 19:46:16.789346 1 tasks_processing.go:71] worker 9 working on storage_classes task. I0701 19:46:16.789410 1 tasks_processing.go:71] worker 16 working on machine_sets task. I0701 19:46:16.789412 1 tasks_processing.go:71] worker 2 working on clusterroles task. I0701 19:46:16.789418 1 tasks_processing.go:71] worker 15 working on image_registries task. I0701 19:46:16.789412 1 tasks_processing.go:71] worker 23 working on metrics task. I0701 19:46:16.789237 1 tasks_processing.go:71] worker 0 working on image task. I0701 19:46:16.789654 1 tasks_processing.go:71] worker 24 working on openshift_logging task. I0701 19:46:16.789716 1 tasks_processing.go:71] worker 13 working on config_maps task. I0701 19:46:16.789415 1 tasks_processing.go:71] worker 27 working on machine_autoscalers task. I0701 19:46:16.789994 1 tasks_processing.go:71] worker 7 working on operators task. I0701 19:46:16.790042 1 tasks_processing.go:71] worker 30 working on nodenetworkstates task. I0701 19:46:16.790146 1 tasks_processing.go:71] worker 1 working on overlapping_namespace_uids task. I0701 19:46:16.790239 1 tasks_processing.go:71] worker 5 working on operators_pods_and_events task. I0701 19:46:16.790295 1 tasks_processing.go:71] worker 31 working on container_runtime_configs task. I0701 19:46:16.790525 1 tasks_processing.go:71] worker 26 working on crds task. I0701 19:46:16.790725 1 tasks_processing.go:69] worker 3 listening for tasks. I0701 19:46:16.790809 1 tasks_processing.go:71] worker 3 working on feature_gates task. I0701 19:46:16.790532 1 tasks_processing.go:71] worker 29 working on machines task. I0701 19:46:16.790540 1 tasks_processing.go:71] worker 20 working on tsdb_status task. I0701 19:46:16.790544 1 tasks_processing.go:71] worker 21 working on qemu_kubevirt_launcher_logs task. I0701 19:46:16.790548 1 tasks_processing.go:71] worker 22 working on nodes task. I0701 19:46:16.790553 1 tasks_processing.go:71] worker 11 working on install_plans task. I0701 19:46:16.790556 1 tasks_processing.go:71] worker 10 working on ingress_certificates task. I0701 19:46:16.790569 1 tasks_processing.go:71] worker 25 working on sap_datahubs task. I0701 19:46:16.790573 1 tasks_processing.go:71] worker 17 working on schedulers task. I0701 19:46:16.790584 1 tasks_processing.go:71] worker 18 working on ceph_cluster task. I0701 19:46:16.790595 1 tasks_processing.go:71] worker 19 working on number_of_pods_and_netnamespaces_with_sdn_annotations task. I0701 19:46:16.790599 1 tasks_processing.go:71] worker 6 working on machine_healthchecks task. I0701 19:46:16.790610 1 tasks_processing.go:71] worker 28 working on openshift_machine_api_events task. I0701 19:46:16.792570 1 tasks_processing.go:71] worker 27 working on sap_config task. I0701 19:46:16.792639 1 gather.go:177] gatherer "clusterconfig" function "machine_autoscalers" took 2.789772ms to process 0 records E0701 19:46:16.794590 1 gather_most_recent_metrics.go:87] Unable to retrieve most recent metrics: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.251.150:9091: connect: connection refused E0701 19:46:16.794602 1 gather_prometheus_tsdb_status.go:49] Unable to tsdb status: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.251.150:9091: connect: connection refused I0701 19:46:16.794615 1 tasks_processing.go:71] worker 23 working on validating_webhook_configurations task. E0701 19:46:16.794622 1 gather.go:140] gatherer "clusterconfig" function "metrics" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/federate?match%5B%5D=cluster_installer&match%5B%5D=namespace%3Acontainer_cpu_usage%3Asum&match%5B%5D=namespace%3Acontainer_memory_usage_bytes%3Asum&match%5B%5D=vsphere_node_hw_version_total&match%5B%5D=virt_platform&match%5B%5D=console_helm_installs_total&match%5B%5D=console_helm_upgrades_total&match%5B%5D=console_helm_uninstalls_total&match%5B%5D=openshift_apps_deploymentconfigs_strategy_total&match%5B%5D=etcd_server_slow_apply_total&match%5B%5D=etcd_server_slow_read_indexes_total&match%5B%5D=haproxy_exporter_server_threshold": dial tcp 172.30.251.150:9091: connect: connection refused I0701 19:46:16.794647 1 gather.go:177] gatherer "clusterconfig" function "metrics" took 5.147609ms to process 0 records E0701 19:46:16.794693 1 gather.go:140] gatherer "clusterconfig" function "tsdb_status" failed with the error: Get "https://prometheus-k8s.openshift-monitoring.svc:9091/api/v1/status/tsdb": dial tcp 172.30.251.150:9091: connect: connection refused I0701 19:46:16.794708 1 tasks_processing.go:71] worker 20 working on machine_configs task. I0701 19:46:16.794710 1 gather.go:177] gatherer "clusterconfig" function "tsdb_status" took 3.647743ms to process 0 records I0701 19:46:16.803850 1 tasks_processing.go:71] worker 31 working on silenced_alerts task. I0701 19:46:16.804462 1 gather.go:177] gatherer "clusterconfig" function "container_runtime_configs" took 13.540767ms to process 0 records I0701 19:46:16.804506 1 controller.go:128] Initializing last reported time to 0001-01-01T00:00:00Z I0701 19:46:16.804551 1 tasks_processing.go:71] worker 16 working on jaegers task. I0701 19:46:16.804567 1 controller.go:326] The initial operator extension status is healthy I0701 19:46:16.804580 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0701 19:46:16.804588 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0701 19:46:16.804591 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0701 19:46:16.804616 1 controller.go:489] The operator is still being initialized I0701 19:46:16.804632 1 controller.go:512] The operator is healthy I0701 19:46:16.804542 1 gather.go:177] gatherer "clusterconfig" function "machine_sets" took 14.458595ms to process 0 records I0701 19:46:16.804804 1 gather.go:177] gatherer "clusterconfig" function "openshift_logging" took 14.287626ms to process 0 records E0701 19:46:16.804814 1 gather.go:140] gatherer "clusterconfig" function "machines" failed with the error: machines.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machines" in API group "machine.openshift.io" at the cluster scope I0701 19:46:16.804823 1 gather.go:177] gatherer "clusterconfig" function "machines" took 13.210474ms to process 0 records W0701 19:46:16.804891 1 operator.go:288] started I0701 19:46:16.804891 1 tasks_processing.go:71] worker 24 working on dvo_metrics task. I0701 19:46:16.804924 1 sca.go:136] Pulling SCA certificates from https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates. Next check is in 8h0m0s I0701 19:46:16.804968 1 recorder.go:75] Recording config/image with fingerprint=18cc3b3b0272fb23ea87516a610eaeacf03d9a4eba9e146011578a94dd5df0c7 I0701 19:46:16.804987 1 gather.go:177] gatherer "clusterconfig" function "image" took 14.589941ms to process 1 records I0701 19:46:16.805016 1 tasks_processing.go:71] worker 12 working on cluster_apiserver task. I0701 19:46:16.805036 1 cluster_transfer.go:83] checking the availability of cluster transfer. Next check is in 12h0m0s I0701 19:46:16.806735 1 base_controller.go:76] Waiting for caches to sync for LoggingSyncer I0701 19:46:16.807456 1 recorder.go:75] Recording config/authentication with fingerprint=7cd088374b1c3d03819b8a517b4bbf62da4e069492b2ae6e51210dfc555f2a00 I0701 19:46:16.808073 1 gather.go:177] gatherer "clusterconfig" function "authentication" took 15.18362ms to process 1 records I0701 19:46:16.807620 1 tasks_processing.go:71] worker 0 working on lokistack task. I0701 19:46:16.807720 1 tasks_processing.go:71] worker 29 working on storage_cluster task. I0701 19:46:16.808218 1 tasks_processing.go:71] worker 15 working on proxies task. I0701 19:46:16.809849 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/config/cluster with fingerprint=b22b8835d5d70fcee330d3cc31a5a04096ec3c934e9553ce154c3c54ac1e59bf I0701 19:46:16.809914 1 gather.go:177] gatherer "clusterconfig" function "image_registries" took 16.735846ms to process 1 records I0701 19:46:16.809926 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkstates" took 17.353788ms to process 0 records I0701 19:46:16.810024 1 recorder.go:75] Recording config/storage/storageclasses/gp2-csi with fingerprint=dafcbe4da28039954663aa320efdc01aaeb8502d5bfb07b8fd26f09716b8c2be I0701 19:46:16.810052 1 recorder.go:75] Recording config/storage/storageclasses/gp3-csi with fingerprint=dfb3948ebf07641d03773ce3aa63cb9592f3a7755de924ba1997e558f7d29d6b I0701 19:46:16.810060 1 gather.go:177] gatherer "clusterconfig" function "storage_classes" took 20.453529ms to process 2 records E0701 19:46:16.810068 1 gather.go:140] gatherer "clusterconfig" function "machine_healthchecks" failed with the error: machinehealthchecks.machine.openshift.io is forbidden: User "system:serviceaccount:openshift-insights:gather" cannot list resource "machinehealthchecks" in API group "machine.openshift.io" at the cluster scope I0701 19:46:16.810077 1 gather.go:177] gatherer "clusterconfig" function "machine_healthchecks" took 17.900744ms to process 0 records I0701 19:46:16.810087 1 tasks_processing.go:71] worker 6 working on olm_operators task. I0701 19:46:16.810221 1 tasks_processing.go:71] worker 9 working on nodenetworkconfigurationpolicies task. I0701 19:46:16.810265 1 tasks_processing.go:71] worker 30 working on image_pruners task. I0701 19:46:16.811090 1 tasks_processing.go:71] worker 22 working on cost_management_metrics_configs task. I0701 19:46:16.811532 1 recorder.go:75] Recording config/node/ip-10-0-0-56.ec2.internal with fingerprint=29c80c79c36b1f0c26107a0686bfaa99c1db33656b668885408f990d0ac03bfa I0701 19:46:16.811888 1 recorder.go:75] Recording config/node/ip-10-0-1-54.ec2.internal with fingerprint=3b991621cd58c1d927afbf6b47d129ce45755644d7abe1a70f2b6b5ca4be6057 I0701 19:46:16.811978 1 recorder.go:75] Recording config/node/ip-10-0-2-238.ec2.internal with fingerprint=99c20c458d8b8d6429d1e7b19a4fed74ddf90c2b7de27675aa2f8027167b3044 I0701 19:46:16.811988 1 gather.go:177] gatherer "clusterconfig" function "nodes" took 19.635151ms to process 3 records I0701 19:46:16.812101 1 recorder.go:75] Recording config/pdbs/openshift-console/console with fingerprint=215e5986e0dadcb9e2166455e88dd9112d275a7c7a870d978d3d7ce8686f8505 I0701 19:46:16.812119 1 tasks_processing.go:71] worker 8 working on machine_config_pools task. I0701 19:46:16.812126 1 recorder.go:75] Recording config/pdbs/openshift-console/downloads with fingerprint=d41f6c8d00f7b7b2b70c5c858ad0a9a8e46fc605f25760207710d6c8f3d36092 I0701 19:46:16.812147 1 recorder.go:75] Recording config/pdbs/openshift-image-registry/image-registry with fingerprint=5cba7228c648a33e80982a67619eb7ea0c5bc9d97e15430d6619ea5463d91f0a I0701 19:46:16.812171 1 recorder.go:75] Recording config/pdbs/openshift-ingress/router-default with fingerprint=fb5eae3474e5f2777e35cdfc7fde70bc167f762910cdfe519fb343fb47ee2e35 I0701 19:46:16.812194 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/alertmanager-main with fingerprint=355e291b1ed7862e91ec56180c754595a7973aaabf794226f1cad4cfc844d7e6 I0701 19:46:16.812214 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/metrics-server with fingerprint=b64d6a6628d417b07c2194afc34ac07adea9f273a683bf4bd0f1cd1386377fdc I0701 19:46:16.812283 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/monitoring-plugin with fingerprint=c3c907b5e88a77cd0d041f3232916f2d80259c50526a78e808aaf3b9fa6d34cb I0701 19:46:16.812309 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-k8s with fingerprint=66b6a384e4d4c9ecaaf0daefb523259ce46ea9af73d9f7d272191450db7c373c I0701 19:46:16.812329 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/prometheus-operator-admission-webhook with fingerprint=63649d04edb5752cf02c11215a0509fc3c3dba381bf0e1cc2ff87ea67a850a71 I0701 19:46:16.812351 1 recorder.go:75] Recording config/pdbs/openshift-monitoring/thanos-querier-pdb with fingerprint=75d7f3adc0754042bd75644b5ca3ffd7dc8658205e7a5a040e53a8d73554fc6a I0701 19:46:16.812372 1 recorder.go:75] Recording config/pdbs/openshift-operator-lifecycle-manager/packageserver-pdb with fingerprint=050ff81c1a532494b7520a533cb47ced7b4ac5b0638960beb99c741f77429315 I0701 19:46:16.812381 1 gather.go:177] gatherer "clusterconfig" function "pdbs" took 21.750308ms to process 11 records I0701 19:46:16.812409 1 gather.go:177] gatherer "clusterconfig" function "node_logs" took 21.778026ms to process 0 records I0701 19:46:16.812460 1 tasks_processing.go:71] worker 4 working on aggregated_monitoring_cr_names task. I0701 19:46:16.812578 1 recorder.go:75] Recording config/featuregate with fingerprint=78dc1d661245e14b7f9679fbe66b7f1ce4ac3c8b7db24789674624a63dd4fa80 I0701 19:46:16.812588 1 gather.go:177] gatherer "clusterconfig" function "feature_gates" took 21.252932ms to process 1 records I0701 19:46:16.812603 1 recorder.go:75] Recording config/namespaces_with_overlapping_uids with fingerprint=4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 I0701 19:46:16.812609 1 gather.go:177] gatherer "clusterconfig" function "overlapping_namespace_uids" took 22.073568ms to process 1 records I0701 19:46:16.812612 1 tasks_processing.go:71] worker 3 working on sap_pods task. I0701 19:46:16.812619 1 tasks_processing.go:71] worker 1 working on support_secret task. I0701 19:46:16.814545 1 tasks_processing.go:71] worker 5 working on infrastructures task. I0701 19:46:16.814575 1 gather.go:177] gatherer "clusterconfig" function "operators_pods_and_events" took 24.285144ms to process 0 records I0701 19:46:16.815398 1 tasks_processing.go:71] worker 25 working on networks task. I0701 19:46:16.815547 1 gather.go:177] gatherer "clusterconfig" function "sap_datahubs" took 23.628472ms to process 0 records I0701 19:46:16.815576 1 tasks_processing.go:71] worker 16 working on container_images task. I0701 19:46:16.815589 1 gather.go:177] gatherer "clusterconfig" function "jaegers" took 11.017154ms to process 0 records I0701 19:46:16.815599 1 gather.go:177] gatherer "clusterconfig" function "ceph_cluster" took 23.635382ms to process 0 records I0701 19:46:16.815641 1 gather_logs.go:145] no pods in namespace were found I0701 19:46:16.815650 1 gather.go:177] gatherer "clusterconfig" function "qemu_kubevirt_launcher_logs" took 24.269833ms to process 0 records I0701 19:46:16.815696 1 tasks_processing.go:71] worker 21 working on mutating_webhook_configurations task. I0701 19:46:16.815702 1 tasks_processing.go:71] worker 18 working on oauths task. I0701 19:46:16.816961 1 tasks_processing.go:71] worker 17 working on version task. I0701 19:46:16.817215 1 recorder.go:75] Recording config/schedulers/cluster with fingerprint=694f21c45546b25b0ff6a2dfa1fa48d02c456708f807cf9ca3d0b4ddbcb40791 I0701 19:46:16.817241 1 gather.go:177] gatherer "clusterconfig" function "schedulers" took 25.13056ms to process 1 records I0701 19:46:16.820158 1 tasks_processing.go:71] worker 27 working on openstack_version task. I0701 19:46:16.820298 1 gather.go:177] gatherer "clusterconfig" function "sap_config" took 27.523163ms to process 0 records I0701 19:46:16.820535 1 tasks_processing.go:71] worker 28 working on openstack_controlplanes task. I0701 19:46:16.820550 1 gather.go:177] gatherer "clusterconfig" function "openshift_machine_api_events" took 28.244541ms to process 0 records I0701 19:46:16.820940 1 tasks_processing.go:71] worker 12 working on certificate_signing_requests task. I0701 19:46:16.821001 1 controller.go:212] Source periodic-workloads *controllerstatus.Simple is not ready I0701 19:46:16.821012 1 controller.go:212] Source scaController *sca.Controller is not ready I0701 19:46:16.821016 1 controller.go:212] Source clusterTransferController *clustertransfer.Controller is not ready I0701 19:46:16.821018 1 controller.go:212] Source periodic-clusterconfig *controllerstatus.Simple is not ready I0701 19:46:16.821021 1 controller.go:212] Source periodic-conditional *controllerstatus.Simple is not ready I0701 19:46:16.821036 1 controller.go:489] The operator is still being initialized I0701 19:46:16.821041 1 controller.go:512] The operator is healthy I0701 19:46:16.821199 1 recorder.go:75] Recording config/apiserver with fingerprint=739036979325439281e75fe52889bf4d695d4a5aa949e5799a643373d69cfaf5 I0701 19:46:16.821217 1 gather.go:177] gatherer "clusterconfig" function "cluster_apiserver" took 15.910799ms to process 1 records I0701 19:46:16.821455 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/admin with fingerprint=0418f4ee496830403a7c19f2e8580adba7a81e4f3fb66fa820d8e0e8cc41f832 I0701 19:46:16.821466 1 tasks_processing.go:71] worker 2 working on openstack_dataplanenodesets task. I0701 19:46:16.821601 1 recorder.go:75] Recording cluster-scoped-resources/rbac.authorization.k8s.io/clusterroles/edit with fingerprint=83c99cbfc8cd2fe8d60e8478f60104dcce0a22e8e4cd6cff9cbd51a7d6d8b4eb I0701 19:46:16.821612 1 gather.go:177] gatherer "clusterconfig" function "clusterroles" took 31.701598ms to process 2 records I0701 19:46:16.823461 1 tasks_processing.go:71] worker 22 working on ingress task. I0701 19:46:16.823464 1 gather.go:177] gatherer "clusterconfig" function "cost_management_metrics_configs" took 12.353006ms to process 0 records I0701 19:46:16.823479 1 gather.go:177] gatherer "clusterconfig" function "nodenetworkconfigurationpolicies" took 13.22426ms to process 0 records I0701 19:46:16.823490 1 tasks_processing.go:71] worker 9 working on service_accounts task. I0701 19:46:16.823497 1 tasks_processing.go:71] worker 29 working on monitoring_persistent_volumes task. I0701 19:46:16.823567 1 gather.go:177] gatherer "clusterconfig" function "storage_cluster" took 15.257239ms to process 0 records I0701 19:46:16.823743 1 tasks_processing.go:71] worker 23 working on pod_network_connectivity_checks task. I0701 19:46:16.823886 1 recorder.go:75] Recording config/validatingwebhookconfigurations/alertmanagerconfigs.openshift.io with fingerprint=6ba4aba9917d5b2673ac7e773507cdac0bf0714babf214d7622488ad9ec5ac31 I0701 19:46:16.823959 1 recorder.go:75] Recording config/validatingwebhookconfigurations/multus.openshift.io with fingerprint=b6fa8730ad6365e4c716fe7541de23f9179d38a87f7c2cc2a814e8f6c43d7a24 I0701 19:46:16.824061 1 recorder.go:75] Recording config/validatingwebhookconfigurations/network-node-identity.openshift.io with fingerprint=7f8b6df96526127f62d3199e1c6de0e3b74f2953a4f1db1b3a7af55c5e0ec546 I0701 19:46:16.824086 1 recorder.go:75] Recording config/validatingwebhookconfigurations/performance-addon-operator with fingerprint=dcf1f03f8c563820f0804f994c91937b6bce10177c4c2c9fcbfb32feddfa7239 I0701 19:46:16.824112 1 recorder.go:75] Recording config/validatingwebhookconfigurations/prometheusrules.openshift.io with fingerprint=7803169ba6013de1fc35f0d49e04a90b3083c03760b71416d8bd575493386312 I0701 19:46:16.824136 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterrolebindings-validation with fingerprint=d25a932e96a57b893321846817079aab9c75898c18bec87a20542b856c77087e I0701 19:46:16.824160 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-clusterroles-validation with fingerprint=06f8f664bef45a36392edc951628cc704f0ec68b01d5e82f5ae9d9274cfb6da8 I0701 19:46:16.824195 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-ingress-config-validation with fingerprint=77e72562dcf618c69f0b8bd0b5346234518f99a91f39d4a7c5164b9298aed29e I0701 19:46:16.824218 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-network-operator-validation with fingerprint=3af199153ef33ac9e8a80a9dfe44c847e28c7bf68ce53ead3bc86b50e9f1e491 I0701 19:46:16.824249 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-regular-user-validation with fingerprint=bcda980323ea089ef148565a1d71276683a1da0efcdd6728b2ad9cb147b72271 I0701 19:46:16.824273 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-scc-validation with fingerprint=88325f8473293c692cebf8d4658e004a239e736c30908ac1c7c73ff5d3488f7a I0701 19:46:16.824297 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-serviceaccount-validation with fingerprint=947db6500ec474b15cebfde95db25d78038fb5a729bc20ea05087b917498993b I0701 19:46:16.824322 1 recorder.go:75] Recording config/validatingwebhookconfigurations/sre-techpreviewnoupgrade-validation with fingerprint=b93958f5e53cf6136591048335af1fd98c5da5b5b52e683a5493375fa552928e I0701 19:46:16.824330 1 gather.go:177] gatherer "clusterconfig" function "validating_webhook_configurations" took 29.113902ms to process 13 records I0701 19:46:16.833389 1 tasks_processing.go:71] worker 3 working on openstack_dataplanedeployments task. I0701 19:46:16.833404 1 gather.go:177] gatherer "clusterconfig" function "sap_pods" took 20.759172ms to process 0 records I0701 19:46:16.833522 1 gather.go:177] gatherer "clusterconfig" function "openstack_version" took 13.289607ms to process 0 records I0701 19:46:16.833530 1 gather.go:177] gatherer "clusterconfig" function "openstack_controlplanes" took 12.932044ms to process 0 records I0701 19:46:16.833535 1 gather.go:177] gatherer "clusterconfig" function "lokistack" took 25.404033ms to process 0 records I0701 19:46:16.833543 1 tasks_processing.go:74] worker 0 stopped. I0701 19:46:16.833549 1 tasks_processing.go:74] worker 27 stopped. I0701 19:46:16.833553 1 tasks_processing.go:74] worker 28 stopped. I0701 19:46:16.833855 1 tasks_processing.go:74] worker 25 stopped. I0701 19:46:16.833993 1 recorder.go:75] Recording config/network with fingerprint=307cbe0f5435b79f5e23eb4152a62c37c239333bb5160c292340e3ad0ff16cf7 I0701 19:46:16.834012 1 gather.go:177] gatherer "clusterconfig" function "networks" took 18.443786ms to process 1 records I0701 19:46:16.834286 1 tasks_processing.go:74] worker 5 stopped. I0701 19:46:16.834803 1 recorder.go:75] Recording config/infrastructure with fingerprint=54a3ab1616b8bf36ec2dfcf43d08ff629024004076211468521ec7fc63f6f1d5 I0701 19:46:16.834821 1 gather.go:177] gatherer "clusterconfig" function "infrastructures" took 19.723322ms to process 1 records I0701 19:46:16.837147 1 tasks_processing.go:74] worker 2 stopped. I0701 19:46:16.837198 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanenodesets" took 15.670254ms to process 0 records E0701 19:46:16.837268 1 gather.go:140] gatherer "clusterconfig" function "pod_network_connectivity_checks" failed with the error: the server could not find the requested resource (get podnetworkconnectivitychecks.controlplane.operator.openshift.io) I0701 19:46:16.837407 1 gather.go:177] gatherer "clusterconfig" function "pod_network_connectivity_checks" took 13.403185ms to process 0 records I0701 19:46:16.837283 1 tasks_processing.go:74] worker 23 stopped. I0701 19:46:16.837826 1 tasks_processing.go:74] worker 26 stopped. I0701 19:46:16.838750 1 recorder.go:75] Recording config/crd/volumesnapshots.snapshot.storage.k8s.io with fingerprint=31207cadfd983176498b9263711956e9c7f9e6fedf79ab09da171567ddc79ac7 I0701 19:46:16.839712 1 recorder.go:75] Recording config/crd/volumesnapshotcontents.snapshot.storage.k8s.io with fingerprint=df0ec1848ef6c3531f1e3bcf810283eb395cbb87b3fb5c214e257143bcc13bc9 I0701 19:46:16.839769 1 gather.go:177] gatherer "clusterconfig" function "crds" took 46.661501ms to process 2 records I0701 19:46:16.839906 1 recorder.go:75] Recording config/proxy with fingerprint=d6a62e8f21418b92247216b5bcce9143d178ef69e5412853361d04daa3126c8f I0701 19:46:16.839940 1 gather.go:177] gatherer "clusterconfig" function "proxies" took 29.266296ms to process 1 records I0701 19:46:16.839962 1 tasks_processing.go:74] worker 15 stopped. I0701 19:46:16.840061 1 tasks_processing.go:74] worker 22 stopped. I0701 19:46:16.840267 1 recorder.go:75] Recording config/ingress with fingerprint=f72b3d7441970fdacc482191c109c8949940ea58536bb4e8551493413b051483 I0701 19:46:16.840307 1 gather.go:177] gatherer "clusterconfig" function "ingress" took 14.324588ms to process 1 records I0701 19:46:16.840494 1 recorder.go:75] Recording config/clusteroperator/imageregistry.operator.openshift.io/imagepruner/cluster with fingerprint=3e5803b7b22e1d697d9f5d9a8ed8ce17e7af63bce3a50118761382fd374039ce I0701 19:46:16.840540 1 gather.go:177] gatherer "clusterconfig" function "image_pruners" took 27.595943ms to process 1 records I0701 19:46:16.840797 1 tasks_processing.go:74] worker 30 stopped. I0701 19:46:16.840882 1 tasks_processing.go:74] worker 18 stopped. I0701 19:46:16.841118 1 recorder.go:75] Recording config/oauth with fingerprint=8c193bd8a2f4e6a1b9f370d3bdd133114da524dd7122902735a6a92d387d10eb I0701 19:46:16.841166 1 gather.go:177] gatherer "clusterconfig" function "oauths" took 22.223786ms to process 1 records I0701 19:46:16.841227 1 recorder.go:75] Recording config/alerts with fingerprint=0dcc8022419866eaf1b4d588f56802cb2553c287967a368ffbf04a61a572dc61 I0701 19:46:16.841268 1 gather.go:177] gatherer "clusterconfig" function "active_alerts" took 51.792193ms to process 1 records I0701 19:46:16.841300 1 tasks_processing.go:74] worker 14 stopped. I0701 19:46:16.842987 1 tasks_processing.go:74] worker 6 stopped. I0701 19:46:16.843006 1 gather.go:177] gatherer "clusterconfig" function "olm_operators" took 32.893151ms to process 0 records I0701 19:46:16.843258 1 tasks_processing.go:74] worker 29 stopped. I0701 19:46:16.843274 1 gather.go:177] gatherer "clusterconfig" function "monitoring_persistent_volumes" took 19.746381ms to process 0 records I0701 19:46:16.843368 1 tasks_processing.go:74] worker 1 stopped. E0701 19:46:16.843441 1 gather.go:140] gatherer "clusterconfig" function "support_secret" failed with the error: secrets "support" not found I0701 19:46:16.843464 1 gather.go:177] gatherer "clusterconfig" function "support_secret" took 30.738877ms to process 0 records I0701 19:46:16.843475 1 gather.go:177] gatherer "clusterconfig" function "openstack_dataplanedeployments" took 9.93836ms to process 0 records I0701 19:46:16.843552 1 tasks_processing.go:74] worker 3 stopped. W0701 19:46:16.843639 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0701 19:46:16.843659 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/aws-pod-identity with fingerprint=b18daeff97304f426f8711c89cd62f7a6976cf3dcb5d40ac5103bb0ee5c0d2d6 I0701 19:46:16.843782 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-podimagespec-mutation with fingerprint=6a0e05f6268c870465453cb05555422d010832fc4cbdfaf95ff38d2d67993fee I0701 19:46:16.843821 1 recorder.go:75] Recording config/mutatingwebhookconfigurations/sre-service-mutation with fingerprint=6b4524f09bdb81b9f4a24a596db94f98744bebc390c6cd68f4fe11fd3bae3b7d I0701 19:46:16.843831 1 gather.go:177] gatherer "clusterconfig" function "mutating_webhook_configurations" took 27.761187ms to process 3 records I0701 19:46:16.843666 1 tasks_processing.go:74] worker 21 stopped. I0701 19:46:16.844428 1 tasks_processing.go:74] worker 31 stopped. I0701 19:46:16.844443 1 recorder.go:75] Recording config/silenced_alerts.json with fingerprint=37517e5f3dc66819f61f5a7bb8ace1921282415f10551d2defa5c3eb0985b570 I0701 19:46:16.844451 1 gather.go:177] gatherer "clusterconfig" function "silenced_alerts" took 40.504975ms to process 1 records I0701 19:46:16.845936 1 requests.go:205] Asking for SCA certificate with "{"arch": ["x86_64"]}" payload I0701 19:46:16.846445 1 tasks_processing.go:74] worker 12 stopped. I0701 19:46:16.846474 1 gather.go:177] gatherer "clusterconfig" function "certificate_signing_requests" took 25.476819ms to process 0 records I0701 19:46:16.856864 1 tasks_processing.go:74] worker 4 stopped. I0701 19:46:16.856882 1 gather.go:177] gatherer "clusterconfig" function "aggregated_monitoring_cr_names" took 44.38549ms to process 0 records I0701 19:46:16.858504 1 tasks_processing.go:74] worker 19 stopped. I0701 19:46:16.858518 1 gather.go:177] gatherer "clusterconfig" function "number_of_pods_and_netnamespaces_with_sdn_annotations" took 66.398078ms to process 0 records I0701 19:46:16.860661 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0701 19:46:16.860705 1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0701 19:46:16.860840 1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController I0701 19:46:16.864306 1 base_controller.go:82] Caches are synced for ConfigController I0701 19:46:16.864322 1 base_controller.go:119] Starting #1 worker of ConfigController controller ... I0701 19:46:16.865088 1 tasks_processing.go:74] worker 13 stopped. E0701 19:46:16.865106 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "cluster-monitoring-config" not found E0701 19:46:16.865113 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "gateway-mode-config" not found E0701 19:46:16.865117 1 gather.go:140] gatherer "clusterconfig" function "config_maps" failed with the error: configmaps "insights-config" not found I0701 19:46:16.865129 1 recorder.go:75] Recording config/configmaps/openshift-config/installer-images/images.json with fingerprint=26b6661162b099a0f5a279859b4f46c867929a79d9a4a41fde4be4e6fe138018 I0701 19:46:16.865200 1 recorder.go:75] Recording config/configmaps/openshift-config/kube-root-ca.crt/ca.crt with fingerprint=f73fcd48d6f9ebbc83253a1b5f23ef59d0683465ec46e474389103c717467dfb I0701 19:46:16.865213 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/invoker with fingerprint=76b482f683cd3ef9da02debac5b26080a5aeb06ff768ee5c21117514dff29d8a I0701 19:46:16.865219 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-install/version with fingerprint=c93090eb0d2a4736885abeb79c91680cfd01fda46464f83456b085d4dc8239f0 I0701 19:46:16.865232 1 recorder.go:75] Recording config/configmaps/openshift-config/openshift-service-ca.crt/service-ca.crt with fingerprint=34cdf5d36d45a9e3ee7c2be31546e3d937ae4ebf144ac07b0210c06a1610a94d I0701 19:46:16.865285 1 recorder.go:75] Recording config/configmaps/openshift-config/rosa-brand-logo/rosa-brand-logo.svg with fingerprint=6ed8ca4dd7a8eee7249182bc006e9649ce84d76c551ddfaaa33e55d8c4cc1ed0 I0701 19:46:16.865316 1 recorder.go:75] Recording config/configmaps/kube-system/cluster-config-v1/install-config with fingerprint=ab3811c6b83fd7b8e920094cfa3080d1b4ee3c35ec4c8379437b21d27bd6608d I0701 19:46:16.865325 1 gather.go:177] gatherer "clusterconfig" function "config_maps" took 75.348368ms to process 7 records I0701 19:46:16.865745 1 tasks_processing.go:74] worker 17 stopped. I0701 19:46:16.866080 1 recorder.go:75] Recording config/version with fingerprint=3e6a03d1b71bbdc1fc679e98123b259b7623b9d532a3699499c6363123106d42 I0701 19:46:16.866093 1 recorder.go:75] Recording config/id with fingerprint=403a989d24a4c30ae891481a69408dd2aebc2ff01ba2b0b07cb4dfa12e4d1305 I0701 19:46:16.866099 1 gather.go:177] gatherer "clusterconfig" function "version" took 48.764825ms to process 2 records I0701 19:46:16.869521 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:46:16.888427 1 cluster_transfer.go:102] no available accepted cluster transfer I0701 19:46:16.888449 1 controllerstatus.go:80] name=clusterTransferController healthy=true reason=NoClusterTransfer message=no available cluster transfer I0701 19:46:16.897204 1 tasks_processing.go:74] worker 16 stopped. I0701 19:46:16.899409 1 recorder.go:75] Recording config/pod/openshift-console-operator/console-operator-575cd97545-hnzxj with fingerprint=1ab42eb24f9c6cf2c46a8067b7f7a7237ec2766243e46ebdd6d637d547e1dfd1 I0701 19:46:16.899616 1 recorder.go:75] Recording config/pod/openshift-multus/multus-8qn89 with fingerprint=7a5c0e349eef512ca4e2f3fdb73369df6e3b469e54732dff720331c61a0790e5 I0701 19:46:16.900096 1 recorder.go:75] Recording config/pod/openshift-ovn-kubernetes/ovnkube-node-ljlcq with fingerprint=2ca3d97a2d151f4d308091ff03cd7e86c69237fd635322867c46aa805628bf81 I0701 19:46:16.900219 1 recorder.go:75] Recording config/running_containers with fingerprint=035f69cae923ba28b31f09b2c6da39eeb7824c06bacc3863e4ac44904d0093d4 I0701 19:46:16.900234 1 gather.go:177] gatherer "clusterconfig" function "container_images" took 81.605767ms to process 4 records I0701 19:46:16.904443 1 tasks_processing.go:74] worker 20 stopped. I0701 19:46:16.904477 1 recorder.go:75] Recording aggregated/unused_machine_configs_count with fingerprint=4bfc9fa984e5dfcd45848faaf05269de7619bf42edf9f781751af5ee05c1a499 I0701 19:46:16.904493 1 gather.go:177] gatherer "clusterconfig" function "machine_configs" took 109.714553ms to process 1 records I0701 19:46:16.907410 1 base_controller.go:82] Caches are synced for LoggingSyncer I0701 19:46:16.907425 1 base_controller.go:119] Starting #1 worker of LoggingSyncer controller ... I0701 19:46:16.914503 1 tasks_processing.go:74] worker 8 stopped. I0701 19:46:16.914518 1 gather.go:177] gatherer "clusterconfig" function "machine_config_pools" took 102.36513ms to process 0 records I0701 19:46:16.918614 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0701 19:46:16.926628 1 gather_cluster_operators.go:184] Unable to get dnsrecords.ingress.operator.openshift.io resource due to: dnsrecords.ingress.operator.openshift.io "default" not found I0701 19:46:16.954463 1 tasks_processing.go:74] worker 10 stopped. E0701 19:46:16.954488 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret 'router-certs-default' in namespace 'openshift-ingress': secrets "router-certs-default" not found E0701 19:46:16.954497 1 gather.go:140] gatherer "clusterconfig" function "ingress_certificates" failed with the error: failed to get secret '2r9hocmc24gutgt57e2bmcpr09hu1mnp-primary-cert-bundle-secret' in namespace 'openshift-ingress-operator': secrets "2r9hocmc24gutgt57e2bmcpr09hu1mnp-primary-cert-bundle-secret" not found I0701 19:46:16.954582 1 recorder.go:75] Recording aggregated/ingress_controllers_certs with fingerprint=7e47a55485f4b6d45c391aec58add9df6fbed5b0cd4b3eb4fbbfaf31b5bd1d15 I0701 19:46:16.954600 1 gather.go:177] gatherer "clusterconfig" function "ingress_certificates" took 162.894227ms to process 1 records I0701 19:46:17.024168 1 sca.go:266] etc-pki-entitlement secret successfully created I0701 19:46:17.024198 1 controllerstatus.go:80] name=scaController healthy=true reason=Updated message=SCA certs successfully updated W0701 19:46:17.840190 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0701 19:46:18.841588 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. I0701 19:46:19.259371 1 tasks_processing.go:74] worker 7 stopped. I0701 19:46:19.259503 1 recorder.go:75] Recording config/clusteroperator/console with fingerprint=34fb32e09fc3cf7075eb1e5c0d40731fbf54c2be23c79886edc77f9df9279369 I0701 19:46:19.259564 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/console/cluster with fingerprint=be84fc75c87766f0f4d4e9fb1f9db618d008a21cc463e8239af81a1c296612d1 I0701 19:46:19.259592 1 recorder.go:75] Recording config/clusteroperator/csi-snapshot-controller with fingerprint=7f288ea96226c2945d3ac61b98407f7f1e54db271d28408cc21579c141f46c5e I0701 19:46:19.259604 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/csisnapshotcontroller/cluster with fingerprint=5adc514f4b63e2f1ecc68bf6f9c0af70c5eea04522a49524e102721b1c41f80e I0701 19:46:19.259635 1 recorder.go:75] Recording config/clusteroperator/dns with fingerprint=aac4d70211578e48784fda6e63528b9df5e9821e7dde81124d6e79b018f42dd5 I0701 19:46:19.259652 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/dns/default with fingerprint=9e7b4ce029030d3d8c3b49af92c556acdcc415000b40d3f969dbdc42c432b47f I0701 19:46:19.259697 1 recorder.go:75] Recording config/clusteroperator/image-registry with fingerprint=efa08fa13b216efd3689b7c88bcd8d5b79192b0e8c98a90f70c18d9679a8a134 I0701 19:46:19.259727 1 recorder.go:75] Recording config/clusteroperator/ingress with fingerprint=498050586dec33c729feb7ac64e580ebd9937c9a63f3cfc3ff1de9081a8bbfbe I0701 19:46:19.259750 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/ingresscontroller/openshift-ingress-operator/default with fingerprint=e72d82e502e67017aee8ab160ed433d4bdaf177447c08a33ad979df9d0fb6b58 I0701 19:46:19.259782 1 recorder.go:75] Recording config/clusteroperator/insights with fingerprint=7e09b7ebc7bcc4a21497fbb27e76c2fceea7bf94a6b4fbf08568e2379abcf382 I0701 19:46:19.260032 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/insightsoperator/cluster with fingerprint=de26adfc40dd6dbc63c297fbcfda796db0bd9fe105549539317808a54fcbe932 I0701 19:46:19.260052 1 recorder.go:75] Recording config/clusteroperator/kube-apiserver with fingerprint=ba6e4d32e33905c570e66d3848849657cbd0b01e86c375f225c6faf739db78b6 I0701 19:46:19.260058 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubeapiserver/cluster with fingerprint=51503bf0b784fcf65ea46bcaf1f72ac1a5c4d5dc211934f18f27871efed05762 I0701 19:46:19.260073 1 recorder.go:75] Recording config/clusteroperator/kube-controller-manager with fingerprint=c6e40124afe0a8279231d6588535c49342f11658df345fd12be6d93d85b78b78 I0701 19:46:19.260084 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubecontrollermanager/cluster with fingerprint=ce90c0d4f367d7da085074268031798382ae7c54fdcb0a21f15a4818fe308c11 I0701 19:46:19.260098 1 recorder.go:75] Recording config/clusteroperator/kube-scheduler with fingerprint=44595e2e2b62e18de59680dd108a1bf9a24a141f2801783873bd622d75c9fc47 I0701 19:46:19.260107 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubescheduler/cluster with fingerprint=f2940fb9fd20c19951dfc295eb363b7fba0c505f5ae61f01967a063099e6b60a I0701 19:46:19.260128 1 recorder.go:75] Recording config/clusteroperator/kube-storage-version-migrator with fingerprint=7af562b9aedcec2e7a8e99f0885cae9e773cbbc8f5020ac37dbd7d6414f72298 I0701 19:46:19.260138 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/kubestorageversionmigrator/cluster with fingerprint=9351181aa7e6ada41ef581ab31e13516c6b934cc95710154bafb2eb222cb58db I0701 19:46:19.260157 1 recorder.go:75] Recording config/clusteroperator/monitoring with fingerprint=559486aa5d64b8f7481a14969c2c5494a742e0f9b50dd2bd44e012206d005465 I0701 19:46:19.260257 1 recorder.go:75] Recording config/clusteroperator/network with fingerprint=21e169adf4003661b5d3dd9d4211342c5f3aef9d067137be046294dc97d9983d I0701 19:46:19.260268 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/ovn with fingerprint=626a89d20e0deaed5b6dfb533acfe65f4bb1618bd200a703b62e60c5d16d94ab I0701 19:46:19.260277 1 recorder.go:75] Recording config/clusteroperator/network.operator.openshift.io/operatorpki/openshift-ovn-kubernetes/signer with fingerprint=90410b16914712b85b3c4578716ad8c0ae072e688f4cd1e022bf76f20da3506d I0701 19:46:19.260332 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/network/cluster with fingerprint=a93d15eaecb455a0e40ecb2826eeecc1533899204ddd3c3921d15ab70af7ae75 I0701 19:46:19.260355 1 recorder.go:75] Recording config/clusteroperator/node-tuning with fingerprint=63423d4b45862fafe026ed715928393159f6ecaea3c2d8500ed33f9f9644ef31 I0701 19:46:19.260378 1 recorder.go:75] Recording config/clusteroperator/openshift-apiserver with fingerprint=a9c1bbb0fee6ca35a4727d43039cc7ba5b4782f8832c110d7df4c019648d031c I0701 19:46:19.260388 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftapiserver/cluster with fingerprint=e712e6cf27339b441e4ed1f4cde91dbde7e952698ba93407e4457db63a4a4c76 I0701 19:46:19.260405 1 recorder.go:75] Recording config/clusteroperator/openshift-controller-manager with fingerprint=aaf532a449d28d58d2a8488410efdb5b2ab571cf5b7467b095159c44ed283bc3 I0701 19:46:19.260415 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/openshiftcontrollermanager/cluster with fingerprint=d71a0f4672f9b45d9fc8293bf1687afc650fd28d32e2e30de27523fe7b4eadf7 I0701 19:46:19.260433 1 recorder.go:75] Recording config/clusteroperator/openshift-samples with fingerprint=ce36fb66388a0da1349f03b131b09446de2cdfe923b9c8eb243e5a1b86c805f1 I0701 19:46:19.260441 1 recorder.go:75] Recording config/clusteroperator/samples.operator.openshift.io/config/cluster with fingerprint=439a4284281b8dcef0621bb14ba23e2175a28a50613f86b33a171c49689474fd I0701 19:46:19.260453 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager with fingerprint=3e8f5e05b403446a8e4a4d36bd14714317e9aa81e132a4d5d789894291c1dba9 I0701 19:46:19.260467 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-catalog with fingerprint=a2e25d7c58fa7161b831c5ff5b031593334bd7689ca03d56e986f7d0b69dccc8 I0701 19:46:19.260483 1 recorder.go:75] Recording config/clusteroperator/operator-lifecycle-manager-packageserver with fingerprint=cbd990fca00907a68f354658a7a1f73029af45d78d65f94c47116c673143c6db I0701 19:46:19.260504 1 recorder.go:75] Recording config/clusteroperator/service-ca with fingerprint=a42b44f7e86d43641209d940241a279c0f2bcf0852f82aa3d848485e1643ab06 I0701 19:46:19.260512 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/serviceca/cluster with fingerprint=812f7edc2cdb30e61e7f2b29454357a40b1a507a4b0c2b7729193b67f0e3b4aa I0701 19:46:19.260539 1 recorder.go:75] Recording config/clusteroperator/storage with fingerprint=df2d50da1e1c20554b54b05880dfca482519ba459559bb970bfd61813aac32ad I0701 19:46:19.260555 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/clustercsidriver/ebs.csi.aws.com with fingerprint=510064d6f6bcced87ab5bd2ddaff3d0edd7f93f4a4f7af2641f29fc53ffab21e I0701 19:46:19.260564 1 recorder.go:75] Recording config/clusteroperator/operator.openshift.io/storage/cluster with fingerprint=8e480f8c1ce1b39baac42d8ec780c57c2592929ae0c801b61ffad49ba13f33ad I0701 19:46:19.260571 1 gather.go:177] gatherer "clusterconfig" function "operators" took 2.469348926s to process 39 records W0701 19:46:19.840375 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0701 19:46:20.840172 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0701 19:46:21.840313 1 gather_dvo_metrics.go:210] Failed to read the DVO metrics. Trying again. W0701 19:46:21.840341 1 gather_dvo_metrics.go:117] Unable to read metrics from endpoint "http://deployment-validation-operator-metrics.openshift-deployment-validation-operator.svc:8383": DVO metrics service was not available within the 5s timeout: context deadline exceeded I0701 19:46:21.840357 1 tasks_processing.go:74] worker 24 stopped. E0701 19:46:21.840367 1 gather.go:140] gatherer "clusterconfig" function "dvo_metrics" failed with the error: DVO metrics service was not available within the 5s timeout: context deadline exceeded I0701 19:46:21.840379 1 recorder.go:75] Recording config/dvo_metrics with fingerprint=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 I0701 19:46:21.840388 1 gather.go:177] gatherer "clusterconfig" function "dvo_metrics" took 5.035445058s to process 1 records I0701 19:46:29.821050 1 tasks_processing.go:74] worker 11 stopped. I0701 19:46:29.821093 1 recorder.go:75] Recording config/installplans with fingerprint=e3c18441424aaa81f54ea9723d194eacb5824eb47178047bd893de11825bc8a2 I0701 19:46:29.821105 1 gather.go:177] gatherer "clusterconfig" function "install_plans" took 13.029571212s to process 1 records I0701 19:46:30.629303 1 tasks_processing.go:74] worker 9 stopped. I0701 19:46:30.629583 1 recorder.go:75] Recording config/serviceaccounts with fingerprint=b5628417db3123c06333720ebefb8c69f4980e1afaf323f1ff9cb97dfe717013 I0701 19:46:30.629600 1 gather.go:177] gatherer "clusterconfig" function "service_accounts" took 13.805796955s to process 1 records E0701 19:46:30.629660 1 periodic.go:247] "Unhandled Error" err="clusterconfig failed after 13.84s with: function \"metrics\" failed with an error, function \"tsdb_status\" failed with an error, function \"machines\" failed with an error, function \"machine_healthchecks\" failed with an error, function \"pod_network_connectivity_checks\" failed with an error, function \"support_secret\" failed with an error, function \"config_maps\" failed with an error, function \"ingress_certificates\" failed with an error, function \"dvo_metrics\" failed with an error" I0701 19:46:30.630801 1 controllerstatus.go:89] name=periodic-clusterconfig healthy=false reason=PeriodicGatherFailed message=Source clusterconfig could not be retrieved: function "metrics" failed with an error, function "tsdb_status" failed with an error, function "machines" failed with an error, function "machine_healthchecks" failed with an error, function "pod_network_connectivity_checks" failed with an error, function "support_secret" failed with an error, function "config_maps" failed with an error, function "ingress_certificates" failed with an error, function "dvo_metrics" failed with an error I0701 19:46:30.630821 1 periodic.go:209] Running workloads gatherer I0701 19:46:30.630840 1 tasks_processing.go:45] number of workers: 2 I0701 19:46:30.630853 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:46:30.630859 1 tasks_processing.go:71] worker 1 working on workload_info task. I0701 19:46:30.630866 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:46:30.630880 1 tasks_processing.go:71] worker 0 working on helmchart_info task. I0701 19:46:30.637459 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-0-56.ec2.internal... I0701 19:46:30.637469 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-1-54.ec2.internal... I0701 19:46:30.637495 1 gather_workloads_runtime_infos.go:63] Gathering workload runtime info for node ip-10-0-2-238.ec2.internal... I0701 19:46:30.656280 1 tasks_processing.go:74] worker 0 stopped. I0701 19:46:30.656300 1 gather.go:177] gatherer "workloads" function "helmchart_info" took 25.384234ms to process 0 records I0701 19:46:35.580426 1 gather_workloads_runtime_infos.go:84] Gathered workload runtime infos in 5s I0701 19:46:35.607375 1 gather_workloads_info.go:278] Loaded pods in 0s, will wait 25s for image data I0701 19:46:35.635630 1 gather_workloads_info.go:387] No image sha256:04c87c054a3f366a7dfbe0a93ebb0c80a098ee16842c5794b67c1202eec61996 (30ms) I0701 19:46:35.643595 1 gather_workloads_info.go:387] No image sha256:765f0d23b637f685f98a31bd47c131b03cf72a40761a3f9a9d6320faa3c33733 (8ms) I0701 19:46:35.651464 1 gather_workloads_info.go:387] No image sha256:2904a78e2eb73fd6a9bb94c105c2a056831fb4113fbb7b0607c50adc9d879c9b (8ms) I0701 19:46:35.659089 1 gather_workloads_info.go:387] No image sha256:b99613494e9d6a4a8207323872d63cc02b830fc80a8bf972e62c64381aa4cd17 (8ms) I0701 19:46:35.666942 1 gather_workloads_info.go:387] No image sha256:a56211d075aa43cbb491f669a5b2e46ee023dc95b7d51dbac28f463948c5ad61 (8ms) I0701 19:46:35.674662 1 gather_workloads_info.go:387] No image sha256:5f0b67cfbbc381243fb91ccc17345b56d05f4d717c667e8c644e5bf05633ba71 (8ms) I0701 19:46:35.682647 1 gather_workloads_info.go:387] No image sha256:50197f22710766515f67944a779e00dd9ae3d17b18732d7324a970353b11f292 (8ms) I0701 19:46:35.691049 1 gather_workloads_info.go:387] No image sha256:101993f6e65d6fda8b90c583b718b720809182961f69bc777295f47e6df4d7ce (8ms) I0701 19:46:35.699140 1 gather_workloads_info.go:387] No image sha256:c940ea87e7d133d75ba0002ef00c0806825eed3db8094cdb260d1bac18127733 (8ms) I0701 19:46:35.707719 1 gather_workloads_info.go:387] No image sha256:7adc1eab05d6724c76ba751f6df816b08d6e70b78dee9eb94fa6fd9690542c98 (9ms) I0701 19:46:35.716019 1 gather_workloads_info.go:387] No image sha256:09e27b3a67ebeda04d3e597e0561bfb40550de78bdecb53f2f9ff3dfb1cfe7ea (8ms) I0701 19:46:35.815701 1 gather_workloads_info.go:387] No image sha256:2598489729a4b258e4ecda4a06f6875133f2a10ced5c5241f8a57a8a05418e36 (100ms) I0701 19:46:35.915774 1 gather_workloads_info.go:387] No image sha256:76cc47f3775560477ec548c7b6fddbd99112ad6604f790697eef546576051fb7 (100ms) I0701 19:46:36.016304 1 gather_workloads_info.go:387] No image sha256:56a85660a445eced5c79a595a0eccf590087c5672d50f49d4c25ad52f9a44f04 (101ms) I0701 19:46:36.115364 1 gather_workloads_info.go:387] No image sha256:6e5df2aab9ce759e8f957341a11c087c233307e3c8f8760d21270efc19ccd3ce (99ms) I0701 19:46:36.216198 1 gather_workloads_info.go:387] No image sha256:a498046d64605bcccee2440aa4f04a4602baaae263cf01d977ec5208e876b1fd (101ms) I0701 19:46:36.316272 1 gather_workloads_info.go:398] Found image sha256:32d4561d4bf1b3c4e3a7878f918f9c1422bab85bdf8711fbdf905d59692fee81 (100ms) I0701 19:46:36.415191 1 gather_workloads_info.go:387] No image sha256:1e65f9f5490fbc930d2dce96f03ddcb28990b839d7dc011ef5de5bec6c5095bf (99ms) I0701 19:46:36.517991 1 gather_workloads_info.go:387] No image sha256:653c666f842c13e0baae2e89a9b1efe0e2ef56f621ffb5b32005115d2a26ab8c (103ms) I0701 19:46:36.614423 1 gather_workloads_info.go:387] No image sha256:0a99240166165eb5718e7516a43282fe32df9c7c5e809b31b58abe44e42ff94d (96ms) I0701 19:46:36.715131 1 gather_workloads_info.go:387] No image sha256:91828234f107c068c8a4966d08370ae7b73e637651dbc6d92c18c4553402c22c (101ms) I0701 19:46:36.815625 1 gather_workloads_info.go:387] No image sha256:74713675d580a47a56b628469c4c05c41e114e0b935960d718ff0cf7e72f613d (100ms) I0701 19:46:36.916084 1 gather_workloads_info.go:387] No image sha256:943018739e3db1763c3184b460dbc409e058abbac76d57b9927faad317be85e4 (100ms) I0701 19:46:37.018410 1 gather_workloads_info.go:387] No image sha256:1477f3287cc75352305274f248f781c1b7c2a8696f4f7bc450c45a4a5f5b10ea (102ms) I0701 19:46:37.118632 1 gather_workloads_info.go:387] No image sha256:a0105d1eb62cf6ac9e5e2ef28d3e89bf6dc514bc594fc7090fe5a5ee18a09c87 (100ms) I0701 19:46:37.221437 1 gather_workloads_info.go:387] No image sha256:2e57e192c3c1240fd935dcd55c8fde5e70e78bf81d6176c96edf21fafe59f8ba (103ms) I0701 19:46:37.315392 1 gather_workloads_info.go:387] No image sha256:95418d0ba6fc0aa96647d97cf6530967f7d7ae97a82ce7f7e41c8e44310b0b72 (94ms) I0701 19:46:37.414815 1 gather_workloads_info.go:387] No image sha256:0af73ad7a47c9ff53269ae2bc5163bf9e389085220c34e4bd42c92c4e7a85f6d (99ms) I0701 19:46:37.516153 1 gather_workloads_info.go:387] No image sha256:2e564f336c77116053f34d4201d364d8da04e789cfffa0ea422574c95f2d6404 (101ms) I0701 19:46:37.614949 1 gather_workloads_info.go:387] No image sha256:25bda9d34e23ba6bc9df5cf7104bdb237269740a3a76ee025f88b34e68b7e2b5 (99ms) I0701 19:46:37.714086 1 gather_workloads_info.go:387] No image sha256:7bd3e88d6d3f878a89203df590b684c5a77bf08df4717451d3197ab74683a80e (99ms) I0701 19:46:37.815634 1 gather_workloads_info.go:387] No image sha256:5a95c19d82767e0235b4edb4a0536482c816904897aae1dc3eb255cb52b87a9f (102ms) I0701 19:46:37.915404 1 gather_workloads_info.go:387] No image sha256:36b9e89c3cfcf1ab9ae500486e38afb6862cba48cb0b4d84a09508ab8f3d299f (100ms) I0701 19:46:38.015034 1 gather_workloads_info.go:387] No image sha256:88437b35173fe2e8a4d9f148ec17c754bee4df3f723b3a5676955eef4f363d31 (100ms) I0701 19:46:38.114519 1 gather_workloads_info.go:387] No image sha256:a258c226562adb14e3a163a1940938526ee6a0928982a7667d85d9a7334ce639 (99ms) I0701 19:46:38.215701 1 gather_workloads_info.go:387] No image sha256:513b1166e376feca472fdd5174f6abf0abf02925d6a21b7b4d89a2ff2fc67bed (101ms) I0701 19:46:38.315238 1 gather_workloads_info.go:387] No image sha256:289816958633a763a72dbc44e1dad40466223164e7e253039514f0d974ea5d21 (100ms) I0701 19:46:38.415347 1 gather_workloads_info.go:387] No image sha256:0092c6ff139f8698c91349d507eb3182bc5d6c4ae040f672e347ff504accdea2 (100ms) I0701 19:46:38.514989 1 gather_workloads_info.go:387] No image sha256:3d6ea9e1633f38070718a1357fcb80e6cb3932aafcc4d65e659795b186c5da15 (100ms) I0701 19:46:38.614069 1 gather_workloads_info.go:387] No image sha256:4556896f77307821531ef91b7b7faccb82b824ea695693b2989f597f0deca038 (99ms) I0701 19:46:38.715225 1 gather_workloads_info.go:398] Found image sha256:b3909bf664c77097f75b3768830863d642eed3815dab2bfb4415c771ca2d5007 (101ms) I0701 19:46:38.815543 1 gather_workloads_info.go:387] No image sha256:03cf4cd7ef1518610c6c7b3ad27d1622d82e98e3dc6e3f8e5d0fceb5c8d3786e (100ms) I0701 19:46:38.916031 1 gather_workloads_info.go:387] No image sha256:90bdf9e678fe18e35401e34e1cb0912d822e9cb32b9e41cc545870540e9a87f2 (100ms) I0701 19:46:39.014388 1 gather_workloads_info.go:387] No image sha256:47154813651033d59751fb655a384dbffb64dd26f10bd7f3be0c3128d0486356 (98ms) I0701 19:46:39.115258 1 gather_workloads_info.go:387] No image sha256:521712486e2c6e3c020dad6a1cb340db8e55665b69f7c208fab9cd9e965fd588 (101ms) I0701 19:46:39.214940 1 gather_workloads_info.go:387] No image sha256:3612c5a433effd17b7aff7ff6edf4fb27b61153908e3a0294158e4886a1d600d (100ms) I0701 19:46:39.316103 1 gather_workloads_info.go:387] No image sha256:7b31223098f08328f5ddea8e5b871dbbd5f5a61ec550e8956f66793c0c6031a9 (101ms) I0701 19:46:39.414214 1 gather_workloads_info.go:387] No image sha256:695cf2f0cc07683c2a3ce1eaf3e56fe18abc6e2bac716f7d9843f5d173b9df52 (98ms) I0701 19:46:39.515111 1 gather_workloads_info.go:387] No image sha256:1a2532940843248c57d52141185dd71fbc393ab28b65d48f682038632c1dbbad (101ms) I0701 19:46:39.515144 1 tasks_processing.go:74] worker 1 stopped. I0701 19:46:39.515437 1 recorder.go:75] Recording config/workload_info with fingerprint=9b645c2a4542e4d62250bb4578e37557dc8f69c586b7aca45c7a5c2070091ea6 I0701 19:46:39.515454 1 gather.go:177] gatherer "workloads" function "workload_info" took 8.88427775s to process 1 records I0701 19:46:39.515469 1 periodic.go:256] Periodic gather workloads completed in 8.884s I0701 19:46:39.515478 1 controllerstatus.go:80] name=periodic-workloads healthy=true reason= message= I0701 19:46:39.515482 1 periodic.go:209] Running conditional gatherer I0701 19:46:39.521548 1 requests.go:294] Making HTTP GET request at: https://console.redhat.com/api/gathering/v2/4.20.8/gathering_rules I0701 19:46:39.708294 1 conditional_gatherer.go:338] updating alerts cache for conditional gatherer I0701 19:46:40.778912 1 conditional_gatherer.go:384] updating version cache for conditional gatherer I0701 19:46:40.786085 1 conditional_gatherer.go:392] cluster version is '4.20.8' I0701 19:46:40.786112 1 tasks_processing.go:45] number of workers: 3 I0701 19:46:40.786120 1 tasks_processing.go:69] worker 2 listening for tasks. I0701 19:46:40.786125 1 tasks_processing.go:71] worker 2 working on conditional_gatherer_rules task. I0701 19:46:40.786135 1 tasks_processing.go:69] worker 0 listening for tasks. I0701 19:46:40.786144 1 tasks_processing.go:69] worker 1 listening for tasks. I0701 19:46:40.786158 1 tasks_processing.go:71] worker 0 working on remote_configuration task. I0701 19:46:40.786159 1 tasks_processing.go:74] worker 1 stopped. I0701 19:46:40.786174 1 tasks_processing.go:71] worker 2 working on rapid_container_logs task. I0701 19:46:40.786215 1 recorder.go:75] Recording insights-operator/conditional-gatherer-rules with fingerprint=c8d4892cd4f2eea27d57353c6055eeb530bc2d811bfe041f6984d90395255a8c I0701 19:46:40.786229 1 gather.go:177] gatherer "conditional" function "conditional_gatherer_rules" took 1.065µs to process 1 records I0701 19:46:40.786261 1 recorder.go:75] Recording insights-operator/remote-configuration with fingerprint=5f681cc1442447fd2c1534f876836dff54caf3c545b582a5a8aae894f9203c5d I0701 19:46:40.786270 1 gather.go:177] gatherer "conditional" function "remote_configuration" took 1.502µs to process 1 records I0701 19:46:40.786275 1 tasks_processing.go:74] worker 0 stopped. I0701 19:46:40.786489 1 tasks_processing.go:74] worker 2 stopped. I0701 19:46:40.786505 1 gather.go:177] gatherer "conditional" function "rapid_container_logs" took 302.578µs to process 0 records I0701 19:46:40.786534 1 controllerstatus.go:80] name=periodic-conditional healthy=true reason=Succeeded message= I0701 19:46:40.786560 1 recorder.go:75] Recording insights-operator/remote-configuration.json with fingerprint=363fa5c8fa2573b18bc685faeca9ca9f021a8f80760895b7701f54e3dcd5c85f I0701 19:46:40.817164 1 recorder.go:75] Recording insights-operator/gathers with fingerprint=69aa86d559fd670d6a74d24f49496b0abf8b75ebd2e155bad07bc7b988ecda68 I0701 19:46:40.817330 1 diskrecorder.go:70] Writing 112 records to /var/lib/insights-operator/insights-2026-07-01-194640.tar.gz I0701 19:46:40.824168 1 diskrecorder.go:51] Wrote 112 records to disk in 6ms I0701 19:46:40.824199 1 periodic.go:278] Gathering cluster info every 2h0m0s I0701 19:46:40.824222 1 periodic.go:279] Configuration is dataReporting: interval: 2h0m0s, uploadEndpoint: https://console.redhat.com/api/ingress/v1/upload, storagePath: /var/lib/insights-operator, downloadEndpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/%s/reports, conditionalGathererEndpoint: https://console.redhat.com/api/gathering/v2/%s/gathering_rules, obfuscation: [] sca: disabled: false, endpoint: https://api.openshift.com/api/accounts_mgmt/v1/entitlement_certificates, interval: 8h0m0s alerting: disabled: false clusterTransfer: endpoint: https://api.openshift.com/api/accounts_mgmt/v1/cluster_transfers/, interval: 12h0m0s proxy: httpProxy: , httpsProxy: , noProxy: I0701 19:47:46.789731 1 diskrecorder.go:223] Found files to send: insights-2026-07-01-194640.tar.gz I0701 19:47:46.789761 1 insightsuploader.go:150] Checking archives to upload periodically every 15m8.801484465s I0701 19:47:46.789769 1 insightsuploader.go:165] Uploading latest report since 0001-01-01T00:00:00Z I0701 19:47:46.801376 1 requests.go:46] Uploading application/vnd.redhat.openshift.periodic to https://console.redhat.com/api/ingress/v1/upload I0701 19:47:47.177188 1 requests.go:87] Successfully reported id=2026-07-01T19:47:46Z x-rh-insights-request-id=a68bcdd0845e4fe09593f2fb8ac3bae8, wrote=54118 I0701 19:47:47.177311 1 insightsuploader.go:187] Uploaded report successfully in 387.534193ms I0701 19:47:47.177335 1 controller.go:128] Initializing last reported time to 2026-07-01T19:47:46Z I0701 19:47:47.177405 1 insightsreport.go:304] Archive uploaded, starting pulling report... I0701 19:47:47.177418 1 insightsreport.go:215] Starting retrieving report from Smart Proxy I0701 19:47:47.177426 1 insightsreport.go:221] Initial delay for pulling: 1m0s I0701 19:47:47.182756 1 controller.go:512] The operator is healthy I0701 19:48:16.810956 1 controller.go:512] The operator is healthy I0701 19:48:53.125706 1 insightsreport.go:137] Pulling report from smart-proxy I0701 19:48:53.125736 1 insightsreport.go:149] Retrieving report I0701 19:48:53.132933 1 requests.go:110] Retrieving report for cluster: b03129ce-c613-4c8d-aff3-0e949bf6aea2 I0701 19:48:53.132948 1 requests.go:111] Endpoint: https://console.redhat.com/api/insights-results-aggregator/v2/cluster/b03129ce-c613-4c8d-aff3-0e949bf6aea2/reports I0701 19:48:53.135881 1 requests.go:121] Retrieving report from https://console.redhat.com/api/insights-results-aggregator/v2/cluster/b03129ce-c613-4c8d-aff3-0e949bf6aea2/reports I0701 19:48:53.303996 1 insightsreport.go:184] Report retrieved I0701 19:48:53.341134 1 insightsreport.go:239] Report retrieved correctly I0701 19:50:16.810488 1 controller.go:512] The operator is healthy I0701 19:51:16.782184 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:51:16.792850 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:51:16.792869 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:51:16.796829 1 secretconfigobserver.go:119] support secret does not exist I0701 19:52:16.817782 1 controller.go:512] The operator is healthy I0701 19:54:16.811331 1 controller.go:512] The operator is healthy I0701 19:56:16.781796 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:56:16.785597 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 19:56:16.797709 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 19:56:16.807136 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 19:56:16.807156 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 19:56:16.813179 1 controller.go:512] The operator is healthy I0701 19:56:16.827297 1 secretconfigobserver.go:119] support secret does not exist I0701 19:58:16.811499 1 controller.go:512] The operator is healthy I0701 20:00:16.811116 1 controller.go:512] The operator is healthy I0701 20:01:16.827892 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:01:16.838650 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:01:16.838696 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:01:16.842547 1 secretconfigobserver.go:119] support secret does not exist I0701 20:02:16.810854 1 controller.go:512] The operator is healthy I0701 20:03:00.979737 1 insightsuploader.go:146] Nothing to report since 2026-07-01T19:47:46Z I0701 20:04:16.812335 1 controller.go:512] The operator is healthy I0701 20:06:16.782182 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 20:06:16.786209 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 20:06:16.810280 1 controller.go:512] The operator is healthy I0701 20:06:16.843275 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:06:16.847638 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:06:16.847656 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:06:16.851025 1 secretconfigobserver.go:119] support secret does not exist I0701 20:08:16.814350 1 controller.go:512] The operator is healthy I0701 20:10:16.813724 1 controller.go:512] The operator is healthy I0701 20:11:16.851881 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:11:16.856642 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:11:16.856662 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:11:16.860016 1 secretconfigobserver.go:119] support secret does not exist I0701 20:12:16.811068 1 controller.go:512] The operator is healthy I0701 20:14:16.812692 1 controller.go:512] The operator is healthy I0701 20:16:16.785639 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 20:16:16.789170 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 20:16:16.810233 1 controller.go:512] The operator is healthy I0701 20:16:16.860233 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:16:16.864534 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:16:16.864556 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:16:16.868298 1 secretconfigobserver.go:119] support secret does not exist I0701 20:18:16.810609 1 controller.go:512] The operator is healthy I0701 20:18:26.074396 1 insightsuploader.go:146] Nothing to report since 2026-07-01T19:47:46Z I0701 20:20:16.814558 1 controller.go:512] The operator is healthy I0701 20:21:16.869120 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:21:16.877037 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:21:16.877060 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:21:16.880967 1 secretconfigobserver.go:119] support secret does not exist I0701 20:22:16.814072 1 controller.go:512] The operator is healthy I0701 20:24:16.810904 1 controller.go:512] The operator is healthy I0701 20:26:16.782719 1 configmapobserver.go:84] configmaps "insights-config" not found I0701 20:26:16.811062 1 controller.go:512] The operator is healthy I0701 20:26:16.881163 1 secretconfigobserver.go:136] Refreshing configuration from cluster pull secret I0701 20:26:16.885430 1 secretconfigobserver.go:249] Found cloud.openshift.com token I0701 20:26:16.885447 1 secretconfigobserver.go:162] Refreshing configuration from cluster support secret I0701 20:26:16.889430 1 secretconfigobserver.go:119] support secret does not exist