Apr 19 15:22:01.057033 ip-10-0-130-182 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Apr 19 15:22:01.057042 ip-10-0-130-182 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Apr 19 15:22:01.057050 ip-10-0-130-182 systemd[1]: kubelet.service: Failed with result 'resources'. Apr 19 15:22:01.057262 ip-10-0-130-182 systemd[1]: Failed to start Kubernetes Kubelet. Apr 19 15:22:11.165237 ip-10-0-130-182 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Apr 19 15:22:11.165253 ip-10-0-130-182 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 714a2495e63e454f982d1b135647321d -- Apr 19 15:24:25.926040 ip-10-0-130-182 systemd[1]: Starting Kubernetes Kubelet... Apr 19 15:24:26.443573 ip-10-0-130-182 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 19 15:24:26.443573 ip-10-0-130-182 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 19 15:24:26.443573 ip-10-0-130-182 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 19 15:24:26.443573 ip-10-0-130-182 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 19 15:24:26.443573 ip-10-0-130-182 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 19 15:24:26.445572 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.445469 2576 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 19 15:24:26.450061 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450038 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 19 15:24:26.450061 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450060 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450065 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450068 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450072 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450075 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450077 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450080 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450092 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450095 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450100 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450105 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450108 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450111 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450114 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450116 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450119 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450122 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450124 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450127 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450130 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 19 15:24:26.450138 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450132 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450136 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450138 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450142 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450145 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450149 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450153 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450156 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450160 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450163 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450165 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450168 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450171 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450173 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450176 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450178 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450181 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450184 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450186 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 19 15:24:26.450609 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450190 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450192 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450196 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450198 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450201 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450204 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450207 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450209 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450212 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450214 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450217 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450220 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450222 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450225 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450227 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450230 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450233 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450236 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450239 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450241 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 19 15:24:26.451108 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450244 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450246 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450249 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450251 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450254 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450257 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450259 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450262 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450265 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450267 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450270 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450272 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450275 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450281 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450284 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450288 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450290 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450293 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450295 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450298 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 19 15:24:26.451637 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450300 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450303 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450305 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450308 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450310 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450313 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450767 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450774 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450777 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450780 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450783 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450786 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450789 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450792 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450794 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450797 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450800 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450802 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450805 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450808 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 19 15:24:26.452126 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450810 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450813 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450815 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450818 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450821 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450824 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450827 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450830 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450833 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450837 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450840 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450842 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450845 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450847 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450850 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450853 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450856 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450859 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450861 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 19 15:24:26.452635 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450864 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450867 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450870 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450872 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450875 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450877 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450880 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450882 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450885 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450887 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450890 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450892 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450895 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450897 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450900 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450902 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450905 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450907 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450911 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450914 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 19 15:24:26.453114 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450917 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450920 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450922 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450925 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450927 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450930 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450932 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450935 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450937 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450940 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450943 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450945 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450948 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450950 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450953 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450955 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450958 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450960 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450963 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450965 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 19 15:24:26.453607 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450968 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450974 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450977 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450981 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450983 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450986 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450989 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450992 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450995 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.450998 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451000 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451003 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451006 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451099 2576 flags.go:64] FLAG: --address="0.0.0.0" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451107 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451115 2576 flags.go:64] FLAG: --anonymous-auth="true" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451120 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451125 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451129 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451133 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451138 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 19 15:24:26.454115 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451142 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451145 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451149 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451152 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451156 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451159 2576 flags.go:64] FLAG: --cgroup-root="" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451162 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451165 2576 flags.go:64] FLAG: --client-ca-file="" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451168 2576 flags.go:64] FLAG: --cloud-config="" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451171 2576 flags.go:64] FLAG: --cloud-provider="external" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451174 2576 flags.go:64] FLAG: --cluster-dns="[]" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451178 2576 flags.go:64] FLAG: --cluster-domain="" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451182 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451185 2576 flags.go:64] FLAG: --config-dir="" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451188 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451191 2576 flags.go:64] FLAG: --container-log-max-files="5" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451196 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451199 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451202 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451206 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451209 2576 flags.go:64] FLAG: --contention-profiling="false" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451212 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451215 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451218 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451222 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 19 15:24:26.454655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451227 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451230 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451233 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451236 2576 flags.go:64] FLAG: --enable-load-reader="false" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451239 2576 flags.go:64] FLAG: --enable-server="true" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451242 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451247 2576 flags.go:64] FLAG: --event-burst="100" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451251 2576 flags.go:64] FLAG: --event-qps="50" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451254 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451257 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451260 2576 flags.go:64] FLAG: --eviction-hard="" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451264 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451267 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451270 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451274 2576 flags.go:64] FLAG: --eviction-soft="" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451276 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451279 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451282 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451285 2576 flags.go:64] FLAG: --experimental-mounter-path="" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451288 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451291 2576 flags.go:64] FLAG: --fail-swap-on="true" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451294 2576 flags.go:64] FLAG: --feature-gates="" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451298 2576 flags.go:64] FLAG: --file-check-frequency="20s" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451302 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451305 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 19 15:24:26.455275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451308 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451311 2576 flags.go:64] FLAG: --healthz-port="10248" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451315 2576 flags.go:64] FLAG: --help="false" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451318 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451321 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451327 2576 flags.go:64] FLAG: --http-check-frequency="20s" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451329 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451334 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451337 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451341 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451343 2576 flags.go:64] FLAG: --image-service-endpoint="" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451346 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451349 2576 flags.go:64] FLAG: --kube-api-burst="100" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451352 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451356 2576 flags.go:64] FLAG: --kube-api-qps="50" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451359 2576 flags.go:64] FLAG: --kube-reserved="" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451362 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451364 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451368 2576 flags.go:64] FLAG: --kubelet-cgroups="" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451370 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451373 2576 flags.go:64] FLAG: --lock-file="" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451376 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451379 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451382 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 19 15:24:26.455935 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451387 2576 flags.go:64] FLAG: --log-json-split-stream="false" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451390 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451393 2576 flags.go:64] FLAG: --log-text-split-stream="false" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451396 2576 flags.go:64] FLAG: --logging-format="text" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451398 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451402 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451405 2576 flags.go:64] FLAG: --manifest-url="" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451407 2576 flags.go:64] FLAG: --manifest-url-header="" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451412 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451415 2576 flags.go:64] FLAG: --max-open-files="1000000" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451424 2576 flags.go:64] FLAG: --max-pods="110" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451427 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451430 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451435 2576 flags.go:64] FLAG: --memory-manager-policy="None" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451437 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451441 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451444 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451447 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451456 2576 flags.go:64] FLAG: --node-status-max-images="50" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451459 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451462 2576 flags.go:64] FLAG: --oom-score-adj="-999" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451465 2576 flags.go:64] FLAG: --pod-cidr="" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451468 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c8cfe89231412ff3ee8cb6207fa0be33cad0f08e88c9c0f1e9f7e8c6f14d6715" Apr 19 15:24:26.456542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451474 2576 flags.go:64] FLAG: --pod-manifest-path="" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451477 2576 flags.go:64] FLAG: --pod-max-pids="-1" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451480 2576 flags.go:64] FLAG: --pods-per-core="0" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451483 2576 flags.go:64] FLAG: --port="10250" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451486 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451490 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-016f0f1d964b356fd" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451493 2576 flags.go:64] FLAG: --qos-reserved="" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451496 2576 flags.go:64] FLAG: --read-only-port="10255" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451499 2576 flags.go:64] FLAG: --register-node="true" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451502 2576 flags.go:64] FLAG: --register-schedulable="true" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451505 2576 flags.go:64] FLAG: --register-with-taints="" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451509 2576 flags.go:64] FLAG: --registry-burst="10" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451512 2576 flags.go:64] FLAG: --registry-qps="5" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451514 2576 flags.go:64] FLAG: --reserved-cpus="" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451517 2576 flags.go:64] FLAG: --reserved-memory="" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451521 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451524 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451527 2576 flags.go:64] FLAG: --rotate-certificates="false" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451530 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451533 2576 flags.go:64] FLAG: --runonce="false" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451536 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451539 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451543 2576 flags.go:64] FLAG: --seccomp-default="false" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451546 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451549 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451553 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 19 15:24:26.457122 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451556 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451560 2576 flags.go:64] FLAG: --storage-driver-password="root" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451563 2576 flags.go:64] FLAG: --storage-driver-secure="false" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451565 2576 flags.go:64] FLAG: --storage-driver-table="stats" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451569 2576 flags.go:64] FLAG: --storage-driver-user="root" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451571 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451575 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451578 2576 flags.go:64] FLAG: --system-cgroups="" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451581 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451587 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451589 2576 flags.go:64] FLAG: --tls-cert-file="" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451592 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451596 2576 flags.go:64] FLAG: --tls-min-version="" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451599 2576 flags.go:64] FLAG: --tls-private-key-file="" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451602 2576 flags.go:64] FLAG: --topology-manager-policy="none" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451605 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451608 2576 flags.go:64] FLAG: --topology-manager-scope="container" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451612 2576 flags.go:64] FLAG: --v="2" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451616 2576 flags.go:64] FLAG: --version="false" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451621 2576 flags.go:64] FLAG: --vmodule="" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451641 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.451645 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451740 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451744 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 19 15:24:26.457812 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451747 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451750 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451753 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451756 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451761 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451763 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451766 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451769 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451772 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451781 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451785 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451788 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451791 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451794 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451797 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451800 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451803 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451806 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451809 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451812 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 19 15:24:26.458740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451815 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451817 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451820 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451822 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451825 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451828 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451830 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451833 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451836 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451838 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451841 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451844 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451846 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451849 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451851 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451854 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451858 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451861 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451864 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 19 15:24:26.459311 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451868 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451871 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451874 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451877 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451880 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451883 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451885 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451892 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451894 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451897 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451899 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451902 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451905 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451907 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451910 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451912 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451915 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451917 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451920 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451923 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 19 15:24:26.459951 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451925 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451928 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451931 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451933 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451936 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451938 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451941 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451943 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451946 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451949 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451952 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451955 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451957 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451960 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451962 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451965 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451968 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451971 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451974 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451978 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 19 15:24:26.460712 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451980 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 19 15:24:26.461241 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451983 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 19 15:24:26.461241 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451985 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 19 15:24:26.461241 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451988 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 19 15:24:26.461241 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.451991 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 19 15:24:26.461241 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.452911 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 19 15:24:26.461665 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.461643 2576 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 19 15:24:26.461709 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.461666 2576 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 19 15:24:26.461740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461722 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 19 15:24:26.461740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461727 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 19 15:24:26.461740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461731 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 19 15:24:26.461740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461734 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 19 15:24:26.461740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461737 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 19 15:24:26.461740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461740 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 19 15:24:26.461740 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461743 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461747 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461749 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461753 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461755 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461758 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461761 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461764 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461766 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461770 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461772 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461775 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461778 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461781 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461783 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461786 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461788 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461791 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461794 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 19 15:24:26.461919 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461797 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461799 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461802 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461805 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461807 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461811 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461816 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461820 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461822 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461825 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461829 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461831 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461834 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461837 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461839 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461844 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461846 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461849 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461852 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 19 15:24:26.462397 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461855 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461858 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461861 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461863 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461866 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461869 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461872 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461874 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461877 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461879 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461882 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461884 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461887 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461890 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461892 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461895 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461899 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461903 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461905 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461908 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 19 15:24:26.462888 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461911 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461913 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461916 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461918 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461921 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461924 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461927 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461930 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461933 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461936 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461939 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461942 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461945 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461948 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461951 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461953 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461956 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461958 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461961 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461964 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461966 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 19 15:24:26.463407 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.461969 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.461974 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462082 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462087 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462090 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462093 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462097 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462101 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462103 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462106 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462108 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462111 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462114 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462116 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462119 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 19 15:24:26.463938 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462122 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462124 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462127 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462129 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462132 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462135 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462138 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462140 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462143 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462147 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462150 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462152 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462155 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462157 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462160 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462163 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462166 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462168 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462171 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 19 15:24:26.464316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462173 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462177 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462180 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462183 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462186 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462189 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462191 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462194 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462197 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462199 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462202 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462204 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462207 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462210 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462212 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462215 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462218 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462221 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462224 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 19 15:24:26.464893 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462227 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462230 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462233 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462235 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462238 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462241 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462243 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462246 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462248 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462251 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462253 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462256 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462258 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462261 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462263 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462266 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462269 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462272 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462274 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462277 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 19 15:24:26.465360 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462279 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462282 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462284 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462287 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462289 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462292 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462294 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462297 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462300 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462302 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462305 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462308 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462311 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462313 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:26.462316 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.462320 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 19 15:24:26.465956 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.463124 2576 server.go:962] "Client rotation is on, will bootstrap in background" Apr 19 15:24:26.466940 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.466925 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 19 15:24:26.468231 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.468219 2576 server.go:1019] "Starting client certificate rotation" Apr 19 15:24:26.468337 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.468318 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 19 15:24:26.468372 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.468367 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 19 15:24:26.496916 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.496889 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 19 15:24:26.499775 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.499752 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 19 15:24:26.515608 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.515585 2576 log.go:25] "Validated CRI v1 runtime API" Apr 19 15:24:26.523264 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.523246 2576 log.go:25] "Validated CRI v1 image API" Apr 19 15:24:26.524728 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.524713 2576 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 19 15:24:26.527147 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.527130 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 19 15:24:26.532656 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.532622 2576 fs.go:135] Filesystem UUIDs: map[1b5af02c-3ea6-41f0-a1e4-2b16871a16e0:/dev/nvme0n1p4 47d361c5-e097-48ce-9ace-491197e3f0e0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Apr 19 15:24:26.532731 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.532655 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 19 15:24:26.538732 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.538580 2576 manager.go:217] Machine: {Timestamp:2026-04-19 15:24:26.536414881 +0000 UTC m=+0.465875033 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099807 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27d1265237bdc1ba24fab37667f779 SystemUUID:ec27d126-5237-bdc1-ba24-fab37667f779 BootID:714a2495-e63e-454f-982d-1b135647321d Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:13:1e:9e:3d:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:13:1e:9e:3d:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:bc:09:6e:dc:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 19 15:24:26.538732 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.538723 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 19 15:24:26.538845 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.538828 2576 manager.go:233] Version: {KernelVersion:5.14.0-570.107.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260414-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 19 15:24:26.540151 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.540115 2576 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 19 15:24:26.540349 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.540154 2576 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-182.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 19 15:24:26.540393 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.540361 2576 topology_manager.go:138] "Creating topology manager with none policy" Apr 19 15:24:26.540393 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.540371 2576 container_manager_linux.go:306] "Creating device plugin manager" Apr 19 15:24:26.540393 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.540386 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 19 15:24:26.541233 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.541220 2576 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 19 15:24:26.543008 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.542994 2576 state_mem.go:36] "Initialized new in-memory state store" Apr 19 15:24:26.543130 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.543120 2576 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 19 15:24:26.545739 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.545725 2576 kubelet.go:491] "Attempting to sync node with API server" Apr 19 15:24:26.545780 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.545744 2576 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 19 15:24:26.545780 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.545758 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 19 15:24:26.545780 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.545769 2576 kubelet.go:397] "Adding apiserver pod source" Apr 19 15:24:26.545780 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.545779 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 19 15:24:26.546260 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.546241 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rg5fg" Apr 19 15:24:26.547042 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.547026 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 19 15:24:26.547127 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.547046 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 19 15:24:26.550534 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.550510 2576 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 19 15:24:26.551908 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.551892 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rg5fg" Apr 19 15:24:26.552537 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.552523 2576 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 19 15:24:26.555481 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555464 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 19 15:24:26.555481 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555484 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555491 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555497 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555502 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555508 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555514 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555519 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555529 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555535 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555543 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 19 15:24:26.555583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.555552 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 19 15:24:26.556850 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.556839 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 19 15:24:26.556904 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.556853 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 19 15:24:26.560024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.560006 2576 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 19 15:24:26.560917 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.560901 2576 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 19 15:24:26.560975 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.560945 2576 server.go:1295] "Started kubelet" Apr 19 15:24:26.561113 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.561054 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 19 15:24:26.561581 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.561527 2576 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 19 15:24:26.561666 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.561606 2576 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 19 15:24:26.561954 ip-10-0-130-182 systemd[1]: Started Kubernetes Kubelet. Apr 19 15:24:26.566663 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.566643 2576 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 19 15:24:26.566813 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.566802 2576 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 19 15:24:26.568452 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.568430 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-130-182.ec2.internal" not found Apr 19 15:24:26.569015 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.568995 2576 server.go:317] "Adding debug handlers to kubelet server" Apr 19 15:24:26.577213 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.577189 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 19 15:24:26.577831 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.577816 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 19 15:24:26.578719 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.578699 2576 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 19 15:24:26.578795 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.578740 2576 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 19 15:24:26.578795 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.578759 2576 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 19 15:24:26.578903 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.578869 2576 reconstruct.go:97] "Volume reconstruction finished" Apr 19 15:24:26.578903 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.578880 2576 reconciler.go:26] "Reconciler: start to sync state" Apr 19 15:24:26.578973 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.578933 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-182.ec2.internal\" not found" Apr 19 15:24:26.579233 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.579215 2576 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 19 15:24:26.579356 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.579341 2576 factory.go:55] Registering systemd factory Apr 19 15:24:26.579417 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.579365 2576 factory.go:223] Registration of the systemd container factory successfully Apr 19 15:24:26.579590 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.579573 2576 factory.go:153] Registering CRI-O factory Apr 19 15:24:26.579590 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.579591 2576 factory.go:223] Registration of the crio container factory successfully Apr 19 15:24:26.579770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.579661 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 19 15:24:26.579770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.579689 2576 factory.go:103] Registering Raw factory Apr 19 15:24:26.579770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.579706 2576 manager.go:1196] Started watching for new ooms in manager Apr 19 15:24:26.580308 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.580281 2576 manager.go:319] Starting recovery of all containers Apr 19 15:24:26.580598 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.580578 2576 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 19 15:24:26.582397 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.582369 2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-182.ec2.internal\" not found" node="ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.582796 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.582780 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-130-182.ec2.internal" not found Apr 19 15:24:26.590579 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.590563 2576 manager.go:324] Recovery completed Apr 19 15:24:26.595223 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.595205 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 19 15:24:26.597283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.597268 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-182.ec2.internal" event="NodeHasSufficientMemory" Apr 19 15:24:26.597356 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.597293 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-182.ec2.internal" event="NodeHasNoDiskPressure" Apr 19 15:24:26.597356 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.597305 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-182.ec2.internal" event="NodeHasSufficientPID" Apr 19 15:24:26.597882 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.597870 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 19 15:24:26.597882 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.597881 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 19 15:24:26.597970 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.597897 2576 state_mem.go:36] "Initialized new in-memory state store" Apr 19 15:24:26.601996 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.601983 2576 policy_none.go:49] "None policy: Start" Apr 19 15:24:26.602045 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.602001 2576 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 19 15:24:26.602045 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.602012 2576 state_mem.go:35] "Initializing new in-memory state store" Apr 19 15:24:26.636407 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.636388 2576 manager.go:341] "Starting Device Plugin manager" Apr 19 15:24:26.636516 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.636422 2576 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 19 15:24:26.636516 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.636434 2576 server.go:85] "Starting device plugin registration server" Apr 19 15:24:26.636769 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.636750 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 19 15:24:26.636975 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.636772 2576 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 19 15:24:26.637080 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.637029 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 19 15:24:26.637254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.637157 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 19 15:24:26.637254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.637191 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 19 15:24:26.637735 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.637714 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 19 15:24:26.637818 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.637759 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-182.ec2.internal\" not found" Apr 19 15:24:26.638793 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.638765 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-130-182.ec2.internal" not found Apr 19 15:24:26.733464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.733379 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 19 15:24:26.735695 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.734811 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 19 15:24:26.735695 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.734843 2576 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 19 15:24:26.735695 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.734864 2576 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 19 15:24:26.735695 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.734872 2576 kubelet.go:2451] "Starting kubelet main sync loop" Apr 19 15:24:26.735695 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.734979 2576 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 19 15:24:26.737158 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.737139 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 19 15:24:26.737998 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.737983 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-182.ec2.internal" event="NodeHasSufficientMemory" Apr 19 15:24:26.738095 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.738010 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-182.ec2.internal" event="NodeHasNoDiskPressure" Apr 19 15:24:26.738095 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.738021 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-182.ec2.internal" event="NodeHasSufficientPID" Apr 19 15:24:26.738095 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.738047 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.741045 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.741030 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 19 15:24:26.744470 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.744446 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.744470 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:26.744470 2576 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-182.ec2.internal\": node \"ip-10-0-130-182.ec2.internal\" not found" Apr 19 15:24:26.835883 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.835848 2576 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal"] Apr 19 15:24:26.840460 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.840440 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.840548 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.840449 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.861217 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.861194 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.865476 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.865458 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.873420 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.873405 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 19 15:24:26.874676 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.874662 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 19 15:24:26.879925 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.879909 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/59e42b1c24a2af4d6251e18611da3c5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal\" (UID: \"59e42b1c24a2af4d6251e18611da3c5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.879981 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.879937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59e42b1c24a2af4d6251e18611da3c5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal\" (UID: \"59e42b1c24a2af4d6251e18611da3c5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.879981 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.879955 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1850971f2e72febbaa8e5e238327e28b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-182.ec2.internal\" (UID: \"1850971f2e72febbaa8e5e238327e28b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.980770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.980746 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/59e42b1c24a2af4d6251e18611da3c5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal\" (UID: \"59e42b1c24a2af4d6251e18611da3c5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.980913 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.980701 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/59e42b1c24a2af4d6251e18611da3c5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal\" (UID: \"59e42b1c24a2af4d6251e18611da3c5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.980913 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.980815 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59e42b1c24a2af4d6251e18611da3c5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal\" (UID: \"59e42b1c24a2af4d6251e18611da3c5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.980913 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.980836 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1850971f2e72febbaa8e5e238327e28b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-182.ec2.internal\" (UID: \"1850971f2e72febbaa8e5e238327e28b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.980913 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.980862 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1850971f2e72febbaa8e5e238327e28b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-182.ec2.internal\" (UID: \"1850971f2e72febbaa8e5e238327e28b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" Apr 19 15:24:26.980913 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:26.980892 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59e42b1c24a2af4d6251e18611da3c5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal\" (UID: \"59e42b1c24a2af4d6251e18611da3c5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:27.176708 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.176686 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" Apr 19 15:24:27.176848 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.176693 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" Apr 19 15:24:27.468178 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.468069 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 19 15:24:27.468801 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.468278 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 19 15:24:27.468801 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.468278 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 19 15:24:27.468801 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.468299 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 19 15:24:27.546808 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.546770 2576 apiserver.go:52] "Watching apiserver" Apr 19 15:24:27.554220 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.554194 2576 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 19 15:24:27.554317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.554189 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-18 15:19:26 +0000 UTC" deadline="2027-09-13 21:23:24.210108362 +0000 UTC" Apr 19 15:24:27.554317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.554283 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12293h58m56.655831881s" Apr 19 15:24:27.555789 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.555767 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-xvhn9","openshift-cluster-node-tuning-operator/tuned-gn47z","openshift-image-registry/node-ca-fw4zx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal","openshift-multus/multus-fjf4g","openshift-network-diagnostics/network-check-target-w2fz9","openshift-network-operator/iptables-alerter-52vcj","kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk","openshift-dns/node-resolver-rlhfq","openshift-multus/multus-additional-cni-plugins-bcjgt","openshift-multus/network-metrics-daemon-r7nm5","openshift-ovn-kubernetes/ovnkube-node-kqc9s"] Apr 19 15:24:27.560554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.560527 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.560709 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.560691 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.562964 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.562943 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mzwnd\"" Apr 19 15:24:27.563064 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.562946 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k824p\"" Apr 19 15:24:27.563121 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.563025 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 19 15:24:27.563175 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.563150 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 19 15:24:27.563227 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.563205 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 19 15:24:27.563321 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.563296 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 19 15:24:27.565310 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.565292 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.569999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.567490 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.569999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.567786 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 19 15:24:27.569999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.567868 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:27.569999 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.568036 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:27.569999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.568210 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 19 15:24:27.569999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.568216 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 19 15:24:27.569999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.568545 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lqrwl\"" Apr 19 15:24:27.570808 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.570787 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 19 15:24:27.570945 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.570919 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 19 15:24:27.571038 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.570948 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 19 15:24:27.571237 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.571224 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.571287 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.571269 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7lpvg\"" Apr 19 15:24:27.572494 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.572478 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 19 15:24:27.573392 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.573375 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-grs4d\"" Apr 19 15:24:27.573614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.573599 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 19 15:24:27.573740 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.573717 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 19 15:24:27.573812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.573738 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.573812 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.573782 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 19 15:24:27.575842 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.575826 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 19 15:24:27.575927 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.575839 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 19 15:24:27.575927 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.575875 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 19 15:24:27.576031 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.575971 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5w29k\"" Apr 19 15:24:27.576464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.576437 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.577281 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.577266 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 19 15:24:27.578602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.578579 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 19 15:24:27.578704 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.578688 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x84qs\"" Apr 19 15:24:27.579133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.579118 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.579380 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.579230 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 19 15:24:27.581323 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.581304 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 19 15:24:27.581426 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.581388 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 19 15:24:27.581603 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.581587 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l6tp9\"" Apr 19 15:24:27.581904 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.581890 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:27.581976 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.581957 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:27.584447 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-os-release\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.584553 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.584553 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584475 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rgww\" (UniqueName: \"kubernetes.io/projected/f8668eb5-0f25-4267-b970-27cee041914b-kube-api-access-8rgww\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.584553 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584499 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-tuned\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.584553 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584527 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-conf-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.584721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584565 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.584721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584595 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25b451a9-9cc5-4da3-8c6f-71b7f71e15dd-konnectivity-ca\") pod \"konnectivity-agent-xvhn9\" (UID: \"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd\") " pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.584721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584619 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-sys\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.584721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584652 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-etc-kubernetes\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.584721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584655 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.584721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584676 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxtbd\" (UniqueName: \"kubernetes.io/projected/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-kube-api-access-vxtbd\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.584721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqw86\" (UniqueName: \"kubernetes.io/projected/2514ec8e-0d56-4eda-929e-7a8759df8f30-kube-api-access-zqw86\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584727 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-cnibin\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584744 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysctl-conf\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-multus-certs\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584783 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-cnibin\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584810 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584860 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2514ec8e-0d56-4eda-929e-7a8759df8f30-hosts-file\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-systemd\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.584983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584953 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-host\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.584992 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c0a785a-7536-42b4-9261-4205121fcf3a-tmp\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585020 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-device-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585047 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-modprobe-d\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585086 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-cni-bin\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585126 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-socket-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585156 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-var-lib-kubelet\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585176 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-netns\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585206 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-hostroot\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585240 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25b451a9-9cc5-4da3-8c6f-71b7f71e15dd-agent-certs\") pod \"konnectivity-agent-xvhn9\" (UID: \"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd\") " pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585270 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7efc9272-6634-4b94-abe5-ba1e076ca391-serviceca\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585292 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-lib-modules\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585306 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-cni-binary-copy\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585325 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-registration-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585360 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-system-cni-dir\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysconfig\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585391 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hfsq\" (UniqueName: \"kubernetes.io/projected/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-kube-api-access-5hfsq\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-kubernetes\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-iptables-alerter-script\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585449 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-sys-fs\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.585464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585465 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7efc9272-6634-4b94-abe5-ba1e076ca391-host\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585480 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8tpf\" (UniqueName: \"kubernetes.io/projected/7efc9272-6634-4b94-abe5-ba1e076ca391-kube-api-access-r8tpf\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585496 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-host-slash\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585510 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-kubelet\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldppb\" (UniqueName: \"kubernetes.io/projected/6c0a785a-7536-42b4-9261-4205121fcf3a-kube-api-access-ldppb\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585557 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-cni-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2514ec8e-0d56-4eda-929e-7a8759df8f30-tmp-dir\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585586 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-run\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-system-cni-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585616 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-k8s-cni-cncf-io\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2r59\" (UniqueName: \"kubernetes.io/projected/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-kube-api-access-p2r59\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585686 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-os-release\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585701 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-daemon-config\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585716 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585741 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysctl-d\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585763 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-socket-dir-parent\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585786 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-cni-multus\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.585864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.585811 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-cni-binary-copy\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.586547 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.586523 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 19 15:24:27.586904 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.586888 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qwwqc\"" Apr 19 15:24:27.586967 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.586952 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 19 15:24:27.588120 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.588104 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 19 15:24:27.588120 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.588110 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 19 15:24:27.588256 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.588121 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 19 15:24:27.588256 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.588145 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 19 15:24:27.588256 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.588182 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 19 15:24:27.608027 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.607970 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jbfdn" Apr 19 15:24:27.615400 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.615375 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jbfdn" Apr 19 15:24:27.679359 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.679337 2576 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 19 15:24:27.686351 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-log-socket\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.686454 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686362 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-os-release\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.686454 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686382 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.686454 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686399 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8rgww\" (UniqueName: \"kubernetes.io/projected/f8668eb5-0f25-4267-b970-27cee041914b-kube-api-access-8rgww\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.686612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686446 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-var-lib-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.686612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686514 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-tuned\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.686612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686464 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-os-release\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.686612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686542 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-conf-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.686612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686577 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.686612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686582 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-conf-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.686612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686599 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25b451a9-9cc5-4da3-8c6f-71b7f71e15dd-konnectivity-ca\") pod \"konnectivity-agent-xvhn9\" (UID: \"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd\") " pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686676 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686674 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-etc-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686781 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-sys\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686826 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-etc-kubernetes\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686835 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-sys\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686855 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vxtbd\" (UniqueName: \"kubernetes.io/projected/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-kube-api-access-vxtbd\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686895 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-etc-kubernetes\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.686931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686926 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zqw86\" (UniqueName: \"kubernetes.io/projected/2514ec8e-0d56-4eda-929e-7a8759df8f30-kube-api-access-zqw86\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686957 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-cni-bin\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.686983 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-cnibin\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687000 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687036 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687011 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-run-ovn-kubernetes\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687067 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-cnibin\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687082 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-ovnkube-config\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687111 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysctl-conf\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687136 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-multus-certs\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687159 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-cnibin\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687168 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25b451a9-9cc5-4da3-8c6f-71b7f71e15dd-konnectivity-ca\") pod \"konnectivity-agent-xvhn9\" (UID: \"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd\") " pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687186 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687196 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-multus-certs\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687213 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysctl-conf\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687219 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2514ec8e-0d56-4eda-929e-7a8759df8f30-hosts-file\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687243 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-cnibin\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687254 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-kubelet\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.687386 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687268 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2514ec8e-0d56-4eda-929e-7a8759df8f30-hosts-file\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-node-log\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687328 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-systemd\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687355 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-host\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687380 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c0a785a-7536-42b4-9261-4205121fcf3a-tmp\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687427 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-systemd\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687446 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-host\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687454 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-device-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687483 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687511 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-modprobe-d\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687538 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-cni-bin\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687564 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687589 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-socket-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687593 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-device-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687615 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-env-overrides\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687658 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-var-lib-kubelet\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687672 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-cni-bin\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.688316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687681 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-netns\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687694 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687705 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-hostroot\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687763 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687765 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-var-lib-kubelet\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687766 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-modprobe-d\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687770 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-netns\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687784 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687797 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-hostroot\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687873 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25b451a9-9cc5-4da3-8c6f-71b7f71e15dd-agent-certs\") pod \"konnectivity-agent-xvhn9\" (UID: \"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd\") " pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687885 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-socket-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687906 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7efc9272-6634-4b94-abe5-ba1e076ca391-serviceca\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.687993 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-lib-modules\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688112 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-cni-binary-copy\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688169 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-registration-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688175 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-lib-modules\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688209 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-system-cni-dir\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.689866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688261 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-registration-dir\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-systemd\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688295 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-system-cni-dir\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688331 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysconfig\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688374 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5hfsq\" (UniqueName: \"kubernetes.io/projected/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-kube-api-access-5hfsq\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688373 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7efc9272-6634-4b94-abe5-ba1e076ca391-serviceca\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688418 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cncl\" (UniqueName: \"kubernetes.io/projected/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-kube-api-access-8cncl\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688446 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-systemd-units\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688479 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-kubernetes\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688507 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-iptables-alerter-script\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688533 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-sys-fs\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688541 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysconfig\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688560 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7efc9272-6634-4b94-abe5-ba1e076ca391-host\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688616 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-kubernetes\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688664 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-cni-binary-copy\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688697 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-sys-fs\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688708 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r8tpf\" (UniqueName: \"kubernetes.io/projected/7efc9272-6634-4b94-abe5-ba1e076ca391-kube-api-access-r8tpf\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.690501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-cni-netd\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688789 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7efc9272-6634-4b94-abe5-ba1e076ca391-host\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fd23153-4cc8-41aa-9038-9780e928b79b-ovn-node-metrics-cert\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-ovnkube-script-lib\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688854 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-host-slash\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688881 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-kubelet\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688914 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ldppb\" (UniqueName: \"kubernetes.io/projected/6c0a785a-7536-42b4-9261-4205121fcf3a-kube-api-access-ldppb\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688917 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-host-slash\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688941 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-cni-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.688967 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2514ec8e-0d56-4eda-929e-7a8759df8f30-tmp-dir\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689004 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-kubelet\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689019 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-cni-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689007 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-ovn\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689062 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-run\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689090 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-system-cni-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689140 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-system-cni-dir\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689140 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-run\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689160 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-k8s-cni-cncf-io\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689188 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p2r59\" (UniqueName: \"kubernetes.io/projected/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-kube-api-access-p2r59\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689200 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-run-k8s-cni-cncf-io\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twxkt\" (UniqueName: \"kubernetes.io/projected/6fd23153-4cc8-41aa-9038-9780e928b79b-kube-api-access-twxkt\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689228 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-os-release\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689243 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-daemon-config\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689267 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689341 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-os-release\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-slash\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689418 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-run-netns\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689394 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8668eb5-0f25-4267-b970-27cee041914b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689419 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2514ec8e-0d56-4eda-929e-7a8759df8f30-tmp-dir\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689461 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysctl-d\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689491 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-socket-dir-parent\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689515 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-cni-multus\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689525 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-iptables-alerter-script\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689540 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-cni-binary-copy\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689566 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-sysctl-d\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.691513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689566 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-socket-dir-parent\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689614 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-host-var-lib-cni-multus\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.689693 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-multus-daemon-config\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.691990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.690070 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8668eb5-0f25-4267-b970-27cee041914b-cni-binary-copy\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.691990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.690219 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6c0a785a-7536-42b4-9261-4205121fcf3a-etc-tuned\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.691990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.690618 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25b451a9-9cc5-4da3-8c6f-71b7f71e15dd-agent-certs\") pod \"konnectivity-agent-xvhn9\" (UID: \"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd\") " pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.691990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.690777 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6c0a785a-7536-42b4-9261-4205121fcf3a-tmp\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.693504 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.693485 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 19 15:24:27.693613 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.693509 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 19 15:24:27.693613 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.693522 2576 projected.go:194] Error preparing data for projected volume kube-api-access-lh7j5 for pod openshift-network-diagnostics/network-check-target-w2fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:27.693613 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.693597 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5 podName:2e551954-ed8a-49b4-8237-75b061234272 nodeName:}" failed. No retries permitted until 2026-04-19 15:24:28.193565653 +0000 UTC m=+2.123025810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lh7j5" (UniqueName: "kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5") pod "network-check-target-w2fz9" (UID: "2e551954-ed8a-49b4-8237-75b061234272") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:27.694851 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.694827 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rgww\" (UniqueName: \"kubernetes.io/projected/f8668eb5-0f25-4267-b970-27cee041914b-kube-api-access-8rgww\") pod \"multus-additional-cni-plugins-bcjgt\" (UID: \"f8668eb5-0f25-4267-b970-27cee041914b\") " pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.694851 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.694835 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqw86\" (UniqueName: \"kubernetes.io/projected/2514ec8e-0d56-4eda-929e-7a8759df8f30-kube-api-access-zqw86\") pod \"node-resolver-rlhfq\" (UID: \"2514ec8e-0d56-4eda-929e-7a8759df8f30\") " pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.694999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.694950 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxtbd\" (UniqueName: \"kubernetes.io/projected/f1e64598-e85a-42b6-9c8e-b2b326e5c8c3-kube-api-access-vxtbd\") pod \"iptables-alerter-52vcj\" (UID: \"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3\") " pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.696208 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.696187 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2r59\" (UniqueName: \"kubernetes.io/projected/6d1f13eb-aa2b-4628-9adc-6d546d487ab3-kube-api-access-p2r59\") pod \"multus-fjf4g\" (UID: \"6d1f13eb-aa2b-4628-9adc-6d546d487ab3\") " pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.696450 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.696434 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hfsq\" (UniqueName: \"kubernetes.io/projected/ee4bfd2c-6d77-4f44-8e67-bc439b8372c2-kube-api-access-5hfsq\") pod \"aws-ebs-csi-driver-node-2kdxk\" (UID: \"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.696968 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.696942 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8tpf\" (UniqueName: \"kubernetes.io/projected/7efc9272-6634-4b94-abe5-ba1e076ca391-kube-api-access-r8tpf\") pod \"node-ca-fw4zx\" (UID: \"7efc9272-6634-4b94-abe5-ba1e076ca391\") " pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.697055 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.697035 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldppb\" (UniqueName: \"kubernetes.io/projected/6c0a785a-7536-42b4-9261-4205121fcf3a-kube-api-access-ldppb\") pod \"tuned-gn47z\" (UID: \"6c0a785a-7536-42b4-9261-4205121fcf3a\") " pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.706650 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.706616 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rlhfq" Apr 19 15:24:27.717161 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.717144 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" Apr 19 15:24:27.734099 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.734069 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1850971f2e72febbaa8e5e238327e28b.slice/crio-cb63f38ecc5ae8f509660dda1f34be698b499ad329229dcc85774da706d278ea WatchSource:0}: Error finding container cb63f38ecc5ae8f509660dda1f34be698b499ad329229dcc85774da706d278ea: Status 404 returned error can't find the container with id cb63f38ecc5ae8f509660dda1f34be698b499ad329229dcc85774da706d278ea Apr 19 15:24:27.735164 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.735143 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59e42b1c24a2af4d6251e18611da3c5e.slice/crio-425260ae189e761500d9f482bf4f2d05b26a582d1d52eab4a095880f338f5f7d WatchSource:0}: Error finding container 425260ae189e761500d9f482bf4f2d05b26a582d1d52eab4a095880f338f5f7d: Status 404 returned error can't find the container with id 425260ae189e761500d9f482bf4f2d05b26a582d1d52eab4a095880f338f5f7d Apr 19 15:24:27.735775 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.735754 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2514ec8e_0d56_4eda_929e_7a8759df8f30.slice/crio-c01024a1e655b8009f44770bc6dc500e15c5a4eef8c0795cc1ce28563f95bb25 WatchSource:0}: Error finding container c01024a1e655b8009f44770bc6dc500e15c5a4eef8c0795cc1ce28563f95bb25: Status 404 returned error can't find the container with id c01024a1e655b8009f44770bc6dc500e15c5a4eef8c0795cc1ce28563f95bb25 Apr 19 15:24:27.737132 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.737018 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8668eb5_0f25_4267_b970_27cee041914b.slice/crio-04f02246ef999923291d15c622bd72408a81e82deb6dd03e993678989225d24f WatchSource:0}: Error finding container 04f02246ef999923291d15c622bd72408a81e82deb6dd03e993678989225d24f: Status 404 returned error can't find the container with id 04f02246ef999923291d15c622bd72408a81e82deb6dd03e993678989225d24f Apr 19 15:24:27.739015 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.738999 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 19 15:24:27.790104 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790074 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-ovn\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790104 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790105 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-twxkt\" (UniqueName: \"kubernetes.io/projected/6fd23153-4cc8-41aa-9038-9780e928b79b-kube-api-access-twxkt\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790122 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-slash\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790137 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-run-netns\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790169 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-log-socket\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790174 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-ovn\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790209 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-slash\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790218 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-run-netns\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790223 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-var-lib-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790226 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-log-socket\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790326 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-var-lib-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790375 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790410 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-etc-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790436 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790470 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-cni-bin\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790481 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-etc-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.790490 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790523 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-openvswitch\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790528 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-run-ovn-kubernetes\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790489 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-run-ovn-kubernetes\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:27.790560 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:24:28.290539747 +0000 UTC m=+2.219999903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:27.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790562 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-cni-bin\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790661 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-ovnkube-config\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790714 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-kubelet\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790739 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-node-log\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790768 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790773 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-kubelet\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-node-log\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790816 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-env-overrides\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790813 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790903 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-systemd\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790942 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-run-systemd\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790949 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8cncl\" (UniqueName: \"kubernetes.io/projected/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-kube-api-access-8cncl\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.790976 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-systemd-units\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791003 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-cni-netd\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791028 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fd23153-4cc8-41aa-9038-9780e928b79b-ovn-node-metrics-cert\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791053 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-ovnkube-script-lib\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791062 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-systemd-units\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791084 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6fd23153-4cc8-41aa-9038-9780e928b79b-host-cni-netd\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791837 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791183 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-env-overrides\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791837 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791213 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-ovnkube-config\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.791837 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.791558 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6fd23153-4cc8-41aa-9038-9780e928b79b-ovnkube-script-lib\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.793195 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.793179 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6fd23153-4cc8-41aa-9038-9780e928b79b-ovn-node-metrics-cert\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.798411 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.798386 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twxkt\" (UniqueName: \"kubernetes.io/projected/6fd23153-4cc8-41aa-9038-9780e928b79b-kube-api-access-twxkt\") pod \"ovnkube-node-kqc9s\" (UID: \"6fd23153-4cc8-41aa-9038-9780e928b79b\") " pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:27.798620 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.798599 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cncl\" (UniqueName: \"kubernetes.io/projected/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-kube-api-access-8cncl\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:27.898610 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.898583 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:27.904459 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.904433 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b451a9_9cc5_4da3_8c6f_71b7f71e15dd.slice/crio-9225819209c53e4f4d7a10b4ac9ede96a7defa97480d959161e1b9562905d267 WatchSource:0}: Error finding container 9225819209c53e4f4d7a10b4ac9ede96a7defa97480d959161e1b9562905d267: Status 404 returned error can't find the container with id 9225819209c53e4f4d7a10b4ac9ede96a7defa97480d959161e1b9562905d267 Apr 19 15:24:27.907268 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.907253 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gn47z" Apr 19 15:24:27.913918 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.913897 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0a785a_7536_42b4_9261_4205121fcf3a.slice/crio-d1ea7003502301debc88341584f7611b0237fe7f980fce02c4840430b8864f50 WatchSource:0}: Error finding container d1ea7003502301debc88341584f7611b0237fe7f980fce02c4840430b8864f50: Status 404 returned error can't find the container with id d1ea7003502301debc88341584f7611b0237fe7f980fce02c4840430b8864f50 Apr 19 15:24:27.928269 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.928250 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fw4zx" Apr 19 15:24:27.933737 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.933717 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7efc9272_6634_4b94_abe5_ba1e076ca391.slice/crio-63fb80ffe761535037b5afcf2d068cf36c989ecb07370bce53293404dc39827f WatchSource:0}: Error finding container 63fb80ffe761535037b5afcf2d068cf36c989ecb07370bce53293404dc39827f: Status 404 returned error can't find the container with id 63fb80ffe761535037b5afcf2d068cf36c989ecb07370bce53293404dc39827f Apr 19 15:24:27.938556 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.938537 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjf4g" Apr 19 15:24:27.945415 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.945388 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1f13eb_aa2b_4628_9adc_6d546d487ab3.slice/crio-d4c97c4c93209a235ba4e1511c1b2258565dcfddac062022d442edae4831dbb8 WatchSource:0}: Error finding container d4c97c4c93209a235ba4e1511c1b2258565dcfddac062022d442edae4831dbb8: Status 404 returned error can't find the container with id d4c97c4c93209a235ba4e1511c1b2258565dcfddac062022d442edae4831dbb8 Apr 19 15:24:27.961401 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.961382 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-52vcj" Apr 19 15:24:27.968583 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.968558 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e64598_e85a_42b6_9c8e_b2b326e5c8c3.slice/crio-53109de5c9e1924a60f233b72291a225b81165d7c37dcf664d606f6fd2ba97d1 WatchSource:0}: Error finding container 53109de5c9e1924a60f233b72291a225b81165d7c37dcf664d606f6fd2ba97d1: Status 404 returned error can't find the container with id 53109de5c9e1924a60f233b72291a225b81165d7c37dcf664d606f6fd2ba97d1 Apr 19 15:24:27.980897 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:27.980878 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" Apr 19 15:24:27.986549 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:27.986529 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4bfd2c_6d77_4f44_8e67_bc439b8372c2.slice/crio-1c72ced2ba74ce506012ffe43620925be24cec839fb610ceb3941701c1ca641c WatchSource:0}: Error finding container 1c72ced2ba74ce506012ffe43620925be24cec839fb610ceb3941701c1ca641c: Status 404 returned error can't find the container with id 1c72ced2ba74ce506012ffe43620925be24cec839fb610ceb3941701c1ca641c Apr 19 15:24:28.023461 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.023442 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:28.028876 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:24:28.028857 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd23153_4cc8_41aa_9038_9780e928b79b.slice/crio-ffd2ed50719478a07dbd4be8391c5d7ed60c7b6d2053d686786c6082c2f388ac WatchSource:0}: Error finding container ffd2ed50719478a07dbd4be8391c5d7ed60c7b6d2053d686786c6082c2f388ac: Status 404 returned error can't find the container with id ffd2ed50719478a07dbd4be8391c5d7ed60c7b6d2053d686786c6082c2f388ac Apr 19 15:24:28.194369 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.194338 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:28.194549 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:28.194522 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 19 15:24:28.194549 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:28.194541 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 19 15:24:28.194685 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:28.194552 2576 projected.go:194] Error preparing data for projected volume kube-api-access-lh7j5 for pod openshift-network-diagnostics/network-check-target-w2fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:28.194685 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:28.194608 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5 podName:2e551954-ed8a-49b4-8237-75b061234272 nodeName:}" failed. No retries permitted until 2026-04-19 15:24:29.194591375 +0000 UTC m=+3.124051517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lh7j5" (UniqueName: "kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5") pod "network-check-target-w2fz9" (UID: "2e551954-ed8a-49b4-8237-75b061234272") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:28.295052 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.294975 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:28.295207 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:28.295161 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:28.295269 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:28.295240 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:24:29.295219152 +0000 UTC m=+3.224679293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:28.306536 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.306509 2576 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 19 15:24:28.306931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.306910 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 19 15:24:28.616457 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.616420 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-18 15:19:27 +0000 UTC" deadline="2027-09-18 21:25:55.433639735 +0000 UTC" Apr 19 15:24:28.616457 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.616456 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12414h1m26.817187703s" Apr 19 15:24:28.627387 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.627360 2576 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 19 15:24:28.760867 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.760798 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerStarted","Data":"04f02246ef999923291d15c622bd72408a81e82deb6dd03e993678989225d24f"} Apr 19 15:24:28.776271 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.776193 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" event={"ID":"1850971f2e72febbaa8e5e238327e28b","Type":"ContainerStarted","Data":"cb63f38ecc5ae8f509660dda1f34be698b499ad329229dcc85774da706d278ea"} Apr 19 15:24:28.788075 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.787916 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" event={"ID":"59e42b1c24a2af4d6251e18611da3c5e","Type":"ContainerStarted","Data":"425260ae189e761500d9f482bf4f2d05b26a582d1d52eab4a095880f338f5f7d"} Apr 19 15:24:28.809610 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.809574 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" event={"ID":"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2","Type":"ContainerStarted","Data":"1c72ced2ba74ce506012ffe43620925be24cec839fb610ceb3941701c1ca641c"} Apr 19 15:24:28.825512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.825474 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fw4zx" event={"ID":"7efc9272-6634-4b94-abe5-ba1e076ca391","Type":"ContainerStarted","Data":"63fb80ffe761535037b5afcf2d068cf36c989ecb07370bce53293404dc39827f"} Apr 19 15:24:28.844271 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.844231 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gn47z" event={"ID":"6c0a785a-7536-42b4-9261-4205121fcf3a","Type":"ContainerStarted","Data":"d1ea7003502301debc88341584f7611b0237fe7f980fce02c4840430b8864f50"} Apr 19 15:24:28.859564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.859516 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rlhfq" event={"ID":"2514ec8e-0d56-4eda-929e-7a8759df8f30","Type":"ContainerStarted","Data":"c01024a1e655b8009f44770bc6dc500e15c5a4eef8c0795cc1ce28563f95bb25"} Apr 19 15:24:28.873064 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.872961 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"ffd2ed50719478a07dbd4be8391c5d7ed60c7b6d2053d686786c6082c2f388ac"} Apr 19 15:24:28.884502 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.884459 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52vcj" event={"ID":"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3","Type":"ContainerStarted","Data":"53109de5c9e1924a60f233b72291a225b81165d7c37dcf664d606f6fd2ba97d1"} Apr 19 15:24:28.901134 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.901099 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjf4g" event={"ID":"6d1f13eb-aa2b-4628-9adc-6d546d487ab3","Type":"ContainerStarted","Data":"d4c97c4c93209a235ba4e1511c1b2258565dcfddac062022d442edae4831dbb8"} Apr 19 15:24:28.912282 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:28.912251 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xvhn9" event={"ID":"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd","Type":"ContainerStarted","Data":"9225819209c53e4f4d7a10b4ac9ede96a7defa97480d959161e1b9562905d267"} Apr 19 15:24:29.203529 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:29.203362 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:29.203771 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.203530 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 19 15:24:29.203771 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.203551 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 19 15:24:29.203771 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.203565 2576 projected.go:194] Error preparing data for projected volume kube-api-access-lh7j5 for pod openshift-network-diagnostics/network-check-target-w2fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:29.203771 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.203645 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5 podName:2e551954-ed8a-49b4-8237-75b061234272 nodeName:}" failed. No retries permitted until 2026-04-19 15:24:31.203608997 +0000 UTC m=+5.133069139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lh7j5" (UniqueName: "kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5") pod "network-check-target-w2fz9" (UID: "2e551954-ed8a-49b4-8237-75b061234272") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:29.303835 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:29.303792 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:29.304010 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.303942 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:29.304069 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.304018 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:24:31.303983375 +0000 UTC m=+5.233443521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:29.617398 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:29.617361 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-18 15:19:27 +0000 UTC" deadline="2027-11-10 21:12:35.681709646 +0000 UTC" Apr 19 15:24:29.617398 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:29.617393 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13685h48m6.064318591s" Apr 19 15:24:29.736669 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:29.735905 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:29.736669 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.736061 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:29.736669 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:29.736495 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:29.736669 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:29.736582 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:31.223730 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:31.223102 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:31.223730 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.223264 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 19 15:24:31.223730 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.223284 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 19 15:24:31.223730 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.223296 2576 projected.go:194] Error preparing data for projected volume kube-api-access-lh7j5 for pod openshift-network-diagnostics/network-check-target-w2fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:31.223730 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.223348 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5 podName:2e551954-ed8a-49b4-8237-75b061234272 nodeName:}" failed. No retries permitted until 2026-04-19 15:24:35.223330197 +0000 UTC m=+9.152790339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lh7j5" (UniqueName: "kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5") pod "network-check-target-w2fz9" (UID: "2e551954-ed8a-49b4-8237-75b061234272") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:31.324209 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:31.324139 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:31.324399 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.324279 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:31.324399 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.324355 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:24:35.324333566 +0000 UTC m=+9.253793729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:31.735761 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:31.735728 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:31.735977 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.735871 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:31.736419 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:31.736272 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:31.736419 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:31.736363 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:33.736104 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:33.735971 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:33.736645 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:33.736115 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:33.736645 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:33.736258 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:33.736645 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:33.736381 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:35.258189 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:35.258077 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:35.258607 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.258286 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 19 15:24:35.258607 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.258311 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 19 15:24:35.258607 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.258326 2576 projected.go:194] Error preparing data for projected volume kube-api-access-lh7j5 for pod openshift-network-diagnostics/network-check-target-w2fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:35.258607 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.258394 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5 podName:2e551954-ed8a-49b4-8237-75b061234272 nodeName:}" failed. No retries permitted until 2026-04-19 15:24:43.258375437 +0000 UTC m=+17.187835581 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lh7j5" (UniqueName: "kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5") pod "network-check-target-w2fz9" (UID: "2e551954-ed8a-49b4-8237-75b061234272") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:35.358958 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:35.358925 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:35.359137 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.359046 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:35.359137 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.359108 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:24:43.359088264 +0000 UTC m=+17.288548405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:35.735438 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:35.735402 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:35.735655 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.735536 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:35.735655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:35.735603 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:35.735794 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:35.735725 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:37.735915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:37.735877 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:37.736285 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:37.735988 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:37.736285 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:37.736043 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:37.736285 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:37.736142 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:39.735313 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:39.735272 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:39.735313 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:39.735295 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:39.735824 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:39.735416 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:39.735824 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:39.735567 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:41.735334 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:41.735290 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:41.735334 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:41.735336 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:41.735883 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:41.735426 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:41.735883 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:41.735567 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:43.319723 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:43.319682 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:43.320211 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.319861 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 19 15:24:43.320211 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.319886 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 19 15:24:43.320211 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.319898 2576 projected.go:194] Error preparing data for projected volume kube-api-access-lh7j5 for pod openshift-network-diagnostics/network-check-target-w2fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:43.320211 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.319960 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5 podName:2e551954-ed8a-49b4-8237-75b061234272 nodeName:}" failed. No retries permitted until 2026-04-19 15:24:59.319945999 +0000 UTC m=+33.249406139 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lh7j5" (UniqueName: "kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5") pod "network-check-target-w2fz9" (UID: "2e551954-ed8a-49b4-8237-75b061234272") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:43.420555 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:43.420513 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:43.420761 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.420650 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:43.420761 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.420722 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:24:59.420701142 +0000 UTC m=+33.350161306 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:43.735912 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:43.735875 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:43.736093 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:43.735875 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:43.736093 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.736017 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:43.736229 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:43.736128 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:45.736213 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:45.736053 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:45.736483 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:45.736119 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:45.736483 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:45.736284 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:45.736483 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:45.736357 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:45.945413 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:45.945382 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjf4g" event={"ID":"6d1f13eb-aa2b-4628-9adc-6d546d487ab3","Type":"ContainerStarted","Data":"07e9f92e3d8ad6447649bff4edc51dd867e6330385cff215ad6875e09e048bb3"} Apr 19 15:24:45.946805 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:45.946780 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" event={"ID":"1850971f2e72febbaa8e5e238327e28b","Type":"ContainerStarted","Data":"233ffaba57691331436790220a56dfc6286eda9085006667d4badce4c2cf9888"} Apr 19 15:24:45.947969 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:45.947944 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gn47z" event={"ID":"6c0a785a-7536-42b4-9261-4205121fcf3a","Type":"ContainerStarted","Data":"c9e24903ab86a075dc06d400da976c9cc8c84b7feaa5688585e7276461cf8a8d"} Apr 19 15:24:45.962233 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:45.962193 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-fjf4g" podStartSLOduration=2.267582837 podStartE2EDuration="19.962180267s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.946960384 +0000 UTC m=+1.876420535" lastFinishedPulling="2026-04-19 15:24:45.641557824 +0000 UTC m=+19.571017965" observedRunningTime="2026-04-19 15:24:45.96177407 +0000 UTC m=+19.891234231" watchObservedRunningTime="2026-04-19 15:24:45.962180267 +0000 UTC m=+19.891640429" Apr 19 15:24:45.981204 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:45.981170 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gn47z" podStartSLOduration=2.254288412 podStartE2EDuration="19.98115866s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.915270075 +0000 UTC m=+1.844730215" lastFinishedPulling="2026-04-19 15:24:45.642140309 +0000 UTC m=+19.571600463" observedRunningTime="2026-04-19 15:24:45.980985841 +0000 UTC m=+19.910446013" watchObservedRunningTime="2026-04-19 15:24:45.98115866 +0000 UTC m=+19.910618819" Apr 19 15:24:46.951472 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.951174 2576 generic.go:358] "Generic (PLEG): container finished" podID="59e42b1c24a2af4d6251e18611da3c5e" containerID="3b520e34e4dfea118dd1da24e99edbb0ef0b37c15fe49539b1e212fe75aaa69a" exitCode=0 Apr 19 15:24:46.952178 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.951257 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" event={"ID":"59e42b1c24a2af4d6251e18611da3c5e","Type":"ContainerDied","Data":"3b520e34e4dfea118dd1da24e99edbb0ef0b37c15fe49539b1e212fe75aaa69a"} Apr 19 15:24:46.953043 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.953021 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" event={"ID":"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2","Type":"ContainerStarted","Data":"c7134979889c7b15d809d542a98232dcb2bbcb2a9bbf248de59a47f77daf2367"} Apr 19 15:24:46.954283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.954263 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fw4zx" event={"ID":"7efc9272-6634-4b94-abe5-ba1e076ca391","Type":"ContainerStarted","Data":"f60253366f1f9991e9d2a67eb39eec1d50226ca025a92a6ad8c67099c0fb51f0"} Apr 19 15:24:46.955596 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.955534 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rlhfq" event={"ID":"2514ec8e-0d56-4eda-929e-7a8759df8f30","Type":"ContainerStarted","Data":"5bf8ab4c11afe8b931566a9de11f7b07aff4870150e65f6778781cc3a57df3f6"} Apr 19 15:24:46.958138 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.958119 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"2cec61095adddc80eeab9cf5cd77595abd459e32ae5fec396f17de8cddabd358"} Apr 19 15:24:46.958223 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.958142 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"1645c63d3e72826c8e0464486f48c0af74195488fc09bcbb4202bdda0bde944c"} Apr 19 15:24:46.958223 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.958150 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"78dafc6b1f8057739d0f76908161b9893daf92d2d52c4419500d6781ad8a9757"} Apr 19 15:24:46.958223 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.958159 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"6602244b2b00d49d0f4b73ef1f0be38f5ee5f4a6f603fc471350c069a7c5f0f4"} Apr 19 15:24:46.958223 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.958166 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"04f9732c46b0656bd2b3a9d35dafda04a3892ce79abe0971bd07fa764b6eb872"} Apr 19 15:24:46.958223 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.958174 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"1c167c0aadac9f4c849425dcc4799f223f53f141c890c34c510ec00f6b93e6f4"} Apr 19 15:24:46.959246 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.959222 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xvhn9" event={"ID":"25b451a9-9cc5-4da3-8c6f-71b7f71e15dd","Type":"ContainerStarted","Data":"a7ecd322149c8fa7ff3feef0bc819397e856c90ea28bcb68e4113f0afca320ca"} Apr 19 15:24:46.960406 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.960385 2576 generic.go:358] "Generic (PLEG): container finished" podID="f8668eb5-0f25-4267-b970-27cee041914b" containerID="a1408f807947115457cc2b989f526c77c76ee5e3f7e28e700019a581568dd399" exitCode=0 Apr 19 15:24:46.960497 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.960474 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerDied","Data":"a1408f807947115457cc2b989f526c77c76ee5e3f7e28e700019a581568dd399"} Apr 19 15:24:46.965973 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.965905 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-182.ec2.internal" podStartSLOduration=20.965892114 podStartE2EDuration="20.965892114s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-19 15:24:45.994606314 +0000 UTC m=+19.924066476" watchObservedRunningTime="2026-04-19 15:24:46.965892114 +0000 UTC m=+20.895352277" Apr 19 15:24:46.978084 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.978041 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fw4zx" podStartSLOduration=3.371171414 podStartE2EDuration="20.978029752s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.935109425 +0000 UTC m=+1.864569565" lastFinishedPulling="2026-04-19 15:24:45.54196776 +0000 UTC m=+19.471427903" observedRunningTime="2026-04-19 15:24:46.977377607 +0000 UTC m=+20.906837779" watchObservedRunningTime="2026-04-19 15:24:46.978029752 +0000 UTC m=+20.907489914" Apr 19 15:24:46.992096 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:46.992053 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-rlhfq" podStartSLOduration=3.189504234 podStartE2EDuration="20.992041039s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.739353406 +0000 UTC m=+1.668813552" lastFinishedPulling="2026-04-19 15:24:45.541890202 +0000 UTC m=+19.471350357" observedRunningTime="2026-04-19 15:24:46.991606688 +0000 UTC m=+20.921066852" watchObservedRunningTime="2026-04-19 15:24:46.992041039 +0000 UTC m=+20.921501201" Apr 19 15:24:47.735478 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.735449 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:47.735596 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.735481 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:47.735667 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:47.735596 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:47.736006 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:47.735979 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:47.846274 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.846244 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 19 15:24:47.964016 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.963931 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-52vcj" event={"ID":"f1e64598-e85a-42b6-9c8e-b2b326e5c8c3","Type":"ContainerStarted","Data":"eab6a5e04dc09479f82bf2a64ba3ddbb0277d0b0e85a27b6eb206a57a6324d8b"} Apr 19 15:24:47.965790 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.965753 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" event={"ID":"59e42b1c24a2af4d6251e18611da3c5e","Type":"ContainerStarted","Data":"e6846bb8791f563ddd91987859980b139508c31582f64735ed2237cbfdf1fae9"} Apr 19 15:24:47.968102 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.968052 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" event={"ID":"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2","Type":"ContainerStarted","Data":"b4d80fa403f2ff9859183a6365ea8c1d3ba203bc4bf55458af25b611859da6ed"} Apr 19 15:24:47.977519 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.977480 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xvhn9" podStartSLOduration=4.353834593 podStartE2EDuration="21.977465283s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.905919489 +0000 UTC m=+1.835379630" lastFinishedPulling="2026-04-19 15:24:45.529550165 +0000 UTC m=+19.459010320" observedRunningTime="2026-04-19 15:24:47.032122164 +0000 UTC m=+20.961582326" watchObservedRunningTime="2026-04-19 15:24:47.977465283 +0000 UTC m=+21.906925445" Apr 19 15:24:47.977836 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:47.977806 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-52vcj" podStartSLOduration=4.418189435 podStartE2EDuration="21.977799049s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.96996504 +0000 UTC m=+1.899425179" lastFinishedPulling="2026-04-19 15:24:45.529574651 +0000 UTC m=+19.459034793" observedRunningTime="2026-04-19 15:24:47.97715562 +0000 UTC m=+21.906615785" watchObservedRunningTime="2026-04-19 15:24:47.977799049 +0000 UTC m=+21.907259211" Apr 19 15:24:48.646509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:48.646404 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-19T15:24:47.846266071Z","UUID":"87f24642-f222-495f-bf54-08453de13370","Handler":null,"Name":"","Endpoint":""} Apr 19 15:24:48.648912 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:48.648886 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 19 15:24:48.648912 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:48.648915 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 19 15:24:48.973690 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:48.973653 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"619e12298e52d6bd76ef2603b2399494d4490d63746fea6c2fc150f23e2ce1fb"} Apr 19 15:24:48.975593 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:48.975556 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" event={"ID":"ee4bfd2c-6d77-4f44-8e67-bc439b8372c2","Type":"ContainerStarted","Data":"e27cea9115cd70c563d1f7735872b9a0b32b85483260396303efe4c844c8dfcc"} Apr 19 15:24:48.992267 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:48.992215 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-182.ec2.internal" podStartSLOduration=22.99220194 podStartE2EDuration="22.99220194s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-19 15:24:47.990659709 +0000 UTC m=+21.920119871" watchObservedRunningTime="2026-04-19 15:24:48.99220194 +0000 UTC m=+22.921662101" Apr 19 15:24:48.992512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:48.992471 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2kdxk" podStartSLOduration=2.259974139 podStartE2EDuration="22.992464617s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.987920521 +0000 UTC m=+1.917380660" lastFinishedPulling="2026-04-19 15:24:48.720410991 +0000 UTC m=+22.649871138" observedRunningTime="2026-04-19 15:24:48.991989411 +0000 UTC m=+22.921449574" watchObservedRunningTime="2026-04-19 15:24:48.992464617 +0000 UTC m=+22.921924781" Apr 19 15:24:49.735864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:49.735833 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:49.736046 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:49.735840 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:49.736046 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:49.735927 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:49.736046 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:49.736007 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:50.227461 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:50.227421 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:50.228251 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:50.228233 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:50.980280 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:50.980248 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:50.980816 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:50.980797 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xvhn9" Apr 19 15:24:51.735549 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.735520 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:51.735956 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.735524 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:51.735956 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:51.735679 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:51.735956 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:51.735722 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:51.984296 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.984213 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" event={"ID":"6fd23153-4cc8-41aa-9038-9780e928b79b","Type":"ContainerStarted","Data":"c3a2ee2e90c09bc2c8e20f8f9d76a2c84fdee75ee889cc211ffbaccbd0df2b6b"} Apr 19 15:24:51.984558 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.984537 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:51.984674 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.984567 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:51.986042 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.986019 2576 generic.go:358] "Generic (PLEG): container finished" podID="f8668eb5-0f25-4267-b970-27cee041914b" containerID="ee5ee949d97d8d1a51c37444ff792cf3cb65da5b16420898467d592bd7cb61bc" exitCode=0 Apr 19 15:24:51.986150 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.986104 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerDied","Data":"ee5ee949d97d8d1a51c37444ff792cf3cb65da5b16420898467d592bd7cb61bc"} Apr 19 15:24:51.999429 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:51.999411 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:52.010711 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:52.010674 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" podStartSLOduration=7.942288688 podStartE2EDuration="26.010660908s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:28.03028085 +0000 UTC m=+1.959740990" lastFinishedPulling="2026-04-19 15:24:46.098653066 +0000 UTC m=+20.028113210" observedRunningTime="2026-04-19 15:24:52.010076164 +0000 UTC m=+25.939536326" watchObservedRunningTime="2026-04-19 15:24:52.010660908 +0000 UTC m=+25.940121069" Apr 19 15:24:52.990472 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:52.990240 2576 generic.go:358] "Generic (PLEG): container finished" podID="f8668eb5-0f25-4267-b970-27cee041914b" containerID="d2b0b55dd6fd2536863aa96f15bfd3ac1e6d3d32a1203fb0eef8bed980696411" exitCode=0 Apr 19 15:24:52.990866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:52.990321 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerDied","Data":"d2b0b55dd6fd2536863aa96f15bfd3ac1e6d3d32a1203fb0eef8bed980696411"} Apr 19 15:24:52.991306 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:52.991289 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:53.004991 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:53.004970 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:24:53.455129 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:53.455085 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w2fz9"] Apr 19 15:24:53.455330 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:53.455249 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:53.455446 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:53.455420 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:53.457266 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:53.457242 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r7nm5"] Apr 19 15:24:53.457373 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:53.457357 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:53.457472 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:53.457450 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:53.994862 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:53.994768 2576 generic.go:358] "Generic (PLEG): container finished" podID="f8668eb5-0f25-4267-b970-27cee041914b" containerID="bb5bc720be2810a6bdcdd5f2e61ed3b22bea1ffb1a9b7b727b3006ca8439ef5a" exitCode=0 Apr 19 15:24:53.995770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:53.994853 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerDied","Data":"bb5bc720be2810a6bdcdd5f2e61ed3b22bea1ffb1a9b7b727b3006ca8439ef5a"} Apr 19 15:24:54.735440 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:54.735407 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:54.735591 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:54.735522 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:55.736233 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:55.735976 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:55.736616 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:55.736267 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:56.736455 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:56.736420 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:56.736899 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:56.736560 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:57.736026 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:57.735985 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:57.736233 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:57.736106 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2fz9" podUID="2e551954-ed8a-49b4-8237-75b061234272" Apr 19 15:24:58.735273 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.735192 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:58.735726 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:58.735329 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7nm5" podUID="7aeaccc6-af60-4b21-9803-f56daf2f7ddb" Apr 19 15:24:58.876163 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.876138 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-182.ec2.internal" event="NodeReady" Apr 19 15:24:58.876317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.876290 2576 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 19 15:24:58.915936 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.915905 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wh4pt"] Apr 19 15:24:58.949797 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.949729 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qcm6b"] Apr 19 15:24:58.949957 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.949881 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:58.953177 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.952559 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 19 15:24:58.953177 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.952664 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dvb5z\"" Apr 19 15:24:58.953177 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.952559 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 19 15:24:58.965001 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.964325 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wh4pt"] Apr 19 15:24:58.965001 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.964355 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qcm6b"] Apr 19 15:24:58.965001 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.964454 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:24:58.967050 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.967027 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 19 15:24:58.967159 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.967087 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zxl4j\"" Apr 19 15:24:58.967159 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.967135 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 19 15:24:58.967159 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:58.967142 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 19 15:24:59.037554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.037475 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f37774-81cc-4b2c-94cb-ec77935ec46d-config-volume\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.037554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.037522 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtkvc\" (UniqueName: \"kubernetes.io/projected/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-kube-api-access-jtkvc\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:24:59.037779 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.037558 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3f37774-81cc-4b2c-94cb-ec77935ec46d-tmp-dir\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.037779 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.037613 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.037779 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.037710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qxnz\" (UniqueName: \"kubernetes.io/projected/e3f37774-81cc-4b2c-94cb-ec77935ec46d-kube-api-access-2qxnz\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.037779 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.037752 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:24:59.138371 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.138336 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.138538 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.138381 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qxnz\" (UniqueName: \"kubernetes.io/projected/e3f37774-81cc-4b2c-94cb-ec77935ec46d-kube-api-access-2qxnz\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.138538 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.138412 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:24:59.138538 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.138464 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f37774-81cc-4b2c-94cb-ec77935ec46d-config-volume\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.138538 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.138483 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jtkvc\" (UniqueName: \"kubernetes.io/projected/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-kube-api-access-jtkvc\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:24:59.138538 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.138503 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:24:59.138834 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.138561 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:24:59.138834 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.138595 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:24:59.638572362 +0000 UTC m=+33.568032507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:24:59.138834 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.138647 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:24:59.638612803 +0000 UTC m=+33.568072947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:24:59.138834 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.138514 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3f37774-81cc-4b2c-94cb-ec77935ec46d-tmp-dir\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.139101 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.138856 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3f37774-81cc-4b2c-94cb-ec77935ec46d-tmp-dir\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.139101 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.139093 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f37774-81cc-4b2c-94cb-ec77935ec46d-config-volume\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.149476 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.149451 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qxnz\" (UniqueName: \"kubernetes.io/projected/e3f37774-81cc-4b2c-94cb-ec77935ec46d-kube-api-access-2qxnz\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.149617 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.149547 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtkvc\" (UniqueName: \"kubernetes.io/projected/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-kube-api-access-jtkvc\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:24:59.339954 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.339919 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:59.340164 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.340069 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 19 15:24:59.340164 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.340088 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 19 15:24:59.340164 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.340097 2576 projected.go:194] Error preparing data for projected volume kube-api-access-lh7j5 for pod openshift-network-diagnostics/network-check-target-w2fz9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:59.340164 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.340164 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5 podName:2e551954-ed8a-49b4-8237-75b061234272 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:31.34014801 +0000 UTC m=+65.269608151 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lh7j5" (UniqueName: "kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5") pod "network-check-target-w2fz9" (UID: "2e551954-ed8a-49b4-8237-75b061234272") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 19 15:24:59.441096 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.441057 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:24:59.441265 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.441201 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:59.441322 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.441268 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:25:31.441247027 +0000 UTC m=+65.370707177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 19 15:24:59.643116 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.643023 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:24:59.643280 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.643142 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:24:59.643280 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.643214 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:24:59.643280 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.643235 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:24:59.643395 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.643319 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:00.64329494 +0000 UTC m=+34.572755102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:24:59.643395 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:24:59.643341 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:25:00.643330547 +0000 UTC m=+34.572790695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:24:59.735336 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.735296 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:24:59.738432 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.738405 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 19 15:24:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.738439 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2zbg\"" Apr 19 15:24:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:24:59.738441 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 19 15:25:00.650906 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:00.650845 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:25:00.651117 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:00.650925 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:25:00.651117 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:00.651004 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:25:00.651117 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:00.651034 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:25:00.651117 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:00.651074 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:25:02.651058052 +0000 UTC m=+36.580518192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:25:00.651117 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:00.651103 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:02.651096858 +0000 UTC m=+36.580556998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:25:00.735783 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:00.735704 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:25:00.738541 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:00.738518 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 19 15:25:00.738681 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:00.738568 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zjn96\"" Apr 19 15:25:01.009424 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:01.009338 2576 generic.go:358] "Generic (PLEG): container finished" podID="f8668eb5-0f25-4267-b970-27cee041914b" containerID="859cba3c9b42bf5dcc4daea5b73f0c16adee9300e901d694805a3631787b3492" exitCode=0 Apr 19 15:25:01.009424 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:01.009386 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerDied","Data":"859cba3c9b42bf5dcc4daea5b73f0c16adee9300e901d694805a3631787b3492"} Apr 19 15:25:01.935190 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:01.935157 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-2zptl"] Apr 19 15:25:01.948575 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:01.948392 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zptl"] Apr 19 15:25:01.948741 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:01.948577 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:01.951093 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:01.951069 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 19 15:25:02.013900 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.013872 2576 generic.go:358] "Generic (PLEG): container finished" podID="f8668eb5-0f25-4267-b970-27cee041914b" containerID="ff14ad1a86181dbe5883b408970370c65c3bba9ee84945cdc2562a379aaad7fa" exitCode=0 Apr 19 15:25:02.014063 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.013934 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerDied","Data":"ff14ad1a86181dbe5883b408970370c65c3bba9ee84945cdc2562a379aaad7fa"} Apr 19 15:25:02.061904 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.061872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/98cebdc7-b692-4937-9ff6-006fa39429e6-dbus\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.062076 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.061936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98cebdc7-b692-4937-9ff6-006fa39429e6-original-pull-secret\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.062076 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.062065 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/98cebdc7-b692-4937-9ff6-006fa39429e6-kubelet-config\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.162818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.162785 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/98cebdc7-b692-4937-9ff6-006fa39429e6-dbus\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.162818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.162823 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98cebdc7-b692-4937-9ff6-006fa39429e6-original-pull-secret\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.163033 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.162926 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/98cebdc7-b692-4937-9ff6-006fa39429e6-kubelet-config\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.163033 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.163006 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/98cebdc7-b692-4937-9ff6-006fa39429e6-kubelet-config\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.163140 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.163084 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/98cebdc7-b692-4937-9ff6-006fa39429e6-dbus\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.166112 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.166089 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/98cebdc7-b692-4937-9ff6-006fa39429e6-original-pull-secret\") pod \"global-pull-secret-syncer-2zptl\" (UID: \"98cebdc7-b692-4937-9ff6-006fa39429e6\") " pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.257461 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.257388 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2zptl" Apr 19 15:25:02.401925 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.401894 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2zptl"] Apr 19 15:25:02.405533 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:25:02.405503 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98cebdc7_b692_4937_9ff6_006fa39429e6.slice/crio-ab0fb6f3c5c6d16691fcbfeeeb8d9375008796c8065247c05bfcb87dbfff7492 WatchSource:0}: Error finding container ab0fb6f3c5c6d16691fcbfeeeb8d9375008796c8065247c05bfcb87dbfff7492: Status 404 returned error can't find the container with id ab0fb6f3c5c6d16691fcbfeeeb8d9375008796c8065247c05bfcb87dbfff7492 Apr 19 15:25:02.666555 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.666518 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:25:02.666555 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:02.666561 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:25:02.666836 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:02.666693 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:25:02.666836 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:02.666726 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:25:02.666836 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:02.666744 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:06.666731328 +0000 UTC m=+40.596191472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:25:02.666836 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:02.666793 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:25:06.666773572 +0000 UTC m=+40.596233733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:25:03.016832 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:03.016751 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zptl" event={"ID":"98cebdc7-b692-4937-9ff6-006fa39429e6","Type":"ContainerStarted","Data":"ab0fb6f3c5c6d16691fcbfeeeb8d9375008796c8065247c05bfcb87dbfff7492"} Apr 19 15:25:03.019477 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:03.019452 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" event={"ID":"f8668eb5-0f25-4267-b970-27cee041914b","Type":"ContainerStarted","Data":"f935201a0aa1f2c80d3bd28f8683c156e787d6b743633c8757bbda9d05ef031a"} Apr 19 15:25:03.042157 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:03.042093 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bcjgt" podStartSLOduration=4.615979822 podStartE2EDuration="37.042070381s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:24:27.739256631 +0000 UTC m=+1.668716771" lastFinishedPulling="2026-04-19 15:25:00.165347183 +0000 UTC m=+34.094807330" observedRunningTime="2026-04-19 15:25:03.040084081 +0000 UTC m=+36.969544242" watchObservedRunningTime="2026-04-19 15:25:03.042070381 +0000 UTC m=+36.971530547" Apr 19 15:25:06.695620 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:06.695409 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:25:06.695620 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:06.695542 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:25:06.696160 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:06.695708 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:25:14.695685235 +0000 UTC m=+48.625145386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:25:06.696160 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:06.695999 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:25:06.696160 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:06.696146 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:25:06.696333 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:06.696204 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:14.69618825 +0000 UTC m=+48.625648393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:25:07.027840 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:07.027756 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2zptl" event={"ID":"98cebdc7-b692-4937-9ff6-006fa39429e6","Type":"ContainerStarted","Data":"5f4b0a48efa7db895c5d557c1838ffbaf4e8c9b76240d72df2094a2725bd5cc8"} Apr 19 15:25:14.749753 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:14.749708 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:25:14.749753 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:14.749754 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:25:14.750184 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:14.749859 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:25:14.750184 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:14.749916 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:25:14.750184 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:14.749926 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:25:30.749910361 +0000 UTC m=+64.679370500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:25:14.750184 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:14.749962 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:30.749949281 +0000 UTC m=+64.679409421 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:25:25.007416 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:25.007388 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kqc9s" Apr 19 15:25:25.046591 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:25.046537 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2zptl" podStartSLOduration=19.850885243 podStartE2EDuration="24.046520009s" podCreationTimestamp="2026-04-19 15:25:01 +0000 UTC" firstStartedPulling="2026-04-19 15:25:02.408882927 +0000 UTC m=+36.338343066" lastFinishedPulling="2026-04-19 15:25:06.604517687 +0000 UTC m=+40.533977832" observedRunningTime="2026-04-19 15:25:07.041378787 +0000 UTC m=+40.970838954" watchObservedRunningTime="2026-04-19 15:25:25.046520009 +0000 UTC m=+58.975980164" Apr 19 15:25:30.759306 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:30.759267 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:25:30.759926 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:30.759411 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:25:30.759926 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:30.759471 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:26:02.759455538 +0000 UTC m=+96.688915678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:25:30.759926 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:30.759470 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:25:30.759926 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:30.759870 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:25:30.760159 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:30.759941 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:26:02.759924216 +0000 UTC m=+96.689384358 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:25:31.363446 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.363390 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:25:31.366453 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.366435 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 19 15:25:31.376394 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.376374 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 19 15:25:31.387319 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.387299 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh7j5\" (UniqueName: \"kubernetes.io/projected/2e551954-ed8a-49b4-8237-75b061234272-kube-api-access-lh7j5\") pod \"network-check-target-w2fz9\" (UID: \"2e551954-ed8a-49b4-8237-75b061234272\") " pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:25:31.464142 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.464105 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:25:31.467059 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.467034 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 19 15:25:31.474940 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:31.474919 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 19 15:25:31.475027 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:31.475007 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs podName:7aeaccc6-af60-4b21-9803-f56daf2f7ddb nodeName:}" failed. No retries permitted until 2026-04-19 15:26:35.47498501 +0000 UTC m=+129.404445149 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs") pod "network-metrics-daemon-r7nm5" (UID: "7aeaccc6-af60-4b21-9803-f56daf2f7ddb") : secret "metrics-daemon-secret" not found Apr 19 15:25:31.549614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.549580 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2zbg\"" Apr 19 15:25:31.557563 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.557542 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:25:31.685151 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:31.685116 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w2fz9"] Apr 19 15:25:31.690577 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:25:31.689619 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e551954_ed8a_49b4_8237_75b061234272.slice/crio-2624571f948ad793eadb7bdb67438fb305dec3c5fdf6dc7c3ec0a7e0ce26ac98 WatchSource:0}: Error finding container 2624571f948ad793eadb7bdb67438fb305dec3c5fdf6dc7c3ec0a7e0ce26ac98: Status 404 returned error can't find the container with id 2624571f948ad793eadb7bdb67438fb305dec3c5fdf6dc7c3ec0a7e0ce26ac98 Apr 19 15:25:32.075099 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:32.075018 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w2fz9" event={"ID":"2e551954-ed8a-49b4-8237-75b061234272","Type":"ContainerStarted","Data":"2624571f948ad793eadb7bdb67438fb305dec3c5fdf6dc7c3ec0a7e0ce26ac98"} Apr 19 15:25:35.083885 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:35.083844 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w2fz9" event={"ID":"2e551954-ed8a-49b4-8237-75b061234272","Type":"ContainerStarted","Data":"8d80909862ce406fb1687666362b195db83f686655c4a3abaec2c36356ecafdb"} Apr 19 15:25:35.084283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:35.084060 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:25:35.098698 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:35.098650 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-w2fz9" podStartSLOduration=66.383675857 podStartE2EDuration="1m9.098619823s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:25:31.693069118 +0000 UTC m=+65.622529262" lastFinishedPulling="2026-04-19 15:25:34.408013082 +0000 UTC m=+68.337473228" observedRunningTime="2026-04-19 15:25:35.098198698 +0000 UTC m=+69.027658861" watchObservedRunningTime="2026-04-19 15:25:35.098619823 +0000 UTC m=+69.028079985" Apr 19 15:25:51.924033 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:51.923906 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng"] Apr 19 15:25:51.928087 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:51.928068 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" Apr 19 15:25:51.930599 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:51.930578 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Apr 19 15:25:51.930599 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:51.930588 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Apr 19 15:25:51.930793 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:51.930663 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mx542\"" Apr 19 15:25:51.933920 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:51.933895 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng"] Apr 19 15:25:52.003117 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.003089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q5dp\" (UniqueName: \"kubernetes.io/projected/1dffa790-7318-45b1-8c6f-981031b70840-kube-api-access-7q5dp\") pod \"volume-data-source-validator-7c6cbb6c87-f6xng\" (UID: \"1dffa790-7318-45b1-8c6f-981031b70840\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" Apr 19 15:25:52.052859 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.052829 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-d7c7c5446-wbdgz"] Apr 19 15:25:52.055680 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.055666 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.058084 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.058063 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Apr 19 15:25:52.058422 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.058401 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Apr 19 15:25:52.058613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.058597 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Apr 19 15:25:52.058799 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.058785 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Apr 19 15:25:52.058862 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.058818 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Apr 19 15:25:52.059133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.059114 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Apr 19 15:25:52.059202 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.059188 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n4wfg\"" Apr 19 15:25:52.069258 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.069239 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d7c7c5446-wbdgz"] Apr 19 15:25:52.103542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.103519 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.103650 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.103558 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7q5dp\" (UniqueName: \"kubernetes.io/projected/1dffa790-7318-45b1-8c6f-981031b70840-kube-api-access-7q5dp\") pod \"volume-data-source-validator-7c6cbb6c87-f6xng\" (UID: \"1dffa790-7318-45b1-8c6f-981031b70840\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" Apr 19 15:25:52.103728 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.103645 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-default-certificate\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.103728 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.103675 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.103833 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.103725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-stats-auth\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.103833 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.103764 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btcj4\" (UniqueName: \"kubernetes.io/projected/41b0c1cd-70f7-47aa-b30e-0368722019e6-kube-api-access-btcj4\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.110691 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.110673 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q5dp\" (UniqueName: \"kubernetes.io/projected/1dffa790-7318-45b1-8c6f-981031b70840-kube-api-access-7q5dp\") pod \"volume-data-source-validator-7c6cbb6c87-f6xng\" (UID: \"1dffa790-7318-45b1-8c6f-981031b70840\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" Apr 19 15:25:52.135513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.135491 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2"] Apr 19 15:25:52.138507 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.138307 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.139605 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.139586 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-qjpxz"] Apr 19 15:25:52.142008 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.141993 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fjltl\"" Apr 19 15:25:52.142169 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.142151 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Apr 19 15:25:52.142169 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.142166 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.142654 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.142616 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Apr 19 15:25:52.142788 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.142772 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Apr 19 15:25:52.142979 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.142966 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Apr 19 15:25:52.144528 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.144511 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Apr 19 15:25:52.145044 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.144827 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Apr 19 15:25:52.145044 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.144868 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Apr 19 15:25:52.145044 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.144886 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Apr 19 15:25:52.145044 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.144886 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jwtls\"" Apr 19 15:25:52.150394 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.150375 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Apr 19 15:25:52.150836 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.150819 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2"] Apr 19 15:25:52.157091 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.157072 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-qjpxz"] Apr 19 15:25:52.205121 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205062 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p28th\" (UniqueName: \"kubernetes.io/projected/142a272e-7901-47e5-a0d4-f9e2be29d84a-kube-api-access-p28th\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.205121 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205094 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f15cf98-a60c-41aa-96e8-be8466dd8e26-trusted-ca\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.205261 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205128 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-default-certificate\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.205261 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205158 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.205261 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/142a272e-7901-47e5-a0d4-f9e2be29d84a-serving-cert\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.205261 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205215 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/142a272e-7901-47e5-a0d4-f9e2be29d84a-config\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.205503 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205258 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-stats-auth\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.205503 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205284 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-btcj4\" (UniqueName: \"kubernetes.io/projected/41b0c1cd-70f7-47aa-b30e-0368722019e6-kube-api-access-btcj4\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.205503 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:52.205265 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 19 15:25:52.205503 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205316 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7k4s\" (UniqueName: \"kubernetes.io/projected/8f15cf98-a60c-41aa-96e8-be8466dd8e26-kube-api-access-d7k4s\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.205503 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f15cf98-a60c-41aa-96e8-be8466dd8e26-serving-cert\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.205503 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:52.205377 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:52.705355939 +0000 UTC m=+86.634816084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : secret "router-metrics-certs-default" not found Apr 19 15:25:52.205503 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205485 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.205821 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.205531 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f15cf98-a60c-41aa-96e8-be8466dd8e26-config\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.205821 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:52.205606 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:52.705592819 +0000 UTC m=+86.635052977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : configmap references non-existent config key: service-ca.crt Apr 19 15:25:52.207564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.207545 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-default-certificate\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.207658 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.207567 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-stats-auth\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.215740 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.215724 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btcj4\" (UniqueName: \"kubernetes.io/projected/41b0c1cd-70f7-47aa-b30e-0368722019e6-kube-api-access-btcj4\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.237425 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.237396 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" Apr 19 15:25:52.306180 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.305968 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/142a272e-7901-47e5-a0d4-f9e2be29d84a-serving-cert\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.306180 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.306015 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/142a272e-7901-47e5-a0d4-f9e2be29d84a-config\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.306180 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.306051 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d7k4s\" (UniqueName: \"kubernetes.io/projected/8f15cf98-a60c-41aa-96e8-be8466dd8e26-kube-api-access-d7k4s\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.306180 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.306081 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f15cf98-a60c-41aa-96e8-be8466dd8e26-serving-cert\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.306415 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.306233 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f15cf98-a60c-41aa-96e8-be8466dd8e26-config\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.306415 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.306287 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p28th\" (UniqueName: \"kubernetes.io/projected/142a272e-7901-47e5-a0d4-f9e2be29d84a-kube-api-access-p28th\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.306415 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.306303 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f15cf98-a60c-41aa-96e8-be8466dd8e26-trusted-ca\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.307326 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.307285 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/142a272e-7901-47e5-a0d4-f9e2be29d84a-config\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.307507 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.307475 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f15cf98-a60c-41aa-96e8-be8466dd8e26-config\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.307692 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.307589 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8f15cf98-a60c-41aa-96e8-be8466dd8e26-trusted-ca\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.308844 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.308812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/142a272e-7901-47e5-a0d4-f9e2be29d84a-serving-cert\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.310167 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.310121 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f15cf98-a60c-41aa-96e8-be8466dd8e26-serving-cert\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.314284 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.314255 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p28th\" (UniqueName: \"kubernetes.io/projected/142a272e-7901-47e5-a0d4-f9e2be29d84a-kube-api-access-p28th\") pod \"service-ca-operator-d6fc45fc5-5j2z2\" (UID: \"142a272e-7901-47e5-a0d4-f9e2be29d84a\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.314412 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.314386 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7k4s\" (UniqueName: \"kubernetes.io/projected/8f15cf98-a60c-41aa-96e8-be8466dd8e26-kube-api-access-d7k4s\") pod \"console-operator-9d4b6777b-qjpxz\" (UID: \"8f15cf98-a60c-41aa-96e8-be8466dd8e26\") " pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.349419 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.349395 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng"] Apr 19 15:25:52.349558 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:25:52.349540 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dffa790_7318_45b1_8c6f_981031b70840.slice/crio-5d03dc6eefec8110ca705c7bc2ec84d56047b7cb7ed735991246334d3f4ebde8 WatchSource:0}: Error finding container 5d03dc6eefec8110ca705c7bc2ec84d56047b7cb7ed735991246334d3f4ebde8: Status 404 returned error can't find the container with id 5d03dc6eefec8110ca705c7bc2ec84d56047b7cb7ed735991246334d3f4ebde8 Apr 19 15:25:52.448652 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.448580 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" Apr 19 15:25:52.454363 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.454339 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:25:52.568595 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.568566 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2"] Apr 19 15:25:52.571431 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:25:52.571405 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod142a272e_7901_47e5_a0d4_f9e2be29d84a.slice/crio-4cb0441afa3bb25541eb87fdd8dd9d16ded51f1d7a06aebcb85ecec7b27eed07 WatchSource:0}: Error finding container 4cb0441afa3bb25541eb87fdd8dd9d16ded51f1d7a06aebcb85ecec7b27eed07: Status 404 returned error can't find the container with id 4cb0441afa3bb25541eb87fdd8dd9d16ded51f1d7a06aebcb85ecec7b27eed07 Apr 19 15:25:52.584723 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.584696 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-qjpxz"] Apr 19 15:25:52.588489 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:25:52.588466 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f15cf98_a60c_41aa_96e8_be8466dd8e26.slice/crio-45a1616c58b7143805c7043b04532a5e611976711e9c5a85f39188bbe210045f WatchSource:0}: Error finding container 45a1616c58b7143805c7043b04532a5e611976711e9c5a85f39188bbe210045f: Status 404 returned error can't find the container with id 45a1616c58b7143805c7043b04532a5e611976711e9c5a85f39188bbe210045f Apr 19 15:25:52.710116 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.710045 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.710116 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:52.710112 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:52.710257 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:52.710207 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 19 15:25:52.710257 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:52.710213 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:53.710194103 +0000 UTC m=+87.639654248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : configmap references non-existent config key: service-ca.crt Apr 19 15:25:52.710257 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:52.710247 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:53.710235278 +0000 UTC m=+87.639695417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : secret "router-metrics-certs-default" not found Apr 19 15:25:53.118910 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:53.118875 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" event={"ID":"142a272e-7901-47e5-a0d4-f9e2be29d84a","Type":"ContainerStarted","Data":"4cb0441afa3bb25541eb87fdd8dd9d16ded51f1d7a06aebcb85ecec7b27eed07"} Apr 19 15:25:53.119961 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:53.119935 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" event={"ID":"8f15cf98-a60c-41aa-96e8-be8466dd8e26","Type":"ContainerStarted","Data":"45a1616c58b7143805c7043b04532a5e611976711e9c5a85f39188bbe210045f"} Apr 19 15:25:53.120908 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:53.120882 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" event={"ID":"1dffa790-7318-45b1-8c6f-981031b70840","Type":"ContainerStarted","Data":"5d03dc6eefec8110ca705c7bc2ec84d56047b7cb7ed735991246334d3f4ebde8"} Apr 19 15:25:53.719453 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:53.719406 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:53.719660 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:53.719614 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:55.719592436 +0000 UTC m=+89.649052591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : configmap references non-existent config key: service-ca.crt Apr 19 15:25:53.719754 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:53.719668 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:53.719846 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:53.719824 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 19 15:25:53.719911 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:53.719901 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:55.71988077 +0000 UTC m=+89.649340912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : secret "router-metrics-certs-default" not found Apr 19 15:25:54.124537 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:54.124505 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" event={"ID":"1dffa790-7318-45b1-8c6f-981031b70840","Type":"ContainerStarted","Data":"08eb32f5a806b88a3a2fab3519a5cf9cf1b5f6356601ba5e615a4fa9ce92b94e"} Apr 19 15:25:54.140557 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:54.140498 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-f6xng" podStartSLOduration=1.5941756759999999 podStartE2EDuration="3.140477743s" podCreationTimestamp="2026-04-19 15:25:51 +0000 UTC" firstStartedPulling="2026-04-19 15:25:52.351218046 +0000 UTC m=+86.280678190" lastFinishedPulling="2026-04-19 15:25:53.897520117 +0000 UTC m=+87.826980257" observedRunningTime="2026-04-19 15:25:54.138689263 +0000 UTC m=+88.068149426" watchObservedRunningTime="2026-04-19 15:25:54.140477743 +0000 UTC m=+88.069937905" Apr 19 15:25:55.738648 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:55.738593 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:55.739060 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:55.738691 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:55.739060 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:55.738767 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:59.738747444 +0000 UTC m=+93.668207601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : configmap references non-existent config key: service-ca.crt Apr 19 15:25:55.739060 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:55.738798 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 19 15:25:55.739060 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:55.738843 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:25:59.738829161 +0000 UTC m=+93.668289300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : secret "router-metrics-certs-default" not found Apr 19 15:25:56.130418 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.130372 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" event={"ID":"142a272e-7901-47e5-a0d4-f9e2be29d84a","Type":"ContainerStarted","Data":"20a3f703548ec75c1dea7ee80c3de2b9b3247c03e0db8034cc44818018c45114"} Apr 19 15:25:56.131818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.131795 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/0.log" Apr 19 15:25:56.131939 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.131839 2576 generic.go:358] "Generic (PLEG): container finished" podID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" containerID="00f59a72b40d9554fd1654ff3a1d5f947de27843b1273d9efbff21e4b22df210" exitCode=255 Apr 19 15:25:56.131939 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.131885 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" event={"ID":"8f15cf98-a60c-41aa-96e8-be8466dd8e26","Type":"ContainerDied","Data":"00f59a72b40d9554fd1654ff3a1d5f947de27843b1273d9efbff21e4b22df210"} Apr 19 15:25:56.132095 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.132080 2576 scope.go:117] "RemoveContainer" containerID="00f59a72b40d9554fd1654ff3a1d5f947de27843b1273d9efbff21e4b22df210" Apr 19 15:25:56.147076 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.147032 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" podStartSLOduration=1.586740129 podStartE2EDuration="4.147019434s" podCreationTimestamp="2026-04-19 15:25:52 +0000 UTC" firstStartedPulling="2026-04-19 15:25:52.573449676 +0000 UTC m=+86.502909816" lastFinishedPulling="2026-04-19 15:25:55.133728974 +0000 UTC m=+89.063189121" observedRunningTime="2026-04-19 15:25:56.145887754 +0000 UTC m=+90.075347915" watchObservedRunningTime="2026-04-19 15:25:56.147019434 +0000 UTC m=+90.076479595" Apr 19 15:25:56.406183 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.403794 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd"] Apr 19 15:25:56.407729 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.407709 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" Apr 19 15:25:56.410270 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.410238 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Apr 19 15:25:56.410270 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.410257 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qp642\"" Apr 19 15:25:56.410464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.410257 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Apr 19 15:25:56.415328 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.415292 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd"] Apr 19 15:25:56.444468 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.444434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6k5t\" (UniqueName: \"kubernetes.io/projected/5a1bd774-11fe-4781-88a0-85b33882c3f5-kube-api-access-l6k5t\") pod \"migrator-74bb7799d9-z5vbd\" (UID: \"5a1bd774-11fe-4781-88a0-85b33882c3f5\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" Apr 19 15:25:56.545236 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.545195 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6k5t\" (UniqueName: \"kubernetes.io/projected/5a1bd774-11fe-4781-88a0-85b33882c3f5-kube-api-access-l6k5t\") pod \"migrator-74bb7799d9-z5vbd\" (UID: \"5a1bd774-11fe-4781-88a0-85b33882c3f5\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" Apr 19 15:25:56.553156 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.553127 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6k5t\" (UniqueName: \"kubernetes.io/projected/5a1bd774-11fe-4781-88a0-85b33882c3f5-kube-api-access-l6k5t\") pod \"migrator-74bb7799d9-z5vbd\" (UID: \"5a1bd774-11fe-4781-88a0-85b33882c3f5\") " pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" Apr 19 15:25:56.718138 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.718051 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" Apr 19 15:25:56.838479 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:56.838449 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd"] Apr 19 15:25:56.841355 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:25:56.841325 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1bd774_11fe_4781_88a0_85b33882c3f5.slice/crio-0e80a92343730297b8243211541e28715ecf998d47d66d34046c302ca68c7e9d WatchSource:0}: Error finding container 0e80a92343730297b8243211541e28715ecf998d47d66d34046c302ca68c7e9d: Status 404 returned error can't find the container with id 0e80a92343730297b8243211541e28715ecf998d47d66d34046c302ca68c7e9d Apr 19 15:25:57.135659 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:57.135608 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" event={"ID":"5a1bd774-11fe-4781-88a0-85b33882c3f5","Type":"ContainerStarted","Data":"0e80a92343730297b8243211541e28715ecf998d47d66d34046c302ca68c7e9d"} Apr 19 15:25:57.136977 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:57.136956 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/1.log" Apr 19 15:25:57.137363 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:57.137351 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/0.log" Apr 19 15:25:57.137412 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:57.137381 2576 generic.go:358] "Generic (PLEG): container finished" podID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" containerID="fe30d421ab51320e131620741d65f0051a12209688935de4640e8869edd393fa" exitCode=255 Apr 19 15:25:57.137496 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:57.137476 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" event={"ID":"8f15cf98-a60c-41aa-96e8-be8466dd8e26","Type":"ContainerDied","Data":"fe30d421ab51320e131620741d65f0051a12209688935de4640e8869edd393fa"} Apr 19 15:25:57.137535 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:57.137520 2576 scope.go:117] "RemoveContainer" containerID="00f59a72b40d9554fd1654ff3a1d5f947de27843b1273d9efbff21e4b22df210" Apr 19 15:25:57.137789 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:57.137766 2576 scope.go:117] "RemoveContainer" containerID="fe30d421ab51320e131620741d65f0051a12209688935de4640e8869edd393fa" Apr 19 15:25:57.137994 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:57.137977 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-qjpxz_openshift-console-operator(8f15cf98-a60c-41aa-96e8-be8466dd8e26)\"" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" podUID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" Apr 19 15:25:58.140602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:58.140578 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/1.log" Apr 19 15:25:58.141049 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:58.141030 2576 scope.go:117] "RemoveContainer" containerID="fe30d421ab51320e131620741d65f0051a12209688935de4640e8869edd393fa" Apr 19 15:25:58.141272 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:58.141250 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-qjpxz_openshift-console-operator(8f15cf98-a60c-41aa-96e8-be8466dd8e26)\"" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" podUID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" Apr 19 15:25:59.144602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.144554 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" event={"ID":"5a1bd774-11fe-4781-88a0-85b33882c3f5","Type":"ContainerStarted","Data":"d3ef5aa5721e4a8635faaf662f7f38c4a63dbfa0035ace9bd437cd935346b043"} Apr 19 15:25:59.144602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.144602 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" event={"ID":"5a1bd774-11fe-4781-88a0-85b33882c3f5","Type":"ContainerStarted","Data":"f2ecb12875b0b090ddaa1d068e8a472748b076b5073ad7c20d144fd85eece1cb"} Apr 19 15:25:59.159635 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.159587 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-74bb7799d9-z5vbd" podStartSLOduration=1.7313565720000001 podStartE2EDuration="3.159573916s" podCreationTimestamp="2026-04-19 15:25:56 +0000 UTC" firstStartedPulling="2026-04-19 15:25:56.843060069 +0000 UTC m=+90.772520214" lastFinishedPulling="2026-04-19 15:25:58.271277415 +0000 UTC m=+92.200737558" observedRunningTime="2026-04-19 15:25:59.158101693 +0000 UTC m=+93.087561855" watchObservedRunningTime="2026-04-19 15:25:59.159573916 +0000 UTC m=+93.089034102" Apr 19 15:25:59.226105 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.226075 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-865cb79987-qclq7"] Apr 19 15:25:59.229135 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.229109 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.231676 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.231653 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Apr 19 15:25:59.231830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.231653 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-82jjz\"" Apr 19 15:25:59.231830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.231653 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Apr 19 15:25:59.231999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.231675 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Apr 19 15:25:59.231999 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.231699 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Apr 19 15:25:59.236204 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.236185 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-qclq7"] Apr 19 15:25:59.271983 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.271951 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-signing-key\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.272123 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.272009 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvfn8\" (UniqueName: \"kubernetes.io/projected/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-kube-api-access-tvfn8\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.272174 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.272139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-signing-cabundle\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.372752 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.372721 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-signing-cabundle\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.372927 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.372769 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-signing-key\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.373018 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.372984 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tvfn8\" (UniqueName: \"kubernetes.io/projected/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-kube-api-access-tvfn8\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.373875 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.373854 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-signing-cabundle\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.375150 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.375130 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-signing-key\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.380922 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.380900 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvfn8\" (UniqueName: \"kubernetes.io/projected/b8fd1dc1-4fc4-48b0-8e9f-e640cae82430-kube-api-access-tvfn8\") pod \"service-ca-865cb79987-qclq7\" (UID: \"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430\") " pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.539266 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.539180 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-qclq7" Apr 19 15:25:59.649676 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.649647 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-qclq7"] Apr 19 15:25:59.652944 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:25:59.652910 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fd1dc1_4fc4_48b0_8e9f_e640cae82430.slice/crio-772dc357ba827fd5373e0b6792006f0728f0425a7b4c007520b24350a30fc8a0 WatchSource:0}: Error finding container 772dc357ba827fd5373e0b6792006f0728f0425a7b4c007520b24350a30fc8a0: Status 404 returned error can't find the container with id 772dc357ba827fd5373e0b6792006f0728f0425a7b4c007520b24350a30fc8a0 Apr 19 15:25:59.776452 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.776417 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:59.776604 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:25:59.776481 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:25:59.776604 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:59.776579 2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Apr 19 15:25:59.776704 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:59.776607 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:26:07.776586316 +0000 UTC m=+101.706046459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : configmap references non-existent config key: service-ca.crt Apr 19 15:25:59.776704 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:25:59.776652 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:26:07.776618314 +0000 UTC m=+101.706078454 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : secret "router-metrics-certs-default" not found Apr 19 15:26:00.150765 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:00.150728 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-qclq7" event={"ID":"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430","Type":"ContainerStarted","Data":"c27f7285f209b4751909f4951db992beb36461f93e802897b79d35b7444aac31"} Apr 19 15:26:00.150765 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:00.150768 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-qclq7" event={"ID":"b8fd1dc1-4fc4-48b0-8e9f-e640cae82430","Type":"ContainerStarted","Data":"772dc357ba827fd5373e0b6792006f0728f0425a7b4c007520b24350a30fc8a0"} Apr 19 15:26:00.154903 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:00.154882 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rlhfq_2514ec8e-0d56-4eda-929e-7a8759df8f30/dns-node-resolver/0.log" Apr 19 15:26:00.166851 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:00.166811 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-865cb79987-qclq7" podStartSLOduration=1.166798225 podStartE2EDuration="1.166798225s" podCreationTimestamp="2026-04-19 15:25:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-19 15:26:00.166200495 +0000 UTC m=+94.095660658" watchObservedRunningTime="2026-04-19 15:26:00.166798225 +0000 UTC m=+94.096258386" Apr 19 15:26:01.154203 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:01.154176 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fw4zx_7efc9272-6634-4b94-abe5-ba1e076ca391/node-ca/0.log" Apr 19 15:26:02.155590 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:02.155558 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-z5vbd_5a1bd774-11fe-4781-88a0-85b33882c3f5/migrator/0.log" Apr 19 15:26:02.354871 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:02.354842 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-z5vbd_5a1bd774-11fe-4781-88a0-85b33882c3f5/graceful-termination/0.log" Apr 19 15:26:02.455276 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:02.455183 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:26:02.455276 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:02.455233 2576 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:26:02.455729 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:02.455714 2576 scope.go:117] "RemoveContainer" containerID="fe30d421ab51320e131620741d65f0051a12209688935de4640e8869edd393fa" Apr 19 15:26:02.455944 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:02.455924 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-qjpxz_openshift-console-operator(8f15cf98-a60c-41aa-96e8-be8466dd8e26)\"" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" podUID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" Apr 19 15:26:02.802963 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:02.802873 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:26:02.802963 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:02.802915 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:26:02.803209 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:02.803052 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 19 15:26:02.803209 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:02.803067 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 19 15:26:02.803209 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:02.803104 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert podName:ea08d4af-7d5b-47ed-ab7f-2e997210a6b9 nodeName:}" failed. No retries permitted until 2026-04-19 15:27:06.803090781 +0000 UTC m=+160.732550921 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert") pod "ingress-canary-qcm6b" (UID: "ea08d4af-7d5b-47ed-ab7f-2e997210a6b9") : secret "canary-serving-cert" not found Apr 19 15:26:02.803209 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:02.803143 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls podName:e3f37774-81cc-4b2c-94cb-ec77935ec46d nodeName:}" failed. No retries permitted until 2026-04-19 15:27:06.803123838 +0000 UTC m=+160.732583982 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls") pod "dns-default-wh4pt" (UID: "e3f37774-81cc-4b2c-94cb-ec77935ec46d") : secret "dns-default-metrics-tls" not found Apr 19 15:26:06.088456 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:06.088426 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-w2fz9" Apr 19 15:26:07.848015 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:07.847966 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:07.848592 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:07.848065 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:07.848592 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:07.848171 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle podName:41b0c1cd-70f7-47aa-b30e-0368722019e6 nodeName:}" failed. No retries permitted until 2026-04-19 15:26:23.848153286 +0000 UTC m=+117.777613426 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle") pod "router-default-d7c7c5446-wbdgz" (UID: "41b0c1cd-70f7-47aa-b30e-0368722019e6") : configmap references non-existent config key: service-ca.crt Apr 19 15:26:07.850474 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:07.850448 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41b0c1cd-70f7-47aa-b30e-0368722019e6-metrics-certs\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:16.737480 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:16.737439 2576 scope.go:117] "RemoveContainer" containerID="fe30d421ab51320e131620741d65f0051a12209688935de4640e8869edd393fa" Apr 19 15:26:17.193690 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:17.193667 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:26:17.194041 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:17.194025 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/1.log" Apr 19 15:26:17.194098 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:17.194058 2576 generic.go:358] "Generic (PLEG): container finished" podID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" containerID="b425caae2d9483f4079d8ace2eab558155ffa94a1f07200067ac5d385b865d4c" exitCode=255 Apr 19 15:26:17.194098 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:17.194087 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" event={"ID":"8f15cf98-a60c-41aa-96e8-be8466dd8e26","Type":"ContainerDied","Data":"b425caae2d9483f4079d8ace2eab558155ffa94a1f07200067ac5d385b865d4c"} Apr 19 15:26:17.194165 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:17.194114 2576 scope.go:117] "RemoveContainer" containerID="fe30d421ab51320e131620741d65f0051a12209688935de4640e8869edd393fa" Apr 19 15:26:17.194458 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:17.194435 2576 scope.go:117] "RemoveContainer" containerID="b425caae2d9483f4079d8ace2eab558155ffa94a1f07200067ac5d385b865d4c" Apr 19 15:26:17.194609 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:17.194590 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-9d4b6777b-qjpxz_openshift-console-operator(8f15cf98-a60c-41aa-96e8-be8466dd8e26)\"" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" podUID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" Apr 19 15:26:18.197962 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:18.197932 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:26:22.431328 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.431290 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rsvd9"] Apr 19 15:26:22.434665 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.434643 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.438724 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.438696 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 19 15:26:22.438886 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.438744 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 19 15:26:22.440029 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.440004 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 19 15:26:22.440167 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.440044 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jvwp2\"" Apr 19 15:26:22.440327 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.440306 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 19 15:26:22.445810 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.445786 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rsvd9"] Apr 19 15:26:22.454880 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.454852 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:26:22.455003 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.454892 2576 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:26:22.455249 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.455236 2576 scope.go:117] "RemoveContainer" containerID="b425caae2d9483f4079d8ace2eab558155ffa94a1f07200067ac5d385b865d4c" Apr 19 15:26:22.455425 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:22.455408 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-9d4b6777b-qjpxz_openshift-console-operator(8f15cf98-a60c-41aa-96e8-be8466dd8e26)\"" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" podUID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" Apr 19 15:26:22.560592 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.560558 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/21a5dc61-a5e7-48cd-a756-316c82784b5b-data-volume\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.560831 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.560716 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/21a5dc61-a5e7-48cd-a756-316c82784b5b-crio-socket\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.560831 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.560753 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc6d2\" (UniqueName: \"kubernetes.io/projected/21a5dc61-a5e7-48cd-a756-316c82784b5b-kube-api-access-dc6d2\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.560952 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.560872 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/21a5dc61-a5e7-48cd-a756-316c82784b5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.560952 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.560921 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/21a5dc61-a5e7-48cd-a756-316c82784b5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662080 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662037 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/21a5dc61-a5e7-48cd-a756-316c82784b5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662080 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662077 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/21a5dc61-a5e7-48cd-a756-316c82784b5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662333 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662263 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/21a5dc61-a5e7-48cd-a756-316c82784b5b-data-volume\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662393 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662336 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/21a5dc61-a5e7-48cd-a756-316c82784b5b-crio-socket\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662393 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662373 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dc6d2\" (UniqueName: \"kubernetes.io/projected/21a5dc61-a5e7-48cd-a756-316c82784b5b-kube-api-access-dc6d2\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662478 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662407 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/21a5dc61-a5e7-48cd-a756-316c82784b5b-crio-socket\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662566 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662548 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/21a5dc61-a5e7-48cd-a756-316c82784b5b-data-volume\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.662602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.662582 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/21a5dc61-a5e7-48cd-a756-316c82784b5b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.664475 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.664450 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/21a5dc61-a5e7-48cd-a756-316c82784b5b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.673362 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.673342 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc6d2\" (UniqueName: \"kubernetes.io/projected/21a5dc61-a5e7-48cd-a756-316c82784b5b-kube-api-access-dc6d2\") pod \"insights-runtime-extractor-rsvd9\" (UID: \"21a5dc61-a5e7-48cd-a756-316c82784b5b\") " pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.743930 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.743836 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rsvd9" Apr 19 15:26:22.861925 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:22.861854 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rsvd9"] Apr 19 15:26:22.864512 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:22.864486 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a5dc61_a5e7_48cd_a756_316c82784b5b.slice/crio-59e83b5ffbef3115d94715558c0e7b9cf6f371e513b5aa8504aea48276230ca2 WatchSource:0}: Error finding container 59e83b5ffbef3115d94715558c0e7b9cf6f371e513b5aa8504aea48276230ca2: Status 404 returned error can't find the container with id 59e83b5ffbef3115d94715558c0e7b9cf6f371e513b5aa8504aea48276230ca2 Apr 19 15:26:23.211451 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:23.211416 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvd9" event={"ID":"21a5dc61-a5e7-48cd-a756-316c82784b5b","Type":"ContainerStarted","Data":"9849e142e710305aba2465039a51ec81d71399c34475502a691710df12676e43"} Apr 19 15:26:23.211451 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:23.211454 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvd9" event={"ID":"21a5dc61-a5e7-48cd-a756-316c82784b5b","Type":"ContainerStarted","Data":"59e83b5ffbef3115d94715558c0e7b9cf6f371e513b5aa8504aea48276230ca2"} Apr 19 15:26:23.872112 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:23.872061 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:23.872708 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:23.872689 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41b0c1cd-70f7-47aa-b30e-0368722019e6-service-ca-bundle\") pod \"router-default-d7c7c5446-wbdgz\" (UID: \"41b0c1cd-70f7-47aa-b30e-0368722019e6\") " pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:24.165138 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:24.165093 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:24.215831 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:24.215803 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvd9" event={"ID":"21a5dc61-a5e7-48cd-a756-316c82784b5b","Type":"ContainerStarted","Data":"ba9af51d90daf432c231992c832a5f6445112f2100bc2e101ae7e1a1c2de4eb9"} Apr 19 15:26:24.280727 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:24.280689 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-d7c7c5446-wbdgz"] Apr 19 15:26:24.286124 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:24.286092 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b0c1cd_70f7_47aa_b30e_0368722019e6.slice/crio-923987f4175c4a4f94761de697fa2bf3d8861872c4e1d357273a7cb55812b63e WatchSource:0}: Error finding container 923987f4175c4a4f94761de697fa2bf3d8861872c4e1d357273a7cb55812b63e: Status 404 returned error can't find the container with id 923987f4175c4a4f94761de697fa2bf3d8861872c4e1d357273a7cb55812b63e Apr 19 15:26:25.220118 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:25.220073 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" event={"ID":"41b0c1cd-70f7-47aa-b30e-0368722019e6","Type":"ContainerStarted","Data":"ffedecab274f88350d33d258668d14354b892930afcdfafcce192d0c3c2bbb65"} Apr 19 15:26:25.220118 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:25.220124 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" event={"ID":"41b0c1cd-70f7-47aa-b30e-0368722019e6","Type":"ContainerStarted","Data":"923987f4175c4a4f94761de697fa2bf3d8861872c4e1d357273a7cb55812b63e"} Apr 19 15:26:25.239961 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:25.239904 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" podStartSLOduration=33.239884721 podStartE2EDuration="33.239884721s" podCreationTimestamp="2026-04-19 15:25:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-19 15:26:25.239518126 +0000 UTC m=+119.168978289" watchObservedRunningTime="2026-04-19 15:26:25.239884721 +0000 UTC m=+119.169344884" Apr 19 15:26:26.166078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:26.166032 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:26.168682 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:26.168660 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:26.226313 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:26.226282 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvd9" event={"ID":"21a5dc61-a5e7-48cd-a756-316c82784b5b","Type":"ContainerStarted","Data":"bd56b2ce675bb9f2513eddd6d01198b6fb9bc25e527a200b54aba9da22131cda"} Apr 19 15:26:26.226725 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:26.226555 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:26.227740 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:26.227716 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-d7c7c5446-wbdgz" Apr 19 15:26:26.246488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:26.246429 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rsvd9" podStartSLOduration=1.6708197729999998 podStartE2EDuration="4.246412423s" podCreationTimestamp="2026-04-19 15:26:22 +0000 UTC" firstStartedPulling="2026-04-19 15:26:22.914150309 +0000 UTC m=+116.843610448" lastFinishedPulling="2026-04-19 15:26:25.489742949 +0000 UTC m=+119.419203098" observedRunningTime="2026-04-19 15:26:26.245619388 +0000 UTC m=+120.175079550" watchObservedRunningTime="2026-04-19 15:26:26.246412423 +0000 UTC m=+120.175872584" Apr 19 15:26:29.836497 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.836463 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5676c8c784-ksfzr"] Apr 19 15:26:29.841685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.841664 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:29.844259 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.844233 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 19 15:26:29.845512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.845486 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Apr 19 15:26:29.845613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.845487 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Apr 19 15:26:29.845613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.845541 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t6z29\"" Apr 19 15:26:29.845613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.845566 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 19 15:26:29.845613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.845575 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 19 15:26:29.848430 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.848273 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5676c8c784-ksfzr"] Apr 19 15:26:29.924549 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.924516 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgwbv\" (UniqueName: \"kubernetes.io/projected/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-kube-api-access-lgwbv\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:29.924763 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.924572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-metrics-client-ca\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:29.924763 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.924669 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:29.924878 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:29.924773 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-prometheus-operator-tls\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.025094 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.025061 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-metrics-client-ca\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.025210 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.025117 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.025210 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.025174 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-prometheus-operator-tls\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.025310 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.025219 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lgwbv\" (UniqueName: \"kubernetes.io/projected/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-kube-api-access-lgwbv\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.025783 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.025757 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-metrics-client-ca\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.027549 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.027532 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-prometheus-operator-tls\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.027652 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.027612 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.032780 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.032760 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgwbv\" (UniqueName: \"kubernetes.io/projected/184a1522-2ca4-48b2-be5c-2fe7ee5ec74c-kube-api-access-lgwbv\") pod \"prometheus-operator-5676c8c784-ksfzr\" (UID: \"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c\") " pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.150918 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.150801 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" Apr 19 15:26:30.284507 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:30.284475 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5676c8c784-ksfzr"] Apr 19 15:26:30.287144 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:30.287116 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184a1522_2ca4_48b2_be5c_2fe7ee5ec74c.slice/crio-dd0f7fab264159d6cf6eb2c90c62a84d9e236735b975cdbfe22583c4a63c3ad0 WatchSource:0}: Error finding container dd0f7fab264159d6cf6eb2c90c62a84d9e236735b975cdbfe22583c4a63c3ad0: Status 404 returned error can't find the container with id dd0f7fab264159d6cf6eb2c90c62a84d9e236735b975cdbfe22583c4a63c3ad0 Apr 19 15:26:31.240379 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:31.240345 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" event={"ID":"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c","Type":"ContainerStarted","Data":"dd0f7fab264159d6cf6eb2c90c62a84d9e236735b975cdbfe22583c4a63c3ad0"} Apr 19 15:26:32.244106 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:32.244071 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" event={"ID":"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c","Type":"ContainerStarted","Data":"f04d605e981d5d2274188dc44fb03281b5d300e31edfec588e8fffab8f484a6b"} Apr 19 15:26:32.244106 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:32.244107 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" event={"ID":"184a1522-2ca4-48b2-be5c-2fe7ee5ec74c","Type":"ContainerStarted","Data":"28a5541d89ec1643aec22ff08c7f551d992d778c2336d0b44153557dcc035cc7"} Apr 19 15:26:32.260725 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:32.260674 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5676c8c784-ksfzr" podStartSLOduration=1.7879043970000001 podStartE2EDuration="3.26065863s" podCreationTimestamp="2026-04-19 15:26:29 +0000 UTC" firstStartedPulling="2026-04-19 15:26:30.288957614 +0000 UTC m=+124.218417755" lastFinishedPulling="2026-04-19 15:26:31.761711848 +0000 UTC m=+125.691171988" observedRunningTime="2026-04-19 15:26:32.25920745 +0000 UTC m=+126.188667606" watchObservedRunningTime="2026-04-19 15:26:32.26065863 +0000 UTC m=+126.190118793" Apr 19 15:26:33.736009 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:33.735973 2576 scope.go:117] "RemoveContainer" containerID="b425caae2d9483f4079d8ace2eab558155ffa94a1f07200067ac5d385b865d4c" Apr 19 15:26:33.736395 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:33.736177 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-9d4b6777b-qjpxz_openshift-console-operator(8f15cf98-a60c-41aa-96e8-be8466dd8e26)\"" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" podUID="8f15cf98-a60c-41aa-96e8-be8466dd8e26" Apr 19 15:26:34.174206 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.174171 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-frtdc"] Apr 19 15:26:34.177943 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.177922 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.180317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.180291 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 19 15:26:34.180539 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.180340 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 19 15:26:34.180748 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.180730 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 19 15:26:34.180833 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.180809 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s2772\"" Apr 19 15:26:34.262855 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.262821 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-sys\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263047 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.262906 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-root\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263047 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.262946 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8c0e9b7-a211-4e29-9a98-59a55972095d-metrics-client-ca\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263047 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.263005 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-textfile\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263047 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.263042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-wtmp\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263251 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.263078 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-tls\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263251 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.263103 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263251 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.263184 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-accelerators-collector-config\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.263251 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.263232 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmg9r\" (UniqueName: \"kubernetes.io/projected/a8c0e9b7-a211-4e29-9a98-59a55972095d-kube-api-access-hmg9r\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364409 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364369 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8c0e9b7-a211-4e29-9a98-59a55972095d-metrics-client-ca\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364608 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364439 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-textfile\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364608 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364482 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-wtmp\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364608 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364519 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-tls\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364608 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364545 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364856 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364704 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-wtmp\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364856 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364759 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-accelerators-collector-config\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364856 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364796 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hmg9r\" (UniqueName: \"kubernetes.io/projected/a8c0e9b7-a211-4e29-9a98-59a55972095d-kube-api-access-hmg9r\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.364856 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364824 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-sys\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.365056 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364895 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-root\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.365056 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364961 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-root\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.365162 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.365101 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a8c0e9b7-a211-4e29-9a98-59a55972095d-metrics-client-ca\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.365219 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:34.365210 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Apr 19 15:26:34.365274 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:34.365270 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-tls podName:a8c0e9b7-a211-4e29-9a98-59a55972095d nodeName:}" failed. No retries permitted until 2026-04-19 15:26:34.865248865 +0000 UTC m=+128.794709014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-tls") pod "node-exporter-frtdc" (UID: "a8c0e9b7-a211-4e29-9a98-59a55972095d") : secret "node-exporter-tls" not found Apr 19 15:26:34.365339 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.365270 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8c0e9b7-a211-4e29-9a98-59a55972095d-sys\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.365339 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.365275 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-accelerators-collector-config\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.365339 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.364824 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-textfile\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.367343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.367315 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.373478 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.373458 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmg9r\" (UniqueName: \"kubernetes.io/projected/a8c0e9b7-a211-4e29-9a98-59a55972095d-kube-api-access-hmg9r\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.868149 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.868111 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-tls\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:34.870421 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:34.870392 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a8c0e9b7-a211-4e29-9a98-59a55972095d-node-exporter-tls\") pod \"node-exporter-frtdc\" (UID: \"a8c0e9b7-a211-4e29-9a98-59a55972095d\") " pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:35.087814 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.087763 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-frtdc" Apr 19 15:26:35.095638 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:35.095599 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c0e9b7_a211_4e29_9a98_59a55972095d.slice/crio-108bbe72c1d4d431558d83f5278e80c768fedc7194baa30c4a79c08f801e6cd2 WatchSource:0}: Error finding container 108bbe72c1d4d431558d83f5278e80c768fedc7194baa30c4a79c08f801e6cd2: Status 404 returned error can't find the container with id 108bbe72c1d4d431558d83f5278e80c768fedc7194baa30c4a79c08f801e6cd2 Apr 19 15:26:35.234656 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.234577 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:26:35.241249 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.239506 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.242401 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.242239 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Apr 19 15:26:35.242401 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.242253 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rrckw\"" Apr 19 15:26:35.242401 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.242332 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Apr 19 15:26:35.242401 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.242379 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Apr 19 15:26:35.242752 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.242580 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Apr 19 15:26:35.242752 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.242675 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Apr 19 15:26:35.243612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.243482 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Apr 19 15:26:35.243612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.243499 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Apr 19 15:26:35.243612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.243540 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Apr 19 15:26:35.243612 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.243606 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Apr 19 15:26:35.252300 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.252275 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:26:35.253513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.253490 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-frtdc" event={"ID":"a8c0e9b7-a211-4e29-9a98-59a55972095d","Type":"ContainerStarted","Data":"108bbe72c1d4d431558d83f5278e80c768fedc7194baa30c4a79c08f801e6cd2"} Apr 19 15:26:35.270154 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270124 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-web-config\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270298 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270187 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j5bq\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-kube-api-access-8j5bq\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270298 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-tls-assets\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270298 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-out\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270298 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270275 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270300 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270339 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270363 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-volume\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270401 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270488 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270820 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270529 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.270820 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.270547 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371107 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371075 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371107 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371113 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371337 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371134 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371337 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371156 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-web-config\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371548 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371520 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8j5bq\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-kube-api-access-8j5bq\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371619 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371566 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-tls-assets\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371619 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371598 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-out\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371772 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371666 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371772 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371695 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371772 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371729 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.371772 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371762 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.372014 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371797 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-volume\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.372014 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371825 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.372014 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.371943 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.372596 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:35.372196 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle podName:95fbdd50-1805-47b5-910f-0a9f64ad7b32 nodeName:}" failed. No retries permitted until 2026-04-19 15:26:35.872172833 +0000 UTC m=+129.801632973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32") : configmap references non-existent config key: ca-bundle.crt Apr 19 15:26:35.372714 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:35.372677 2576 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Apr 19 15:26:35.372802 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:26:35.372733 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls podName:95fbdd50-1805-47b5-910f-0a9f64ad7b32 nodeName:}" failed. No retries permitted until 2026-04-19 15:26:35.872715082 +0000 UTC m=+129.802175222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32") : secret "alertmanager-main-tls" not found Apr 19 15:26:35.373099 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.373050 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.374314 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.374286 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.374762 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.374744 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-web-config\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.374858 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.374741 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.375270 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.375241 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-out\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.375387 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.375320 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.375451 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.375423 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.375800 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.375782 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-volume\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.376428 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.376413 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-tls-assets\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.379971 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.379945 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j5bq\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-kube-api-access-8j5bq\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.575603 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.575216 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:26:35.580720 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.580698 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7aeaccc6-af60-4b21-9803-f56daf2f7ddb-metrics-certs\") pod \"network-metrics-daemon-r7nm5\" (UID: \"7aeaccc6-af60-4b21-9803-f56daf2f7ddb\") " pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:26:35.847885 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.847848 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zjn96\"" Apr 19 15:26:35.855975 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.855933 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7nm5" Apr 19 15:26:35.877317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.877274 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.877317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.877327 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.878146 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.878119 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:35.880006 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:35.879984 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:36.041026 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.041000 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r7nm5"] Apr 19 15:26:36.043616 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:36.043592 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aeaccc6_af60_4b21_9803_f56daf2f7ddb.slice/crio-fb10dfa46cfd07ed4d09abfb6ce637736b4eb5824da60ea3de057b451fa79272 WatchSource:0}: Error finding container fb10dfa46cfd07ed4d09abfb6ce637736b4eb5824da60ea3de057b451fa79272: Status 404 returned error can't find the container with id fb10dfa46cfd07ed4d09abfb6ce637736b4eb5824da60ea3de057b451fa79272 Apr 19 15:26:36.144832 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.144795 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-786dd4d45d-wfd4m"] Apr 19 15:26:36.149514 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.148108 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.150768 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.150739 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:26:36.150963 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.150946 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Apr 19 15:26:36.151040 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.150961 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vzglx\"" Apr 19 15:26:36.151040 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.150958 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Apr 19 15:26:36.151168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.151070 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-16qiqcm56keru\"" Apr 19 15:26:36.151866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.151823 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Apr 19 15:26:36.151866 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.151835 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Apr 19 15:26:36.151971 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.151824 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Apr 19 15:26:36.159097 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.159073 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-786dd4d45d-wfd4m"] Apr 19 15:26:36.180705 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.180676 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.180830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.180710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-grpc-tls\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.180830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.180735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.180933 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.180827 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.180933 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.180905 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-metrics-client-ca\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.181086 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.180936 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d6ff\" (UniqueName: \"kubernetes.io/projected/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-kube-api-access-6d6ff\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.181086 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.180973 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-tls\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.181086 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.181075 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.256705 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.256656 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7nm5" event={"ID":"7aeaccc6-af60-4b21-9803-f56daf2f7ddb","Type":"ContainerStarted","Data":"fb10dfa46cfd07ed4d09abfb6ce637736b4eb5824da60ea3de057b451fa79272"} Apr 19 15:26:36.257985 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.257955 2576 generic.go:358] "Generic (PLEG): container finished" podID="a8c0e9b7-a211-4e29-9a98-59a55972095d" containerID="9ed77b22b029035d4dea2684072c1305ea7b3fb9c80fb94142fa2db3476fa16e" exitCode=0 Apr 19 15:26:36.258097 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.258006 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-frtdc" event={"ID":"a8c0e9b7-a211-4e29-9a98-59a55972095d","Type":"ContainerDied","Data":"9ed77b22b029035d4dea2684072c1305ea7b3fb9c80fb94142fa2db3476fa16e"} Apr 19 15:26:36.271771 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.271751 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:26:36.274898 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:36.274862 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95fbdd50_1805_47b5_910f_0a9f64ad7b32.slice/crio-457335e38e13272396831fc89a66185d1a51cab382c30035dbf77014ee05db5b WatchSource:0}: Error finding container 457335e38e13272396831fc89a66185d1a51cab382c30035dbf77014ee05db5b: Status 404 returned error can't find the container with id 457335e38e13272396831fc89a66185d1a51cab382c30035dbf77014ee05db5b Apr 19 15:26:36.281574 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.281552 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.281700 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.281584 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.281700 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.281602 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-grpc-tls\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.281700 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.281677 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.282181 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.282111 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.283027 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.282535 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-metrics-client-ca\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.283027 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.282573 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6d6ff\" (UniqueName: \"kubernetes.io/projected/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-kube-api-access-6d6ff\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.283027 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.282678 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-tls\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.283696 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.283670 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-metrics-client-ca\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.284621 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.284574 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.284621 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.284610 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.284843 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.284711 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.284843 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.284740 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-grpc-tls\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.284997 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.284975 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.284997 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.284993 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-secret-thanos-querier-tls\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.291168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.291147 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d6ff\" (UniqueName: \"kubernetes.io/projected/7e60fe67-3a75-44a3-a571-b3cff8fb84e1-kube-api-access-6d6ff\") pod \"thanos-querier-786dd4d45d-wfd4m\" (UID: \"7e60fe67-3a75-44a3-a571-b3cff8fb84e1\") " pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.469989 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.469951 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:36.608188 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:36.608136 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-786dd4d45d-wfd4m"] Apr 19 15:26:36.612769 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:36.612731 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e60fe67_3a75_44a3_a571_b3cff8fb84e1.slice/crio-c1016d64f4932423a54f83e5d22f128a123846e82a2003e44be6c76497818146 WatchSource:0}: Error finding container c1016d64f4932423a54f83e5d22f128a123846e82a2003e44be6c76497818146: Status 404 returned error can't find the container with id c1016d64f4932423a54f83e5d22f128a123846e82a2003e44be6c76497818146 Apr 19 15:26:37.263839 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:37.263790 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-frtdc" event={"ID":"a8c0e9b7-a211-4e29-9a98-59a55972095d","Type":"ContainerStarted","Data":"29d18f4b75ca3a07f5de3b946af2138cc1025a2af3c7f0e3fd91bd4dd0dd1636"} Apr 19 15:26:37.263839 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:37.263844 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-frtdc" event={"ID":"a8c0e9b7-a211-4e29-9a98-59a55972095d","Type":"ContainerStarted","Data":"e4808281c3d120e4c4dabf49c5f6dfdb01d1724ca079ae0fc444d4e71c7f7091"} Apr 19 15:26:37.265448 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:37.265421 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerStarted","Data":"457335e38e13272396831fc89a66185d1a51cab382c30035dbf77014ee05db5b"} Apr 19 15:26:37.266905 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:37.266880 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" event={"ID":"7e60fe67-3a75-44a3-a571-b3cff8fb84e1","Type":"ContainerStarted","Data":"c1016d64f4932423a54f83e5d22f128a123846e82a2003e44be6c76497818146"} Apr 19 15:26:37.268670 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:37.268648 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7nm5" event={"ID":"7aeaccc6-af60-4b21-9803-f56daf2f7ddb","Type":"ContainerStarted","Data":"688bda1e217720ecbc0364bc94b0172316d37ac11669127739addf9bf484bfd7"} Apr 19 15:26:37.284222 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:37.283781 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-frtdc" podStartSLOduration=2.404506659 podStartE2EDuration="3.28376604s" podCreationTimestamp="2026-04-19 15:26:34 +0000 UTC" firstStartedPulling="2026-04-19 15:26:35.097157063 +0000 UTC m=+129.026617202" lastFinishedPulling="2026-04-19 15:26:35.976416425 +0000 UTC m=+129.905876583" observedRunningTime="2026-04-19 15:26:37.281843775 +0000 UTC m=+131.211303929" watchObservedRunningTime="2026-04-19 15:26:37.28376604 +0000 UTC m=+131.213226204" Apr 19 15:26:38.273377 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:38.273341 2576 generic.go:358] "Generic (PLEG): container finished" podID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerID="9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db" exitCode=0 Apr 19 15:26:38.273843 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:38.273433 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db"} Apr 19 15:26:38.275283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:38.275256 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7nm5" event={"ID":"7aeaccc6-af60-4b21-9803-f56daf2f7ddb","Type":"ContainerStarted","Data":"d44ee8d48b4fd1ca15926f45ba2760deebc080634a69ee231a65b05dd33b903c"} Apr 19 15:26:38.311282 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:38.311233 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r7nm5" podStartSLOduration=131.253079008 podStartE2EDuration="2m12.311218679s" podCreationTimestamp="2026-04-19 15:24:26 +0000 UTC" firstStartedPulling="2026-04-19 15:26:36.045761185 +0000 UTC m=+129.975221328" lastFinishedPulling="2026-04-19 15:26:37.103900848 +0000 UTC m=+131.033360999" observedRunningTime="2026-04-19 15:26:38.309801116 +0000 UTC m=+132.239261302" watchObservedRunningTime="2026-04-19 15:26:38.311218679 +0000 UTC m=+132.240678842" Apr 19 15:26:39.279523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.279483 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" event={"ID":"7e60fe67-3a75-44a3-a571-b3cff8fb84e1","Type":"ContainerStarted","Data":"addfcade6f9ed08872cc7808447a6405df38b3da3262cf1a5fa457efd3ae67ad"} Apr 19 15:26:39.279523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.279523 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" event={"ID":"7e60fe67-3a75-44a3-a571-b3cff8fb84e1","Type":"ContainerStarted","Data":"ec5b077017eae02f1fd9af9660f12f1c865f0d308c1f9b0d5c10aa0a6786e2bc"} Apr 19 15:26:39.280073 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.279533 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" event={"ID":"7e60fe67-3a75-44a3-a571-b3cff8fb84e1","Type":"ContainerStarted","Data":"aca77a3a8aad7de1425ccfe5fe53daec3d84a947c3118b4d0fbe7957444cf020"} Apr 19 15:26:39.373221 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.373187 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-cc96676dd-jgq7w"] Apr 19 15:26:39.376871 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.376838 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.379699 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.379478 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rqr28\"" Apr 19 15:26:39.379699 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.379486 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Apr 19 15:26:39.379699 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.379548 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Apr 19 15:26:39.379699 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.379590 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Apr 19 15:26:39.379995 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.379715 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Apr 19 15:26:39.380149 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.380121 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Apr 19 15:26:39.384835 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.384817 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Apr 19 15:26:39.390576 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.390478 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cc96676dd-jgq7w"] Apr 19 15:26:39.413143 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-secret-telemeter-client\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.413143 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413150 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.413348 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413180 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-metrics-client-ca\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.413348 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413219 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-federate-client-tls\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.413348 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413324 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-telemeter-client-tls\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.413491 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413360 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hngxj\" (UniqueName: \"kubernetes.io/projected/a880e944-1654-4de1-a3e3-54bf9bebb4de-kube-api-access-hngxj\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.413491 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.413491 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.413438 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-serving-certs-ca-bundle\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514486 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514448 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-telemeter-client-tls\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514486 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514483 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hngxj\" (UniqueName: \"kubernetes.io/projected/a880e944-1654-4de1-a3e3-54bf9bebb4de-kube-api-access-hngxj\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514762 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514678 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514762 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514750 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-serving-certs-ca-bundle\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514872 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514824 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-secret-telemeter-client\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514872 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514855 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514976 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514880 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-metrics-client-ca\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.514976 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.514928 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-federate-client-tls\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.515795 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.515741 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-serving-certs-ca-bundle\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.515795 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.515781 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.515990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.515797 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a880e944-1654-4de1-a3e3-54bf9bebb4de-metrics-client-ca\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.517424 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.517394 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-secret-telemeter-client\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.517580 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.517563 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-telemeter-client-tls\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.517785 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.517725 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.517864 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.517831 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a880e944-1654-4de1-a3e3-54bf9bebb4de-federate-client-tls\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.522532 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.522508 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hngxj\" (UniqueName: \"kubernetes.io/projected/a880e944-1654-4de1-a3e3-54bf9bebb4de-kube-api-access-hngxj\") pod \"telemeter-client-cc96676dd-jgq7w\" (UID: \"a880e944-1654-4de1-a3e3-54bf9bebb4de\") " pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:39.692883 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:39.692849 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" Apr 19 15:26:40.364157 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.364135 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cc96676dd-jgq7w"] Apr 19 15:26:40.366333 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:26:40.366307 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda880e944_1654_4de1_a3e3_54bf9bebb4de.slice/crio-b6649a796418c9dbe35839869592c44a743298c82f01263f5348171fd3923a47 WatchSource:0}: Error finding container b6649a796418c9dbe35839869592c44a743298c82f01263f5348171fd3923a47: Status 404 returned error can't find the container with id b6649a796418c9dbe35839869592c44a743298c82f01263f5348171fd3923a47 Apr 19 15:26:40.436459 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.435682 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:26:40.442842 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.440298 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.443741 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.443585 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 19 15:26:40.443844 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.443808 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3cr70q92qqbb4\"" Apr 19 15:26:40.444312 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.444174 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 19 15:26:40.444312 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.444222 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-64mhl\"" Apr 19 15:26:40.444881 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.444860 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 19 15:26:40.446524 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.445134 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 19 15:26:40.446524 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.445360 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 19 15:26:40.446524 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.445540 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 19 15:26:40.446524 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.445975 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 19 15:26:40.446524 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.446316 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Apr 19 15:26:40.446826 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.446708 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 19 15:26:40.447434 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.446942 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 19 15:26:40.447434 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.446990 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 19 15:26:40.447434 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.447228 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 19 15:26:40.449353 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.449263 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 19 15:26:40.460043 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.459996 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:26:40.524987 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.524927 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-config\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.524987 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.524977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525187 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkcf8\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-kube-api-access-rkcf8\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525187 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525052 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525187 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525092 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-config-out\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525187 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525187 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525153 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525187 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525183 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-web-config\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525220 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525246 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525276 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525315 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525344 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525381 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525402 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525441 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525425 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525767 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525507 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.525767 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.525576 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.626762 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.626728 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.626869 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.626794 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.626869 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.626850 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-config\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.626980 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.626888 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.626980 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.626914 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rkcf8\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-kube-api-access-rkcf8\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627077 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627046 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627135 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-config-out\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627169 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627209 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627245 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-web-config\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627284 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627313 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627347 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627376 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627416 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627463 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.627488 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627493 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.628153 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627524 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.628153 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627664 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.628153 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.627726 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.628337 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.628306 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.628426 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.628400 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.631472 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.631451 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.632252 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.632206 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.632573 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.632529 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.633267 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.633241 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.633461 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.633241 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-config-out\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.633461 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.633446 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.633800 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.633779 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.633908 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.633832 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.634119 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.634074 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.634266 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.634138 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-config\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.634384 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.634335 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.634492 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.634470 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-web-config\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.634645 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.634570 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.639411 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.639376 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkcf8\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-kube-api-access-rkcf8\") pod \"prometheus-k8s-0\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.758034 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.757952 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:26:40.901396 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:40.901312 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:26:41.298844 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.298808 2576 generic.go:358] "Generic (PLEG): container finished" podID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" exitCode=0 Apr 19 15:26:41.299017 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.298897 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f"} Apr 19 15:26:41.299017 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.298934 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerStarted","Data":"3193a1d4bf2c986b359ab3753a2e913b21c813587c2ba9b69dd4cc0eb7129b0f"} Apr 19 15:26:41.302363 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.302319 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerStarted","Data":"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7"} Apr 19 15:26:41.302363 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.302354 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerStarted","Data":"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789"} Apr 19 15:26:41.302497 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.302367 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerStarted","Data":"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635"} Apr 19 15:26:41.302497 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.302379 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerStarted","Data":"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268"} Apr 19 15:26:41.302497 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.302391 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerStarted","Data":"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822"} Apr 19 15:26:41.302497 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.302403 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerStarted","Data":"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf"} Apr 19 15:26:41.303674 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.303648 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" event={"ID":"a880e944-1654-4de1-a3e3-54bf9bebb4de","Type":"ContainerStarted","Data":"b6649a796418c9dbe35839869592c44a743298c82f01263f5348171fd3923a47"} Apr 19 15:26:41.306717 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.306691 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" event={"ID":"7e60fe67-3a75-44a3-a571-b3cff8fb84e1","Type":"ContainerStarted","Data":"744c920bdeb87627cc3c079e238bc6d812b35e4bc7e4cb35ff823c9f69a60457"} Apr 19 15:26:41.306814 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.306730 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" event={"ID":"7e60fe67-3a75-44a3-a571-b3cff8fb84e1","Type":"ContainerStarted","Data":"2bb411cd1911a35ea2ad70b4bfe17bedf91a4f9dba9cdc06035ef606ceff2bcf"} Apr 19 15:26:41.306814 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.306746 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" event={"ID":"7e60fe67-3a75-44a3-a571-b3cff8fb84e1","Type":"ContainerStarted","Data":"c8b957f958a55da201a74de2d5dd2682ae3cb2517a6115fd3ac5fb099b3a5b39"} Apr 19 15:26:41.306916 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.306893 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:41.347196 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.347144 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" podStartSLOduration=1.7568967020000001 podStartE2EDuration="5.34712882s" podCreationTimestamp="2026-04-19 15:26:36 +0000 UTC" firstStartedPulling="2026-04-19 15:26:36.615033055 +0000 UTC m=+130.544493195" lastFinishedPulling="2026-04-19 15:26:40.205265158 +0000 UTC m=+134.134725313" observedRunningTime="2026-04-19 15:26:41.345889896 +0000 UTC m=+135.275350068" watchObservedRunningTime="2026-04-19 15:26:41.34712882 +0000 UTC m=+135.276588982" Apr 19 15:26:41.376581 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:41.376531 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.4496915599999998 podStartE2EDuration="6.376515776s" podCreationTimestamp="2026-04-19 15:26:35 +0000 UTC" firstStartedPulling="2026-04-19 15:26:36.276777461 +0000 UTC m=+130.206237601" lastFinishedPulling="2026-04-19 15:26:40.203601664 +0000 UTC m=+134.133061817" observedRunningTime="2026-04-19 15:26:41.374622344 +0000 UTC m=+135.304082507" watchObservedRunningTime="2026-04-19 15:26:41.376515776 +0000 UTC m=+135.305975970" Apr 19 15:26:43.315208 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:43.315167 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" event={"ID":"a880e944-1654-4de1-a3e3-54bf9bebb4de","Type":"ContainerStarted","Data":"f9dec5757f206de6c530708afa7ddfd5878b100c664cf47288c5170233f2e202"} Apr 19 15:26:43.315208 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:43.315210 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" event={"ID":"a880e944-1654-4de1-a3e3-54bf9bebb4de","Type":"ContainerStarted","Data":"0c0c319830bc646cf1f1a63b35c5f392d573746d9153e658ab468e66896e4750"} Apr 19 15:26:43.315617 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:43.315221 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" event={"ID":"a880e944-1654-4de1-a3e3-54bf9bebb4de","Type":"ContainerStarted","Data":"7df416c6849267a26978b9a7a5030036ba3e1522ec5a3bd8a10e203db36a3177"} Apr 19 15:26:43.337282 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:43.337229 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-cc96676dd-jgq7w" podStartSLOduration=2.155718851 podStartE2EDuration="4.337209964s" podCreationTimestamp="2026-04-19 15:26:39 +0000 UTC" firstStartedPulling="2026-04-19 15:26:40.368383745 +0000 UTC m=+134.297843884" lastFinishedPulling="2026-04-19 15:26:42.549874857 +0000 UTC m=+136.479334997" observedRunningTime="2026-04-19 15:26:43.335081046 +0000 UTC m=+137.264541210" watchObservedRunningTime="2026-04-19 15:26:43.337209964 +0000 UTC m=+137.266670124" Apr 19 15:26:44.736263 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:44.736230 2576 scope.go:117] "RemoveContainer" containerID="b425caae2d9483f4079d8ace2eab558155ffa94a1f07200067ac5d385b865d4c" Apr 19 15:26:45.323312 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:45.323281 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerStarted","Data":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} Apr 19 15:26:45.323437 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:45.323322 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerStarted","Data":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} Apr 19 15:26:45.325719 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:45.325687 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:26:45.325832 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:45.325814 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" event={"ID":"8f15cf98-a60c-41aa-96e8-be8466dd8e26","Type":"ContainerStarted","Data":"cc568684f84a03eef6f924ee4230f77d0987489e5d46023b87115f5a6902e05f"} Apr 19 15:26:45.327142 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:45.326863 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:26:45.343752 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:45.342417 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" podStartSLOduration=50.785103978 podStartE2EDuration="53.34239825s" podCreationTimestamp="2026-04-19 15:25:52 +0000 UTC" firstStartedPulling="2026-04-19 15:25:52.590211381 +0000 UTC m=+86.519671531" lastFinishedPulling="2026-04-19 15:25:55.147505659 +0000 UTC m=+89.076965803" observedRunningTime="2026-04-19 15:26:45.342104499 +0000 UTC m=+139.271564661" watchObservedRunningTime="2026-04-19 15:26:45.34239825 +0000 UTC m=+139.271858413" Apr 19 15:26:46.137234 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:46.137206 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-9d4b6777b-qjpxz" Apr 19 15:26:46.331576 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:46.331532 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerStarted","Data":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} Apr 19 15:26:46.331770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:46.331582 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerStarted","Data":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} Apr 19 15:26:46.331770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:46.331601 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerStarted","Data":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} Apr 19 15:26:46.331770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:46.331685 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerStarted","Data":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} Apr 19 15:26:46.364547 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:46.364453 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.509083118 podStartE2EDuration="6.364435656s" podCreationTimestamp="2026-04-19 15:26:40 +0000 UTC" firstStartedPulling="2026-04-19 15:26:41.300395818 +0000 UTC m=+135.229855965" lastFinishedPulling="2026-04-19 15:26:45.155748362 +0000 UTC m=+139.085208503" observedRunningTime="2026-04-19 15:26:46.363058726 +0000 UTC m=+140.292518888" watchObservedRunningTime="2026-04-19 15:26:46.364435656 +0000 UTC m=+140.293895819" Apr 19 15:26:47.318214 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:47.318188 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-786dd4d45d-wfd4m" Apr 19 15:26:50.758809 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:26:50.758772 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:01.962719 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:01.962673 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wh4pt" podUID="e3f37774-81cc-4b2c-94cb-ec77935ec46d" Apr 19 15:27:01.982436 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:01.982404 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-qcm6b" podUID="ea08d4af-7d5b-47ed-ab7f-2e997210a6b9" Apr 19 15:27:02.378685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:02.378654 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wh4pt" Apr 19 15:27:06.875734 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:06.875688 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:27:06.876266 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:06.875808 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:27:06.878153 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:06.878132 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ea08d4af-7d5b-47ed-ab7f-2e997210a6b9-cert\") pod \"ingress-canary-qcm6b\" (UID: \"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9\") " pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:27:06.878220 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:06.878156 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e3f37774-81cc-4b2c-94cb-ec77935ec46d-metrics-tls\") pod \"dns-default-wh4pt\" (UID: \"e3f37774-81cc-4b2c-94cb-ec77935ec46d\") " pod="openshift-dns/dns-default-wh4pt" Apr 19 15:27:06.882501 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:06.882481 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dvb5z\"" Apr 19 15:27:06.890642 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:06.890611 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wh4pt" Apr 19 15:27:07.037387 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:07.037361 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wh4pt"] Apr 19 15:27:07.040051 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:27:07.040021 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f37774_81cc_4b2c_94cb_ec77935ec46d.slice/crio-0bb576712772e18968649073eb6922a17c177e67ff4516498330709c0d1c891a WatchSource:0}: Error finding container 0bb576712772e18968649073eb6922a17c177e67ff4516498330709c0d1c891a: Status 404 returned error can't find the container with id 0bb576712772e18968649073eb6922a17c177e67ff4516498330709c0d1c891a Apr 19 15:27:07.393990 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:07.393955 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wh4pt" event={"ID":"e3f37774-81cc-4b2c-94cb-ec77935ec46d","Type":"ContainerStarted","Data":"0bb576712772e18968649073eb6922a17c177e67ff4516498330709c0d1c891a"} Apr 19 15:27:08.884275 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:08.884247 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d7c7c5446-wbdgz_41b0c1cd-70f7-47aa-b30e-0368722019e6/router/0.log" Apr 19 15:27:09.401868 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:09.401834 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wh4pt" event={"ID":"e3f37774-81cc-4b2c-94cb-ec77935ec46d","Type":"ContainerStarted","Data":"5b174a0108fe8239530965db0d3da008d48ad92a2b08b664ea9e6c5c7f42fe18"} Apr 19 15:27:09.401868 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:09.401870 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wh4pt" event={"ID":"e3f37774-81cc-4b2c-94cb-ec77935ec46d","Type":"ContainerStarted","Data":"766388742d37f56f95dd1e00404b137805f256de5df812cb00926a30d1aa942e"} Apr 19 15:27:09.402068 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:09.401894 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wh4pt" Apr 19 15:27:09.418092 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:09.418042 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-wh4pt" podStartSLOduration=129.928747427 podStartE2EDuration="2m11.418025965s" podCreationTimestamp="2026-04-19 15:24:58 +0000 UTC" firstStartedPulling="2026-04-19 15:27:07.041851914 +0000 UTC m=+160.971312054" lastFinishedPulling="2026-04-19 15:27:08.531130436 +0000 UTC m=+162.460590592" observedRunningTime="2026-04-19 15:27:09.41725646 +0000 UTC m=+163.346716622" watchObservedRunningTime="2026-04-19 15:27:09.418025965 +0000 UTC m=+163.347486126" Apr 19 15:27:11.409576 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:11.409543 2576 generic.go:358] "Generic (PLEG): container finished" podID="142a272e-7901-47e5-a0d4-f9e2be29d84a" containerID="20a3f703548ec75c1dea7ee80c3de2b9b3247c03e0db8034cc44818018c45114" exitCode=0 Apr 19 15:27:11.409956 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:11.409596 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" event={"ID":"142a272e-7901-47e5-a0d4-f9e2be29d84a","Type":"ContainerDied","Data":"20a3f703548ec75c1dea7ee80c3de2b9b3247c03e0db8034cc44818018c45114"} Apr 19 15:27:11.409956 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:11.409916 2576 scope.go:117] "RemoveContainer" containerID="20a3f703548ec75c1dea7ee80c3de2b9b3247c03e0db8034cc44818018c45114" Apr 19 15:27:12.415166 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:12.415132 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-5j2z2" event={"ID":"142a272e-7901-47e5-a0d4-f9e2be29d84a","Type":"ContainerStarted","Data":"dfae250a72e56863501a2680f7bb0a8e11db49020ebfc8dfd06815496b30f856"} Apr 19 15:27:13.736643 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:13.736534 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:27:13.739841 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:13.739817 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zxl4j\"" Apr 19 15:27:13.748168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:13.748132 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qcm6b" Apr 19 15:27:13.868820 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:13.868783 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qcm6b"] Apr 19 15:27:13.872281 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:27:13.872255 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea08d4af_7d5b_47ed_ab7f_2e997210a6b9.slice/crio-a7eee6e9eea2d06acbbcc548d9e19cae668f7c8eb13324c119454583176d296c WatchSource:0}: Error finding container a7eee6e9eea2d06acbbcc548d9e19cae668f7c8eb13324c119454583176d296c: Status 404 returned error can't find the container with id a7eee6e9eea2d06acbbcc548d9e19cae668f7c8eb13324c119454583176d296c Apr 19 15:27:14.421675 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:14.421639 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qcm6b" event={"ID":"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9","Type":"ContainerStarted","Data":"a7eee6e9eea2d06acbbcc548d9e19cae668f7c8eb13324c119454583176d296c"} Apr 19 15:27:16.430697 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:16.430657 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qcm6b" event={"ID":"ea08d4af-7d5b-47ed-ab7f-2e997210a6b9","Type":"ContainerStarted","Data":"5c152e81913aec23f923506ce2c7d7ee624518ff323f31236ca7484cfac489ed"} Apr 19 15:27:16.449753 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:16.449695 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qcm6b" podStartSLOduration=136.630816267 podStartE2EDuration="2m18.449679701s" podCreationTimestamp="2026-04-19 15:24:58 +0000 UTC" firstStartedPulling="2026-04-19 15:27:13.874167595 +0000 UTC m=+167.803627734" lastFinishedPulling="2026-04-19 15:27:15.693031028 +0000 UTC m=+169.622491168" observedRunningTime="2026-04-19 15:27:16.447969398 +0000 UTC m=+170.377429560" watchObservedRunningTime="2026-04-19 15:27:16.449679701 +0000 UTC m=+170.379139862" Apr 19 15:27:19.407875 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:19.407847 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wh4pt" Apr 19 15:27:40.758340 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:40.758305 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:40.774819 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:40.774790 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:41.521329 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:41.521305 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:54.466196 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:54.466153 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:27:54.466818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:54.466593 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="alertmanager" containerID="cri-o://d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf" gracePeriod=120 Apr 19 15:27:54.466818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:54.466689 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-metric" containerID="cri-o://ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789" gracePeriod=120 Apr 19 15:27:54.466818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:54.466713 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="prom-label-proxy" containerID="cri-o://59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7" gracePeriod=120 Apr 19 15:27:54.466818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:54.466742 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-web" containerID="cri-o://2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268" gracePeriod=120 Apr 19 15:27:54.466818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:54.466797 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy" containerID="cri-o://f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635" gracePeriod=120 Apr 19 15:27:54.466818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:54.466804 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="config-reloader" containerID="cri-o://0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822" gracePeriod=120 Apr 19 15:27:55.553340 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553311 2576 generic.go:358] "Generic (PLEG): container finished" podID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerID="59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7" exitCode=0 Apr 19 15:27:55.553340 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553338 2576 generic.go:358] "Generic (PLEG): container finished" podID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerID="f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635" exitCode=0 Apr 19 15:27:55.553713 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553347 2576 generic.go:358] "Generic (PLEG): container finished" podID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerID="0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822" exitCode=0 Apr 19 15:27:55.553713 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553353 2576 generic.go:358] "Generic (PLEG): container finished" podID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerID="d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf" exitCode=0 Apr 19 15:27:55.553713 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553382 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7"} Apr 19 15:27:55.553713 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553419 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635"} Apr 19 15:27:55.553713 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553431 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822"} Apr 19 15:27:55.553713 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.553440 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf"} Apr 19 15:27:55.713255 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.713232 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:55.802915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.802821 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-out\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.802915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.802868 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.802915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.802896 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-cluster-tls-config\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.802937 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-web-config\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.802961 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803150 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j5bq\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-kube-api-access-8j5bq\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803198 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-main-db\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803485 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803280 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-metric\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803485 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803311 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-metrics-client-ca\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803485 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803331 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:27:55.803485 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803351 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-tls-assets\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803485 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803379 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-web\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803485 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803416 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803485 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803455 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-volume\") pod \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\" (UID: \"95fbdd50-1805-47b5-910f-0a9f64ad7b32\") " Apr 19 15:27:55.803830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803738 2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.803885 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.803848 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 19 15:27:55.804365 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.804299 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:27:55.805764 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.805713 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:55.806211 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.806176 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-kube-api-access-8j5bq" (OuterVolumeSpecName: "kube-api-access-8j5bq") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "kube-api-access-8j5bq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:27:55.806680 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.806653 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-volume" (OuterVolumeSpecName: "config-volume") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:55.806781 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.806743 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:55.806845 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.806768 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:55.806845 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.806837 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-out" (OuterVolumeSpecName: "config-out") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 19 15:27:55.807225 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.807203 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:27:55.807853 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.807837 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:55.811430 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.811400 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:55.818452 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.818431 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-web-config" (OuterVolumeSpecName: "web-config") pod "95fbdd50-1805-47b5-910f-0a9f64ad7b32" (UID: "95fbdd50-1805-47b5-910f-0a9f64ad7b32"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:55.904683 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904646 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904683 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904680 2576 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95fbdd50-1805-47b5-910f-0a9f64ad7b32-metrics-client-ca\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904683 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904692 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-tls-assets\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904702 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904714 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-main-tls\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904723 2576 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-volume\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904731 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-config-out\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904740 2576 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-cluster-tls-config\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904748 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-web-config\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904758 2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/95fbdd50-1805-47b5-910f-0a9f64ad7b32-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904768 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j5bq\" (UniqueName: \"kubernetes.io/projected/95fbdd50-1805-47b5-910f-0a9f64ad7b32-kube-api-access-8j5bq\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:55.904921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:55.904777 2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/95fbdd50-1805-47b5-910f-0a9f64ad7b32-alertmanager-main-db\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:56.559727 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.559693 2576 generic.go:358] "Generic (PLEG): container finished" podID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerID="ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789" exitCode=0 Apr 19 15:27:56.559727 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.559720 2576 generic.go:358] "Generic (PLEG): container finished" podID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerID="2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268" exitCode=0 Apr 19 15:27:56.560283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.559750 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789"} Apr 19 15:27:56.560283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.559776 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268"} Apr 19 15:27:56.560283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.559786 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"95fbdd50-1805-47b5-910f-0a9f64ad7b32","Type":"ContainerDied","Data":"457335e38e13272396831fc89a66185d1a51cab382c30035dbf77014ee05db5b"} Apr 19 15:27:56.560283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.559800 2576 scope.go:117] "RemoveContainer" containerID="59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7" Apr 19 15:27:56.560283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.559825 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.567826 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.567801 2576 scope.go:117] "RemoveContainer" containerID="ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789" Apr 19 15:27:56.574695 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.574675 2576 scope.go:117] "RemoveContainer" containerID="f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635" Apr 19 15:27:56.581231 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.581201 2576 scope.go:117] "RemoveContainer" containerID="2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268" Apr 19 15:27:56.583670 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.583618 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:27:56.587468 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.587446 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:27:56.592128 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.591936 2576 scope.go:117] "RemoveContainer" containerID="0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822" Apr 19 15:27:56.598774 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.598758 2576 scope.go:117] "RemoveContainer" containerID="d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf" Apr 19 15:27:56.605471 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.605440 2576 scope.go:117] "RemoveContainer" containerID="9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db" Apr 19 15:27:56.611723 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.611704 2576 scope.go:117] "RemoveContainer" containerID="59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7" Apr 19 15:27:56.611986 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:56.611964 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7\": container with ID starting with 59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7 not found: ID does not exist" containerID="59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7" Apr 19 15:27:56.612026 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.611995 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7"} err="failed to get container status \"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7\": rpc error: code = NotFound desc = could not find container \"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7\": container with ID starting with 59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7 not found: ID does not exist" Apr 19 15:27:56.612068 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.612032 2576 scope.go:117] "RemoveContainer" containerID="ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789" Apr 19 15:27:56.612292 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:56.612263 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789\": container with ID starting with ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789 not found: ID does not exist" containerID="ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789" Apr 19 15:27:56.612382 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.612300 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789"} err="failed to get container status \"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789\": rpc error: code = NotFound desc = could not find container \"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789\": container with ID starting with ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789 not found: ID does not exist" Apr 19 15:27:56.612382 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.612321 2576 scope.go:117] "RemoveContainer" containerID="f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635" Apr 19 15:27:56.612517 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:56.612502 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635\": container with ID starting with f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635 not found: ID does not exist" containerID="f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635" Apr 19 15:27:56.612554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.612519 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635"} err="failed to get container status \"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635\": rpc error: code = NotFound desc = could not find container \"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635\": container with ID starting with f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635 not found: ID does not exist" Apr 19 15:27:56.612554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.612532 2576 scope.go:117] "RemoveContainer" containerID="2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268" Apr 19 15:27:56.612787 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:56.612766 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268\": container with ID starting with 2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268 not found: ID does not exist" containerID="2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268" Apr 19 15:27:56.612876 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.612806 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268"} err="failed to get container status \"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268\": rpc error: code = NotFound desc = could not find container \"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268\": container with ID starting with 2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268 not found: ID does not exist" Apr 19 15:27:56.612876 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.612832 2576 scope.go:117] "RemoveContainer" containerID="0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822" Apr 19 15:27:56.613206 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:56.613176 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822\": container with ID starting with 0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822 not found: ID does not exist" containerID="0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822" Apr 19 15:27:56.613295 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.613208 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822"} err="failed to get container status \"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822\": rpc error: code = NotFound desc = could not find container \"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822\": container with ID starting with 0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822 not found: ID does not exist" Apr 19 15:27:56.613295 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.613228 2576 scope.go:117] "RemoveContainer" containerID="d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf" Apr 19 15:27:56.613566 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:56.613540 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf\": container with ID starting with d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf not found: ID does not exist" containerID="d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf" Apr 19 15:27:56.613700 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.613572 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf"} err="failed to get container status \"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf\": rpc error: code = NotFound desc = could not find container \"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf\": container with ID starting with d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf not found: ID does not exist" Apr 19 15:27:56.613700 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.613592 2576 scope.go:117] "RemoveContainer" containerID="9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db" Apr 19 15:27:56.613863 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:56.613844 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db\": container with ID starting with 9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db not found: ID does not exist" containerID="9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db" Apr 19 15:27:56.613942 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.613865 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db"} err="failed to get container status \"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db\": rpc error: code = NotFound desc = could not find container \"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db\": container with ID starting with 9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db not found: ID does not exist" Apr 19 15:27:56.613942 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.613878 2576 scope.go:117] "RemoveContainer" containerID="59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7" Apr 19 15:27:56.614169 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614124 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7"} err="failed to get container status \"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7\": rpc error: code = NotFound desc = could not find container \"59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7\": container with ID starting with 59e642b2c6bf30d92c3528c2425709d92f7e44f58967fb4ef7db8fb10feb35c7 not found: ID does not exist" Apr 19 15:27:56.614225 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614171 2576 scope.go:117] "RemoveContainer" containerID="ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789" Apr 19 15:27:56.614391 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614370 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789"} err="failed to get container status \"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789\": rpc error: code = NotFound desc = could not find container \"ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789\": container with ID starting with ee39316b80dc3e78df1d08e5dc9e7e2804a523a624f2751c49d5e9ec86c3f789 not found: ID does not exist" Apr 19 15:27:56.614391 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614391 2576 scope.go:117] "RemoveContainer" containerID="f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635" Apr 19 15:27:56.614650 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614616 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635"} err="failed to get container status \"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635\": rpc error: code = NotFound desc = could not find container \"f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635\": container with ID starting with f10af481eb8704d5135635930c42675127b502e3a56d2e3900dff1d1563f4635 not found: ID does not exist" Apr 19 15:27:56.614717 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614649 2576 scope.go:117] "RemoveContainer" containerID="2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268" Apr 19 15:27:56.614888 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614872 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268"} err="failed to get container status \"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268\": rpc error: code = NotFound desc = could not find container \"2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268\": container with ID starting with 2a4db3ef958b69d8b03ee18a9bfaa93f9dba48d135718396c7b6bd1037b61268 not found: ID does not exist" Apr 19 15:27:56.614946 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.614889 2576 scope.go:117] "RemoveContainer" containerID="0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822" Apr 19 15:27:56.615112 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615092 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822"} err="failed to get container status \"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822\": rpc error: code = NotFound desc = could not find container \"0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822\": container with ID starting with 0db3957b0c51e4ce189760ac037f37196627f7ce78b62178f6dfd1bf9d6a1822 not found: ID does not exist" Apr 19 15:27:56.615189 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615114 2576 scope.go:117] "RemoveContainer" containerID="d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf" Apr 19 15:27:56.615189 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615135 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:27:56.615344 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615326 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf"} err="failed to get container status \"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf\": rpc error: code = NotFound desc = could not find container \"d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf\": container with ID starting with d4bfb60f44189059833d3623de69b87044919ab38a77bd5a99f76945381d1ccf not found: ID does not exist" Apr 19 15:27:56.615392 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615345 2576 scope.go:117] "RemoveContainer" containerID="9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db" Apr 19 15:27:56.615537 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615519 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db"} err="failed to get container status \"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db\": rpc error: code = NotFound desc = could not find container \"9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db\": container with ID starting with 9b763ef0e071cec3a98907cd4eb1f1045959fb325759226cd13d85559fda53db not found: ID does not exist" Apr 19 15:27:56.615586 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615552 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy" Apr 19 15:27:56.615586 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615564 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615593 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="init-config-reloader" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615599 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="init-config-reloader" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615612 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-web" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615619 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-web" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615647 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="config-reloader" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615653 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="config-reloader" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615659 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-metric" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615664 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-metric" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615671 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="alertmanager" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615677 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="alertmanager" Apr 19 15:27:56.615685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615688 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="prom-label-proxy" Apr 19 15:27:56.616078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615695 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="prom-label-proxy" Apr 19 15:27:56.616078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615744 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="config-reloader" Apr 19 15:27:56.616078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615752 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="prom-label-proxy" Apr 19 15:27:56.616078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615760 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy" Apr 19 15:27:56.616078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615768 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-web" Apr 19 15:27:56.616078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615774 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="kube-rbac-proxy-metric" Apr 19 15:27:56.616078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.615781 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" containerName="alertmanager" Apr 19 15:27:56.620861 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.620844 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.623595 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.623573 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.623957 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.623979 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.624008 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.624018 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.624036 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.623962 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.623964 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Apr 19 15:27:56.624168 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.624021 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rrckw\"" Apr 19 15:27:56.629142 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.629120 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Apr 19 15:27:56.631595 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.631574 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:27:56.712985 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.712956 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.712985 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.712991 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c08f491-98f2-4085-aaca-3c71f4601377-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713186 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713186 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713074 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c08f491-98f2-4085-aaca-3c71f4601377-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713186 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713107 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s62g9\" (UniqueName: \"kubernetes.io/projected/4c08f491-98f2-4085-aaca-3c71f4601377-kube-api-access-s62g9\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713186 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713127 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713186 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713146 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c08f491-98f2-4085-aaca-3c71f4601377-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713208 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713222 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-web-config\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713241 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713256 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c08f491-98f2-4085-aaca-3c71f4601377-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713279 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c08f491-98f2-4085-aaca-3c71f4601377-config-out\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.713343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.713298 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.740117 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.740091 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95fbdd50-1805-47b5-910f-0a9f64ad7b32" path="/var/lib/kubelet/pods/95fbdd50-1805-47b5-910f-0a9f64ad7b32/volumes" Apr 19 15:27:56.814331 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814261 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814331 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814297 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c08f491-98f2-4085-aaca-3c71f4601377-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814331 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814326 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s62g9\" (UniqueName: \"kubernetes.io/projected/4c08f491-98f2-4085-aaca-3c71f4601377-kube-api-access-s62g9\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814551 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814345 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814551 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814482 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c08f491-98f2-4085-aaca-3c71f4601377-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814689 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814582 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814689 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814617 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-web-config\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814689 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814681 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814834 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814713 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c08f491-98f2-4085-aaca-3c71f4601377-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814834 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814767 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c08f491-98f2-4085-aaca-3c71f4601377-config-out\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814834 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814809 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814986 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814891 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.814986 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.814926 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c08f491-98f2-4085-aaca-3c71f4601377-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.815325 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.815297 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c08f491-98f2-4085-aaca-3c71f4601377-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.815585 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.815545 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c08f491-98f2-4085-aaca-3c71f4601377-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.815715 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.815646 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c08f491-98f2-4085-aaca-3c71f4601377-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.817547 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.817523 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-web-config\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.817681 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.817608 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.817681 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.817607 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.818012 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.817986 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.818129 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.818109 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.818236 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.818200 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.818369 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.818350 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c08f491-98f2-4085-aaca-3c71f4601377-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.818507 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.818491 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c08f491-98f2-4085-aaca-3c71f4601377-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.819477 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.819460 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c08f491-98f2-4085-aaca-3c71f4601377-config-out\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.821464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.821447 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s62g9\" (UniqueName: \"kubernetes.io/projected/4c08f491-98f2-4085-aaca-3c71f4601377-kube-api-access-s62g9\") pod \"alertmanager-main-0\" (UID: \"4c08f491-98f2-4085-aaca-3c71f4601377\") " pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:56.930842 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:56.930794 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Apr 19 15:27:57.055289 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:57.055256 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Apr 19 15:27:57.058811 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:27:57.058772 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c08f491_98f2_4085_aaca_3c71f4601377.slice/crio-c08be737974c197f4a63bc892ac68f296c5feca0e61570fc8daca6da9ffcb4a2 WatchSource:0}: Error finding container c08be737974c197f4a63bc892ac68f296c5feca0e61570fc8daca6da9ffcb4a2: Status 404 returned error can't find the container with id c08be737974c197f4a63bc892ac68f296c5feca0e61570fc8daca6da9ffcb4a2 Apr 19 15:27:57.565068 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:57.565031 2576 generic.go:358] "Generic (PLEG): container finished" podID="4c08f491-98f2-4085-aaca-3c71f4601377" containerID="1e9cf150493ffcd530627e33d49126e57da4e39235d052bf5c47053cfd30a44c" exitCode=0 Apr 19 15:27:57.565555 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:57.565083 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerDied","Data":"1e9cf150493ffcd530627e33d49126e57da4e39235d052bf5c47053cfd30a44c"} Apr 19 15:27:57.565555 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:57.565104 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerStarted","Data":"c08be737974c197f4a63bc892ac68f296c5feca0e61570fc8daca6da9ffcb4a2"} Apr 19 15:27:58.572149 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.572110 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerStarted","Data":"840ac803a7e861d6bc83f77d60de531bdd831e91a35be678582608f9ef6666af"} Apr 19 15:27:58.572534 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.572155 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerStarted","Data":"beff431c1655ec7cba0e13b40d16cd77b7e4a5609363d5c8c2caed6b3318dde9"} Apr 19 15:27:58.572534 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.572171 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerStarted","Data":"82267ef116345edee2f1dc854c3b542aa9b4f65edeba05e6ac11d1a7d3629b0e"} Apr 19 15:27:58.572534 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.572184 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerStarted","Data":"56fb757b5337f2bdc9b3b197fbe56a9f58c642b4c34d8b8fa6cc4906838d3cbb"} Apr 19 15:27:58.572534 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.572212 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerStarted","Data":"040203f2328b8fd65a3fba248ce87039d5ac68f2c4bd085df5f09b4a9577cfa6"} Apr 19 15:27:58.572534 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.572225 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c08f491-98f2-4085-aaca-3c71f4601377","Type":"ContainerStarted","Data":"77bc2a0f06f1cf29952d3769cfcc18a1963d848068f5d3e6e0625e6ad49cf810"} Apr 19 15:27:58.600788 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.600732 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.60070919 podStartE2EDuration="2.60070919s" podCreationTimestamp="2026-04-19 15:27:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-19 15:27:58.599643053 +0000 UTC m=+212.529103214" watchObservedRunningTime="2026-04-19 15:27:58.60070919 +0000 UTC m=+212.530169352" Apr 19 15:27:58.687779 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.687741 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:27:58.688523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.688347 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="prometheus" containerID="cri-o://8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" gracePeriod=600 Apr 19 15:27:58.688523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.688382 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy" containerID="cri-o://c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" gracePeriod=600 Apr 19 15:27:58.688523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.688396 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="thanos-sidecar" containerID="cri-o://afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" gracePeriod=600 Apr 19 15:27:58.688523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.688426 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="config-reloader" containerID="cri-o://1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" gracePeriod=600 Apr 19 15:27:58.688523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.688394 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" gracePeriod=600 Apr 19 15:27:58.688523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.688420 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-web" containerID="cri-o://6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" gracePeriod=600 Apr 19 15:27:58.946216 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:58.945901 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.031831 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.031799 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.031831 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.031836 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-thanos-prometheus-http-client-file\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032064 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.031854 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-web-config\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032064 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.031876 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-rulefiles-0\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032064 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032012 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-tls-assets\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032064 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032058 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-grpc-tls\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032269 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032088 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-metrics-client-ca\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032269 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032118 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-config\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032269 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032144 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkcf8\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-kube-api-access-rkcf8\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032269 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032172 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-tls\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032269 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032204 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-kube-rbac-proxy\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032278 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-db\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032348 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-metrics-client-certs\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032383 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-kubelet-serving-ca-bundle\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032430 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032472 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-trusted-ca-bundle\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032512 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032503 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-config-out\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.032806 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.032532 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-serving-certs-ca-bundle\") pod \"b870f93c-f6f0-4b6f-836d-ad46f917224f\" (UID: \"b870f93c-f6f0-4b6f-836d-ad46f917224f\") " Apr 19 15:27:59.033196 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.033169 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:27:59.033413 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.033387 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:27:59.033957 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.033927 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:27:59.034473 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.034439 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.034583 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.034478 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.034766 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.034742 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:27:59.034819 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.034751 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.035021 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.035000 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 19 15:27:59.035143 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.035122 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:27:59.035213 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.035161 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:27:59.035268 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.035204 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.035772 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.035744 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.036469 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.036443 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-config" (OuterVolumeSpecName: "config") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.036876 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.036856 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-kube-api-access-rkcf8" (OuterVolumeSpecName: "kube-api-access-rkcf8") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "kube-api-access-rkcf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:27:59.037159 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.037143 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.037322 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.037310 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.037399 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.037377 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-config-out" (OuterVolumeSpecName: "config-out") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 19 15:27:59.045942 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.045921 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-web-config" (OuterVolumeSpecName: "web-config") pod "b870f93c-f6f0-4b6f-836d-ad46f917224f" (UID: "b870f93c-f6f0-4b6f-836d-ad46f917224f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133554 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-metrics-client-certs\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133593 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133605 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133615 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133644 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-config-out\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133654 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133663 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133672 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.133667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133682 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-web-config\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133692 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133700 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-tls-assets\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133709 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-grpc-tls\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133717 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b870f93c-f6f0-4b6f-836d-ad46f917224f-configmap-metrics-client-ca\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133728 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-config\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133737 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkcf8\" (UniqueName: \"kubernetes.io/projected/b870f93c-f6f0-4b6f-836d-ad46f917224f-kube-api-access-rkcf8\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133746 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133756 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b870f93c-f6f0-4b6f-836d-ad46f917224f-secret-kube-rbac-proxy\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.134030 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.133764 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b870f93c-f6f0-4b6f-836d-ad46f917224f-prometheus-k8s-db\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:27:59.578051 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578012 2576 generic.go:358] "Generic (PLEG): container finished" podID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" exitCode=0 Apr 19 15:27:59.578051 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578042 2576 generic.go:358] "Generic (PLEG): container finished" podID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" exitCode=0 Apr 19 15:27:59.578051 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578053 2576 generic.go:358] "Generic (PLEG): container finished" podID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" exitCode=0 Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578061 2576 generic.go:358] "Generic (PLEG): container finished" podID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" exitCode=0 Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578068 2576 generic.go:358] "Generic (PLEG): container finished" podID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" exitCode=0 Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578076 2576 generic.go:358] "Generic (PLEG): container finished" podID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" exitCode=0 Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578096 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578138 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578153 2576 scope.go:117] "RemoveContainer" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578141 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578278 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578296 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578312 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578326 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} Apr 19 15:27:59.578545 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.578339 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b870f93c-f6f0-4b6f-836d-ad46f917224f","Type":"ContainerDied","Data":"3193a1d4bf2c986b359ab3753a2e913b21c813587c2ba9b69dd4cc0eb7129b0f"} Apr 19 15:27:59.586138 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.586111 2576 scope.go:117] "RemoveContainer" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.595756 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.595735 2576 scope.go:117] "RemoveContainer" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.602054 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.602037 2576 scope.go:117] "RemoveContainer" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.605922 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.605900 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:27:59.609412 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.609329 2576 scope.go:117] "RemoveContainer" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.611885 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.611861 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:27:59.615648 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.615612 2576 scope.go:117] "RemoveContainer" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.622183 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.622167 2576 scope.go:117] "RemoveContainer" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.628491 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.628474 2576 scope.go:117] "RemoveContainer" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.628779 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:59.628759 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": container with ID starting with c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1 not found: ID does not exist" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.628830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.628788 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} err="failed to get container status \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": rpc error: code = NotFound desc = could not find container \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": container with ID starting with c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1 not found: ID does not exist" Apr 19 15:27:59.628830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.628807 2576 scope.go:117] "RemoveContainer" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.629068 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:59.629050 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": container with ID starting with c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7 not found: ID does not exist" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.629113 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629073 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} err="failed to get container status \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": rpc error: code = NotFound desc = could not find container \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": container with ID starting with c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7 not found: ID does not exist" Apr 19 15:27:59.629113 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629089 2576 scope.go:117] "RemoveContainer" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.629309 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:59.629293 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": container with ID starting with 6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3 not found: ID does not exist" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.629358 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629314 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} err="failed to get container status \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": rpc error: code = NotFound desc = could not find container \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": container with ID starting with 6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3 not found: ID does not exist" Apr 19 15:27:59.629358 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629329 2576 scope.go:117] "RemoveContainer" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.629544 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:59.629526 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": container with ID starting with afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5 not found: ID does not exist" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.629580 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629548 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} err="failed to get container status \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": rpc error: code = NotFound desc = could not find container \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": container with ID starting with afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5 not found: ID does not exist" Apr 19 15:27:59.629580 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629560 2576 scope.go:117] "RemoveContainer" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.629824 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:59.629798 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": container with ID starting with 1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22 not found: ID does not exist" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.629824 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629817 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} err="failed to get container status \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": rpc error: code = NotFound desc = could not find container \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": container with ID starting with 1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22 not found: ID does not exist" Apr 19 15:27:59.629955 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.629830 2576 scope.go:117] "RemoveContainer" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.630054 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:59.630038 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": container with ID starting with 8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45 not found: ID does not exist" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.630101 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630059 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} err="failed to get container status \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": rpc error: code = NotFound desc = could not find container \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": container with ID starting with 8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45 not found: ID does not exist" Apr 19 15:27:59.630101 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630073 2576 scope.go:117] "RemoveContainer" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.630287 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:27:59.630271 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": container with ID starting with 234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f not found: ID does not exist" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.630328 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630290 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f"} err="failed to get container status \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": rpc error: code = NotFound desc = could not find container \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": container with ID starting with 234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f not found: ID does not exist" Apr 19 15:27:59.630328 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630303 2576 scope.go:117] "RemoveContainer" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.630522 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630503 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} err="failed to get container status \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": rpc error: code = NotFound desc = could not find container \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": container with ID starting with c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1 not found: ID does not exist" Apr 19 15:27:59.630569 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630524 2576 scope.go:117] "RemoveContainer" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.630762 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630744 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} err="failed to get container status \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": rpc error: code = NotFound desc = could not find container \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": container with ID starting with c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7 not found: ID does not exist" Apr 19 15:27:59.630833 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630763 2576 scope.go:117] "RemoveContainer" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.630965 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630951 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} err="failed to get container status \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": rpc error: code = NotFound desc = could not find container \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": container with ID starting with 6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3 not found: ID does not exist" Apr 19 15:27:59.630965 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.630965 2576 scope.go:117] "RemoveContainer" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.631171 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631152 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} err="failed to get container status \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": rpc error: code = NotFound desc = could not find container \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": container with ID starting with afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5 not found: ID does not exist" Apr 19 15:27:59.631211 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631172 2576 scope.go:117] "RemoveContainer" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.631376 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631358 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} err="failed to get container status \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": rpc error: code = NotFound desc = could not find container \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": container with ID starting with 1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22 not found: ID does not exist" Apr 19 15:27:59.631418 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631378 2576 scope.go:117] "RemoveContainer" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.631572 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631555 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} err="failed to get container status \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": rpc error: code = NotFound desc = could not find container \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": container with ID starting with 8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45 not found: ID does not exist" Apr 19 15:27:59.631610 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631573 2576 scope.go:117] "RemoveContainer" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.631778 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631759 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f"} err="failed to get container status \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": rpc error: code = NotFound desc = could not find container \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": container with ID starting with 234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f not found: ID does not exist" Apr 19 15:27:59.631817 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.631780 2576 scope.go:117] "RemoveContainer" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.632062 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.632031 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} err="failed to get container status \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": rpc error: code = NotFound desc = could not find container \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": container with ID starting with c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1 not found: ID does not exist" Apr 19 15:27:59.632149 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.632063 2576 scope.go:117] "RemoveContainer" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.632562 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.632426 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} err="failed to get container status \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": rpc error: code = NotFound desc = could not find container \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": container with ID starting with c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7 not found: ID does not exist" Apr 19 15:27:59.632562 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.632484 2576 scope.go:117] "RemoveContainer" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.632767 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.632747 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} err="failed to get container status \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": rpc error: code = NotFound desc = could not find container \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": container with ID starting with 6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3 not found: ID does not exist" Apr 19 15:27:59.632767 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.632766 2576 scope.go:117] "RemoveContainer" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.633009 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.632989 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} err="failed to get container status \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": rpc error: code = NotFound desc = could not find container \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": container with ID starting with afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5 not found: ID does not exist" Apr 19 15:27:59.633009 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633008 2576 scope.go:117] "RemoveContainer" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.633224 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633204 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} err="failed to get container status \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": rpc error: code = NotFound desc = could not find container \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": container with ID starting with 1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22 not found: ID does not exist" Apr 19 15:27:59.633302 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633225 2576 scope.go:117] "RemoveContainer" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.633464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633431 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} err="failed to get container status \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": rpc error: code = NotFound desc = could not find container \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": container with ID starting with 8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45 not found: ID does not exist" Apr 19 15:27:59.633464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633462 2576 scope.go:117] "RemoveContainer" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.633735 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633717 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f"} err="failed to get container status \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": rpc error: code = NotFound desc = could not find container \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": container with ID starting with 234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f not found: ID does not exist" Apr 19 15:27:59.633809 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633736 2576 scope.go:117] "RemoveContainer" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.633920 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633902 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:27:59.633972 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633935 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} err="failed to get container status \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": rpc error: code = NotFound desc = could not find container \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": container with ID starting with c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1 not found: ID does not exist" Apr 19 15:27:59.633972 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.633956 2576 scope.go:117] "RemoveContainer" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.634222 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634208 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} err="failed to get container status \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": rpc error: code = NotFound desc = could not find container \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": container with ID starting with c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7 not found: ID does not exist" Apr 19 15:27:59.634222 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634223 2576 scope.go:117] "RemoveContainer" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.634324 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634309 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="thanos-sidecar" Apr 19 15:27:59.634324 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634326 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="thanos-sidecar" Apr 19 15:27:59.634324 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634342 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-thanos" Apr 19 15:27:59.634324 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634350 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-thanos" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634364 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="config-reloader" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634373 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="config-reloader" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634384 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="init-config-reloader" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634392 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="init-config-reloader" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634402 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634410 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634432 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-web" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634440 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-web" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634454 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="prometheus" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634462 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="prometheus" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634464 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} err="failed to get container status \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": rpc error: code = NotFound desc = could not find container \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": container with ID starting with 6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3 not found: ID does not exist" Apr 19 15:27:59.634487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634483 2576 scope.go:117] "RemoveContainer" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634540 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="thanos-sidecar" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634553 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="prometheus" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634563 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-web" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634572 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy-thanos" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634582 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="kube-rbac-proxy" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634593 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" containerName="config-reloader" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634752 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} err="failed to get container status \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": rpc error: code = NotFound desc = could not find container \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": container with ID starting with afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5 not found: ID does not exist" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.634776 2576 scope.go:117] "RemoveContainer" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.635024 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.635018 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} err="failed to get container status \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": rpc error: code = NotFound desc = could not find container \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": container with ID starting with 1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22 not found: ID does not exist" Apr 19 15:27:59.635413 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.635038 2576 scope.go:117] "RemoveContainer" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.635413 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.635263 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} err="failed to get container status \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": rpc error: code = NotFound desc = could not find container \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": container with ID starting with 8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45 not found: ID does not exist" Apr 19 15:27:59.635413 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.635308 2576 scope.go:117] "RemoveContainer" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.635554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.635508 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f"} err="failed to get container status \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": rpc error: code = NotFound desc = could not find container \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": container with ID starting with 234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f not found: ID does not exist" Apr 19 15:27:59.635554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.635526 2576 scope.go:117] "RemoveContainer" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.636081 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.635993 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} err="failed to get container status \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": rpc error: code = NotFound desc = could not find container \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": container with ID starting with c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1 not found: ID does not exist" Apr 19 15:27:59.636081 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.636033 2576 scope.go:117] "RemoveContainer" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.636320 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.636298 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} err="failed to get container status \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": rpc error: code = NotFound desc = could not find container \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": container with ID starting with c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7 not found: ID does not exist" Apr 19 15:27:59.636366 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.636321 2576 scope.go:117] "RemoveContainer" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.636566 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.636548 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} err="failed to get container status \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": rpc error: code = NotFound desc = could not find container \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": container with ID starting with 6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3 not found: ID does not exist" Apr 19 15:27:59.636613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.636568 2576 scope.go:117] "RemoveContainer" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.636782 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.636759 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} err="failed to get container status \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": rpc error: code = NotFound desc = could not find container \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": container with ID starting with afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5 not found: ID does not exist" Apr 19 15:27:59.636830 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.636787 2576 scope.go:117] "RemoveContainer" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.637668 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.637641 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} err="failed to get container status \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": rpc error: code = NotFound desc = could not find container \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": container with ID starting with 1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22 not found: ID does not exist" Apr 19 15:27:59.637668 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.637669 2576 scope.go:117] "RemoveContainer" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.637947 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.637925 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} err="failed to get container status \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": rpc error: code = NotFound desc = could not find container \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": container with ID starting with 8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45 not found: ID does not exist" Apr 19 15:27:59.637995 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.637947 2576 scope.go:117] "RemoveContainer" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.638189 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638170 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f"} err="failed to get container status \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": rpc error: code = NotFound desc = could not find container \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": container with ID starting with 234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f not found: ID does not exist" Apr 19 15:27:59.638228 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638190 2576 scope.go:117] "RemoveContainer" containerID="c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1" Apr 19 15:27:59.638408 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638388 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1"} err="failed to get container status \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": rpc error: code = NotFound desc = could not find container \"c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1\": container with ID starting with c59012951ef2a9add5bb7aa54c0767339ee65b091eb87ae2c9616dcee16350d1 not found: ID does not exist" Apr 19 15:27:59.638453 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638411 2576 scope.go:117] "RemoveContainer" containerID="c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7" Apr 19 15:27:59.638663 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638641 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7"} err="failed to get container status \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": rpc error: code = NotFound desc = could not find container \"c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7\": container with ID starting with c9667280f9259de374a4b4b56d7a2f22f909a19fb3fde37a1ddcf973385c57c7 not found: ID does not exist" Apr 19 15:27:59.638663 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638662 2576 scope.go:117] "RemoveContainer" containerID="6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3" Apr 19 15:27:59.638882 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638866 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3"} err="failed to get container status \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": rpc error: code = NotFound desc = could not find container \"6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3\": container with ID starting with 6be863c8c5b6166abca1524e6dab34a6e7abfedf1c834256e43dd84ea47dbcf3 not found: ID does not exist" Apr 19 15:27:59.638937 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.638883 2576 scope.go:117] "RemoveContainer" containerID="afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5" Apr 19 15:27:59.639092 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.639077 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5"} err="failed to get container status \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": rpc error: code = NotFound desc = could not find container \"afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5\": container with ID starting with afea738979dc9868b89a27c693dc4dc16ede347b76d303d390a9251fb14757e5 not found: ID does not exist" Apr 19 15:27:59.639135 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.639093 2576 scope.go:117] "RemoveContainer" containerID="1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22" Apr 19 15:27:59.639289 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.639274 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22"} err="failed to get container status \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": rpc error: code = NotFound desc = could not find container \"1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22\": container with ID starting with 1878c3049c0d310c43912d4a0f4bb04a7230ce476f1d1c9485293a8dbe805b22 not found: ID does not exist" Apr 19 15:27:59.639326 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.639290 2576 scope.go:117] "RemoveContainer" containerID="8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45" Apr 19 15:27:59.639494 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.639473 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45"} err="failed to get container status \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": rpc error: code = NotFound desc = could not find container \"8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45\": container with ID starting with 8895270cded1ff37203dbe530ec55ead862a817069b6ca69e076d6053dd0ad45 not found: ID does not exist" Apr 19 15:27:59.639566 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.639496 2576 scope.go:117] "RemoveContainer" containerID="234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f" Apr 19 15:27:59.639716 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.639699 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f"} err="failed to get container status \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": rpc error: code = NotFound desc = could not find container \"234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f\": container with ID starting with 234bd25309897168d11a007f8110ac0f003ab91e9dd09fdab6375c564d275c2f not found: ID does not exist" Apr 19 15:27:59.640592 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.640578 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.643154 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643131 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 19 15:27:59.643254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643151 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 19 15:27:59.643254 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643224 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 19 15:27:59.643410 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643391 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3cr70q92qqbb4\"" Apr 19 15:27:59.643521 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643503 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-64mhl\"" Apr 19 15:27:59.643616 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643600 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 19 15:27:59.643869 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643850 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 19 15:27:59.643948 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643922 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 19 15:27:59.644008 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643928 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 19 15:27:59.644008 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.643958 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 19 15:27:59.644099 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.644058 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 19 15:27:59.644099 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.644077 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 19 15:27:59.644192 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.644077 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Apr 19 15:27:59.646573 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.646555 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 19 15:27:59.648894 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.648769 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 19 15:27:59.650478 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.650457 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:27:59.738233 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7c53410-373e-4e5b-9ed1-9635d81f405e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738233 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738438 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738253 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-config\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738438 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738314 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738438 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738375 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738438 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738439 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738468 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7c53410-373e-4e5b-9ed1-9635d81f405e-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738492 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738519 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738541 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738564 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738747 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738573 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738747 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738592 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738747 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738608 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738747 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738649 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738747 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tnzt\" (UniqueName: \"kubernetes.io/projected/d7c53410-373e-4e5b-9ed1-9635d81f405e-kube-api-access-8tnzt\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.738747 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.738704 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.839912 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.839852 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.840089 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.839944 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.840089 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.839974 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.840089 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.839993 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.840089 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840015 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.840089 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840032 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.840089 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840049 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840453 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8tnzt\" (UniqueName: \"kubernetes.io/projected/d7c53410-373e-4e5b-9ed1-9635d81f405e-kube-api-access-8tnzt\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840500 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840577 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7c53410-373e-4e5b-9ed1-9635d81f405e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840604 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840656 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-config\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840700 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840715 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840753 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840778 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840806 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840837 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7c53410-373e-4e5b-9ed1-9635d81f405e-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.841584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.840863 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.842283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.842150 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.843198 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.843170 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.843311 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.843221 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.843311 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.843274 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-web-config\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.843436 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.843318 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.843521 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.843494 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.844100 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.844073 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.844581 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.844555 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.845060 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.844973 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7c53410-373e-4e5b-9ed1-9635d81f405e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.845288 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.845237 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7c53410-373e-4e5b-9ed1-9635d81f405e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.845428 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.845405 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.845746 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.845728 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-config\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.845908 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.845888 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.846009 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.845988 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.846048 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.846024 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d7c53410-373e-4e5b-9ed1-9635d81f405e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.846267 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.846250 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7c53410-373e-4e5b-9ed1-9635d81f405e-config-out\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.848240 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.848218 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tnzt\" (UniqueName: \"kubernetes.io/projected/d7c53410-373e-4e5b-9ed1-9635d81f405e-kube-api-access-8tnzt\") pod \"prometheus-k8s-0\" (UID: \"d7c53410-373e-4e5b-9ed1-9635d81f405e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:27:59.952126 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:27:59.952067 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:28:00.078509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:00.078402 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 19 15:28:00.080603 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:28:00.080574 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c53410_373e_4e5b_9ed1_9635d81f405e.slice/crio-a0ec78c194d914fb4879c321b668c40f551902d133144e3f9ef65adec1b5e4b3 WatchSource:0}: Error finding container a0ec78c194d914fb4879c321b668c40f551902d133144e3f9ef65adec1b5e4b3: Status 404 returned error can't find the container with id a0ec78c194d914fb4879c321b668c40f551902d133144e3f9ef65adec1b5e4b3 Apr 19 15:28:00.583375 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:00.583341 2576 generic.go:358] "Generic (PLEG): container finished" podID="d7c53410-373e-4e5b-9ed1-9635d81f405e" containerID="ddbaaad75daecdf4f1490a3386ab43554714b9b422480b20cb987e292bc1a2cd" exitCode=0 Apr 19 15:28:00.583753 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:00.583389 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerDied","Data":"ddbaaad75daecdf4f1490a3386ab43554714b9b422480b20cb987e292bc1a2cd"} Apr 19 15:28:00.583753 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:00.583412 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerStarted","Data":"a0ec78c194d914fb4879c321b668c40f551902d133144e3f9ef65adec1b5e4b3"} Apr 19 15:28:00.744693 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:00.743694 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b870f93c-f6f0-4b6f-836d-ad46f917224f" path="/var/lib/kubelet/pods/b870f93c-f6f0-4b6f-836d-ad46f917224f/volumes" Apr 19 15:28:01.589690 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:01.589657 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerStarted","Data":"bfb28553fe386be75a2b6958918417023a789add932ae9489da9d4f560256403"} Apr 19 15:28:01.589690 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:01.589692 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerStarted","Data":"abcff8cf8f57c60488120bd8d6e63784cc08766a47b02f3736d473e2f530d437"} Apr 19 15:28:01.590210 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:01.589706 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerStarted","Data":"3487763be335968ca1f7ca2de77cbf11617b13b7de7329dd448e496d74dc4b38"} Apr 19 15:28:01.590210 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:01.589717 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerStarted","Data":"c80f89751197023e368c67af469c8b9fc8671a3b703e415b30cf324d0fcd3edf"} Apr 19 15:28:01.590210 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:01.589727 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerStarted","Data":"2c06a04e38548e8da60be01c84bba9fe274f880c56daa32445ec2613eec54410"} Apr 19 15:28:01.590210 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:01.589737 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d7c53410-373e-4e5b-9ed1-9635d81f405e","Type":"ContainerStarted","Data":"31f651f64ebd026401aac3cfc2916a7df6c32dcf31ad3279c5b36ee9499b0de8"} Apr 19 15:28:01.615558 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:01.615506 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.615490936 podStartE2EDuration="2.615490936s" podCreationTimestamp="2026-04-19 15:27:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-19 15:28:01.613219377 +0000 UTC m=+215.542679562" watchObservedRunningTime="2026-04-19 15:28:01.615490936 +0000 UTC m=+215.544951140" Apr 19 15:28:04.952268 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:04.952211 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:28:59.953171 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:59.953065 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:28:59.968444 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:28:59.968416 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:29:00.783487 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:00.783461 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 19 15:29:26.593917 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:26.593877 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:29:26.594712 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:26.594691 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:29:26.604227 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:26.604206 2576 kubelet.go:1628] "Image garbage collection succeeded" Apr 19 15:29:54.819382 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.819339 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh"] Apr 19 15:29:54.822721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.822705 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:54.831133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.831116 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Apr 19 15:29:54.831443 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.831425 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Apr 19 15:29:54.831515 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.831425 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Apr 19 15:29:54.831515 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.831497 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Apr 19 15:29:54.831515 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.831506 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pcf6s\"" Apr 19 15:29:54.844502 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.844472 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh"] Apr 19 15:29:54.919020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.918977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/836ff1ff-0345-4d96-b088-44d0cac8a98e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:54.919212 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.919048 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqxjg\" (UniqueName: \"kubernetes.io/projected/836ff1ff-0345-4d96-b088-44d0cac8a98e-kube-api-access-cqxjg\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:54.919212 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:54.919113 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/836ff1ff-0345-4d96-b088-44d0cac8a98e-webhook-cert\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.019520 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.019490 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/836ff1ff-0345-4d96-b088-44d0cac8a98e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.019719 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.019547 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cqxjg\" (UniqueName: \"kubernetes.io/projected/836ff1ff-0345-4d96-b088-44d0cac8a98e-kube-api-access-cqxjg\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.019719 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.019584 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/836ff1ff-0345-4d96-b088-44d0cac8a98e-webhook-cert\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.021784 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.021758 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/836ff1ff-0345-4d96-b088-44d0cac8a98e-webhook-cert\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.022066 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.022045 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/836ff1ff-0345-4d96-b088-44d0cac8a98e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.027388 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.027364 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqxjg\" (UniqueName: \"kubernetes.io/projected/836ff1ff-0345-4d96-b088-44d0cac8a98e-kube-api-access-cqxjg\") pod \"opendatahub-operator-controller-manager-67944f454b-8lkwh\" (UID: \"836ff1ff-0345-4d96-b088-44d0cac8a98e\") " pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.132782 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.132749 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:55.263776 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.263756 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh"] Apr 19 15:29:55.266707 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:29:55.266674 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836ff1ff_0345_4d96_b088_44d0cac8a98e.slice/crio-eff42d389dd25a058411e15b936c3a832e63838c6d89ce54468f1e631bab9b6d WatchSource:0}: Error finding container eff42d389dd25a058411e15b936c3a832e63838c6d89ce54468f1e631bab9b6d: Status 404 returned error can't find the container with id eff42d389dd25a058411e15b936c3a832e63838c6d89ce54468f1e631bab9b6d Apr 19 15:29:55.268421 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.268401 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 19 15:29:55.935083 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:55.935045 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" event={"ID":"836ff1ff-0345-4d96-b088-44d0cac8a98e","Type":"ContainerStarted","Data":"eff42d389dd25a058411e15b936c3a832e63838c6d89ce54468f1e631bab9b6d"} Apr 19 15:29:58.945979 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:58.945940 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" event={"ID":"836ff1ff-0345-4d96-b088-44d0cac8a98e","Type":"ContainerStarted","Data":"cff02f5fbb01ee76ac56125d5ba59967b43eb1f1964f7c8c4a1f4dc962b993f7"} Apr 19 15:29:58.946394 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:58.946056 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:29:58.965464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:29:58.965419 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" podStartSLOduration=2.290911708 podStartE2EDuration="4.965405458s" podCreationTimestamp="2026-04-19 15:29:54 +0000 UTC" firstStartedPulling="2026-04-19 15:29:55.268563952 +0000 UTC m=+329.198024092" lastFinishedPulling="2026-04-19 15:29:57.943057701 +0000 UTC m=+331.872517842" observedRunningTime="2026-04-19 15:29:58.964023732 +0000 UTC m=+332.893483895" watchObservedRunningTime="2026-04-19 15:29:58.965405458 +0000 UTC m=+332.894865617" Apr 19 15:30:09.951462 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:09.951432 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67944f454b-8lkwh" Apr 19 15:30:12.263400 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.263353 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l"] Apr 19 15:30:12.267017 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.267000 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.277080 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.277063 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Apr 19 15:30:12.278511 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.278495 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Apr 19 15:30:12.278898 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.278884 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Apr 19 15:30:12.279306 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.279291 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Apr 19 15:30:12.279343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.279289 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8s5vj\"" Apr 19 15:30:12.279520 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.279505 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Apr 19 15:30:12.293549 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.293525 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l"] Apr 19 15:30:12.374841 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.374806 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38ac9bf7-f584-4ceb-8d54-01fae655f31c-cert\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.375023 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.374870 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcqgm\" (UniqueName: \"kubernetes.io/projected/38ac9bf7-f584-4ceb-8d54-01fae655f31c-kube-api-access-vcqgm\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.375023 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.374895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38ac9bf7-f584-4ceb-8d54-01fae655f31c-metrics-cert\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.375023 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.374920 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38ac9bf7-f584-4ceb-8d54-01fae655f31c-manager-config\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.476175 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.476133 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vcqgm\" (UniqueName: \"kubernetes.io/projected/38ac9bf7-f584-4ceb-8d54-01fae655f31c-kube-api-access-vcqgm\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.476175 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.476176 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38ac9bf7-f584-4ceb-8d54-01fae655f31c-metrics-cert\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.476395 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.476222 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38ac9bf7-f584-4ceb-8d54-01fae655f31c-manager-config\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.476395 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.476250 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38ac9bf7-f584-4ceb-8d54-01fae655f31c-cert\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.476919 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.476893 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/38ac9bf7-f584-4ceb-8d54-01fae655f31c-manager-config\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.478903 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.478881 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/38ac9bf7-f584-4ceb-8d54-01fae655f31c-cert\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.478953 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.478929 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/38ac9bf7-f584-4ceb-8d54-01fae655f31c-metrics-cert\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.486765 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.486740 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcqgm\" (UniqueName: \"kubernetes.io/projected/38ac9bf7-f584-4ceb-8d54-01fae655f31c-kube-api-access-vcqgm\") pod \"lws-controller-manager-65b77f8fcd-7zg6l\" (UID: \"38ac9bf7-f584-4ceb-8d54-01fae655f31c\") " pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.576230 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.576123 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:12.716215 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.716175 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l"] Apr 19 15:30:12.718122 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:30:12.718091 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38ac9bf7_f584_4ceb_8d54_01fae655f31c.slice/crio-64fbe7c30ef07c56cfc3595d4b6fbe96e8c064d5604eae885516712fab04cca6 WatchSource:0}: Error finding container 64fbe7c30ef07c56cfc3595d4b6fbe96e8c064d5604eae885516712fab04cca6: Status 404 returned error can't find the container with id 64fbe7c30ef07c56cfc3595d4b6fbe96e8c064d5604eae885516712fab04cca6 Apr 19 15:30:12.994281 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:12.994186 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" event={"ID":"38ac9bf7-f584-4ceb-8d54-01fae655f31c","Type":"ContainerStarted","Data":"64fbe7c30ef07c56cfc3595d4b6fbe96e8c064d5604eae885516712fab04cca6"} Apr 19 15:30:17.014858 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:17.014806 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" event={"ID":"38ac9bf7-f584-4ceb-8d54-01fae655f31c","Type":"ContainerStarted","Data":"23591761a6137dd8ded085b60914cd44cf882444d55038ec66e03070f2470abb"} Apr 19 15:30:17.015261 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:17.014924 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:30:17.032693 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:17.032641 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" podStartSLOduration=1.425821073 podStartE2EDuration="5.032612737s" podCreationTimestamp="2026-04-19 15:30:12 +0000 UTC" firstStartedPulling="2026-04-19 15:30:12.719889823 +0000 UTC m=+346.649349963" lastFinishedPulling="2026-04-19 15:30:16.326681484 +0000 UTC m=+350.256141627" observedRunningTime="2026-04-19 15:30:17.030490331 +0000 UTC m=+350.959950493" watchObservedRunningTime="2026-04-19 15:30:17.032612737 +0000 UTC m=+350.962072898" Apr 19 15:30:18.285465 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.285433 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l"] Apr 19 15:30:18.288837 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.288816 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.291355 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.291330 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xd27t\"" Apr 19 15:30:18.291355 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.291347 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Apr 19 15:30:18.291355 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.291363 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Apr 19 15:30:18.296648 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.296345 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l"] Apr 19 15:30:18.426337 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.426301 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b5378ca-a08d-43fd-ad45-35a6fa21f495-tmp\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.426525 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.426361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5378ca-a08d-43fd-ad45-35a6fa21f495-tls-certs\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.426525 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.426392 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm5zx\" (UniqueName: \"kubernetes.io/projected/0b5378ca-a08d-43fd-ad45-35a6fa21f495-kube-api-access-nm5zx\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.527869 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.527831 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b5378ca-a08d-43fd-ad45-35a6fa21f495-tmp\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.528046 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.527961 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5378ca-a08d-43fd-ad45-35a6fa21f495-tls-certs\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.528046 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.528012 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nm5zx\" (UniqueName: \"kubernetes.io/projected/0b5378ca-a08d-43fd-ad45-35a6fa21f495-kube-api-access-nm5zx\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.529980 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.529951 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b5378ca-a08d-43fd-ad45-35a6fa21f495-tmp\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.530292 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.530270 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b5378ca-a08d-43fd-ad45-35a6fa21f495-tls-certs\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.535579 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.535520 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm5zx\" (UniqueName: \"kubernetes.io/projected/0b5378ca-a08d-43fd-ad45-35a6fa21f495-kube-api-access-nm5zx\") pod \"kube-auth-proxy-745fb64bc5-2s47l\" (UID: \"0b5378ca-a08d-43fd-ad45-35a6fa21f495\") " pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.599690 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.599653 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" Apr 19 15:30:18.716293 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:18.716269 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l"] Apr 19 15:30:18.718682 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:30:18.718655 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b5378ca_a08d_43fd_ad45_35a6fa21f495.slice/crio-dd5e997a499057c5d0c2a13c3c1a4f60ae856e445caa6880419e798468d54a56 WatchSource:0}: Error finding container dd5e997a499057c5d0c2a13c3c1a4f60ae856e445caa6880419e798468d54a56: Status 404 returned error can't find the container with id dd5e997a499057c5d0c2a13c3c1a4f60ae856e445caa6880419e798468d54a56 Apr 19 15:30:19.024756 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:19.024721 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" event={"ID":"0b5378ca-a08d-43fd-ad45-35a6fa21f495","Type":"ContainerStarted","Data":"dd5e997a499057c5d0c2a13c3c1a4f60ae856e445caa6880419e798468d54a56"} Apr 19 15:30:23.039270 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:23.039235 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" event={"ID":"0b5378ca-a08d-43fd-ad45-35a6fa21f495","Type":"ContainerStarted","Data":"93e1ecca8e113c0167bc241321ebf3bb7722828349883aa1d6b495157f1e4907"} Apr 19 15:30:23.055795 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:23.055737 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-745fb64bc5-2s47l" podStartSLOduration=1.7114388649999999 podStartE2EDuration="5.055721107s" podCreationTimestamp="2026-04-19 15:30:18 +0000 UTC" firstStartedPulling="2026-04-19 15:30:18.720936642 +0000 UTC m=+352.650396782" lastFinishedPulling="2026-04-19 15:30:22.065218869 +0000 UTC m=+355.994679024" observedRunningTime="2026-04-19 15:30:23.054614623 +0000 UTC m=+356.984074785" watchObservedRunningTime="2026-04-19 15:30:23.055721107 +0000 UTC m=+356.985181269" Apr 19 15:30:28.022380 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:30:28.022346 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65b77f8fcd-7zg6l" Apr 19 15:32:10.310059 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.310021 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk"] Apr 19 15:32:10.313490 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.313471 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:10.317904 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.317883 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ltvvm\"" Apr 19 15:32:10.318020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.317928 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 19 15:32:10.318020 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.317937 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 19 15:32:10.324765 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.324742 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk"] Apr 19 15:32:10.403678 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.403647 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj6rd\" (UniqueName: \"kubernetes.io/projected/5e8dccfa-828f-4b02-a961-7b2151f4d02b-kube-api-access-vj6rd\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:10.403887 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.403770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e8dccfa-828f-4b02-a961-7b2151f4d02b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:10.417022 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.416992 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk"] Apr 19 15:32:10.417266 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:32:10.417246 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-vj6rd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" podUID="5e8dccfa-828f-4b02-a961-7b2151f4d02b" Apr 19 15:32:10.504881 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.504842 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vj6rd\" (UniqueName: \"kubernetes.io/projected/5e8dccfa-828f-4b02-a961-7b2151f4d02b-kube-api-access-vj6rd\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:10.505055 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.504935 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e8dccfa-828f-4b02-a961-7b2151f4d02b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:10.505286 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.505269 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e8dccfa-828f-4b02-a961-7b2151f4d02b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:10.522326 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.522299 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj6rd\" (UniqueName: \"kubernetes.io/projected/5e8dccfa-828f-4b02-a961-7b2151f4d02b-kube-api-access-vj6rd\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:10.885882 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.885849 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk"] Apr 19 15:32:10.894756 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:10.894716 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk"] Apr 19 15:32:11.392674 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.392642 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:11.395062 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.395028 2576 status_manager.go:895] "Failed to get status for pod" podUID="5e8dccfa-828f-4b02-a961-7b2151f4d02b" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" is forbidden: User \"system:node:ip-10-0-130-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-182.ec2.internal' and this object" Apr 19 15:32:11.397838 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.397818 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:11.399915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.399891 2576 status_manager.go:895] "Failed to get status for pod" podUID="5e8dccfa-828f-4b02-a961-7b2151f4d02b" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" is forbidden: User \"system:node:ip-10-0-130-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-182.ec2.internal' and this object" Apr 19 15:32:11.514228 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.514193 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e8dccfa-828f-4b02-a961-7b2151f4d02b-extensions-socket-volume\") pod \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " Apr 19 15:32:11.514414 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.514292 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj6rd\" (UniqueName: \"kubernetes.io/projected/5e8dccfa-828f-4b02-a961-7b2151f4d02b-kube-api-access-vj6rd\") pod \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\" (UID: \"5e8dccfa-828f-4b02-a961-7b2151f4d02b\") " Apr 19 15:32:11.514477 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.514451 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8dccfa-828f-4b02-a961-7b2151f4d02b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5e8dccfa-828f-4b02-a961-7b2151f4d02b" (UID: "5e8dccfa-828f-4b02-a961-7b2151f4d02b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 19 15:32:11.514589 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.514574 2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e8dccfa-828f-4b02-a961-7b2151f4d02b-extensions-socket-volume\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:32:11.516345 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.516317 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e8dccfa-828f-4b02-a961-7b2151f4d02b-kube-api-access-vj6rd" (OuterVolumeSpecName: "kube-api-access-vj6rd") pod "5e8dccfa-828f-4b02-a961-7b2151f4d02b" (UID: "5e8dccfa-828f-4b02-a961-7b2151f4d02b"). InnerVolumeSpecName "kube-api-access-vj6rd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:32:11.615686 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:11.615654 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj6rd\" (UniqueName: \"kubernetes.io/projected/5e8dccfa-828f-4b02-a961-7b2151f4d02b-kube-api-access-vj6rd\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:32:12.395770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:12.395736 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" Apr 19 15:32:12.398043 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:12.398007 2576 status_manager.go:895] "Failed to get status for pod" podUID="5e8dccfa-828f-4b02-a961-7b2151f4d02b" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" is forbidden: User \"system:node:ip-10-0-130-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-182.ec2.internal' and this object" Apr 19 15:32:12.403886 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:12.403860 2576 status_manager.go:895] "Failed to get status for pod" podUID="5e8dccfa-828f-4b02-a961-7b2151f4d02b" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-wh8tk\" is forbidden: User \"system:node:ip-10-0-130-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-182.ec2.internal' and this object" Apr 19 15:32:12.739685 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:32:12.739602 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e8dccfa-828f-4b02-a961-7b2151f4d02b" path="/var/lib/kubelet/pods/5e8dccfa-828f-4b02-a961-7b2151f4d02b/volumes" Apr 19 15:33:09.079419 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.079384 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-4nlv5"] Apr 19 15:33:09.082696 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.082681 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.085317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.085296 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 19 15:33:09.085317 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.085301 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 19 15:33:09.086288 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.086272 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qzbc7\"" Apr 19 15:33:09.086341 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.086273 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Apr 19 15:33:09.089439 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.089417 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-4nlv5"] Apr 19 15:33:09.121666 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.121610 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtnhp\" (UniqueName: \"kubernetes.io/projected/41644fed-c458-44d7-8312-fb9810b2790c-kube-api-access-rtnhp\") pod \"limitador-limitador-7d549b5b-4nlv5\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.121834 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.121714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41644fed-c458-44d7-8312-fb9810b2790c-config-file\") pod \"limitador-limitador-7d549b5b-4nlv5\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.181602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.181562 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-4nlv5"] Apr 19 15:33:09.223140 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.223104 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rtnhp\" (UniqueName: \"kubernetes.io/projected/41644fed-c458-44d7-8312-fb9810b2790c-kube-api-access-rtnhp\") pod \"limitador-limitador-7d549b5b-4nlv5\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.223308 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.223169 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41644fed-c458-44d7-8312-fb9810b2790c-config-file\") pod \"limitador-limitador-7d549b5b-4nlv5\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.223770 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.223751 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41644fed-c458-44d7-8312-fb9810b2790c-config-file\") pod \"limitador-limitador-7d549b5b-4nlv5\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.230596 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.230564 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtnhp\" (UniqueName: \"kubernetes.io/projected/41644fed-c458-44d7-8312-fb9810b2790c-kube-api-access-rtnhp\") pod \"limitador-limitador-7d549b5b-4nlv5\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.393793 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.393762 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:09.486343 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.486313 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:33:09.490888 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.490860 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.496610 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.496582 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:33:09.516044 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.516018 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-4nlv5"] Apr 19 15:33:09.518853 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:33:09.518829 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41644fed_c458_44d7_8312_fb9810b2790c.slice/crio-4571343621b0a8bcf4b5510313b874865a43103e060de003f0a1625e460666ba WatchSource:0}: Error finding container 4571343621b0a8bcf4b5510313b874865a43103e060de003f0a1625e460666ba: Status 404 returned error can't find the container with id 4571343621b0a8bcf4b5510313b874865a43103e060de003f0a1625e460666ba Apr 19 15:33:09.521374 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.521352 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:33:09.526258 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.526238 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm2mf\" (UniqueName: \"kubernetes.io/projected/b145825a-c143-47c1-b575-9a3864bf4030-kube-api-access-nm2mf\") pod \"limitador-limitador-78c99df468-kt8g2\" (UID: \"b145825a-c143-47c1-b575-9a3864bf4030\") " pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.526364 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.526280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b145825a-c143-47c1-b575-9a3864bf4030-config-file\") pod \"limitador-limitador-78c99df468-kt8g2\" (UID: \"b145825a-c143-47c1-b575-9a3864bf4030\") " pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.581198 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.581164 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" event={"ID":"41644fed-c458-44d7-8312-fb9810b2790c","Type":"ContainerStarted","Data":"4571343621b0a8bcf4b5510313b874865a43103e060de003f0a1625e460666ba"} Apr 19 15:33:09.627349 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.627321 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nm2mf\" (UniqueName: \"kubernetes.io/projected/b145825a-c143-47c1-b575-9a3864bf4030-kube-api-access-nm2mf\") pod \"limitador-limitador-78c99df468-kt8g2\" (UID: \"b145825a-c143-47c1-b575-9a3864bf4030\") " pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.627517 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.627365 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b145825a-c143-47c1-b575-9a3864bf4030-config-file\") pod \"limitador-limitador-78c99df468-kt8g2\" (UID: \"b145825a-c143-47c1-b575-9a3864bf4030\") " pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.628019 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.627999 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b145825a-c143-47c1-b575-9a3864bf4030-config-file\") pod \"limitador-limitador-78c99df468-kt8g2\" (UID: \"b145825a-c143-47c1-b575-9a3864bf4030\") " pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.635181 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.635158 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm2mf\" (UniqueName: \"kubernetes.io/projected/b145825a-c143-47c1-b575-9a3864bf4030-kube-api-access-nm2mf\") pod \"limitador-limitador-78c99df468-kt8g2\" (UID: \"b145825a-c143-47c1-b575-9a3864bf4030\") " pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.802713 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.802611 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:09.920722 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:09.920695 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:33:09.923513 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:33:09.923472 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb145825a_c143_47c1_b575_9a3864bf4030.slice/crio-4b587a10a0512d11e4ea702f23d3f53c4e3c98e69cdcce97b35bee27bc21ae6c WatchSource:0}: Error finding container 4b587a10a0512d11e4ea702f23d3f53c4e3c98e69cdcce97b35bee27bc21ae6c: Status 404 returned error can't find the container with id 4b587a10a0512d11e4ea702f23d3f53c4e3c98e69cdcce97b35bee27bc21ae6c Apr 19 15:33:10.088888 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.088852 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-9thnz"] Apr 19 15:33:10.092209 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.092192 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9thnz" Apr 19 15:33:10.094780 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.094758 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-q77m2\"" Apr 19 15:33:10.097364 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.097333 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-9thnz"] Apr 19 15:33:10.130642 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.130595 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpl4k\" (UniqueName: \"kubernetes.io/projected/31e9a570-61cb-45a2-b9d4-a3b6c9a39e56-kube-api-access-gpl4k\") pod \"authorino-7498df8756-9thnz\" (UID: \"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56\") " pod="kuadrant-system/authorino-7498df8756-9thnz" Apr 19 15:33:10.232139 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.232102 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpl4k\" (UniqueName: \"kubernetes.io/projected/31e9a570-61cb-45a2-b9d4-a3b6c9a39e56-kube-api-access-gpl4k\") pod \"authorino-7498df8756-9thnz\" (UID: \"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56\") " pod="kuadrant-system/authorino-7498df8756-9thnz" Apr 19 15:33:10.243563 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.243499 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpl4k\" (UniqueName: \"kubernetes.io/projected/31e9a570-61cb-45a2-b9d4-a3b6c9a39e56-kube-api-access-gpl4k\") pod \"authorino-7498df8756-9thnz\" (UID: \"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56\") " pod="kuadrant-system/authorino-7498df8756-9thnz" Apr 19 15:33:10.412955 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.412479 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9thnz" Apr 19 15:33:10.608365 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.608297 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" event={"ID":"b145825a-c143-47c1-b575-9a3864bf4030","Type":"ContainerStarted","Data":"4b587a10a0512d11e4ea702f23d3f53c4e3c98e69cdcce97b35bee27bc21ae6c"} Apr 19 15:33:10.619214 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:10.619184 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-9thnz"] Apr 19 15:33:10.623319 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:33:10.623286 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31e9a570_61cb_45a2_b9d4_a3b6c9a39e56.slice/crio-8abab703467ceae3b3d9067cb885eb5c44bb53f82c8762afae5cf2ccd1a35267 WatchSource:0}: Error finding container 8abab703467ceae3b3d9067cb885eb5c44bb53f82c8762afae5cf2ccd1a35267: Status 404 returned error can't find the container with id 8abab703467ceae3b3d9067cb885eb5c44bb53f82c8762afae5cf2ccd1a35267 Apr 19 15:33:11.616790 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:11.616685 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9thnz" event={"ID":"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56","Type":"ContainerStarted","Data":"8abab703467ceae3b3d9067cb885eb5c44bb53f82c8762afae5cf2ccd1a35267"} Apr 19 15:33:14.628818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.628787 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" event={"ID":"41644fed-c458-44d7-8312-fb9810b2790c","Type":"ContainerStarted","Data":"ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef"} Apr 19 15:33:14.629261 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.628900 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:14.630163 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.630137 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9thnz" event={"ID":"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56","Type":"ContainerStarted","Data":"200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33"} Apr 19 15:33:14.631443 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.631421 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" event={"ID":"b145825a-c143-47c1-b575-9a3864bf4030","Type":"ContainerStarted","Data":"3470f7766e572449b755b46c5d187cf227e50f5ed4bfe71ba989a7167dc2def9"} Apr 19 15:33:14.631546 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.631511 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:14.646276 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.646167 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" podStartSLOduration=0.757647096 podStartE2EDuration="5.646151563s" podCreationTimestamp="2026-04-19 15:33:09 +0000 UTC" firstStartedPulling="2026-04-19 15:33:09.521506322 +0000 UTC m=+523.450966473" lastFinishedPulling="2026-04-19 15:33:14.410010796 +0000 UTC m=+528.339470940" observedRunningTime="2026-04-19 15:33:14.644475044 +0000 UTC m=+528.573935210" watchObservedRunningTime="2026-04-19 15:33:14.646151563 +0000 UTC m=+528.575611726" Apr 19 15:33:14.667800 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.667750 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" podStartSLOduration=1.191644116 podStartE2EDuration="5.667737119s" podCreationTimestamp="2026-04-19 15:33:09 +0000 UTC" firstStartedPulling="2026-04-19 15:33:09.925712239 +0000 UTC m=+523.855172389" lastFinishedPulling="2026-04-19 15:33:14.401805252 +0000 UTC m=+528.331265392" observedRunningTime="2026-04-19 15:33:14.66672719 +0000 UTC m=+528.596187351" watchObservedRunningTime="2026-04-19 15:33:14.667737119 +0000 UTC m=+528.597197280" Apr 19 15:33:14.683142 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:14.683095 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-9thnz" podStartSLOduration=0.90663159 podStartE2EDuration="4.683081215s" podCreationTimestamp="2026-04-19 15:33:10 +0000 UTC" firstStartedPulling="2026-04-19 15:33:10.62535456 +0000 UTC m=+524.554814707" lastFinishedPulling="2026-04-19 15:33:14.401804189 +0000 UTC m=+528.331264332" observedRunningTime="2026-04-19 15:33:14.682652865 +0000 UTC m=+528.612113024" watchObservedRunningTime="2026-04-19 15:33:14.683081215 +0000 UTC m=+528.612541377" Apr 19 15:33:25.636773 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:25.636743 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:25.637244 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:25.636795 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-kt8g2" Apr 19 15:33:25.701762 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:25.701731 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-4nlv5"] Apr 19 15:33:25.701984 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:25.701937 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" podUID="41644fed-c458-44d7-8312-fb9810b2790c" containerName="limitador" containerID="cri-o://ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef" gracePeriod=30 Apr 19 15:33:26.243835 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.243808 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:26.376478 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.376445 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41644fed-c458-44d7-8312-fb9810b2790c-config-file\") pod \"41644fed-c458-44d7-8312-fb9810b2790c\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " Apr 19 15:33:26.376675 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.376498 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtnhp\" (UniqueName: \"kubernetes.io/projected/41644fed-c458-44d7-8312-fb9810b2790c-kube-api-access-rtnhp\") pod \"41644fed-c458-44d7-8312-fb9810b2790c\" (UID: \"41644fed-c458-44d7-8312-fb9810b2790c\") " Apr 19 15:33:26.376855 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.376826 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41644fed-c458-44d7-8312-fb9810b2790c-config-file" (OuterVolumeSpecName: "config-file") pod "41644fed-c458-44d7-8312-fb9810b2790c" (UID: "41644fed-c458-44d7-8312-fb9810b2790c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 19 15:33:26.378722 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.378683 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41644fed-c458-44d7-8312-fb9810b2790c-kube-api-access-rtnhp" (OuterVolumeSpecName: "kube-api-access-rtnhp") pod "41644fed-c458-44d7-8312-fb9810b2790c" (UID: "41644fed-c458-44d7-8312-fb9810b2790c"). InnerVolumeSpecName "kube-api-access-rtnhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:33:26.477328 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.477294 2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41644fed-c458-44d7-8312-fb9810b2790c-config-file\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:33:26.477328 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.477325 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtnhp\" (UniqueName: \"kubernetes.io/projected/41644fed-c458-44d7-8312-fb9810b2790c-kube-api-access-rtnhp\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:33:26.673546 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.673451 2576 generic.go:358] "Generic (PLEG): container finished" podID="41644fed-c458-44d7-8312-fb9810b2790c" containerID="ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef" exitCode=0 Apr 19 15:33:26.673546 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.673520 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" Apr 19 15:33:26.673546 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.673528 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" event={"ID":"41644fed-c458-44d7-8312-fb9810b2790c","Type":"ContainerDied","Data":"ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef"} Apr 19 15:33:26.674131 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.673562 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-4nlv5" event={"ID":"41644fed-c458-44d7-8312-fb9810b2790c","Type":"ContainerDied","Data":"4571343621b0a8bcf4b5510313b874865a43103e060de003f0a1625e460666ba"} Apr 19 15:33:26.674131 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.673578 2576 scope.go:117] "RemoveContainer" containerID="ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef" Apr 19 15:33:26.682591 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.682567 2576 scope.go:117] "RemoveContainer" containerID="ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef" Apr 19 15:33:26.682996 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:33:26.682974 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef\": container with ID starting with ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef not found: ID does not exist" containerID="ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef" Apr 19 15:33:26.683093 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.683002 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef"} err="failed to get container status \"ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef\": rpc error: code = NotFound desc = could not find container \"ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef\": container with ID starting with ab6f8fff4219885d6abdea89f97fa08372ff12f4901e468656206e4d915eb2ef not found: ID does not exist" Apr 19 15:33:26.696902 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.696869 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-4nlv5"] Apr 19 15:33:26.700845 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.700824 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-4nlv5"] Apr 19 15:33:26.747209 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:26.747174 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41644fed-c458-44d7-8312-fb9810b2790c" path="/var/lib/kubelet/pods/41644fed-c458-44d7-8312-fb9810b2790c/volumes" Apr 19 15:33:41.059079 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.059034 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-92995"] Apr 19 15:33:41.059677 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.059575 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41644fed-c458-44d7-8312-fb9810b2790c" containerName="limitador" Apr 19 15:33:41.059677 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.059599 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41644fed-c458-44d7-8312-fb9810b2790c" containerName="limitador" Apr 19 15:33:41.059802 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.059729 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="41644fed-c458-44d7-8312-fb9810b2790c" containerName="limitador" Apr 19 15:33:41.070484 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.070459 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-92995"] Apr 19 15:33:41.070650 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.070580 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-92995" Apr 19 15:33:41.208490 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.208449 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m89hq\" (UniqueName: \"kubernetes.io/projected/7125ae18-1b23-4bf5-8699-9d63d0a2807b-kube-api-access-m89hq\") pod \"authorino-8b475cf9f-92995\" (UID: \"7125ae18-1b23-4bf5-8699-9d63d0a2807b\") " pod="kuadrant-system/authorino-8b475cf9f-92995" Apr 19 15:33:41.224880 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.224849 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-92995"] Apr 19 15:33:41.225104 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:33:41.225083 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-m89hq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8b475cf9f-92995" podUID="7125ae18-1b23-4bf5-8699-9d63d0a2807b" Apr 19 15:33:41.254976 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.254943 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56fdd757f5-926wj"] Apr 19 15:33:41.258294 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.258278 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.260592 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.260571 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Apr 19 15:33:41.264961 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.264934 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-926wj"] Apr 19 15:33:41.286382 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.286359 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-926wj"] Apr 19 15:33:41.286606 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:33:41.286589 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vtcjk tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-vtcjk tls-cert]: context canceled" pod="kuadrant-system/authorino-56fdd757f5-926wj" podUID="cdf651d9-c7fa-4b00-a810-c8f9f8226b98" Apr 19 15:33:41.309036 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.309011 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m89hq\" (UniqueName: \"kubernetes.io/projected/7125ae18-1b23-4bf5-8699-9d63d0a2807b-kube-api-access-m89hq\") pod \"authorino-8b475cf9f-92995\" (UID: \"7125ae18-1b23-4bf5-8699-9d63d0a2807b\") " pod="kuadrant-system/authorino-8b475cf9f-92995" Apr 19 15:33:41.310294 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.310247 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5fb6576bb5-gwjvc"] Apr 19 15:33:41.313419 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.313402 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.319411 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.319383 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m89hq\" (UniqueName: \"kubernetes.io/projected/7125ae18-1b23-4bf5-8699-9d63d0a2807b-kube-api-access-m89hq\") pod \"authorino-8b475cf9f-92995\" (UID: \"7125ae18-1b23-4bf5-8699-9d63d0a2807b\") " pod="kuadrant-system/authorino-8b475cf9f-92995" Apr 19 15:33:41.320283 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.320259 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fb6576bb5-gwjvc"] Apr 19 15:33:41.410570 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.410533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-tls-cert\") pod \"authorino-56fdd757f5-926wj\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.410766 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.410587 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75j9m\" (UniqueName: \"kubernetes.io/projected/ae93d22f-b41e-45de-a99c-5e98535334ae-kube-api-access-75j9m\") pod \"authorino-5fb6576bb5-gwjvc\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.410766 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.410610 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtcjk\" (UniqueName: \"kubernetes.io/projected/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-kube-api-access-vtcjk\") pod \"authorino-56fdd757f5-926wj\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.410766 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.410728 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae93d22f-b41e-45de-a99c-5e98535334ae-tls-cert\") pod \"authorino-5fb6576bb5-gwjvc\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.511860 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.511801 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae93d22f-b41e-45de-a99c-5e98535334ae-tls-cert\") pod \"authorino-5fb6576bb5-gwjvc\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.512071 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.511879 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-tls-cert\") pod \"authorino-56fdd757f5-926wj\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.512071 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.511924 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-75j9m\" (UniqueName: \"kubernetes.io/projected/ae93d22f-b41e-45de-a99c-5e98535334ae-kube-api-access-75j9m\") pod \"authorino-5fb6576bb5-gwjvc\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.512071 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.511976 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vtcjk\" (UniqueName: \"kubernetes.io/projected/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-kube-api-access-vtcjk\") pod \"authorino-56fdd757f5-926wj\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.514362 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.514338 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-tls-cert\") pod \"authorino-56fdd757f5-926wj\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.514483 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.514425 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae93d22f-b41e-45de-a99c-5e98535334ae-tls-cert\") pod \"authorino-5fb6576bb5-gwjvc\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.519370 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.519345 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75j9m\" (UniqueName: \"kubernetes.io/projected/ae93d22f-b41e-45de-a99c-5e98535334ae-kube-api-access-75j9m\") pod \"authorino-5fb6576bb5-gwjvc\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.519496 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.519444 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtcjk\" (UniqueName: \"kubernetes.io/projected/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-kube-api-access-vtcjk\") pod \"authorino-56fdd757f5-926wj\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.631162 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.631126 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:33:41.722183 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.722156 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-92995" Apr 19 15:33:41.722339 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.722156 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.728119 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.728016 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-92995" Apr 19 15:33:41.732220 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.732201 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:41.749324 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.749250 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5fb6576bb5-gwjvc"] Apr 19 15:33:41.753312 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:33:41.753279 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae93d22f_b41e_45de_a99c_5e98535334ae.slice/crio-d6233fa93735c8800fc02844a0dc98ec1861e934cb23e1c19217f0eb94b89735 WatchSource:0}: Error finding container d6233fa93735c8800fc02844a0dc98ec1861e934cb23e1c19217f0eb94b89735: Status 404 returned error can't find the container with id d6233fa93735c8800fc02844a0dc98ec1861e934cb23e1c19217f0eb94b89735 Apr 19 15:33:41.915460 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.915387 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-tls-cert\") pod \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " Apr 19 15:33:41.915460 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.915423 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtcjk\" (UniqueName: \"kubernetes.io/projected/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-kube-api-access-vtcjk\") pod \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\" (UID: \"cdf651d9-c7fa-4b00-a810-c8f9f8226b98\") " Apr 19 15:33:41.915654 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.915464 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m89hq\" (UniqueName: \"kubernetes.io/projected/7125ae18-1b23-4bf5-8699-9d63d0a2807b-kube-api-access-m89hq\") pod \"7125ae18-1b23-4bf5-8699-9d63d0a2807b\" (UID: \"7125ae18-1b23-4bf5-8699-9d63d0a2807b\") " Apr 19 15:33:41.917493 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.917466 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "cdf651d9-c7fa-4b00-a810-c8f9f8226b98" (UID: "cdf651d9-c7fa-4b00-a810-c8f9f8226b98"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:33:41.917602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.917504 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-kube-api-access-vtcjk" (OuterVolumeSpecName: "kube-api-access-vtcjk") pod "cdf651d9-c7fa-4b00-a810-c8f9f8226b98" (UID: "cdf651d9-c7fa-4b00-a810-c8f9f8226b98"). InnerVolumeSpecName "kube-api-access-vtcjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:33:41.917602 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:41.917509 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7125ae18-1b23-4bf5-8699-9d63d0a2807b-kube-api-access-m89hq" (OuterVolumeSpecName: "kube-api-access-m89hq") pod "7125ae18-1b23-4bf5-8699-9d63d0a2807b" (UID: "7125ae18-1b23-4bf5-8699-9d63d0a2807b"). InnerVolumeSpecName "kube-api-access-m89hq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:33:42.016712 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.016682 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m89hq\" (UniqueName: \"kubernetes.io/projected/7125ae18-1b23-4bf5-8699-9d63d0a2807b-kube-api-access-m89hq\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:33:42.016712 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.016709 2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-tls-cert\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:33:42.016712 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.016719 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtcjk\" (UniqueName: \"kubernetes.io/projected/cdf651d9-c7fa-4b00-a810-c8f9f8226b98-kube-api-access-vtcjk\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:33:42.726889 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.726861 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56fdd757f5-926wj" Apr 19 15:33:42.726889 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.726889 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-92995" Apr 19 15:33:42.727318 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.726861 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" event={"ID":"ae93d22f-b41e-45de-a99c-5e98535334ae","Type":"ContainerStarted","Data":"ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b"} Apr 19 15:33:42.727318 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.726981 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" event={"ID":"ae93d22f-b41e-45de-a99c-5e98535334ae","Type":"ContainerStarted","Data":"d6233fa93735c8800fc02844a0dc98ec1861e934cb23e1c19217f0eb94b89735"} Apr 19 15:33:42.747733 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.747676 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" podStartSLOduration=1.38644261 podStartE2EDuration="1.747656066s" podCreationTimestamp="2026-04-19 15:33:41 +0000 UTC" firstStartedPulling="2026-04-19 15:33:41.754494281 +0000 UTC m=+555.683954421" lastFinishedPulling="2026-04-19 15:33:42.115707723 +0000 UTC m=+556.045167877" observedRunningTime="2026-04-19 15:33:42.745795678 +0000 UTC m=+556.675255845" watchObservedRunningTime="2026-04-19 15:33:42.747656066 +0000 UTC m=+556.677116228" Apr 19 15:33:42.768750 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.768715 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-92995"] Apr 19 15:33:42.775936 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.775907 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-92995"] Apr 19 15:33:42.777516 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.777490 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-9thnz"] Apr 19 15:33:42.777765 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.777731 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-9thnz" podUID="31e9a570-61cb-45a2-b9d4-a3b6c9a39e56" containerName="authorino" containerID="cri-o://200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33" gracePeriod=30 Apr 19 15:33:42.795063 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.795040 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-926wj"] Apr 19 15:33:42.804724 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:42.804699 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56fdd757f5-926wj"] Apr 19 15:33:43.012869 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.012849 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9thnz" Apr 19 15:33:43.126916 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.126879 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpl4k\" (UniqueName: \"kubernetes.io/projected/31e9a570-61cb-45a2-b9d4-a3b6c9a39e56-kube-api-access-gpl4k\") pod \"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56\" (UID: \"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56\") " Apr 19 15:33:43.128920 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.128896 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31e9a570-61cb-45a2-b9d4-a3b6c9a39e56-kube-api-access-gpl4k" (OuterVolumeSpecName: "kube-api-access-gpl4k") pod "31e9a570-61cb-45a2-b9d4-a3b6c9a39e56" (UID: "31e9a570-61cb-45a2-b9d4-a3b6c9a39e56"). InnerVolumeSpecName "kube-api-access-gpl4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:33:43.228330 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.228296 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpl4k\" (UniqueName: \"kubernetes.io/projected/31e9a570-61cb-45a2-b9d4-a3b6c9a39e56-kube-api-access-gpl4k\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:33:43.731683 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.731621 2576 generic.go:358] "Generic (PLEG): container finished" podID="31e9a570-61cb-45a2-b9d4-a3b6c9a39e56" containerID="200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33" exitCode=0 Apr 19 15:33:43.732148 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.731695 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-9thnz" Apr 19 15:33:43.732148 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.731722 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9thnz" event={"ID":"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56","Type":"ContainerDied","Data":"200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33"} Apr 19 15:33:43.732148 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.731758 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-9thnz" event={"ID":"31e9a570-61cb-45a2-b9d4-a3b6c9a39e56","Type":"ContainerDied","Data":"8abab703467ceae3b3d9067cb885eb5c44bb53f82c8762afae5cf2ccd1a35267"} Apr 19 15:33:43.732148 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.731771 2576 scope.go:117] "RemoveContainer" containerID="200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33" Apr 19 15:33:43.740765 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.740742 2576 scope.go:117] "RemoveContainer" containerID="200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33" Apr 19 15:33:43.741032 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:33:43.741010 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33\": container with ID starting with 200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33 not found: ID does not exist" containerID="200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33" Apr 19 15:33:43.741079 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.741042 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33"} err="failed to get container status \"200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33\": rpc error: code = NotFound desc = could not find container \"200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33\": container with ID starting with 200ec43572fd4eb2a48c4c4d6fb232477746143581d859c2a4c9345cc8a74a33 not found: ID does not exist" Apr 19 15:33:43.760468 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.760439 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-9thnz"] Apr 19 15:33:43.766464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:43.766439 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-9thnz"] Apr 19 15:33:44.739247 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:44.739214 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31e9a570-61cb-45a2-b9d4-a3b6c9a39e56" path="/var/lib/kubelet/pods/31e9a570-61cb-45a2-b9d4-a3b6c9a39e56/volumes" Apr 19 15:33:44.739655 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:44.739547 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7125ae18-1b23-4bf5-8699-9d63d0a2807b" path="/var/lib/kubelet/pods/7125ae18-1b23-4bf5-8699-9d63d0a2807b/volumes" Apr 19 15:33:44.739809 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:44.739796 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf651d9-c7fa-4b00-a810-c8f9f8226b98" path="/var/lib/kubelet/pods/cdf651d9-c7fa-4b00-a810-c8f9f8226b98/volumes" Apr 19 15:33:50.082112 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:33:50.082083 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:34:19.090078 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:19.090039 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:34:26.619575 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:26.619543 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:34:26.620019 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:26.619785 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:34:28.012328 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.012292 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps"] Apr 19 15:34:28.012804 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.012696 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31e9a570-61cb-45a2-b9d4-a3b6c9a39e56" containerName="authorino" Apr 19 15:34:28.012804 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.012709 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e9a570-61cb-45a2-b9d4-a3b6c9a39e56" containerName="authorino" Apr 19 15:34:28.012804 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.012780 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="31e9a570-61cb-45a2-b9d4-a3b6c9a39e56" containerName="authorino" Apr 19 15:34:28.016987 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.016965 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.021095 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.021072 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w98zh\"" Apr 19 15:34:28.021266 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.021105 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Apr 19 15:34:28.021266 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.021071 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Apr 19 15:34:28.021266 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.021215 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Apr 19 15:34:28.024187 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.024154 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps"] Apr 19 15:34:28.116728 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.116694 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.116902 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.116738 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.116902 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.116813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb2k5\" (UniqueName: \"kubernetes.io/projected/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-kube-api-access-lb2k5\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.116989 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.116929 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.116989 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.116959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.116989 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.116983 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218165 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218130 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218165 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218168 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218423 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218193 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lb2k5\" (UniqueName: \"kubernetes.io/projected/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-kube-api-access-lb2k5\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218423 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218267 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218423 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218302 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218423 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218333 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218675 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218604 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218729 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218712 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.218782 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.218759 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.220437 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.220416 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.220717 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.220700 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.225243 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.225224 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb2k5\" (UniqueName: \"kubernetes.io/projected/7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e-kube-api-access-lb2k5\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jgkps\" (UID: \"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.327160 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.327078 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:28.455852 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.455823 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps"] Apr 19 15:34:28.458866 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:34:28.458828 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c55559c_b6ba_41cb_abcb_bf0fb4f81f2e.slice/crio-f95be5fc6d61ee4fd6de912470c8a72002f98a5cfd3c58031895f6f7b701296d WatchSource:0}: Error finding container f95be5fc6d61ee4fd6de912470c8a72002f98a5cfd3c58031895f6f7b701296d: Status 404 returned error can't find the container with id f95be5fc6d61ee4fd6de912470c8a72002f98a5cfd3c58031895f6f7b701296d Apr 19 15:34:28.495221 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.495187 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:34:28.880219 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:28.880176 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" event={"ID":"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e","Type":"ContainerStarted","Data":"f95be5fc6d61ee4fd6de912470c8a72002f98a5cfd3c58031895f6f7b701296d"} Apr 19 15:34:33.900466 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:33.900379 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" event={"ID":"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e","Type":"ContainerStarted","Data":"920a6b1948ffafe631e187c6b6511aaaa666ba0c9608436ec80349e6193b6a16"} Apr 19 15:34:36.787083 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:36.787046 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:34:40.503937 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:40.503893 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:34:42.930277 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:42.930199 2576 generic.go:358] "Generic (PLEG): container finished" podID="7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e" containerID="920a6b1948ffafe631e187c6b6511aaaa666ba0c9608436ec80349e6193b6a16" exitCode=0 Apr 19 15:34:42.930673 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:42.930273 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" event={"ID":"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e","Type":"ContainerDied","Data":"920a6b1948ffafe631e187c6b6511aaaa666ba0c9608436ec80349e6193b6a16"} Apr 19 15:34:44.939156 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:44.939119 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" event={"ID":"7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e","Type":"ContainerStarted","Data":"282d76de340951a92093dace858779eaf67c5f4847c8b95df080d146412b0fd6"} Apr 19 15:34:44.939540 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:44.939313 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:34:44.955690 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:44.955639 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" podStartSLOduration=2.462446198 podStartE2EDuration="17.955589015s" podCreationTimestamp="2026-04-19 15:34:27 +0000 UTC" firstStartedPulling="2026-04-19 15:34:28.460842552 +0000 UTC m=+602.390302693" lastFinishedPulling="2026-04-19 15:34:43.953985357 +0000 UTC m=+617.883445510" observedRunningTime="2026-04-19 15:34:44.954532166 +0000 UTC m=+618.883992350" watchObservedRunningTime="2026-04-19 15:34:44.955589015 +0000 UTC m=+618.885049176" Apr 19 15:34:55.956595 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:34:55.956566 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jgkps" Apr 19 15:35:20.808013 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.807979 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4"] Apr 19 15:35:20.811396 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.811380 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.813917 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.813898 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Apr 19 15:35:20.819345 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.819321 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4"] Apr 19 15:35:20.889104 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.889071 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.889262 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.889123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.889262 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.889194 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.889262 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.889246 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbnfr\" (UniqueName: \"kubernetes.io/projected/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-kube-api-access-qbnfr\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.889367 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.889329 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.889412 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.889395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990138 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990102 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990138 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990139 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnfr\" (UniqueName: \"kubernetes.io/projected/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-kube-api-access-qbnfr\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990353 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990167 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990353 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990209 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990353 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990248 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990353 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990291 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990559 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990491 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990659 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990619 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.990736 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.990675 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.992442 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.992421 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.992839 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.992816 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:20.997798 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:20.997777 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbnfr\" (UniqueName: \"kubernetes.io/projected/e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707-kube-api-access-qbnfr\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4\" (UID: \"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:21.123313 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:21.123275 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:21.243789 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:21.243766 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4"] Apr 19 15:35:21.246165 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:35:21.246120 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ce4ee1_b60f_4ddb_9c0a_8dc57832b707.slice/crio-d62d57d070c6f45a3e3bf685470f0e0437933239bc673cbbb10b21b3879e30b6 WatchSource:0}: Error finding container d62d57d070c6f45a3e3bf685470f0e0437933239bc673cbbb10b21b3879e30b6: Status 404 returned error can't find the container with id d62d57d070c6f45a3e3bf685470f0e0437933239bc673cbbb10b21b3879e30b6 Apr 19 15:35:21.248350 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:21.248332 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 19 15:35:22.067410 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:22.067373 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" event={"ID":"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707","Type":"ContainerStarted","Data":"fb16d7c2cdf5b0f6f0ddab2355b3b32544995c670748efef72bd3abc273bad35"} Apr 19 15:35:22.067798 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:22.067419 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" event={"ID":"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707","Type":"ContainerStarted","Data":"d62d57d070c6f45a3e3bf685470f0e0437933239bc673cbbb10b21b3879e30b6"} Apr 19 15:35:22.089065 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:22.089034 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:35:25.112052 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.112020 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7"] Apr 19 15:35:25.115661 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.115643 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.118099 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.118070 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Apr 19 15:35:25.123930 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.123907 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7"] Apr 19 15:35:25.231335 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.231296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.231505 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.231340 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.231505 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.231373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.231505 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.231394 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.231505 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.231428 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/70106617-c01c-4646-bf07-4fe53f6e0025-kube-api-access-2hdh4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.231731 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.231596 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70106617-c01c-4646-bf07-4fe53f6e0025-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.332642 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.332593 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70106617-c01c-4646-bf07-4fe53f6e0025-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.332813 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.332716 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.332813 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.332756 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.332969 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.332938 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.333043 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.332990 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.333104 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.333052 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/70106617-c01c-4646-bf07-4fe53f6e0025-kube-api-access-2hdh4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.333210 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.333189 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.333430 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.333371 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.333430 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.333382 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.335209 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.335186 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70106617-c01c-4646-bf07-4fe53f6e0025-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.335340 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.335322 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70106617-c01c-4646-bf07-4fe53f6e0025-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.340544 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.340516 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/70106617-c01c-4646-bf07-4fe53f6e0025-kube-api-access-2hdh4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7\" (UID: \"70106617-c01c-4646-bf07-4fe53f6e0025\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.427446 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.427357 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:25.556166 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:25.556136 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7"] Apr 19 15:35:25.558583 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:35:25.558553 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70106617_c01c_4646_bf07_4fe53f6e0025.slice/crio-5a3b85e1da36ce3387b5cf45d55ff7c1b09148991e6111b3041e992fd05e2f8a WatchSource:0}: Error finding container 5a3b85e1da36ce3387b5cf45d55ff7c1b09148991e6111b3041e992fd05e2f8a: Status 404 returned error can't find the container with id 5a3b85e1da36ce3387b5cf45d55ff7c1b09148991e6111b3041e992fd05e2f8a Apr 19 15:35:26.082971 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:26.082935 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" event={"ID":"70106617-c01c-4646-bf07-4fe53f6e0025","Type":"ContainerStarted","Data":"48b9da143e286a4c13d6710093602f9a990948c302fc644c97d1cb036e8c8b4f"} Apr 19 15:35:26.082971 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:26.082973 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" event={"ID":"70106617-c01c-4646-bf07-4fe53f6e0025","Type":"ContainerStarted","Data":"5a3b85e1da36ce3387b5cf45d55ff7c1b09148991e6111b3041e992fd05e2f8a"} Apr 19 15:35:26.391238 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:26.391151 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:35:27.087554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:27.087517 2576 generic.go:358] "Generic (PLEG): container finished" podID="e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707" containerID="fb16d7c2cdf5b0f6f0ddab2355b3b32544995c670748efef72bd3abc273bad35" exitCode=0 Apr 19 15:35:27.087734 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:27.087588 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" event={"ID":"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707","Type":"ContainerDied","Data":"fb16d7c2cdf5b0f6f0ddab2355b3b32544995c670748efef72bd3abc273bad35"} Apr 19 15:35:28.092749 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:28.092715 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" event={"ID":"e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707","Type":"ContainerStarted","Data":"2e1113c3f6475af8c495887db545bbf409ec7d82837c84225939b975113e38ad"} Apr 19 15:35:28.093175 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:28.092925 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:28.109804 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:28.109703 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" podStartSLOduration=7.942327594 podStartE2EDuration="8.109686698s" podCreationTimestamp="2026-04-19 15:35:20 +0000 UTC" firstStartedPulling="2026-04-19 15:35:27.088383547 +0000 UTC m=+661.017843687" lastFinishedPulling="2026-04-19 15:35:27.255742639 +0000 UTC m=+661.185202791" observedRunningTime="2026-04-19 15:35:28.108927014 +0000 UTC m=+662.038387174" watchObservedRunningTime="2026-04-19 15:35:28.109686698 +0000 UTC m=+662.039146861" Apr 19 15:35:31.105149 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:31.105113 2576 generic.go:358] "Generic (PLEG): container finished" podID="70106617-c01c-4646-bf07-4fe53f6e0025" containerID="48b9da143e286a4c13d6710093602f9a990948c302fc644c97d1cb036e8c8b4f" exitCode=0 Apr 19 15:35:31.105509 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:31.105188 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" event={"ID":"70106617-c01c-4646-bf07-4fe53f6e0025","Type":"ContainerDied","Data":"48b9da143e286a4c13d6710093602f9a990948c302fc644c97d1cb036e8c8b4f"} Apr 19 15:35:32.110698 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:32.110662 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" event={"ID":"70106617-c01c-4646-bf07-4fe53f6e0025","Type":"ContainerStarted","Data":"4f6580c8e2ce762d9a60e7e17d78d9a6ddab8e89441ff8b74dab75fce4f56fdd"} Apr 19 15:35:32.111079 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:32.110871 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:32.130221 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:32.130165 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" podStartSLOduration=6.957024768 podStartE2EDuration="7.130151115s" podCreationTimestamp="2026-04-19 15:35:25 +0000 UTC" firstStartedPulling="2026-04-19 15:35:31.105818281 +0000 UTC m=+665.035278422" lastFinishedPulling="2026-04-19 15:35:31.278944627 +0000 UTC m=+665.208404769" observedRunningTime="2026-04-19 15:35:32.128564413 +0000 UTC m=+666.058024577" watchObservedRunningTime="2026-04-19 15:35:32.130151115 +0000 UTC m=+666.059611278" Apr 19 15:35:39.110219 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:39.110187 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4" Apr 19 15:35:43.127799 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:43.127767 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7" Apr 19 15:35:51.348381 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.348344 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fb6576bb5-gwjvc"] Apr 19 15:35:51.348895 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.348607 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" podUID="ae93d22f-b41e-45de-a99c-5e98535334ae" containerName="authorino" containerID="cri-o://ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b" gracePeriod=30 Apr 19 15:35:51.595595 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.595573 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:35:51.663932 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.663850 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae93d22f-b41e-45de-a99c-5e98535334ae-tls-cert\") pod \"ae93d22f-b41e-45de-a99c-5e98535334ae\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " Apr 19 15:35:51.663932 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.663894 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75j9m\" (UniqueName: \"kubernetes.io/projected/ae93d22f-b41e-45de-a99c-5e98535334ae-kube-api-access-75j9m\") pod \"ae93d22f-b41e-45de-a99c-5e98535334ae\" (UID: \"ae93d22f-b41e-45de-a99c-5e98535334ae\") " Apr 19 15:35:51.665931 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.665894 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae93d22f-b41e-45de-a99c-5e98535334ae-kube-api-access-75j9m" (OuterVolumeSpecName: "kube-api-access-75j9m") pod "ae93d22f-b41e-45de-a99c-5e98535334ae" (UID: "ae93d22f-b41e-45de-a99c-5e98535334ae"). InnerVolumeSpecName "kube-api-access-75j9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 19 15:35:51.673613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.673588 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae93d22f-b41e-45de-a99c-5e98535334ae-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ae93d22f-b41e-45de-a99c-5e98535334ae" (UID: "ae93d22f-b41e-45de-a99c-5e98535334ae"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 19 15:35:51.765445 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.765413 2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ae93d22f-b41e-45de-a99c-5e98535334ae-tls-cert\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:35:51.765445 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:51.765446 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75j9m\" (UniqueName: \"kubernetes.io/projected/ae93d22f-b41e-45de-a99c-5e98535334ae-kube-api-access-75j9m\") on node \"ip-10-0-130-182.ec2.internal\" DevicePath \"\"" Apr 19 15:35:52.176656 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.176597 2576 generic.go:358] "Generic (PLEG): container finished" podID="ae93d22f-b41e-45de-a99c-5e98535334ae" containerID="ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b" exitCode=0 Apr 19 15:35:52.176857 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.176666 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" Apr 19 15:35:52.176857 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.176674 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" event={"ID":"ae93d22f-b41e-45de-a99c-5e98535334ae","Type":"ContainerDied","Data":"ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b"} Apr 19 15:35:52.176857 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.176710 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5fb6576bb5-gwjvc" event={"ID":"ae93d22f-b41e-45de-a99c-5e98535334ae","Type":"ContainerDied","Data":"d6233fa93735c8800fc02844a0dc98ec1861e934cb23e1c19217f0eb94b89735"} Apr 19 15:35:52.176857 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.176725 2576 scope.go:117] "RemoveContainer" containerID="ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b" Apr 19 15:35:52.185350 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.185333 2576 scope.go:117] "RemoveContainer" containerID="ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b" Apr 19 15:35:52.185595 ip-10-0-130-182 kubenswrapper[2576]: E0419 15:35:52.185578 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b\": container with ID starting with ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b not found: ID does not exist" containerID="ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b" Apr 19 15:35:52.185735 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.185601 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b"} err="failed to get container status \"ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b\": rpc error: code = NotFound desc = could not find container \"ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b\": container with ID starting with ce81da7cba6ec3bc9b7f978d2347444c343fd7c5fcba64d8800372d25526be0b not found: ID does not exist" Apr 19 15:35:52.199340 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.199312 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5fb6576bb5-gwjvc"] Apr 19 15:35:52.201241 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.201222 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5fb6576bb5-gwjvc"] Apr 19 15:35:52.740384 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:35:52.740350 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae93d22f-b41e-45de-a99c-5e98535334ae" path="/var/lib/kubelet/pods/ae93d22f-b41e-45de-a99c-5e98535334ae/volumes" Apr 19 15:36:16.198057 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:36:16.197958 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:36:26.891295 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:36:26.891261 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:36:36.285872 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:36:36.285836 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:36:45.788727 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:36:45.788692 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:36:54.490463 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:36:54.490422 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:37:05.989825 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:37:05.989789 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:38:06.990374 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:38:06.990289 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:38:22.784319 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:38:22.784285 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:39:00.793606 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:39:00.793571 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:39:17.386818 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:39:17.386729 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:39:26.646717 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:39:26.646676 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:39:26.647340 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:39:26.647322 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:39:32.490049 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:39:32.490008 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:39:47.495041 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:39:47.494996 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:40:41.999657 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:40:41.999596 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:40:50.585340 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:40:50.585255 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:41:07.384376 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:41:07.384340 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:41:15.686613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:41:15.686582 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:41:33.502784 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:41:33.502748 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:41:40.890705 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:41:40.890666 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:42:13.397002 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:42:13.396908 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:42:22.503868 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:42:22.503836 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:42:30.410547 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:42:30.410507 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:42:39.187308 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:42:39.187265 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:42:47.482521 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:42:47.482483 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:43:05.290617 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:43:05.290577 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:43:17.790614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:43:17.790569 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:44:04.291978 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:04.291894 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:44:12.685847 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:12.685813 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:44:20.785553 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:20.785519 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:44:26.671638 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:26.671603 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:44:26.675245 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:26.675218 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:44:29.188464 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:29.188429 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:44:39.288889 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:39.288852 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:44:47.682044 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:47.682009 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:44:55.983496 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:44:55.983456 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:01.383968 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:01.383933 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:06.792341 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:06.792304 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:13.788096 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:13.788009 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:22.386479 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:22.386434 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:31.690743 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:31.690707 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:39.183572 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:39.183524 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:47.884198 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:47.884155 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:45:57.085549 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:45:57.085513 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:46:05.686031 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:46:05.685993 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:46:13.688421 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:46:13.688384 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:46:22.694076 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:46:22.694040 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:46:31.298056 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:46:31.298018 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:48:48.691542 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:48:48.691508 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:48:53.689325 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:48:53.689290 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:49:18.892101 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:49:18.892066 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:49:26.291071 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:49:26.291027 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:49:26.700736 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:49:26.700705 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:49:26.703875 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:49:26.703853 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:49:35.693352 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:49:35.693319 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:49:45.479417 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:49:45.479335 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:49:54.886323 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:49:54.886290 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:50:05.889161 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:50:05.889114 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:50:14.100759 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:50:14.100720 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:50:24.996439 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:50:24.996398 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:50:33.094891 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:50:33.093846 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:50:43.694121 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:50:43.694082 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:50:53.201963 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:50:53.201929 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:51:25.797188 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:51:25.797156 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:52:08.390660 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:52:08.390612 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:52:16.787648 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:52:16.787604 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:52:26.291173 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:52:26.291141 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:52:33.589150 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:52:33.589109 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:52:43.492299 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:52:43.492208 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:52:56.484613 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:52:56.484581 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:53:04.493557 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:53:04.493515 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:53:12.791667 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:53:12.791616 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:53:20.586889 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:53:20.586850 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:53:28.892637 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:53:28.892597 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:53:38.194717 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:53:38.194685 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:53:49.789176 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:53:49.789139 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:54:07.895180 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:07.895141 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:54:15.992021 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:15.991934 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:54:25.086777 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:25.086738 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:54:26.729876 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:26.729835 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:54:26.733476 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:26.733447 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:54:32.994153 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:32.994120 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:54:49.985089 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:49.985058 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:54:58.198496 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:54:58.198458 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:55:07.490202 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:55:07.490157 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:55:15.487004 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:55:15.486969 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:55:24.396205 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:55:24.396169 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:55:33.094133 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:55:33.094096 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:55:42.486211 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:55:42.486170 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:55:55.288050 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:55:55.287966 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:56:04.493548 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:56:04.493504 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:56:17.992130 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:56:17.992091 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:56:26.183865 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:56:26.183828 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:56:34.688680 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:56:34.688642 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:56:42.985402 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:56:42.985357 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:56:51.089571 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:56:51.089540 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:57:06.381706 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:57:06.381668 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:57:14.683039 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:57:14.682943 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:57:23.785084 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:57:23.785047 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:57:31.081164 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:57:31.081123 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:57:56.186725 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:57:56.186690 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:58:08.790554 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:08.790507 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-kt8g2"] Apr 19 15:58:14.166190 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:14.166153 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67944f454b-8lkwh_836ff1ff-0345-4d96-b088-44d0cac8a98e/manager/0.log" Apr 19 15:58:16.201767 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:16.201738 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-kt8g2_b145825a-c143-47c1-b575-9a3864bf4030/limitador/0.log" Apr 19 15:58:16.838160 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:16.838126 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745fb64bc5-2s47l_0b5378ca-a08d-43fd-ad45-35a6fa21f495/kube-auth-proxy/0.log" Apr 19 15:58:17.149387 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:17.149308 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d7c7c5446-wbdgz_41b0c1cd-70f7-47aa-b30e-0368722019e6/router/0.log" Apr 19 15:58:17.468769 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:17.468682 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4_e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707/storage-initializer/0.log" Apr 19 15:58:17.475286 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:17.475262 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-fcbd4_e2ce4ee1-b60f-4ddb-9c0a-8dc57832b707/main/0.log" Apr 19 15:58:17.804689 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:17.804596 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7_70106617-c01c-4646-bf07-4fe53f6e0025/main/0.log" Apr 19 15:58:17.811287 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:17.811261 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccrb6h7_70106617-c01c-4646-bf07-4fe53f6e0025/storage-initializer/0.log" Apr 19 15:58:18.026436 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:18.026408 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-jgkps_7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e/storage-initializer/0.log" Apr 19 15:58:18.034252 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:18.034224 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-jgkps_7c55559c-b6ba-41cb-abcb-bf0fb4f81f2e/main/0.log" Apr 19 15:58:21.571921 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.571877 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w8x7s/must-gather-88gv6"] Apr 19 15:58:21.572320 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.572306 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae93d22f-b41e-45de-a99c-5e98535334ae" containerName="authorino" Apr 19 15:58:21.572363 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.572320 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae93d22f-b41e-45de-a99c-5e98535334ae" containerName="authorino" Apr 19 15:58:21.572397 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.572382 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="ae93d22f-b41e-45de-a99c-5e98535334ae" containerName="authorino" Apr 19 15:58:21.575859 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.575838 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:21.578565 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.578534 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w8x7s\"/\"kube-root-ca.crt\"" Apr 19 15:58:21.578565 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.578536 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w8x7s\"/\"default-dockercfg-775qz\"" Apr 19 15:58:21.578780 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.578579 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w8x7s\"/\"openshift-service-ca.crt\"" Apr 19 15:58:21.589884 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.589859 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8x7s/must-gather-88gv6"] Apr 19 15:58:21.649823 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.649788 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvqm7\" (UniqueName: \"kubernetes.io/projected/1206b7ae-2a6e-40fb-b668-6583d8e2170b-kube-api-access-zvqm7\") pod \"must-gather-88gv6\" (UID: \"1206b7ae-2a6e-40fb-b668-6583d8e2170b\") " pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:21.650000 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.649857 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1206b7ae-2a6e-40fb-b668-6583d8e2170b-must-gather-output\") pod \"must-gather-88gv6\" (UID: \"1206b7ae-2a6e-40fb-b668-6583d8e2170b\") " pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:21.751186 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.751145 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zvqm7\" (UniqueName: \"kubernetes.io/projected/1206b7ae-2a6e-40fb-b668-6583d8e2170b-kube-api-access-zvqm7\") pod \"must-gather-88gv6\" (UID: \"1206b7ae-2a6e-40fb-b668-6583d8e2170b\") " pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:21.751359 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.751213 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1206b7ae-2a6e-40fb-b668-6583d8e2170b-must-gather-output\") pod \"must-gather-88gv6\" (UID: \"1206b7ae-2a6e-40fb-b668-6583d8e2170b\") " pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:21.751523 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.751508 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1206b7ae-2a6e-40fb-b668-6583d8e2170b-must-gather-output\") pod \"must-gather-88gv6\" (UID: \"1206b7ae-2a6e-40fb-b668-6583d8e2170b\") " pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:21.758614 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.758586 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvqm7\" (UniqueName: \"kubernetes.io/projected/1206b7ae-2a6e-40fb-b668-6583d8e2170b-kube-api-access-zvqm7\") pod \"must-gather-88gv6\" (UID: \"1206b7ae-2a6e-40fb-b668-6583d8e2170b\") " pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:21.898316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:21.898281 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8x7s/must-gather-88gv6" Apr 19 15:58:22.018665 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:22.018639 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8x7s/must-gather-88gv6"] Apr 19 15:58:22.021499 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:58:22.021471 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1206b7ae_2a6e_40fb_b668_6583d8e2170b.slice/crio-6fbab13e2804a2cbf4d878686875da9ccf30aedc1c1c27e153d8387a23ac8ed6 WatchSource:0}: Error finding container 6fbab13e2804a2cbf4d878686875da9ccf30aedc1c1c27e153d8387a23ac8ed6: Status 404 returned error can't find the container with id 6fbab13e2804a2cbf4d878686875da9ccf30aedc1c1c27e153d8387a23ac8ed6 Apr 19 15:58:22.023547 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:22.023525 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 19 15:58:22.822294 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:22.822256 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8x7s/must-gather-88gv6" event={"ID":"1206b7ae-2a6e-40fb-b668-6583d8e2170b","Type":"ContainerStarted","Data":"6fbab13e2804a2cbf4d878686875da9ccf30aedc1c1c27e153d8387a23ac8ed6"} Apr 19 15:58:23.829126 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:23.829086 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8x7s/must-gather-88gv6" event={"ID":"1206b7ae-2a6e-40fb-b668-6583d8e2170b","Type":"ContainerStarted","Data":"9f724ef0ccc5086920caeff2a53102374a8bcb1711b9e0c87f72bab6ff629f02"} Apr 19 15:58:23.829126 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:23.829127 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8x7s/must-gather-88gv6" event={"ID":"1206b7ae-2a6e-40fb-b668-6583d8e2170b","Type":"ContainerStarted","Data":"eee8709cc6d5648ac7b72514c491a2362b7d7604f8d37c1800a697f21968a34f"} Apr 19 15:58:23.847300 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:23.847232 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-w8x7s/must-gather-88gv6" podStartSLOduration=2.029999143 podStartE2EDuration="2.847207274s" podCreationTimestamp="2026-04-19 15:58:21 +0000 UTC" firstStartedPulling="2026-04-19 15:58:22.0237207 +0000 UTC m=+2035.953180840" lastFinishedPulling="2026-04-19 15:58:22.840928828 +0000 UTC m=+2036.770388971" observedRunningTime="2026-04-19 15:58:23.844613226 +0000 UTC m=+2037.774073388" watchObservedRunningTime="2026-04-19 15:58:23.847207274 +0000 UTC m=+2037.776667437" Apr 19 15:58:24.398751 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:24.398717 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2zptl_98cebdc7-b692-4937-9ff6-006fa39429e6/global-pull-secret-syncer/0.log" Apr 19 15:58:24.574419 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:24.574383 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xvhn9_25b451a9-9cc5-4da3-8c6f-71b7f71e15dd/konnectivity-agent/0.log" Apr 19 15:58:24.595513 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:24.595479 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-182.ec2.internal_1850971f2e72febbaa8e5e238327e28b/haproxy/0.log" Apr 19 15:58:28.695045 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:28.695005 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-kt8g2_b145825a-c143-47c1-b575-9a3864bf4030/limitador/0.log" Apr 19 15:58:30.164965 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.164933 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c08f491-98f2-4085-aaca-3c71f4601377/alertmanager/0.log" Apr 19 15:58:30.197132 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.197096 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c08f491-98f2-4085-aaca-3c71f4601377/config-reloader/0.log" Apr 19 15:58:30.218301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.218276 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c08f491-98f2-4085-aaca-3c71f4601377/kube-rbac-proxy-web/0.log" Apr 19 15:58:30.243246 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.243220 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c08f491-98f2-4085-aaca-3c71f4601377/kube-rbac-proxy/0.log" Apr 19 15:58:30.265362 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.265314 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c08f491-98f2-4085-aaca-3c71f4601377/kube-rbac-proxy-metric/0.log" Apr 19 15:58:30.285287 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.285217 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c08f491-98f2-4085-aaca-3c71f4601377/prom-label-proxy/0.log" Apr 19 15:58:30.304951 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.304924 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c08f491-98f2-4085-aaca-3c71f4601377/init-config-reloader/0.log" Apr 19 15:58:30.559806 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.559777 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-frtdc_a8c0e9b7-a211-4e29-9a98-59a55972095d/node-exporter/0.log" Apr 19 15:58:30.581591 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.581558 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-frtdc_a8c0e9b7-a211-4e29-9a98-59a55972095d/kube-rbac-proxy/0.log" Apr 19 15:58:30.604047 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.604018 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-frtdc_a8c0e9b7-a211-4e29-9a98-59a55972095d/init-textfile/0.log" Apr 19 15:58:30.782932 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.782872 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7c53410-373e-4e5b-9ed1-9635d81f405e/prometheus/0.log" Apr 19 15:58:30.805537 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.805439 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7c53410-373e-4e5b-9ed1-9635d81f405e/config-reloader/0.log" Apr 19 15:58:30.851986 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.851956 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7c53410-373e-4e5b-9ed1-9635d81f405e/thanos-sidecar/0.log" Apr 19 15:58:30.874143 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.874118 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7c53410-373e-4e5b-9ed1-9635d81f405e/kube-rbac-proxy-web/0.log" Apr 19 15:58:30.890946 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.890923 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7c53410-373e-4e5b-9ed1-9635d81f405e/kube-rbac-proxy/0.log" Apr 19 15:58:30.911301 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.911250 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7c53410-373e-4e5b-9ed1-9635d81f405e/kube-rbac-proxy-thanos/0.log" Apr 19 15:58:30.931243 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.931195 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d7c53410-373e-4e5b-9ed1-9635d81f405e/init-config-reloader/0.log" Apr 19 15:58:30.960031 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.960000 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5676c8c784-ksfzr_184a1522-2ca4-48b2-be5c-2fe7ee5ec74c/prometheus-operator/0.log" Apr 19 15:58:30.976729 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:30.976700 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5676c8c784-ksfzr_184a1522-2ca4-48b2-be5c-2fe7ee5ec74c/kube-rbac-proxy/0.log" Apr 19 15:58:31.053760 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.053722 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cc96676dd-jgq7w_a880e944-1654-4de1-a3e3-54bf9bebb4de/telemeter-client/0.log" Apr 19 15:58:31.113816 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.113781 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cc96676dd-jgq7w_a880e944-1654-4de1-a3e3-54bf9bebb4de/reload/0.log" Apr 19 15:58:31.149068 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.149037 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cc96676dd-jgq7w_a880e944-1654-4de1-a3e3-54bf9bebb4de/kube-rbac-proxy/0.log" Apr 19 15:58:31.182186 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.182154 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786dd4d45d-wfd4m_7e60fe67-3a75-44a3-a571-b3cff8fb84e1/thanos-query/0.log" Apr 19 15:58:31.205346 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.205315 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786dd4d45d-wfd4m_7e60fe67-3a75-44a3-a571-b3cff8fb84e1/kube-rbac-proxy-web/0.log" Apr 19 15:58:31.224591 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.224560 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786dd4d45d-wfd4m_7e60fe67-3a75-44a3-a571-b3cff8fb84e1/kube-rbac-proxy/0.log" Apr 19 15:58:31.246281 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.246249 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786dd4d45d-wfd4m_7e60fe67-3a75-44a3-a571-b3cff8fb84e1/prom-label-proxy/0.log" Apr 19 15:58:31.265585 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.265550 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786dd4d45d-wfd4m_7e60fe67-3a75-44a3-a571-b3cff8fb84e1/kube-rbac-proxy-rules/0.log" Apr 19 15:58:31.286094 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:31.286067 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786dd4d45d-wfd4m_7e60fe67-3a75-44a3-a571-b3cff8fb84e1/kube-rbac-proxy-metrics/0.log" Apr 19 15:58:32.933721 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:32.933686 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/2.log" Apr 19 15:58:32.942774 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:32.942741 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-qjpxz_8f15cf98-a60c-41aa-96e8-be8466dd8e26/console-operator/3.log" Apr 19 15:58:32.974069 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:32.974032 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf"] Apr 19 15:58:32.980176 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:32.980135 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:32.990349 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:32.990319 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf"] Apr 19 15:58:33.074407 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.074366 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-sys\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.074915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.074707 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-lib-modules\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.074915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.074746 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bthwp\" (UniqueName: \"kubernetes.io/projected/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-kube-api-access-bthwp\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.074915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.074773 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-podres\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.074915 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.074822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-proc\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.176222 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.176185 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-sys\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.176527 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.176324 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-sys\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.176794 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.176503 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-lib-modules\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.176952 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.176936 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bthwp\" (UniqueName: \"kubernetes.io/projected/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-kube-api-access-bthwp\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.177068 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.177055 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-podres\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.177205 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.177193 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-proc\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.177316 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.176647 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-lib-modules\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.177375 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.177296 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-proc\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.177375 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.177237 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-podres\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.190693 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.185461 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bthwp\" (UniqueName: \"kubernetes.io/projected/82d38fe1-29c6-4031-ab01-b804d8dc4eb4-kube-api-access-bthwp\") pod \"perf-node-gather-daemonset-xksxf\" (UID: \"82d38fe1-29c6-4031-ab01-b804d8dc4eb4\") " pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.296816 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.296780 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.459375 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.459345 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf"] Apr 19 15:58:33.462316 ip-10-0-130-182 kubenswrapper[2576]: W0419 15:58:33.462279 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod82d38fe1_29c6_4031_ab01_b804d8dc4eb4.slice/crio-70676b7eb9e36baff127d72ddc42e67e25cda739f8aad594f517aedc11790484 WatchSource:0}: Error finding container 70676b7eb9e36baff127d72ddc42e67e25cda739f8aad594f517aedc11790484: Status 404 returned error can't find the container with id 70676b7eb9e36baff127d72ddc42e67e25cda739f8aad594f517aedc11790484 Apr 19 15:58:33.895873 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.895840 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" event={"ID":"82d38fe1-29c6-4031-ab01-b804d8dc4eb4","Type":"ContainerStarted","Data":"7e1b371c654d37c42a430ad1c49152c202a5ed260696606074eadf07c5f06cd3"} Apr 19 15:58:33.895873 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.895885 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" event={"ID":"82d38fe1-29c6-4031-ab01-b804d8dc4eb4","Type":"ContainerStarted","Data":"70676b7eb9e36baff127d72ddc42e67e25cda739f8aad594f517aedc11790484"} Apr 19 15:58:33.896145 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.895927 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:33.912666 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.912570 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" podStartSLOduration=1.912552953 podStartE2EDuration="1.912552953s" podCreationTimestamp="2026-04-19 15:58:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-19 15:58:33.910358978 +0000 UTC m=+2047.839819141" watchObservedRunningTime="2026-04-19 15:58:33.912552953 +0000 UTC m=+2047.842013118" Apr 19 15:58:33.954942 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:33.954881 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7c6cbb6c87-f6xng_1dffa790-7318-45b1-8c6f-981031b70840/volume-data-source-validator/0.log" Apr 19 15:58:34.847333 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:34.847306 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wh4pt_e3f37774-81cc-4b2c-94cb-ec77935ec46d/dns/0.log" Apr 19 15:58:34.867728 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:34.867690 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wh4pt_e3f37774-81cc-4b2c-94cb-ec77935ec46d/kube-rbac-proxy/0.log" Apr 19 15:58:34.941042 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:34.941016 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rlhfq_2514ec8e-0d56-4eda-929e-7a8759df8f30/dns-node-resolver/0.log" Apr 19 15:58:35.443418 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:35.443385 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fw4zx_7efc9272-6634-4b94-abe5-ba1e076ca391/node-ca/0.log" Apr 19 15:58:36.312473 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:36.312445 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-745fb64bc5-2s47l_0b5378ca-a08d-43fd-ad45-35a6fa21f495/kube-auth-proxy/0.log" Apr 19 15:58:36.410376 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:36.410322 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-d7c7c5446-wbdgz_41b0c1cd-70f7-47aa-b30e-0368722019e6/router/0.log" Apr 19 15:58:36.932761 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:36.932733 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qcm6b_ea08d4af-7d5b-47ed-ab7f-2e997210a6b9/serve-healthcheck-canary/0.log" Apr 19 15:58:37.479687 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:37.479660 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rsvd9_21a5dc61-a5e7-48cd-a756-316c82784b5b/kube-rbac-proxy/0.log" Apr 19 15:58:37.498052 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:37.498026 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rsvd9_21a5dc61-a5e7-48cd-a756-316c82784b5b/exporter/0.log" Apr 19 15:58:37.517281 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:37.517244 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rsvd9_21a5dc61-a5e7-48cd-a756-316c82784b5b/extractor/0.log" Apr 19 15:58:39.463754 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:39.463701 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67944f454b-8lkwh_836ff1ff-0345-4d96-b088-44d0cac8a98e/manager/0.log" Apr 19 15:58:39.912688 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:39.912656 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w8x7s/perf-node-gather-daemonset-xksxf" Apr 19 15:58:40.609223 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:40.609193 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65b77f8fcd-7zg6l_38ac9bf7-f584-4ceb-8d54-01fae655f31c/manager/0.log" Apr 19 15:58:45.022086 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:45.022053 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-z5vbd_5a1bd774-11fe-4781-88a0-85b33882c3f5/migrator/0.log" Apr 19 15:58:45.040692 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:45.040600 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-74bb7799d9-z5vbd_5a1bd774-11fe-4781-88a0-85b33882c3f5/graceful-termination/0.log" Apr 19 15:58:46.544128 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.544101 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bcjgt_f8668eb5-0f25-4267-b970-27cee041914b/kube-multus-additional-cni-plugins/0.log" Apr 19 15:58:46.563577 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.563550 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bcjgt_f8668eb5-0f25-4267-b970-27cee041914b/egress-router-binary-copy/0.log" Apr 19 15:58:46.582257 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.582235 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bcjgt_f8668eb5-0f25-4267-b970-27cee041914b/cni-plugins/0.log" Apr 19 15:58:46.600584 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.600563 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bcjgt_f8668eb5-0f25-4267-b970-27cee041914b/bond-cni-plugin/0.log" Apr 19 15:58:46.619148 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.619122 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bcjgt_f8668eb5-0f25-4267-b970-27cee041914b/routeoverride-cni/0.log" Apr 19 15:58:46.638650 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.638615 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bcjgt_f8668eb5-0f25-4267-b970-27cee041914b/whereabouts-cni-bincopy/0.log" Apr 19 15:58:46.657758 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.657728 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bcjgt_f8668eb5-0f25-4267-b970-27cee041914b/whereabouts-cni/0.log" Apr 19 15:58:46.833900 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.833872 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fjf4g_6d1f13eb-aa2b-4628-9adc-6d546d487ab3/kube-multus/0.log" Apr 19 15:58:46.970000 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.969967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r7nm5_7aeaccc6-af60-4b21-9803-f56daf2f7ddb/network-metrics-daemon/0.log" Apr 19 15:58:46.987353 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:46.987328 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r7nm5_7aeaccc6-af60-4b21-9803-f56daf2f7ddb/kube-rbac-proxy/0.log" Apr 19 15:58:47.868143 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:47.868096 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/ovn-controller/0.log" Apr 19 15:58:47.901785 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:47.901754 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/ovn-acl-logging/0.log" Apr 19 15:58:47.922826 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:47.922798 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/kube-rbac-proxy-node/0.log" Apr 19 15:58:47.941514 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:47.941479 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/kube-rbac-proxy-ovn-metrics/0.log" Apr 19 15:58:47.959197 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:47.959164 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/northd/0.log" Apr 19 15:58:47.976899 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:47.976862 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/nbdb/0.log" Apr 19 15:58:47.995000 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:47.994971 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/sbdb/0.log" Apr 19 15:58:48.163910 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:48.163834 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kqc9s_6fd23153-4cc8-41aa-9038-9780e928b79b/ovnkube-controller/0.log" Apr 19 15:58:49.820913 ip-10-0-130-182 kubenswrapper[2576]: I0419 15:58:49.820878 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-w2fz9_2e551954-ed8a-49b4-8237-75b061234272/network-check-target-container/0.log"